Skip to contentSkip to conversation

Unifying LKL into UML

RFC Superseded Hajime Tazaki 8 participants 248 messages Feb 5, 2020 – Mar 18, 2021 1 message not in the archive Page 1 of 3

Subsystems (10)
Documentation x86 Kbuild Kselftest +6 more subsystems
lore
Share & tools
lore
#0 lore raw

v3 cover Unifying LKL into UML

This patchset is to ask opinions from folks, whether LKL codes is good to integrate into UML code base. We wish to have any kind of feedback from your kind reviews. There are numbers of commits which should be asked for reviews to other mailing lists; we will do it later once we got discussed in this mailing list.

Changes in rfc v3:
- use UML drivers (net, block) from LKL programs
- drop virtio device implementations
- drop mingw32 (Windows) host
- drop android (arm/aarch64) host
- drop FreeBSD (x86_64) host
- drop LD_PRELOAD (hijack) support
- update milestone

rfc v2:
- use UMMODE instead of SUBARCH to switch UML or LKL
- tools/lkl directory is still there. I confirmed we can move under arch/um (e.g., arch/um/lkl/hosts). I will move it IF this is preferable.
- drop several patches involved non-uml directory
- drop several patches which are not required
- refine commit logs
- document updated

LKL (Linux Kernel Library) is aiming to allow reusing the Linux kernel code as extensively as possible with minimal effort and reduced maintenance overhead.

Examples of how LKL can be used are: creating userspace applications (running on Linux and other operating systems) that can read or write Linux filesystems or can use the Linux networking stack, creating kernel drivers for other operating systems that can read Linux filesystems, bootloaders support for reading/writing Linux filesystems, etc.

With LKL, the kernel code is compiled into an object file that can be directly linked by applications. The API offered by LKL is based on the Linux system call interface.

LKL is originally implemented as an architecture port in arch/lkl, but this series of commits tries to integrate this into arch/um as one of the mode of UML. This was discussed during RFC email of LKL (*1).

The latest LKL version can be found at https://github.com/lkl/linux

Milestone
=========
This patches is a first step toward upstreaming *library mode* of Linux kernel, but we think we need to have several steps toward our goal, describing in the below.

1. Put LKL code under arch/um (arch/um/lkl), and build it in a separate way from UML.
2. Use UML driver implementations in LKL as a minimum set of patches
- Only support x86 sub architecture (dependency to UML drivers)
3. Support broader host supports
- add virtio device features

For the step 1, we put LKL as one of UMMODE in order to make less effort to integrate (make ARCH=um UMMODE=library). The modification to existing UML code is trying to be minimized.

Building LKL the host library and LKL applications
==================================================

% cd tools/lkl
% make

will build LKL as a object file, it will install it in tools/lkl/lib together with the headers files in tools/lkl/include then will build the host library, tests and a few of application examples:

* tests/boot - a simple applications that uses LKL and exercises the basic LKL APIs

* tests/net-test - a simple applications that uses network feature of LKL and exercises the basic network-related APIs

* fs2tar - a tool that converts a filesystem image to a tar archive

* cptofs/cpfromfs - a tool that copies files to/from a filesystem image

* lklfuse - a tool that can mount a filesystem image in userspace, without root priviledges, using FUSE

% make run-tests

should run the above tests/boot and tests/net-test and report errors if there are any.

Supported hosts
===============

Currently LKL supports Linux userspace applications. New hosts can be added relatively easy if the host supports gcc and GNU ld. Previous versions of LKL supported Windows kernel and Haiku kernel hosts, and we also have WIP patches with rump-hypercall interface, used in UEFI, as well as macOS userspace (part of POSIX).

There is also musl-libc port for LKL, which might be interested in for some folks.

Further readings about LKL
=========================

- Discussion in github LKL issue
https://github.com/lkl/linux/issues/304

- LKL (an article)
https://www.researchgate.net/profile/Nicolae_Tapus2/publication/224164682_LKL_The_Linux_kernel_library/links/02bfe50fd921ab4f7c000000.pdf

*1 RFC email to LKML (back in 2015)
https://www.mail-archive.com/[email protected]/msg1012277.html

Please review the following changes for suitability for inclusion. If you have any objections or suggestions for improvement, please respond to the patches. If you agree with the changes, please provide your Acked-by.

The following changes since commit d65197ad52494bed3b5e64708281b8295f76c391:

um: Fix time-travel=inf-cpu with xor/raid6 (2020-01-19 22:42:06 +0100)

are available in the Git repository at:

git://github.com/thehajime/linux b3072caaef01f2cef5f34bc012e066826798500c
https://github.com/thehajime/linux/tree/upstream-to-uml-5.6-rc1-v3

Hajime Tazaki (6):
lkl tools: host lib: filesystem helpers lkl tools: host lib: networking helpers
um lkl: add CI scripts to conduct regression tests um lkl: add UML network driver for lkl
um lkl: add UML block device driver (ubd) for lkl
um: fix clone flags to be familar with valgrind

Octavian Purdila (20):
asm-generic: atomic64: allow using generic atomic64 on 64bit platforms
arch: add __SYSCALL_DEFINE_ARCH
um lkl: architecture skeleton for Linux kernel library
um lkl: host interface
um lkl: memory handling
um lkl: kernel threads support
um lkl: interrupt support
um lkl: system call interface and application API
um lkl: timers, time and delay support
um lkl: basic kernel console support
um lkl: initialization and cleanup
um lkl: plug in the build system
lkl tools: skeleton for host side library
lkl tools: host lib: add utilities functions
lkl tools: host lib: posix host operations
lkl tools: add test programs
lkl tools: cptofs that reads/writes to/from a filesystem image lkl tools: fs2tar that converts a filesystem image to tar lkl tools: add lklfuse
um lkl: add documentation

 .circleci/config.yml                       |  198 ++++
 Documentation/virt/uml/lkl.txt             |   64 ++
 MAINTAINERS                                |    8 +
 arch/um/Kconfig                            |   13 +
 arch/um/Makefile                           |   19 +-
 arch/um/Makefile.um                        |   10 +
 arch/um/drivers/Makefile                   |    2 +
 arch/um/drivers/ubd_kern.c                 |    6 +-
 arch/um/drivers/ubd_user.c                 |    3 +-
 arch/um/kernel/Makefile                    |    4 +
 arch/um/kernel/irq.c                       |    4 +
 arch/um/lkl/.gitignore                     |    2 +
 arch/um/lkl/Kconfig                        |   88 ++
 arch/um/lkl/Kconfig.debug                  |    0
 arch/um/lkl/Makefile                       |   69 ++
 arch/um/lkl/auto.conf                      |    1 +
 arch/um/lkl/configs/lkl_defconfig          | 1052 ++++++++++++++++++++
 arch/um/lkl/include/asm/Kbuild             |   80 ++
 arch/um/lkl/include/asm/bitsperlong.h      |   11 +
 arch/um/lkl/include/asm/byteorder.h        |    7 +
 arch/um/lkl/include/asm/cpu.h              |   14 +
 arch/um/lkl/include/asm/elf.h              |   15 +
 arch/um/lkl/include/asm/host_ops.h         |   10 +
 arch/um/lkl/include/asm/irq.h              |   13 +
 arch/um/lkl/include/asm/page.h             |   14 +
 arch/um/lkl/include/asm/pgtable.h          |   57 ++
 arch/um/lkl/include/asm/processor.h        |   60 ++
 arch/um/lkl/include/asm/ptrace.h           |   25 +
 arch/um/lkl/include/asm/sched.h            |   23 +
 arch/um/lkl/include/asm/setup.h            |    7 +
 arch/um/lkl/include/asm/syscalls.h         |   18 +
 arch/um/lkl/include/asm/syscalls_32.h      |   43 +
 arch/um/lkl/include/asm/thread_info.h      |   70 ++
 arch/um/lkl/include/asm/tlb.h              |   12 +
 arch/um/lkl/include/asm/uaccess.h          |   64 ++
 arch/um/lkl/include/asm/unistd.h           |   27 +
 arch/um/lkl/include/asm/unistd_32.h        |   31 +
 arch/um/lkl/include/asm/vmlinux.lds.h      |   14 +
 arch/um/lkl/include/asm/xor.h              |    9 +
 arch/um/lkl/include/uapi/asm/Kbuild        |    9 +
 arch/um/lkl/include/uapi/asm/bitsperlong.h |   13 +
 arch/um/lkl/include/uapi/asm/byteorder.h   |   11 +
 arch/um/lkl/include/uapi/asm/host_ops.h    |  143 +++
 arch/um/lkl/include/uapi/asm/irq.h         |   36 +
 arch/um/lkl/include/uapi/asm/sigcontext.h  |   16 +
 arch/um/lkl/include/uapi/asm/siginfo.h     |   11 +
 arch/um/lkl/include/uapi/asm/swab.h        |   11 +
 arch/um/lkl/include/uapi/asm/syscalls.h    |  349 +++++++
 arch/um/lkl/include/uapi/asm/unistd.h      |   16 +
 arch/um/lkl/kernel/Makefile                |    4 +
 arch/um/lkl/kernel/asm-offsets.c           |    3 +
 arch/um/lkl/kernel/console.c               |   42 +
 arch/um/lkl/kernel/cpu.c                   |  223 +++++
 arch/um/lkl/kernel/irq.c                   |  209 ++++
 arch/um/lkl/kernel/misc.c                  |   60 ++
 arch/um/lkl/kernel/setup.c                 |  198 ++++
 arch/um/lkl/kernel/syscalls.c              |  192 ++++
 arch/um/lkl/kernel/syscalls_32.c           |  159 +++
 arch/um/lkl/kernel/threads.c               |  227 +++++
 arch/um/lkl/kernel/time.c                  |  145 +++
 arch/um/lkl/kernel/vmlinux.lds.S           |   51 +
 arch/um/lkl/mm/Makefile                    |    1 +
 arch/um/lkl/mm/bootmem.c                   |   99 ++
 arch/um/lkl/scripts/headers_install.py     |  196 ++++
 arch/um/os-Linux/Makefile                  |    5 +
 arch/um/os-Linux/file.c                    |    9 +
 include/asm-generic/atomic64.h             |    2 +
 include/linux/syscalls.h                   |    6 +
 tools/lkl/.gitignore                       |   13 +
 tools/lkl/Build                            |    6 +
 tools/lkl/Makefile                         |  131 +++
 tools/lkl/Makefile.autoconf                |   67 ++
 tools/lkl/Targets                          |   17 +
 tools/lkl/cptofs.c                         |  636 ++++++++++++
 tools/lkl/fs2tar.c                         |  412 ++++++++
 tools/lkl/include/.gitignore               |    1 +
 tools/lkl/include/lkl.h                    |  788 +++++++++++++++
 tools/lkl/include/lkl_config.h             |   61 ++
 tools/lkl/include/lkl_host.h               |  112 +++
 tools/lkl/lib/.gitignore                   |    3 +
 tools/lkl/lib/Build                        |   13 +
 tools/lkl/lib/config.c                     |  750 ++++++++++++++
 tools/lkl/lib/dbg.c                        |  300 ++++++
 tools/lkl/lib/dbg_handler.c                |   44 +
 tools/lkl/lib/endian.h                     |   31 +
 tools/lkl/lib/fs.c                         |  471 +++++++++
 tools/lkl/lib/jmp_buf.c                    |   14 +
 tools/lkl/lib/jmp_buf.h                    |    8 +
 tools/lkl/lib/net.c                        |  818 +++++++++++++++
 tools/lkl/lib/posix-host.c                 |  356 +++++++
 tools/lkl/lib/um/Build                     |    3 +
 tools/lkl/lib/um/um_block.c                |   17 +
 tools/lkl/lib/um/um_glue.c                 |   58 ++
 tools/lkl/lib/um/um_net.c                  |   25 +
 tools/lkl/lib/utils.c                      |  266 +++++
 tools/lkl/lklfuse.c                        |  659 ++++++++++++
 tools/lkl/scripts/checkpatch.sh            |   60 ++
 tools/lkl/scripts/lkl-jenkins.sh           |   21 +
 tools/lkl/tests/Build                      |    3 +
 tools/lkl/tests/boot.c                     |  562 +++++++++++
 tools/lkl/tests/boot.sh                    |    9 +
 tools/lkl/tests/cla.c                      |  159 +++
 tools/lkl/tests/cla.h                      |   33 +
 tools/lkl/tests/disk.c                     |  203 ++++
 tools/lkl/tests/disk.sh                    |   61 ++
 tools/lkl/tests/lklfuse.sh                 |  110 ++
 tools/lkl/tests/net-setup.sh               |   74 ++
 tools/lkl/tests/net-test.c                 |  291 ++++++
 tools/lkl/tests/net.sh                     |  108 ++
 tools/lkl/tests/run.py                     |  180 ++++
 tools/lkl/tests/tap13.py                   |  209 ++++
 tools/lkl/tests/test.c                     |  126 +++
 tools/lkl/tests/test.h                     |   72 ++
 tools/lkl/tests/test.sh                    |  182 ++++
 tools/lkl/tests/valgrind.supp              |  105 ++
 tools/lkl/tests/valgrind2xunit.py          |   69 ++
 116 files changed, 13327 insertions(+), 3 deletions(-)
 create mode 100644 .circleci/config.yml
 create mode 100644 Documentation/virt/uml/lkl.txt
 create mode 100644 arch/um/Makefile.um
 create mode 100644 arch/um/lkl/.gitignore
 create mode 100644 arch/um/lkl/Kconfig
 create mode 100644 arch/um/lkl/Kconfig.debug
 create mode 100644 arch/um/lkl/Makefile
 create mode 100644 arch/um/lkl/auto.conf
 create mode 100644 arch/um/lkl/configs/lkl_defconfig
 create mode 100644 arch/um/lkl/include/asm/Kbuild
 create mode 100644 arch/um/lkl/include/asm/bitsperlong.h
 create mode 100644 arch/um/lkl/include/asm/byteorder.h
 create mode 100644 arch/um/lkl/include/asm/cpu.h
 create mode 100644 arch/um/lkl/include/asm/elf.h
 create mode 100644 arch/um/lkl/include/asm/host_ops.h
 create mode 100644 arch/um/lkl/include/asm/irq.h
 create mode 100644 arch/um/lkl/include/asm/page.h
 create mode 100644 arch/um/lkl/include/asm/pgtable.h
 create mode 100644 arch/um/lkl/include/asm/processor.h
 create mode 100644 arch/um/lkl/include/asm/ptrace.h
 create mode 100644 arch/um/lkl/include/asm/sched.h
 create mode 100644 arch/um/lkl/include/asm/setup.h
 create mode 100644 arch/um/lkl/include/asm/syscalls.h
 create mode 100644 arch/um/lkl/include/asm/syscalls_32.h
 create mode 100644 arch/um/lkl/include/asm/thread_info.h
 create mode 100644 arch/um/lkl/include/asm/tlb.h
 create mode 100644 arch/um/lkl/include/asm/uaccess.h
 create mode 100644 arch/um/lkl/include/asm/unistd.h
 create mode 100644 arch/um/lkl/include/asm/unistd_32.h
 create mode 100644 arch/um/lkl/include/asm/vmlinux.lds.h
 create mode 100644 arch/um/lkl/include/asm/xor.h
 create mode 100644 arch/um/lkl/include/uapi/asm/Kbuild
 create mode 100644 arch/um/lkl/include/uapi/asm/bitsperlong.h
 create mode 100644 arch/um/lkl/include/uapi/asm/byteorder.h
 create mode 100644 arch/um/lkl/include/uapi/asm/host_ops.h
 create mode 100644 arch/um/lkl/include/uapi/asm/irq.h
 create mode 100644 arch/um/lkl/include/uapi/asm/sigcontext.h
 create mode 100644 arch/um/lkl/include/uapi/asm/siginfo.h
 create mode 100644 arch/um/lkl/include/uapi/asm/swab.h
 create mode 100644 arch/um/lkl/include/uapi/asm/syscalls.h
 create mode 100644 arch/um/lkl/include/uapi/asm/unistd.h
 create mode 100644 arch/um/lkl/kernel/Makefile
 create mode 100644 arch/um/lkl/kernel/asm-offsets.c
 create mode 100644 arch/um/lkl/kernel/console.c
 create mode 100644 arch/um/lkl/kernel/cpu.c
 create mode 100644 arch/um/lkl/kernel/irq.c
 create mode 100644 arch/um/lkl/kernel/misc.c
 create mode 100644 arch/um/lkl/kernel/setup.c
 create mode 100644 arch/um/lkl/kernel/syscalls.c
 create mode 100644 arch/um/lkl/kernel/syscalls_32.c
 create mode 100644 arch/um/lkl/kernel/threads.c
 create mode 100644 arch/um/lkl/kernel/time.c
 create mode 100644 arch/um/lkl/kernel/vmlinux.lds.S
 create mode 100644 arch/um/lkl/mm/Makefile
 create mode 100644 arch/um/lkl/mm/bootmem.c
 create mode 100755 arch/um/lkl/scripts/headers_install.py
 create mode 100644 tools/lkl/.gitignore
 create mode 100644 tools/lkl/Build
 create mode 100644 tools/lkl/Makefile
 create mode 100644 tools/lkl/Makefile.autoconf
 create mode 100644 tools/lkl/Targets
 create mode 100644 tools/lkl/cptofs.c
 create mode 100644 tools/lkl/fs2tar.c
 create mode 100644 tools/lkl/include/.gitignore
 create mode 100644 tools/lkl/include/lkl.h
 create mode 100644 tools/lkl/include/lkl_config.h
 create mode 100644 tools/lkl/include/lkl_host.h
 create mode 100644 tools/lkl/lib/.gitignore
 create mode 100644 tools/lkl/lib/Build
 create mode 100644 tools/lkl/lib/config.c
 create mode 100644 tools/lkl/lib/dbg.c
 create mode 100644 tools/lkl/lib/dbg_handler.c
 create mode 100644 tools/lkl/lib/endian.h
 create mode 100644 tools/lkl/lib/fs.c
 create mode 100644 tools/lkl/lib/jmp_buf.c
 create mode 100644 tools/lkl/lib/jmp_buf.h
 create mode 100644 tools/lkl/lib/net.c
 create mode 100644 tools/lkl/lib/posix-host.c
 create mode 100644 tools/lkl/lib/um/Build
 create mode 100644 tools/lkl/lib/um/um_block.c
 create mode 100644 tools/lkl/lib/um/um_glue.c
 create mode 100644 tools/lkl/lib/um/um_net.c
 create mode 100644 tools/lkl/lib/utils.c
 create mode 100644 tools/lkl/lklfuse.c
 create mode 100755 tools/lkl/scripts/checkpatch.sh
 create mode 100755 tools/lkl/scripts/lkl-jenkins.sh
 create mode 100644 tools/lkl/tests/Build
 create mode 100644 tools/lkl/tests/boot.c
 create mode 100755 tools/lkl/tests/boot.sh
 create mode 100644 tools/lkl/tests/cla.c
 create mode 100644 tools/lkl/tests/cla.h
 create mode 100644 tools/lkl/tests/disk.c
 create mode 100755 tools/lkl/tests/disk.sh
 create mode 100755 tools/lkl/tests/lklfuse.sh
 create mode 100644 tools/lkl/tests/net-setup.sh
 create mode 100644 tools/lkl/tests/net-test.c
 create mode 100755 tools/lkl/tests/net.sh
 create mode 100755 tools/lkl/tests/run.py
 create mode 100644 tools/lkl/tests/tap13.py
 create mode 100644 tools/lkl/tests/test.c
 create mode 100644 tools/lkl/tests/test.h
 create mode 100644 tools/lkl/tests/test.sh
 create mode 100644 tools/lkl/tests/valgrind.supp
 create mode 100755 tools/lkl/tests/valgrind2xunit.py
Signature · 1 line
2.21.0 (Apple Git-122.2)
#1 lore raw

RFC v3 1/26 asm-generic: atomic64: allow using generic atomic64 on 64bit platforms

From: Octavian Purdila <[email protected]>

With CONFIG_64BIT enabled, atomic64 via CONFIG_GENERIC_ATOMIC64 options are not compiled due to type conflict of atomic64_t defined in linux/type.h.

This commit fixes the issue and allow using generic atomic64 ops.

Currently, LKL is only the user which defines GENERIC_ATOMIC64 (lib/atomic64.c) under CONFIG_64BIT environment. Thus, there is no issues before this commit.

Cc Will Deacon <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Boqun Feng <[email protected]>
---
 include/asm-generic/atomic64.h | 2 ++
 1 file changed, 2 insertions(+)
include/asm-generic/atomic64.h +2 -0
index 370f01d4450f..9b15847baae5 100644
@@ -9,9 +9,11 @@
 #define _ASM_GENERIC_ATOMIC64_H
 #include <linux/types.h>
 
+#ifndef CONFIG_64BIT
 typedef struct {
 	s64 counter;
 } atomic64_t;
+#endif
 
 #define ATOMIC64_INIT(i)	{ (i) }
 
Signature · 1 line
2.21.0 (Apple Git-122.2)
maintainer
#2 lore raw

Re: [RFC v3 01/26] asm-generic: atomic64: allow using generic atomic64 on 64bit platforms

Quoted 11 lines · Hajime Tazaki

From: Octavian Purdila <[email protected]>

With CONFIG_64BIT enabled, atomic64 via CONFIG_GENERIC_ATOMIC64 options are not compiled due to type conflict of atomic64_t defined in linux/type.h.

This commit fixes the issue and allow using generic atomic64 ops.

Currently, LKL is only the user which defines GENERIC_ATOMIC64 (lib/atomic64.c) under CONFIG_64BIT environment. Thus, there is no issues before this commit.

Uhhhhh, no.

Not allowing GENERIC_ATOMIC64 on 64BIT is a *feature*.

Any 64bit arch that needs GENERIC_ATOMIC64 is an utter piece of crap.

Please explain more.

#3 lore raw

Re: [RFC v3 01/26] asm-generic: atomic64: allow using generic atomic64 on 64bit platforms

Quoted 14 lines · Peter Zijlstra
Quoted 11 lines · Hajime Tazaki

From: Octavian Purdila <[email protected]>

With CONFIG_64BIT enabled, atomic64 via CONFIG_GENERIC_ATOMIC64 options are not compiled due to type conflict of atomic64_t defined in linux/type.h.

This commit fixes the issue and allow using generic atomic64 ops.

Currently, LKL is only the user which defines GENERIC_ATOMIC64 (lib/atomic64.c) under CONFIG_64BIT environment. Thus, there is no issues before this commit.

Quoted 8 lines

Uhhhhh, no.

Not allowing GENERIC_ATOMIC64 on 64BIT is a *feature*.

Any 64bit arch that needs GENERIC_ATOMIC64 is an utter piece of crap.

Please explain more.

Hi Peter,

I was not aware that not allowing GENERIC_ATOMIC64 was intentional. I understand your point that a 64 bit architecture that can't handle 64 bit atomic operation is broken.

One way to deal with this in LKL would be to use GCC atomic builtins or if that doesn't work expose them as host operations. This would keep LKL as a meta-arch that can run on multiple physical architectures. I'll give it a try.

#4 lore raw

Re: [RFC v3 01/26] asm-generic: atomic64: allow using generic atomic64 on 64bit platforms

Quoted 35 lines · Octavian Purdila
Quoted 14 lines · Peter Zijlstra
Quoted 11 lines · Hajime Tazaki

From: Octavian Purdila <[email protected]>

With CONFIG_64BIT enabled, atomic64 via CONFIG_GENERIC_ATOMIC64 options are not compiled due to type conflict of atomic64_t defined in linux/type.h.

This commit fixes the issue and allow using generic atomic64 ops.

Currently, LKL is only the user which defines GENERIC_ATOMIC64 (lib/atomic64.c) under CONFIG_64BIT environment. Thus, there is no issues before this commit.

Quoted 8 lines

Uhhhhh, no.

Not allowing GENERIC_ATOMIC64 on 64BIT is a *feature*.

Any 64bit arch that needs GENERIC_ATOMIC64 is an utter piece of crap.

Please explain more.

Hi Peter,

I was not aware that not allowing GENERIC_ATOMIC64 was intentional. I understand your point that a 64 bit architecture that can't handle 64 bit atomic operation is broken.

One way to deal with this in LKL would be to use GCC atomic builtins or if that doesn't work expose them as host operations. This would keep LKL as a meta-arch that can run on multiple physical architectures. I'll give it a try.

Quoted 5 lines

_______________________________________________
linux-um mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-um

You can lift a set of defines which do that for most compilers/arches out of OVS code.

Have a look at lib/ovs-atomic*.h

It should do exactly what you want (+/- cutting it down to things you need).

Signature · 2 lines
Anton R. Ivanov
https://www.kot-begemot.co.uk/
maintainer
#5 lore raw

Re: [RFC v3 01/26] asm-generic: atomic64: allow using generic atomic64 on 64bit platforms

In reply to Octavian Purdila
Quoted 1 line · Octavian Purdila

I was not aware that not allowing GENERIC_ATOMIC64 was intentional. I

It might not have been, but presented with this patch, I feel like it should've been :-)

Quoted 2 lines

understand your point that a 64 bit architecture that can't handle 64 bit atomic operation is broken.

(sadly they actually exist, I shall name no names)

Quoted 4 lines

One way to deal with this in LKL would be to use GCC atomic builtins or if that doesn't work expose them as host operations. This would keep LKL as a meta-arch that can run on multiple physical architectures. I'll give it a try.

What is this LKL you speak of and how does it do the 32bit atomics?

One thing to keep in mind is that the C11 atomics (_Atomic) don't trivially map to the LKMM -- although I keep forgetting the exact details, there is a paper on it somewhere. Also, once you're limited to a specific arch the issue also becomes much easier than C11 vs LKMM in general.

_______________________________________________
linux-um mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-um

#6 lore raw

Re: [RFC v3 01/26] asm-generic: atomic64: allow using generic atomic64 on 64bit platforms

Quoted 10 lines · Peter Zijlstra
Quoted 1 line · Octavian Purdila

I was not aware that not allowing GENERIC_ATOMIC64 was intentional. I

It might not have been, but presented with this patch, I feel like it should've been :-)

Quoted 2 lines

understand your point that a 64 bit architecture that can't handle 64 bit atomic operation is broken.

Quoted 9 lines

(sadly they actually exist, I shall name no names)

Quoted 4 lines

One way to deal with this in LKL would be to use GCC atomic builtins or if that doesn't work expose them as host operations. This would keep LKL as a meta-arch that can run on multiple physical architectures. I'll give it a try.

What is this LKL you speak of and how does it do the 32bit atomics?

LKL is a build of the Linux kernel as a library that can run in many environments including multiple architectures and OSes [1]

For 32bit atomics LKL also uses the asm-generic implementation. It is very similar with generic 64bit atomic implementation and it is used by multiple 32bit arches. I think this was my original reasoning for this patch and not going with C11 atomics.

Quoted 5 lines

One thing to keep in mind is that the C11 atomics (_Atomic) don't trivially map to the LKMM -- although I keep forgetting the exact details, there is a paper on it somewhere. Also, once you're limited to a specific arch the issue also becomes much easier than C11 vs LKMM in general.

Thanks, I will look it up.

[1] https://github.com/lkl/linux

maintainer
#7 lore raw

Re: [RFC v3 01/26] asm-generic: atomic64: allow using generic atomic64 on 64bit platforms

Quoted 18 lines · Octavian Purdila
Quoted 17 lines · Peter Zijlstra
Quoted 1 line · Octavian Purdila

I was not aware that not allowing GENERIC_ATOMIC64 was intentional. I

It might not have been, but presented with this patch, I feel like it should've been :-)

Quoted 2 lines

understand your point that a 64 bit architecture that can't handle 64 bit atomic operation is broken.

(sadly they actually exist, I shall name no names)

Quoted 4 lines

One way to deal with this in LKL would be to use GCC atomic builtins or if that doesn't work expose them as host operations. This would keep LKL as a meta-arch that can run on multiple physical architectures. I'll give it a try.

Quoted 5 lines
Quoted 2 lines

What is this LKL you speak of and how does it do the 32bit atomics?

LKL is a build of the Linux kernel as a library that can run in many environments including multiple architectures and OSes [1]

Thanks, I'll put it on the to-read list.

Quoted 4 lines

For 32bit atomics LKL also uses the asm-generic implementation. It is very similar with generic 64bit atomic implementation and it is used by multiple 32bit arches. I think this was my original reasoning for this patch and not going with C11 atomics.

Uh no, asm-generic/atomic.h is radically different from lib/atomic64.c.

asm-generic/atomic.h builds all required atomic operations from cmpxchg() (loops), while lib/atomic64.c builds 64bit atomics by using a hashed set of spinlocks.

The asm-generic stuff gives you real atomic ops, albeit sub-optimal, lib/atomic64.c gives you a turd.

_______________________________________________
linux-um mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-um

#8 lore raw

Re: [RFC v3 01/26] asm-generic: atomic64: allow using generic atomic64 on 64bit platforms

Quoted 33 lines · Peter Zijlstra
Quoted 18 lines · Octavian Purdila
Quoted 17 lines · Peter Zijlstra
Quoted 1 line · Octavian Purdila

I was not aware that not allowing GENERIC_ATOMIC64 was intentional. I

It might not have been, but presented with this patch, I feel like it should've been :-)

Quoted 2 lines

understand your point that a 64 bit architecture that can't handle 64 bit atomic operation is broken.

(sadly they actually exist, I shall name no names)

Quoted 4 lines

One way to deal with this in LKL would be to use GCC atomic builtins or if that doesn't work expose them as host operations. This would keep LKL as a meta-arch that can run on multiple physical architectures. I'll give it a try.

Quoted 5 lines
Quoted 2 lines

What is this LKL you speak of and how does it do the 32bit atomics?

LKL is a build of the Linux kernel as a library that can run in many environments including multiple architectures and OSes [1]

Thanks, I'll put it on the to-read list.

Quoted 4 lines

For 32bit atomics LKL also uses the asm-generic implementation. It is very similar with generic 64bit atomic implementation and it is used by multiple 32bit arches. I think this was my original reasoning for this patch and not going with C11 atomics.

Quoted 8 lines

Uh no, asm-generic/atomic.h is radically different from lib/atomic64.c.

asm-generic/atomic.h builds all required atomic operations from cmpxchg() (loops), while lib/atomic64.c builds 64bit atomics by using a hashed set of spinlocks.

The asm-generic stuff gives you real atomic ops, albeit sub-optimal, lib/atomic64.c gives you a turd.

Ah, yes, I overlooked that.

(I think that they are equivalent for LKL because it is a non-SMP arch only at this moment, but I agree that the better approach is to have a proper implementation in LKL)

A message that is not in the archive (sent off-list or not yet received): cover.1580882335.git.thehajime-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

#10 lore raw

RFC v3 2/26 arch: add __SYSCALL_DEFINE_ARCH

From: Octavian Purdila <[email protected]>

This allows the architecture code to process the system call definitions. It is used by LKL to create strong typed function definitions for system calls.

---
 include/linux/syscalls.h | 6 ++++++
 1 file changed, 6 insertions(+)
include/linux/syscalls.h +6 -0
index 5262b7a76d39..d61069aa869a 100644
@@ -203,9 +203,14 @@ static inline int is_syscall_trace_event(struct trace_event_call *tp_event)
 }
 #endif
 
+#ifndef __SYSCALL_DEFINE_ARCH
+#define __SYSCALL_DEFINE_ARCH(x, sname, ...)
+#endif
+
 #ifndef SYSCALL_DEFINE0
 #define SYSCALL_DEFINE0(sname)					\
 	SYSCALL_METADATA(_##sname, 0);				\
+	__SYSCALL_DEFINE_ARCH(0, _##sname);			\
 	asmlinkage long sys_##sname(void);			\
 	ALLOW_ERROR_INJECTION(sys_##sname, ERRNO);		\
 	asmlinkage long sys_##sname(void)
@@ -222,6 +227,7 @@ static inline int is_syscall_trace_event(struct trace_event_call *tp_event)
 
 #define SYSCALL_DEFINEx(x, sname, ...)				\
 	SYSCALL_METADATA(sname, x, __VA_ARGS__)			\
+	__SYSCALL_DEFINE_ARCH(x, sname, __VA_ARGS__)		\
 	__SYSCALL_DEFINEx(x, sname, __VA_ARGS__)
 
 #define __PROTECT(...) asmlinkage_protect(__VA_ARGS__)
Signature · 1 line
2.21.0 (Apple Git-122.2)
#11 lore raw

RFC v3 3/26 um lkl: architecture skeleton for Linux kernel library

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

Adds the LKL Kconfig, vmlinux linker script, basic architecture headers and miscellaneous basic functions or stubs such as dump_stack(), show_regs() and cpuinfo proc ops.

The headers we introduce in this patch are simple wrappers to the asm-generic headers or stubs for things we don't support, such as ptrace, DMA, signals, ELF handling and low level processor operations.

The kernel configuration is automatically updated to reflect the endianness of the host, 64bit support or the output format for vmlinux's linker script. We do this by looking at the ld's default output format.

Cc Andreas Abel <[email protected]>
Cc Conrad Meyer <[email protected]>
Cc Edison M. Castro <[email protected]>
Cc H.K. Jerry Chu <[email protected]>
Cc Jens Staal <[email protected]>
Cc Lai Jiangshan <[email protected]>
Cc Levente Kurusa <[email protected]>
Cc Luca Dariz <[email protected]>
Cc Mark Stillwell <[email protected]>
Cc Matthieu Coudron <[email protected]>
Cc Michael Zimmermann <[email protected]>
Cc Motomu Utsumi <[email protected]>
Cc Patrick Collins <[email protected]>
Cc Petros Angelatos <[email protected]>
Cc Pierre-Hugues Husson <[email protected]>
Cc Xiao Jia <[email protected]>
Cc Yuan Liu <[email protected]>
---
 MAINTAINERS                                |    8 +
 arch/um/lkl/.gitignore                     |    2 +
 arch/um/lkl/Kconfig                        |   56 ++
 arch/um/lkl/Kconfig.debug                  |    0
 arch/um/lkl/configs/lkl_defconfig          | 1052 ++++++++++++++++++++
 arch/um/lkl/include/asm/Kbuild             |   80 ++
 arch/um/lkl/include/asm/bitsperlong.h      |   11 +
 arch/um/lkl/include/asm/byteorder.h        |    7 +
 arch/um/lkl/include/asm/cpu.h              |   14 +
 arch/um/lkl/include/asm/elf.h              |   15 +
 arch/um/lkl/include/asm/processor.h        |   60 ++
 arch/um/lkl/include/asm/ptrace.h           |   25 +
 arch/um/lkl/include/asm/sched.h            |   23 +
 arch/um/lkl/include/asm/syscalls.h         |   18 +
 arch/um/lkl/include/asm/syscalls_32.h      |   43 +
 arch/um/lkl/include/asm/tlb.h              |   12 +
 arch/um/lkl/include/asm/uaccess.h          |   64 ++
 arch/um/lkl/include/asm/unistd_32.h        |   31 +
 arch/um/lkl/include/asm/vmlinux.lds.h      |   14 +
 arch/um/lkl/include/asm/xor.h              |    9 +
 arch/um/lkl/include/uapi/asm/Kbuild        |    9 +
 arch/um/lkl/include/uapi/asm/bitsperlong.h |   13 +
 arch/um/lkl/include/uapi/asm/byteorder.h   |   11 +
 arch/um/lkl/include/uapi/asm/siginfo.h     |   11 +
 arch/um/lkl/include/uapi/asm/swab.h        |   11 +
 arch/um/lkl/include/uapi/asm/syscalls.h    |  348 +++++++
 arch/um/lkl/kernel/asm-offsets.c           |    2 +
 arch/um/lkl/kernel/misc.c                  |   60 ++
 arch/um/lkl/kernel/vmlinux.lds.S           |   51 +
 29 files changed, 2060 insertions(+)
 create mode 100644 arch/um/lkl/.gitignore
 create mode 100644 arch/um/lkl/Kconfig
 create mode 100644 arch/um/lkl/Kconfig.debug
 create mode 100644 arch/um/lkl/configs/lkl_defconfig
 create mode 100644 arch/um/lkl/include/asm/Kbuild
 create mode 100644 arch/um/lkl/include/asm/bitsperlong.h
 create mode 100644 arch/um/lkl/include/asm/byteorder.h
 create mode 100644 arch/um/lkl/include/asm/cpu.h
 create mode 100644 arch/um/lkl/include/asm/elf.h
 create mode 100644 arch/um/lkl/include/asm/processor.h
 create mode 100644 arch/um/lkl/include/asm/ptrace.h
 create mode 100644 arch/um/lkl/include/asm/sched.h
 create mode 100644 arch/um/lkl/include/asm/syscalls.h
 create mode 100644 arch/um/lkl/include/asm/syscalls_32.h
 create mode 100644 arch/um/lkl/include/asm/tlb.h
 create mode 100644 arch/um/lkl/include/asm/uaccess.h
 create mode 100644 arch/um/lkl/include/asm/unistd_32.h
 create mode 100644 arch/um/lkl/include/asm/vmlinux.lds.h
 create mode 100644 arch/um/lkl/include/asm/xor.h
 create mode 100644 arch/um/lkl/include/uapi/asm/Kbuild
 create mode 100644 arch/um/lkl/include/uapi/asm/bitsperlong.h
 create mode 100644 arch/um/lkl/include/uapi/asm/byteorder.h
 create mode 100644 arch/um/lkl/include/uapi/asm/siginfo.h
 create mode 100644 arch/um/lkl/include/uapi/asm/swab.h
 create mode 100644 arch/um/lkl/include/uapi/asm/syscalls.h
 create mode 100644 arch/um/lkl/kernel/asm-offsets.c
 create mode 100644 arch/um/lkl/kernel/misc.c
 create mode 100644 arch/um/lkl/kernel/vmlinux.lds.S
MAINTAINERS +8 -0
index 4017e6b760be..12afa00402a1 100644
@@ -9606,6 +9606,14 @@ F:	Documentation/core-api/atomic_ops.rst
 F:	Documentation/core-api/refcount-vs-atomic.rst
 F:	Documentation/memory-barriers.txt
 
+LINUX KERNEL LIBRARY
+M:	Octavian Purdila <[email protected]>
+M:	Hajime Tazaki <[email protected]>
+L:	[email protected]
+S:	Maintained
+F:	arch/um/lkl/
+F:	tools/lkl/
+
 LIS3LV02D ACCELEROMETER DRIVER
 M:	Eric Piel <[email protected]>
 S:	Maintained
arch/um/lkl/.gitignore new file +2 -0
new file mode 100644
index 000000000000..ced1c60d8235
@@ -0,0 +1,2 @@
+kernel/vmlinux.lds
+include/generated
arch/um/lkl/Kconfig new file +56 -0

56 lines not on this page · raw message

arch/um/lkl/Kconfig.debug new file +0 -0
new file mode 100644
index 000000000000..e69de29bb2d1
arch/um/lkl/configs/lkl_defconfig new file +1052 -0

1,052 lines not on this page · raw message

arch/um/lkl/include/asm/Kbuild new file +80 -0

80 lines not on this page · raw message

arch/um/lkl/include/asm/bitsperlong.h new file +11 -0
new file mode 100644
index 000000000000..5745d5e51274
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __LKL_BITSPERLONG_H
+#define __LKL_BITSPERLONG_H
+
+#include <uapi/asm/bitsperlong.h>
+
+#define BITS_PER_LONG __BITS_PER_LONG
+
+#define BITS_PER_LONG_LONG 64
+
+#endif
arch/um/lkl/include/asm/byteorder.h new file +7 -0
new file mode 100644
index 000000000000..5d0c4efaa44b
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_BYTEORDER_H
+#define _ASM_LKL_BYTEORDER_H
+
+#include <uapi/asm/byteorder.h>
+
+#endif /* _ASM_LKL_BYTEORDER_H */
arch/um/lkl/include/asm/cpu.h new file +14 -0
new file mode 100644
index 000000000000..d2b8c501c7b1
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_CPU_H
+#define _ASM_LKL_CPU_H
+
+int lkl_cpu_get(void);
+void lkl_cpu_put(void);
+int lkl_cpu_try_run_irq(int irq);
+int lkl_cpu_init(void);
+void lkl_cpu_shutdown(void);
+void lkl_cpu_wait_shutdown(void);
+void lkl_cpu_change_owner(lkl_thread_t owner);
+void lkl_cpu_set_irqs_pending(void);
+
+#endif /* _ASM_LKL_CPU_H */
arch/um/lkl/include/asm/elf.h new file +15 -0
new file mode 100644
index 000000000000..bb2456d638f4
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_ELF_H
+#define _ASM_LKL_ELF_H
+
+#define elf_check_arch(x) 0
+
+#ifdef CONFIG_64BIT
+#define ELF_CLASS ELFCLASS64
+#else
+#define ELF_CLASS ELFCLASS32
+#endif
+
+#define elf_gregset_t long
+#define elf_fpregset_t double
+#endif
arch/um/lkl/include/asm/processor.h new file +60 -0

60 lines not on this page · raw message

arch/um/lkl/include/asm/ptrace.h new file +25 -0
new file mode 100644
index 000000000000..28199be26dc0
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_PTRACE_H
+#define _ASM_LKL_PTRACE_H
+
+#include <linux/errno.h>
+
+struct task_struct;
+
+#define user_mode(regs) 0
+#define kernel_mode(regs) 1
+#define profile_pc(regs) 0
+#define instruction_pointer(regs) 0
+#define user_stack_pointer(regs) 0
+
+static inline long arch_ptrace(struct task_struct *child, long request,
+			       unsigned long addr, unsigned long data)
+{
+	return -EINVAL;
+}
+
+static inline void ptrace_disable(struct task_struct *child)
+{
+}
+
+#endif
arch/um/lkl/include/asm/sched.h new file +23 -0
new file mode 100644
index 000000000000..4c2635921ec8
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_SCHED_H
+#define _ASM_LKL_SCHED_H
+
+#include <linux/sched.h>
+#include <uapi/asm/host_ops.h>
+
+static inline void thread_sched_jb(void)
+{
+	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD)) {
+		set_ti_thread_flag(current_thread_info(), TIF_SCHED_JB);
+		set_current_state(TASK_UNINTERRUPTIBLE);
+		lkl_ops->jmp_buf_set(&current_thread_info()->sched_jb,
+				     schedule);
+	} else {
+		lkl_bug("%s() can be used only for host task\n", __func__);
+	}
+}
+
+void switch_to_host_task(struct task_struct *);
+int host_task_stub(void *unused);
+
+#endif /*  _ASM_LKL_SCHED_H */
arch/um/lkl/include/asm/syscalls.h new file +18 -0
new file mode 100644
index 000000000000..2eaa870a9020
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_SYSCALLS_H
+#define _ASM_LKL_SYSCALLS_H
+
+int syscalls_init(void);
+void syscalls_cleanup(void);
+long lkl_syscall(long no, long *params);
+void wakeup_idle_host_task(void);
+
+#define sys_mmap sys_mmap_pgoff
+#define sys_mmap2 sys_mmap_pgoff
+#define sys_clone sys_ni_syscall
+#define sys_vfork sys_ni_syscall
+#define sys_rt_sigreturn sys_ni_syscall
+
+#include <asm-generic/syscalls.h>
+
+#endif /* _ASM_LKL_SYSCALLS_H */
arch/um/lkl/include/asm/syscalls_32.h new file +43 -0

43 lines not on this page · raw message

arch/um/lkl/include/asm/tlb.h new file +12 -0
new file mode 100644
index 000000000000..d474890d317d
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_TLB_H
+#define _ASM_LKL_TLB_H
+
+#define tlb_start_vma(tlb, vma)				do { } while (0)
+#define tlb_end_vma(tlb, vma)				do { } while (0)
+#define __tlb_remove_tlb_entry(tlb, pte, address)	do { } while (0)
+#define tlb_flush(tlb)					do { } while (0)
+
+#include <asm-generic/tlb.h>
+
+#endif /* _ASM_LKL_TLB_H */
arch/um/lkl/include/asm/uaccess.h new file +64 -0

64 lines not on this page · raw message

arch/um/lkl/include/asm/unistd_32.h new file +31 -0
new file mode 100644
index 000000000000..8582a55e61e2
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <asm/bitsperlong.h>
+
+#ifndef __SYSCALL
+#define __SYSCALL(x, y)
+#endif
+
+#if __BITS_PER_LONG == 32
+__SYSCALL(__NR3264_truncate, sys32_truncate64)
+__SYSCALL(__NR3264_ftruncate, sys32_ftruncate64)
+
+#ifdef CONFIG_MMU
+__SYSCALL(__NR3264_mmap, sys32_mmap)
+#endif
+
+__SYSCALL(__NR_wait4, sys32_wait4)
+
+__SYSCALL(__NR_pread64, sys32_pread64)
+__SYSCALL(__NR_pwrite64, sys32_pwrite64)
+
+__SYSCALL(__NR_readahead, sys32_readahead)
+#ifdef __ARCH_WANT_SYNC_FILE_RANGE2
+__SYSCALL(__NR_sync_file_range2, sys32_sync_file_range2)
+#else
+__SYSCALL(__NR_sync_file_range, sys32_sync_file_range)
+#endif
+/* mm/fadvise.c */
+__SYSCALL(__NR3264_fadvise64, sys32_fadvise64_64)
+__SYSCALL(__NR_fallocate, sys32_fallocate)
+
+#endif
arch/um/lkl/include/asm/vmlinux.lds.h new file +14 -0
new file mode 100644
index 000000000000..a3c285882dc4
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_VMLINUX_LDS_H
+#define _LKL_VMLINUX_LDS_H
+
+/* we encode our own __ro_after_init section */
+#define RO_AFTER_INIT_DATA
+
+#ifdef __MINGW32__
+#define RODATA_SECTION .rdata
+#endif
+
+#include <asm-generic/vmlinux.lds.h>
+
+#endif
arch/um/lkl/include/asm/xor.h new file +9 -0
new file mode 100644
index 000000000000..286ce75b5d9d
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_XOR_H
+#define _ASM_LKL_XOR_H
+
+#include <asm-generic/xor.h>
+
+#define XOR_SELECT_TEMPLATE(x) (&xor_block_8regs)
+
+#endif /* _ASM_LKL_XOR_H */
arch/um/lkl/include/uapi/asm/Kbuild new file +9 -0
new file mode 100644
index 000000000000..39d9a1f2e8f5
@@ -0,0 +1,9 @@
+# UAPI Header export list
+
+generic-y += elf.h
+generic-y += kvm_para.h
+generic-y += shmparam.h
+generic-y += timex.h
+
+# no header-y since we need special user headers handling
+# see arch/lkl/script/headers.py
arch/um/lkl/include/uapi/asm/bitsperlong.h new file +13 -0
new file mode 100644
index 000000000000..8b4ebf2b0264
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_BITSPERLONG_H
+#define _ASM_UAPI_LKL_BITSPERLONG_H
+
+#ifdef CONFIG_64BIT
+#define __BITS_PER_LONG 64
+#else
+#define __BITS_PER_LONG 32
+#endif
+
+#define __ARCH_WANT_STAT64
+
+#endif /* _ASM_UAPI_LKL_BITSPERLONG_H */
arch/um/lkl/include/uapi/asm/byteorder.h new file +11 -0
new file mode 100644
index 000000000000..3c4a58d2062f
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_BYTEORDER_H
+#define _ASM_UAPI_LKL_BYTEORDER_H
+
+#if defined(CONFIG_BIG_ENDIAN)
+#include <linux/byteorder/big_endian.h>
+#else
+#include <linux/byteorder/little_endian.h>
+#endif
+
+#endif /* _ASM_UAPI_LKL_BYTEORDER_H */
arch/um/lkl/include/uapi/asm/siginfo.h new file +11 -0
new file mode 100644
index 000000000000..811916cf42c8
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_LKL_SIGINFO_H
+#define _ASM_LKL_SIGINFO_H
+
+#ifdef CONFIG_64BIT
+#define __ARCH_SI_PREAMBLE_SIZE	(4 * sizeof(int))
+#endif
+
+#include <asm-generic/siginfo.h>
+
+#endif /* _ASM_LKL_SIGINFO_H */
arch/um/lkl/include/uapi/asm/swab.h new file +11 -0
new file mode 100644
index 000000000000..1a1773e1bd35
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_LKL_SWAB_H
+#define _ASM_LKL_SWAB_H
+
+#ifndef __arch_swab32
+#define __arch_swab32(x) ___constant_swab32(x)
+#endif
+
+#include <asm-generic/swab.h>
+
+#endif /* _ASM_LKL_SWAB_H */
arch/um/lkl/include/uapi/asm/syscalls.h new file +348 -0

348 lines not on this page · raw message

arch/um/lkl/kernel/asm-offsets.c new file +2 -0
new file mode 100644
index 000000000000..6be0763698dc
@@ -0,0 +1,2 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Dummy asm-offsets.c file. Required by kbuild and ready to be used - hint! */
arch/um/lkl/kernel/misc.c new file +60 -0

60 lines not on this page · raw message

arch/um/lkl/kernel/vmlinux.lds.S new file +51 -0

51 lines not on this page · raw message

Signature · 1 line
2.21.0 (Apple Git-122.2)
#12 lore raw

RFC v3 4/26 um lkl: host interface

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

This patch introduces the host operations that define the interface between the LKL and the host. These operations must be provided either by a host library or by the application itself.

Cc Michael Zimmermann <[email protected]>
Cc Patrick Collins <[email protected]>
Cc Pierre-Hugues Husson <[email protected]>
Cc Yuan Liu <[email protected]>
---
 arch/um/lkl/include/asm/host_ops.h      | 10 ++++++++++
 arch/um/lkl/include/uapi/asm/host_ops.h | 26 +++++++++++++++++++++++++
 2 files changed, 36 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/host_ops.h
 create mode 100644 arch/um/lkl/include/uapi/asm/host_ops.h
arch/um/lkl/include/asm/host_ops.h new file +10 -0
new file mode 100644
index 000000000000..65850f394b79
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_HOST_OPS_H
+#define _ASM_LKL_HOST_OPS_H
+
+#include "irq.h"
+#include <uapi/asm/host_ops.h>
+
+extern struct lkl_host_operations *lkl_ops;
+
+#endif
arch/um/lkl/include/uapi/asm/host_ops.h new file +26 -0
new file mode 100644
index 000000000000..7cfb0a93e6a6
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_HOST_OPS_H
+#define _ASM_UAPI_LKL_HOST_OPS_H
+
+/* Defined in {posix,nt}-host.c */
+struct lkl_mutex;
+struct lkl_sem;
+struct lkl_tls_key;
+typedef unsigned long lkl_thread_t;
+struct lkl_jmp_buf {
+	unsigned long buf[32];
+};
+
+/**
+ * lkl_host_operations - host operations used by the Linux kernel
+ *
+ * These operations must be provided by a host library or by the application
+ * itself.
+ *
+ */
+struct lkl_host_operations {
+};
+
+void lkl_bug(const char *fmt, ...);
+
+#endif
Signature · 1 line
2.21.0 (Apple Git-122.2)
#13 lore raw

RFC v3 5/26 um lkl: memory handling

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

LKL is a non MMU architecture and hence there is not much work left to do other than initializing the boot allocator and providing the page and page table definitions.

The backstore memory is allocated via a host operation and the memory size to be used is specified when the kernel is started, in the lkl_start_kernel call.

---
 arch/um/lkl/include/asm/page.h          | 14 ++++++
 arch/um/lkl/include/asm/pgtable.h       | 57 +++++++++++++++++++++
 arch/um/lkl/include/uapi/asm/host_ops.h |  5 ++
 arch/um/lkl/mm/bootmem.c                | 66 +++++++++++++++++++++++++
 4 files changed, 142 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/page.h
 create mode 100644 arch/um/lkl/include/asm/pgtable.h
 create mode 100644 arch/um/lkl/mm/bootmem.c
arch/um/lkl/include/asm/page.h new file +14 -0
new file mode 100644
index 000000000000..e77f3da22031
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_PAGE_H
+#define _ASM_LKL_PAGE_H
+
+#define CONFIG_KERNEL_RAM_BASE_ADDRESS memory_start
+
+#ifndef __ASSEMBLY__
+void free_mem(void);
+void bootmem_init(unsigned long mem_size);
+#endif
+
+#include <asm-generic/page.h>
+
+#endif /* _ASM_LKL_PAGE_H */
arch/um/lkl/include/asm/pgtable.h new file +57 -0

57 lines not on this page · raw message

arch/um/lkl/include/uapi/asm/host_ops.h +5 -0
index 7cfb0a93e6a6..6bbc94c120be 100644
@@ -17,8 +17,13 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @mem_alloc - allocate memory
+ * @mem_free - free memory
+ *
  */
 struct lkl_host_operations {
+	void *(*mem_alloc)(unsigned long mem);
+	void (*mem_free)(void *mem);
 };
 
 void lkl_bug(const char *fmt, ...);
arch/um/lkl/mm/bootmem.c new file +66 -0

66 lines not on this page · raw message

Signature · 1 line
2.21.0 (Apple Git-122.2)
#14 lore raw

RFC v3 6/26 um lkl: kernel threads support

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

LKL does not support user processes but it must support kernel threads as part as the normal kernel work-flow. It uses host operations to create and terminate host threads that are going to run the kernel threads. It also uses semaphores to synchronize those threads and to allow the Linux kernel scheduler to control how the kernel threads run.

Each kernel thread runs in a host threads and has a host semaphore associated with it - the thread's scheduling semaphore. The semaphore counter is initialized to 0. The first thing a kernel thread does after getting spawned, before running any kernel code, is to perform a down operation to block the thread.

The kernel controls host threads scheduling by performing up and down operations on the scheduling semaphore. In __switch_context an up operation on the next thread is performed to wake up a blocked thread, and a down operation is performed on the prev thread to block it.

A thread is terminated by marking it in free_thread_info and performing an up operation on the scheduling semaphore at which point the marked thread will terminate itself.

---
 arch/um/lkl/include/asm/thread_info.h   |  70 ++++++++
 arch/um/lkl/include/uapi/asm/host_ops.h |  55 ++++++
 arch/um/lkl/kernel/cpu.c                | 223 +++++++++++++++++++++++
 arch/um/lkl/kernel/threads.c            | 227 ++++++++++++++++++++++++
 4 files changed, 575 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/thread_info.h
 create mode 100644 arch/um/lkl/kernel/cpu.c
 create mode 100644 arch/um/lkl/kernel/threads.c
arch/um/lkl/include/asm/thread_info.h new file +70 -0
new file mode 100644
index 000000000000..da4e75fc7b10
@@ -0,0 +1,70 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_THREAD_INFO_H
+#define _ASM_LKL_THREAD_INFO_H
+
+#define THREAD_SIZE	       (4096)
+
+#ifndef __ASSEMBLY__
+#include <asm/types.h>
+#include <asm/processor.h>
+#include <asm/host_ops.h>
+
+typedef struct {
+	unsigned long seg;
+} mm_segment_t;
+
+struct thread_info {
+	struct task_struct *task;
+	unsigned long flags;
+	int preempt_count;
+	mm_segment_t addr_limit;
+	struct lkl_sem *sched_sem;
+	struct lkl_jmp_buf sched_jb;
+	bool dead;
+	lkl_thread_t tid;
+	struct task_struct *prev_sched;
+	unsigned long stackend;
+};
+
+#define INIT_THREAD_INFO(tsk)				\
+{							\
+	.task		= &tsk,				\
+	.preempt_count	= INIT_PREEMPT_COUNT,		\
+	.flags		= 0,				\
+	.addr_limit	= KERNEL_DS,			\
+}
+
+/* how to get the thread information struct from C */
+extern struct thread_info *_current_thread_info;
+static inline struct thread_info *current_thread_info(void)
+{
+	return _current_thread_info;
+}
+
+/* thread information allocation */
+unsigned long *alloc_thread_stack_node(struct task_struct *, int node);
+void free_thread_stack(struct task_struct *tsk);
+
+void threads_init(void);
+void threads_cleanup(void);
+
+#define TIF_SYSCALL_TRACE		0
+#define TIF_NOTIFY_RESUME		1
+#define TIF_SIGPENDING			2
+#define TIF_NEED_RESCHED		3
+#define TIF_RESTORE_SIGMASK		4
+#define TIF_MEMDIE			5
+#define TIF_NOHZ			6
+#define TIF_SCHED_JB			7
+#define TIF_HOST_THREAD			8
+
+#define __HAVE_THREAD_FUNCTIONS
+
+#define task_thread_info(task)	((struct thread_info *)(task)->stack)
+#define task_stack_page(task)	((task)->stack)
+void setup_thread_stack(struct task_struct *p, struct task_struct *org);
+#define end_of_stack(p) (&task_thread_info(p)->stackend)
+
+#endif /* __ASSEMBLY__ */
+
+#endif
arch/um/lkl/include/uapi/asm/host_ops.h +55 -0

70 lines not on this page · raw message

arch/um/lkl/kernel/cpu.c new file +223 -0

223 lines not on this page · raw message

arch/um/lkl/kernel/threads.c new file +227 -0

227 lines not on this page · raw message

Signature · 1 line
2.21.0 (Apple Git-122.2)
#15 lore raw

RFC v3 7/26 um lkl: interrupt support

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

Add APIs that allows the host to reserve and free and interrupt number and also to trigger an interrupt.

The trigger operation will simply store the interrupt data in queue. The interrupt handler is run later, at the first opportunity it has to avoid races with any kernel threads.

Currently, interrupts are run on the first interrupt enable operation if interrupts are disabled and if we are not already in interrupt context.

When triggering an interrupt, it uses GCC's built-in functions for atomic memory access to synchronize and simple boolean flags.

---
 arch/um/lkl/include/asm/irq.h             |  13 ++
 arch/um/lkl/include/uapi/asm/irq.h        |  36 ++++
 arch/um/lkl/include/uapi/asm/sigcontext.h |  16 ++
 arch/um/lkl/kernel/irq.c                  | 193 ++++++++++++++++++++++
 4 files changed, 258 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/irq.h
 create mode 100644 arch/um/lkl/include/uapi/asm/irq.h
 create mode 100644 arch/um/lkl/include/uapi/asm/sigcontext.h
 create mode 100644 arch/um/lkl/kernel/irq.c
arch/um/lkl/include/asm/irq.h new file +13 -0
new file mode 100644
index 000000000000..948fc54cb76c
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_IRQ_H
+#define _ASM_LKL_IRQ_H
+
+#define IRQ_STATUS_BITS		(sizeof(long) * 8)
+#define NR_IRQS			((int)(IRQ_STATUS_BITS * IRQ_STATUS_BITS))
+
+void run_irqs(void);
+void set_irq_pending(int irq);
+
+#include <uapi/asm/irq.h>
+
+#endif
arch/um/lkl/include/uapi/asm/irq.h new file +36 -0
new file mode 100644
index 000000000000..666628b233eb
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_IRQ_H
+#define _ASM_UAPI_LKL_IRQ_H
+
+/**
+ * lkl_trigger_irq - generate an interrupt
+ *
+ * This function is used by the device host side to signal its Linux counterpart
+ * that some event happened.
+ *
+ * @irq - the irq number to signal
+ */
+int lkl_trigger_irq(int irq);
+
+/**
+ * lkl_get_free_irq - find and reserve a free IRQ number
+ *
+ * This function is called by the host device code to find an unused IRQ number
+ * and reserved it for its own use.
+ *
+ * @user - a string to identify the user
+ * @returns - and irq number that can be used by request_irq or an negative
+ * value in case of an error
+ */
+int lkl_get_free_irq(const char *user);
+
+/**
+ * lkl_put_irq - release an IRQ number previously obtained with lkl_get_free_irq
+ *
+ * @irq - irq number to release
+ * @user - string identifying the user; should be the same as the one passed to
+ * lkl_get_free_irq when the irq number was obtained
+ */
+void lkl_put_irq(int irq, const char *name);
+
+#endif
arch/um/lkl/include/uapi/asm/sigcontext.h new file +16 -0
new file mode 100644
index 000000000000..2f4848843d1d
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_SIGCONTEXT_H
+#define _ASM_UAPI_LKL_SIGCONTEXT_H
+
+#include <asm/ptrace.h>
+
+struct pt_regs {
+	void *irq_data;
+};
+
+struct sigcontext {
+	struct pt_regs regs;
+	unsigned long oldmask;
+};
+
+#endif
arch/um/lkl/kernel/irq.c new file +193 -0

193 lines not on this page · raw message

Signature · 1 line
2.21.0 (Apple Git-122.2)
#16 lore raw

Re: [RFC v3 07/26] um lkl: interrupt support

Quoted 212 lines · Hajime Tazaki

From: Octavian Purdila <[email protected]>

Add APIs that allows the host to reserve and free and interrupt number and also to trigger an interrupt.

The trigger operation will simply store the interrupt data in queue. The interrupt handler is run later, at the first opportunity it has to avoid races with any kernel threads.

Currently, interrupts are run on the first interrupt enable operation if interrupts are disabled and if we are not already in interrupt context.

When triggering an interrupt, it uses GCC's built-in functions for atomic memory access to synchronize and simple boolean flags.

---
  arch/um/lkl/include/asm/irq.h             |  13 ++
  arch/um/lkl/include/uapi/asm/irq.h        |  36 ++++
  arch/um/lkl/include/uapi/asm/sigcontext.h |  16 ++
  arch/um/lkl/kernel/irq.c                  | 193 ++++++++++++++++++++++
  4 files changed, 258 insertions(+)

create mode 100644 arch/um/lkl/include/asm/irq.h create mode 100644 arch/um/lkl/include/uapi/asm/irq.h create mode 100644 arch/um/lkl/include/uapi/asm/sigcontext.h create mode 100644 arch/um/lkl/kernel/irq.c

arch/um/lkl/include/asm/irq.h new file +13 -0
new file mode 100644
index 000000000000..948fc54cb76c
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_IRQ_H
+#define _ASM_LKL_IRQ_H
+
+#define IRQ_STATUS_BITS		(sizeof(long) * 8)
+#define NR_IRQS			((int)(IRQ_STATUS_BITS * IRQ_STATUS_BITS))
+
+void run_irqs(void);
+void set_irq_pending(int irq);
+
+#include <uapi/asm/irq.h>
+
+#endif
arch/um/lkl/include/uapi/asm/irq.h new file +36 -0
new file mode 100644
index 000000000000..666628b233eb
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_IRQ_H
+#define _ASM_UAPI_LKL_IRQ_H
+
+/**
+ * lkl_trigger_irq - generate an interrupt
+ *
+ * This function is used by the device host side to signal its Linux counterpart
+ * that some event happened.
+ *
+ * @irq - the irq number to signal
+ */
+int lkl_trigger_irq(int irq);
+
+/**
+ * lkl_get_free_irq - find and reserve a free IRQ number
+ *
+ * This function is called by the host device code to find an unused IRQ number
+ * and reserved it for its own use.
+ *
+ * @user - a string to identify the user
+ * @returns - and irq number that can be used by request_irq or an negative
+ * value in case of an error
+ */
+int lkl_get_free_irq(const char *user);
+
+/**
+ * lkl_put_irq - release an IRQ number previously obtained with lkl_get_free_irq
+ *
+ * @irq - irq number to release
+ * @user - string identifying the user; should be the same as the one passed to
+ * lkl_get_free_irq when the irq number was obtained
+ */
+void lkl_put_irq(int irq, const char *name);
+
+#endif
arch/um/lkl/include/uapi/asm/sigcontext.h new file +16 -0
new file mode 100644
index 000000000000..2f4848843d1d
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_SIGCONTEXT_H
+#define _ASM_UAPI_LKL_SIGCONTEXT_H
+
+#include <asm/ptrace.h>
+
+struct pt_regs {
+	void *irq_data;
+};
+
+struct sigcontext {
+	struct pt_regs regs;
+	unsigned long oldmask;
+};
+
+#endif
arch/um/lkl/kernel/irq.c new file +93 -0
new file mode 100644
index 000000000000..e3b59e46ca50
@@ -0,0 +1,193 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/kernel.h>
+#include <linux/init.h>
+#include <linux/irq.h>
+#include <linux/hardirq.h>
+#include <asm/irq_regs.h>
+#include <linux/sched.h>
+#include <linux/seq_file.h>
+#include <linux/tick.h>
+#include <asm/irqflags.h>
+#include <asm/host_ops.h>
+#include <asm/cpu.h>
+
+/*
+ * To avoid much overhead we use an indirect approach: the irqs are marked using
+ * a bitmap (array of longs) and a summary of the modified bits is kept in a
+ * separate "index" long - one bit for each sizeof(long). Thus we can support
+ * 4096 irqs on 64bit platforms and 1024 irqs on 32bit platforms.
+ *
+ * Whenever an irq is trigger both the array and the index is updated. To find
+ * which irqs were triggered we first search the index and then the
+ * corresponding part of the arrary.
+ */
+static unsigned long irq_status[NR_IRQS / IRQ_STATUS_BITS];
+static unsigned long irq_index_status;
+
+static inline unsigned long test_and_clear_irq_index_status(void)
+{
+	if (!irq_index_status)
+		return 0;
+	return __sync_fetch_and_and(&irq_index_status, 0);
+}
+
+static inline unsigned long test_and_clear_irq_status(int index)
+{
+	if (!&irq_status[index])
+		return 0;
+	return __sync_fetch_and_and(&irq_status[index], 0);
+}
+
+void set_irq_pending(int irq)
+{
+	int index = irq / IRQ_STATUS_BITS;
+	int bit = irq % IRQ_STATUS_BITS;
+
+	__sync_fetch_and_or(&irq_status[index], BIT(bit));
+	__sync_fetch_and_or(&irq_index_status, BIT(index));
+}
+
+static struct irq_info {
+	const char *user;
+} irqs[NR_IRQS];
+
+static bool irqs_enabled;
+
+static struct pt_regs dummy;
+
+static void run_irq(int irq)
+{
+	unsigned long flags;
+	struct pt_regs *old_regs = set_irq_regs((struct pt_regs *)&dummy);
+
+	/* interrupt handlers need to run with interrupts disabled */
+	local_irq_save(flags);
+	irq_enter();
+	generic_handle_irq(irq);
+	irq_exit();
+	set_irq_regs(old_regs);
+	local_irq_restore(flags);
+}
+
+/**
+ * This function can be called from arbitrary host threads, so do not
+ * issue any Linux calls (e.g. prink) if lkl_cpu_get() was not issued
+ * before.
+ */
+int lkl_trigger_irq(int irq)
+{
+	int ret;
+
+	if (!irq || irq > NR_IRQS)
+		return -EINVAL;
+
+	ret = lkl_cpu_try_run_irq(irq);
+	if (ret <= 0)
+		return ret;
+
+	/*
+	 * Since this can be called from Linux context (e.g. lkl_trigger_irq ->
+	 * IRQ -> softirq -> lkl_trigger_irq) make sure we are actually allowed
+	 * to run irqs at this point
+	 */
+	if (!irqs_enabled) {

arch/um/lkl/kernel/irq.c line 103

Quoted 10 lines
arch/um/lkl/kernel/irq.c new file +10 -0
+		set_irq_pending(irq);
+		lkl_cpu_put();
+		return 0;
+	}
+
+	run_irq(irq);
+
+	lkl_cpu_put();
+
+	return 0;

Isn't that just:

	if (irqs_enabled)
		run_irq(irq);
	else
		set_irq_pending(irq);

	lkl_cpu_put();

	return 0;
Quoted 91 lines
arch/um/lkl/kernel/irq.c new file +90 -0
+}
+
+static inline void for_each_bit(unsigned long word, void (*f)(int, int), int j)
+{
+	int i = 0;
+
+	while (word) {
+		if (word & 1)
+			f(i, j);
+		word >>= 1;
+		i++;
+	}
+}
+
+static inline void deliver_irq(int bit, int index)
+{
+	run_irq(index * IRQ_STATUS_BITS + bit);
+}
+
+static inline void check_irq_status(int i, int unused)
+{
+	for_each_bit(test_and_clear_irq_status(i), deliver_irq, i);
+}
+
+void run_irqs(void)
+{
+	for_each_bit(test_and_clear_irq_index_status(), check_irq_status, 0);
+}
+
+int show_interrupts(struct seq_file *p, void *v)
+{
+	return 0;
+}
+
+int lkl_get_free_irq(const char *user)
+{
+	int i;
+	int ret = -EBUSY;
+
+	/* 0 is not a valid IRQ */
+	for (i = 1; i < NR_IRQS; i++) {
+		if (!irqs[i].user) {
+			irqs[i].user = user;
+			irq_set_chip_and_handler(i, &dummy_irq_chip,
+						 handle_simple_irq);
+			ret = i;
+			break;
+		}
+	}
+
+	return ret;
+}
+
+void lkl_put_irq(int i, const char *user)
+{
+	if (!irqs[i].user || strcmp(irqs[i].user, user) != 0) {
+		WARN("%s tried to release %s's irq %d", user, irqs[i].user, i);
+		return;
+	}
+
+	irqs[i].user = NULL;
+}
+
+unsigned long arch_local_save_flags(void)
+{
+	return irqs_enabled;
+}
+
+void arch_local_irq_restore(unsigned long flags)
+{
+	if (flags == ARCH_IRQ_ENABLED && irqs_enabled == ARCH_IRQ_DISABLED &&
+	    !in_interrupt())
+		run_irqs();
+	irqs_enabled = flags;
+}
+
+void init_IRQ(void)
+{
+	int i;
+
+	for (i = 0; i < NR_IRQS; i++)
+		irq_set_chip_and_handler(i, &dummy_irq_chip, handle_simple_irq);
+
+	pr_info("lkl: irqs initialized\n");
+}
+
+void cpu_yield_to_irqs(void)
+{
+	cpu_relax();
+}
Signature · 2 lines
Anton R. Ivanov
https://www.kot-begemot.co.uk/
#17 lore raw

Re: [RFC v3 07/26] um lkl: interrupt support

Quoted 35 lines · Anton Ivanov
Quoted 22 lines
arch/um/lkl/kernel/irq.c new file +22 -0
+/**
+ * This function can be called from arbitrary host threads, so do not
+ * issue any Linux calls (e.g. prink) if lkl_cpu_get() was not issued
+ * before.
+ */
+int lkl_trigger_irq(int irq)
+{
+	int ret;
+
+	if (!irq || irq > NR_IRQS)
+		return -EINVAL;
+
+	ret = lkl_cpu_try_run_irq(irq);
+	if (ret <= 0)
+		return ret;
+
+	/*
+	 * Since this can be called from Linux context (e.g. lkl_trigger_irq ->
+	 * IRQ -> softirq -> lkl_trigger_irq) make sure we are actually allowed
+	 * to run irqs at this point
+	 */
+	if (!irqs_enabled) {

arch/um/lkl/kernel/irq.c line 103

Quoted 10 lines
arch/um/lkl/kernel/irq.c new file +10 -0
+		set_irq_pending(irq);
+		lkl_cpu_put();
+		return 0;
+	}
+
+	run_irq(irq);
+
+	lkl_cpu_put();
+
+	return 0;

Isn't that just:

Quoted 8 lines
	if (irqs_enabled)
		run_irq(irq);
	else
		set_irq_pending(irq);

	lkl_cpu_put();

	return 0;

Thanks, this is much cleaner. I will fix this in the next turn.

-- Hajime

#18 lore raw

RFC v3 8/26 um lkl: system call interface and application API

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

The LKL application API is based on the kernel system call interface in order to offer a stable API to applications. Note that we can't offer the full Linux system call interface due to LKL limitations such as lack of virtual memory, signal, user processes, etc.

The host is using the LKL interrupt mechanism (lkl_trigger_irq) to initiate a system call. The system call is executed in the context of the init process.

To avoid collisions between the Linux API and the LKL API (e.g. struct stat, MKNOD, etc.) we use a python script to modify the user headers and to prefix all of the global symbols (structures, typedefs, defines) with LKL, lkl, _LKL, _lkl, __LKL or __lkl.

Cc Conrad Meyer <[email protected]>
Cc Jens Staal <[email protected]>
Cc Lai Jiangshan <[email protected]>
Cc Luca Dariz <[email protected]>
Cc Michael Zimmermann <[email protected]>
Cc Patrick Collins <[email protected]>
Cc Pierre-Hugues Husson <[email protected]>
Cc Yuan Liu <[email protected]>
---
 arch/um/lkl/include/asm/unistd.h        |  27 ++++
 arch/um/lkl/include/uapi/asm/host_ops.h |  14 ++
 arch/um/lkl/include/uapi/asm/unistd.h   |  16 ++
 arch/um/lkl/kernel/syscalls.c           | 192 +++++++++++++++++++++++
 arch/um/lkl/kernel/syscalls_32.c        | 159 +++++++++++++++++++
 arch/um/lkl/scripts/headers_install.py  | 196 ++++++++++++++++++++++++
 6 files changed, 604 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/unistd.h
 create mode 100644 arch/um/lkl/include/uapi/asm/unistd.h
 create mode 100644 arch/um/lkl/kernel/syscalls.c
 create mode 100644 arch/um/lkl/kernel/syscalls_32.c
 create mode 100755 arch/um/lkl/scripts/headers_install.py
arch/um/lkl/include/asm/unistd.h new file +27 -0
new file mode 100644
index 000000000000..ca0eac0f2827
@@ -0,0 +1,27 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <uapi/asm/unistd.h>
+
+#define __SC_ASCII(t, a) #t "," #a
+
+#define __ASCII_MAP0(m, ...)
+#define __ASCII_MAP1(m, t, a) m(t, a)
+#define __ASCII_MAP2(m, t, a, ...) m(t, a) "," __ASCII_MAP1(m, __VA_ARGS__)
+#define __ASCII_MAP3(m, t, a, ...) m(t, a) "," __ASCII_MAP2(m, __VA_ARGS__)
+#define __ASCII_MAP4(m, t, a, ...) m(t, a) "," __ASCII_MAP3(m, __VA_ARGS__)
+#define __ASCII_MAP5(m, t, a, ...) m(t, a) "," __ASCII_MAP4(m, __VA_ARGS__)
+#define __ASCII_MAP6(m, t, a, ...) m(t, a) "," __ASCII_MAP5(m, __VA_ARGS__)
+#define __ASCII_MAP(n, ...) __ASCII_MAP##n(__VA_ARGS__)
+
+#ifdef __MINGW32__
+#define SECTION_ATTRS "n0"
+#else
+#define SECTION_ATTRS "a"
+#endif
+
+#define __SYSCALL_DEFINE_ARCH(x, name, ...)				\
+	asm(".section .syscall_defs,\"" SECTION_ATTRS "\"\n"		\
+	    ".ascii \"#ifdef __NR" #name "\\n\"\n"			\
+	    ".ascii \"SYSCALL_DEFINE" #x "(" #name ","			\
+	    __ASCII_MAP(x, __SC_ASCII, __VA_ARGS__) ")\\n\"\n"		\
+	    ".ascii \"#endif\\n\"\n"					\
+	    ".section .text\n");
arch/um/lkl/include/uapi/asm/host_ops.h +14 -0
index 19924fc7c718..1c839d7139f8 100644
@@ -36,6 +36,15 @@ struct lkl_jmp_buf {
  * @thread_join - wait for the given thread to terminate. Returns 0
  * for success, -1 otherwise
  *
+ * @tls_alloc - allocate a thread local storage key; returns 0 if successful; if
+ * destructor is not NULL it will be called when a thread terminates with its
+ * argument set to the current thread local storage value
+ * @tls_free - frees a thread local storage key; returns 0 if successful
+ * @tls_set - associate data to the thread local storage key; returns 0 if
+ * successful
+ * @tls_get - return data associated with the thread local storage key or NULL
+ * on error
+ *
  * @mem_alloc - allocate memory
  * @mem_free - free memory
  *
@@ -71,6 +80,11 @@ struct lkl_host_operations {
 	lkl_thread_t (*thread_self)(void);
 	int (*thread_equal)(lkl_thread_t a, lkl_thread_t b);
 
+	struct lkl_tls_key *(*tls_alloc)(void (*destructor)(void *));
+	void (*tls_free)(struct lkl_tls_key *key);
+	int (*tls_set)(struct lkl_tls_key *key, void *data);
+	void *(*tls_get)(struct lkl_tls_key *key);
+
 	void *(*mem_alloc)(unsigned long mem);
 	void (*mem_free)(void *mem);
 
arch/um/lkl/include/uapi/asm/unistd.h new file +16 -0
new file mode 100644
index 000000000000..072474a4060d
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#define __ARCH_WANT_SYSCALL_NO_AT
+#define __ARCH_WANT_SYSCALL_DEPRECATED
+#define __ARCH_WANT_SYSCALL_NO_FLAGS
+#define __ARCH_WANT_RENAMEAT
+#define __ARCH_WANT_NEW_STAT
+#define __ARCH_WANT_SET_GET_RLIMIT
+#define __ARCH_WANT_TIME32_SYSCALLS
+
+#include <asm/bitsperlong.h>
+
+#if __BITS_PER_LONG == 64
+#define __ARCH_WANT_SYS_NEWFSTATAT
+#endif
+
+#include <asm-generic/unistd.h>
arch/um/lkl/kernel/syscalls.c new file +192 -0

192 lines not on this page · raw message

arch/um/lkl/kernel/syscalls_32.c new file +159 -0

159 lines not on this page · raw message

arch/um/lkl/scripts/headers_install.py new file +196 -0

196 lines not on this page · raw message

Signature · 1 line
2.21.0 (Apple Git-122.2)
#19 lore raw

RFC v3 9/26 um lkl: timers, time and delay support

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

Clockevent driver based on host timer operations and clocksource driver and udelay support based on host time operations.

---
 arch/um/lkl/include/uapi/asm/host_ops.h |  13 +++
 arch/um/lkl/kernel/time.c               | 145 ++++++++++++++++++++++++
 2 files changed, 158 insertions(+)
 create mode 100644 arch/um/lkl/kernel/time.c
arch/um/lkl/include/uapi/asm/host_ops.h +13 -0
index 1c839d7139f8..c9f77dd7fbe7 100644
@@ -48,6 +48,13 @@ struct lkl_jmp_buf {
  * @mem_alloc - allocate memory
  * @mem_free - free memory
  *
+ * @timer_create - allocate a host timer that runs fn(arg) when the timer
+ * fires.
+ * @timer_free - disarms and free the timer
+ * @timer_set_oneshot - arm the timer to fire once, after delta ns.
+ * @timer_set_periodic - arm the timer to fire periodically, with a period of
+ * delta ns.
+ *
  * @gettid - returns the host thread id of the caller, which need not
  * be the same as the handle returned by thread_create
  *
@@ -88,6 +95,12 @@ struct lkl_host_operations {
 	void *(*mem_alloc)(unsigned long mem);
 	void (*mem_free)(void *mem);
 
+	unsigned long long (*time)(void);
+
+	void *(*timer_alloc)(void (*fn)(void *), void *arg);
+	int (*timer_set_oneshot)(void *timer, unsigned long delta);
+	void (*timer_free)(void *timer);
+
 	long (*gettid)(void);
 
 	void (*jmp_buf_set)(struct lkl_jmp_buf *jmpb, void (*f)(void));
arch/um/lkl/kernel/time.c new file +145 -0

145 lines not on this page · raw message

Signature · 7 lines
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-um
#20 lore raw

RFC v3 10/26 um lkl: basic kernel console support

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

Write operations are deferred to the host print operation.

---
 arch/um/lkl/include/uapi/asm/host_ops.h |  4 +++
 arch/um/lkl/kernel/console.c            | 42 +++++++++++++++++++++++++
 2 files changed, 46 insertions(+)
 create mode 100644 arch/um/lkl/kernel/console.c
arch/um/lkl/include/uapi/asm/host_ops.h +4 -0
index c9f77dd7fbe7..986340ba9d8d 100644
@@ -17,6 +17,8 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @print - optional operation that receives console messages
+ *
  * @sem_alloc - allocate a host semaphore an initialize it to count
  * @sem_free - free a host semaphore
  * @sem_up - perform an up operation on the semaphore
@@ -70,6 +72,8 @@ struct lkl_jmp_buf {
  * @jmp_buf_longjmp - perform a jump back to the saved jump buffer
  */
 struct lkl_host_operations {
+	void (*print)(const char *str, int len);
+
 	struct lkl_sem *(*sem_alloc)(int count);
 	void (*sem_free)(struct lkl_sem *sem);
 	void (*sem_up)(struct lkl_sem *sem);
arch/um/lkl/kernel/console.c new file +42 -0

42 lines not on this page · raw message

Signature · 1 line
2.21.0 (Apple Git-122.2)
#21 lore raw

RFC v3 11/26 um lkl: initialization and cleanup

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

Add the lkl_start_kernel and lkl_sys_halt APIs that start and respectively stops the Linux kernel.

lkl_start_kernel creates a separate threads that will run the initial and idle kernel thread. It waits for the kernel to complete initialization before returning, to avoid races with system calls issues by the host application.

During the setup phase, we create "/init" in initial ramfs root filesystem to avoid mounting the "real" rootfs since ramfs is good enough for now.

lkl_sys_halt will shutdown the kernel, terminate all threads and free all host resources used by the kernel before returning.

This patch also introduces idle CPU handling since it is closely related to the shutdown process. A host semaphore is used to wait for new interrupts when the kernel switches the CPU to idle to avoid wasting host CPU cycles. When the kernel is shutdown we terminate the idle thread at the first CPU idle event.

Cc Michael Zimmermann <[email protected]>
Cc Patrick Collins <[email protected]>
Cc Yuan Liu <[email protected]>
---
 arch/um/lkl/include/asm/setup.h         |   7 +
 arch/um/lkl/include/uapi/asm/host_ops.h |  20 +++
 arch/um/lkl/kernel/setup.c              | 189 ++++++++++++++++++++++++
 3 files changed, 216 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/setup.h
 create mode 100644 arch/um/lkl/kernel/setup.c
arch/um/lkl/include/asm/setup.h new file +7 -0
new file mode 100644
index 000000000000..b40955208cc6
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_SETUP_H
+#define _ASM_LKL_SETUP_H
+
+#define COMMAND_LINE_SIZE 4096
+
+#endif
arch/um/lkl/include/uapi/asm/host_ops.h +20 -0
index 986340ba9d8d..fe4382c3050a 100644
@@ -19,6 +19,8 @@ struct lkl_jmp_buf {
  *
  * @print - optional operation that receives console messages
  *
+ * @panic - called during a kernel panic
+ *
  * @sem_alloc - allocate a host semaphore an initialize it to count
  * @sem_free - free a host semaphore
  * @sem_up - perform an up operation on the semaphore
@@ -73,6 +75,7 @@ struct lkl_jmp_buf {
  */
 struct lkl_host_operations {
 	void (*print)(const char *str, int len);
+	void (*panic)(void);
 
 	struct lkl_sem *(*sem_alloc)(int count);
 	void (*sem_free)(struct lkl_sem *sem);
@@ -111,6 +114,23 @@ struct lkl_host_operations {
 	void (*jmp_buf_longjmp)(struct lkl_jmp_buf *jmpb, int val);
 };
 
+/**
+ * lkl_start_kernel - registers the host operations and starts the kernel
+ *
+ * The function returns only after the kernel is shutdown with lkl_sys_halt.
+ *
+ * @lkl_ops - pointer to host operations
+ * @cmd_line - format for command line string that is going to be used to
+ * generate the Linux kernel command line
+ */
+int lkl_start_kernel(struct lkl_host_operations *lkl_ops, const char *cmd_line,
+		     ...);
+
+/**
+ * lkl_is_running - returns 1 if the kernel is currently running
+ */
+int lkl_is_running(void);
+
 int lkl_printf(const char *fmt, ...);
 void lkl_bug(const char *fmt, ...);
 
arch/um/lkl/kernel/setup.c new file +189 -0

189 lines not on this page · raw message

Signature · 1 line
2.21.0 (Apple Git-122.2)
#22 lore raw

RFC v3 12/26 um lkl: plug in the build system

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

Basic Makefiles for building LKL. Add a new architecture specific target for installing the resulting library files and headers.

To make LKL binaries build, UML introduced an additional option, UMMODE variable, to switch the output file of build: kernel (default), or library (LKL). Those modes are not able to be ON at the same time.

To build on library mode, users do the following:

make defconfig ARCH=um UMMODE=library
make ARCH=um UMMODE=library

---
 arch/um/Kconfig             | 13 +++++++
 arch/um/Makefile            | 19 ++++++++++-
 arch/um/Makefile.um         | 10 ++++++
 arch/um/lkl/Kconfig         | 31 +++++++++++++++--
 arch/um/lkl/Makefile        | 67 +++++++++++++++++++++++++++++++++++++
 arch/um/lkl/auto.conf       |  1 +
 arch/um/lkl/kernel/Makefile |  4 +++
 arch/um/lkl/mm/Makefile     |  1 +
 8 files changed, 143 insertions(+), 3 deletions(-)
 create mode 100644 arch/um/Makefile.um
 create mode 100644 arch/um/lkl/Makefile
 create mode 100644 arch/um/lkl/auto.conf
 create mode 100644 arch/um/lkl/kernel/Makefile
 create mode 100644 arch/um/lkl/mm/Makefile
arch/um/Kconfig +13 -0
index 0917f8443c28..fce63e42198d 100644
@@ -5,6 +5,10 @@ menu "UML-specific options"
 config UML
 	bool
 	default y
+
+config UMMODE_KERN
+	bool "UML mode: kernel mode"
+	default y if "$(UMMODE)" = "kernel"
 	select ARCH_HAS_KCOV
 	select ARCH_NO_PREEMPT
 	select HAVE_ARCH_AUDITSYSCALL
@@ -20,7 +24,12 @@ config UML
 	select GENERIC_CLOCKEVENTS
 	select HAVE_GCC_PLUGINS
 	select TTY # Needed for line.c
+        help
+	  This mode switches a mode to build a regular kernel executable
+          of UML.
+
 
+if UMMODE_KERN
 config MMU
 	bool
 	default y
@@ -200,6 +209,10 @@ config UML_TIME_TRAVEL_SUPPORT
 
 	  It is safe to say Y, but you probably don't need this.
 
+endif #UMMODE_KERN
+
 endmenu
 
 source "arch/um/drivers/Kconfig"
+
+source "arch/um/lkl/Kconfig"
arch/um/Makefile +18 -1
index d2daa206872d..f2a537f700c2 100644
@@ -1,3 +1,4 @@
+# SPDX-License-Identifier: GPL-2.0
 #
 # This file is included by the global makefile so that you can add your own
 # architecture-specific flags and dependencies.
@@ -44,7 +45,7 @@ HOST_DIR := arch/$(HEADER_ARCH)
 include $(ARCH_DIR)/Makefile-skas
 include $(HOST_DIR)/Makefile.um
 
-core-y += $(HOST_DIR)/um/
+
 
 SHARED_HEADERS	:= $(ARCH_DIR)/include/shared
 ARCH_INCLUDE	:= -I$(srctree)/$(SHARED_HEADERS)
@@ -144,5 +145,21 @@ CLEAN_FILES += linux x.i gmon.out
 archclean:
 	@find . \( -name '*.bb' -o -name '*.bbg' -o -name '*.da' \
 		-o -name '*.gcov' \) -type f -print | xargs rm -f
+	$(Q)rm -rf $(srctree)/$(LKL_DIR)/include/generated
 
 export HEADER_ARCH SUBARCH USER_CFLAGS CFLAGS_NO_HARDENING OS DEV_NULL_PATH
+
+
+
+# SPDX-License-Identifier: GPL-2.0
+# select mode of UML build
+UMMODE ?= kernel
+LKL_DIR := $(ARCH_DIR)/lkl
+
+ifeq ($(UMMODE),kernel)
+	include $(ARCH_DIR)/Makefile.um
+else ifeq ($(UMMODE),library)
+	include $(ARCH_DIR)/lkl/Makefile
+endif
+
+export UMMODE LKL_DIR
arch/um/Makefile.um new file +10 -0
new file mode 100644
index 000000000000..be7ee4d1adde
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# This file is included by the global makefile so that you can add your own
+# architecture-specific flags and dependencies.
+#
+# Copyright (C) 2002 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
+# Licensed under the GPL
+#
+
+core-y += $(HOST_DIR)/um/
arch/um/lkl/Kconfig +29 -2
index f7b641ea7aef..f72b423fad5b 100644
@@ -1,6 +1,25 @@
 # SPDX-License-Identifier: GPL-2.0
 
-config UML_LKL
+menu "LKL-specific options"
+
+config UML
+	bool
+	default y
+
+config UMMODE_LIB
+	bool "UML mode: library mode"
+	depends on !UMMODE_KERN
+	select LKL
+	default y if "$(UMMODE)" = "library"
+	help
+	  This mode switches a mode to build a library of UML (Linux
+	  Kernel Library/LKL).  This switch is exclusive to "kernel mode"
+	  of UML, which is traditional mode of UML.
+
+	  For more detail about LKL, see
+	  <file:Documentation/virt/uml/lkl.txt>.
+
+config LKL
        def_bool y
        depends on !SMP && !MMU && !COREDUMP && !SECCOMP && !UPROBES && !COMPAT && !USER_RETURN_NOTIFIER
        select ARCH_THREAD_STACK_ALLOCATOR
@@ -29,6 +48,12 @@ config OUTPUT_FORMAT
 config ARCH_DMA_ADDR_T_64BIT
        def_bool 64BIT
 
+config X86_64
+       def_bool y if "$(OUTPUT_FORMAT)" = "elf64-x86-64"
+
+config X86_32
+       def_bool y if "$(OUTPUT_FORMAT)" = "elf32-i386"
+
 config 64BIT
        def_bool n
 
@@ -39,7 +64,7 @@ config BIG_ENDIAN
        def_bool n
 
 config GENERIC_CSUM
-       def_bool y
+       def_bool LKL
 
 config GENERIC_HWEIGHT
        def_bool y
@@ -54,3 +79,5 @@ config RWSEM_GENERIC_SPINLOCK
 config HZ
         int
         default 100
+
+endmenu
arch/um/lkl/Makefile new file +67 -0

67 lines not on this page · raw message

arch/um/lkl/auto.conf new file +1 -0
new file mode 100644
index 000000000000..4bfd65a02d73
@@ -0,0 +1 @@
+export OUTPUT_FORMAT=$(shell $(LD) -r -print-output-format)
arch/um/lkl/kernel/Makefile new file +4 -0
new file mode 100644
index 000000000000..ef489f2f7176
@@ -0,0 +1,4 @@
+extra-y := vmlinux.lds
+
+obj-y = setup.o threads.o irq.o time.o syscalls.o misc.o console.o \
+	syscalls_32.o cpu.o
arch/um/lkl/mm/Makefile new file +1 -0
new file mode 100644
index 000000000000..2af6e3051897
@@ -0,0 +1 @@
+obj-y = bootmem.o
Signature · 1 line
2.21.0 (Apple Git-122.2)
#23 lore raw

RFC v3 13/26 lkl tools: skeleton for host side library

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

This patch adds the skeleton for the host library.

The host library is implementing the host operations needed by LKL and is split into host dependent (depends on a specific host, e.g. POSIX hosts) and host independent parts (will work on all supported hosts).

Cc Ben Wolsieffer <[email protected]>
Cc Conrad Meyer <[email protected]>
Cc H.K. Jerry Chu <[email protected]>
Cc Luca Dariz <[email protected]>
Cc Mark Stillwell <[email protected]>
Cc Michael Zimmermann <[email protected]>
Cc Motomu Utsumi <[email protected]>
Cc Patrick Collins <[email protected]>
Cc Petros Angelatos <[email protected]>
Cc Thomas Liebetraut <[email protected]>
Cc Xiao Jia <[email protected]>
Cc Yuan Liu <[email protected]>
---
 tools/lkl/.gitignore         |   4 +
 tools/lkl/Build              |   0
 tools/lkl/Makefile           | 125 ++++++++++++
 tools/lkl/Makefile.autoconf  |  65 +++++++
 tools/lkl/Targets            |   3 +
 tools/lkl/include/.gitignore |   1 +
 tools/lkl/include/lkl.h      | 358 +++++++++++++++++++++++++++++++++++
 tools/lkl/include/lkl_host.h |  19 ++
 tools/lkl/lib/.gitignore     |   3 +
 tools/lkl/lib/Build          |   1 +
 10 files changed, 579 insertions(+)
 create mode 100644 tools/lkl/.gitignore
 create mode 100644 tools/lkl/Build
 create mode 100644 tools/lkl/Makefile
 create mode 100644 tools/lkl/Makefile.autoconf
 create mode 100644 tools/lkl/Targets
 create mode 100644 tools/lkl/include/.gitignore
 create mode 100644 tools/lkl/include/lkl.h
 create mode 100644 tools/lkl/include/lkl_host.h
 create mode 100644 tools/lkl/lib/.gitignore
 create mode 100644 tools/lkl/lib/Build
tools/lkl/.gitignore new file +4 -0
new file mode 100644
index 000000000000..1aed58bfe171
@@ -0,0 +1,4 @@
+Makefile.conf
+include/lkl_autoconf.h
+tests/autoconf.sh
+bin/stat
tools/lkl/Build new file +0 -0
new file mode 100644
index 000000000000..e69de29bb2d1
tools/lkl/Makefile new file +125 -0

125 lines not on this page · raw message

tools/lkl/Makefile.autoconf new file +65 -0

65 lines not on this page · raw message

tools/lkl/Targets new file +3 -0
new file mode 100644
index 000000000000..24c985e64638
@@ -0,0 +1,3 @@
+libs-y += lib/liblkl
+
+
tools/lkl/include/.gitignore new file +1 -0
new file mode 100644
index 000000000000..c41a463c898d
@@ -0,0 +1 @@
+lkl/
\ No newline at end of file
tools/lkl/include/lkl.h new file +358 -0

358 lines not on this page · raw message

tools/lkl/include/lkl_host.h new file +19 -0
new file mode 100644
index 000000000000..b5f96096fe69
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_HOST_H
+#define _LKL_HOST_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <lkl/asm/host_ops.h>
+#include <lkl.h>
+
+extern struct lkl_host_operations lkl_host_ops;
+
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
tools/lkl/lib/.gitignore new file +3 -0
new file mode 100644
index 000000000000..427ae0273fdd
@@ -0,0 +1,3 @@
+lkl.o
+liblkl.a
+
tools/lkl/lib/Build new file +1 -0
new file mode 100644
index 000000000000..8b137891791f
@@ -0,0 +1 @@
+
Signature · 1 line
2.21.0 (Apple Git-122.2)
#24 lore raw

RFC v3 14/26 lkl tools: host lib: add utilities functions

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

Add basic utility functions for getting a string from a kernel error code and a fprintf like function that uses the host print operation. The latter is useful for informing the user about errors that occur in the host library.

Other configuration and debug utilities are also added.

Cc Conrad Meyer <[email protected]>
Cc Patrick Collins <[email protected]>
Cc Yuan Liu <[email protected]>
Cc Motomu Utsumi <[email protected]>
---
 tools/lkl/include/lkl_config.h |  61 +++
 tools/lkl/include/lkl_host.h   |   7 +
 tools/lkl/lib/Build            |   7 +
 tools/lkl/lib/config.c         | 744 +++++++++++++++++++++++++++++++++
 tools/lkl/lib/dbg.c            | 300 +++++++++++++
 tools/lkl/lib/dbg_handler.c    |  44 ++
 tools/lkl/lib/endian.h         |  31 ++
 tools/lkl/lib/jmp_buf.c        |  14 +
 tools/lkl/lib/jmp_buf.h        |   8 +
 tools/lkl/lib/utils.c          | 266 ++++++++++++
 10 files changed, 1482 insertions(+)
 create mode 100644 tools/lkl/include/lkl_config.h
 create mode 100644 tools/lkl/lib/config.c
 create mode 100644 tools/lkl/lib/dbg.c
 create mode 100644 tools/lkl/lib/dbg_handler.c
 create mode 100644 tools/lkl/lib/endian.h
 create mode 100644 tools/lkl/lib/jmp_buf.c
 create mode 100644 tools/lkl/lib/jmp_buf.h
 create mode 100644 tools/lkl/lib/utils.c
tools/lkl/include/lkl_config.h new file +61 -0
new file mode 100644
index 000000000000..d3edf8b414cf
@@ -0,0 +1,61 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_LIB_CONFIG_H
+#define _LKL_LIB_CONFIG_H
+
+#define LKL_CONFIG_JSON_TOKEN_MAX 300
+
+struct lkl_config_iface {
+	struct lkl_config_iface *next;
+	struct lkl_netdev *nd;
+
+	/* OBSOLETE: should use IFTYPE and IFPARAMS */
+	char *iftap;
+	char *iftype;
+	char *ifparams;
+	char *ifmtu_str;
+	char *ifip;
+	char *ifipv6;
+	char *ifgateway;
+	char *ifgateway6;
+	char *ifmac_str;
+	char *ifnetmask_len;
+	char *ifnetmask6_len;
+	char *ifoffload_str;
+	char *ifneigh_entries;
+	char *ifqdisc_entries;
+};
+
+struct lkl_config {
+	int ifnum;
+	struct lkl_config_iface *ifaces;
+
+	char *gateway;
+	char *gateway6;
+	char *debug;
+	char *mount;
+	/* single_cpu mode:
+	 * 0: Don't pin to single CPU (default).
+	 * 1: Pin only LKL kernel threads to single CPU.
+	 * 2: Pin all LKL threads to single CPU including all LKL kernel threads
+	 * and device polling threads. Avoid this mode if having busy polling
+	 * threads.
+	 *
+	 * mode 2 can achieve better TCP_RR but worse TCP_STREAM than mode 1.
+	 * You should choose the best for your application and virtio device
+	 * type.
+	 */
+	char *single_cpu;
+	char *sysctls;
+	char *boot_cmdline;
+	char *dump;
+	char *delay_main;
+};
+
+int lkl_load_config_json(struct lkl_config *cfg, char *jstr);
+int lkl_load_config_env(struct lkl_config *cfg);
+void lkl_show_config(struct lkl_config *cfg);
+int lkl_load_config_pre(struct lkl_config *cfg);
+int lkl_load_config_post(struct lkl_config *cfg);
+int lkl_unload_config(struct lkl_config *cfg);
+
+#endif /* _LKL_LIB_CONFIG_H */
tools/lkl/include/lkl_host.h +7 -0
index b5f96096fe69..85e80eb4ad0d 100644
@@ -11,6 +11,13 @@ extern "C" {
 
 extern struct lkl_host_operations lkl_host_ops;
 
+/**
+ * lkl_printf - print a message via the host print operation
+ *
+ * @fmt: printf like format string
+ */
+int lkl_printf(const char *fmt, ...);
+
 
 #ifdef __cplusplus
 }
tools/lkl/lib/Build +7 -0
index 8b137891791f..658bfa865b9c 100644
@@ -1 +1,8 @@
+CFLAGS_config.o += -I$(srctree)/tools/perf/pmu-events
 
+liblkl-y += jmp_buf.o
+liblkl-y += utils.o
+liblkl-y += dbg.o
+liblkl-y += dbg_handler.o
+liblkl-y += ../../perf/pmu-events/jsmn.o
+liblkl-y += config.o
tools/lkl/lib/config.c new file +744 -0

744 lines not on this page · raw message

tools/lkl/lib/dbg.c new file +300 -0

300 lines not on this page · raw message

tools/lkl/lib/dbg_handler.c new file +44 -0

44 lines not on this page · raw message

tools/lkl/lib/endian.h new file +31 -0
new file mode 100644
index 000000000000..aaccfa0edb65
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_LIB_ENDIAN_H
+#define _LKL_LIB_ENDIAN_H
+
+#if defined(__FreeBSD__)
+#include <sys/endian.h>
+#elif defined(__ANDROID__)
+#include <sys/endian.h>
+#elif defined(__MINGW32__)
+#include <winsock.h>
+#define le32toh(x) (x)
+#define le16toh(x) (x)
+#define htole32(x) (x)
+#define htole16(x) (x)
+#define le64toh(x) (x)
+#define htobe32(x) htonl(x)
+#define htobe16(x) htons(x)
+#define be32toh(x) ntohl(x)
+#define be16toh(x) ntohs(x)
+#else
+#include <endian.h>
+#endif
+
+#ifndef htonl
+#define htonl(x) htobe32(x)
+#define htons(x) htobe16(x)
+#define ntohl(x) be32toh(x)
+#define ntohs(x) be16toh(x)
+#endif
+
+#endif /* _LKL_LIB_ENDIAN_H */
tools/lkl/lib/jmp_buf.c new file +14 -0
new file mode 100644
index 000000000000..f6bdd7e4bd83
@@ -0,0 +1,14 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <setjmp.h>
+#include <lkl_host.h>
+
+void jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void))
+{
+	if (!setjmp(*((jmp_buf *)jmpb->buf)))
+		f();
+}
+
+void jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val)
+{
+	longjmp(*((jmp_buf *)jmpb->buf), val);
+}
tools/lkl/lib/jmp_buf.h new file +8 -0
new file mode 100644
index 000000000000..8782cbaaf51f
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_LIB_JMP_BUF_H
+#define _LKL_LIB_JMP_BUF_H
+
+void jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void));
+void jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val);
+
+#endif
tools/lkl/lib/utils.c new file +266 -0

266 lines not on this page · raw message

Signature · 1 line
2.21.0 (Apple Git-122.2)
#25 lore raw

RFC v3 15/26 lkl tools: host lib: filesystem helpers

In reply to Hajime Tazaki

Add LKL applications APIs to mount and unmount a filesystem from a disk added via lkl_disk_add().

Also add open/close/read directory wrappers on top of lkl_sys_getdents64.

Cc Conrad Meyer <[email protected]>
Cc Michael Zimmermann <[email protected]>
Cc Yuan Liu <[email protected]>
---
 tools/lkl/include/lkl.h | 165 ++++++++++++++
 tools/lkl/lib/Build     |   1 +
 tools/lkl/lib/fs.c      | 471 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 637 insertions(+)
 create mode 100644 tools/lkl/lib/fs.c
tools/lkl/include/lkl.h +165 -0
index 4b95d0ef8e5b..cdae92300320 100644
@@ -350,6 +350,171 @@ const char *lkl_strerror(int err);
  */
 void lkl_perror(char *msg, int err);
 
+/**
+ * struct lkl_dev_blk_ops - block device host operations, defined in lkl_host.h.
+ */
+struct lkl_dev_blk_ops;
+
+/**
+ * lkl_disk - host disk handle
+ *
+ * @dev - a pointer to private information for this disk backend
+ * @fd - a POSIX file descriptor that can be used by preadv/pwritev
+ * @handle - an NT file handle that can be used by ReadFile/WriteFile
+ */
+struct lkl_disk {
+	void *dev;
+	union {
+		int fd;
+		void *handle;
+	};
+	struct lkl_dev_blk_ops *ops;
+};
+
+/**
+ * lkl_disk_add - add a new disk
+ *
+ * @disk - the host disk handle
+ * @returns a disk id (0 is valid) or a strictly negative value in case of error
+ */
+int lkl_disk_add(struct lkl_disk *disk);
+
+/**
+ * lkl_disk_remove - remove a disk
+ *
+ * This function makes a cleanup of the @disk's private information
+ * that was initialized by lkl_disk_add before.
+ *
+ * @disk - the host disk handle
+ */
+int lkl_disk_remove(struct lkl_disk disk);
+
+/**
+ * lkl_encode_dev_from_sysfs_blkdev - extract device id from sysfs
+ *
+ * This function returns the device id for the given sysfs dev node.
+ * The content of the node has to be in the form 'MAJOR:MINOR'.
+ * Also, this function expects an absolute path which means that sysfs
+ * already has to be mounted at the given path
+ *
+ * @sysfs_path - absolute path to the sysfs dev node
+ * @pdevid - pointer to memory where dev id will be returned
+ * @returns - 0 on success, a negative value on error
+ */
+int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid);
+
+/**
+ * lkl_mount_dev - mount a disk
+ *
+ * This functions creates a device file for the given disk, creates a mount
+ * point and mounts the device over the mount point.
+ *
+ * @disk_id - the disk id identifying the disk to be mounted
+ * @part - disk partition or zero for full disk
+ * @fs_type - filesystem type
+ * @flags - mount flags
+ * @opts - additional filesystem specific mount options
+ * @mnt_str - a string that will be filled by this function with the path where
+ * the filesystem has been mounted
+ * @mnt_str_len - size of mnt_str
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_mount_dev(unsigned int disk_id, unsigned int part, const char *fs_type,
+		   int flags, const char *opts,
+		   char *mnt_str, unsigned int mnt_str_len);
+
+/**
+ * lkl_umount_dev - umount a disk
+ *
+ * This functions umounts the given disks and removes the device file and the
+ * mount point.
+ *
+ * @disk_id - the disk id identifying the disk to be mounted
+ * @part - disk partition or zero for full disk
+ * @flags - umount flags
+ * @timeout_ms - timeout to wait for the kernel to flush closed files so that
+ * umount can succeed
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
+		    long timeout_ms);
+
+/**
+ * lkl_umount_timeout - umount filesystem with timeout
+ *
+ * @path - the path to unmount
+ * @flags - umount flags
+ * @timeout_ms - timeout to wait for the kernel to flush closed files so that
+ * umount can succeed
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_umount_timeout(char *path, int flags, long timeout_ms);
+
+/**
+ * lkl_opendir - open a directory
+ *
+ * @path - directory path
+ * @err - pointer to store the error in case of failure
+ * @returns - a handle to be used when calling lkl_readdir
+ */
+struct lkl_dir *lkl_opendir(const char *path, int *err);
+
+/**
+ * lkl_fdopendir - open a directory
+ *
+ * @fd - file descriptor
+ * @err - pointer to store the error in case of failure
+ * @returns - a handle to be used when calling lkl_readdir
+ */
+struct lkl_dir *lkl_fdopendir(int fd, int *err);
+
+/**
+ * lkl_rewinddir - reset directory stream
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ */
+void lkl_rewinddir(struct lkl_dir *dir);
+
+/**
+ * lkl_closedir - close the directory
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ */
+int lkl_closedir(struct lkl_dir *dir);
+
+/**
+ * lkl_readdir - get the next available entry of the directory
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ * @returns - a lkl_dirent64 entry or NULL if the end of the directory stream is
+ * reached or if an error occurred; check lkl_errdir() to distinguish between
+ * errors or end of the directory stream
+ */
+struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir);
+
+/**
+ * lkl_errdir - checks if an error occurred during the last lkl_readdir call
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ * @returns - 0 if no error occurred, or a negative value otherwise
+ */
+int lkl_errdir(struct lkl_dir *dir);
+
+/**
+ * lkl_dirfd - gets the file descriptor associated with the directory handle
+ *
+ * @dir - the directory handle as returned by lkl_opendir
+ * @returns - a positive value,which is the LKL file descriptor associated with
+ * the directory handle, or a negative value otherwise
+ */
+int lkl_dirfd(struct lkl_dir *dir);
+
+/**
+ * lkl_mount_fs - mount a file system type like proc, sys
+ * @fstype - file system type. e.g. proc, sys
+ * @returns - 0 on success. 1 if it's already mounted. negative on failure.
+ */
+int lkl_mount_fs(char *fstype);
 
 #ifdef __cplusplus
 }
tools/lkl/lib/Build +1 -0
index 658bfa865b9c..4a444337b0e7 100644
@@ -1,5 +1,6 @@
 CFLAGS_config.o += -I$(srctree)/tools/perf/pmu-events
 
+liblkl-y += fs.o
 liblkl-y += jmp_buf.o
 liblkl-y += utils.o
 liblkl-y += dbg.o
tools/lkl/lib/fs.c new file +471 -0

471 lines not on this page · raw message

Signature · 1 line
2.21.0 (Apple Git-122.2)
#26 lore raw

RFC v3 16/26 lkl tools: host lib: networking helpers

In reply to Hajime Tazaki

This commit adds LKL application APIs to control network related resources of kernel via LKL syscall, ioctl, and via netlink messages.

---
 tools/lkl/include/lkl.h      | 241 +++++++++++
 tools/lkl/include/lkl_host.h |  85 ++++
 tools/lkl/lib/Build          |   1 +
 tools/lkl/lib/net.c          | 818 +++++++++++++++++++++++++++++++++++
 4 files changed, 1145 insertions(+)
 create mode 100644 tools/lkl/lib/net.c
tools/lkl/include/lkl.h +241 -0
index cdae92300320..1f4291ad9455 100644
@@ -516,6 +516,247 @@ int lkl_dirfd(struct lkl_dir *dir);
  */
 int lkl_mount_fs(char *fstype);
 
+/**
+ * lkl_if_up - activate network interface
+ *
+ * @ifindex - the ifindex of the interface
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_up(int ifindex);
+
+/**
+ * lkl_if_down - deactivate network interface
+ *
+ * @ifindex - the ifindex of the interface
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_down(int ifindex);
+
+/**
+ * lkl_if_set_mtu - set MTU on interface
+ *
+ * @ifindex - the ifindex of the interface
+ * @mtu - the requested MTU size
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_set_mtu(int ifindex, int mtu);
+
+/**
+ * lkl_if_set_ipv4 - set IPv4 address on interface
+ *
+ * @ifindex - the ifindex of the interface
+ * @addr - 4-byte IP address (i.e., struct in_addr)
+ * @netmask_len - prefix length of the @addr
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_set_ipv4(int ifindex, unsigned int addr, unsigned int netmask_len);
+
+/**
+ * lkl_set_ipv4_gateway - add an IPv4 default route
+ *
+ * @addr - 4-byte IP address of the gateway (i.e., struct in_addr)
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_set_ipv4_gateway(unsigned int addr);
+
+/**
+ * lkl_if_set_ipv4_gateway - add an IPv4 default route in rule table
+ *
+ * @ifindex - the ifindex of the interface, used for tableid calculation
+ * @addr - 4-byte IP address of the interface
+ * @netmask_len - prefix length of the @addr
+ * @gw_addr - 4-byte IP address of the gateway
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_set_ipv4_gateway(int ifindex, unsigned int addr,
+		unsigned int netmask_len, unsigned int gw_addr);
+
+/**
+ * lkl_if_set_ipv6 - set IPv6 address on interface
+ * must be called after interface is up.
+ *
+ * @ifindex - the ifindex of the interface
+ * @addr - 16-byte IPv6 address (i.e., struct in6_addr)
+ * @netprefix_len - prefix length of the @addr
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_set_ipv6(int ifindex, void *addr, unsigned int netprefix_len);
+
+/**
+ * lkl_set_ipv6_gateway - add an IPv6 default route
+ *
+ * @addr - 16-byte IPv6 address of the gateway (i.e., struct in6_addr)
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_set_ipv6_gateway(void *addr);
+
+/**
+ * lkl_if_set_ipv6_gateway - add an IPv6 default route in rule table
+ *
+ * @ifindex - the ifindex of the interface, used for tableid calculation
+ * @addr - 16-byte IP address of the interface
+ * @netmask_len - prefix length of the @addr
+ * @gw_addr - 16-byte IP address of the gateway (i.e., struct in_addr)
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_set_ipv6_gateway(int ifindex, void *addr,
+		unsigned int netmask_len, void *gw_addr);
+
+/**
+ * lkl_ifname_to_ifindex - obtain ifindex of an interface by name
+ *
+ * @name - string of an interface
+ * @returns - return an integer of ifindex if no error
+ */
+int lkl_ifname_to_ifindex(const char *name);
+
+/**
+ * lkl_netdev_get_ifindex - retrieve the interface index for a given network
+ * device id
+ *
+ * @id - the network device id
+ * @returns the interface index or a stricly negative value in case of error
+ */
+int lkl_netdev_get_ifindex(int id);
+
+/**
+ * lkl_add_neighbor - add a permanent arp entry
+ * @ifindex - the ifindex of the interface
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @ip - ip address of the entry in network byte order
+ * @mac - mac address of the entry
+ */
+int lkl_add_neighbor(int ifindex, int af, void *addr, void *mac);
+
+/**
+ * lkl_if_add_ip - add an ip address
+ * @ifindex - the ifindex of the interface
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @addr - ip address of the entry in network byte order
+ * @netprefix_len - prefix length of the @addr
+ */
+int lkl_if_add_ip(int ifindex, int af, void *addr, unsigned int netprefix_len);
+
+/**
+ * lkl_if_del_ip - add an ip address
+ * @ifindex - the ifindex of the interface
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @addr - ip address of the entry in network byte order
+ * @netprefix_len - prefix length of the @addr
+ */
+int lkl_if_del_ip(int ifindex, int af, void *addr, unsigned int netprefix_len);
+
+/**
+ * lkl_add_gateway - add a gateway
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @gwaddr - 4-byte IP address of the gateway (i.e., struct in_addr)
+ */
+int lkl_add_gateway(int af, void *gwaddr);
+
+/**
+ * XXX Should I use OIF selector?
+ * temporary table idx = ifindex * 2 + 0 <- ipv4
+ * temporary table idx = ifindex * 2 + 1 <- ipv6
+ */
+/**
+ * lkl_if_add_rule_from_addr - create an ip rule table with "from" selector
+ * @ifindex - the ifindex of the interface, used for table id calculation
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @saddr - network byte order ip address, "from" selector address of this rule
+ */
+int lkl_if_add_rule_from_saddr(int ifindex, int af, void *saddr);
+
+/**
+ * lkl_if_add_gateway - add gateway to rule table
+ * @ifindex - the ifindex of the interface, used for table id calculation
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @gwaddr - 4-byte IP address of the gateway (i.e., struct in_addr)
+ */
+int lkl_if_add_gateway(int ifindex, int af, void *gwaddr);
+
+/**
+ * lkl_if_add_linklocal - add linklocal route to rule table
+ * @ifindex - the ifindex of the interface, used for table id calculation
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @addr - ip address of the entry in network byte order
+ * @netprefix_len - prefix length of the @addr
+ */
+int lkl_if_add_linklocal(int ifindex, int af,  void *addr, int netprefix_len);
+
+/**
+ * lkl_if_wait_ipv6_dad - wait for DAD to be done for a ipv6 address
+ * must be called after interface is up
+ *
+ * @ifindex - the ifindex of the interface
+ * @addr - ip address of the entry in network byte order
+ */
+int lkl_if_wait_ipv6_dad(int ifindex, void *addr);
+
+/**
+ * lkl_set_fd_limit - set the maximum number of file descriptors allowed
+ * @fd_limit - fd max limit
+ */
+int lkl_set_fd_limit(unsigned int fd_limit);
+
+/**
+ * lkl_qdisc_add - set qdisc rule onto an interface
+ *
+ * @ifindex - the ifindex of the interface
+ * @root - the name of root class (e.g., "root");
+ * @type - the type of qdisc (e.g., "fq")
+ */
+int lkl_qdisc_add(int ifindex, const char *root, const char *type);
+
+/**
+ * lkl_qdisc_parse_add - Add a qdisc entry for an interface with strings
+ *
+ * @ifindex - the ifindex of the interface
+ * @entries - strings of qdisc configurations in the form of
+ *            "root|type;root|type;..."
+ */
+void lkl_qdisc_parse_add(int ifindex, const char *entries);
+
+/**
+ * lkl_netdev - host network device handle, defined in lkl_host.h.
+ */
+struct lkl_netdev;
+
+/**
+ * lkl_netdev_args - arguments to lkl_netdev_add
+ * @mac - optional MAC address for the device
+ * @offload - offload bits for the device
+ */
+struct lkl_netdev_args {
+	void *mac;
+	unsigned int offload;
+};
+
+/*
+ * lkl_register_dbg_handler- register a signal handler that loads a debug lib.
+ *
+ * The signal handler is triggered by Ctrl-Z. It creates a new pthread which
+ * call dbg_entrance().
+ *
+ * If you run the program from shell script, make sure you ignore SIGTSTP by
+ * "trap '' TSTP" in the shell script.
+ */
+void lkl_register_dbg_handler(void);
+
+/**
+ * lkl_sysctl - write a sysctl value
+ *
+ * @path - the path to an sysctl entry (e.g., "net.ipv4.tcp_wmem");
+ * @value - the value of the sysctl (e.g., "4096 87380 2147483647")
+ */
+int lkl_sysctl(const char *path, const char *value);
+
+/**
+ * lkl_sysctl_parse_write - Configure sysctl parameters with strings
+ *
+ * @sysctls - Configure sysctl parameters as the form of "key=value;..."
+ */
+void lkl_sysctl_parse_write(const char *sysctls);
+
 #ifdef __cplusplus
 }
 #endif
tools/lkl/include/lkl_host.h +85 -0

91 lines not on this page · raw message

tools/lkl/lib/Build +1 -0
index 4a444337b0e7..dba530424e4a 100644
@@ -1,6 +1,7 @@
 CFLAGS_config.o += -I$(srctree)/tools/perf/pmu-events
 
 liblkl-y += fs.o
+liblkl-y += net.o
 liblkl-y += jmp_buf.o
 liblkl-y += utils.o
 liblkl-y += dbg.o
tools/lkl/lib/net.c new file +818 -0

818 lines not on this page · raw message

Signature · 1 line
2.21.0 (Apple Git-122.2)
#27 lore raw

RFC v3 17/26 lkl tools: host lib: posix host operations

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

Implement LKL host operations for POSIX hosts.

Cc Conrad Meyer <[email protected]>
Cc Mark Stillwell <[email protected]>
Cc Patrick Collins <[email protected]>
Cc Pierre-Hugues Husson <[email protected]>
Cc Thomas Liebetraut <[email protected]>
Cc Yuan Liu <[email protected]>
---
 tools/lkl/lib/Build        |   2 +
 tools/lkl/lib/posix-host.c | 353 +++++++++++++++++++++++++++++++++++++
 2 files changed, 355 insertions(+)
 create mode 100644 tools/lkl/lib/posix-host.c
tools/lkl/lib/Build +2 -0
index dba530424e4a..0e711e260a3a 100644
@@ -1,8 +1,10 @@
 CFLAGS_config.o += -I$(srctree)/tools/perf/pmu-events
+CFLAGS_posix-host.o += -D_FILE_OFFSET_BITS=64
 
 liblkl-y += fs.o
 liblkl-y += net.o
 liblkl-y += jmp_buf.o
+liblkl-$(LKL_HOST_CONFIG_POSIX) += posix-host.o
 liblkl-y += utils.o
 liblkl-y += dbg.o
 liblkl-y += dbg_handler.o
tools/lkl/lib/posix-host.c new file +353 -0

353 lines not on this page · raw message

Signature · 1 line
2.21.0 (Apple Git-122.2)
#28 lore raw

RFC v3 18/26 lkl tools: add test programs

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

Add a simple LKL test application (boot) that starts the kernel and performs simple tests that minimally exercise the LKL API. Networking and block device tests are also added which are useful to detect regressions.

Cc Conrad Meyer <[email protected]>
Cc David Disseldorp <[email protected]>
Cc H.K. Jerry Chu <[email protected]>
Cc Lai Jiangshan <[email protected]>
Cc Luca Dariz <[email protected]>
Cc Mark Stillwell <[email protected]>
Cc Motomu Utsumi <[email protected]>
Cc Patrick Collins <[email protected]>
Cc Petros Angelatos <[email protected]>
Cc Thomas Liebetraut <[email protected]>
Cc Yuan Liu <[email protected]>
---
 tools/lkl/.gitignore              |   5 +
 tools/lkl/Makefile                |   5 +-
 tools/lkl/Targets                 |   3 +
 tools/lkl/tests/Build             |   3 +
 tools/lkl/tests/boot.c            | 562 ++++++++++++++++++++++++++++++
 tools/lkl/tests/boot.sh           |   9 +
 tools/lkl/tests/cla.c             | 159 +++++++++
 tools/lkl/tests/cla.h             |  33 ++
 tools/lkl/tests/disk.c            | 189 ++++++++++
 tools/lkl/tests/disk.sh           |  61 ++++
 tools/lkl/tests/net-setup.sh      |  73 ++++
 tools/lkl/tests/net-test.c        | 287 +++++++++++++++
 tools/lkl/tests/net.sh            |  90 +++++
 tools/lkl/tests/run.py            | 179 ++++++++++
 tools/lkl/tests/tap13.py          | 209 +++++++++++
 tools/lkl/tests/test.c            | 126 +++++++
 tools/lkl/tests/test.h            |  72 ++++
 tools/lkl/tests/test.sh           | 182 ++++++++++
 tools/lkl/tests/valgrind.supp     | 105 ++++++
 tools/lkl/tests/valgrind2xunit.py |  69 ++++
 20 files changed, 2420 insertions(+), 1 deletion(-)
 create mode 100644 tools/lkl/tests/Build
 create mode 100644 tools/lkl/tests/boot.c
 create mode 100755 tools/lkl/tests/boot.sh
 create mode 100644 tools/lkl/tests/cla.c
 create mode 100644 tools/lkl/tests/cla.h
 create mode 100644 tools/lkl/tests/disk.c
 create mode 100755 tools/lkl/tests/disk.sh
 create mode 100644 tools/lkl/tests/net-setup.sh
 create mode 100644 tools/lkl/tests/net-test.c
 create mode 100755 tools/lkl/tests/net.sh
 create mode 100755 tools/lkl/tests/run.py
 create mode 100644 tools/lkl/tests/tap13.py
 create mode 100644 tools/lkl/tests/test.c
 create mode 100644 tools/lkl/tests/test.h
 create mode 100644 tools/lkl/tests/test.sh
 create mode 100644 tools/lkl/tests/valgrind.supp
 create mode 100755 tools/lkl/tests/valgrind2xunit.py
tools/lkl/.gitignore +5 -0
index 1aed58bfe171..4e08254dbd46 100644
@@ -2,3 +2,8 @@ Makefile.conf
 include/lkl_autoconf.h
 tests/autoconf.sh
 bin/stat
+tests/net-test
+tests/disk
+tests/boot
+tests/valgrind*.xml
+*.pyc
tools/lkl/Makefile +4 -1
index 11ea7e9095bb..0abccb9d86b6 100644
@@ -117,8 +117,11 @@ programs_install: $(progs-y:%=$(OUTPUT)%$(EXESUF))
 install: headers_install libraries_install programs_install
 
 
+run-tests:
+	./tests/run.py $(tests)
+
 FORCE: ;
-.PHONY: all clean FORCE
+.PHONY: all clean FORCE run-tests
 .PHONY: headers_install libraries_install programs_install install
 .NOTPARALLEL : lib/lkl.o
 .SECONDARY:
tools/lkl/Targets +3 -0
index 24c985e64638..a9f74c3cc8fb 100644
@@ -1,3 +1,6 @@
 libs-y += lib/liblkl
 
+progs-y += tests/boot
+progs-y += tests/disk
+progs-y += tests/net-test
 
tools/lkl/tests/Build new file +3 -0
new file mode 100644
index 000000000000..ace86a3d3438
@@ -0,0 +1,3 @@
+boot-y += boot.o test.o
+disk-y += disk.o cla.o test.o
+net-test-y += net-test.o cla.o test.o
tools/lkl/tests/boot.c new file +562 -0

562 lines not on this page · raw message

tools/lkl/tests/boot.sh new file +9 -0
new file mode 100755
index 000000000000..d985c04b0ac1
@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+source $script_dir/test.sh
+
+lkl_test_plan 1 "boot"
+lkl_test_run 1
+lkl_test_exec $script_dir/boot
tools/lkl/tests/cla.c new file +159 -0

159 lines not on this page · raw message

tools/lkl/tests/cla.h new file +33 -0
new file mode 100644
index 000000000000..f8369be02e5a
@@ -0,0 +1,33 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_TEST_CLA_H
+#define _LKL_TEST_CLA_H
+
+enum cl_arg_type {
+	CL_ARG_USER = 0,
+	CL_ARG_BOOL,
+	CL_ARG_INT,
+	CL_ARG_STR,
+	CL_ARG_STR_SET,
+	CL_ARG_IPV4,
+	CL_ARG_END,
+};
+
+struct cl_arg;
+
+typedef int (*cl_arg_parser_t)(struct cl_arg *arg, const char *value);
+
+struct cl_arg {
+	const char *long_name;
+	char short_name;
+	const char *help;
+	int has_arg;
+	enum cl_arg_type type;
+	void *store;
+	void *set;
+	cl_arg_parser_t parser;
+};
+
+int parse_args(int argc, const char **argv, struct cl_arg *args);
+
+
+#endif /* _LKL_TEST_CLA_H */
tools/lkl/tests/disk.c new file +189 -0

189 lines not on this page · raw message

tools/lkl/tests/disk.sh new file +61 -0

61 lines not on this page · raw message

tools/lkl/tests/net-setup.sh new file +73 -0

73 lines not on this page · raw message

tools/lkl/tests/net-test.c new file +287 -0

287 lines not on this page · raw message

tools/lkl/tests/net.sh new file +90 -0

90 lines not on this page · raw message

tools/lkl/tests/run.py new file +179 -0

179 lines not on this page · raw message

tools/lkl/tests/tap13.py new file +209 -0

209 lines not on this page · raw message

tools/lkl/tests/test.c new file +126 -0

126 lines not on this page · raw message

tools/lkl/tests/test.h new file +72 -0

72 lines not on this page · raw message

tools/lkl/tests/test.sh new file +182 -0

182 lines not on this page · raw message

tools/lkl/tests/valgrind.supp new file +105 -0

105 lines not on this page · raw message

tools/lkl/tests/valgrind2xunit.py new file +69 -0

69 lines not on this page · raw message

Signature · 1 line
2.21.0 (Apple Git-122.2)
#29 lore raw

RFC v3 19/26 lkl tools: cptofs that reads/writes to/from a filesystem image

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

A tool to read/write to/from a filesystem image without mounting the file to host filesystem. Thus there is no root privilege to modify the contents.

Cc Conrad Meyer <[email protected]>
Cc Dan Peebles <[email protected]>
Cc Petros Angelatos <[email protected]>
Cc Tuomas Tynkkynen <[email protected]>
Cc Yuriy Taraday <[email protected]>
---
 tools/lkl/.gitignore |   2 +
 tools/lkl/Build      |   2 +
 tools/lkl/Makefile   |   3 +
 tools/lkl/Targets    |   4 +
 tools/lkl/cptofs.c   | 636 +++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 647 insertions(+)
 create mode 100644 tools/lkl/cptofs.c
tools/lkl/.gitignore +2 -0
index 4e08254dbd46..1a8210f4d9c4 100644
@@ -7,3 +7,5 @@ tests/disk
 tests/boot
 tests/valgrind*.xml
 *.pyc
+cptofs
+cpfromfs
tools/lkl/Build +2 -0
index e69de29bb2d1..a9d12c5ca260 100644
@@ -0,0 +1,2 @@
+cptofs-$(LKL_HOST_CONFIG_ARCHIVE) += cptofs.o
+
tools/lkl/Makefile +3 -0
index 0abccb9d86b6..03750aab98d8 100644
@@ -86,6 +86,9 @@ $(OUTPUT)%$(EXESUF): $(OUTPUT)%-in.o $(OUTPUT)liblkl.a
 $(OUTPUT)%-in.o: $(OUTPUT)lib/lkl.o FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
+$(OUTPUT)cpfromfs$(EXESUF): $(OUTPUT)cptofs$(EXESUF)
+	$(Q)if ! [ -e $@ ]; then ln -s $< $@; fi
+
 
 clean:
 	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
tools/lkl/Targets +4 -0
index a9f74c3cc8fb..17de965f92b3 100644
@@ -4,3 +4,7 @@ progs-y += tests/boot
 progs-y += tests/disk
 progs-y += tests/net-test
 
+progs-$(LKL_HOST_CONFIG_ARCHIVE) += cptofs
+progs-$(LKL_HOST_CONFIG_ARCHIVE) += cpfromfs
+LDLIBS_cptofs-y := -larchive
+LDLIBS_cptofs-$(LKL_HOST_CONFIG_NEEDS_LARGP) += -largp
tools/lkl/cptofs.c new file +636 -0

636 lines not on this page · raw message

Signature · 1 line
2.21.0 (Apple Git-122.2)
#30 lore raw

RFC v3 20/26 lkl tools: fs2tar that converts a filesystem image to tar

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

Simple utility that converts a filesystem image to a tar file, preserving file rights and extended attributes.

---
 tools/lkl/.gitignore |   1 +
 tools/lkl/Build      |   1 +
 tools/lkl/Targets    |   4 +
 tools/lkl/fs2tar.c   | 412 +++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 418 insertions(+)
 create mode 100644 tools/lkl/fs2tar.c
tools/lkl/.gitignore +1 -0
index 1a8210f4d9c4..138e65efcad2 100644
@@ -9,3 +9,4 @@ tests/valgrind*.xml
 *.pyc
 cptofs
 cpfromfs
+fs2tar
tools/lkl/Build +1 -0
index a9d12c5ca260..73b37363a6de 100644
@@ -1,2 +1,3 @@
 cptofs-$(LKL_HOST_CONFIG_ARCHIVE) += cptofs.o
+fs2tar-$(LKL_HOST_CONFIG_ARCHIVE) += fs2tar.o
 
tools/lkl/Targets +4 -0
index 17de965f92b3..3451a1856955 100644
@@ -8,3 +8,7 @@ progs-$(LKL_HOST_CONFIG_ARCHIVE) += cptofs
 progs-$(LKL_HOST_CONFIG_ARCHIVE) += cpfromfs
 LDLIBS_cptofs-y := -larchive
 LDLIBS_cptofs-$(LKL_HOST_CONFIG_NEEDS_LARGP) += -largp
+
+progs-$(LKL_HOST_CONFIG_ARCHIVE) += fs2tar
+LDLIBS_fs2tar-y := -larchive
+LDLIBS_fs2tar-$(LKL_HOST_CONFIG_NEEDS_LARGP) += -largp
tools/lkl/fs2tar.c new file +412 -0

412 lines not on this page · raw message

Signature · 1 line
2.21.0 (Apple Git-122.2)
#31 lore raw

RFC v3 21/26 lkl tools: add lklfuse

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

Add a simple fuse based program that can mount filesystem in userspace using LKL.

Cc Conrad Meyer <[email protected]>
Cc Quentin Anglade <[email protected]>
Cc Rafael Gieschke <[email protected]>
---
 tools/lkl/.gitignore       |   1 +
 tools/lkl/Build            |   3 +
 tools/lkl/Targets          |   3 +
 tools/lkl/lklfuse.c        | 659 +++++++++++++++++++++++++++++++++++++
 tools/lkl/tests/lklfuse.sh | 110 +++++++
 5 files changed, 776 insertions(+)
 create mode 100644 tools/lkl/lklfuse.c
 create mode 100755 tools/lkl/tests/lklfuse.sh
tools/lkl/.gitignore +1 -0
index 138e65efcad2..c78ec268e4b0 100644
@@ -10,3 +10,4 @@ tests/valgrind*.xml
 cptofs
 cpfromfs
 fs2tar
+lklfuse
tools/lkl/Build +3 -0
index 73b37363a6de..6048440d0e1b 100644
@@ -1,3 +1,6 @@
+CFLAGS_lklfuse.o += -D_FILE_OFFSET_BITS=64
+
 cptofs-$(LKL_HOST_CONFIG_ARCHIVE) += cptofs.o
 fs2tar-$(LKL_HOST_CONFIG_ARCHIVE) += fs2tar.o
+lklfuse-$(LKL_HOST_CONFIG_FUSE) += lklfuse.o
 
tools/lkl/Targets +3 -0
index 3451a1856955..03d24362a195 100644
@@ -12,3 +12,6 @@ LDLIBS_cptofs-$(LKL_HOST_CONFIG_NEEDS_LARGP) += -largp
 progs-$(LKL_HOST_CONFIG_ARCHIVE) += fs2tar
 LDLIBS_fs2tar-y := -larchive
 LDLIBS_fs2tar-$(LKL_HOST_CONFIG_NEEDS_LARGP) += -largp
+
+progs-$(LKL_HOST_CONFIG_FUSE) += lklfuse
+LDLIBS_lklfuse-y := -lfuse
tools/lkl/lklfuse.c new file +659 -0

659 lines not on this page · raw message

tools/lkl/tests/lklfuse.sh new file +110 -0

110 lines not on this page · raw message

Signature · 1 line
2.21.0 (Apple Git-122.2)
#32 lore raw

RFC v3 22/26 um lkl: add documentation

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

A document describing brief introduction of LKL, why it is needed, and how it is used is added. The document is located under uml/ directory.

Cc Chenyang Zhong <[email protected]>
Cc Conrad Meyer <[email protected]>
Cc Gustavo Bittencourt <[email protected]>
Cc H.K. Jerry Chu <[email protected]>
Cc Motomu Utsumi <[email protected]>
Cc Patrick Collins <[email protected]>
Cc Thomas Liebetraut <[email protected]>
Cc Yuan Liu <[email protected]>
---
 Documentation/virt/uml/lkl.txt | 64 ++++++++++++++++++++++++++++++++++
 1 file changed, 64 insertions(+)
 create mode 100644 Documentation/virt/uml/lkl.txt
Documentation/virt/uml/lkl.txt new file +64 -0
new file mode 100644
index 000000000000..f6d581092331
@@ -0,0 +1,64 @@
+
+Introduction
+============
+
+LKL (Linux Kernel Library) is aiming to allow reusing the Linux kernel code as
+extensively as possible with minimal effort and reduced maintenance overhead.
+
+Examples of how LKL can be used are: creating userspace applications (running on
+Linux and other operating systems) that can read or write Linux filesystems or
+can use the Linux networking stack, creating kernel drivers for other operating
+systems that can read Linux filesystems, bootloaders support for reading/writing
+Linux filesystems, etc.
+
+With LKL, the kernel code is compiled into an object file that can be directly
+linked by applications. The API offered by LKL is based on the Linux system call
+interface.
+
+LKL is implemented as one of the mode of UML (arch/um). It uses host operations
+defined by the application or a host library (tools/lkl/lib).
+
+
+Supported hosts
+===============
+
+The supported host for now is Linux (x86 architecture) userspace applications.
+
+
+Building LKL the host library and LKL applications
+==================================================
+
+    $ make -C tools/lkl
+
+will build LKL as a object file, it will install it in tools/lkl/lib together
+with the headers files in tools/lkl/include then will build the host library,
+tests and a few of application examples:
+
+* tests/boot - a simple applications that uses LKL and exercises the basic LKL
+  APIs
+
+* tests/net-test - a simple applications that uses network feature of LKL and
+  exercises the basic network-related APIs
+
+* fs2tar - a tool that converts a filesystem image to a tar archive
+
+* cptofs/cpfromfs - a tool that copies files to/from a filesystem image
+
+* lklfuse - a tool that can mount a filesystem image in userspace,
+  without root privileges, using FUSE
+
+
+Building LKL on Ubuntu
+-----------------------
+
+    $ sudo apt-get install libfuse-dev libarchive-dev xfsprogs
+
+    # Optional, if you would like to be able to run tests
+    $ sudo apt-get install btrfs-tools
+    $ pip install yamlish junit_xml
+
+    $ make -C tools/lkl
+
+    # To check that everything works:
+    $ cd tools/lkl
+    $ make run-tests
Signature · 1 line
2.21.0 (Apple Git-122.2)
#33 lore raw

RFC v3 23/26 um lkl: add CI scripts to conduct regression tests

In reply to Hajime Tazaki

For the continuous integration (CI), we use CircleCI with multiple test targets triggered with a single commit. The test also conducts memory checks using valgrind in order to prevent enbug.

---
 .circleci/config.yml             | 196 +++++++++++++++++++++++++++++++
 tools/lkl/scripts/checkpatch.sh  |  60 ++++++++++
 tools/lkl/scripts/lkl-jenkins.sh |  21 ++++
 3 files changed, 277 insertions(+)
 create mode 100644 .circleci/config.yml
 create mode 100755 tools/lkl/scripts/checkpatch.sh
 create mode 100755 tools/lkl/scripts/lkl-jenkins.sh
.circleci/config.yml new file +196 -0
new file mode 100644
index 000000000000..5bdf059014c0
@@ -0,0 +1,196 @@
+version: 2
+general:
+  artifacts:
+
+do_steps: &do_steps
+ steps:
+  - run: echo "$CROSS_COMPILE" > ~/_cross_compile
+  - restore_cache:
+      key: code-tree-shallow-{{ .Environment.CACHE_VERSION }}
+  - run:
+      name: checkout build tree
+      command: |
+        mkdir -p ~/.ssh/
+        ssh-keyscan -H github.com >> ~/.ssh/known_hosts
+        if ! [ -d .git ]; then
+          git clone --depth=1 $CIRCLE_REPOSITORY_URL .;
+        fi
+        if [[ $CIRCLE_BRANCH == pull/* ]]; then
+           git fetch --depth=1 origin $CIRCLE_BRANCH/head;
+        else
+           git fetch --depth=1 origin $CIRCLE_BRANCH;
+        fi
+        git reset --hard $CIRCLE_SHA1
+  - save_cache:
+      key: code-tree-shallow-{{ .Environment.CACHE_VERSION }}-{{ epoch }}
+      paths:
+        - /home/ubuntu/project/.git
+  - run:
+      name: clean
+      command: |
+        make mrproper
+        cd tools/lkl && make clean-conf
+        rm -rf ~/junit
+  - run: mkdir -p /home/ubuntu/.ccache
+  - restore_cache:
+      key: compiler-cache-{{ checksum "~/_cross_compile" }}-{{ .Environment.CACHE_VERSION }}
+  - run: cd tools/lkl && make -j8 ${MKARG}
+  - run: mkdir -p ~/destdir && cd tools/lkl && make DESTDIR=~/destdir
+  - save_cache:
+     paths:
+       - /home/ubuntu/.ccache
+     key: compiler-cache-{{ checksum "~/_cross_compile" }}-{{ .Environment.CACHE_VERSION }}-{{ epoch }}
+  - run:
+      name: run tests
+      command: |
+        mkdir -p ~/junit
+        make -C tools/lkl run-tests tests="--junit-dir ~/junit"
+      no_output_timeout: "90m"
+  - run:
+      name: collecting test results
+      command: |
+        find ./tools/lkl/ -type f -name "*.xml" -exec mv {} ~/junit/ \;
+  - store_test_results:
+      path: ~/junit
+  - store_artifacts:
+      path: ~/junit
+
+
+do_uml_steps: &do_uml_steps
+ steps:
+  - run: echo "$CROSS_COMPILE" > ~/_cross_compile
+  - restore_cache:
+      key: code-tree-shallow-{{ .Environment.CACHE_VERSION }}
+  - run:
+      name: checkout build tree
+      command: |
+        mkdir -p ~/.ssh/
+        ssh-keyscan -H github.com >> ~/.ssh/known_hosts
+        if ! [ -d .git ]; then
+          git clone --depth=1 $CIRCLE_REPOSITORY_URL .;
+        fi
+        if [[ $CIRCLE_BRANCH == pull/* ]]; then
+           git fetch --depth=1 origin $CIRCLE_BRANCH/head;
+        else
+           git fetch --depth=1 origin $CIRCLE_BRANCH;
+        fi
+        git reset --hard $CIRCLE_SHA1
+  - save_cache:
+      key: code-tree-shallow-{{ .Environment.CACHE_VERSION }}-{{ epoch }}
+      paths:
+        - /home/ubuntu/project/.git
+  - run: mkdir -p /home/ubuntu/.ccache
+  - restore_cache:
+      key: compiler-cache-{{ checksum "~/_cross_compile" }}-{{ .Environment.CACHE_VERSION }}
+  - run:
+      name: build
+      command: |
+        sudo apt-get update
+        sudo apt-get install -y gcc-multilib g++-multilib
+        make defconfig ARCH=um SUBARCH=$SUBARCH
+        make ARCH=um SUBARCH=$SUBARCH
+  - save_cache:
+     paths:
+       - /home/ubuntu/.ccache
+     key: compiler-cache-{{ checksum "~/_cross_compile" }}-{{ .Environment.CACHE_VERSION }}-{{ epoch }}
+  - run:
+      name: test
+      command: |
+        # XXX: i386 build doesn't work with the test
+        if [ $CIRCLE_STAGE = "i386_uml" ] || [ $CIRCLE_STAGE = "i386_uml_on_x86_64" ]; then
+          exit 0
+        fi
+        ./linux rootfstype=hostfs ro mem=1g loglevel=10 init="/bin/bash -c exit" || export RETVAL=$?
+        # SIGABRT=6 => 128+6
+        if [ $RETVAL != "134" ]; then
+          exit 1
+        fi
+
+## Customize the test machine
+jobs:
+  x86_64:
+   docker:
+     - image: lkldocker/circleci-x86_64:0.7
+   environment:
+     CROSS_COMPILE: ""
+   <<: *do_steps
+
+  i386:
+   docker:
+     - image: lkldocker/circleci-i386:0.1
+   environment:
+     CROSS_COMPILE: ""
+   <<: *do_steps
+
+  x86_64_valgrind:
+   docker:
+     - image: lkldocker/circleci-x86_64:0.7
+   environment:
+     CROSS_COMPILE: ""
+     VALGRIND: 1
+   <<: *do_steps
+
+  x86_64_uml:
+   docker:
+     - image: lkldocker/circleci-x86_64:0.7
+   environment:
+     CROSS_COMPILE: ""
+     TMPDIR: "/tmp" # required for not using /dev/shm
+     SUBARCH: "x86_64"
+   <<: *do_uml_steps
+
+  i386_uml:
+   docker:
+     - image: lkldocker/circleci-i386:0.1
+   environment:
+     CROSS_COMPILE: ""
+     SUBARCH: "i386"
+     TMPDIR: "/tmp" # required for not using /dev/shm
+   <<: *do_uml_steps
+
+  i386_uml_on_x86_64:
+   docker:
+     - image: lkldocker/circleci-x86_64:0.7
+   environment:
+     CROSS_COMPILE: ""
+     TMPDIR: "/tmp" # required for not using /dev/shm
+     SUBARCH: "i386"
+   <<: *do_uml_steps
+
+  checkpatch:
+   docker:
+     - image: lkldocker/circleci:0.5
+   environment:
+   steps:
+     - restore_cache:
+        key: code-tree-full-history-{{ .Environment.CACHE_VERSION }}
+     - checkout
+     - run: sudo pip install ply
+     - run: tools/lkl/scripts/checkpatch.sh
+     - save_cache:
+        key: code-tree-full-history-{{ .Environment.CACHE_VERSION }}-{{ epoch }}
+        paths:
+          - /home/ubuntu/project/.git
+        when: always
+
+workflows:
+  version: 2
+  build:
+    jobs:
+     - x86_64
+     - x86_64_valgrind
+     - checkpatch
+     - i386
+     - x86_64_uml
+     - i386_uml
+     - i386_uml_on_x86_64
+  nightly:
+    triggers:
+      - schedule:
+          cron: "0 0 * * *"
+          filters:
+            branches:
+              only:
+                - master
+    jobs:
+      - x86_64_valgrind
tools/lkl/scripts/checkpatch.sh new file +60 -0

60 lines not on this page · raw message

tools/lkl/scripts/lkl-jenkins.sh new file +21 -0
new file mode 100755
index 000000000000..eaadc6e90143
@@ -0,0 +1,21 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+set -e
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+basedir=$(cd $script_dir/../../..; pwd)
+
+export PATH=$PATH:/sbin
+
+build_and_test()
+{
+    cd $basedir
+    make mrproper
+    cd tools/lkl
+    make clean-conf
+    make -j4
+    make run-tests
+}
+
+build_and_test
Signature · 1 line
2.21.0 (Apple Git-122.2)
#34 lore raw

RFC v3 24/26 um lkl: add UML network driver for lkl

In reply to Hajime Tazaki

This commit adds the use of the UML drivers of network interfaces which should be able to use any backend configured by boot command line parameters. Currently slirp and tuntap backends are tested.

Since the UML drivers use LKL irq infrastructure, the build system only picks required object files of UML, and LKL adds trivial glue code and ifdefs into UML code so that existing driver codes can be used for LKL as-is.

---
 .circleci/config.yml                    |  2 ++
 arch/um/drivers/Makefile                |  2 ++
 arch/um/kernel/Makefile                 |  4 +++
 arch/um/kernel/irq.c                    |  4 +++
 arch/um/lkl/Kconfig                     |  2 ++
 arch/um/lkl/Makefile                    |  2 ++
 arch/um/lkl/include/uapi/asm/host_ops.h |  6 +++++
 arch/um/lkl/kernel/asm-offsets.c        |  1 +
 arch/um/lkl/kernel/irq.c                | 16 ++++++++++++
 arch/um/lkl/kernel/setup.c              |  4 +++
 arch/um/lkl/mm/bootmem.c                | 33 +++++++++++++++++++++++++
 arch/um/os-Linux/Makefile               |  5 ++++
 arch/um/os-Linux/file.c                 |  9 +++++++
 tools/lkl/Makefile.autoconf             |  2 ++
 tools/lkl/include/lkl.h                 |  9 +++++++
 tools/lkl/include/lkl_host.h            |  1 +
 tools/lkl/lib/Build                     |  1 +
 tools/lkl/lib/config.c                  |  6 +++++
 tools/lkl/lib/posix-host.c              |  3 +++
 tools/lkl/lib/um/Build                  |  2 ++
 tools/lkl/lib/um/um_glue.c              | 33 +++++++++++++++++++++++++
 tools/lkl/lib/um/um_net.c               | 25 +++++++++++++++++++
 tools/lkl/tests/net-setup.sh            |  1 +
 tools/lkl/tests/net-test.c              | 14 +++++++----
 tools/lkl/tests/net.sh                  | 18 ++++++++++++++
 tools/lkl/tests/run.py                  |  1 +
 26 files changed, 201 insertions(+), 5 deletions(-)
 create mode 100644 tools/lkl/lib/um/Build
 create mode 100644 tools/lkl/lib/um/um_glue.c
 create mode 100644 tools/lkl/lib/um/um_net.c
.circleci/config.yml +2 -0
index 5bdf059014c0..df1554078a4b 100644
@@ -43,6 +43,8 @@ do_steps: &do_steps
   - run:
       name: run tests
       command: |
+        sudo apt-get update
+        sudo apt-get install -y slirp
         mkdir -p ~/junit
         make -C tools/lkl run-tests tests="--junit-dir ~/junit"
       no_output_timeout: "90m"
arch/um/drivers/Makefile +2 -0
index a290821e355c..3cce6c81079e 100644
@@ -37,7 +37,9 @@ $(obj)/vde.o: $(obj)/vde_kern.o $(obj)/vde_user.o
 # When the above is fixed, don't forget to add this too!
 #targets += $(obj)/pcap.o
 
+ifeq ($(UMMODE),kernel)
 obj-y := stdio_console.o fd.o chan_kern.o chan_user.o line.o
+endif
 obj-$(CONFIG_SSL) += ssl.o
 obj-$(CONFIG_STDERR_CONSOLE) += stderr_console.o
 
arch/um/kernel/Makefile +4 -0
index 5aa882011e04..78b49eb9bf81 100644
@@ -12,12 +12,16 @@ CPPFLAGS_vmlinux.lds := -DSTART=$(LDS_START)		\
                         -DELF_ARCH=$(LDS_ELF_ARCH)	\
                         -DELF_FORMAT=$(LDS_ELF_FORMAT)	\
 			$(LDS_EXTRA)
+ifeq ($(UMMODE),kernel)
 extra-y := vmlinux.lds
 
 obj-y = config.o exec.o exitcode.o irq.o ksyms.o mem.o \
 	physmem.o process.o ptrace.o reboot.o sigio.o \
 	signal.o syscall.o sysrq.o time.o tlb.o trap.o \
 	um_arch.o umid.o maccess.o kmsg_dump.o skas/
+else ifeq ($(UMMODE),library)
+obj-y = irq.o
+endif
 
 obj-$(CONFIG_BLK_DEV_INITRD) += initrd.o
 obj-$(CONFIG_GPROF)	+= gprof_syms.o
arch/um/kernel/irq.c +4 -0
index 3577118bb4a5..580a889c2277 100644
@@ -406,6 +406,7 @@ int deactivate_all_fds(void)
 	return 0;
 }
 
+#ifdef CONFIG_UMMODE_KERN
 /*
  * do_IRQ handles all normal device IRQs (the special
  * SMP cross-CPU interrupts have their own specific
@@ -420,6 +421,7 @@ unsigned int do_IRQ(int irq, struct uml_pt_regs *regs)
 	set_irq_regs(old_regs);
 	return 1;
 }
+#endif /* !CONFIG_UMMODE_KERN */
 
 void um_free_irq(unsigned int irq, void *dev)
 {
@@ -446,6 +448,7 @@ int um_request_irq(unsigned int irq, int fd, int type,
 
 EXPORT_SYMBOL(um_request_irq);
 
+#ifdef CONFIG_UMMODE_KERN
 /*
  * irq_chip must define at least enable/disable and ack when
  * the edge handler is used.
@@ -597,3 +600,4 @@ unsigned long from_irq_stack(int nested)
 	return mask & ~1;
 }
 
+#endif /* !CONFIG_UMMODE_KERN */
arch/um/lkl/Kconfig +2 -0
index f72b423fad5b..3568f33d5ccf 100644
@@ -81,3 +81,5 @@ config HZ
         default 100
 
 endmenu
+
+source "arch/um/drivers/Kconfig"
arch/um/lkl/Makefile +2 -0
index e1161fa3fb63..17ff6d00d1db 100644
@@ -4,6 +4,8 @@ include $(LKL_DIR)/auto.conf
 
 # fixup CFLAGS of um build
 KBUILD_CFLAGS := $(subst $(CFLAGS),,$(KBUILD_CFLAGS))
+# XXX
+KBUILD_CFLAGS	+= -DTIMER_IRQ=0 -DUM_ETH_IRQ=5 -DLAST_IRQ=15
 
 SRCARCH := um/lkl
 ARCH_INCLUDE += -I$(srctree)/$(LKL_DIR)/um/include
arch/um/lkl/include/uapi/asm/host_ops.h +6 -0
index fe4382c3050a..0230885f4f64 100644
@@ -17,6 +17,10 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @um_devices - string containg the list of UML devices in command line
+ * format. This string is appended to the kernel command line and
+ * is provided here for convenience to be implemented by the host library.
+ *
  * @print - optional operation that receives console messages
  *
  * @panic - called during a kernel panic
@@ -74,6 +78,8 @@ struct lkl_jmp_buf {
  * @jmp_buf_longjmp - perform a jump back to the saved jump buffer
  */
 struct lkl_host_operations {
+	const char *um_devices;
+
 	void (*print)(const char *str, int len);
 	void (*panic)(void);
 
arch/um/lkl/kernel/asm-offsets.c +1 -0
index 6be0763698dc..6fdca6df21a2 100644
@@ -1,2 +1,3 @@
 // SPDX-License-Identifier: GPL-2.0
 /* Dummy asm-offsets.c file. Required by kbuild and ready to be used - hint! */
+#include <sysdep/kernel-offsets.h>
arch/um/lkl/kernel/irq.c +16 -0
index e3b59e46ca50..7de65b8f699d 100644
@@ -11,6 +11,10 @@
 #include <asm/host_ops.h>
 #include <asm/cpu.h>
 
+#if defined(__linux) && (defined(__i386) || defined(__x86_64))
+#include <os.h>
+#endif
+
 /*
  * To avoid much overhead we use an indirect approach: the irqs are marked using
  * a bitmap (array of longs) and a summary of the modified bits is kept in a
@@ -184,6 +188,10 @@ void init_IRQ(void)
 	for (i = 0; i < NR_IRQS; i++)
 		irq_set_chip_and_handler(i, &dummy_irq_chip, handle_simple_irq);
 
+#if defined(__linux) && (defined(__i386) || defined(__x86_64))
+	/* Initialize EPOLL Loop */
+	os_setup_epoll();
+#endif
 	pr_info("lkl: irqs initialized\n");
 }
 
@@ -191,3 +199,11 @@ void cpu_yield_to_irqs(void)
 {
 	cpu_relax();
 }
+
+#if defined(__linux) && (defined(__i386) || defined(__x86_64))
+unsigned int do_IRQ(int irq, struct uml_pt_regs *regs)
+{
+	lkl_trigger_irq(irq);
+	return 1;
+}
+#endif
arch/um/lkl/kernel/setup.c +4 -0
index 36c199d3aa22..39b7b7c79581 100644
@@ -61,6 +61,10 @@ int __init lkl_start_kernel(struct lkl_host_operations *ops, const char *fmt,
 	ret = vsnprintf(boot_command_line, COMMAND_LINE_SIZE, fmt, ap);
 	va_end(ap);
 
+	if (ops->um_devices)
+		strscpy(boot_command_line + ret, ops->um_devices,
+			COMMAND_LINE_SIZE - ret);
+
 	memcpy(cmd_line, boot_command_line, COMMAND_LINE_SIZE);
 
 	init_sem = lkl_ops->sem_alloc(0);
arch/um/lkl/mm/bootmem.c +33 -0
index 39dd0d22b44e..3175dadee13f 100644
@@ -64,3 +64,36 @@ void free_mem(void)
 {
 	lkl_ops->mem_free((void *)_memory_start);
 }
+
+void *uml_kmalloc(int size, int flags)
+{
+	return kmalloc(size, flags);
+}
+
+char *uml_strdup(const char *string)
+{
+	return kstrdup(string, GFP_KERNEL);
+}
+
+void free_stack(unsigned long stack, int order)
+{
+	free_pages(stack, order);
+}
+
+unsigned long alloc_stack(int order, int atomic)
+{
+	unsigned long page;
+	gfp_t flags = GFP_KERNEL;
+
+	if (atomic)
+		flags = GFP_ATOMIC;
+	page = __get_free_pages(flags, order);
+
+	return page;
+}
+
+int __cant_sleep(void)
+{
+	return in_atomic() || irqs_disabled() || in_interrupt();
+	/* Is in_interrupt() really needed? */
+}
arch/um/os-Linux/Makefile +5 -0
index 839915b8c31c..a74a2486e178 100644
@@ -6,9 +6,14 @@
 # Don't instrument UML-specific code
 KCOV_INSTRUMENT                := n
 
+ifeq ($(UMMODE),kernel)
 obj-y = execvp.o file.o helper.o irq.o main.o mem.o process.o \
 	registers.o sigio.o signal.o start_up.o time.o tty.o \
 	umid.o user_syms.o util.o drivers/ skas/
+else
+obj-y = execvp.o file.o helper.o irq.o drivers/
+endif
+
 
 obj-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
 
arch/um/os-Linux/file.c +9 -0
index 5133e3afb96f..6656a2ec418a 100644
@@ -397,6 +397,14 @@ int os_pipe(int *fds, int stream, int close_on_exec)
 	return err;
 }
 
+static void sig_handler(int sig)
+{
+	if (sig != SIGIO)
+		return;
+
+	sigio_handler(sig, NULL, NULL);
+}
+
 int os_set_fd_async(int fd)
 {
 	int err, flags;
@@ -413,6 +421,7 @@ int os_set_fd_async(int fd)
 		return err;
 	}
 
+	signal(SIGIO, sig_handler);
 	if ((fcntl(fd, F_SETSIG, SIGIO) < 0) ||
 	    (fcntl(fd, F_SETOWN, os_getpid()) < 0)) {
 		err = -errno;
tools/lkl/Makefile.autoconf +2 -0
index 268c367d9962..d2dcaa3b85f2 100644
@@ -41,6 +41,8 @@ define posix_host
   $(if $(strip $(call find_include,archive.h)),$(call set_autoconf_var,ARCHIVE,y))
   $(if $(filter $(1),elf64-x86-64-freebsd),$(call set_autoconf_var,NEEDS_LARGP,y))
   $(if $(filter $(1),elf32-i386),$(call set_autoconf_var,I386,y))
+  $(if $(filter $(1),elf64-x86-64),$(call set_autoconf_var,UML_DEV,y))
+  $(if $(filter $(1),elf32-i386),$(call set_autoconf_var,UML_DEV,y))
 endef
 
 define do_autoconf
tools/lkl/include/lkl.h +9 -0
index 1f4291ad9455..952d11e30868 100644
@@ -731,6 +731,15 @@ struct lkl_netdev_args {
 	unsigned int offload;
 };
 
+#ifdef LKL_HOST_CONFIG_UML_DEV
+struct lkl_netdev *lkl_um_netdev_create(const char *ifparams);
+#else
+static inline struct lkl_netdev *lkl_um_netdev_create(const char *ifparams)
+{
+	return NULL;
+}
+#endif
+
 /*
  * lkl_register_dbg_handler- register a signal handler that loads a debug lib.
  *
tools/lkl/include/lkl_host.h +1 -0
index 4e6d6e031498..7ac30bdcaf0a 100644
@@ -10,6 +10,7 @@ extern "C" {
 #include <lkl.h>
 
 extern struct lkl_host_operations lkl_host_ops;
+extern char lkl_um_devs[4096];
 
 /**
  * lkl_printf - print a message via the host print operation
tools/lkl/lib/Build +1 -0
index 0e711e260a3a..ad8a4b806375 100644
@@ -10,3 +10,4 @@ liblkl-y += dbg.o
 liblkl-y += dbg_handler.o
 liblkl-y += ../../perf/pmu-events/jsmn.o
 liblkl-y += config.o
+liblkl-$(LKL_HOST_CONFIG_UML_DEV) += um/
tools/lkl/lib/config.c +6 -0
index 37f8ac4d942a..db3af68446d3 100644
@@ -470,6 +470,12 @@ static int lkl_config_netdev_create(struct lkl_config *cfg,
 	memset(&nd_args, 0, sizeof(struct lkl_netdev_args));
 
 	if (!nd && iface->iftype && iface->ifparams) {
+		if ((strcmp(iface->iftype, "um") == 0)) {
+			nd = lkl_um_netdev_create(iface->ifparams);
+			iface->nd = nd;
+			/* um_netdev doesn't use virtio device */
+			return 0;
+		}
 	}
 
 	if (nd) {
tools/lkl/lib/posix-host.c +3 -0
index 4be1611a8942..d6394d41e125 100644
@@ -346,6 +346,9 @@ struct lkl_host_operations lkl_host_ops = {
 	.print = print,
 	.mem_alloc = (void *)malloc,
 	.mem_free = free,
+#ifdef LKL_HOST_CONFIG_UML_DEV
+	.um_devices = lkl_um_devs,
+#endif
 	.gettid = _gettid,
 	.jmp_buf_set = jmp_buf_set,
 	.jmp_buf_longjmp = jmp_buf_longjmp,
tools/lkl/lib/um/Build new file +2 -0
new file mode 100644
index 000000000000..09a60975ecd6
@@ -0,0 +1,2 @@
+liblkl-y += um_glue.o
+liblkl-y += um_net.o
tools/lkl/lib/um/um_glue.c new file +33 -0
new file mode 100644
index 000000000000..10ca8524e682
@@ -0,0 +1,33 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+#include <errno.h>
+
+
+
+char lkl_um_devs[4096];
+
+/* from sigio.c */
+void maybe_sigio_broken(int fd, int read)
+{
+}
+
+/* from process.c */
+int os_getpid(void)
+{
+	return getpid();
+}
+
+
+/* from chan_kern.c */
+void free_irqs(void)
+{
+}
+
+/* from sigio.c */
+int ignore_sigio_fd(int fd)
+{
+	return 0;
+}
tools/lkl/lib/um/um_net.c new file +25 -0
new file mode 100644
index 000000000000..edd0c65fc08e
@@ -0,0 +1,25 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <lkl_host.h>
+
+static int registered_net_dev_idx;
+
+struct lkl_netdev *lkl_um_netdev_create(const char *ifparams)
+{
+	struct lkl_netdev *nd;
+
+	nd = lkl_host_ops.mem_alloc(sizeof(struct lkl_netdev));
+	if (!nd)
+		return NULL;
+
+	memset(nd, 0, sizeof(struct lkl_netdev));
+
+	nd->id = registered_net_dev_idx++;
+	/* concat strings */
+	snprintf(lkl_um_devs + strlen(lkl_um_devs), sizeof(lkl_um_devs),
+		 " eth%d=%s", nd->id, ifparams);
+
+	return nd;
+}
tools/lkl/tests/net-setup.sh +1 -0
index 0cfc42a30a54..fa390fb904b5 100644
@@ -8,6 +8,7 @@ TEST_IP6_NETMASK=64
 TEST_MAC0="aa:bb:cc:dd:ee:ff"
 TEST_MAC1="aa:bb:cc:dd:ee:aa"
 TEST_NETSERVER_PORT=11223
+TEST_UM_SLIRP_PARMS="slirp,,`which slirp`"
 
 # $1 - count
 # $2 - netcount
tools/lkl/tests/net-test.c +9 -5
index edc37150270b..c3f818742263 100644
@@ -22,9 +22,10 @@
 
 enum {
 	BACKEND_NONE,
+	BACKEND_UM,
 };
 
-const char *backends[] = { "loopback", NULL };
+const char *backends[] = { "loopback", "um", NULL };
 static struct {
 	int backend;
 	const char *ifname;
@@ -164,12 +165,17 @@ static int lkl_test_icmp(void)
 }
 
 static struct lkl_netdev *nd;
+static int nd_id;
 
 static int lkl_test_nd_create(void)
 {
 	switch (cla.backend) {
 	case BACKEND_NONE:
 		return TEST_SKIP;
+	case BACKEND_UM:
+		nd = lkl_um_netdev_create(cla.ifname);
+		nd_id = nd->id;
+		break;
 	}
 
 	if (!nd) {
@@ -180,11 +186,9 @@ static int lkl_test_nd_create(void)
 	return TEST_SUCCESS;
 }
 
-static int nd_id;
-
 static int lkl_test_nd_add(void)
 {
-	if (cla.backend == BACKEND_NONE)
+	if (cla.backend == BACKEND_NONE || cla.backend == BACKEND_UM)
 		return TEST_SKIP;
 
 	return TEST_SUCCESS;
@@ -192,7 +196,7 @@ static int lkl_test_nd_add(void)
 
 static int lkl_test_nd_remove(void)
 {
-	if (cla.backend == BACKEND_NONE)
+	if (cla.backend == BACKEND_NONE || cla.backend == BACKEND_UM)
 		return TEST_SKIP;
 
 	return TEST_SUCCESS;
tools/lkl/tests/net.sh +18 -0
index 32e8452b0406..089ebfb332da 100755
@@ -13,6 +13,8 @@ cleanup_backend()
     case "$1" in
     "loopback")
         ;;
+    "um")
+        ;;
     esac
 }
 
@@ -47,6 +49,16 @@ setup_backend()
     case "$1" in
     "loopback")
         ;;
+    "um")
+	# only intel arch is capable with um-net backent
+	if [ -z "$LKL_HOST_CONFIG_UML_DEV" ]; then
+            return $TEST_SKIP
+	fi
+	# slirp's helper process doesn't work with valgrind
+	if [ -n "$VALGRIND" ]; then
+            return $TEST_SKIP
+	fi
+        ;;
     *)
         echo "don't know how to setup backend $1"
         return $TEST_FAILED
@@ -60,6 +72,12 @@ run_tests()
     "loopback")
         lkl_test_exec $script_dir/net-test --dst 127.0.0.1
         ;;
+    "um")
+        lkl_test_exec $script_dir/net-test --backend um \
+                      --ifname $TEST_UM_SLIRP_PARMS \
+                      --ip 10.0.2.15 --netmask-len 8 \
+                      --dst 10.0.2.2
+        ;;
     esac
 }
 
tools/lkl/tests/run.py +1 -0
index b72299aaabee..23d5c5489e2c 100755
@@ -54,6 +54,7 @@ tests = [
     'disk.sh -t vfat',
     'disk.sh -t btrfs',
     'net.sh -b loopback',
+    'net.sh -b um',
     'lklfuse.sh -t ext4',
     'lklfuse.sh -t vfat',
     'lklfuse.sh -t btrfs',
Signature · 1 line
2.21.0 (Apple Git-122.2)
#35 lore raw

RFC v3 25/26 um lkl: add UML block device driver (ubd) for lkl

In reply to Hajime Tazaki

This commit adds a support to utilize exising UML drivers of block devices for LKL. The goal is similar to networking drivers, add minimum amount of code and use drivers code as-is.

This commit also adds a test code with several filesystems (ext4, vfat, btrfs at the moment) to exercise the basic operations of disk.

---
 arch/um/drivers/ubd_kern.c              |  6 +++++-
 arch/um/lkl/Kconfig                     |  3 +++
 arch/um/lkl/Makefile                    |  2 +-
 arch/um/lkl/include/uapi/asm/syscalls.h |  1 +
 arch/um/lkl/kernel/setup.c              |  5 +++++
 tools/lkl/include/lkl.h                 | 15 ++++++++++++++
 tools/lkl/lib/um/Build                  |  1 +
 tools/lkl/lib/um/um_block.c             | 17 ++++++++++++++++
 tools/lkl/lib/um/um_glue.c              | 27 ++++++++++++++++++++++++-
 tools/lkl/tests/disk.c                  | 14 +++++++++++++
 10 files changed, 88 insertions(+), 3 deletions(-)
 create mode 100644 tools/lkl/lib/um/um_block.c
arch/um/drivers/ubd_kern.c +5 -1
index 6627d7c30f37..be1dc01de963 100644
@@ -544,7 +544,7 @@ static irqreturn_t ubd_intr(int irq, void *dev)
 /* Only changed by ubd_init, which is an initcall. */
 static int io_pid = -1;
 
-static void kill_io_thread(void)
+void kill_io_thread(void)
 {
 	if(io_pid != -1)
 		os_kill_process(io_pid, 1);
@@ -818,6 +818,7 @@ static int ubd_open_dev(struct ubd *ubd_dev)
 	}
 	ubd_dev->fd = fd;
 
+#ifdef CONFIG_UMMODE_KERN
 	if(ubd_dev->cow.file != NULL){
 		blk_queue_max_hw_sectors(ubd_dev->queue, 8 * sizeof(long));
 
@@ -842,6 +843,7 @@ static int ubd_open_dev(struct ubd *ubd_dev)
 		if(err < 0) goto error;
 		ubd_dev->cow.fd = err;
 	}
+#endif	/* CONFIG_UMMODE_KERN */
 	if (ubd_dev->no_trim == 0) {
 		ubd_dev->queue->limits.discard_granularity = SECTOR_SIZE;
 		ubd_dev->queue->limits.discard_alignment = SECTOR_SIZE;
@@ -851,9 +853,11 @@ static int ubd_open_dev(struct ubd *ubd_dev)
 	}
 	blk_queue_flag_set(QUEUE_FLAG_NONROT, ubd_dev->queue);
 	return 0;
+#ifdef CONFIG_UMMODE_KERN
  error:
 	os_close_file(ubd_dev->fd);
 	return err;
+#endif
 }
 
 static void ubd_device_release(struct device *dev)
arch/um/lkl/Kconfig +3 -0
index 3568f33d5ccf..fdd7ef2653a3 100644
@@ -48,6 +48,9 @@ config OUTPUT_FORMAT
 config ARCH_DMA_ADDR_T_64BIT
        def_bool 64BIT
 
+config BLK_DEV_UBD
+       def_bool y if "$(OUTPUT_FORMAT)" = "elf32-i386" || "$(OUTPUT_FORMAT)" = "elf64-x86-64"
+
 config X86_64
        def_bool y if "$(OUTPUT_FORMAT)" = "elf64-x86-64"
 
arch/um/lkl/Makefile +1 -1
index 17ff6d00d1db..0ed34f87848b 100644
@@ -5,7 +5,7 @@ include $(LKL_DIR)/auto.conf
 # fixup CFLAGS of um build
 KBUILD_CFLAGS := $(subst $(CFLAGS),,$(KBUILD_CFLAGS))
 # XXX
-KBUILD_CFLAGS	+= -DTIMER_IRQ=0 -DUM_ETH_IRQ=5 -DLAST_IRQ=15
+KBUILD_CFLAGS	+= -DTIMER_IRQ=0 -DUBD_IRQ=4 -DUM_ETH_IRQ=5 -DLAST_IRQ=15
 
 SRCARCH := um/lkl
 ARCH_INCLUDE += -I$(srctree)/$(LKL_DIR)/um/include
arch/um/lkl/include/uapi/asm/syscalls.h +1 -0
index a81534ffccb7..1384c34137d6 100644
@@ -164,6 +164,7 @@ struct sockaddr {
 #include <linux/virtio_ring.h>
 #include <linux/pkt_sched.h>
 #include <linux/io_uring.h>
+#include <linux/major.h>
 
 struct user_msghdr {
 	void		__user *msg_name;
arch/um/lkl/kernel/setup.c +5 -0
index 39b7b7c79581..daf55c216db9 100644
@@ -126,6 +126,11 @@ long lkl_sys_halt(void)
 		LINUX_REBOOT_CMD_RESTART,
 	};
 
+#ifdef CONFIG_BLK_DEV_UBD
+void kill_io_thread(void);
+	kill_io_thread();
+#endif
+
 	err = lkl_syscall(__NR_reboot, params);
 	if (err < 0)
 		return err;
tools/lkl/include/lkl.h +15 -0
index 952d11e30868..7d164e4dcad1 100644
@@ -355,6 +355,10 @@ void lkl_perror(char *msg, int err);
  */
 struct lkl_dev_blk_ops;
 
+enum {
+	BLK_BACKEND_UM,
+};
+
 /**
  * lkl_disk - host disk handle
  *
@@ -368,6 +372,7 @@ struct lkl_disk {
 		int fd;
 		void *handle;
 	};
+	int backend;
 	struct lkl_dev_blk_ops *ops;
 };
 
@@ -403,6 +408,16 @@ int lkl_disk_remove(struct lkl_disk disk);
  */
 int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid);
 
+
+#ifdef LKL_HOST_CONFIG_UML_DEV
+int lkl_disk_um_add(struct lkl_disk *disk, const char *blkparams);
+#else
+static inline int lkl_disk_um_add(struct lkl_disk *disk, const char *blkparams)
+{
+	return -LKL_ENOSYS;
+}
+#endif
+
 /**
  * lkl_mount_dev - mount a disk
  *
tools/lkl/lib/um/Build +1 -0
index 09a60975ecd6..52573c05a797 100644
@@ -1,2 +1,3 @@
 liblkl-y += um_glue.o
 liblkl-y += um_net.o
+liblkl-y += um_block.o
tools/lkl/lib/um/um_block.c new file +17 -0
new file mode 100644
index 000000000000..70dc624ec43c
@@ -0,0 +1,17 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <lkl_host.h>
+
+
+static int registered_blk_dev_idx;
+
+int lkl_disk_um_add(struct lkl_disk *disk, const char *blkparams)
+{
+	/* concat strings */
+	snprintf(lkl_um_devs + strlen(lkl_um_devs), sizeof(lkl_um_devs),
+		 " ubd%d=%s", registered_blk_dev_idx, blkparams);
+
+	return registered_blk_dev_idx++;
+}
tools/lkl/lib/um/um_glue.c +26 -1
index 10ca8524e682..ed82ff8ced52 100644
@@ -2,9 +2,10 @@
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
+#include <signal.h>
 #include <unistd.h>
 #include <errno.h>
-
+#include <sys/wait.h>
 
 
 char lkl_um_devs[4096];
@@ -31,3 +32,27 @@ int ignore_sigio_fd(int fd)
 {
 	return 0;
 }
+
+/* from util.c */
+/*
+ * UML helper threads must not handle SIGWINCH/INT/TERM
+ */
+void os_fix_helper_signals(void)
+{
+	signal(SIGWINCH, SIG_IGN);
+	signal(SIGINT, SIG_DFL);
+	signal(SIGTERM, SIG_DFL);
+}
+
+
+
+void os_kill_process(int pid, int reap_child)
+{
+	kill(pid, SIGKILL);
+	if (reap_child) {
+		while ((errno = 0, ((waitpid(pid, NULL, __WALL)) < 0))
+		       && (errno == EINTR))
+			;
+	}
+
+}
tools/lkl/tests/disk.c +14 -0
index 0aa039876b54..f7801014d054 100644
@@ -23,12 +23,17 @@ static struct {
 	const char *disk;
 	const char *fstype;
 	int partition;
+	int backend;
 } cla;
 
+const char *backends[] = { "um", NULL };
+
 struct cl_arg args[] = {
 	{"disk", 'd', "disk file to use", 1, CL_ARG_STR, &cla.disk},
 	{"partition", 'P', "partition to mount", 1, CL_ARG_INT, &cla.partition},
 	{"type", 't', "filesystem type", 1, CL_ARG_STR, &cla.fstype},
+	{"backend", 'b', "blockd evice backend type", 1, CL_ARG_STR_SET,
+	 &cla.backend, backends},
 	{0},
 };
 
@@ -50,6 +55,15 @@ int lkl_test_disk_add(void)
 
 	disk.ops = NULL;
 
+	disk.backend = cla.backend;
+
+	switch (disk.backend) {
+	case BLK_BACKEND_UM:
+		disk.dev = (char *)cla.disk;
+	default:
+		break;
+	}
+
 	disk_id = lkl_disk_add(&disk);
 	if (disk_id < 0)
 		goto out_close;
Signature · 1 line
2.21.0 (Apple Git-122.2)
#36 lore raw

RFC v3 26/26 um: fix clone flags to be familar with valgrind

In reply to Hajime Tazaki

Valgrind reports the following error with UML when ubd driver is compiled in.

==29588== Unsupported clone() flags: 0x500 ==29588==
==29588== The only supported clone() uses are:
==29588== - via a threads library (LinuxThreads or NPTL) ==29588== - via the implementation of fork or vfork

Adding CLONE_FS flags silences this issue.

---
 arch/um/drivers/ubd_user.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
arch/um/drivers/ubd_user.c +2 -1
index a1afe414ce48..fabb50e91c37 100644
@@ -47,7 +47,8 @@ int start_io_thread(unsigned long sp, int *fd_out)
 		goto out_close;
 	}
 
-	pid = clone(io_thread, (void *) sp, CLONE_FILES | CLONE_VM, NULL);
+	pid = clone(io_thread, (void *) sp,
+		    CLONE_FILES | CLONE_VM | CLONE_FS, NULL);
 	if(pid < 0){
 		err = -errno;
 		printk("start_io_thread - clone failed : errno = %d\n", errno);
Signature · 1 line
2.21.0 (Apple Git-122.2)
#37 lore raw

v4 cover Unifying LKL into UML

In reply to Hajime Tazaki

This is another spin of the unification of LKL into UML. I've updated the patchset by addressing comments received so far.

Changes in rfc v4:
- Rebase on the current uml/master branch
- Fix IRQ handling (bug fix)
- drop a patch for CONFIG_GENERIC_ATOMIC64 support (comment by Peter Zijlstra)
- implement vector net driver for UMMODE_LIB (comment by Anton)
- clean up uapi headers to avoid duplicates
- clean up IRQ handling code (comment by Anton)
- fix error handling in test code (comments by David Disseldorp)

Changes in rfc v3:
- use UML drivers (net, block) from LKL programs
- drop virtio device implementations
- drop mingw32 (Windows) host
- drop android (arm/aarch64) host
- drop FreeBSD (x86_64) host
- drop LD_PRELOAD (hijack) support
- update milestone

rfc v2:
- use UMMODE instead of SUBARCH to switch UML or LKL
- tools/lkl directory is still there. I confirmed we can move under arch/um (e.g., arch/um/lkl/hosts). I will move it IF this is preferable.
- drop several patches involved non-uml directory
- drop several patches which are not required
- refine commit logs
- document updated

LKL (Linux Kernel Library) is aiming to allow reusing the Linux kernel code as extensively as possible with minimal effort and reduced maintenance overhead.

Examples of how LKL can be used are: creating userspace applications (running on Linux and other operating systems) that can read or write Linux filesystems or can use the Linux networking stack, creating kernel drivers for other operating systems that can read Linux filesystems, bootloaders support for reading/writing Linux filesystems, etc.

With LKL, the kernel code is compiled into an object file that can be directly linked by applications. The API offered by LKL is based on the Linux system call interface.

LKL is originally implemented as an architecture port in arch/lkl, but this series of commits tries to integrate this into arch/um as one of the mode of UML. This was discussed during RFC email of LKL (*1).

The latest LKL version can be found at https://github.com/lkl/linux

Milestone
=========
This patches is a first step toward upstreaming *library mode* of Linux kernel, but we think we need to have several steps toward our goal, describing in the below.

1. Put LKL code under arch/um (arch/um/lkl), and build it in a separate way from UML.
2. Use UML driver implementations in LKL as a minimum set of patches
- Only support x86 sub architecture (dependency to UML drivers)
3. Support broader host supports
- add virtio device features

For the step 1, we put LKL as one of UMMODE in order to make less effort to integrate (make ARCH=um UMMODE=library). The modification to existing UML code is trying to be minimized.

Building LKL the host library and LKL applications
==================================================

% cd tools/lkl
% make

will build LKL as a object file, it will install it in tools/lkl/lib together with the headers files in tools/lkl/include then will build the host library, tests and a few of application examples:

* tests/boot - a simple applications that uses LKL and exercises the basic LKL APIs

* tests/net-test - a simple applications that uses network feature of LKL and exercises the basic network-related APIs

* fs2tar - a tool that converts a filesystem image to a tar archive

* cptofs/cpfromfs - a tool that copies files to/from a filesystem image

* lklfuse - a tool that can mount a filesystem image in userspace, without root priviledges, using FUSE

% make run-tests

should run the above tests/boot and tests/net-test and report errors if there are any.

Supported hosts
===============

Currently LKL supports Linux userspace applications. New hosts can be added relatively easy if the host supports gcc and GNU ld. Previous versions of LKL supported Windows kernel and Haiku kernel hosts, and we also have WIP patches with rump-hypercall interface, used in UEFI, as well as macOS userspace (part of POSIX).

There is also musl-libc port for LKL, which might be interested in for some folks.

Further readings about LKL
=========================

- Discussion in github LKL issue
https://github.com/lkl/linux/issues/304

- LKL (an article)
https://www.researchgate.net/profile/Nicolae_Tapus2/publication/224164682_LKL_The_Linux_kernel_library/links/02bfe50fd921ab4f7c000000.pdf

*1 RFC email to LKML (back in 2015)
https://www.mail-archive.com/[email protected]/msg1012277.html

Please review the following changes for suitability for inclusion. If you have any objections or suggestions for improvement, please respond to the patches. If you agree with the changes, please provide your Acked-by.

The following changes since commit 4a7c46247f9c620c0390a15cb00b6ef9576b9c23:

um: Remove some unnecessary NULL checks in vector_user.c (2020-03-29 23:56:47 +0200)

are available in the Git repository at:

git://github.com/thehajime/linux 59ed4e23efa7abc9c7d52e6b5fb31fb8ce795c5c
https://github.com/thehajime/linux/tree/upstream-to-uml-5.6-rc7-v4

Hajime Tazaki (6):
lkl tools: host lib: filesystem helpers lkl tools: host lib: networking helpers
um lkl: add CI scripts to conduct regression tests um lkl: add UML network driver for lkl
um lkl: add UML block device driver (ubd) for lkl
um: fix clone flags to be familiar with valgrind

Octavian Purdila (19):
arch: add __SYSCALL_DEFINE_ARCH
um lkl: architecture skeleton for Linux kernel library um lkl: host interface
um lkl: memory handling
um lkl: kernel threads support
um lkl: interrupt support
um lkl: system call interface and application API
um lkl: timers, time and delay support
um lkl: basic kernel console support
um lkl: initialization and cleanup
um lkl: plug in the build system
lkl tools: skeleton for host side library
lkl tools: host lib: add utilities functions
lkl tools: host lib: posix host operations
lkl tools: add test programs
lkl tools: cptofs that reads/writes to/from a filesystem image lkl tools: fs2tar that converts a filesystem image to tar lkl tools: add lklfuse
um lkl: add documentation

 .circleci/config.yml                       |  198 ++++
 Documentation/virt/uml/lkl.txt             |   64 ++
 MAINTAINERS                                |    8 +
 arch/um/Kconfig                            |   13 +
 arch/um/Makefile                           |   19 +-
 arch/um/Makefile.um                        |   10 +
 arch/um/drivers/Makefile                   |    2 +
 arch/um/drivers/ubd_kern.c                 |    6 +-
 arch/um/drivers/ubd_user.c                 |    3 +-
 arch/um/include/asm/irq.h                  |    2 +
 arch/um/kernel/Makefile                    |    4 +
 arch/um/kernel/irq.c                       |    4 +
 arch/um/lkl/.gitignore                     |    2 +
 arch/um/lkl/Kconfig                        |   88 ++
 arch/um/lkl/Kconfig.debug                  |    0
 arch/um/lkl/Makefile                       |   67 ++
 arch/um/lkl/auto.conf                      |    1 +
 arch/um/lkl/configs/lkl_defconfig          | 1052 ++++++++++++++++++++
 arch/um/lkl/include/asm/Kbuild             |   79 ++
 arch/um/lkl/include/asm/atomic.h           |   11 +
 arch/um/lkl/include/asm/atomic64.h         |  114 +++
 arch/um/lkl/include/asm/bitsperlong.h      |   11 +
 arch/um/lkl/include/asm/byteorder.h        |    7 +
 arch/um/lkl/include/asm/cpu.h              |   14 +
 arch/um/lkl/include/asm/elf.h              |   15 +
 arch/um/lkl/include/asm/host_ops.h         |   10 +
 arch/um/lkl/include/asm/irq.h              |   16 +
 arch/um/lkl/include/asm/page.h             |   14 +
 arch/um/lkl/include/asm/pgtable.h          |   57 ++
 arch/um/lkl/include/asm/processor.h        |   60 ++
 arch/um/lkl/include/asm/ptrace.h           |   25 +
 arch/um/lkl/include/asm/sched.h            |   23 +
 arch/um/lkl/include/asm/setup.h            |    7 +
 arch/um/lkl/include/asm/syscalls.h         |   18 +
 arch/um/lkl/include/asm/syscalls_32.h      |   43 +
 arch/um/lkl/include/asm/thread_info.h      |   70 ++
 arch/um/lkl/include/asm/tlb.h              |   12 +
 arch/um/lkl/include/asm/uaccess.h          |   64 ++
 arch/um/lkl/include/asm/unistd.h           |   27 +
 arch/um/lkl/include/asm/unistd_32.h        |   31 +
 arch/um/lkl/include/asm/vmalloc.h          |    5 +
 arch/um/lkl/include/asm/vmlinux.lds.h      |   14 +
 arch/um/lkl/include/asm/xor.h              |    9 +
 arch/um/lkl/include/uapi/asm/Kbuild        |   11 +
 arch/um/lkl/include/uapi/asm/bitsperlong.h |   13 +
 arch/um/lkl/include/uapi/asm/byteorder.h   |   11 +
 arch/um/lkl/include/uapi/asm/host_ops.h    |  143 +++
 arch/um/lkl/include/uapi/asm/irq.h         |   36 +
 arch/um/lkl/include/uapi/asm/sigcontext.h  |   16 +
 arch/um/lkl/include/uapi/asm/syscalls.h    |  349 +++++++
 arch/um/lkl/include/uapi/asm/unistd.h      |   13 +
 arch/um/lkl/kernel/Makefile                |    4 +
 arch/um/lkl/kernel/asm-offsets.c           |    3 +
 arch/um/lkl/kernel/console.c               |   42 +
 arch/um/lkl/kernel/cpu.c                   |  223 +++++
 arch/um/lkl/kernel/irq.c                   |  223 +++++
 arch/um/lkl/kernel/misc.c                  |   60 ++
 arch/um/lkl/kernel/setup.c                 |  198 ++++
 arch/um/lkl/kernel/syscalls.c              |  192 ++++
 arch/um/lkl/kernel/syscalls_32.c           |  159 +++
 arch/um/lkl/kernel/threads.c               |  227 +++++
 arch/um/lkl/kernel/time.c                  |  145 +++
 arch/um/lkl/kernel/vmlinux.lds.S           |   51 +
 arch/um/lkl/mm/Makefile                    |    1 +
 arch/um/lkl/mm/bootmem.c                   |   99 ++
 arch/um/lkl/scripts/headers_install.py     |  196 ++++
 arch/um/os-Linux/Makefile                  |    5 +
 include/linux/syscalls.h                   |    6 +
 tools/lkl/.gitignore                       |   13 +
 tools/lkl/Build                            |    6 +
 tools/lkl/Makefile                         |  131 +++
 tools/lkl/Makefile.autoconf                |   67 ++
 tools/lkl/Targets                          |   17 +
 tools/lkl/cptofs.c                         |  636 ++++++++++++
 tools/lkl/fs2tar.c                         |  412 ++++++++
 tools/lkl/include/.gitignore               |    1 +
 tools/lkl/include/lkl.h                    |  788 +++++++++++++++
 tools/lkl/include/lkl_config.h             |   61 ++
 tools/lkl/include/lkl_host.h               |  112 +++
 tools/lkl/lib/.gitignore                   |    3 +
 tools/lkl/lib/Build                        |   13 +
 tools/lkl/lib/config.c                     |  750 ++++++++++++++
 tools/lkl/lib/dbg.c                        |  300 ++++++
 tools/lkl/lib/dbg_handler.c                |   44 +
 tools/lkl/lib/endian.h                     |   31 +
 tools/lkl/lib/fs.c                         |  471 +++++++++
 tools/lkl/lib/jmp_buf.c                    |   14 +
 tools/lkl/lib/jmp_buf.h                    |    8 +
 tools/lkl/lib/net.c                        |  834 ++++++++++++++++
 tools/lkl/lib/posix-host.c                 |  356 +++++++
 tools/lkl/lib/um/Build                     |    3 +
 tools/lkl/lib/um/um_block.c                |   17 +
 tools/lkl/lib/um/um_glue.c                 |   61 ++
 tools/lkl/lib/um/um_net.c                  |   30 +
 tools/lkl/lib/utils.c                      |  266 +++++
 tools/lkl/lklfuse.c                        |  659 ++++++++++++
 tools/lkl/scripts/checkpatch.sh            |   60 ++
 tools/lkl/scripts/lkl-jenkins.sh           |   21 +
 tools/lkl/tests/Build                      |    3 +
 tools/lkl/tests/boot.c                     |  562 +++++++++++
 tools/lkl/tests/boot.sh                    |    9 +
 tools/lkl/tests/cla.c                      |  159 +++
 tools/lkl/tests/cla.h                      |   33 +
 tools/lkl/tests/disk.c                     |  203 ++++
 tools/lkl/tests/disk.sh                    |   61 ++
 tools/lkl/tests/lklfuse.sh                 |  110 ++
 tools/lkl/tests/net-setup.sh               |  136 +++
 tools/lkl/tests/net-test.c                 |  307 ++++++
 tools/lkl/tests/net.sh                     |  135 +++
 tools/lkl/tests/run.py                     |  181 ++++
 tools/lkl/tests/tap13.py                   |  209 ++++
 tools/lkl/tests/test.c                     |  126 +++
 tools/lkl/tests/test.h                     |   72 ++
 tools/lkl/tests/test.sh                    |  182 ++++
 tools/lkl/tests/valgrind.supp              |  105 ++
 tools/lkl/tests/valgrind2xunit.py          |   69 ++
 116 files changed, 13569 insertions(+), 3 deletions(-)
 create mode 100644 .circleci/config.yml
 create mode 100644 Documentation/virt/uml/lkl.txt
 create mode 100644 arch/um/Makefile.um
 create mode 100644 arch/um/lkl/.gitignore
 create mode 100644 arch/um/lkl/Kconfig
 create mode 100644 arch/um/lkl/Kconfig.debug
 create mode 100644 arch/um/lkl/Makefile
 create mode 100644 arch/um/lkl/auto.conf
 create mode 100644 arch/um/lkl/configs/lkl_defconfig
 create mode 100644 arch/um/lkl/include/asm/Kbuild
 create mode 100644 arch/um/lkl/include/asm/atomic.h
 create mode 100644 arch/um/lkl/include/asm/atomic64.h
 create mode 100644 arch/um/lkl/include/asm/bitsperlong.h
 create mode 100644 arch/um/lkl/include/asm/byteorder.h
 create mode 100644 arch/um/lkl/include/asm/cpu.h
 create mode 100644 arch/um/lkl/include/asm/elf.h
 create mode 100644 arch/um/lkl/include/asm/host_ops.h
 create mode 100644 arch/um/lkl/include/asm/irq.h
 create mode 100644 arch/um/lkl/include/asm/page.h
 create mode 100644 arch/um/lkl/include/asm/pgtable.h
 create mode 100644 arch/um/lkl/include/asm/processor.h
 create mode 100644 arch/um/lkl/include/asm/ptrace.h
 create mode 100644 arch/um/lkl/include/asm/sched.h
 create mode 100644 arch/um/lkl/include/asm/setup.h
 create mode 100644 arch/um/lkl/include/asm/syscalls.h
 create mode 100644 arch/um/lkl/include/asm/syscalls_32.h
 create mode 100644 arch/um/lkl/include/asm/thread_info.h
 create mode 100644 arch/um/lkl/include/asm/tlb.h
 create mode 100644 arch/um/lkl/include/asm/uaccess.h
 create mode 100644 arch/um/lkl/include/asm/unistd.h
 create mode 100644 arch/um/lkl/include/asm/unistd_32.h
 create mode 100644 arch/um/lkl/include/asm/vmalloc.h
 create mode 100644 arch/um/lkl/include/asm/vmlinux.lds.h
 create mode 100644 arch/um/lkl/include/asm/xor.h
 create mode 100644 arch/um/lkl/include/uapi/asm/Kbuild
 create mode 100644 arch/um/lkl/include/uapi/asm/bitsperlong.h
 create mode 100644 arch/um/lkl/include/uapi/asm/byteorder.h
 create mode 100644 arch/um/lkl/include/uapi/asm/host_ops.h
 create mode 100644 arch/um/lkl/include/uapi/asm/irq.h
 create mode 100644 arch/um/lkl/include/uapi/asm/sigcontext.h
 create mode 100644 arch/um/lkl/include/uapi/asm/syscalls.h
 create mode 100644 arch/um/lkl/include/uapi/asm/unistd.h
 create mode 100644 arch/um/lkl/kernel/Makefile
 create mode 100644 arch/um/lkl/kernel/asm-offsets.c
 create mode 100644 arch/um/lkl/kernel/console.c
 create mode 100644 arch/um/lkl/kernel/cpu.c
 create mode 100644 arch/um/lkl/kernel/irq.c
 create mode 100644 arch/um/lkl/kernel/misc.c
 create mode 100644 arch/um/lkl/kernel/setup.c
 create mode 100644 arch/um/lkl/kernel/syscalls.c
 create mode 100644 arch/um/lkl/kernel/syscalls_32.c
 create mode 100644 arch/um/lkl/kernel/threads.c
 create mode 100644 arch/um/lkl/kernel/time.c
 create mode 100644 arch/um/lkl/kernel/vmlinux.lds.S
 create mode 100644 arch/um/lkl/mm/Makefile
 create mode 100644 arch/um/lkl/mm/bootmem.c
 create mode 100755 arch/um/lkl/scripts/headers_install.py
 create mode 100644 tools/lkl/.gitignore
 create mode 100644 tools/lkl/Build
 create mode 100644 tools/lkl/Makefile
 create mode 100644 tools/lkl/Makefile.autoconf
 create mode 100644 tools/lkl/Targets
 create mode 100644 tools/lkl/cptofs.c
 create mode 100644 tools/lkl/fs2tar.c
 create mode 100644 tools/lkl/include/.gitignore
 create mode 100644 tools/lkl/include/lkl.h
 create mode 100644 tools/lkl/include/lkl_config.h
 create mode 100644 tools/lkl/include/lkl_host.h
 create mode 100644 tools/lkl/lib/.gitignore
 create mode 100644 tools/lkl/lib/Build
 create mode 100644 tools/lkl/lib/config.c
 create mode 100644 tools/lkl/lib/dbg.c
 create mode 100644 tools/lkl/lib/dbg_handler.c
 create mode 100644 tools/lkl/lib/endian.h
 create mode 100644 tools/lkl/lib/fs.c
 create mode 100644 tools/lkl/lib/jmp_buf.c
 create mode 100644 tools/lkl/lib/jmp_buf.h
 create mode 100644 tools/lkl/lib/net.c
 create mode 100644 tools/lkl/lib/posix-host.c
 create mode 100644 tools/lkl/lib/um/Build
 create mode 100644 tools/lkl/lib/um/um_block.c
 create mode 100644 tools/lkl/lib/um/um_glue.c
 create mode 100644 tools/lkl/lib/um/um_net.c
 create mode 100644 tools/lkl/lib/utils.c
 create mode 100644 tools/lkl/lklfuse.c
 create mode 100755 tools/lkl/scripts/checkpatch.sh
 create mode 100755 tools/lkl/scripts/lkl-jenkins.sh
 create mode 100644 tools/lkl/tests/Build
 create mode 100644 tools/lkl/tests/boot.c
 create mode 100755 tools/lkl/tests/boot.sh
 create mode 100644 tools/lkl/tests/cla.c
 create mode 100644 tools/lkl/tests/cla.h
 create mode 100644 tools/lkl/tests/disk.c
 create mode 100755 tools/lkl/tests/disk.sh
 create mode 100755 tools/lkl/tests/lklfuse.sh
 create mode 100644 tools/lkl/tests/net-setup.sh
 create mode 100644 tools/lkl/tests/net-test.c
 create mode 100755 tools/lkl/tests/net.sh
 create mode 100755 tools/lkl/tests/run.py
 create mode 100644 tools/lkl/tests/tap13.py
 create mode 100644 tools/lkl/tests/test.c
 create mode 100644 tools/lkl/tests/test.h
 create mode 100644 tools/lkl/tests/test.sh
 create mode 100644 tools/lkl/tests/valgrind.supp
 create mode 100755 tools/lkl/tests/valgrind2xunit.py
Signature · 1 line
2.21.0 (Apple Git-122.2)
#38 lore raw

RFC v4 1/25 arch: add __SYSCALL_DEFINE_ARCH

From: Octavian Purdila <[email protected]>

This allows the architecture code to process the system call definitions. It is used by LKL to create strong typed function definitions for system calls.

---
 include/linux/syscalls.h | 6 ++++++
 1 file changed, 6 insertions(+)
include/linux/syscalls.h +6 -0
index 1815065d52f3..e45815a3ee10 100644
@@ -203,9 +203,14 @@ static inline int is_syscall_trace_event(struct trace_event_call *tp_event)
 }
 #endif
 
+#ifndef __SYSCALL_DEFINE_ARCH
+#define __SYSCALL_DEFINE_ARCH(x, sname, ...)
+#endif
+
 #ifndef SYSCALL_DEFINE0
 #define SYSCALL_DEFINE0(sname)					\
 	SYSCALL_METADATA(_##sname, 0);				\
+	__SYSCALL_DEFINE_ARCH(0, _##sname);			\
 	asmlinkage long sys_##sname(void);			\
 	ALLOW_ERROR_INJECTION(sys_##sname, ERRNO);		\
 	asmlinkage long sys_##sname(void)
@@ -222,6 +227,7 @@ static inline int is_syscall_trace_event(struct trace_event_call *tp_event)
 
 #define SYSCALL_DEFINEx(x, sname, ...)				\
 	SYSCALL_METADATA(sname, x, __VA_ARGS__)			\
+	__SYSCALL_DEFINE_ARCH(x, sname, __VA_ARGS__)		\
 	__SYSCALL_DEFINEx(x, sname, __VA_ARGS__)
 
 #define __PROTECT(...) asmlinkage_protect(__VA_ARGS__)
Signature · 1 line
2.21.0 (Apple Git-122.2)
#39 lore raw

RFC v4 2/25 um lkl: architecture skeleton for Linux kernel library

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

Adds the LKL Kconfig, vmlinux linker script, basic architecture headers and miscellaneous basic functions or stubs such as dump_stack(), show_regs() and cpuinfo proc ops.

The headers we introduce in this patch are simple wrappers to the asm-generic headers or stubs for things we don't support, such as ptrace, DMA, signals, ELF handling and low level processor operations.

The kernel configuration is automatically updated to reflect the endianness of the host, 64bit support or the output format for vmlinux's linker script. We do this by looking at the ld's default output format.

Cc Andreas Abel <[email protected]>
Cc Conrad Meyer <[email protected]>
Cc Edison M. Castro <[email protected]>
Cc H.K. Jerry Chu <[email protected]>
Cc Jens Staal <[email protected]>
Cc Lai Jiangshan <[email protected]>
Cc Levente Kurusa <[email protected]>
Cc Luca Dariz <[email protected]>
Cc Mark Stillwell <[email protected]>
Cc Matthieu Coudron <[email protected]>
Cc Michael Zimmermann <[email protected]>
Cc Motomu Utsumi <[email protected]>
Cc Patrick Collins <[email protected]>
Cc Petros Angelatos <[email protected]>
Cc Pierre-Hugues Husson <[email protected]>
Cc Xiao Jia <[email protected]>
Cc Yuan Liu <[email protected]>
---
 MAINTAINERS                                |    8 +
 arch/um/lkl/.gitignore                     |    2 +
 arch/um/lkl/Kconfig                        |   56 ++
 arch/um/lkl/Kconfig.debug                  |    0
 arch/um/lkl/configs/lkl_defconfig          | 1052 ++++++++++++++++++++
 arch/um/lkl/include/asm/Kbuild             |   79 ++
 arch/um/lkl/include/asm/atomic.h           |   11 +
 arch/um/lkl/include/asm/atomic64.h         |  114 +++
 arch/um/lkl/include/asm/bitsperlong.h      |   11 +
 arch/um/lkl/include/asm/byteorder.h        |    7 +
 arch/um/lkl/include/asm/cpu.h              |   14 +
 arch/um/lkl/include/asm/elf.h              |   15 +
 arch/um/lkl/include/asm/processor.h        |   60 ++
 arch/um/lkl/include/asm/ptrace.h           |   25 +
 arch/um/lkl/include/asm/sched.h            |   23 +
 arch/um/lkl/include/asm/syscalls.h         |   18 +
 arch/um/lkl/include/asm/syscalls_32.h      |   43 +
 arch/um/lkl/include/asm/tlb.h              |   12 +
 arch/um/lkl/include/asm/uaccess.h          |   64 ++
 arch/um/lkl/include/asm/unistd_32.h        |   31 +
 arch/um/lkl/include/asm/vmalloc.h          |    5 +
 arch/um/lkl/include/asm/vmlinux.lds.h      |   14 +
 arch/um/lkl/include/asm/xor.h              |    9 +
 arch/um/lkl/include/uapi/asm/Kbuild        |   11 +
 arch/um/lkl/include/uapi/asm/bitsperlong.h |   13 +
 arch/um/lkl/include/uapi/asm/byteorder.h   |   11 +
 arch/um/lkl/include/uapi/asm/syscalls.h    |  348 +++++++
 arch/um/lkl/kernel/asm-offsets.c           |    2 +
 arch/um/lkl/kernel/misc.c                  |   60 ++
 arch/um/lkl/kernel/vmlinux.lds.S           |   51 +
 30 files changed, 2169 insertions(+)
 create mode 100644 arch/um/lkl/.gitignore
 create mode 100644 arch/um/lkl/Kconfig
 create mode 100644 arch/um/lkl/Kconfig.debug
 create mode 100644 arch/um/lkl/configs/lkl_defconfig
 create mode 100644 arch/um/lkl/include/asm/Kbuild
 create mode 100644 arch/um/lkl/include/asm/atomic.h
 create mode 100644 arch/um/lkl/include/asm/atomic64.h
 create mode 100644 arch/um/lkl/include/asm/bitsperlong.h
 create mode 100644 arch/um/lkl/include/asm/byteorder.h
 create mode 100644 arch/um/lkl/include/asm/cpu.h
 create mode 100644 arch/um/lkl/include/asm/elf.h
 create mode 100644 arch/um/lkl/include/asm/processor.h
 create mode 100644 arch/um/lkl/include/asm/ptrace.h
 create mode 100644 arch/um/lkl/include/asm/sched.h
 create mode 100644 arch/um/lkl/include/asm/syscalls.h
 create mode 100644 arch/um/lkl/include/asm/syscalls_32.h
 create mode 100644 arch/um/lkl/include/asm/tlb.h
 create mode 100644 arch/um/lkl/include/asm/uaccess.h
 create mode 100644 arch/um/lkl/include/asm/unistd_32.h
 create mode 100644 arch/um/lkl/include/asm/vmalloc.h
 create mode 100644 arch/um/lkl/include/asm/vmlinux.lds.h
 create mode 100644 arch/um/lkl/include/asm/xor.h
 create mode 100644 arch/um/lkl/include/uapi/asm/Kbuild
 create mode 100644 arch/um/lkl/include/uapi/asm/bitsperlong.h
 create mode 100644 arch/um/lkl/include/uapi/asm/byteorder.h
 create mode 100644 arch/um/lkl/include/uapi/asm/syscalls.h
 create mode 100644 arch/um/lkl/kernel/asm-offsets.c
 create mode 100644 arch/um/lkl/kernel/misc.c
 create mode 100644 arch/um/lkl/kernel/vmlinux.lds.S
MAINTAINERS +8 -0
index cc1d18cb5d18..f3913964eabe 100644
@@ -9710,6 +9710,14 @@ F:	Documentation/core-api/atomic_ops.rst
 F:	Documentation/core-api/refcount-vs-atomic.rst
 F:	Documentation/memory-barriers.txt
 
+LINUX KERNEL LIBRARY
+M:	Octavian Purdila <[email protected]>
+M:	Hajime Tazaki <[email protected]>
+L:	[email protected]
+S:	Maintained
+F:	arch/um/lkl/
+F:	tools/lkl/
+
 LIS3LV02D ACCELEROMETER DRIVER
 M:	Eric Piel <[email protected]>
 S:	Maintained
arch/um/lkl/.gitignore new file +2 -0
new file mode 100644
index 000000000000..ced1c60d8235
@@ -0,0 +1,2 @@
+kernel/vmlinux.lds
+include/generated
arch/um/lkl/Kconfig new file +56 -0

56 lines not on this page · raw message

arch/um/lkl/Kconfig.debug new file +0 -0
new file mode 100644
index 000000000000..e69de29bb2d1
arch/um/lkl/configs/lkl_defconfig new file +1052 -0

1,052 lines not on this page · raw message

arch/um/lkl/include/asm/Kbuild new file +79 -0

79 lines not on this page · raw message

arch/um/lkl/include/asm/atomic.h new file +11 -0
new file mode 100644
index 000000000000..0c5f9f8e4b30
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __LKL_ATOMIC_H
+#define __LKL_ATOMIC_H
+
+#include <asm-generic/atomic.h>
+
+#ifndef CONFIG_GENERIC_ATOMIC64
+#include "atomic64.h"
+#endif /* CONFIG_GENERIC_ATOMIC64 */
+
+#endif
arch/um/lkl/include/asm/atomic64.h new file +114 -0

114 lines not on this page · raw message

arch/um/lkl/include/asm/bitsperlong.h new file +11 -0
new file mode 100644
index 000000000000..5745d5e51274
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __LKL_BITSPERLONG_H
+#define __LKL_BITSPERLONG_H
+
+#include <uapi/asm/bitsperlong.h>
+
+#define BITS_PER_LONG __BITS_PER_LONG
+
+#define BITS_PER_LONG_LONG 64
+
+#endif
arch/um/lkl/include/asm/byteorder.h new file +7 -0
new file mode 100644
index 000000000000..5d0c4efaa44b
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_BYTEORDER_H
+#define _ASM_LKL_BYTEORDER_H
+
+#include <uapi/asm/byteorder.h>
+
+#endif /* _ASM_LKL_BYTEORDER_H */
arch/um/lkl/include/asm/cpu.h new file +14 -0
new file mode 100644
index 000000000000..d2b8c501c7b1
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_CPU_H
+#define _ASM_LKL_CPU_H
+
+int lkl_cpu_get(void);
+void lkl_cpu_put(void);
+int lkl_cpu_try_run_irq(int irq);
+int lkl_cpu_init(void);
+void lkl_cpu_shutdown(void);
+void lkl_cpu_wait_shutdown(void);
+void lkl_cpu_change_owner(lkl_thread_t owner);
+void lkl_cpu_set_irqs_pending(void);
+
+#endif /* _ASM_LKL_CPU_H */
arch/um/lkl/include/asm/elf.h new file +15 -0
new file mode 100644
index 000000000000..bb2456d638f4
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_ELF_H
+#define _ASM_LKL_ELF_H
+
+#define elf_check_arch(x) 0
+
+#ifdef CONFIG_64BIT
+#define ELF_CLASS ELFCLASS64
+#else
+#define ELF_CLASS ELFCLASS32
+#endif
+
+#define elf_gregset_t long
+#define elf_fpregset_t double
+#endif
arch/um/lkl/include/asm/processor.h new file +60 -0

60 lines not on this page · raw message

arch/um/lkl/include/asm/ptrace.h new file +25 -0
new file mode 100644
index 000000000000..28199be26dc0
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_PTRACE_H
+#define _ASM_LKL_PTRACE_H
+
+#include <linux/errno.h>
+
+struct task_struct;
+
+#define user_mode(regs) 0
+#define kernel_mode(regs) 1
+#define profile_pc(regs) 0
+#define instruction_pointer(regs) 0
+#define user_stack_pointer(regs) 0
+
+static inline long arch_ptrace(struct task_struct *child, long request,
+			       unsigned long addr, unsigned long data)
+{
+	return -EINVAL;
+}
+
+static inline void ptrace_disable(struct task_struct *child)
+{
+}
+
+#endif
arch/um/lkl/include/asm/sched.h new file +23 -0
new file mode 100644
index 000000000000..4c2635921ec8
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_SCHED_H
+#define _ASM_LKL_SCHED_H
+
+#include <linux/sched.h>
+#include <uapi/asm/host_ops.h>
+
+static inline void thread_sched_jb(void)
+{
+	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD)) {
+		set_ti_thread_flag(current_thread_info(), TIF_SCHED_JB);
+		set_current_state(TASK_UNINTERRUPTIBLE);
+		lkl_ops->jmp_buf_set(&current_thread_info()->sched_jb,
+				     schedule);
+	} else {
+		lkl_bug("%s() can be used only for host task\n", __func__);
+	}
+}
+
+void switch_to_host_task(struct task_struct *);
+int host_task_stub(void *unused);
+
+#endif /*  _ASM_LKL_SCHED_H */
arch/um/lkl/include/asm/syscalls.h new file +18 -0
new file mode 100644
index 000000000000..2eaa870a9020
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_SYSCALLS_H
+#define _ASM_LKL_SYSCALLS_H
+
+int syscalls_init(void);
+void syscalls_cleanup(void);
+long lkl_syscall(long no, long *params);
+void wakeup_idle_host_task(void);
+
+#define sys_mmap sys_mmap_pgoff
+#define sys_mmap2 sys_mmap_pgoff
+#define sys_clone sys_ni_syscall
+#define sys_vfork sys_ni_syscall
+#define sys_rt_sigreturn sys_ni_syscall
+
+#include <asm-generic/syscalls.h>
+
+#endif /* _ASM_LKL_SYSCALLS_H */
arch/um/lkl/include/asm/syscalls_32.h new file +43 -0

43 lines not on this page · raw message

arch/um/lkl/include/asm/tlb.h new file +12 -0
new file mode 100644
index 000000000000..d474890d317d
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_TLB_H
+#define _ASM_LKL_TLB_H
+
+#define tlb_start_vma(tlb, vma)				do { } while (0)
+#define tlb_end_vma(tlb, vma)				do { } while (0)
+#define __tlb_remove_tlb_entry(tlb, pte, address)	do { } while (0)
+#define tlb_flush(tlb)					do { } while (0)
+
+#include <asm-generic/tlb.h>
+
+#endif /* _ASM_LKL_TLB_H */
arch/um/lkl/include/asm/uaccess.h new file +64 -0

64 lines not on this page · raw message

arch/um/lkl/include/asm/unistd_32.h new file +31 -0
new file mode 100644
index 000000000000..8582a55e61e2
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <asm/bitsperlong.h>
+
+#ifndef __SYSCALL
+#define __SYSCALL(x, y)
+#endif
+
+#if __BITS_PER_LONG == 32
+__SYSCALL(__NR3264_truncate, sys32_truncate64)
+__SYSCALL(__NR3264_ftruncate, sys32_ftruncate64)
+
+#ifdef CONFIG_MMU
+__SYSCALL(__NR3264_mmap, sys32_mmap)
+#endif
+
+__SYSCALL(__NR_wait4, sys32_wait4)
+
+__SYSCALL(__NR_pread64, sys32_pread64)
+__SYSCALL(__NR_pwrite64, sys32_pwrite64)
+
+__SYSCALL(__NR_readahead, sys32_readahead)
+#ifdef __ARCH_WANT_SYNC_FILE_RANGE2
+__SYSCALL(__NR_sync_file_range2, sys32_sync_file_range2)
+#else
+__SYSCALL(__NR_sync_file_range, sys32_sync_file_range)
+#endif
+/* mm/fadvise.c */
+__SYSCALL(__NR3264_fadvise64, sys32_fadvise64_64)
+__SYSCALL(__NR_fallocate, sys32_fallocate)
+
+#endif
arch/um/lkl/include/asm/vmalloc.h new file +5 -0
new file mode 100644
index 000000000000..46de2e4f03c7
@@ -0,0 +1,5 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_VMALLOC_H
+#define _LKL_VMALLOC_H
+
+#endif /* _LKL_VMALLOC_H */
arch/um/lkl/include/asm/vmlinux.lds.h new file +14 -0
new file mode 100644
index 000000000000..a3c285882dc4
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_VMLINUX_LDS_H
+#define _LKL_VMLINUX_LDS_H
+
+/* we encode our own __ro_after_init section */
+#define RO_AFTER_INIT_DATA
+
+#ifdef __MINGW32__
+#define RODATA_SECTION .rdata
+#endif
+
+#include <asm-generic/vmlinux.lds.h>
+
+#endif
arch/um/lkl/include/asm/xor.h new file +9 -0
new file mode 100644
index 000000000000..286ce75b5d9d
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_XOR_H
+#define _ASM_LKL_XOR_H
+
+#include <asm-generic/xor.h>
+
+#define XOR_SELECT_TEMPLATE(x) (&xor_block_8regs)
+
+#endif /* _ASM_LKL_XOR_H */
arch/um/lkl/include/uapi/asm/Kbuild new file +11 -0
new file mode 100644
index 000000000000..e1f554364a56
@@ -0,0 +1,11 @@
+# UAPI Header export list
+
+generic-y += elf.h
+generic-y += kvm_para.h
+generic-y += shmparam.h
+generic-y += siginfo.h
+generic-y += swab.h
+generic-y += timex.h
+
+# no header-y since we need special user headers handling
+# see arch/lkl/script/headers.py
arch/um/lkl/include/uapi/asm/bitsperlong.h new file +13 -0
new file mode 100644
index 000000000000..8b4ebf2b0264
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_BITSPERLONG_H
+#define _ASM_UAPI_LKL_BITSPERLONG_H
+
+#ifdef CONFIG_64BIT
+#define __BITS_PER_LONG 64
+#else
+#define __BITS_PER_LONG 32
+#endif
+
+#define __ARCH_WANT_STAT64
+
+#endif /* _ASM_UAPI_LKL_BITSPERLONG_H */
arch/um/lkl/include/uapi/asm/byteorder.h new file +11 -0
new file mode 100644
index 000000000000..3c4a58d2062f
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_BYTEORDER_H
+#define _ASM_UAPI_LKL_BYTEORDER_H
+
+#if defined(CONFIG_BIG_ENDIAN)
+#include <linux/byteorder/big_endian.h>
+#else
+#include <linux/byteorder/little_endian.h>
+#endif
+
+#endif /* _ASM_UAPI_LKL_BYTEORDER_H */
arch/um/lkl/include/uapi/asm/syscalls.h new file +348 -0

348 lines not on this page · raw message

arch/um/lkl/kernel/asm-offsets.c new file +2 -0
new file mode 100644
index 000000000000..6be0763698dc
@@ -0,0 +1,2 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Dummy asm-offsets.c file. Required by kbuild and ready to be used - hint! */
arch/um/lkl/kernel/misc.c new file +60 -0

60 lines not on this page · raw message

arch/um/lkl/kernel/vmlinux.lds.S new file +51 -0

51 lines not on this page · raw message

Signature · 1 line
2.21.0 (Apple Git-122.2)
maintainer
#40 lore raw

Re: [RFC v4 02/25] um lkl: architecture skeleton for Linux kernel library

Quoted 6 lines · Hajime Tazaki

From: Octavian Purdila <[email protected]>

Adds the LKL Kconfig, vmlinux linker script, basic architecture headers and miscellaneous basic functions or stubs such as dump_stack(), show_regs() and cpuinfo proc ops.

Quoted 8 lines

The headers we introduce in this patch are simple wrappers to the asm-generic headers or stubs for things we don't support, such as ptrace, DMA, signals, ELF handling and low level processor operations.

The kernel configuration is automatically updated to reflect the endianness of the host, 64bit support or the output format for vmlinux's linker script. We do this by looking at the ld's default output format.

Can you elaborate what the plan is here?

I mean, you're not actually "unifying" anything with ARCH=um, you're just basically splitting ARCH=um into ARCH=um-um and ARCH=um-lkl or something. Is there much point?

Even the basic underlying building blocks are _very_ different, e.g. in UML the host interface is just a bunch of functions that must be implemented (os_*()), while you have a "struct lkl_host_operations". If we can't even unify at that trivial level, is there any point in it at all? I'm not even really sure what those ops are used for - are all of those things that the *application* using LKL necessarily must provide?

Similarly with the IRQ routing mechanism - two completely different concepts. Where's the "unifying"?

Ultimately, I can see two ways this goes:

1) We give up, and get ARCH=lkl, sharing just (some of) the drivers while moving them into the right drivers/somewhere/ place. Even that looks somewhat awkward looking at the later patches in this set, but seems like that at *least* should be done.

2) Ideally, instead, we actually unify: LKL grows support for userspace processes using UML infrastructure, the "in-kernel" IRQ mechanisms are unified, UML stuff moves into lkl-ops, and the UML binary basically becomes a user of the LKL library to start everything up. There may be some bits remaining that are just not interesting (e.g. some drivers you don't care about would continue to make direct calls to the user side instead of lkl-ops, and then they're just not compatible with lkl, only with the uml special case of lkl), but then things are clean.

Now, of course it seems like (2) would actually be better - LKL would actually get support for userspace processes using UML's tricks, most of the code is unified, and even LKL's users can take advantage of new things. At the same time, all of the duplication is avoided.

However, I just don't know if it's actually _possible_ to do that though. Conceptually, it seems like it should be - why shouldn't a library be able to spawn other userspace processes - I mean, it's not really in the model that LKL really _wants_ since it's supposed to offer the syscall API, but you could make a "bool run_normal_init" or something in the lkl-ops for the user of the library to determine what should happen, right?

However, there clearly are huge differences between LKL and UML in all respects, and I don't know if this wouldn't conflict with the library model, e.g. there may be linker issues etc. Or maybe the specific UML interrupt handling is required in UML and cannot be done in LKL (but then perhaps it could be put into the hypothetical UML-application vs. UML-the-LKL-library?)

Ultimately, personally I think it's going to have to be one or the other of those two options though, I don't really see much value in what you're doing here in the patchset now. This way just messes up everything, it's not clear what's UML and what's LKL, and they're intertwined with ifdefs everywhere; just look at where you have to add ifdefs in patch 23 - how would anyone later understand which part gets compiled for which of them?

johannes

PS: actually having something like lkl-ops in UML would've been nice also for my "time-travel" work, since it neatly abstracts the timers out. I do wonder a bit about the overhead of jumping through function pointers all the time though, it may be worth rethinking that overall anyway!

#41 lore raw

Re: [RFC v4 02/25] um lkl: architecture skeleton for Linux kernel library

Johannes, Hajime,

Quoted 33 lines · Johannes Berg
Quoted 6 lines · Hajime Tazaki

From: Octavian Purdila <[email protected]>

Adds the LKL Kconfig, vmlinux linker script, basic architecture headers and miscellaneous basic functions or stubs such as dump_stack(), show_regs() and cpuinfo proc ops.

Quoted 8 lines

The headers we introduce in this patch are simple wrappers to the asm-generic headers or stubs for things we don't support, such as ptrace, DMA, signals, ELF handling and low level processor operations.

The kernel configuration is automatically updated to reflect the endianness of the host, 64bit support or the output format for vmlinux's linker script. We do this by looking at the ld's default output format.

Can you elaborate what the plan is here?

I mean, you're not actually "unifying" anything with ARCH=um, you're just basically splitting ARCH=um into ARCH=um-um and ARCH=um-lkl or something. Is there much point?

Even the basic underlying building blocks are _very_ different, e.g. in UML the host interface is just a bunch of functions that must be implemented (os_*()), while you have a "struct lkl_host_operations". If we can't even unify at that trivial level, is there any point in it at all? I'm not even really sure what those ops are used for - are all of those things that the *application* using LKL necessarily must provide?

Similarly with the IRQ routing mechanism - two completely different concepts. Where's the "unifying"?

Quoted 6 lines

Ultimately, I can see two ways this goes:

1) We give up, and get ARCH=lkl, sharing just (some of) the drivers while moving them into the right drivers/somewhere/ place. Even that looks somewhat awkward looking at the later patches in this set, but seems like that at *least* should be done.

Yeah, this would be a goal.
UML and LKL are quite different but they should share at least their userspace drivers.
I also don't mind if we don't share every driver at the beginning but it should be
a feasible goal for the future.

Quoted 9 lines

2) Ideally, instead, we actually unify: LKL grows support for userspace processes using UML infrastructure, the "in-kernel" IRQ mechanisms are unified, UML stuff moves into lkl-ops, and the UML binary basically becomes a user of the LKL library to start everything up. There may be some bits remaining that are just not interesting (e.g. some drivers you don't care about would continue to make direct calls to the user side instead of lkl-ops, and then they're just not compatible with lkl, only with the uml special case of lkl), but then things are clean.

A few months ago I though this is doable but now I'm not so sure anymore.

Quoted 30 lines

Now, of course it seems like (2) would actually be better - LKL would actually get support for userspace processes using UML's tricks, most of the code is unified, and even LKL's users can take advantage of new things. At the same time, all of the duplication is avoided.

However, I just don't know if it's actually _possible_ to do that though. Conceptually, it seems like it should be - why shouldn't a library be able to spawn other userspace processes - I mean, it's not really in the model that LKL really _wants_ since it's supposed to offer the syscall API, but you could make a "bool run_normal_init" or something in the lkl-ops for the user of the library to determine what should happen, right?

However, there clearly are huge differences between LKL and UML in all respects, and I don't know if this wouldn't conflict with the library model, e.g. there may be linker issues etc. Or maybe the specific UML interrupt handling is required in UML and cannot be done in LKL (but then perhaps it could be put into the hypothetical UML-application vs. UML-the-LKL-library?)

Ultimately, personally I think it's going to have to be one or the other of those two options though, I don't really see much value in what you're doing here in the patchset now. This way just messes up everything, it's not clear what's UML and what's LKL, and they're intertwined with ifdefs everywhere; just look at where you have to add ifdefs in patch 23 - how would anyone later understand which part gets compiled for which of them?

Quoted 7 lines

johannes

PS: actually having something like lkl-ops in UML would've been nice also for my "time-travel" work, since it neatly abstracts the timers out. I do wonder a bit about the overhead of jumping through function pointers all the time though, it may be worth rethinking that overall anyway!

Agreed. UML can also learn from LKL. :-)

Signature · 2 lines
Thanks,
//richard
#42 lore raw

Re: [RFC v4 02/25] um lkl: architecture skeleton for Linux kernel library

Hello Johannes, Richard

Quoted 23 lines · Richard Weinberger

Johannes, Hajime,

Quoted 18 lines · Johannes Berg
Quoted 6 lines · Hajime Tazaki

From: Octavian Purdila <[email protected]>

Adds the LKL Kconfig, vmlinux linker script, basic architecture headers and miscellaneous basic functions or stubs such as dump_stack(), show_regs() and cpuinfo proc ops.

Quoted 8 lines

The headers we introduce in this patch are simple wrappers to the asm-generic headers or stubs for things we don't support, such as ptrace, DMA, signals, ELF handling and low level processor operations.

The kernel configuration is automatically updated to reflect the endianness of the host, 64bit support or the output format for vmlinux's linker script. We do this by looking at the ld's default output format.

Can you elaborate what the plan is here?

I always wrote down the current plan that I have in my mind in the cover letter (patch 00); please take a look at the "Milestone" block.

Quoted:

Quoted 13 lines

Milestone
=========
This patches is a first step toward upstreaming *library mode* of Linux kernel, but we think we need to have several steps toward our goal, describing in the below.

1. Put LKL code under arch/um (arch/um/lkl), and build it in a separate way from UML.
2. Use UML driver implementations in LKL as a minimum set of patches
- Only support x86 sub architecture (dependency to UML drivers)
3. Support broader host supports
- add virtio device features

Quoted 8 lines

For the step 1, we put LKL as one of UMMODE in order to make less effort to integrate (make ARCH=um UMMODE=library). The modification to existing UML code is trying to be minimized.

Quoted 3 lines

I mean, you're not actually "unifying" anything with ARCH=um, you're just basically splitting ARCH=um into ARCH=um-um and ARCH=um-lkl or something. Is there much point?

The current step (1 in the milestone) tries to cover this goal:
splitting ARCH=um into UMMODE_KERNEL and UMMODE_LIB.

Quoted 17 lines
Quoted 10 lines

Even the basic underlying building blocks are _very_ different, e.g. in UML the host interface is just a bunch of functions that must be implemented (os_*()), while you have a "struct lkl_host_operations". If we can't even unify at that trivial level, is there any point in it at all? I'm not even really sure what those ops are used for - are all of those things that the *application* using LKL necessarily must provide?

Similarly with the IRQ routing mechanism - two completely different concepts. Where's the "unifying"?

Quoted 6 lines

Ultimately, I can see two ways this goes:

1) We give up, and get ARCH=lkl, sharing just (some of) the drivers while moving them into the right drivers/somewhere/ place. Even that looks somewhat awkward looking at the later patches in this set, but seems like that at *least* should be done.

Quoted 6 lines

Yeah, this would be a goal.
UML and LKL are quite different but they should share at least their userspace drivers.
I also don't mind if we don't share every driver at the beginning but it should be
a feasible goal for the future.

Sharing drivers code is also included in this patchset, step 2 in the milestone.

I was thinking that implementing os_*() functions with lkl_host_ops would be the further goal (e.g., step 3 or 4).

Quoted 11 lines
Quoted 9 lines

2) Ideally, instead, we actually unify: LKL grows support for userspace processes using UML infrastructure, the "in-kernel" IRQ mechanisms are unified, UML stuff moves into lkl-ops, and the UML binary basically becomes a user of the LKL library to start everything up. There may be some bits remaining that are just not interesting (e.g. some drivers you don't care about would continue to make direct calls to the user side instead of lkl-ops, and then they're just not compatible with lkl, only with the uml special case of lkl), but then things are clean.

A few months ago I though this is doable but now I'm not so sure anymore.

For the part of (2) which Johannes pointed out (I mean the part "UML stuff moves into lkl-ops"), I become to think that implementing os_*() functions using lkl_host_ops would be also interesting if those re-implementation makes the glue code eliminated.

I'll work on that.

Quoted 13 lines
Quoted 13 lines

Now, of course it seems like (2) would actually be better - LKL would actually get support for userspace processes using UML's tricks, most of the code is unified, and even LKL's users can take advantage of new things. At the same time, all of the duplication is avoided.

However, I just don't know if it's actually _possible_ to do that though. Conceptually, it seems like it should be - why shouldn't a library be able to spawn other userspace processes - I mean, it's not really in the model that LKL really _wants_ since it's supposed to offer the syscall API, but you could make a "bool run_normal_init" or something in the lkl-ops for the user of the library to determine what should happen, right?

Quoted 6 lines
Quoted 6 lines

However, there clearly are huge differences between LKL and UML in all respects, and I don't know if this wouldn't conflict with the library model, e.g. there may be linker issues etc. Or maybe the specific UML interrupt handling is required in UML and cannot be done in LKL (but then perhaps it could be put into the hypothetical UML-application vs. UML-the-LKL-library?)

For the other part of (2), I agree that your definition of the unification will be the best and final goal of this integration.

But, especially the support for UML userspace processes in LKL is not easy as far as I can see, at least the current LKL doesn't have, I treat those extensions (from the LKL pov) as further goals.

Or the title of the cover letter is somehow overstatement: instead, "Minimum integration of UML with LKL" or something like this would be better ?

Quoted 7 lines
Quoted 7 lines

Ultimately, personally I think it's going to have to be one or the other of those two options though, I don't really see much value in what you're doing here in the patchset now. This way just messes up everything, it's not clear what's UML and what's LKL, and they're intertwined with ifdefs everywhere; just look at where you have to add ifdefs in patch 23 - how would anyone later understand which part gets compiled for which of them?

Since the patchset of LKL is relatively huge, I was trying to make a minimum patchset for the first step. Because having #ifdefs and glue code makes existing arch/um code untouched, I took this way to make the patchset as small as possible.

But if this is not the case, I will look for the other way and touch arch/um code to fit the current LKL implementation.

What do you think ?

# For the particular ifdefs in patch 23, I will try to make it easy to
understand.

Quoted 9 lines
Quoted 7 lines

johannes

PS: actually having something like lkl-ops in UML would've been nice also for my "time-travel" work, since it neatly abstracts the timers out. I do wonder a bit about the overhead of jumping through function pointers all the time though, it may be worth rethinking that overall anyway!

Agreed. UML can also learn from LKL. :-)

Ditto; this is also my wish.

-- Hajime

_______________________________________________
linux-um mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-um

maintainer
#43 lore raw

Re: [RFC v4 02/25] um lkl: architecture skeleton for Linux kernel library

Hi,

Quoted 2 lines

I always wrote down the current plan that I have in my mind in the cover letter (patch 00); please take a look at the "Milestone" block.

Right, sorry.

Quoted 11 lines
Quoted 11 lines

This patches is a first step toward upstreaming *library mode* of Linux kernel, but we think we need to have several steps toward our goal, describing in the below.

1. Put LKL code under arch/um (arch/um/lkl), and build it in a separate way from UML.
2. Use UML driver implementations in LKL as a minimum set of patches
- Only support x86 sub architecture (dependency to UML drivers)
3. Support broader host supports
- add virtio device features

Quoted 6 lines
Quoted 3 lines

For the step 1, we put LKL as one of UMMODE in order to make less effort to integrate (make ARCH=um UMMODE=library). The modification to existing UML code is trying to be minimized.

The current step (1 in the milestone) tries to cover this goal:
splitting ARCH=um into UMMODE_KERNEL and UMMODE_LIB.

So maybe we're doing this backwards?

I mean ... you're trying to minimize the UML code changes, while I'm sort of arguing for maximizing them, to achieve a cleaner split.

In a sense, I think if this is to happen, then we're in it for the long haul. Meaning that we don't actually need all of this working immediately.

So I think conceptually we should answer the questions that I raised below first (basically a kind of "can it be done?" question), and then work towards that goal? IMHO.

Quoted 12 lines
Quoted 11 lines
Quoted 4 lines

1) We give up, and get ARCH=lkl, sharing just (some of) the drivers while moving them into the right drivers/somewhere/ place. Even that looks somewhat awkward looking at the later patches in this set, but seems like that at *least* should be done.

Yeah, this would be a goal.
UML and LKL are quite different but they should share at least their userspace drivers.
I also don't mind if we don't share every driver at the beginning but it should be
a feasible goal for the future.

Quoted 5 lines

Sharing drivers code is also included in this patchset, step 2 in the milestone.

I was thinking that implementing os_*() functions with lkl_host_ops would be the further goal (e.g., step 3 or 4).

Personally, I think this is backwards. That step is the actually *interesting* part, because if this turns out not to be possible, then we should pick option (1) instead of trying to do option (2), failing, and leaving the code a mess (at least personally I think that after this patchset, the code is kinda a mess with all the ifdefs, duplication, etc.) Yes, I know you're in this for the long haul, but still - it'd be a shame to have to do that.

So in a sense, I myself would actually prefer to have an LKL _without_ drivers, but integrated well with UML, over the one that you have now.

Quoted 10 lines
Quoted 10 lines
Quoted 9 lines

2) Ideally, instead, we actually unify: LKL grows support for userspace processes using UML infrastructure, the "in-kernel" IRQ mechanisms are unified, UML stuff moves into lkl-ops, and the UML binary basically becomes a user of the LKL library to start everything up. There may be some bits remaining that are just not interesting (e.g. some drivers you don't care about would continue to make direct calls to the user side instead of lkl-ops, and then they're just not compatible with lkl, only with the uml special case of lkl), but then things are clean.

Quoted 8 lines
Quoted 1 line

A few months ago I though this is doable but now I'm not so sure anymore.

For the part of (2) which Johannes pointed out (I mean the part "UML stuff moves into lkl-ops"), I become to think that implementing os_*() functions using lkl_host_ops would be also interesting if those re-implementation makes the glue code eliminated.

I'll work on that.

Don't go too fast :-)

I really think that this only makes sense if we can also share much of the other code, e.g. the interrupt processing, thread model, etc. If we just share the lkl ops underneath, and then end up implementing two IRQ models and all on top of those, IMHO we've won nothing.

So I (at least) really see it as a choice between these two options:

1) add LKL as arch/lkl/ and share the drivers, but not the arch code

2) really unify LKL and UML, and have them share the arch code, and make UML a special case of LKL, but not in the sense that it has vastly different arch code (like special interrupt handling, etc.)

Now, you allude to the fact that UML is pretty much x86 only, and perhaps that's a point where we can do (2) but only support userspace programs on x86, or such? I don't know where the host architecture actually comes in much in UML, and where that may or may not be the case in LKL.

Quoted 2 lines

For the other part of (2), I agree that your definition of the unification will be the best and final goal of this integration.

Fair, but the problem is that we have to decide *now* whether it's actually possible or not. If not, then IMHO it's a bad choice to even put LKL under arch/um/.

Quoted 2 lines

But, especially the support for UML userspace processes in LKL is not easy as far as I can see

OK, I'll bite - why not? I mean, what's different enough in LKL and UML to make this "not easy"?

I'm not trying to paint you into a corner here with that, I'm just trying to understand the innards of LKL better. I have a _bit_ of a grasp of the UML internals by now, but of course not LKL.

So where do they differ? Conceptually, they seem very similar, but the details actually are different.

But I have the same question on e.g. the IRQ model. I mean, OK, I understand that LKL started from a different base and all, but is it actually *necessary* for LKL to have a different IRQ model? Or is that "just" intertia?

Quoted 3 lines

Or the title of the cover letter is somehow overstatement: instead, "Minimum integration of UML with LKL" or something like this would be better ?

Heh, well, doesn't really matter?

But again, there are a few different aspects here:
- what's technically feasible
- what this patchset achieves
- where we want to be in the end

I think right now these are diverging enough that we can't even answer that last question, much less find the road to get there.

Quoted 9 lines

Since the patchset of LKL is relatively huge, I was trying to make a minimum patchset for the first step. Because having #ifdefs and glue code makes existing arch/um code untouched, I took this way to make the patchset as small as possible.

But if this is not the case, I will look for the other way and touch arch/um code to fit the current LKL implementation.

What do you think ?

I think that'd be fine, if indeed that's what we want to do.

I really think we're beating around the bush, and need to first figure out the technical differences between UML and LKL and decide between the options (1) and (2) above. Maybe there's a compromise there somewhere, where some small bits of code still _are_ different, but IMHO having two (IRQ, thread, memory) models, two host interfaces (lkl-ops vs. os_* functions), even two include/asm/ source trees (and so on) is not appropriate.

This may take some patches, and some experimentation. I'd leave drivers out of this initially - you should be able to test LKL with something simpler, right? The API surface is basically the syscall interface as functions, so you can start the library and call something simple initially? Though I guess you need some driver for the IRQ model to make sense, etc.

And like I said before, that decision will frame everything else. I really don't think we can make significant progress here without having decided whether this is possible.

Perhaps UML *can* become a "special case" of LKL, with a special API function (that's not part of the syscall surface) to "boot(cmdline)" or something. But if it can't, and has to remain as separated as the two are today, I would argue we're better off just not calling them the same architecture.

johannes

_______________________________________________
linux-um mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-um

#44 lore raw

Re: [RFC v4 02/25] um lkl: architecture skeleton for Linux kernel library

Hello,

# dropped an address from Cc as it's not reachable.
Quoted 11 lines · Johannes Berg
Quoted 6 lines
Quoted 3 lines

For the step 1, we put LKL as one of UMMODE in order to make less effort to integrate (make ARCH=um UMMODE=library). The modification to existing UML code is trying to be minimized.

The current step (1 in the milestone) tries to cover this goal:
splitting ARCH=um into UMMODE_KERNEL and UMMODE_LIB.

So maybe we're doing this backwards?

I mean ... you're trying to minimize the UML code changes, while I'm sort of arguing for maximizing them, to achieve a cleaner split.

I see the point.

Quoted 7 lines

In a sense, I think if this is to happen, then we're in it for the long haul. Meaning that we don't actually need all of this working immediately.

So I think conceptually we should answer the questions that I raised below first (basically a kind of "can it be done?" question), and then work towards that goal? IMHO.

okay, agree.

Quoted 26 lines
Quoted 12 lines
Quoted 11 lines
Quoted 4 lines

1) We give up, and get ARCH=lkl, sharing just (some of) the drivers while moving them into the right drivers/somewhere/ place. Even that looks somewhat awkward looking at the later patches in this set, but seems like that at *least* should be done.

Yeah, this would be a goal.
UML and LKL are quite different but they should share at least their userspace drivers.
I also don't mind if we don't share every driver at the beginning but it should be
a feasible goal for the future.

Quoted 5 lines

Sharing drivers code is also included in this patchset, step 2 in the milestone.

I was thinking that implementing os_*() functions with lkl_host_ops would be the further goal (e.g., step 3 or 4).

Personally, I think this is backwards. That step is the actually *interesting* part, because if this turns out not to be possible, then we should pick option (1) instead of trying to do option (2), failing, and leaving the code a mess (at least personally I think that after this patchset, the code is kinda a mess with all the ifdefs, duplication, etc.) Yes, I know you're in this for the long haul, but still - it'd be a shame to have to do that.

Quoted 2 lines

So in a sense, I myself would actually prefer to have an LKL _without_ drivers, but integrated well with UML, over the one that you have now.

LKL without drivers might be nothing, but let's see if this will end a clean, and minimal viable patchset.

Quoted 26 lines
Quoted 10 lines
Quoted 10 lines
Quoted 9 lines

2) Ideally, instead, we actually unify: LKL grows support for userspace processes using UML infrastructure, the "in-kernel" IRQ mechanisms are unified, UML stuff moves into lkl-ops, and the UML binary basically becomes a user of the LKL library to start everything up. There may be some bits remaining that are just not interesting (e.g. some drivers you don't care about would continue to make direct calls to the user side instead of lkl-ops, and then they're just not compatible with lkl, only with the uml special case of lkl), but then things are clean.

Quoted 8 lines
Quoted 1 line

A few months ago I though this is doable but now I'm not so sure anymore.

For the part of (2) which Johannes pointed out (I mean the part "UML stuff moves into lkl-ops"), I become to think that implementing os_*() functions using lkl_host_ops would be also interesting if those re-implementation makes the glue code eliminated.

I'll work on that.

Don't go too fast :-)

I really think that this only makes sense if we can also share much of the other code, e.g. the interrupt processing, thread model, etc. If we just share the lkl ops underneath, and then end up implementing two IRQ models and all on top of those, IMHO we've won nothing.

Quoted 7 lines

So I (at least) really see it as a choice between these two options:

1) add LKL as arch/lkl/ and share the drivers, but not the arch code

2) really unify LKL and UML, and have them share the arch code, and make UML a special case of LKL, but not in the sense that it has vastly different arch code (like special interrupt handling, etc.)

1) is not an option; we discussed this before not to have similar archs in different directories.

# Richard, correct me if I'm wrong.

2) may be doable, the followings would be the list of things what we need:
- unify the driver codes
- unify interrupt handling, need to be x86/linux-independent
- unify thread model (i.e., struct thread_info)
- unify code scheduling (e.g., __switch_to())
- unify memory management (mmu v.s. nommu)
- unify host interface (os_*() v.s. lkl_host_ops)
- support multiple syscall handlings (ptrace-based interception, direct func call, etc)
- (may still miss something..)

Those are actually the list of differences between UML and LKL. And each item is inter-related: interrupt handling depends on how the code/thread is executed, thread implementation interacts with memory management, etc.

Thus for instance, unifying IRQ mechanism may involve several points of the above list to be reorganized/reimplemented UML code to support the library mode. I thought this makes broader changes to UML, which I was trying to avoid in v4 patch.

Quoted 3 lines

Now, you allude to the fact that UML is pretty much x86 only, and perhaps that's a point where we can do (2) but only support userspace programs on x86, or such?

x86/linux only (LKL also has x86/win32 support).

Quoted 17 lines

I don't know where the host architecture
actually comes in much in UML, and where that may or may not be the case in LKL.

Quoted 2 lines

For the other part of (2), I agree that your definition of the unification will be the best and final goal of this integration.

Fair, but the problem is that we have to decide *now* whether it's actually possible or not. If not, then IMHO it's a bad choice to even put LKL under arch/um/.

Quoted 2 lines

But, especially the support for UML userspace processes in LKL is not easy as far as I can see

OK, I'll bite - why not? I mean, what's different enough in LKL and UML to make this "not easy"?

Quoted 7 lines

I'm not trying to paint you into a corner here with that, I'm just trying to understand the innards of LKL better. I have a _bit_ of a grasp of the UML internals by now, but of course not LKL.

So where do they differ? Conceptually, they seem very similar, but the details actually are different.

The differences (and also conflicts) between UML and LKL are pretty much the unification list of above 2).

The current LKL design assumes that there is only a single process/application (but can have multiple threads) in a single LKL kernel instance.

This assumption makes the design simple in several places of the kernel:

- no memory protection across multiple users/processes needed
- no address space separation between user- and kernel-code

as a result, userspace code can directly call syscalls as function calls.

This part is needed to expand more to support multiple processes running/spawned (as UML does), which I mentioned the current LKL doesn't have.

Quoted 4 lines

But I have the same question on e.g. the IRQ model. I mean, OK, I understand that LKL started from a different base and all, but is it actually *necessary* for LKL to have a different IRQ model? Or is that "just" intertia?

As for the interrupt handling, the currently LKL's interrupt is triggered by the function call (lkl_trigger_irq or equivalent) while UML's one is triggered by fd notification (epoll).
# LKL actually uses fd events but encapsulates into the lkl_host_ops.

So now we have three options for interrupt handling:
- use UML IRQ model in LKL (and UML)
- use LKL IRQ model in UML (and LKL)
- use two IRQ models (the current v4 patch)

I gave up to take the 1st approach as it drops host-independent characteristics of LKL, thus took 3rd approach. So to answer your question, yes, it was needed to have a different IRQ model.

But maybe I should try the 2nd approach as well to avoid duplication. I will conduct another experiment if this sounds the right direction.

Quoted 6 lines
Quoted 3 lines

Or the title of the cover letter is somehow overstatement: instead, "Minimum integration of UML with LKL" or something like this would be better ?

Heh, well, doesn't really matter?

Quoted 7 lines

But again, there are a few different aspects here:
- what's technically feasible
- what this patchset achieves
- where we want to be in the end

I think right now these are diverging enough that we can't even answer that last question, much less find the road to get there.

I think the option 2) is the final goal.

Quoted 20 lines
Quoted 9 lines

Since the patchset of LKL is relatively huge, I was trying to make a minimum patchset for the first step. Because having #ifdefs and glue code makes existing arch/um code untouched, I took this way to make the patchset as small as possible.

But if this is not the case, I will look for the other way and touch arch/um code to fit the current LKL implementation.

What do you think ?

I think that'd be fine, if indeed that's what we want to do.

I really think we're beating around the bush, and need to first figure out the technical differences between UML and LKL and decide between the options (1) and (2) above. Maybe there's a compromise there somewhere, where some small bits of code still _are_ different, but IMHO having two (IRQ, thread, memory) models, two host interfaces (lkl-ops vs. os_* functions), even two include/asm/ source trees (and so on) is not appropriate.

Quoted 6 lines

This may take some patches, and some experimentation. I'd leave drivers out of this initially - you should be able to test LKL with something simpler, right? The API surface is basically the syscall interface as functions, so you can start the library and call something simple initially? Though I guess you need some driver for the IRQ model to make sense, etc.

okay.

Quoted 9 lines

And like I said before, that decision will frame everything else. I really don't think we can make significant progress here without having decided whether this is possible.

Perhaps UML *can* become a "special case" of LKL, with a special API function (that's not part of the syscall surface) to "boot(cmdline)" or something. But if it can't, and has to remain as separated as the two are today, I would argue we're better off just not calling them the same architecture.

I agree with this if the unification has all completed.

Thanks,
-- Hajime

#45 lore raw

Re: [RFC v4 02/25] um lkl: architecture skeleton for Linux kernel library

<snip>

Quoted 4 lines
Quoted 4 lines

And like I said before, that decision will frame everything else. I really don't think we can make significant progress here without having decided whether this is possible.

Quoted 8 lines
Quoted 5 lines

Perhaps UML *can* become a "special case" of LKL, with a special API function (that's not part of the syscall surface) to "boot(cmdline)" or something. But if it can't, and has to remain as separated as the two are today, I would argue we're better off just not calling them the same architecture.

I agree with this if the unification has all completed.

I tought a lot about this and I agree with Johannes that if we want to unity UML and LKL we should start with the hard parts.

I am also starting to think that it is unlikely to be able to merge the two "nicely" and that we should probably turn this on its head and start with reworking UML towards the LKL features. We will end up re-implementing *some* of the LKL concepts from scratch in UML but I think at this point this is unavoidable.

Here are my thoughts on a very rough plan for doing that:

Milestone 1: LKL lib on top of UML
* Kernel - Host build split
Build UML as a relocatable object using the UML’s kernel linker script.
Move the ptrace and other well isolated os code out of arch/um to tools/um (or maybe start directly with tools/lkl?)
Use standard host toolchain to create a static library stripped of the ptrace code. Use standard host toolchain to build the main UML executable.
Add library init API that creates the UML kernel process and starts UML.
* System calls APIs
Add new system call interface dbased on fd irq.
Use the LKL scripts to export the required headers to create system calls APIs that use the UML system calls infrastructure. Keep the underlying host and driver operations (threads, irqs, etc.) as they are now in UML.
* Boot test
Port the LKL boot test to verify that we are able to programatically issue system calls.

Milestone 2: add virtio disk support
* Export asm/io.h operations to host/os. Create IO access operations and redirect them to weak os_ variants that use the current UML implementation.
* Add the LKL IO access layer including generic virtio handling and the virtio block device code.
* Port LKL disk test and disk apps (lklfuse, fs2tar, cptofs)

Milestone 3: new arch ports
* Abstrac the system call / IRQ mode the move the implementation to host
* Abstract the thread model and move the implementation to host
* Add LKL thread model and LKL ports

Thanks,
Tavi

#46 lore raw

RFC v4 3/25 um lkl: host interface

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

This patch introduces the host operations that define the interface between the LKL and the host. These operations must be provided either by a host library or by the application itself.

Cc Michael Zimmermann <[email protected]>
Cc Patrick Collins <[email protected]>
Cc Pierre-Hugues Husson <[email protected]>
Cc Yuan Liu <[email protected]>
---
 arch/um/lkl/include/asm/host_ops.h      | 10 ++++++++++
 arch/um/lkl/include/uapi/asm/host_ops.h | 26 +++++++++++++++++++++++++
 2 files changed, 36 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/host_ops.h
 create mode 100644 arch/um/lkl/include/uapi/asm/host_ops.h
arch/um/lkl/include/asm/host_ops.h new file +10 -0
new file mode 100644
index 000000000000..65850f394b79
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_HOST_OPS_H
+#define _ASM_LKL_HOST_OPS_H
+
+#include "irq.h"
+#include <uapi/asm/host_ops.h>
+
+extern struct lkl_host_operations *lkl_ops;
+
+#endif
arch/um/lkl/include/uapi/asm/host_ops.h new file +26 -0
new file mode 100644
index 000000000000..7cfb0a93e6a6
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_HOST_OPS_H
+#define _ASM_UAPI_LKL_HOST_OPS_H
+
+/* Defined in {posix,nt}-host.c */
+struct lkl_mutex;
+struct lkl_sem;
+struct lkl_tls_key;
+typedef unsigned long lkl_thread_t;
+struct lkl_jmp_buf {
+	unsigned long buf[32];
+};
+
+/**
+ * lkl_host_operations - host operations used by the Linux kernel
+ *
+ * These operations must be provided by a host library or by the application
+ * itself.
+ *
+ */
+struct lkl_host_operations {
+};
+
+void lkl_bug(const char *fmt, ...);
+
+#endif
Signature · 7 lines
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-um
#47 lore raw

RFC v4 4/25 um lkl: memory handling

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

LKL is a non MMU architecture and hence there is not much work left to do other than initializing the boot allocator and providing the page and page table definitions.

The backstore memory is allocated via a host operation and the memory size to be used is specified when the kernel is started, in the lkl_start_kernel call.

---
 arch/um/lkl/include/asm/page.h          | 14 ++++++
 arch/um/lkl/include/asm/pgtable.h       | 57 +++++++++++++++++++++
 arch/um/lkl/include/uapi/asm/host_ops.h |  5 ++
 arch/um/lkl/mm/bootmem.c                | 66 +++++++++++++++++++++++++
 4 files changed, 142 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/page.h
 create mode 100644 arch/um/lkl/include/asm/pgtable.h
 create mode 100644 arch/um/lkl/mm/bootmem.c
arch/um/lkl/include/asm/page.h new file +14 -0
new file mode 100644
index 000000000000..e77f3da22031
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_PAGE_H
+#define _ASM_LKL_PAGE_H
+
+#define CONFIG_KERNEL_RAM_BASE_ADDRESS memory_start
+
+#ifndef __ASSEMBLY__
+void free_mem(void);
+void bootmem_init(unsigned long mem_size);
+#endif
+
+#include <asm-generic/page.h>
+
+#endif /* _ASM_LKL_PAGE_H */
arch/um/lkl/include/asm/pgtable.h new file +57 -0

57 lines not on this page · raw message

arch/um/lkl/include/uapi/asm/host_ops.h +5 -0
index 7cfb0a93e6a6..6bbc94c120be 100644
@@ -17,8 +17,13 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @mem_alloc - allocate memory
+ * @mem_free - free memory
+ *
  */
 struct lkl_host_operations {
+	void *(*mem_alloc)(unsigned long mem);
+	void (*mem_free)(void *mem);
 };
 
 void lkl_bug(const char *fmt, ...);
arch/um/lkl/mm/bootmem.c new file +66 -0

66 lines not on this page · raw message

Signature · 7 lines
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-um
#48 lore raw

RFC v4 5/25 um lkl: kernel threads support

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

LKL does not support user processes but it must support kernel threads as part as the normal kernel work-flow. It uses host operations to create and terminate host threads that are going to run the kernel threads. It also uses semaphores to synchronize those threads and to allow the Linux kernel scheduler to control how the kernel threads run.

Each kernel thread runs in a host threads and has a host semaphore associated with it - the thread's scheduling semaphore. The semaphore counter is initialized to 0. The first thing a kernel thread does after getting spawned, before running any kernel code, is to perform a down operation to block the thread.

The kernel controls host threads scheduling by performing up and down operations on the scheduling semaphore. In __switch_context an up operation on the next thread is performed to wake up a blocked thread, and a down operation is performed on the prev thread to block it.

A thread is terminated by marking it in free_thread_info and performing an up operation on the scheduling semaphore at which point the marked thread will terminate itself.

---
 arch/um/lkl/include/asm/thread_info.h   |  70 ++++++++
 arch/um/lkl/include/uapi/asm/host_ops.h |  55 ++++++
 arch/um/lkl/kernel/cpu.c                | 223 +++++++++++++++++++++++
 arch/um/lkl/kernel/threads.c            | 227 ++++++++++++++++++++++++
 4 files changed, 575 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/thread_info.h
 create mode 100644 arch/um/lkl/kernel/cpu.c
 create mode 100644 arch/um/lkl/kernel/threads.c
arch/um/lkl/include/asm/thread_info.h new file +70 -0
new file mode 100644
index 000000000000..da4e75fc7b10
@@ -0,0 +1,70 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_THREAD_INFO_H
+#define _ASM_LKL_THREAD_INFO_H
+
+#define THREAD_SIZE	       (4096)
+
+#ifndef __ASSEMBLY__
+#include <asm/types.h>
+#include <asm/processor.h>
+#include <asm/host_ops.h>
+
+typedef struct {
+	unsigned long seg;
+} mm_segment_t;
+
+struct thread_info {
+	struct task_struct *task;
+	unsigned long flags;
+	int preempt_count;
+	mm_segment_t addr_limit;
+	struct lkl_sem *sched_sem;
+	struct lkl_jmp_buf sched_jb;
+	bool dead;
+	lkl_thread_t tid;
+	struct task_struct *prev_sched;
+	unsigned long stackend;
+};
+
+#define INIT_THREAD_INFO(tsk)				\
+{							\
+	.task		= &tsk,				\
+	.preempt_count	= INIT_PREEMPT_COUNT,		\
+	.flags		= 0,				\
+	.addr_limit	= KERNEL_DS,			\
+}
+
+/* how to get the thread information struct from C */
+extern struct thread_info *_current_thread_info;
+static inline struct thread_info *current_thread_info(void)
+{
+	return _current_thread_info;
+}
+
+/* thread information allocation */
+unsigned long *alloc_thread_stack_node(struct task_struct *, int node);
+void free_thread_stack(struct task_struct *tsk);
+
+void threads_init(void);
+void threads_cleanup(void);
+
+#define TIF_SYSCALL_TRACE		0
+#define TIF_NOTIFY_RESUME		1
+#define TIF_SIGPENDING			2
+#define TIF_NEED_RESCHED		3
+#define TIF_RESTORE_SIGMASK		4
+#define TIF_MEMDIE			5
+#define TIF_NOHZ			6
+#define TIF_SCHED_JB			7
+#define TIF_HOST_THREAD			8
+
+#define __HAVE_THREAD_FUNCTIONS
+
+#define task_thread_info(task)	((struct thread_info *)(task)->stack)
+#define task_stack_page(task)	((task)->stack)
+void setup_thread_stack(struct task_struct *p, struct task_struct *org);
+#define end_of_stack(p) (&task_thread_info(p)->stackend)
+
+#endif /* __ASSEMBLY__ */
+
+#endif
arch/um/lkl/include/uapi/asm/host_ops.h +55 -0

70 lines not on this page · raw message

arch/um/lkl/kernel/cpu.c new file +223 -0

223 lines not on this page · raw message

arch/um/lkl/kernel/threads.c new file +227 -0

227 lines not on this page · raw message

Signature · 1 line
2.21.0 (Apple Git-122.2)
#49 lore raw

RFC v4 6/25 um lkl: interrupt support

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

Add APIs that allows the host to reserve and free and interrupt number and also to trigger an interrupt.

The trigger operation will simply store the interrupt data in queue. The interrupt handler is run later, at the first opportunity it has to avoid races with any kernel threads.

Currently, interrupts are run on the first interrupt enable operation if interrupts are disabled and if we are not already in interrupt context.

When triggering an interrupt, it uses GCC's built-in functions for atomic memory access to synchronize and simple boolean flags.

---
 arch/um/lkl/include/asm/irq.h             |  13 ++
 arch/um/lkl/include/uapi/asm/irq.h        |  36 ++++
 arch/um/lkl/include/uapi/asm/sigcontext.h |  16 ++
 arch/um/lkl/kernel/irq.c                  | 190 ++++++++++++++++++++++
 4 files changed, 255 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/irq.h
 create mode 100644 arch/um/lkl/include/uapi/asm/irq.h
 create mode 100644 arch/um/lkl/include/uapi/asm/sigcontext.h
 create mode 100644 arch/um/lkl/kernel/irq.c
arch/um/lkl/include/asm/irq.h new file +13 -0
new file mode 100644
index 000000000000..948fc54cb76c
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_IRQ_H
+#define _ASM_LKL_IRQ_H
+
+#define IRQ_STATUS_BITS		(sizeof(long) * 8)
+#define NR_IRQS			((int)(IRQ_STATUS_BITS * IRQ_STATUS_BITS))
+
+void run_irqs(void);
+void set_irq_pending(int irq);
+
+#include <uapi/asm/irq.h>
+
+#endif
arch/um/lkl/include/uapi/asm/irq.h new file +36 -0
new file mode 100644
index 000000000000..666628b233eb
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_IRQ_H
+#define _ASM_UAPI_LKL_IRQ_H
+
+/**
+ * lkl_trigger_irq - generate an interrupt
+ *
+ * This function is used by the device host side to signal its Linux counterpart
+ * that some event happened.
+ *
+ * @irq - the irq number to signal
+ */
+int lkl_trigger_irq(int irq);
+
+/**
+ * lkl_get_free_irq - find and reserve a free IRQ number
+ *
+ * This function is called by the host device code to find an unused IRQ number
+ * and reserved it for its own use.
+ *
+ * @user - a string to identify the user
+ * @returns - and irq number that can be used by request_irq or an negative
+ * value in case of an error
+ */
+int lkl_get_free_irq(const char *user);
+
+/**
+ * lkl_put_irq - release an IRQ number previously obtained with lkl_get_free_irq
+ *
+ * @irq - irq number to release
+ * @user - string identifying the user; should be the same as the one passed to
+ * lkl_get_free_irq when the irq number was obtained
+ */
+void lkl_put_irq(int irq, const char *name);
+
+#endif
arch/um/lkl/include/uapi/asm/sigcontext.h new file +16 -0
new file mode 100644
index 000000000000..2f4848843d1d
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_SIGCONTEXT_H
+#define _ASM_UAPI_LKL_SIGCONTEXT_H
+
+#include <asm/ptrace.h>
+
+struct pt_regs {
+	void *irq_data;
+};
+
+struct sigcontext {
+	struct pt_regs regs;
+	unsigned long oldmask;
+};
+
+#endif
arch/um/lkl/kernel/irq.c new file +190 -0

190 lines not on this page · raw message

Signature · 1 line
2.21.0 (Apple Git-122.2)
#50 lore raw

RFC v4 7/25 um lkl: system call interface and application API

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

The LKL application API is based on the kernel system call interface in order to offer a stable API to applications. Note that we can't offer the full Linux system call interface due to LKL limitations such as lack of virtual memory, signal, user processes, etc.

The host is using the LKL interrupt mechanism (lkl_trigger_irq) to initiate a system call. The system call is executed in the context of the init process.

To avoid collisions between the Linux API and the LKL API (e.g. struct stat, MKNOD, etc.) we use a python script to modify the user headers and to prefix all of the global symbols (structures, typedefs, defines) with LKL, lkl, _LKL, _lkl, __LKL or __lkl.

Cc Conrad Meyer <[email protected]>
Cc Jens Staal <[email protected]>
Cc Lai Jiangshan <[email protected]>
Cc Luca Dariz <[email protected]>
Cc Michael Zimmermann <[email protected]>
Cc Patrick Collins <[email protected]>
Cc Pierre-Hugues Husson <[email protected]>
Cc Yuan Liu <[email protected]>
---
 arch/um/lkl/include/asm/unistd.h        |  27 ++++
 arch/um/lkl/include/uapi/asm/host_ops.h |  14 ++
 arch/um/lkl/include/uapi/asm/unistd.h   |  13 ++
 arch/um/lkl/kernel/syscalls.c           | 192 +++++++++++++++++++++++
 arch/um/lkl/kernel/syscalls_32.c        | 159 +++++++++++++++++++
 arch/um/lkl/scripts/headers_install.py  | 196 ++++++++++++++++++++++++
 6 files changed, 601 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/unistd.h
 create mode 100644 arch/um/lkl/include/uapi/asm/unistd.h
 create mode 100644 arch/um/lkl/kernel/syscalls.c
 create mode 100644 arch/um/lkl/kernel/syscalls_32.c
 create mode 100755 arch/um/lkl/scripts/headers_install.py
arch/um/lkl/include/asm/unistd.h new file +27 -0
new file mode 100644
index 000000000000..ca0eac0f2827
@@ -0,0 +1,27 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <uapi/asm/unistd.h>
+
+#define __SC_ASCII(t, a) #t "," #a
+
+#define __ASCII_MAP0(m, ...)
+#define __ASCII_MAP1(m, t, a) m(t, a)
+#define __ASCII_MAP2(m, t, a, ...) m(t, a) "," __ASCII_MAP1(m, __VA_ARGS__)
+#define __ASCII_MAP3(m, t, a, ...) m(t, a) "," __ASCII_MAP2(m, __VA_ARGS__)
+#define __ASCII_MAP4(m, t, a, ...) m(t, a) "," __ASCII_MAP3(m, __VA_ARGS__)
+#define __ASCII_MAP5(m, t, a, ...) m(t, a) "," __ASCII_MAP4(m, __VA_ARGS__)
+#define __ASCII_MAP6(m, t, a, ...) m(t, a) "," __ASCII_MAP5(m, __VA_ARGS__)
+#define __ASCII_MAP(n, ...) __ASCII_MAP##n(__VA_ARGS__)
+
+#ifdef __MINGW32__
+#define SECTION_ATTRS "n0"
+#else
+#define SECTION_ATTRS "a"
+#endif
+
+#define __SYSCALL_DEFINE_ARCH(x, name, ...)				\
+	asm(".section .syscall_defs,\"" SECTION_ATTRS "\"\n"		\
+	    ".ascii \"#ifdef __NR" #name "\\n\"\n"			\
+	    ".ascii \"SYSCALL_DEFINE" #x "(" #name ","			\
+	    __ASCII_MAP(x, __SC_ASCII, __VA_ARGS__) ")\\n\"\n"		\
+	    ".ascii \"#endif\\n\"\n"					\
+	    ".section .text\n");
arch/um/lkl/include/uapi/asm/host_ops.h +14 -0
index 19924fc7c718..1c839d7139f8 100644
@@ -36,6 +36,15 @@ struct lkl_jmp_buf {
  * @thread_join - wait for the given thread to terminate. Returns 0
  * for success, -1 otherwise
  *
+ * @tls_alloc - allocate a thread local storage key; returns 0 if successful; if
+ * destructor is not NULL it will be called when a thread terminates with its
+ * argument set to the current thread local storage value
+ * @tls_free - frees a thread local storage key; returns 0 if successful
+ * @tls_set - associate data to the thread local storage key; returns 0 if
+ * successful
+ * @tls_get - return data associated with the thread local storage key or NULL
+ * on error
+ *
  * @mem_alloc - allocate memory
  * @mem_free - free memory
  *
@@ -71,6 +80,11 @@ struct lkl_host_operations {
 	lkl_thread_t (*thread_self)(void);
 	int (*thread_equal)(lkl_thread_t a, lkl_thread_t b);
 
+	struct lkl_tls_key *(*tls_alloc)(void (*destructor)(void *));
+	void (*tls_free)(struct lkl_tls_key *key);
+	int (*tls_set)(struct lkl_tls_key *key, void *data);
+	void *(*tls_get)(struct lkl_tls_key *key);
+
 	void *(*mem_alloc)(unsigned long mem);
 	void (*mem_free)(void *mem);
 
arch/um/lkl/include/uapi/asm/unistd.h new file +13 -0
new file mode 100644
index 000000000000..7670e93f0152
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#define __ARCH_WANT_RENAMEAT
+#define __ARCH_WANT_NEW_STAT
+#define __ARCH_WANT_SET_GET_RLIMIT
+#define __ARCH_WANT_TIME32_SYSCALLS
+
+#include <asm/bitsperlong.h>
+
+#if __BITS_PER_LONG == 64
+#define __ARCH_WANT_SYS_NEWFSTATAT
+#endif
+
+#include <asm-generic/unistd.h>
arch/um/lkl/kernel/syscalls.c new file +192 -0

192 lines not on this page · raw message

arch/um/lkl/kernel/syscalls_32.c new file +159 -0

159 lines not on this page · raw message

arch/um/lkl/scripts/headers_install.py new file +196 -0

196 lines not on this page · raw message

Signature · 1 line
2.21.0 (Apple Git-122.2)
#51 lore raw

RFC v4 8/25 um lkl: timers, time and delay support

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

Clockevent driver based on host timer operations and clocksource driver and udelay support based on host time operations.

---
 arch/um/lkl/include/uapi/asm/host_ops.h |  13 +++
 arch/um/lkl/kernel/time.c               | 145 ++++++++++++++++++++++++
 2 files changed, 158 insertions(+)
 create mode 100644 arch/um/lkl/kernel/time.c
arch/um/lkl/include/uapi/asm/host_ops.h +13 -0
index 1c839d7139f8..c9f77dd7fbe7 100644
@@ -48,6 +48,13 @@ struct lkl_jmp_buf {
  * @mem_alloc - allocate memory
  * @mem_free - free memory
  *
+ * @timer_create - allocate a host timer that runs fn(arg) when the timer
+ * fires.
+ * @timer_free - disarms and free the timer
+ * @timer_set_oneshot - arm the timer to fire once, after delta ns.
+ * @timer_set_periodic - arm the timer to fire periodically, with a period of
+ * delta ns.
+ *
  * @gettid - returns the host thread id of the caller, which need not
  * be the same as the handle returned by thread_create
  *
@@ -88,6 +95,12 @@ struct lkl_host_operations {
 	void *(*mem_alloc)(unsigned long mem);
 	void (*mem_free)(void *mem);
 
+	unsigned long long (*time)(void);
+
+	void *(*timer_alloc)(void (*fn)(void *), void *arg);
+	int (*timer_set_oneshot)(void *timer, unsigned long delta);
+	void (*timer_free)(void *timer);
+
 	long (*gettid)(void);
 
 	void (*jmp_buf_set)(struct lkl_jmp_buf *jmpb, void (*f)(void));
arch/um/lkl/kernel/time.c new file +145 -0

145 lines not on this page · raw message

Signature · 1 line
2.21.0 (Apple Git-122.2)
#52 lore raw

RFC v4 9/25 um lkl: basic kernel console support

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

This patch adds a basic structure of console support in kernel. Write operations are deferred to the host print operation.

---
 arch/um/lkl/include/uapi/asm/host_ops.h |  4 +++
 arch/um/lkl/kernel/console.c            | 42 +++++++++++++++++++++++++
 2 files changed, 46 insertions(+)
 create mode 100644 arch/um/lkl/kernel/console.c
arch/um/lkl/include/uapi/asm/host_ops.h +4 -0
index c9f77dd7fbe7..986340ba9d8d 100644
@@ -17,6 +17,8 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @print - optional operation that receives console messages
+ *
  * @sem_alloc - allocate a host semaphore an initialize it to count
  * @sem_free - free a host semaphore
  * @sem_up - perform an up operation on the semaphore
@@ -70,6 +72,8 @@ struct lkl_jmp_buf {
  * @jmp_buf_longjmp - perform a jump back to the saved jump buffer
  */
 struct lkl_host_operations {
+	void (*print)(const char *str, int len);
+
 	struct lkl_sem *(*sem_alloc)(int count);
 	void (*sem_free)(struct lkl_sem *sem);
 	void (*sem_up)(struct lkl_sem *sem);
arch/um/lkl/kernel/console.c new file +42 -0

42 lines not on this page · raw message

Signature · 1 line
2.21.0 (Apple Git-122.2)
#53 lore raw

RFC v4 10/25 um lkl: initialization and cleanup

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

Add the lkl_start_kernel and lkl_sys_halt APIs that start and respectively stops the Linux kernel.

lkl_start_kernel creates a separate threads that will run the initial and idle kernel thread. It waits for the kernel to complete initialization before returning, to avoid races with system calls issues by the host application.

During the setup phase, we create "/init" in initial ramfs root filesystem to avoid mounting the "real" rootfs since ramfs is good enough for now.

lkl_sys_halt will shutdown the kernel, terminate all threads and free all host resources used by the kernel before returning.

This patch also introduces idle CPU handling since it is closely related to the shutdown process. A host semaphore is used to wait for new interrupts when the kernel switches the CPU to idle to avoid wasting host CPU cycles. When the kernel is shutdown we terminate the idle thread at the first CPU idle event.

Cc Michael Zimmermann <[email protected]>
Cc Patrick Collins <[email protected]>
Cc Yuan Liu <[email protected]>
---
 arch/um/lkl/include/asm/setup.h         |   7 +
 arch/um/lkl/include/uapi/asm/host_ops.h |  20 +++
 arch/um/lkl/kernel/setup.c              | 189 ++++++++++++++++++++++++
 3 files changed, 216 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/setup.h
 create mode 100644 arch/um/lkl/kernel/setup.c
arch/um/lkl/include/asm/setup.h new file +7 -0
new file mode 100644
index 000000000000..b40955208cc6
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_SETUP_H
+#define _ASM_LKL_SETUP_H
+
+#define COMMAND_LINE_SIZE 4096
+
+#endif
arch/um/lkl/include/uapi/asm/host_ops.h +20 -0
index 986340ba9d8d..fe4382c3050a 100644
@@ -19,6 +19,8 @@ struct lkl_jmp_buf {
  *
  * @print - optional operation that receives console messages
  *
+ * @panic - called during a kernel panic
+ *
  * @sem_alloc - allocate a host semaphore an initialize it to count
  * @sem_free - free a host semaphore
  * @sem_up - perform an up operation on the semaphore
@@ -73,6 +75,7 @@ struct lkl_jmp_buf {
  */
 struct lkl_host_operations {
 	void (*print)(const char *str, int len);
+	void (*panic)(void);
 
 	struct lkl_sem *(*sem_alloc)(int count);
 	void (*sem_free)(struct lkl_sem *sem);
@@ -111,6 +114,23 @@ struct lkl_host_operations {
 	void (*jmp_buf_longjmp)(struct lkl_jmp_buf *jmpb, int val);
 };
 
+/**
+ * lkl_start_kernel - registers the host operations and starts the kernel
+ *
+ * The function returns only after the kernel is shutdown with lkl_sys_halt.
+ *
+ * @lkl_ops - pointer to host operations
+ * @cmd_line - format for command line string that is going to be used to
+ * generate the Linux kernel command line
+ */
+int lkl_start_kernel(struct lkl_host_operations *lkl_ops, const char *cmd_line,
+		     ...);
+
+/**
+ * lkl_is_running - returns 1 if the kernel is currently running
+ */
+int lkl_is_running(void);
+
 int lkl_printf(const char *fmt, ...);
 void lkl_bug(const char *fmt, ...);
 
arch/um/lkl/kernel/setup.c new file +189 -0

189 lines not on this page · raw message

Signature · 1 line
2.21.0 (Apple Git-122.2)
#54 lore raw

RFC v4 11/25 um lkl: plug in the build system

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

Basic Makefiles for building LKL. Add a new architecture specific target for installing the resulting library files and headers.

To make LKL binaries build, UML introduced an additional option, UMMODE variable, to switch the output file of build: kernel (default), or library (LKL). Those modes are not able to be ON at the same time.

To build on library mode, users do the following:

make defconfig ARCH=um UMMODE=library
make ARCH=um UMMODE=library

---
 arch/um/Kconfig             | 13 +++++++
 arch/um/Makefile            | 19 ++++++++++-
 arch/um/Makefile.um         | 10 ++++++
 arch/um/lkl/Kconfig         | 31 +++++++++++++++--
 arch/um/lkl/Makefile        | 67 +++++++++++++++++++++++++++++++++++++
 arch/um/lkl/auto.conf       |  1 +
 arch/um/lkl/kernel/Makefile |  4 +++
 arch/um/lkl/mm/Makefile     |  1 +
 8 files changed, 143 insertions(+), 3 deletions(-)
 create mode 100644 arch/um/Makefile.um
 create mode 100644 arch/um/lkl/Makefile
 create mode 100644 arch/um/lkl/auto.conf
 create mode 100644 arch/um/lkl/kernel/Makefile
 create mode 100644 arch/um/lkl/mm/Makefile
arch/um/Kconfig +13 -0
index 96ab7026b037..17a6bc25b615 100644
@@ -5,6 +5,10 @@ menu "UML-specific options"
 config UML
 	bool
 	default y
+
+config UMMODE_KERN
+	bool "UML mode: kernel mode"
+	default y if "$(UMMODE)" = "kernel"
 	select ARCH_HAS_KCOV
 	select ARCH_NO_PREEMPT
 	select HAVE_ARCH_AUDITSYSCALL
@@ -20,7 +24,12 @@ config UML
 	select GENERIC_CLOCKEVENTS
 	select HAVE_GCC_PLUGINS
 	select TTY # Needed for line.c
+        help
+	  This mode switches a mode to build a regular kernel executable
+          of UML.
+
 
+if UMMODE_KERN
 config MMU
 	bool
 	default y
@@ -207,6 +216,10 @@ config UML_TIME_TRAVEL_SUPPORT
 
 	  It is safe to say Y, but you probably don't need this.
 
+endif #UMMODE_KERN
+
 endmenu
 
 source "arch/um/drivers/Kconfig"
+
+source "arch/um/lkl/Kconfig"
arch/um/Makefile +18 -1
index d2daa206872d..f2a537f700c2 100644
@@ -1,3 +1,4 @@
+# SPDX-License-Identifier: GPL-2.0
 #
 # This file is included by the global makefile so that you can add your own
 # architecture-specific flags and dependencies.
@@ -44,7 +45,7 @@ HOST_DIR := arch/$(HEADER_ARCH)
 include $(ARCH_DIR)/Makefile-skas
 include $(HOST_DIR)/Makefile.um
 
-core-y += $(HOST_DIR)/um/
+
 
 SHARED_HEADERS	:= $(ARCH_DIR)/include/shared
 ARCH_INCLUDE	:= -I$(srctree)/$(SHARED_HEADERS)
@@ -144,5 +145,21 @@ CLEAN_FILES += linux x.i gmon.out
 archclean:
 	@find . \( -name '*.bb' -o -name '*.bbg' -o -name '*.da' \
 		-o -name '*.gcov' \) -type f -print | xargs rm -f
+	$(Q)rm -rf $(srctree)/$(LKL_DIR)/include/generated
 
 export HEADER_ARCH SUBARCH USER_CFLAGS CFLAGS_NO_HARDENING OS DEV_NULL_PATH
+
+
+
+# SPDX-License-Identifier: GPL-2.0
+# select mode of UML build
+UMMODE ?= kernel
+LKL_DIR := $(ARCH_DIR)/lkl
+
+ifeq ($(UMMODE),kernel)
+	include $(ARCH_DIR)/Makefile.um
+else ifeq ($(UMMODE),library)
+	include $(ARCH_DIR)/lkl/Makefile
+endif
+
+export UMMODE LKL_DIR
arch/um/Makefile.um new file +10 -0
new file mode 100644
index 000000000000..be7ee4d1adde
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# This file is included by the global makefile so that you can add your own
+# architecture-specific flags and dependencies.
+#
+# Copyright (C) 2002 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
+# Licensed under the GPL
+#
+
+core-y += $(HOST_DIR)/um/
arch/um/lkl/Kconfig +29 -2
index f7e8d707a12b..c72c40226509 100644
@@ -1,6 +1,25 @@
 # SPDX-License-Identifier: GPL-2.0
 
-config UML_LKL
+menu "LKL-specific options"
+
+config UML
+	bool
+	default y
+
+config UMMODE_LIB
+	bool "UML mode: library mode"
+	depends on !UMMODE_KERN
+	select LKL
+	default y if "$(UMMODE)" = "library"
+	help
+	  This mode switches a mode to build a library of UML (Linux
+	  Kernel Library/LKL).  This switch is exclusive to "kernel mode"
+	  of UML, which is traditional mode of UML.
+
+	  For more detail about LKL, see
+	  <file:Documentation/virt/uml/lkl.txt>.
+
+config LKL
        def_bool y
        depends on !SMP && !MMU && !COREDUMP && !SECCOMP && !UPROBES && !COMPAT && !USER_RETURN_NOTIFIER
        select ARCH_THREAD_STACK_ALLOCATOR
@@ -29,6 +48,12 @@ config OUTPUT_FORMAT
 config ARCH_DMA_ADDR_T_64BIT
        def_bool 64BIT
 
+config X86_64
+       def_bool y if "$(OUTPUT_FORMAT)" = "elf64-x86-64"
+
+config X86_32
+       def_bool y if "$(OUTPUT_FORMAT)" = "elf32-i386"
+
 config 64BIT
        def_bool n
 
@@ -39,7 +64,7 @@ config BIG_ENDIAN
        def_bool n
 
 config GENERIC_CSUM
-       def_bool y
+       def_bool LKL
 
 config GENERIC_HWEIGHT
        def_bool y
@@ -54,3 +79,5 @@ config RWSEM_GENERIC_SPINLOCK
 config HZ
         int
         default 100
+
+endmenu
arch/um/lkl/Makefile new file +67 -0

67 lines not on this page · raw message

arch/um/lkl/auto.conf new file +1 -0
new file mode 100644
index 000000000000..4bfd65a02d73
@@ -0,0 +1 @@
+export OUTPUT_FORMAT=$(shell $(LD) -r -print-output-format)
arch/um/lkl/kernel/Makefile new file +4 -0
new file mode 100644
index 000000000000..ef489f2f7176
@@ -0,0 +1,4 @@
+extra-y := vmlinux.lds
+
+obj-y = setup.o threads.o irq.o time.o syscalls.o misc.o console.o \
+	syscalls_32.o cpu.o
arch/um/lkl/mm/Makefile new file +1 -0
new file mode 100644
index 000000000000..2af6e3051897
@@ -0,0 +1 @@
+obj-y = bootmem.o
Signature · 1 line
2.21.0 (Apple Git-122.2)
#55 lore raw

RFC v4 12/25 lkl tools: skeleton for host side library

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

This patch adds the skeleton for the host library.

The host library is implementing the host operations needed by LKL and is split into host dependent (depends on a specific host, e.g. POSIX hosts) and host independent parts (will work on all supported hosts).

Cc Ben Wolsieffer <[email protected]>
Cc Conrad Meyer <[email protected]>
Cc H.K. Jerry Chu <[email protected]>
Cc Luca Dariz <[email protected]>
Cc Mark Stillwell <[email protected]>
Cc Michael Zimmermann <[email protected]>
Cc Motomu Utsumi <[email protected]>
Cc Patrick Collins <[email protected]>
Cc Petros Angelatos <[email protected]>
Cc Thomas Liebetraut <[email protected]>
Cc Xiao Jia <[email protected]>
Cc Yuan Liu <[email protected]>
---
 tools/lkl/.gitignore         |   4 +
 tools/lkl/Build              |   0
 tools/lkl/Makefile           | 125 ++++++++++++
 tools/lkl/Makefile.autoconf  |  65 +++++++
 tools/lkl/Targets            |   3 +
 tools/lkl/include/.gitignore |   1 +
 tools/lkl/include/lkl.h      | 358 +++++++++++++++++++++++++++++++++++
 tools/lkl/include/lkl_host.h |  19 ++
 tools/lkl/lib/.gitignore     |   3 +
 tools/lkl/lib/Build          |   1 +
 10 files changed, 579 insertions(+)
 create mode 100644 tools/lkl/.gitignore
 create mode 100644 tools/lkl/Build
 create mode 100644 tools/lkl/Makefile
 create mode 100644 tools/lkl/Makefile.autoconf
 create mode 100644 tools/lkl/Targets
 create mode 100644 tools/lkl/include/.gitignore
 create mode 100644 tools/lkl/include/lkl.h
 create mode 100644 tools/lkl/include/lkl_host.h
 create mode 100644 tools/lkl/lib/.gitignore
 create mode 100644 tools/lkl/lib/Build
tools/lkl/.gitignore new file +4 -0
new file mode 100644
index 000000000000..1aed58bfe171
@@ -0,0 +1,4 @@
+Makefile.conf
+include/lkl_autoconf.h
+tests/autoconf.sh
+bin/stat
tools/lkl/Build new file +0 -0
new file mode 100644
index 000000000000..e69de29bb2d1
tools/lkl/Makefile new file +125 -0

125 lines not on this page · raw message

tools/lkl/Makefile.autoconf new file +65 -0

65 lines not on this page · raw message

tools/lkl/Targets new file +3 -0
new file mode 100644
index 000000000000..24c985e64638
@@ -0,0 +1,3 @@
+libs-y += lib/liblkl
+
+
tools/lkl/include/.gitignore new file +1 -0
new file mode 100644
index 000000000000..c41a463c898d
@@ -0,0 +1 @@
+lkl/
\ No newline at end of file
tools/lkl/include/lkl.h new file +358 -0

358 lines not on this page · raw message

tools/lkl/include/lkl_host.h new file +19 -0
new file mode 100644
index 000000000000..b5f96096fe69
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_HOST_H
+#define _LKL_HOST_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <lkl/asm/host_ops.h>
+#include <lkl.h>
+
+extern struct lkl_host_operations lkl_host_ops;
+
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
tools/lkl/lib/.gitignore new file +3 -0
new file mode 100644
index 000000000000..427ae0273fdd
@@ -0,0 +1,3 @@
+lkl.o
+liblkl.a
+
tools/lkl/lib/Build new file +1 -0
new file mode 100644
index 000000000000..8b137891791f
@@ -0,0 +1 @@
+
Signature · 1 line
2.21.0 (Apple Git-122.2)
#56 lore raw

RFC v4 13/25 lkl tools: host lib: add utilities functions

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

Add basic utility functions for getting a string from a kernel error code and a fprintf like function that uses the host print operation. The latter is useful for informing the user about errors that occur in the host library.

Other configuration and debug utilities are also added.

Cc Conrad Meyer <[email protected]>
Cc Patrick Collins <[email protected]>
Cc Yuan Liu <[email protected]>
Cc Motomu Utsumi <[email protected]>
---
 tools/lkl/include/lkl_config.h |  61 +++
 tools/lkl/include/lkl_host.h   |   7 +
 tools/lkl/lib/Build            |   7 +
 tools/lkl/lib/config.c         | 744 +++++++++++++++++++++++++++++++++
 tools/lkl/lib/dbg.c            | 300 +++++++++++++
 tools/lkl/lib/dbg_handler.c    |  44 ++
 tools/lkl/lib/endian.h         |  31 ++
 tools/lkl/lib/jmp_buf.c        |  14 +
 tools/lkl/lib/jmp_buf.h        |   8 +
 tools/lkl/lib/utils.c          | 266 ++++++++++++
 10 files changed, 1482 insertions(+)
 create mode 100644 tools/lkl/include/lkl_config.h
 create mode 100644 tools/lkl/lib/config.c
 create mode 100644 tools/lkl/lib/dbg.c
 create mode 100644 tools/lkl/lib/dbg_handler.c
 create mode 100644 tools/lkl/lib/endian.h
 create mode 100644 tools/lkl/lib/jmp_buf.c
 create mode 100644 tools/lkl/lib/jmp_buf.h
 create mode 100644 tools/lkl/lib/utils.c
tools/lkl/include/lkl_config.h new file +61 -0
new file mode 100644
index 000000000000..d3edf8b414cf
@@ -0,0 +1,61 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_LIB_CONFIG_H
+#define _LKL_LIB_CONFIG_H
+
+#define LKL_CONFIG_JSON_TOKEN_MAX 300
+
+struct lkl_config_iface {
+	struct lkl_config_iface *next;
+	struct lkl_netdev *nd;
+
+	/* OBSOLETE: should use IFTYPE and IFPARAMS */
+	char *iftap;
+	char *iftype;
+	char *ifparams;
+	char *ifmtu_str;
+	char *ifip;
+	char *ifipv6;
+	char *ifgateway;
+	char *ifgateway6;
+	char *ifmac_str;
+	char *ifnetmask_len;
+	char *ifnetmask6_len;
+	char *ifoffload_str;
+	char *ifneigh_entries;
+	char *ifqdisc_entries;
+};
+
+struct lkl_config {
+	int ifnum;
+	struct lkl_config_iface *ifaces;
+
+	char *gateway;
+	char *gateway6;
+	char *debug;
+	char *mount;
+	/* single_cpu mode:
+	 * 0: Don't pin to single CPU (default).
+	 * 1: Pin only LKL kernel threads to single CPU.
+	 * 2: Pin all LKL threads to single CPU including all LKL kernel threads
+	 * and device polling threads. Avoid this mode if having busy polling
+	 * threads.
+	 *
+	 * mode 2 can achieve better TCP_RR but worse TCP_STREAM than mode 1.
+	 * You should choose the best for your application and virtio device
+	 * type.
+	 */
+	char *single_cpu;
+	char *sysctls;
+	char *boot_cmdline;
+	char *dump;
+	char *delay_main;
+};
+
+int lkl_load_config_json(struct lkl_config *cfg, char *jstr);
+int lkl_load_config_env(struct lkl_config *cfg);
+void lkl_show_config(struct lkl_config *cfg);
+int lkl_load_config_pre(struct lkl_config *cfg);
+int lkl_load_config_post(struct lkl_config *cfg);
+int lkl_unload_config(struct lkl_config *cfg);
+
+#endif /* _LKL_LIB_CONFIG_H */
tools/lkl/include/lkl_host.h +7 -0
index b5f96096fe69..85e80eb4ad0d 100644
@@ -11,6 +11,13 @@ extern "C" {
 
 extern struct lkl_host_operations lkl_host_ops;
 
+/**
+ * lkl_printf - print a message via the host print operation
+ *
+ * @fmt: printf like format string
+ */
+int lkl_printf(const char *fmt, ...);
+
 
 #ifdef __cplusplus
 }
tools/lkl/lib/Build +7 -0
index 8b137891791f..658bfa865b9c 100644
@@ -1 +1,8 @@
+CFLAGS_config.o += -I$(srctree)/tools/perf/pmu-events
 
+liblkl-y += jmp_buf.o
+liblkl-y += utils.o
+liblkl-y += dbg.o
+liblkl-y += dbg_handler.o
+liblkl-y += ../../perf/pmu-events/jsmn.o
+liblkl-y += config.o
tools/lkl/lib/config.c new file +744 -0

744 lines not on this page · raw message

tools/lkl/lib/dbg.c new file +300 -0

300 lines not on this page · raw message

tools/lkl/lib/dbg_handler.c new file +44 -0

44 lines not on this page · raw message

tools/lkl/lib/endian.h new file +31 -0
new file mode 100644
index 000000000000..aaccfa0edb65
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_LIB_ENDIAN_H
+#define _LKL_LIB_ENDIAN_H
+
+#if defined(__FreeBSD__)
+#include <sys/endian.h>
+#elif defined(__ANDROID__)
+#include <sys/endian.h>
+#elif defined(__MINGW32__)
+#include <winsock.h>
+#define le32toh(x) (x)
+#define le16toh(x) (x)
+#define htole32(x) (x)
+#define htole16(x) (x)
+#define le64toh(x) (x)
+#define htobe32(x) htonl(x)
+#define htobe16(x) htons(x)
+#define be32toh(x) ntohl(x)
+#define be16toh(x) ntohs(x)
+#else
+#include <endian.h>
+#endif
+
+#ifndef htonl
+#define htonl(x) htobe32(x)
+#define htons(x) htobe16(x)
+#define ntohl(x) be32toh(x)
+#define ntohs(x) be16toh(x)
+#endif
+
+#endif /* _LKL_LIB_ENDIAN_H */
tools/lkl/lib/jmp_buf.c new file +14 -0
new file mode 100644
index 000000000000..f6bdd7e4bd83
@@ -0,0 +1,14 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <setjmp.h>
+#include <lkl_host.h>
+
+void jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void))
+{
+	if (!setjmp(*((jmp_buf *)jmpb->buf)))
+		f();
+}
+
+void jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val)
+{
+	longjmp(*((jmp_buf *)jmpb->buf), val);
+}
tools/lkl/lib/jmp_buf.h new file +8 -0
new file mode 100644
index 000000000000..8782cbaaf51f
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_LIB_JMP_BUF_H
+#define _LKL_LIB_JMP_BUF_H
+
+void jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void));
+void jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val);
+
+#endif
tools/lkl/lib/utils.c new file +266 -0

266 lines not on this page · raw message

Signature · 1 line
2.21.0 (Apple Git-122.2)
#57 lore raw

RFC v4 14/25 lkl tools: host lib: filesystem helpers

In reply to Hajime Tazaki

Add LKL applications APIs to mount and unmount a filesystem from a disk added via lkl_disk_add().

Also add open/close/read directory wrappers on top of lkl_sys_getdents64.

Cc Conrad Meyer <[email protected]>
Cc Michael Zimmermann <[email protected]>
Cc Yuan Liu <[email protected]>
---
 tools/lkl/include/lkl.h | 165 ++++++++++++++
 tools/lkl/lib/Build     |   1 +
 tools/lkl/lib/fs.c      | 471 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 637 insertions(+)
 create mode 100644 tools/lkl/lib/fs.c
tools/lkl/include/lkl.h +165 -0
index 4b95d0ef8e5b..cdae92300320 100644
@@ -350,6 +350,171 @@ const char *lkl_strerror(int err);
  */
 void lkl_perror(char *msg, int err);
 
+/**
+ * struct lkl_dev_blk_ops - block device host operations, defined in lkl_host.h.
+ */
+struct lkl_dev_blk_ops;
+
+/**
+ * lkl_disk - host disk handle
+ *
+ * @dev - a pointer to private information for this disk backend
+ * @fd - a POSIX file descriptor that can be used by preadv/pwritev
+ * @handle - an NT file handle that can be used by ReadFile/WriteFile
+ */
+struct lkl_disk {
+	void *dev;
+	union {
+		int fd;
+		void *handle;
+	};
+	struct lkl_dev_blk_ops *ops;
+};
+
+/**
+ * lkl_disk_add - add a new disk
+ *
+ * @disk - the host disk handle
+ * @returns a disk id (0 is valid) or a strictly negative value in case of error
+ */
+int lkl_disk_add(struct lkl_disk *disk);
+
+/**
+ * lkl_disk_remove - remove a disk
+ *
+ * This function makes a cleanup of the @disk's private information
+ * that was initialized by lkl_disk_add before.
+ *
+ * @disk - the host disk handle
+ */
+int lkl_disk_remove(struct lkl_disk disk);
+
+/**
+ * lkl_encode_dev_from_sysfs_blkdev - extract device id from sysfs
+ *
+ * This function returns the device id for the given sysfs dev node.
+ * The content of the node has to be in the form 'MAJOR:MINOR'.
+ * Also, this function expects an absolute path which means that sysfs
+ * already has to be mounted at the given path
+ *
+ * @sysfs_path - absolute path to the sysfs dev node
+ * @pdevid - pointer to memory where dev id will be returned
+ * @returns - 0 on success, a negative value on error
+ */
+int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid);
+
+/**
+ * lkl_mount_dev - mount a disk
+ *
+ * This functions creates a device file for the given disk, creates a mount
+ * point and mounts the device over the mount point.
+ *
+ * @disk_id - the disk id identifying the disk to be mounted
+ * @part - disk partition or zero for full disk
+ * @fs_type - filesystem type
+ * @flags - mount flags
+ * @opts - additional filesystem specific mount options
+ * @mnt_str - a string that will be filled by this function with the path where
+ * the filesystem has been mounted
+ * @mnt_str_len - size of mnt_str
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_mount_dev(unsigned int disk_id, unsigned int part, const char *fs_type,
+		   int flags, const char *opts,
+		   char *mnt_str, unsigned int mnt_str_len);
+
+/**
+ * lkl_umount_dev - umount a disk
+ *
+ * This functions umounts the given disks and removes the device file and the
+ * mount point.
+ *
+ * @disk_id - the disk id identifying the disk to be mounted
+ * @part - disk partition or zero for full disk
+ * @flags - umount flags
+ * @timeout_ms - timeout to wait for the kernel to flush closed files so that
+ * umount can succeed
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
+		    long timeout_ms);
+
+/**
+ * lkl_umount_timeout - umount filesystem with timeout
+ *
+ * @path - the path to unmount
+ * @flags - umount flags
+ * @timeout_ms - timeout to wait for the kernel to flush closed files so that
+ * umount can succeed
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_umount_timeout(char *path, int flags, long timeout_ms);
+
+/**
+ * lkl_opendir - open a directory
+ *
+ * @path - directory path
+ * @err - pointer to store the error in case of failure
+ * @returns - a handle to be used when calling lkl_readdir
+ */
+struct lkl_dir *lkl_opendir(const char *path, int *err);
+
+/**
+ * lkl_fdopendir - open a directory
+ *
+ * @fd - file descriptor
+ * @err - pointer to store the error in case of failure
+ * @returns - a handle to be used when calling lkl_readdir
+ */
+struct lkl_dir *lkl_fdopendir(int fd, int *err);
+
+/**
+ * lkl_rewinddir - reset directory stream
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ */
+void lkl_rewinddir(struct lkl_dir *dir);
+
+/**
+ * lkl_closedir - close the directory
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ */
+int lkl_closedir(struct lkl_dir *dir);
+
+/**
+ * lkl_readdir - get the next available entry of the directory
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ * @returns - a lkl_dirent64 entry or NULL if the end of the directory stream is
+ * reached or if an error occurred; check lkl_errdir() to distinguish between
+ * errors or end of the directory stream
+ */
+struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir);
+
+/**
+ * lkl_errdir - checks if an error occurred during the last lkl_readdir call
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ * @returns - 0 if no error occurred, or a negative value otherwise
+ */
+int lkl_errdir(struct lkl_dir *dir);
+
+/**
+ * lkl_dirfd - gets the file descriptor associated with the directory handle
+ *
+ * @dir - the directory handle as returned by lkl_opendir
+ * @returns - a positive value,which is the LKL file descriptor associated with
+ * the directory handle, or a negative value otherwise
+ */
+int lkl_dirfd(struct lkl_dir *dir);
+
+/**
+ * lkl_mount_fs - mount a file system type like proc, sys
+ * @fstype - file system type. e.g. proc, sys
+ * @returns - 0 on success. 1 if it's already mounted. negative on failure.
+ */
+int lkl_mount_fs(char *fstype);
 
 #ifdef __cplusplus
 }
tools/lkl/lib/Build +1 -0
index 658bfa865b9c..4a444337b0e7 100644
@@ -1,5 +1,6 @@
 CFLAGS_config.o += -I$(srctree)/tools/perf/pmu-events
 
+liblkl-y += fs.o
 liblkl-y += jmp_buf.o
 liblkl-y += utils.o
 liblkl-y += dbg.o
tools/lkl/lib/fs.c new file +471 -0

471 lines not on this page · raw message

Signature · 1 line
2.21.0 (Apple Git-122.2)
#58 lore raw

RFC v4 15/25 lkl tools: host lib: networking helpers

In reply to Hajime Tazaki

This commit adds LKL application APIs to control network related resources of kernel via LKL syscall, ioctl, and via netlink messages.

---
 tools/lkl/include/lkl.h      | 241 ++++++++++
 tools/lkl/include/lkl_host.h |  85 ++++
 tools/lkl/lib/Build          |   1 +
 tools/lkl/lib/net.c          | 826 +++++++++++++++++++++++++++++++++++
 4 files changed, 1153 insertions(+)
 create mode 100644 tools/lkl/lib/net.c
tools/lkl/include/lkl.h +241 -0
index cdae92300320..1f4291ad9455 100644
@@ -516,6 +516,247 @@ int lkl_dirfd(struct lkl_dir *dir);
  */
 int lkl_mount_fs(char *fstype);
 
+/**
+ * lkl_if_up - activate network interface
+ *
+ * @ifindex - the ifindex of the interface
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_up(int ifindex);
+
+/**
+ * lkl_if_down - deactivate network interface
+ *
+ * @ifindex - the ifindex of the interface
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_down(int ifindex);
+
+/**
+ * lkl_if_set_mtu - set MTU on interface
+ *
+ * @ifindex - the ifindex of the interface
+ * @mtu - the requested MTU size
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_set_mtu(int ifindex, int mtu);
+
+/**
+ * lkl_if_set_ipv4 - set IPv4 address on interface
+ *
+ * @ifindex - the ifindex of the interface
+ * @addr - 4-byte IP address (i.e., struct in_addr)
+ * @netmask_len - prefix length of the @addr
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_set_ipv4(int ifindex, unsigned int addr, unsigned int netmask_len);
+
+/**
+ * lkl_set_ipv4_gateway - add an IPv4 default route
+ *
+ * @addr - 4-byte IP address of the gateway (i.e., struct in_addr)
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_set_ipv4_gateway(unsigned int addr);
+
+/**
+ * lkl_if_set_ipv4_gateway - add an IPv4 default route in rule table
+ *
+ * @ifindex - the ifindex of the interface, used for tableid calculation
+ * @addr - 4-byte IP address of the interface
+ * @netmask_len - prefix length of the @addr
+ * @gw_addr - 4-byte IP address of the gateway
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_set_ipv4_gateway(int ifindex, unsigned int addr,
+		unsigned int netmask_len, unsigned int gw_addr);
+
+/**
+ * lkl_if_set_ipv6 - set IPv6 address on interface
+ * must be called after interface is up.
+ *
+ * @ifindex - the ifindex of the interface
+ * @addr - 16-byte IPv6 address (i.e., struct in6_addr)
+ * @netprefix_len - prefix length of the @addr
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_set_ipv6(int ifindex, void *addr, unsigned int netprefix_len);
+
+/**
+ * lkl_set_ipv6_gateway - add an IPv6 default route
+ *
+ * @addr - 16-byte IPv6 address of the gateway (i.e., struct in6_addr)
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_set_ipv6_gateway(void *addr);
+
+/**
+ * lkl_if_set_ipv6_gateway - add an IPv6 default route in rule table
+ *
+ * @ifindex - the ifindex of the interface, used for tableid calculation
+ * @addr - 16-byte IP address of the interface
+ * @netmask_len - prefix length of the @addr
+ * @gw_addr - 16-byte IP address of the gateway (i.e., struct in_addr)
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_set_ipv6_gateway(int ifindex, void *addr,
+		unsigned int netmask_len, void *gw_addr);
+
+/**
+ * lkl_ifname_to_ifindex - obtain ifindex of an interface by name
+ *
+ * @name - string of an interface
+ * @returns - return an integer of ifindex if no error
+ */
+int lkl_ifname_to_ifindex(const char *name);
+
+/**
+ * lkl_netdev_get_ifindex - retrieve the interface index for a given network
+ * device id
+ *
+ * @id - the network device id
+ * @returns the interface index or a stricly negative value in case of error
+ */
+int lkl_netdev_get_ifindex(int id);
+
+/**
+ * lkl_add_neighbor - add a permanent arp entry
+ * @ifindex - the ifindex of the interface
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @ip - ip address of the entry in network byte order
+ * @mac - mac address of the entry
+ */
+int lkl_add_neighbor(int ifindex, int af, void *addr, void *mac);
+
+/**
+ * lkl_if_add_ip - add an ip address
+ * @ifindex - the ifindex of the interface
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @addr - ip address of the entry in network byte order
+ * @netprefix_len - prefix length of the @addr
+ */
+int lkl_if_add_ip(int ifindex, int af, void *addr, unsigned int netprefix_len);
+
+/**
+ * lkl_if_del_ip - add an ip address
+ * @ifindex - the ifindex of the interface
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @addr - ip address of the entry in network byte order
+ * @netprefix_len - prefix length of the @addr
+ */
+int lkl_if_del_ip(int ifindex, int af, void *addr, unsigned int netprefix_len);
+
+/**
+ * lkl_add_gateway - add a gateway
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @gwaddr - 4-byte IP address of the gateway (i.e., struct in_addr)
+ */
+int lkl_add_gateway(int af, void *gwaddr);
+
+/**
+ * XXX Should I use OIF selector?
+ * temporary table idx = ifindex * 2 + 0 <- ipv4
+ * temporary table idx = ifindex * 2 + 1 <- ipv6
+ */
+/**
+ * lkl_if_add_rule_from_addr - create an ip rule table with "from" selector
+ * @ifindex - the ifindex of the interface, used for table id calculation
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @saddr - network byte order ip address, "from" selector address of this rule
+ */
+int lkl_if_add_rule_from_saddr(int ifindex, int af, void *saddr);
+
+/**
+ * lkl_if_add_gateway - add gateway to rule table
+ * @ifindex - the ifindex of the interface, used for table id calculation
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @gwaddr - 4-byte IP address of the gateway (i.e., struct in_addr)
+ */
+int lkl_if_add_gateway(int ifindex, int af, void *gwaddr);
+
+/**
+ * lkl_if_add_linklocal - add linklocal route to rule table
+ * @ifindex - the ifindex of the interface, used for table id calculation
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @addr - ip address of the entry in network byte order
+ * @netprefix_len - prefix length of the @addr
+ */
+int lkl_if_add_linklocal(int ifindex, int af,  void *addr, int netprefix_len);
+
+/**
+ * lkl_if_wait_ipv6_dad - wait for DAD to be done for a ipv6 address
+ * must be called after interface is up
+ *
+ * @ifindex - the ifindex of the interface
+ * @addr - ip address of the entry in network byte order
+ */
+int lkl_if_wait_ipv6_dad(int ifindex, void *addr);
+
+/**
+ * lkl_set_fd_limit - set the maximum number of file descriptors allowed
+ * @fd_limit - fd max limit
+ */
+int lkl_set_fd_limit(unsigned int fd_limit);
+
+/**
+ * lkl_qdisc_add - set qdisc rule onto an interface
+ *
+ * @ifindex - the ifindex of the interface
+ * @root - the name of root class (e.g., "root");
+ * @type - the type of qdisc (e.g., "fq")
+ */
+int lkl_qdisc_add(int ifindex, const char *root, const char *type);
+
+/**
+ * lkl_qdisc_parse_add - Add a qdisc entry for an interface with strings
+ *
+ * @ifindex - the ifindex of the interface
+ * @entries - strings of qdisc configurations in the form of
+ *            "root|type;root|type;..."
+ */
+void lkl_qdisc_parse_add(int ifindex, const char *entries);
+
+/**
+ * lkl_netdev - host network device handle, defined in lkl_host.h.
+ */
+struct lkl_netdev;
+
+/**
+ * lkl_netdev_args - arguments to lkl_netdev_add
+ * @mac - optional MAC address for the device
+ * @offload - offload bits for the device
+ */
+struct lkl_netdev_args {
+	void *mac;
+	unsigned int offload;
+};
+
+/*
+ * lkl_register_dbg_handler- register a signal handler that loads a debug lib.
+ *
+ * The signal handler is triggered by Ctrl-Z. It creates a new pthread which
+ * call dbg_entrance().
+ *
+ * If you run the program from shell script, make sure you ignore SIGTSTP by
+ * "trap '' TSTP" in the shell script.
+ */
+void lkl_register_dbg_handler(void);
+
+/**
+ * lkl_sysctl - write a sysctl value
+ *
+ * @path - the path to an sysctl entry (e.g., "net.ipv4.tcp_wmem");
+ * @value - the value of the sysctl (e.g., "4096 87380 2147483647")
+ */
+int lkl_sysctl(const char *path, const char *value);
+
+/**
+ * lkl_sysctl_parse_write - Configure sysctl parameters with strings
+ *
+ * @sysctls - Configure sysctl parameters as the form of "key=value;..."
+ */
+void lkl_sysctl_parse_write(const char *sysctls);
+
 #ifdef __cplusplus
 }
 #endif
tools/lkl/include/lkl_host.h +85 -0

91 lines not on this page · raw message

tools/lkl/lib/Build +1 -0
index 4a444337b0e7..dba530424e4a 100644
@@ -1,6 +1,7 @@
 CFLAGS_config.o += -I$(srctree)/tools/perf/pmu-events
 
 liblkl-y += fs.o
+liblkl-y += net.o
 liblkl-y += jmp_buf.o
 liblkl-y += utils.o
 liblkl-y += dbg.o
tools/lkl/lib/net.c new file +826 -0

826 lines not on this page · raw message

Signature · 1 line
2.21.0 (Apple Git-122.2)
#59 lore raw

RFC v4 16/25 lkl tools: host lib: posix host operations

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

This commit implements LKL host operations for POSIX hosts. The operations are implemented to be portable in various POSIX OSs.

Cc Conrad Meyer <[email protected]>
Cc Mark Stillwell <[email protected]>
Cc Patrick Collins <[email protected]>
Cc Pierre-Hugues Husson <[email protected]>
Cc Thomas Liebetraut <[email protected]>
Cc Yuan Liu <[email protected]>
---
 tools/lkl/lib/Build        |   2 +
 tools/lkl/lib/posix-host.c | 353 +++++++++++++++++++++++++++++++++++++
 2 files changed, 355 insertions(+)
 create mode 100644 tools/lkl/lib/posix-host.c
tools/lkl/lib/Build +2 -0
index dba530424e4a..0e711e260a3a 100644
@@ -1,8 +1,10 @@
 CFLAGS_config.o += -I$(srctree)/tools/perf/pmu-events
+CFLAGS_posix-host.o += -D_FILE_OFFSET_BITS=64
 
 liblkl-y += fs.o
 liblkl-y += net.o
 liblkl-y += jmp_buf.o
+liblkl-$(LKL_HOST_CONFIG_POSIX) += posix-host.o
 liblkl-y += utils.o
 liblkl-y += dbg.o
 liblkl-y += dbg_handler.o
tools/lkl/lib/posix-host.c new file +353 -0

353 lines not on this page · raw message

Signature · 1 line
2.21.0 (Apple Git-122.2)
#60 lore raw

RFC v4 17/25 lkl tools: add test programs

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

Add a simple LKL test application (boot) that starts the kernel and performs simple tests that minimally exercise the LKL API. Networking and block device tests are also added which are useful to detect regressions.

Cc Conrad Meyer <[email protected]>
Cc David Disseldorp <[email protected]>
Cc H.K. Jerry Chu <[email protected]>
Cc Lai Jiangshan <[email protected]>
Cc Luca Dariz <[email protected]>
Cc Mark Stillwell <[email protected]>
Cc Motomu Utsumi <[email protected]>
Cc Patrick Collins <[email protected]>
Cc Petros Angelatos <[email protected]>
Cc Thomas Liebetraut <[email protected]>
Cc Yuan Liu <[email protected]>
---
 tools/lkl/.gitignore              |   5 +
 tools/lkl/Makefile                |   5 +-
 tools/lkl/Targets                 |   3 +
 tools/lkl/tests/Build             |   3 +
 tools/lkl/tests/boot.c            | 562 ++++++++++++++++++++++++++++++
 tools/lkl/tests/boot.sh           |   9 +
 tools/lkl/tests/cla.c             | 159 +++++++++
 tools/lkl/tests/cla.h             |  33 ++
 tools/lkl/tests/disk.c            | 189 ++++++++++
 tools/lkl/tests/disk.sh           |  61 ++++
 tools/lkl/tests/net-setup.sh      |  73 ++++
 tools/lkl/tests/net-test.c        | 299 ++++++++++++++++
 tools/lkl/tests/net.sh            |  90 +++++
 tools/lkl/tests/run.py            | 179 ++++++++++
 tools/lkl/tests/tap13.py          | 209 +++++++++++
 tools/lkl/tests/test.c            | 126 +++++++
 tools/lkl/tests/test.h            |  72 ++++
 tools/lkl/tests/test.sh           | 182 ++++++++++
 tools/lkl/tests/valgrind.supp     | 105 ++++++
 tools/lkl/tests/valgrind2xunit.py |  69 ++++
 20 files changed, 2432 insertions(+), 1 deletion(-)
 create mode 100644 tools/lkl/tests/Build
 create mode 100644 tools/lkl/tests/boot.c
 create mode 100755 tools/lkl/tests/boot.sh
 create mode 100644 tools/lkl/tests/cla.c
 create mode 100644 tools/lkl/tests/cla.h
 create mode 100644 tools/lkl/tests/disk.c
 create mode 100755 tools/lkl/tests/disk.sh
 create mode 100644 tools/lkl/tests/net-setup.sh
 create mode 100644 tools/lkl/tests/net-test.c
 create mode 100755 tools/lkl/tests/net.sh
 create mode 100755 tools/lkl/tests/run.py
 create mode 100644 tools/lkl/tests/tap13.py
 create mode 100644 tools/lkl/tests/test.c
 create mode 100644 tools/lkl/tests/test.h
 create mode 100644 tools/lkl/tests/test.sh
 create mode 100644 tools/lkl/tests/valgrind.supp
 create mode 100755 tools/lkl/tests/valgrind2xunit.py
tools/lkl/.gitignore +5 -0
index 1aed58bfe171..4e08254dbd46 100644
@@ -2,3 +2,8 @@ Makefile.conf
 include/lkl_autoconf.h
 tests/autoconf.sh
 bin/stat
+tests/net-test
+tests/disk
+tests/boot
+tests/valgrind*.xml
+*.pyc
tools/lkl/Makefile +4 -1
index 11ea7e9095bb..0abccb9d86b6 100644
@@ -117,8 +117,11 @@ programs_install: $(progs-y:%=$(OUTPUT)%$(EXESUF))
 install: headers_install libraries_install programs_install
 
 
+run-tests:
+	./tests/run.py $(tests)
+
 FORCE: ;
-.PHONY: all clean FORCE
+.PHONY: all clean FORCE run-tests
 .PHONY: headers_install libraries_install programs_install install
 .NOTPARALLEL : lib/lkl.o
 .SECONDARY:
tools/lkl/Targets +3 -0
index 24c985e64638..a9f74c3cc8fb 100644
@@ -1,3 +1,6 @@
 libs-y += lib/liblkl
 
+progs-y += tests/boot
+progs-y += tests/disk
+progs-y += tests/net-test
 
tools/lkl/tests/Build new file +3 -0
new file mode 100644
index 000000000000..ace86a3d3438
@@ -0,0 +1,3 @@
+boot-y += boot.o test.o
+disk-y += disk.o cla.o test.o
+net-test-y += net-test.o cla.o test.o
tools/lkl/tests/boot.c new file +562 -0

562 lines not on this page · raw message

tools/lkl/tests/boot.sh new file +9 -0
new file mode 100755
index 000000000000..d985c04b0ac1
@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+source $script_dir/test.sh
+
+lkl_test_plan 1 "boot"
+lkl_test_run 1
+lkl_test_exec $script_dir/boot
tools/lkl/tests/cla.c new file +159 -0

159 lines not on this page · raw message

tools/lkl/tests/cla.h new file +33 -0
new file mode 100644
index 000000000000..f8369be02e5a
@@ -0,0 +1,33 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_TEST_CLA_H
+#define _LKL_TEST_CLA_H
+
+enum cl_arg_type {
+	CL_ARG_USER = 0,
+	CL_ARG_BOOL,
+	CL_ARG_INT,
+	CL_ARG_STR,
+	CL_ARG_STR_SET,
+	CL_ARG_IPV4,
+	CL_ARG_END,
+};
+
+struct cl_arg;
+
+typedef int (*cl_arg_parser_t)(struct cl_arg *arg, const char *value);
+
+struct cl_arg {
+	const char *long_name;
+	char short_name;
+	const char *help;
+	int has_arg;
+	enum cl_arg_type type;
+	void *store;
+	void *set;
+	cl_arg_parser_t parser;
+};
+
+int parse_args(int argc, const char **argv, struct cl_arg *args);
+
+
+#endif /* _LKL_TEST_CLA_H */
tools/lkl/tests/disk.c new file +189 -0

189 lines not on this page · raw message

tools/lkl/tests/disk.sh new file +61 -0

61 lines not on this page · raw message

tools/lkl/tests/net-setup.sh new file +73 -0

73 lines not on this page · raw message

tools/lkl/tests/net-test.c new file +299 -0

299 lines not on this page · raw message

tools/lkl/tests/net.sh new file +90 -0

90 lines not on this page · raw message

tools/lkl/tests/run.py new file +179 -0

179 lines not on this page · raw message

tools/lkl/tests/tap13.py new file +209 -0

209 lines not on this page · raw message

tools/lkl/tests/test.c new file +126 -0

126 lines not on this page · raw message

tools/lkl/tests/test.h new file +72 -0

72 lines not on this page · raw message

tools/lkl/tests/test.sh new file +182 -0

182 lines not on this page · raw message

tools/lkl/tests/valgrind.supp new file +105 -0

105 lines not on this page · raw message

tools/lkl/tests/valgrind2xunit.py new file +69 -0

69 lines not on this page · raw message

Signature · 1 line
2.21.0 (Apple Git-122.2)
#61 lore raw

RFC v4 18/25 lkl tools: cptofs that reads/writes to/from a filesystem image

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

A tool to read/write to/from a filesystem image without mounting the file to host filesystem. Thus there is no root privilege to modify the contents.

Cc Conrad Meyer <[email protected]>
Cc Dan Peebles <[email protected]>
Cc Petros Angelatos <[email protected]>
Cc Tuomas Tynkkynen <[email protected]>
Cc Yuriy Taraday <[email protected]>
---
 tools/lkl/.gitignore |   2 +
 tools/lkl/Build      |   2 +
 tools/lkl/Makefile   |   3 +
 tools/lkl/Targets    |   4 +
 tools/lkl/cptofs.c   | 636 +++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 647 insertions(+)
 create mode 100644 tools/lkl/cptofs.c
tools/lkl/.gitignore +2 -0
index 4e08254dbd46..1a8210f4d9c4 100644
@@ -7,3 +7,5 @@ tests/disk
 tests/boot
 tests/valgrind*.xml
 *.pyc
+cptofs
+cpfromfs
tools/lkl/Build +2 -0
index e69de29bb2d1..a9d12c5ca260 100644
@@ -0,0 +1,2 @@
+cptofs-$(LKL_HOST_CONFIG_ARCHIVE) += cptofs.o
+
tools/lkl/Makefile +3 -0
index 0abccb9d86b6..03750aab98d8 100644
@@ -86,6 +86,9 @@ $(OUTPUT)%$(EXESUF): $(OUTPUT)%-in.o $(OUTPUT)liblkl.a
 $(OUTPUT)%-in.o: $(OUTPUT)lib/lkl.o FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
+$(OUTPUT)cpfromfs$(EXESUF): $(OUTPUT)cptofs$(EXESUF)
+	$(Q)if ! [ -e $@ ]; then ln -s $< $@; fi
+
 
 clean:
 	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
tools/lkl/Targets +4 -0
index a9f74c3cc8fb..17de965f92b3 100644
@@ -4,3 +4,7 @@ progs-y += tests/boot
 progs-y += tests/disk
 progs-y += tests/net-test
 
+progs-$(LKL_HOST_CONFIG_ARCHIVE) += cptofs
+progs-$(LKL_HOST_CONFIG_ARCHIVE) += cpfromfs
+LDLIBS_cptofs-y := -larchive
+LDLIBS_cptofs-$(LKL_HOST_CONFIG_NEEDS_LARGP) += -largp
tools/lkl/cptofs.c new file +636 -0

636 lines not on this page · raw message

Signature · 7 lines
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-um
#62 lore raw

RFC v4 19/25 lkl tools: fs2tar that converts a filesystem image to tar

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

Simple utility that converts a filesystem image to a tar file, preserving file rights and extended attributes.

---
 tools/lkl/.gitignore |   1 +
 tools/lkl/Build      |   1 +
 tools/lkl/Targets    |   4 +
 tools/lkl/fs2tar.c   | 412 +++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 418 insertions(+)
 create mode 100644 tools/lkl/fs2tar.c
tools/lkl/.gitignore +1 -0
index 1a8210f4d9c4..138e65efcad2 100644
@@ -9,3 +9,4 @@ tests/valgrind*.xml
 *.pyc
 cptofs
 cpfromfs
+fs2tar
tools/lkl/Build +1 -0
index a9d12c5ca260..73b37363a6de 100644
@@ -1,2 +1,3 @@
 cptofs-$(LKL_HOST_CONFIG_ARCHIVE) += cptofs.o
+fs2tar-$(LKL_HOST_CONFIG_ARCHIVE) += fs2tar.o
 
tools/lkl/Targets +4 -0
index 17de965f92b3..3451a1856955 100644
@@ -8,3 +8,7 @@ progs-$(LKL_HOST_CONFIG_ARCHIVE) += cptofs
 progs-$(LKL_HOST_CONFIG_ARCHIVE) += cpfromfs
 LDLIBS_cptofs-y := -larchive
 LDLIBS_cptofs-$(LKL_HOST_CONFIG_NEEDS_LARGP) += -largp
+
+progs-$(LKL_HOST_CONFIG_ARCHIVE) += fs2tar
+LDLIBS_fs2tar-y := -larchive
+LDLIBS_fs2tar-$(LKL_HOST_CONFIG_NEEDS_LARGP) += -largp
tools/lkl/fs2tar.c new file +412 -0

412 lines not on this page · raw message

Signature · 1 line
2.21.0 (Apple Git-122.2)
#63 lore raw

RFC v4 20/25 lkl tools: add lklfuse

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

Add a simple fuse based program that can mount filesystem in userspace using LKL.

Cc Conrad Meyer <[email protected]>
Cc Quentin Anglade <[email protected]>
Cc Rafael Gieschke <[email protected]>
---
 tools/lkl/.gitignore       |   1 +
 tools/lkl/Build            |   3 +
 tools/lkl/Targets          |   3 +
 tools/lkl/lklfuse.c        | 659 +++++++++++++++++++++++++++++++++++++
 tools/lkl/tests/lklfuse.sh | 110 +++++++
 5 files changed, 776 insertions(+)
 create mode 100644 tools/lkl/lklfuse.c
 create mode 100755 tools/lkl/tests/lklfuse.sh
tools/lkl/.gitignore +1 -0
index 138e65efcad2..c78ec268e4b0 100644
@@ -10,3 +10,4 @@ tests/valgrind*.xml
 cptofs
 cpfromfs
 fs2tar
+lklfuse
tools/lkl/Build +3 -0
index 73b37363a6de..6048440d0e1b 100644
@@ -1,3 +1,6 @@
+CFLAGS_lklfuse.o += -D_FILE_OFFSET_BITS=64
+
 cptofs-$(LKL_HOST_CONFIG_ARCHIVE) += cptofs.o
 fs2tar-$(LKL_HOST_CONFIG_ARCHIVE) += fs2tar.o
+lklfuse-$(LKL_HOST_CONFIG_FUSE) += lklfuse.o
 
tools/lkl/Targets +3 -0
index 3451a1856955..03d24362a195 100644
@@ -12,3 +12,6 @@ LDLIBS_cptofs-$(LKL_HOST_CONFIG_NEEDS_LARGP) += -largp
 progs-$(LKL_HOST_CONFIG_ARCHIVE) += fs2tar
 LDLIBS_fs2tar-y := -larchive
 LDLIBS_fs2tar-$(LKL_HOST_CONFIG_NEEDS_LARGP) += -largp
+
+progs-$(LKL_HOST_CONFIG_FUSE) += lklfuse
+LDLIBS_lklfuse-y := -lfuse
tools/lkl/lklfuse.c new file +659 -0

659 lines not on this page · raw message

tools/lkl/tests/lklfuse.sh new file +110 -0

110 lines not on this page · raw message

Signature · 1 line
2.21.0 (Apple Git-122.2)
#64 lore raw

RFC v4 21/25 um lkl: add documentation

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

A document describing brief introduction of LKL, why it is needed, and how it is used is added. The document is located under uml/ directory.

Cc Chenyang Zhong <[email protected]>
Cc Conrad Meyer <[email protected]>
Cc Gustavo Bittencourt <[email protected]>
Cc H.K. Jerry Chu <[email protected]>
Cc Motomu Utsumi <[email protected]>
Cc Patrick Collins <[email protected]>
Cc Thomas Liebetraut <[email protected]>
Cc Yuan Liu <[email protected]>
---
 Documentation/virt/uml/lkl.txt | 64 ++++++++++++++++++++++++++++++++++
 1 file changed, 64 insertions(+)
 create mode 100644 Documentation/virt/uml/lkl.txt
Documentation/virt/uml/lkl.txt new file +64 -0
new file mode 100644
index 000000000000..f6d581092331
@@ -0,0 +1,64 @@
+
+Introduction
+============
+
+LKL (Linux Kernel Library) is aiming to allow reusing the Linux kernel code as
+extensively as possible with minimal effort and reduced maintenance overhead.
+
+Examples of how LKL can be used are: creating userspace applications (running on
+Linux and other operating systems) that can read or write Linux filesystems or
+can use the Linux networking stack, creating kernel drivers for other operating
+systems that can read Linux filesystems, bootloaders support for reading/writing
+Linux filesystems, etc.
+
+With LKL, the kernel code is compiled into an object file that can be directly
+linked by applications. The API offered by LKL is based on the Linux system call
+interface.
+
+LKL is implemented as one of the mode of UML (arch/um). It uses host operations
+defined by the application or a host library (tools/lkl/lib).
+
+
+Supported hosts
+===============
+
+The supported host for now is Linux (x86 architecture) userspace applications.
+
+
+Building LKL the host library and LKL applications
+==================================================
+
+    $ make -C tools/lkl
+
+will build LKL as a object file, it will install it in tools/lkl/lib together
+with the headers files in tools/lkl/include then will build the host library,
+tests and a few of application examples:
+
+* tests/boot - a simple applications that uses LKL and exercises the basic LKL
+  APIs
+
+* tests/net-test - a simple applications that uses network feature of LKL and
+  exercises the basic network-related APIs
+
+* fs2tar - a tool that converts a filesystem image to a tar archive
+
+* cptofs/cpfromfs - a tool that copies files to/from a filesystem image
+
+* lklfuse - a tool that can mount a filesystem image in userspace,
+  without root privileges, using FUSE
+
+
+Building LKL on Ubuntu
+-----------------------
+
+    $ sudo apt-get install libfuse-dev libarchive-dev xfsprogs
+
+    # Optional, if you would like to be able to run tests
+    $ sudo apt-get install btrfs-tools
+    $ pip install yamlish junit_xml
+
+    $ make -C tools/lkl
+
+    # To check that everything works:
+    $ cd tools/lkl
+    $ make run-tests
Signature · 1 line
2.21.0 (Apple Git-122.2)
#65 lore raw

RFC v4 22/25 um lkl: add CI scripts to conduct regression tests

In reply to Hajime Tazaki

For the continuous integration (CI), we use CircleCI with multiple test targets triggered with a single commit. The test also conducts memory checks using valgrind in order to prevent enbug.

---
 .circleci/config.yml             | 196 +++++++++++++++++++++++++++++++
 tools/lkl/scripts/checkpatch.sh  |  60 ++++++++++
 tools/lkl/scripts/lkl-jenkins.sh |  21 ++++
 3 files changed, 277 insertions(+)
 create mode 100644 .circleci/config.yml
 create mode 100755 tools/lkl/scripts/checkpatch.sh
 create mode 100755 tools/lkl/scripts/lkl-jenkins.sh
.circleci/config.yml new file +196 -0
new file mode 100644
index 000000000000..5bdf059014c0
@@ -0,0 +1,196 @@
+version: 2
+general:
+  artifacts:
+
+do_steps: &do_steps
+ steps:
+  - run: echo "$CROSS_COMPILE" > ~/_cross_compile
+  - restore_cache:
+      key: code-tree-shallow-{{ .Environment.CACHE_VERSION }}
+  - run:
+      name: checkout build tree
+      command: |
+        mkdir -p ~/.ssh/
+        ssh-keyscan -H github.com >> ~/.ssh/known_hosts
+        if ! [ -d .git ]; then
+          git clone --depth=1 $CIRCLE_REPOSITORY_URL .;
+        fi
+        if [[ $CIRCLE_BRANCH == pull/* ]]; then
+           git fetch --depth=1 origin $CIRCLE_BRANCH/head;
+        else
+           git fetch --depth=1 origin $CIRCLE_BRANCH;
+        fi
+        git reset --hard $CIRCLE_SHA1
+  - save_cache:
+      key: code-tree-shallow-{{ .Environment.CACHE_VERSION }}-{{ epoch }}
+      paths:
+        - /home/ubuntu/project/.git
+  - run:
+      name: clean
+      command: |
+        make mrproper
+        cd tools/lkl && make clean-conf
+        rm -rf ~/junit
+  - run: mkdir -p /home/ubuntu/.ccache
+  - restore_cache:
+      key: compiler-cache-{{ checksum "~/_cross_compile" }}-{{ .Environment.CACHE_VERSION }}
+  - run: cd tools/lkl && make -j8 ${MKARG}
+  - run: mkdir -p ~/destdir && cd tools/lkl && make DESTDIR=~/destdir
+  - save_cache:
+     paths:
+       - /home/ubuntu/.ccache
+     key: compiler-cache-{{ checksum "~/_cross_compile" }}-{{ .Environment.CACHE_VERSION }}-{{ epoch }}
+  - run:
+      name: run tests
+      command: |
+        mkdir -p ~/junit
+        make -C tools/lkl run-tests tests="--junit-dir ~/junit"
+      no_output_timeout: "90m"
+  - run:
+      name: collecting test results
+      command: |
+        find ./tools/lkl/ -type f -name "*.xml" -exec mv {} ~/junit/ \;
+  - store_test_results:
+      path: ~/junit
+  - store_artifacts:
+      path: ~/junit
+
+
+do_uml_steps: &do_uml_steps
+ steps:
+  - run: echo "$CROSS_COMPILE" > ~/_cross_compile
+  - restore_cache:
+      key: code-tree-shallow-{{ .Environment.CACHE_VERSION }}
+  - run:
+      name: checkout build tree
+      command: |
+        mkdir -p ~/.ssh/
+        ssh-keyscan -H github.com >> ~/.ssh/known_hosts
+        if ! [ -d .git ]; then
+          git clone --depth=1 $CIRCLE_REPOSITORY_URL .;
+        fi
+        if [[ $CIRCLE_BRANCH == pull/* ]]; then
+           git fetch --depth=1 origin $CIRCLE_BRANCH/head;
+        else
+           git fetch --depth=1 origin $CIRCLE_BRANCH;
+        fi
+        git reset --hard $CIRCLE_SHA1
+  - save_cache:
+      key: code-tree-shallow-{{ .Environment.CACHE_VERSION }}-{{ epoch }}
+      paths:
+        - /home/ubuntu/project/.git
+  - run: mkdir -p /home/ubuntu/.ccache
+  - restore_cache:
+      key: compiler-cache-{{ checksum "~/_cross_compile" }}-{{ .Environment.CACHE_VERSION }}
+  - run:
+      name: build
+      command: |
+        sudo apt-get update
+        sudo apt-get install -y gcc-multilib g++-multilib
+        make defconfig ARCH=um SUBARCH=$SUBARCH
+        make ARCH=um SUBARCH=$SUBARCH
+  - save_cache:
+     paths:
+       - /home/ubuntu/.ccache
+     key: compiler-cache-{{ checksum "~/_cross_compile" }}-{{ .Environment.CACHE_VERSION }}-{{ epoch }}
+  - run:
+      name: test
+      command: |
+        # XXX: i386 build doesn't work with the test
+        if [ $CIRCLE_STAGE = "i386_uml" ] || [ $CIRCLE_STAGE = "i386_uml_on_x86_64" ]; then
+          exit 0
+        fi
+        ./linux rootfstype=hostfs ro mem=1g loglevel=10 init="/bin/bash -c exit" || export RETVAL=$?
+        # SIGABRT=6 => 128+6
+        if [ $RETVAL != "134" ]; then
+          exit 1
+        fi
+
+## Customize the test machine
+jobs:
+  x86_64:
+   docker:
+     - image: lkldocker/circleci-x86_64:0.7
+   environment:
+     CROSS_COMPILE: ""
+   <<: *do_steps
+
+  i386:
+   docker:
+     - image: lkldocker/circleci-i386:0.1
+   environment:
+     CROSS_COMPILE: ""
+   <<: *do_steps
+
+  x86_64_valgrind:
+   docker:
+     - image: lkldocker/circleci-x86_64:0.7
+   environment:
+     CROSS_COMPILE: ""
+     VALGRIND: 1
+   <<: *do_steps
+
+  x86_64_uml:
+   docker:
+     - image: lkldocker/circleci-x86_64:0.7
+   environment:
+     CROSS_COMPILE: ""
+     TMPDIR: "/tmp" # required for not using /dev/shm
+     SUBARCH: "x86_64"
+   <<: *do_uml_steps
+
+  i386_uml:
+   docker:
+     - image: lkldocker/circleci-i386:0.1
+   environment:
+     CROSS_COMPILE: ""
+     SUBARCH: "i386"
+     TMPDIR: "/tmp" # required for not using /dev/shm
+   <<: *do_uml_steps
+
+  i386_uml_on_x86_64:
+   docker:
+     - image: lkldocker/circleci-x86_64:0.7
+   environment:
+     CROSS_COMPILE: ""
+     TMPDIR: "/tmp" # required for not using /dev/shm
+     SUBARCH: "i386"
+   <<: *do_uml_steps
+
+  checkpatch:
+   docker:
+     - image: lkldocker/circleci:0.5
+   environment:
+   steps:
+     - restore_cache:
+        key: code-tree-full-history-{{ .Environment.CACHE_VERSION }}
+     - checkout
+     - run: sudo pip install ply
+     - run: tools/lkl/scripts/checkpatch.sh
+     - save_cache:
+        key: code-tree-full-history-{{ .Environment.CACHE_VERSION }}-{{ epoch }}
+        paths:
+          - /home/ubuntu/project/.git
+        when: always
+
+workflows:
+  version: 2
+  build:
+    jobs:
+     - x86_64
+     - x86_64_valgrind
+     - checkpatch
+     - i386
+     - x86_64_uml
+     - i386_uml
+     - i386_uml_on_x86_64
+  nightly:
+    triggers:
+      - schedule:
+          cron: "0 0 * * *"
+          filters:
+            branches:
+              only:
+                - master
+    jobs:
+      - x86_64_valgrind
tools/lkl/scripts/checkpatch.sh new file +60 -0

60 lines not on this page · raw message

tools/lkl/scripts/lkl-jenkins.sh new file +21 -0
new file mode 100755
index 000000000000..eaadc6e90143
@@ -0,0 +1,21 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+set -e
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+basedir=$(cd $script_dir/../../..; pwd)
+
+export PATH=$PATH:/sbin
+
+build_and_test()
+{
+    cd $basedir
+    make mrproper
+    cd tools/lkl
+    make clean-conf
+    make -j4
+    make run-tests
+}
+
+build_and_test
Signature · 1 line
2.21.0 (Apple Git-122.2)
#66 lore raw

RFC v4 23/25 um lkl: add UML network driver for lkl

In reply to Hajime Tazaki

This commit adds the use of the UML drivers of network interfaces which should be able to use any backend configured by boot command line parameters. Currently vector tap backend and (obsolte) slirp and tuntap backends are tested.

Since the UML drivers use LKL irq infrastructure, the build system only picks required object files of UML, and LKL adds trivial glue code and ifdefs into UML code so that existing driver codes can be used for LKL as-is.

Here is a benchmark number with netperf (TCP_STREAM/TCP_MAERTS in Mbps) over 10Gbps ethernet.

|TCP_STREAM | TCP_MAERTS
-------------------- --------------------------

 UMMODE_LIB (um-tap)  | 1932.71    |    8.58
 UMMODE_LIB (vec-tap) | 7565.97    | 9216.49
 UMMODE_LIB (virtio)  | 8743.91    | 9310.35
 UMMODE_KERN (um-tap) | 1819.09    |    0.84
 UMMODE_KERN (vec-tap)| 5418.51    | 9412.07

- setup

We varied client side (netperf) with different net drivers/devices. tso,tx/rx csum are enabled if possible.

               +--docker0--+
               |           |     10GbEth
 netperf +---tap0        eth0 +==========+ eth0 +---+ netserver
 (client)                (ixgbe)          (ixgbe)

<-- Linux box (4.18.5) --> <-- Linux box (4.17.19) -->

- summary
With vector driver over tap backend, UMMODE_LIB can reach similar results with the one of virtio. And outperforming to UMMODE_KERN is the result of simplified user/kernel-space transision (just function calls rather than interposision by ptrace&co).

---
 .circleci/config.yml                    |  2 +
 arch/um/drivers/Makefile                |  2 +
 arch/um/include/asm/irq.h               |  2 +
 arch/um/kernel/Makefile                 |  4 ++
 arch/um/kernel/irq.c                    |  4 ++
 arch/um/lkl/Kconfig                     |  2 +
 arch/um/lkl/configs/lkl_defconfig       |  2 +-
 arch/um/lkl/include/asm/irq.h           |  3 ++
 arch/um/lkl/include/uapi/asm/host_ops.h |  6 +++
 arch/um/lkl/kernel/asm-offsets.c        |  1 +
 arch/um/lkl/kernel/irq.c                | 33 +++++++++++++
 arch/um/lkl/kernel/setup.c              |  4 ++
 arch/um/lkl/mm/bootmem.c                | 33 +++++++++++++
 arch/um/os-Linux/Makefile               |  5 ++
 tools/lkl/Makefile.autoconf             |  2 +
 tools/lkl/include/lkl.h                 |  9 ++++
 tools/lkl/include/lkl_host.h            |  1 +
 tools/lkl/lib/Build                     |  1 +
 tools/lkl/lib/config.c                  |  6 +++
 tools/lkl/lib/net.c                     |  8 ++++
 tools/lkl/lib/posix-host.c              |  3 ++
 tools/lkl/lib/um/Build                  |  2 +
 tools/lkl/lib/um/um_glue.c              | 39 +++++++++++++++
 tools/lkl/lib/um/um_net.c               | 30 ++++++++++++
 tools/lkl/tests/net-setup.sh            | 63 +++++++++++++++++++++++++
 tools/lkl/tests/net-test.c              | 18 +++++--
 tools/lkl/tests/net.sh                  | 45 ++++++++++++++++++
 tools/lkl/tests/run.py                  |  2 +
 28 files changed, 326 insertions(+), 6 deletions(-)
 create mode 100644 tools/lkl/lib/um/Build
 create mode 100644 tools/lkl/lib/um/um_glue.c
 create mode 100644 tools/lkl/lib/um/um_net.c
.circleci/config.yml +2 -0
index 5bdf059014c0..df1554078a4b 100644
@@ -43,6 +43,8 @@ do_steps: &do_steps
   - run:
       name: run tests
       command: |
+        sudo apt-get update
+        sudo apt-get install -y slirp
         mkdir -p ~/junit
         make -C tools/lkl run-tests tests="--junit-dir ~/junit"
       no_output_timeout: "90m"
arch/um/drivers/Makefile +2 -0
index a290821e355c..3cce6c81079e 100644
@@ -37,7 +37,9 @@ $(obj)/vde.o: $(obj)/vde_kern.o $(obj)/vde_user.o
 # When the above is fixed, don't forget to add this too!
 #targets += $(obj)/pcap.o
 
+ifeq ($(UMMODE),kernel)
 obj-y := stdio_console.o fd.o chan_kern.o chan_user.o line.o
+endif
 obj-$(CONFIG_SSL) += ssl.o
 obj-$(CONFIG_STDERR_CONSOLE) += stderr_console.o
 
arch/um/include/asm/irq.h +2 -0
index 42c6205e2dc4..839b86f29b8a 100644
@@ -32,6 +32,8 @@
 
 #endif
 
+#ifdef CONFIG_UMMODE_KERN
 #define NR_IRQS (LAST_IRQ + 1)
+#endif /* CONFIG_UMMODE_KERN */
 
 #endif
arch/um/kernel/Makefile +4 -0
index 5aa882011e04..78b49eb9bf81 100644
@@ -12,12 +12,16 @@ CPPFLAGS_vmlinux.lds := -DSTART=$(LDS_START)		\
                         -DELF_ARCH=$(LDS_ELF_ARCH)	\
                         -DELF_FORMAT=$(LDS_ELF_FORMAT)	\
 			$(LDS_EXTRA)
+ifeq ($(UMMODE),kernel)
 extra-y := vmlinux.lds
 
 obj-y = config.o exec.o exitcode.o irq.o ksyms.o mem.o \
 	physmem.o process.o ptrace.o reboot.o sigio.o \
 	signal.o syscall.o sysrq.o time.o tlb.o trap.o \
 	um_arch.o umid.o maccess.o kmsg_dump.o skas/
+else ifeq ($(UMMODE),library)
+obj-y = irq.o
+endif
 
 obj-$(CONFIG_BLK_DEV_INITRD) += initrd.o
 obj-$(CONFIG_GPROF)	+= gprof_syms.o
arch/um/kernel/irq.c +4 -0
index 3577118bb4a5..26e6bd804f99 100644
@@ -406,6 +406,7 @@ int deactivate_all_fds(void)
 	return 0;
 }
 
+#ifdef CONFIG_UMMODE_KERN
 /*
  * do_IRQ handles all normal device IRQs (the special
  * SMP cross-CPU interrupts have their own specific
@@ -420,6 +421,7 @@ unsigned int do_IRQ(int irq, struct uml_pt_regs *regs)
 	set_irq_regs(old_regs);
 	return 1;
 }
+#endif /* CONFIG_UMMODE_KERN */
 
 void um_free_irq(unsigned int irq, void *dev)
 {
@@ -446,6 +448,7 @@ int um_request_irq(unsigned int irq, int fd, int type,
 
 EXPORT_SYMBOL(um_request_irq);
 
+#ifdef CONFIG_UMMODE_KERN
 /*
  * irq_chip must define at least enable/disable and ack when
  * the edge handler is used.
@@ -597,3 +600,4 @@ unsigned long from_irq_stack(int nested)
 	return mask & ~1;
 }
 
+#endif /* CONFIG_UMMODE_KERN */
arch/um/lkl/Kconfig +2 -0
index c72c40226509..2b6b4063045c 100644
@@ -81,3 +81,5 @@ config HZ
         default 100
 
 endmenu
+
+source "arch/um/drivers/Kconfig"
arch/um/lkl/configs/lkl_defconfig +1 -1
index 7050c233a17e..f2f218a17185 100644
@@ -113,7 +113,7 @@ CONFIG_UML_NET=y
 CONFIG_UML_NET_TUNTAP=y
 # CONFIG_UML_NET_SLIP is not set
 # CONFIG_UML_NET_DAEMON is not set
-# CONFIG_UML_NET_VECTOR is not set
+CONFIG_UML_NET_VECTOR=y
 # CONFIG_UML_NET_VDE is not set
 # CONFIG_UML_NET_MCAST is not set
 # CONFIG_UML_NET_PCAP is not set
arch/um/lkl/include/asm/irq.h +3 -0
index 948fc54cb76c..2ee00ffde21c 100644
@@ -2,6 +2,9 @@
 #ifndef _ASM_LKL_IRQ_H
 #define _ASM_LKL_IRQ_H
 
+/* pull UML's definitions */
+#include "../../../include/asm/irq.h"
+
 #define IRQ_STATUS_BITS		(sizeof(long) * 8)
 #define NR_IRQS			((int)(IRQ_STATUS_BITS * IRQ_STATUS_BITS))
 
arch/um/lkl/include/uapi/asm/host_ops.h +6 -0
index fe4382c3050a..0230885f4f64 100644
@@ -17,6 +17,10 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @um_devices - string containg the list of UML devices in command line
+ * format. This string is appended to the kernel command line and
+ * is provided here for convenience to be implemented by the host library.
+ *
  * @print - optional operation that receives console messages
  *
  * @panic - called during a kernel panic
@@ -74,6 +78,8 @@ struct lkl_jmp_buf {
  * @jmp_buf_longjmp - perform a jump back to the saved jump buffer
  */
 struct lkl_host_operations {
+	const char *um_devices;
+
 	void (*print)(const char *str, int len);
 	void (*panic)(void);
 
arch/um/lkl/kernel/asm-offsets.c +1 -0
index 6be0763698dc..6fdca6df21a2 100644
@@ -1,2 +1,3 @@
 // SPDX-License-Identifier: GPL-2.0
 /* Dummy asm-offsets.c file. Required by kbuild and ready to be used - hint! */
+#include <sysdep/kernel-offsets.h>
arch/um/lkl/kernel/irq.c +33 -0
index c794412f85d9..59e8572862a2 100644
@@ -11,6 +11,11 @@
 #include <asm/host_ops.h>
 #include <asm/cpu.h>
 
+#if defined(__linux) && (defined(__i386) || defined(__x86_64))
+#include <os.h>
+#endif
+void *um_os_signal(int signum, void *handler);
+
 /*
  * To avoid much overhead we use an indirect approach: the irqs are marked using
  * a bitmap (array of longs) and a summary of the modified bits is kept in a
@@ -174,6 +179,16 @@ void arch_local_irq_restore(unsigned long flags)
 	irqs_enabled = flags;
 }
 
+#if defined(__linux) && (defined(__i386) || defined(__x86_64))
+static void sig_handler(int sig)
+{
+	if (sig != SIGIO)
+		return;
+
+	sigio_handler(sig, NULL, NULL);
+}
+#endif
+
 void init_IRQ(void)
 {
 	int i;
@@ -181,6 +196,11 @@ void init_IRQ(void)
 	for (i = 0; i < NR_IRQS; i++)
 		irq_set_chip_and_handler(i, &dummy_irq_chip, handle_simple_irq);
 
+#if defined(__linux) && (defined(__i386) || defined(__x86_64))
+	/* Initialize EPOLL Loop */
+	os_setup_epoll();
+	um_os_signal(SIGIO, sig_handler);
+#endif
 	pr_info("lkl: irqs initialized\n");
 }
 
@@ -188,3 +208,16 @@ void cpu_yield_to_irqs(void)
 {
 	cpu_relax();
 }
+
+#if defined(__linux) && (defined(__i386) || defined(__x86_64))
+unsigned int do_IRQ(int irq, struct uml_pt_regs *regs)
+{
+	/*
+	 * this might be called in signal handler, so dispatch to different
+	 * thread to avoid race
+	 */
+	set_irq_pending(irq);
+
+	return 1;
+}
+#endif
arch/um/lkl/kernel/setup.c +4 -0
index 36c199d3aa22..39b7b7c79581 100644
@@ -61,6 +61,10 @@ int __init lkl_start_kernel(struct lkl_host_operations *ops, const char *fmt,
 	ret = vsnprintf(boot_command_line, COMMAND_LINE_SIZE, fmt, ap);
 	va_end(ap);
 
+	if (ops->um_devices)
+		strscpy(boot_command_line + ret, ops->um_devices,
+			COMMAND_LINE_SIZE - ret);
+
 	memcpy(cmd_line, boot_command_line, COMMAND_LINE_SIZE);
 
 	init_sem = lkl_ops->sem_alloc(0);
arch/um/lkl/mm/bootmem.c +33 -0
index 39dd0d22b44e..3175dadee13f 100644
@@ -64,3 +64,36 @@ void free_mem(void)
 {
 	lkl_ops->mem_free((void *)_memory_start);
 }
+
+void *uml_kmalloc(int size, int flags)
+{
+	return kmalloc(size, flags);
+}
+
+char *uml_strdup(const char *string)
+{
+	return kstrdup(string, GFP_KERNEL);
+}
+
+void free_stack(unsigned long stack, int order)
+{
+	free_pages(stack, order);
+}
+
+unsigned long alloc_stack(int order, int atomic)
+{
+	unsigned long page;
+	gfp_t flags = GFP_KERNEL;
+
+	if (atomic)
+		flags = GFP_ATOMIC;
+	page = __get_free_pages(flags, order);
+
+	return page;
+}
+
+int __cant_sleep(void)
+{
+	return in_atomic() || irqs_disabled() || in_interrupt();
+	/* Is in_interrupt() really needed? */
+}
arch/um/os-Linux/Makefile +5 -0
index 839915b8c31c..a74a2486e178 100644
@@ -6,9 +6,14 @@
 # Don't instrument UML-specific code
 KCOV_INSTRUMENT                := n
 
+ifeq ($(UMMODE),kernel)
 obj-y = execvp.o file.o helper.o irq.o main.o mem.o process.o \
 	registers.o sigio.o signal.o start_up.o time.o tty.o \
 	umid.o user_syms.o util.o drivers/ skas/
+else
+obj-y = execvp.o file.o helper.o irq.o drivers/
+endif
+
 
 obj-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
 
tools/lkl/Makefile.autoconf +2 -0
index 268c367d9962..d2dcaa3b85f2 100644
@@ -41,6 +41,8 @@ define posix_host
   $(if $(strip $(call find_include,archive.h)),$(call set_autoconf_var,ARCHIVE,y))
   $(if $(filter $(1),elf64-x86-64-freebsd),$(call set_autoconf_var,NEEDS_LARGP,y))
   $(if $(filter $(1),elf32-i386),$(call set_autoconf_var,I386,y))
+  $(if $(filter $(1),elf64-x86-64),$(call set_autoconf_var,UML_DEV,y))
+  $(if $(filter $(1),elf32-i386),$(call set_autoconf_var,UML_DEV,y))
 endef
 
 define do_autoconf
tools/lkl/include/lkl.h +9 -0
index 1f4291ad9455..952d11e30868 100644
@@ -731,6 +731,15 @@ struct lkl_netdev_args {
 	unsigned int offload;
 };
 
+#ifdef LKL_HOST_CONFIG_UML_DEV
+struct lkl_netdev *lkl_um_netdev_create(const char *ifparams);
+#else
+static inline struct lkl_netdev *lkl_um_netdev_create(const char *ifparams)
+{
+	return NULL;
+}
+#endif
+
 /*
  * lkl_register_dbg_handler- register a signal handler that loads a debug lib.
  *
tools/lkl/include/lkl_host.h +1 -0
index 4e6d6e031498..7ac30bdcaf0a 100644
@@ -10,6 +10,7 @@ extern "C" {
 #include <lkl.h>
 
 extern struct lkl_host_operations lkl_host_ops;
+extern char lkl_um_devs[4096];
 
 /**
  * lkl_printf - print a message via the host print operation
tools/lkl/lib/Build +1 -0
index 0e711e260a3a..ad8a4b806375 100644
@@ -10,3 +10,4 @@ liblkl-y += dbg.o
 liblkl-y += dbg_handler.o
 liblkl-y += ../../perf/pmu-events/jsmn.o
 liblkl-y += config.o
+liblkl-$(LKL_HOST_CONFIG_UML_DEV) += um/
tools/lkl/lib/config.c +6 -0
index 37f8ac4d942a..db3af68446d3 100644
@@ -470,6 +470,12 @@ static int lkl_config_netdev_create(struct lkl_config *cfg,
 	memset(&nd_args, 0, sizeof(struct lkl_netdev_args));
 
 	if (!nd && iface->iftype && iface->ifparams) {
+		if ((strcmp(iface->iftype, "um") == 0)) {
+			nd = lkl_um_netdev_create(iface->ifparams);
+			iface->nd = nd;
+			/* um_netdev doesn't use virtio device */
+			return 0;
+		}
 	}
 
 	if (nd) {
tools/lkl/lib/net.c +8 -0
index cf6894c35e46..4d4cd2382b2a 100644
@@ -179,6 +179,14 @@ int lkl_netdev_get_ifindex(int id)
 
 	snprintf(ifr.lkl_ifr_name, sizeof(ifr.lkl_ifr_name), "eth%d", id);
 	ret = lkl_sys_ioctl(sock, LKL_SIOCGIFINDEX, (long)&ifr);
+
+	/* retry with vector device */
+	if (ret < 0) {
+		snprintf(ifr.lkl_ifr_name, sizeof(ifr.lkl_ifr_name),
+			 "vec%d",id);
+		ret = lkl_sys_ioctl(sock, LKL_SIOCGIFINDEX, (long)&ifr);
+	}
+
 	lkl_sys_close(sock);
 
 	return ret < 0 ? ret : ifr.lkl_ifr_ifindex;
tools/lkl/lib/posix-host.c +3 -0
index 4be1611a8942..d6394d41e125 100644
@@ -346,6 +346,9 @@ struct lkl_host_operations lkl_host_ops = {
 	.print = print,
 	.mem_alloc = (void *)malloc,
 	.mem_free = free,
+#ifdef LKL_HOST_CONFIG_UML_DEV
+	.um_devices = lkl_um_devs,
+#endif
 	.gettid = _gettid,
 	.jmp_buf_set = jmp_buf_set,
 	.jmp_buf_longjmp = jmp_buf_longjmp,
tools/lkl/lib/um/Build new file +2 -0
new file mode 100644
index 000000000000..09a60975ecd6
@@ -0,0 +1,2 @@
+liblkl-y += um_glue.o
+liblkl-y += um_net.o
tools/lkl/lib/um/um_glue.c new file +39 -0
new file mode 100644
index 000000000000..f59e31a706de
@@ -0,0 +1,39 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+#include <errno.h>
+
+
+
+char lkl_um_devs[4096];
+
+/* from sigio.c */
+void maybe_sigio_broken(int fd, int read)
+{
+}
+
+/* from process.c */
+int os_getpid(void)
+{
+	return getpid();
+}
+
+
+/* from chan_kern.c */
+void free_irqs(void)
+{
+}
+
+/* from sigio.c */
+int ignore_sigio_fd(int fd)
+{
+	return 0;
+}
+
+/* new functions */
+void *um_os_signal(int signum, void *handler)
+{
+	return signal(signum, handler);
+}
tools/lkl/lib/um/um_net.c new file +30 -0
new file mode 100644
index 000000000000..eed2dcd9a91c
@@ -0,0 +1,30 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <lkl_host.h>
+
+static int registered_net_dev_idx;
+
+struct lkl_netdev *lkl_um_netdev_create(const char *ifparams)
+{
+	struct lkl_netdev *nd;
+
+	nd = lkl_host_ops.mem_alloc(sizeof(struct lkl_netdev));
+	if (!nd)
+		return NULL;
+
+	memset(nd, 0, sizeof(struct lkl_netdev));
+
+	nd->id = registered_net_dev_idx++;
+	/* concat strings */
+	/* XXX: better vector device detection ? */
+	if (strncmp(ifparams, "trans", 5) == 0)
+		snprintf(lkl_um_devs + strlen(lkl_um_devs), sizeof(lkl_um_devs),
+			 " vec%d:%s", nd->id, ifparams);
+	else
+		snprintf(lkl_um_devs + strlen(lkl_um_devs), sizeof(lkl_um_devs),
+			 " eth%d=%s", nd->id, ifparams);
+
+	return nd;
+}
tools/lkl/tests/net-setup.sh +63 -0

78 lines not on this page · raw message

tools/lkl/tests/net-test.c +13 -5
index 5a8e39a52417..1050b0df3134 100644
@@ -22,9 +22,11 @@
 
 enum {
 	BACKEND_NONE,
+	BACKEND_UM,
+	BACKEND_UM_VECTOR_TAP,
 };
 
-const char *backends[] = { "loopback", NULL };
+const char *backends[] = { "loopback", "um", "um-vector-tap", NULL };
 static struct {
 	int backend;
 	const char *ifname;
@@ -176,12 +178,18 @@ static int lkl_test_icmp(void)
 }
 
 static struct lkl_netdev *nd;
+static int nd_id;
 
 static int lkl_test_nd_create(void)
 {
 	switch (cla.backend) {
 	case BACKEND_NONE:
 		return TEST_SKIP;
+	case BACKEND_UM:
+	case BACKEND_UM_VECTOR_TAP:
+		nd = lkl_um_netdev_create(cla.ifname);
+		nd_id = nd->id;
+		break;
 	}
 
 	if (!nd) {
@@ -192,11 +200,10 @@ static int lkl_test_nd_create(void)
 	return TEST_SUCCESS;
 }
 
-static int nd_id;
-
 static int lkl_test_nd_add(void)
 {
-	if (cla.backend == BACKEND_NONE)
+	if (cla.backend == BACKEND_NONE || cla.backend == BACKEND_UM
+	    || cla.backend == BACKEND_UM_VECTOR_TAP)
 		return TEST_SKIP;
 
 	return TEST_SUCCESS;
@@ -204,7 +211,8 @@ static int lkl_test_nd_add(void)
 
 static int lkl_test_nd_remove(void)
 {
-	if (cla.backend == BACKEND_NONE)
+	if (cla.backend == BACKEND_NONE || cla.backend == BACKEND_UM
+	    || cla.backend == BACKEND_UM_VECTOR_TAP)
 		return TEST_SKIP;
 
 	return TEST_SUCCESS;
tools/lkl/tests/net.sh +45 -0

65 lines not on this page · raw message

tools/lkl/tests/run.py +2 -0
index b72299aaabee..2cc918a6e11a 100755
@@ -54,6 +54,8 @@ tests = [
     'disk.sh -t vfat',
     'disk.sh -t btrfs',
     'net.sh -b loopback',
+    'net.sh -b um',
+    'net.sh -b um-vector-tap',
     'lklfuse.sh -t ext4',
     'lklfuse.sh -t vfat',
     'lklfuse.sh -t btrfs',
Signature · 1 line
2.21.0 (Apple Git-122.2)
maintainer
#67 lore raw

Re: [RFC v4 23/25] um lkl: add UML network driver for lkl

arch/um/lkl/include/asm/irq.h line 6

Quoted 8 lines

+++ b/arch/um/lkl/include/asm/irq.h

arch/um/lkl/include/asm/irq.h +2 -0
@@ -2,6 +2,9 @@
 #ifndef _ASM_LKL_IRQ_H
 #define _ASM_LKL_IRQ_H
 
+/* pull UML's definitions */
+#include "../../../include/asm/irq.h"

This is _really_ ugly.

arch/um/lkl/kernel/irq.c line 17

Quoted 4 lines
arch/um/lkl/kernel/irq.c +4 -0
+#if defined(__linux) && (defined(__i386) || defined(__x86_64))
+#include <os.h>
+#endif
+void *um_os_signal(int signum, void *handler);

and arguably those random declarations you're sprinkling are worse.

arch/um/lkl/kernel/irq.c void init_IRQ(void) · line 199

Quoted 5 lines
arch/um/lkl/kernel/irq.c +1 -0
@@ -181,6 +196,11 @@ void init_IRQ(void)
 	for (i = 0; i < NR_IRQS; i++)
 		irq_set_chip_and_handler(i, &dummy_irq_chip, handle_simple_irq);
 
+#if defined(__linux) && (defined(__i386) || defined(__x86_64))

What's with all those ifdefs with this condition?

Quoted 13 lines
tools/lkl/lib/um/um_glue.c new file +12 -0
+++ b/tools/lkl/lib/um/um_glue.c
@@ -0,0 +1,39 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+#include <errno.h>
+
+
+
+char lkl_um_devs[4096];
+

tools/lkl/lib/um/um_glue.c line 21

Quoted 10 lines
tools/lkl/lib/um/um_glue.c new file +10 -0
+/* from sigio.c */
+void maybe_sigio_broken(int fd, int read)
+{
+}
+
+/* from process.c */
+int os_getpid(void)
+{
+	return getpid();
+}

All of this really is quite ugly - are you sure it's needed for just the vector network driver??

johannes

#68 lore raw

Re: [RFC v4 23/25] um lkl: add UML network driver for lkl

Thanks for the comments, Johannes.

Quoted 11 lines · Johannes Berg

arch/um/lkl/include/asm/irq.h line 6

Quoted 9 lines

+++ b/arch/um/lkl/include/asm/irq.h

arch/um/lkl/include/asm/irq.h +2 -0
@@ -2,6 +2,9 @@
 #ifndef _ASM_LKL_IRQ_H
 #define _ASM_LKL_IRQ_H
 
+/* pull UML's definitions */
+#include "../../../include/asm/irq.h"

This is _really_ ugly.

Hmm, in previous patchset (until v3), I was using the worse approach (I thought) to avoid this include.

+KBUILD_CFLAGS += -DTIMER_IRQ=0 -DUBD_IRQ=4 -DUM_ETH_IRQ=5 -DLAST_IRQ=15

And I thought the current way is better than before.

Quoted 6 lines

arch/um/lkl/kernel/irq.c line 17

Quoted 4 lines
arch/um/lkl/kernel/irq.c +4 -0
+#if defined(__linux) && (defined(__i386) || defined(__x86_64))
+#include <os.h>
+#endif
+void *um_os_signal(int signum, void *handler);

and arguably those random declarations you're sprinkling are worse.

It means that those are only used when UML drivers (net, block) are enabled, which are only available on Intel/linux hosts. OTOH, UMMODE_LIB _will_ support non-Intel/Linux hosts in the future thus, this ifdefs exist.

Quoted 7 lines

arch/um/lkl/kernel/irq.c void init_IRQ(void) · line 199

Quoted 5 lines
arch/um/lkl/kernel/irq.c +1 -0
@@ -181,6 +196,11 @@ void init_IRQ(void)
 	for (i = 0; i < NR_IRQS; i++)
 		irq_set_chip_and_handler(i, &dummy_irq_chip, handle_simple_irq);
 
+#if defined(__linux) && (defined(__i386) || defined(__x86_64))

What's with all those ifdefs with this condition?

Same as above.
but I agree that the ifdefs are cryptic; I'll try to make it more understandable if I use ifdefs.

Quoted 18 lines
Quoted 18 lines
tools/lkl/lib/um/um_glue.c new file +17 -0
+++ b/tools/lkl/lib/um/um_glue.c
@@ -0,0 +1,39 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+#include <errno.h>
+
+
+
+char lkl_um_devs[4096];
+
+/* from sigio.c */
+void maybe_sigio_broken(int fd, int read)
+{
+}
+
Quoted 8 lines

tools/lkl/lib/um/um_glue.c line 21

Quoted 5 lines
tools/lkl/lib/um/um_glue.c new file +5 -0
+/* from process.c */
+int os_getpid(void)
+{
+	return getpid();
+}

All of this really is quite ugly - are you sure it's needed for just the vector network driver??

Those are needed when we use UML_NET or BLK_DEV_UBD is enabled. I was trying to minimize those glue code as much as possible; I need another try to check if we can remove more or eliminate completely by other ways.

-- Hajime

maintainer
#69 lore raw

Re: [RFC v4 23/25] um lkl: add UML network driver for lkl

Quoted 15 lines · Hajime Tazaki

Thanks for the comments, Johannes.

Quoted 10 lines · Johannes Berg

arch/um/lkl/include/asm/irq.h line 6

Quoted 8 lines

+++ b/arch/um/lkl/include/asm/irq.h

arch/um/lkl/include/asm/irq.h +2 -0
@@ -2,6 +2,9 @@
 #ifndef _ASM_LKL_IRQ_H
 #define _ASM_LKL_IRQ_H
 
+/* pull UML's definitions */
+#include "../../../include/asm/irq.h"

This is _really_ ugly.

Quoted 6 lines

Hmm, in previous patchset (until v3), I was using the worse approach (I thought) to avoid this include.

+KBUILD_CFLAGS += -DTIMER_IRQ=0 -DUBD_IRQ=4 -DUM_ETH_IRQ=5 -DLAST_IRQ=15

And I thought the current way is better than before.

Yeah, ok, that's worse :)

But why is it even needed? It kinda seems to me that this means we're not splitting the code well.

IMHO, if we even want to treat LKL/UML as sub-arches, then we should still split the driver code off in a cleaner way, rather than linking half here half there?

And maybe reorg the code... but I'll reply over in your other email more.

Quoted 11 lines
Quoted 7 lines

arch/um/lkl/kernel/irq.c void init_IRQ(void) · line 199

Quoted 5 lines
arch/um/lkl/kernel/irq.c +1 -0
@@ -181,6 +196,11 @@ void init_IRQ(void)
 	for (i = 0; i < NR_IRQS; i++)
 		irq_set_chip_and_handler(i, &dummy_irq_chip, handle_simple_irq);
 
+#if defined(__linux) && (defined(__i386) || defined(__x86_64))

What's with all those ifdefs with this condition?

Same as above.
but I agree that the ifdefs are cryptic; I'll try to make it more understandable if I use ifdefs.

I'm also generally not convinced that it's a good idea to sprinkle these kinds of ifdefs over the place.

johannes

#70 lore raw

RFC v4 24/25 um lkl: add UML block device driver (ubd) for lkl

In reply to Hajime Tazaki

This commit adds a support to utilize exising UML drivers of block devices for LKL. The goal is similar to networking drivers, add minimum amount of code and use drivers code as-is.

This commit also adds a test code with several filesystems (ext4, vfat, btrfs at the moment) to exercise the basic operations of disk.

---
 arch/um/drivers/ubd_kern.c              |  6 +++++-
 arch/um/lkl/Kconfig                     |  3 +++
 arch/um/lkl/include/uapi/asm/syscalls.h |  1 +
 arch/um/lkl/kernel/setup.c              |  5 +++++
 tools/lkl/include/lkl.h                 | 15 +++++++++++++++
 tools/lkl/lib/um/Build                  |  1 +
 tools/lkl/lib/um/um_block.c             | 17 +++++++++++++++++
 tools/lkl/lib/um/um_glue.c              | 24 +++++++++++++++++++++++-
 tools/lkl/tests/disk.c                  | 14 ++++++++++++++
 9 files changed, 84 insertions(+), 2 deletions(-)
 create mode 100644 tools/lkl/lib/um/um_block.c
arch/um/drivers/ubd_kern.c +5 -1
index eae8c83364f7..248bac8e36b8 100644
@@ -545,7 +545,7 @@ static irqreturn_t ubd_intr(int irq, void *dev)
 /* Only changed by ubd_init, which is an initcall. */
 static int io_pid = -1;
 
-static void kill_io_thread(void)
+void kill_io_thread(void)
 {
 	if(io_pid != -1)
 		os_kill_process(io_pid, 1);
@@ -819,6 +819,7 @@ static int ubd_open_dev(struct ubd *ubd_dev)
 	}
 	ubd_dev->fd = fd;
 
+#ifdef CONFIG_UMMODE_KERN
 	if(ubd_dev->cow.file != NULL){
 		blk_queue_max_hw_sectors(ubd_dev->queue, 8 * sizeof(long));
 
@@ -843,6 +844,7 @@ static int ubd_open_dev(struct ubd *ubd_dev)
 		if(err < 0) goto error;
 		ubd_dev->cow.fd = err;
 	}
+#endif	/* CONFIG_UMMODE_KERN */
 	if (ubd_dev->no_trim == 0) {
 		ubd_dev->queue->limits.discard_granularity = SECTOR_SIZE;
 		ubd_dev->queue->limits.discard_alignment = SECTOR_SIZE;
@@ -852,9 +854,11 @@ static int ubd_open_dev(struct ubd *ubd_dev)
 	}
 	blk_queue_flag_set(QUEUE_FLAG_NONROT, ubd_dev->queue);
 	return 0;
+#ifdef CONFIG_UMMODE_KERN
  error:
 	os_close_file(ubd_dev->fd);
 	return err;
+#endif
 }
 
 static void ubd_device_release(struct device *dev)
arch/um/lkl/Kconfig +3 -0
index 2b6b4063045c..8e4e0a465fc8 100644
@@ -48,6 +48,9 @@ config OUTPUT_FORMAT
 config ARCH_DMA_ADDR_T_64BIT
        def_bool 64BIT
 
+config BLK_DEV_UBD
+       def_bool y if "$(OUTPUT_FORMAT)" = "elf32-i386" || "$(OUTPUT_FORMAT)" = "elf64-x86-64"
+
 config X86_64
        def_bool y if "$(OUTPUT_FORMAT)" = "elf64-x86-64"
 
arch/um/lkl/include/uapi/asm/syscalls.h +1 -0
index a81534ffccb7..1384c34137d6 100644
@@ -164,6 +164,7 @@ struct sockaddr {
 #include <linux/virtio_ring.h>
 #include <linux/pkt_sched.h>
 #include <linux/io_uring.h>
+#include <linux/major.h>
 
 struct user_msghdr {
 	void		__user *msg_name;
arch/um/lkl/kernel/setup.c +5 -0
index 39b7b7c79581..daf55c216db9 100644
@@ -126,6 +126,11 @@ long lkl_sys_halt(void)
 		LINUX_REBOOT_CMD_RESTART,
 	};
 
+#ifdef CONFIG_BLK_DEV_UBD
+void kill_io_thread(void);
+	kill_io_thread();
+#endif
+
 	err = lkl_syscall(__NR_reboot, params);
 	if (err < 0)
 		return err;
tools/lkl/include/lkl.h +15 -0
index 952d11e30868..7d164e4dcad1 100644
@@ -355,6 +355,10 @@ void lkl_perror(char *msg, int err);
  */
 struct lkl_dev_blk_ops;
 
+enum {
+	BLK_BACKEND_UM,
+};
+
 /**
  * lkl_disk - host disk handle
  *
@@ -368,6 +372,7 @@ struct lkl_disk {
 		int fd;
 		void *handle;
 	};
+	int backend;
 	struct lkl_dev_blk_ops *ops;
 };
 
@@ -403,6 +408,16 @@ int lkl_disk_remove(struct lkl_disk disk);
  */
 int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid);
 
+
+#ifdef LKL_HOST_CONFIG_UML_DEV
+int lkl_disk_um_add(struct lkl_disk *disk, const char *blkparams);
+#else
+static inline int lkl_disk_um_add(struct lkl_disk *disk, const char *blkparams)
+{
+	return -LKL_ENOSYS;
+}
+#endif
+
 /**
  * lkl_mount_dev - mount a disk
  *
tools/lkl/lib/um/Build +1 -0
index 09a60975ecd6..52573c05a797 100644
@@ -1,2 +1,3 @@
 liblkl-y += um_glue.o
 liblkl-y += um_net.o
+liblkl-y += um_block.o
tools/lkl/lib/um/um_block.c new file +17 -0
new file mode 100644
index 000000000000..70dc624ec43c
@@ -0,0 +1,17 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <lkl_host.h>
+
+
+static int registered_blk_dev_idx;
+
+int lkl_disk_um_add(struct lkl_disk *disk, const char *blkparams)
+{
+	/* concat strings */
+	snprintf(lkl_um_devs + strlen(lkl_um_devs), sizeof(lkl_um_devs),
+		 " ubd%d=%s", registered_blk_dev_idx, blkparams);
+
+	return registered_blk_dev_idx++;
+}
tools/lkl/lib/um/um_glue.c +23 -1
index f59e31a706de..fd5a338bcb9f 100644
@@ -2,9 +2,10 @@
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
+#include <signal.h>
 #include <unistd.h>
 #include <errno.h>
-
+#include <sys/wait.h>
 
 
 char lkl_um_devs[4096];
@@ -37,3 +38,24 @@ void *um_os_signal(int signum, void *handler)
 {
 	return signal(signum, handler);
 }
+
+/* from util.c */
+/*
+ * UML helper threads must not handle SIGWINCH/INT/TERM
+ */
+void os_fix_helper_signals(void)
+{
+	signal(SIGWINCH, SIG_IGN);
+	signal(SIGINT, SIG_DFL);
+	signal(SIGTERM, SIG_DFL);
+}
+
+void os_kill_process(int pid, int reap_child)
+{
+	kill(pid, SIGKILL);
+	if (reap_child) {
+		while ((errno = 0, ((waitpid(pid, NULL, __WALL)) < 0))
+		       && (errno == EINTR))
+			;
+	}
+}
tools/lkl/tests/disk.c +14 -0
index 0aa039876b54..f7801014d054 100644
@@ -23,12 +23,17 @@ static struct {
 	const char *disk;
 	const char *fstype;
 	int partition;
+	int backend;
 } cla;
 
+const char *backends[] = { "um", NULL };
+
 struct cl_arg args[] = {
 	{"disk", 'd', "disk file to use", 1, CL_ARG_STR, &cla.disk},
 	{"partition", 'P', "partition to mount", 1, CL_ARG_INT, &cla.partition},
 	{"type", 't', "filesystem type", 1, CL_ARG_STR, &cla.fstype},
+	{"backend", 'b', "blockd evice backend type", 1, CL_ARG_STR_SET,
+	 &cla.backend, backends},
 	{0},
 };
 
@@ -50,6 +55,15 @@ int lkl_test_disk_add(void)
 
 	disk.ops = NULL;
 
+	disk.backend = cla.backend;
+
+	switch (disk.backend) {
+	case BLK_BACKEND_UM:
+		disk.dev = (char *)cla.disk;
+	default:
+		break;
+	}
+
 	disk_id = lkl_disk_add(&disk);
 	if (disk_id < 0)
 		goto out_close;
Signature · 1 line
2.21.0 (Apple Git-122.2)
#71 lore raw

RFC v4 25/25 um: fix clone flags to be familiar with valgrind

In reply to Hajime Tazaki

Valgrind reports the following error with UML when ubd driver is compiled in.

==29588== Unsupported clone() flags: 0x500 ==29588==
==29588== The only supported clone() uses are:
==29588== - via a threads library (LinuxThreads or NPTL) ==29588== - via the implementation of fork or vfork

Adding CLONE_FS flags silences this issue.

---
 arch/um/drivers/ubd_user.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
arch/um/drivers/ubd_user.c +2 -1
index a1afe414ce48..fabb50e91c37 100644
@@ -47,7 +47,8 @@ int start_io_thread(unsigned long sp, int *fd_out)
 		goto out_close;
 	}
 
-	pid = clone(io_thread, (void *) sp, CLONE_FILES | CLONE_VM, NULL);
+	pid = clone(io_thread, (void *) sp,
+		    CLONE_FILES | CLONE_VM | CLONE_FS, NULL);
 	if(pid < 0){
 		err = -errno;
 		printk("start_io_thread - clone failed : errno = %d\n", errno);
Signature · 1 line
2.21.0 (Apple Git-122.2)
#72 lore raw

v5 cover Unifying LKL into UML

In reply to Hajime Tazaki

This is another spin of the unification of LKL into UML. Based on the discussion of v4 patchset, we have tried to address issue raised and rewrote the patchset from scratch. The summary is listed in the changelog below.

Although there are still bugs in the patchset, we'd like to ask your opinions on the design we changed.

The milestone section is also updated: this patchset is for the milestone 1, though the common init API is still not implemented yet.

Changes in rfc v5:
- rewrite whole patchset from scratch
- move arch-dependent code of arch/um and arch/x86/um to tools/um
- mainly code under os-Linux/ involved
- introduce 2-stage build (kernel, and host-dependent parts)
- clean up vmlinux.lds.S
- put LKL-specific implementations as a SUBARCH under arch/um/nommu
- introduce !CONFIG_MMU in arch/um
- use struct arch_thread and arch_switch_to() for subarch-specific thread implementation
- integrate with the IRQ infrastructure of UML
- tested with block device drivers (ubd) for the proof

Changes in rfc v4: (https://lwn.net/Articles/816276/)
- Rebase on the current uml/master branch
- Fix IRQ handling (bug fix)
- drop a patch for CONFIG_GENERIC_ATOMIC64 (comment by Peter Zijlstra)
- implement vector net driver for UMMODE_LIB (comment by Anton)
- clean up uapi headers to avoid duplicates
- clean up IRQ handling code (comment by Anton)
- fix error handling in test code (comments by David Disseldorp)

Changes in rfc v3:
- use UML drivers (net, block) from LKL programs
- drop virtio device implementations
- drop mingw32 (Windows) host
- drop android (arm/aarch64) host
- drop FreeBSD (x86_64) host
- drop LD_PRELOAD (hijack) support
- update milestone

rfc v2:
- use UMMODE instead of SUBARCH to switch UML or LKL
- tools/lkl directory is still there. I confirmed we can move under arch/um (e.g., arch/um/lkl/hosts). I will move it IF this is preferable.
- drop several patches involved non-uml directory
- drop several patches which are not required
- refine commit logs
- document updated

LKL (Linux Kernel Library) is aiming to allow reusing the Linux kernel code as extensively as possible with minimal effort and reduced maintenance overhead.

Examples of how LKL can be used are: creating userspace applications (running on Linux and other operating systems) that can read or write Linux filesystems or can use the Linux networking stack, creating kernel drivers for other operating systems that can read Linux filesystems, bootloaders support for reading/writing Linux filesystems, etc.

With LKL, the kernel code is compiled into an object file that can be directly linked by applications. The API offered by LKL is based on the Linux system call interface.

LKL is originally implemented as an architecture port in arch/lkl, but this series of commits tries to integrate this into arch/um as one of the mode of UML. This was discussed during RFC email of LKL (*1).

The latest LKL version can be found at https://github.com/lkl/linux

Milestone
=========
This patches is a first step toward upstreaming *library mode* of Linux kernel, but we think we need to have several steps toward our goal, describing in the below.

Milestone 1: LKL lib on top of UML
* Kernel - Host build split
- Build UML as a relocatable object using the UML's kernel linker script.
- Move the ptrace and other well isolated os code out of arch/um to tools/um
- Use standard host toolchain to create a static library stripped of the ptrace code. Use standard host toolchain to build the main UML executable.
- Add library init API that creates the UML kernel process and starts UML.
* System calls APIs
- Add new system call interface based on UML's irq facility.
- Use the LKL scripts to export the required headers to create system calls APIs that use the UML system calls infrastructure.
- Keep the underlying host and driver operations (threads, irqs, etc.) as they are now in UML.
* Boot test
- Port the LKL boot test to verify that we are able to programatically issue system calls.

Milestone 2: add virtio disk support
* Export asm/io.h operations to host/os. Create IO access operations and redirect them to weak os_ variants that use the current UML implementation.
* Add the LKL IO access layer including generic virtio handling and the virtio block device code.
* Port LKL disk test and disk apps (lklfuse, fs2tar, cptofs)

Milestone 3: new arch ports
* Abstract the system call / IRQ mode the move the implementation to host
* Abstract the thread model and move the implementation to host
* Add LKL thread model and LKL ports

Building LKL the host library and LKL applications
==================================================

% cd tools/um
% make

will build LKL as a object file, it will install it in tools/um/lib together with the headers files in tools/um/include then will build the host library, tests and a few of application examples:

* tests/boot - a simple applications that uses LKL and exercises the basic LKL APIs

% make run-tests

should run the above tests/boot and tests/net-test and report errors if there are any.

Supported hosts
===============

Currently LKL supports Linux userspace applications. New hosts can be added relatively easy if the host supports gcc and GNU ld. Previous versions of LKL supported Windows kernel and Haiku kernel hosts, and we also have WIP patches with rump-hypercall interface, used in UEFI, as well as macOS userspace (part of POSIX).

There is also musl-libc port for LKL, which might be interested in for some folks.

Further readings about LKL
=========================

- Discussion in github LKL issue
https://github.com/lkl/linux/issues/304

- LKL (an article)
https://www.researchgate.net/profile/Nicolae_Tapus2/publication/224164682_LKL_The_Linux_kernel_library/links/02bfe50fd921ab4f7c000000.pdf

*1 RFC email to LKML (back in 2015)
https://www.mail-archive.com/[email protected]/msg1012277.html

Please review the following changes for suitability for inclusion. If you have any objections or suggestions for improvement, please respond to the patches. If you agree with the changes, please provide your Acked-by.

The following changes since commit f6e8c474390be2e6f5bd0b8966e19958214609ff:

um: virtio: Replace zero-length array with flexible-array (2020-06-02 22:38:00 +0200)

are available in the Git repository at:

git://github.com/thehajime/linux b6eab5512a8168bbb46a485e9386e8cbd812b511
https://github.com/thehajime/linux/tree/uml-lkl-5.8rc1-v5

Hajime Tazaki (18):
um: move arch/um/os-Linux dir to tools/um/uml
um: move arch/x86/um/os-Linux to tools/um/uml/
scritps: um: suppress warnings if SRCARCH=um
um: extend arch_switch_to for alternate SUBARCH
um: add nommu mode for UML library mode
um: nommu: host interface
um: nommu: memory handling
um: nommu: kernel thread support
um: nommu: system call interface and application API
um: nommu: basic console support
um: nommu: initialization and cleanup
um: nommu: integrate with irq infrastructure of UML
um: nommu: plug in the build system
um: host: add nommu build for ARCH=um
um: host: add utilities functions
um: host: posix host operations
um: host: add test programs
um: nommu: add block device support of UML

Octavian Purdila (3):
um: split build in kernel and host parts
um: add os init and exit calls
um: host: implement os_initcalls and os_exitcalls

 arch/um/Kconfig                               |  29 +-
 arch/um/Makefile                              |  25 +-
 arch/um/drivers/Makefile                      |  10 +-
 .../um/{os-Linux => }/drivers/ethertap_kern.c |   0
 arch/um/{os-Linux => }/drivers/tuntap_kern.c  |   0
 arch/um/include/asm/common.lds.S              |  99 ----
 arch/um/include/asm/host_ops.h                |   9 +
 arch/um/include/asm/mmu.h                     |   3 +
 arch/um/include/asm/mmu_context.h             |   8 +
 arch/um/include/asm/page.h                    |  15 +
 arch/um/include/asm/pgtable.h                 |  27 +
 arch/um/include/asm/thread_info.h             |  24 +
 arch/um/include/asm/uaccess.h                 |   6 +
 arch/um/include/asm/xor.h                     |   3 +-
 arch/um/include/shared/as-layout.h            |   1 +
 .../drivers => include/shared}/etap.h         |   0
 arch/um/include/shared/init.h                 |  18 +-
 arch/um/include/shared/os.h                   |   1 +
 .../drivers => include/shared}/tuntap.h       |   0
 arch/um/include/uapi/asm/Kbuild               |   2 +
 arch/um/kernel/Makefile                       |  12 +-
 arch/um/kernel/dyn.lds.S                      | 171 -------
 arch/um/kernel/irq.c                          |  13 +
 arch/um/kernel/process.c                      |  14 +-
 arch/um/kernel/reboot.c                       |   5 +
 arch/um/kernel/time.c                         |   2 +
 arch/um/kernel/um_arch.c                      |  16 +
 arch/um/kernel/uml.lds.S                      | 115 -----
 arch/um/kernel/vmlinux.lds.S                  |  91 +++-
 arch/um/nommu/Makefile                        |   1 +
 arch/um/nommu/Makefile.um                     |  18 +
 arch/um/nommu/include/asm/Kbuild              |   6 +
 arch/um/nommu/include/asm/archparam.h         |   1 +
 arch/um/nommu/include/asm/atomic.h            |  11 +
 arch/um/nommu/include/asm/atomic64.h          | 114 +++++
 arch/um/nommu/include/asm/bitsperlong.h       |  12 +
 arch/um/nommu/include/asm/byteorder.h         |   7 +
 arch/um/nommu/include/asm/cpu.h               |  16 +
 arch/um/nommu/include/asm/elf.h               |  15 +
 arch/um/nommu/include/asm/mm_context.h        |   8 +
 arch/um/nommu/include/asm/processor.h         |  46 ++
 arch/um/nommu/include/asm/ptrace.h            |  21 +
 arch/um/nommu/include/asm/sched.h             |  23 +
 arch/um/nommu/include/asm/segment.h           |   9 +
 arch/um/nommu/include/asm/syscall_wrapper.h   |  57 +++
 arch/um/nommu/include/asm/syscalls.h          |  15 +
 arch/um/nommu/include/uapi/asm/Kbuild         |   4 +
 arch/um/nommu/include/uapi/asm/bitsperlong.h  |  11 +
 arch/um/nommu/include/uapi/asm/byteorder.h    |  11 +
 arch/um/nommu/include/uapi/asm/host_ops.h     | 122 +++++
 arch/um/nommu/include/uapi/asm/sigcontext.h   |  12 +
 arch/um/nommu/include/uapi/asm/syscalls.h     | 287 +++++++++++
 arch/um/nommu/include/uapi/asm/unistd.h       |  17 +
 arch/um/nommu/um/Kconfig                      |  45 ++
 arch/um/nommu/um/Makefile                     |   4 +
 arch/um/nommu/um/bootmem.c                    |  86 ++++
 arch/um/nommu/um/console.c                    |  42 ++
 arch/um/nommu/um/cpu.c                        | 247 ++++++++++
 arch/um/nommu/um/delay.c                      |  31 ++
 arch/um/nommu/um/setup.c                      | 178 +++++++
 arch/um/nommu/um/shared/sysdep/archsetjmp.h   |  13 +
 arch/um/nommu/um/shared/sysdep/faultinfo.h    |   8 +
 .../nommu/um/shared/sysdep/kernel-offsets.h   |  12 +
 arch/um/nommu/um/shared/sysdep/mcontext.h     |   9 +
 arch/um/nommu/um/shared/sysdep/ptrace.h       |  42 ++
 arch/um/nommu/um/shared/sysdep/ptrace_user.h  |   7 +
 arch/um/nommu/um/syscalls.c                   | 199 ++++++++
 arch/um/nommu/um/threads.c                    | 261 ++++++++++
 arch/um/nommu/um/unimplemented.c              |  70 +++
 arch/um/nommu/um/user_constants.h             |  13 +
 arch/um/os-Linux/Makefile                     |  19 -
 arch/um/os-Linux/drivers/Makefile             |  13 -
 arch/um/scripts/headers_install.py            | 197 ++++++++
 arch/x86/um/Makefile                          |   2 +-
 arch/x86/um/os-Linux/Makefile                 |  13 -
 arch/x86/um/ptrace_32.c                       |   2 +-
 arch/x86/um/syscalls_64.c                     |   2 +-
 scripts/headers_install.sh                    |   3 +
 scripts/link-vmlinux.sh                       |  42 +-
 tools/um/Makefile                             |  82 ++++
 tools/um/Targets                              |  13 +
 tools/um/include/lkl.h                        | 357 ++++++++++++++
 tools/um/include/lkl_host.h                   |  27 +
 tools/um/lib/Build                            |   7 +
 tools/um/lib/fs.c                             | 461 ++++++++++++++++++
 tools/um/lib/jmp_buf.c                        |  14 +
 tools/um/lib/jmp_buf.h                        |   8 +
 tools/um/lib/posix-host.c                     | 293 +++++++++++
 tools/um/lib/utils.c                          | 213 ++++++++
 tools/um/tests/Build                          |   2 +
 tools/um/tests/boot.c                         | 393 +++++++++++++++
 tools/um/tests/boot.sh                        |   9 +
 tools/um/tests/cla.c                          | 159 ++++++
 tools/um/tests/cla.h                          |  33 ++
 tools/um/tests/disk.c                         | 168 +++++++
 tools/um/tests/disk.sh                        |  70 +++
 tools/um/tests/run.py                         | 174 +++++++
 tools/um/tests/tap13.py                       | 209 ++++++++
 tools/um/tests/test.c                         | 126 +++++
 tools/um/tests/test.h                         |  72 +++
 tools/um/tests/test.sh                        | 181 +++++++
 tools/um/uml/Build                            |  57 +++
 tools/um/uml/drivers/Build                    |  10 +
 .../um/uml}/drivers/ethertap_user.c           |   0
 .../um/uml}/drivers/tuntap_user.c             |   0
 {arch/um/os-Linux => tools/um/uml}/elf_aux.c  |   0
 {arch/um/os-Linux => tools/um/uml}/execvp.c   |   4 -
 {arch/um/os-Linux => tools/um/uml}/file.c     |   0
 {arch/um/os-Linux => tools/um/uml}/helper.c   |   0
 {arch/um/os-Linux => tools/um/uml}/irq.c      |   0
 {arch/um/os-Linux => tools/um/uml}/main.c     |   0
 {arch/um/os-Linux => tools/um/uml}/mem.c      |   0
 tools/um/uml/nommu/Build                      |   1 +
 tools/um/uml/nommu/registers.c                |  21 +
 tools/um/uml/nommu/unimplemented.c            |  21 +
 {arch/um/os-Linux => tools/um/uml}/process.c  |   2 +
 .../um/os-Linux => tools/um/uml}/registers.c  |   0
 {arch/um/os-Linux => tools/um/uml}/sigio.c    |   0
 {arch/um/os-Linux => tools/um/uml}/signal.c   |  12 +-
 .../skas/Makefile => tools/um/uml/skas/Build  |   6 +-
 {arch/um/os-Linux => tools/um/uml}/skas/mem.c |   0
 .../os-Linux => tools/um/uml}/skas/process.c  |   3 +-
 {arch/um/os-Linux => tools/um/uml}/start_up.c |   0
 {arch/um/os-Linux => tools/um/uml}/time.c     |   0
 {arch/um/os-Linux => tools/um/uml}/tty.c      |   0
 {arch/um/os-Linux => tools/um/uml}/umid.c     |   0
 .../um/os-Linux => tools/um/uml}/user_syms.c  |   1 +
 {arch/um/os-Linux => tools/um/uml}/util.c     |  26 +
 tools/um/uml/x86/Build                        |  11 +
 .../os-Linux => tools/um/uml/x86}/mcontext.c  |   0
 .../um/os-Linux => tools/um/uml/x86}/prctl.c  |   0
 .../os-Linux => tools/um/uml/x86}/registers.c |   0
 .../os-Linux => tools/um/uml/x86}/task_size.c |   0
 .../um/os-Linux => tools/um/uml/x86}/tls.c    |   0
 134 files changed, 5846 insertions(+), 523 deletions(-)
 rename arch/um/{os-Linux => }/drivers/ethertap_kern.c (100%)
 rename arch/um/{os-Linux => }/drivers/tuntap_kern.c (100%)
 delete mode 100644 arch/um/include/asm/common.lds.S
 create mode 100644 arch/um/include/asm/host_ops.h
 rename arch/um/{os-Linux/drivers => include/shared}/etap.h (100%)
 rename arch/um/{os-Linux/drivers => include/shared}/tuntap.h (100%)
 create mode 100644 arch/um/include/uapi/asm/Kbuild
 delete mode 100644 arch/um/kernel/dyn.lds.S
 delete mode 100644 arch/um/kernel/uml.lds.S
 create mode 100644 arch/um/nommu/Makefile
 create mode 100644 arch/um/nommu/Makefile.um
 create mode 100644 arch/um/nommu/include/asm/Kbuild
 create mode 100644 arch/um/nommu/include/asm/archparam.h
 create mode 100644 arch/um/nommu/include/asm/atomic.h
 create mode 100644 arch/um/nommu/include/asm/atomic64.h
 create mode 100644 arch/um/nommu/include/asm/bitsperlong.h
 create mode 100644 arch/um/nommu/include/asm/byteorder.h
 create mode 100644 arch/um/nommu/include/asm/cpu.h
 create mode 100644 arch/um/nommu/include/asm/elf.h
 create mode 100644 arch/um/nommu/include/asm/mm_context.h
 create mode 100644 arch/um/nommu/include/asm/processor.h
 create mode 100644 arch/um/nommu/include/asm/ptrace.h
 create mode 100644 arch/um/nommu/include/asm/sched.h
 create mode 100644 arch/um/nommu/include/asm/segment.h
 create mode 100644 arch/um/nommu/include/asm/syscall_wrapper.h
 create mode 100644 arch/um/nommu/include/asm/syscalls.h
 create mode 100644 arch/um/nommu/include/uapi/asm/Kbuild
 create mode 100644 arch/um/nommu/include/uapi/asm/bitsperlong.h
 create mode 100644 arch/um/nommu/include/uapi/asm/byteorder.h
 create mode 100644 arch/um/nommu/include/uapi/asm/host_ops.h
 create mode 100644 arch/um/nommu/include/uapi/asm/sigcontext.h
 create mode 100644 arch/um/nommu/include/uapi/asm/syscalls.h
 create mode 100644 arch/um/nommu/include/uapi/asm/unistd.h
 create mode 100644 arch/um/nommu/um/Kconfig
 create mode 100644 arch/um/nommu/um/Makefile
 create mode 100644 arch/um/nommu/um/bootmem.c
 create mode 100644 arch/um/nommu/um/console.c
 create mode 100644 arch/um/nommu/um/cpu.c
 create mode 100644 arch/um/nommu/um/delay.c
 create mode 100644 arch/um/nommu/um/setup.c
 create mode 100644 arch/um/nommu/um/shared/sysdep/archsetjmp.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/faultinfo.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/kernel-offsets.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/mcontext.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace_user.h
 create mode 100644 arch/um/nommu/um/syscalls.c
 create mode 100644 arch/um/nommu/um/threads.c
 create mode 100644 arch/um/nommu/um/unimplemented.c
 create mode 100644 arch/um/nommu/um/user_constants.h
 delete mode 100644 arch/um/os-Linux/Makefile
 delete mode 100644 arch/um/os-Linux/drivers/Makefile
 create mode 100755 arch/um/scripts/headers_install.py
 delete mode 100644 arch/x86/um/os-Linux/Makefile
 create mode 100644 tools/um/Makefile
 create mode 100644 tools/um/Targets
 create mode 100644 tools/um/include/lkl.h
 create mode 100644 tools/um/include/lkl_host.h
 create mode 100644 tools/um/lib/Build
 create mode 100644 tools/um/lib/fs.c
 create mode 100644 tools/um/lib/jmp_buf.c
 create mode 100644 tools/um/lib/jmp_buf.h
 create mode 100644 tools/um/lib/posix-host.c
 create mode 100644 tools/um/lib/utils.c
 create mode 100644 tools/um/tests/Build
 create mode 100644 tools/um/tests/boot.c
 create mode 100755 tools/um/tests/boot.sh
 create mode 100644 tools/um/tests/cla.c
 create mode 100644 tools/um/tests/cla.h
 create mode 100644 tools/um/tests/disk.c
 create mode 100755 tools/um/tests/disk.sh
 create mode 100755 tools/um/tests/run.py
 create mode 100644 tools/um/tests/tap13.py
 create mode 100644 tools/um/tests/test.c
 create mode 100644 tools/um/tests/test.h
 create mode 100644 tools/um/tests/test.sh
 create mode 100644 tools/um/uml/Build
 create mode 100644 tools/um/uml/drivers/Build
 rename {arch/um/os-Linux => tools/um/uml}/drivers/ethertap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/drivers/tuntap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/elf_aux.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/execvp.c (98%)
 rename {arch/um/os-Linux => tools/um/uml}/file.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/helper.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/irq.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/main.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/mem.c (100%)
 create mode 100644 tools/um/uml/nommu/Build
 create mode 100644 tools/um/uml/nommu/registers.c
 create mode 100644 tools/um/uml/nommu/unimplemented.c
 rename {arch/um/os-Linux => tools/um/uml}/process.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/registers.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/sigio.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/signal.c (96%)
 rename arch/um/os-Linux/skas/Makefile => tools/um/uml/skas/Build (56%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/mem.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/process.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/start_up.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/time.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/tty.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/umid.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/user_syms.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/util.c (89%)
 create mode 100644 tools/um/uml/x86/Build
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/mcontext.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/prctl.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/registers.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/task_size.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/tls.c (100%)
Signature · 1 line
2.21.0 (Apple Git-122.2)
#73 lore raw

RFC v5 1/21 um: split build in kernel and host parts

From: Octavian Purdila <[email protected]>

This patch splits the UML build in two parts: a kernel part that generates a relocatable object (linux.o) and a host build part that links the relocatable object into an executable.

This allows us to simplify the linker script since we can now remove the host bits (e.g. .rel sections). It also gives us greater flexibility since it will be much easier to support other architectures and OSes if we use the standard linker script.

The host build part has been implemented in tools/um so that we can reuse the available host build infrastructure.

The patch also changes the UML build invocation, if before

 $ make ARCH=um defconfig
 $ make ARCH=um

was generating the executable now this only generates the relocatable object.

To fully build UML use:

 $ make -C tools/um

which will generate tools/lkl/linux as the UML executable. To statically compiled UML use:

 $ make -C tools/lkl UML_STATIC=y
---
 arch/um/Kconfig                  |  12 +--
 arch/um/Makefile                 |  14 ++-
 arch/um/include/asm/common.lds.S |  99 ------------------
 arch/um/kernel/dyn.lds.S         | 171 -------------------------------
 arch/um/kernel/uml.lds.S         | 115 ---------------------
 arch/um/kernel/vmlinux.lds.S     |  91 ++++++++++++++--
 scripts/link-vmlinux.sh          |  42 +++-----
 tools/um/Makefile                |  72 +++++++++++++
 tools/um/Targets                 |   4 +
 tools/um/uml/Build               |   0
 10 files changed, 184 insertions(+), 436 deletions(-)
 delete mode 100644 arch/um/include/asm/common.lds.S
 delete mode 100644 arch/um/kernel/dyn.lds.S
 delete mode 100644 arch/um/kernel/uml.lds.S
 create mode 100644 tools/um/Makefile
 create mode 100644 tools/um/Targets
 create mode 100644 tools/um/uml/Build
arch/um/Kconfig +1 -11
index 96ab7026b037..a0a9fd3ab96c 100644
@@ -20,6 +20,7 @@ config UML
 	select GENERIC_CLOCKEVENTS
 	select HAVE_GCC_PLUGINS
 	select TTY # Needed for line.c
+	select MODULE_REL_CRCS if MODVERSIONS
 
 config MMU
 	bool
@@ -79,17 +80,6 @@ config STATIC_LINK
 	  NOTE: This option is incompatible with some networking features which
 	  depend on features that require being dynamically loaded (like NSS).
 
-config LD_SCRIPT_STATIC
-	bool
-	default y
-	depends on STATIC_LINK
-
-config LD_SCRIPT_DYN
-	bool
-	default y
-	depends on !LD_SCRIPT_STATIC
-	select MODULE_REL_CRCS if MODVERSIONS
-
 config HOSTFS
 	tristate "Host filesystem"
 	help
arch/um/Makefile +10 -4
index 275f5ffdf6f0..c952ddefcc1c 100644
@@ -95,14 +95,20 @@ KBUILD_CPPFLAGS += -I$(srctree)/$(HOST_DIR)/include \
 KERNEL_DEFINES = $(strip -Derrno=kernel_errno -Dsigprocmask=kernel_sigprocmask \
 			 -Dmktime=kernel_mktime $(ARCH_KERNEL_DEFINES))
 KBUILD_CFLAGS += $(KERNEL_DEFINES)
+LDFLAGS_vmlinux += -r
 
-PHONY += linux
+PHONY += linux.o
 
-all: linux
+all: linux.o
 
-linux: vmlinux
+linux.o: vmlinux
 	@echo '  LINK $@'
-	$(Q)ln -f $< $@
+	$(Q)$(OBJCOPY) -R .eh_frame $< $@
+
+install: linux.o
+	@echo "  INSTALL $(INSTALL_PATH)/lib/$<"
+	@mkdir -p $(INSTALL_PATH)/lib/
+	@cp $< $(INSTALL_PATH)/lib/
 
 define archhelp
   echo '* linux		- Binary kernel image (./linux) - for backward'
arch/um/include/asm/common.lds.S deleted +0 -99

99 lines not on this page · raw message

arch/um/kernel/dyn.lds.S deleted +0 -171

171 lines not on this page · raw message

arch/um/kernel/uml.lds.S deleted +0 -115

115 lines not on this page · raw message

arch/um/kernel/vmlinux.lds.S +85 -6

93 lines not on this page · raw message

scripts/link-vmlinux.sh +12 -30

48 lines not on this page · raw message

tools/um/Makefile new file +72 -0

72 lines not on this page · raw message

tools/um/Targets new file +4 -0
new file mode 100644
index 000000000000..a4711f1ef422
@@ -0,0 +1,4 @@
+progs-y += uml/linux
+LDLIBS_linux-y := -lrt -lpthread -lutil
+LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
+LDFLAGS_linux-$(UML_STATIC) += -static
tools/um/uml/Build new file +0 -0
new file mode 100644
index 000000000000..e69de29bb2d1
Signature · 1 line
2.21.0 (Apple Git-122.2)
maintainer
#74 lore raw

Re: [RFC v5 01/21] um: split build in kernel and host parts

tools/um/uml/Build

Quoted 757 lines · Hajime Tazaki

From: Octavian Purdila <[email protected]>

This patch splits the UML build in two parts: a kernel part that generates a relocatable object (linux.o) and a host build part that links the relocatable object into an executable.

This allows us to simplify the linker script since we can now remove the host bits (e.g. .rel sections). It also gives us greater flexibility since it will be much easier to support other architectures and OSes if we use the standard linker script.

The host build part has been implemented in tools/um so that we can reuse the available host build infrastructure.

The patch also changes the UML build invocation, if before

  $ make ARCH=um defconfig
  $ make ARCH=um

was generating the executable now this only generates the relocatable object.

To fully build UML use:

  $ make -C tools/um

which will generate tools/lkl/linux as the UML executable. To statically compiled UML use:

  $ make -C tools/lkl UML_STATIC=y
---
  arch/um/Kconfig                  |  12 +--
  arch/um/Makefile                 |  14 ++-
  arch/um/include/asm/common.lds.S |  99 ------------------
  arch/um/kernel/dyn.lds.S         | 171 -------------------------------
  arch/um/kernel/uml.lds.S         | 115 ---------------------
  arch/um/kernel/vmlinux.lds.S     |  91 ++++++++++++++--
  scripts/link-vmlinux.sh          |  42 +++-----
  tools/um/Makefile                |  72 +++++++++++++
  tools/um/Targets                 |   4 +
  tools/um/uml/Build               |   0
  10 files changed, 184 insertions(+), 436 deletions(-)

delete mode 100644 arch/um/include/asm/common.lds.S delete mode 100644 arch/um/kernel/dyn.lds.S delete mode 100644 arch/um/kernel/uml.lds.S create mode 100644 tools/um/Makefile
create mode 100644 tools/um/Targets
create mode 100644 tools/um/uml/Build

arch/um/Kconfig +1 -11
index 96ab7026b037..a0a9fd3ab96c 100644
@@ -20,6 +20,7 @@ config UML
  	select GENERIC_CLOCKEVENTS
  	select HAVE_GCC_PLUGINS
  	select TTY # Needed for line.c
+	select MODULE_REL_CRCS if MODVERSIONS
 
  config MMU
  	bool
@@ -79,17 +80,6 @@ config STATIC_LINK
  	  NOTE: This option is incompatible with some networking features which
  	  depend on features that require being dynamically loaded (like NSS).
 
-config LD_SCRIPT_STATIC
-	bool
-	default y
-	depends on STATIC_LINK
-
-config LD_SCRIPT_DYN
-	bool
-	default y
-	depends on !LD_SCRIPT_STATIC
-	select MODULE_REL_CRCS if MODVERSIONS
-
  config HOSTFS
  	tristate "Host filesystem"
  	help
arch/um/Makefile +10 -4
index 275f5ffdf6f0..c952ddefcc1c 100644
@@ -95,14 +95,20 @@ KBUILD_CPPFLAGS += -I$(srctree)/$(HOST_DIR)/include \
  KERNEL_DEFINES = $(strip -Derrno=kernel_errno -Dsigprocmask=kernel_sigprocmask \
  			 -Dmktime=kernel_mktime $(ARCH_KERNEL_DEFINES))
  KBUILD_CFLAGS += $(KERNEL_DEFINES)
+LDFLAGS_vmlinux += -r
 
-PHONY += linux
+PHONY += linux.o
 
-all: linux
+all: linux.o
 
-linux: vmlinux
+linux.o: vmlinux
  	@echo '  LINK $@'
-	$(Q)ln -f $< $@
+	$(Q)$(OBJCOPY) -R .eh_frame $< $@
+
+install: linux.o
+	@echo "  INSTALL $(INSTALL_PATH)/lib/$<"
+	@mkdir -p $(INSTALL_PATH)/lib/
+	@cp $< $(INSTALL_PATH)/lib/
 
  define archhelp
    echo '* linux		- Binary kernel image (./linux) - for backward'
arch/um/include/asm/common.lds.S deleted +0 -99
deleted file mode 100644
index eca6c452a41b..000000000000
@@ -1,99 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#include <asm-generic/vmlinux.lds.h>
-
-  .fini      : { *(.fini)    } =0x9090
-  _etext = .;
-  PROVIDE (etext = .);
-
-  . = ALIGN(4096);
-  _sdata = .;
-  PROVIDE (sdata = .);
-
-  RO_DATA(4096)
-
-  .unprotected : { *(.unprotected) }
-  . = ALIGN(4096);
-  PROVIDE (_unprotected_end = .);
-
-  . = ALIGN(4096);
-  EXCEPTION_TABLE(0)
-
-  BUG_TABLE
-
-  .uml.setup.init : {
-	__uml_setup_start = .;
-	*(.uml.setup.init)
-	__uml_setup_end = .;
-  }
-	
-  .uml.help.init : {
-	__uml_help_start = .;
-	*(.uml.help.init)
-	__uml_help_end = .;
-  }
-	
-  .uml.postsetup.init : {
-	__uml_postsetup_start = .;
-	*(.uml.postsetup.init)
-	__uml_postsetup_end = .;
-  }
-	
-  .init.setup : {
-	INIT_SETUP(0)
-  }
-
-  PERCPU_SECTION(32)
-	
-  .initcall.init : {
-	INIT_CALLS
-  }
-
-  .con_initcall.init : {
-	CON_INITCALL
-  }
-
-  .exitcall : {
-	__exitcall_begin = .;
-	*(.exitcall.exit)
-	__exitcall_end = .;
-  }
-
-  .uml.exitcall : {
-	__uml_exitcall_begin = .;
-	*(.uml.exitcall.exit)
-	__uml_exitcall_end = .;
-  }
-
-  . = ALIGN(4);
-  .altinstructions : {
-	__alt_instructions = .;
-	*(.altinstructions)
-	__alt_instructions_end = .;
-  }
-  .altinstr_replacement : { *(.altinstr_replacement) }
-  /* .exit.text is discard at runtime, not link time, to deal with references
-     from .altinstructions and .eh_frame */
-  .exit.text : { EXIT_TEXT }
-  .exit.data : { *(.exit.data) }
-
-  .preinit_array : {
-	__preinit_array_start = .;
-	*(.preinit_array)
-	__preinit_array_end = .;
-  }
-  .init_array : {
-	__init_array_start = .;
-	*(.init_array)
-	__init_array_end = .;
-  }
-  .fini_array : {
-	__fini_array_start = .;
-	*(.fini_array)
-	__fini_array_end = .;
-  }
-
-   . = ALIGN(4096);
-  .init.ramfs : {
-	INIT_RAM_FS
-  }
-
arch/um/kernel/dyn.lds.S deleted +0 -171
deleted file mode 100644
index f5001481010c..000000000000
@@ -1,171 +0,0 @@
-#include <asm/vmlinux.lds.h>
-#include <asm/page.h>
-
-OUTPUT_FORMAT(ELF_FORMAT)
-OUTPUT_ARCH(ELF_ARCH)
-ENTRY(_start)
-jiffies = jiffies_64;
-
-SECTIONS
-{
-  PROVIDE (__executable_start = START);
-  . = START + SIZEOF_HEADERS;
-  .interp         : { *(.interp) }
-  __binary_start = .;
-  . = ALIGN(4096);		/* Init code and data */
-  _text = .;
-  INIT_TEXT_SECTION(PAGE_SIZE)
-
-  . = ALIGN(PAGE_SIZE);
-
-  /* Read-only sections, merged into text segment: */
-  .hash           : { *(.hash) }
-  .gnu.hash       : { *(.gnu.hash) }
-  .dynsym         : { *(.dynsym) }
-  .dynstr         : { *(.dynstr) }
-  .gnu.version    : { *(.gnu.version) }
-  .gnu.version_d  : { *(.gnu.version_d) }
-  .gnu.version_r  : { *(.gnu.version_r) }
-  .rel.init       : { *(.rel.init) }
-  .rela.init      : { *(.rela.init) }
-  .rel.text       : { *(.rel.text .rel.text.* .rel.gnu.linkonce.t.*) }
-  .rela.text      : { *(.rela.text .rela.text.* .rela.gnu.linkonce.t.*) }
-  .rel.fini       : { *(.rel.fini) }
-  .rela.fini      : { *(.rela.fini) }
-  .rel.rodata     : { *(.rel.rodata .rel.rodata.* .rel.gnu.linkonce.r.*) }
-  .rela.rodata    : { *(.rela.rodata .rela.rodata.* .rela.gnu.linkonce.r.*) }
-  .rel.data       : { *(.rel.data .rel.data.* .rel.gnu.linkonce.d.*) }
-  .rela.data      : { *(.rela.data .rela.data.* .rela.gnu.linkonce.d.*) }
-  .rel.tdata	  : { *(.rel.tdata .rel.tdata.* .rel.gnu.linkonce.td.*) }
-  .rela.tdata	  : { *(.rela.tdata .rela.tdata.* .rela.gnu.linkonce.td.*) }
-  .rel.tbss	  : { *(.rel.tbss .rel.tbss.* .rel.gnu.linkonce.tb.*) }
-  .rela.tbss	  : { *(.rela.tbss .rela.tbss.* .rela.gnu.linkonce.tb.*) }
-  .rel.ctors      : { *(.rel.ctors) }
-  .rela.ctors     : { *(.rela.ctors) }
-  .rel.dtors      : { *(.rel.dtors) }
-  .rela.dtors     : { *(.rela.dtors) }
-  .rel.got        : { *(.rel.got) }
-  .rela.got       : { *(.rela.got) }
-  .rel.bss        : { *(.rel.bss .rel.bss.* .rel.gnu.linkonce.b.*) }
-  .rela.bss       : { *(.rela.bss .rela.bss.* .rela.gnu.linkonce.b.*) }
-  .rel.plt : {
-	*(.rel.plt)
-	PROVIDE_HIDDEN(__rel_iplt_start = .);
-	*(.rel.iplt)
-	PROVIDE_HIDDEN(__rel_iplt_end = .);
-  }
-  .rela.plt : {
-	*(.rela.plt)
-	PROVIDE_HIDDEN(__rela_iplt_start = .);
-	*(.rela.iplt)
-	PROVIDE_HIDDEN(__rela_iplt_end = .);
-  }
-  .init           : {
-    KEEP (*(.init))
-  } =0x90909090
-  .plt            : { *(.plt) }
-  .text           : {
-    _stext = .;
-    TEXT_TEXT
-    SCHED_TEXT
-    CPUIDLE_TEXT
-    LOCK_TEXT
-    IRQENTRY_TEXT
-    SOFTIRQENTRY_TEXT
-    *(.fixup)
-    *(.stub .text.* .gnu.linkonce.t.*)
-    /* .gnu.warning sections are handled specially by elf32.em.  */
-    *(.gnu.warning)
-
-    . = ALIGN(PAGE_SIZE);
-  } =0x90909090
-  . = ALIGN(PAGE_SIZE);
-  .syscall_stub : {
-	__syscall_stub_start = .;
-	*(.__syscall_stub*)
-	__syscall_stub_end = .;
-  }
-  .fini           : {
-    KEEP (*(.fini))
-  } =0x90909090
-
-  .kstrtab : { *(.kstrtab) }
-
-  #include <asm/common.lds.S>
-
-  __init_begin = .;
-  init.data : { INIT_DATA }
-  __init_end = .;
-
-  /* Ensure the __preinit_array_start label is properly aligned.  We
-     could instead move the label definition inside the section, but
-     the linker would then create the section even if it turns out to
-     be empty, which isn't pretty.  */
-  . = ALIGN(32 / 8);
-  .preinit_array     : { *(.preinit_array) }
-  .init_array     : { *(.init_array) }
-  .fini_array     : { *(.fini_array) }
-  .data           : {
-    INIT_TASK_DATA(KERNEL_STACK_SIZE)
-    . = ALIGN(KERNEL_STACK_SIZE);
-    *(.data..init_irqstack)
-    DATA_DATA
-    *(.data.* .gnu.linkonce.d.*)
-    SORT(CONSTRUCTORS)
-  }
-  .data1          : { *(.data1) }
-  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
-  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
-  .eh_frame       : { KEEP (*(.eh_frame)) }
-  .gcc_except_table   : { *(.gcc_except_table) }
-  .dynamic        : { *(.dynamic) }
-  .ctors          : {
-    /* gcc uses crtbegin.o to find the start of
-       the constructors, so we make sure it is
-       first.  Because this is a wildcard, it
-       doesn't matter if the user does not
-       actually link against crtbegin.o; the
-       linker won't look for a file to match a
-       wildcard.  The wildcard also means that it
-       doesn't matter which directory crtbegin.o
-       is in.  */
-    KEEP (*crtbegin.o(.ctors))
-    /* We don't want to include the .ctor section from
-       from the crtend.o file until after the sorted ctors.
-       The .ctor section from the crtend file contains the
-       end of ctors marker and it must be last */
-    KEEP (*(EXCLUDE_FILE (*crtend.o ) .ctors))
-    KEEP (*(SORT(.ctors.*)))
-    KEEP (*(.ctors))
-  }
-  .dtors          : {
-    KEEP (*crtbegin.o(.dtors))
-    KEEP (*(EXCLUDE_FILE (*crtend.o ) .dtors))
-    KEEP (*(SORT(.dtors.*)))
-    KEEP (*(.dtors))
-  }
-  .jcr            : { KEEP (*(.jcr)) }
-  .got            : { *(.got.plt) *(.got) }
-  _edata = .;
-  PROVIDE (edata = .);
-  .bss            : {
-   __bss_start = .;
-   *(.dynbss)
-   *(.bss .bss.* .gnu.linkonce.b.*)
-   *(COMMON)
-   /* Align here to ensure that the .bss section occupies space up to
-      _end.  Align after .bss to ensure correct alignment even if the
-      .bss section disappears because there are no input sections.  */
-   . = ALIGN(32 / 8);
-  . = ALIGN(32 / 8);
-  }
-   __bss_stop = .;
-  _end = .;
-  PROVIDE (end = .);
-
-  STABS_DEBUG
-
-  DWARF_DEBUG
-
-  DISCARDS
-}
arch/um/kernel/uml.lds.S deleted +0 -115
deleted file mode 100644
index 3b6dab3d4501..000000000000
@@ -1,115 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#include <asm/vmlinux.lds.h>
-#include <asm/page.h>
-
-OUTPUT_FORMAT(ELF_FORMAT)
-OUTPUT_ARCH(ELF_ARCH)
-ENTRY(_start)
-jiffies = jiffies_64;
-
-SECTIONS
-{
-  /* This must contain the right address - not quite the default ELF one.*/
-  PROVIDE (__executable_start = START);
-  /* Static binaries stick stuff here, like the sigreturn trampoline,
-   * invisibly to objdump.  So, just make __binary_start equal to the very
-   * beginning of the executable, and if there are unmapped pages after this,
-   * they are forever unusable.
-   */
-  __binary_start = START;
-
-  . = START + SIZEOF_HEADERS;
-  . = ALIGN(PAGE_SIZE);
-
-  _text = .;
-  INIT_TEXT_SECTION(0)
-
-  .text      :
-  {
-    _stext = .;
-    TEXT_TEXT
-    SCHED_TEXT
-    CPUIDLE_TEXT
-    LOCK_TEXT
-    IRQENTRY_TEXT
-    SOFTIRQENTRY_TEXT
-    *(.fixup)
-    /* .gnu.warning sections are handled specially by elf32.em.  */
-    *(.gnu.warning)
-    *(.gnu.linkonce.t*)
-  }
-
-  . = ALIGN(PAGE_SIZE);
-  .syscall_stub : {
-	__syscall_stub_start = .;
-	*(.__syscall_stub*)
-	__syscall_stub_end = .;
-  }
-
-  /*
-   * These are needed even in a static link, even if they wind up being empty.
-   * Newer glibc needs these __rel{,a}_iplt_{start,end} symbols.
-   */
-  .rel.plt : {
-	*(.rel.plt)
-	PROVIDE_HIDDEN(__rel_iplt_start = .);
-	*(.rel.iplt)
-	PROVIDE_HIDDEN(__rel_iplt_end = .);
-  }
-  .rela.plt : {
-	*(.rela.plt)
-	PROVIDE_HIDDEN(__rela_iplt_start = .);
-	*(.rela.iplt)
-	PROVIDE_HIDDEN(__rela_iplt_end = .);
-  }
-
-  #include <asm/common.lds.S>
-
-  __init_begin = .;
-  init.data : { INIT_DATA }
-  __init_end = .;
-
-  .data    :
-  {
-    INIT_TASK_DATA(KERNEL_STACK_SIZE)
-    . = ALIGN(KERNEL_STACK_SIZE);
-    *(.data..init_irqstack)
-    DATA_DATA
-    *(.gnu.linkonce.d*)
-    CONSTRUCTORS
-  }
-  .data1   : { *(.data1) }
-  .ctors         :
-  {
-    *(.ctors)
-  }
-  .dtors         :
-  {
-    *(.dtors)
-  }
-
-  .got           : { *(.got.plt) *(.got) }
-  .dynamic       : { *(.dynamic) }
-  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
-  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
-  /* We want the small data sections together, so single-instruction offsets
-     can access them all, and initialized data all before uninitialized, so
-     we can shorten the on-disk segment size.  */
-  .sdata     : { *(.sdata) }
-  _edata  =  .;
-  PROVIDE (edata = .);
-  . = ALIGN(PAGE_SIZE);
-  __bss_start = .;
-  PROVIDE(_bss_start = .);
-  SBSS(0)
-  BSS(0)
-   __bss_stop = .;
-  _end = .;
-  PROVIDE (end = .);
-
-  STABS_DEBUG
-
-  DWARF_DEBUG
-
-  DISCARDS
-}
arch/um/kernel/vmlinux.lds.S +85 -6
index 16e49bfa2b42..aaedd66772fa 100644
@@ -1,8 +1,87 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <asm/vmlinux.lds.h>
+#include <asm/thread_info.h>
+#include <asm/page.h>
+#include <asm/thread_info.h>
+#include <asm/cache.h>
+#include <linux/export.h>
 
-KERNEL_STACK_SIZE = 4096 * (1 << CONFIG_KERNEL_STACK_ORDER);
+OUTPUT_FORMAT(ELF_FORMAT)
+jiffies = jiffies_64;
 
-#ifdef CONFIG_LD_SCRIPT_STATIC
-#include "uml.lds.S"
-#else
-#include "dyn.lds.S"
-#endif
+
+SECTIONS
+{
+	__init_begin = . ;
+	HEAD_TEXT_SECTION
+	INIT_TEXT_SECTION(PAGE_SIZE)
+	INIT_DATA_SECTION(16)
+	PERCPU_SECTION(L1_CACHE_BYTES)
+	__init_end = . ;
+
+	_stext = . ;
+	_text = . ;
+	.text : ALIGN(THREAD_SIZE)
+	{
+		__binary_start = . ;
+		TEXT_TEXT
+		SCHED_TEXT
+		LOCK_TEXT
+		CPUIDLE_TEXT
+		IRQENTRY_TEXT
+		SOFTIRQENTRY_TEXT
+	}
+	_etext = . ;
+
+	.syscall_stub : ALIGN(PAGE_SIZE)
+	{
+		__syscall_stub_start = .;
+		*(.__syscall_stub*)
+		__syscall_stub_end = .;
+	}
+
+	_sdata = . ;
+	RO_DATA(PAGE_SIZE)
+	RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE)
+	.data : ALIGN(THREAD_SIZE)  { }
+
+	EXCEPTION_TABLE(16)
+
+	.uml.init_irqstack : ALIGN(THREAD_SIZE) {
+		*(.data..init_irqstack)
+	}
+
+	.uml.setup.init : ALIGN(8) {
+		__uml_setup_start = .;
+		*(.uml.setup.init)
+		__uml_setup_end = .;
+	}
+
+	.uml.help.init : ALIGN(8)  {
+		__uml_help_start = .;
+		*(.uml.help.init)
+		__uml_help_end = .;
+	}
+
+	.uml.postsetup.init : ALIGN(8) {
+		__uml_postsetup_start = .;
+		*(.uml.postsetup.init)
+		__uml_postsetup_end = .;
+	}
+
+	.uml.exitcall : ALIGN(8) {
+		__uml_exitcall_begin = .;
+		*(.uml.exitcall.exit)
+		__uml_exitcall_end = .;
+	}
+	_edata = . ;
+
+	BSS_SECTION(0, 0, 0)
+	_end = . ;
+
+	STABS_DEBUG
+
+	DWARF_DEBUG
+
+	DISCARDS
+}
scripts/link-vmlinux.sh +12 -30
index d09ab4afbda4..0bbb626ace81 100755
@@ -75,36 +75,18 @@ vmlinux_link()
  		strip_debug=-Wl,--strip-debug
  	fi
 
-	if [ "${SRCARCH}" != "um" ]; then
-		objects="--whole-archive			\
-			${KBUILD_VMLINUX_OBJS}			\
-			--no-whole-archive			\
-			--start-group				\
-			${KBUILD_VMLINUX_LIBS}			\
-			--end-group				\
-			${@}"
-
-		${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
-			${strip_debug#-Wl,}			\
-			-o ${output}				\
-			-T ${lds} ${objects}
-	else
-		objects="-Wl,--whole-archive			\
-			${KBUILD_VMLINUX_OBJS}			\
-			-Wl,--no-whole-archive			\
-			-Wl,--start-group			\
-			${KBUILD_VMLINUX_LIBS}			\
-			-Wl,--end-group				\
-			${@}"
-
-		${CC} ${CFLAGS_vmlinux}				\
-			${strip_debug}				\
-			-o ${output}				\
-			-Wl,-T,${lds}				\
-			${objects}				\
-			-lutil -lrt -lpthread
-		rm -f linux
-	fi
+	objects="--whole-archive			\
+		${KBUILD_VMLINUX_OBJS}			\
+		--no-whole-archive			\
+		--start-group				\
+		${KBUILD_VMLINUX_LIBS}			\
+		--end-group				\
+		${@}"
+
+	${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
+		${strip_debug#-Wl,}			\
+		-o ${output}				\
+		-T ${lds} ${objects}
  }
 
  # generate .BTF typeinfo from DWARF debuginfo
tools/um/Makefile new file +72 -0
new file mode 100644
index 000000000000..a63466ef7ef5
@@ -0,0 +1,72 @@
+# Do not use make's built-in rules
+# (this improves performance and avoids hard-to-debug behaviour);
+# also do not print "Entering directory..." messages from make
+.SUFFIXES:
+MAKEFLAGS += -r --no-print-directory
+
+KCONFIG?=defconfig
+
+ifneq ($(silent),1)
+  ifneq ($(V),1)
+	QUIET_AUTOCONF       = @echo '  AUTOCONF '$@;
+	Q = @
+  endif
+endif
+
+PREFIX   := /usr
+
+ifeq (,$(srctree))
+  srctree := $(patsubst %/,%,$(dir $(shell pwd)))
+  srctree := $(patsubst %/,%,$(dir $(srctree)))
+endif
+export srctree
+
+-include ../scripts/Makefile.include
+
+# OUTPUT fixup should be *after* include ../scripts/Makefile.include
+ifneq ($(OUTPUT),)
+  OUTPUT := $(OUTPUT)/tools/um/
+else
+  OUTPUT := $(CURDIR)/
+endif
+export OUTPUT
+export objtree := $(OUTPUT)/../..
+
+
+all:
+
+export CFLAGS += -I$(OUTPUT)/include -Iinclude -Wall -g -O2 -Wextra -fPIC \
+	 -Wno-unused-parameter \
+	 -Wno-missing-field-initializers -fno-strict-aliasing
+
+-include Targets
+
+TARGETS := $(progs-y:%=$(OUTPUT)%)
+TARGETS += $(libs-y:%=$(OUTPUT)%)
+all: $(TARGETS)
+
+# rule to build linux.o
+$(OUTPUT)lib/linux.o:
+	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) $(KCONFIG)
+	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) install INSTALL_PATH=$(OUTPUT)
+
+$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o
+	$(QUIET_AR)$(AR) -rc $@ $^
+
+# rule to link programs
+$(OUTPUT)%: $(OUTPUT)%-in.o $(OUTPUT)liblinux.a
+	$(QUIET_LINK)$(CC) $(LDFLAGS) $(LDFLAGS_$(notdir $*)-y) -o $@ $^ $(LDLIBS) $(LDLIBS_$(notdir $*)-y)
+
+$(OUTPUT)%-in.o: FORCE
+	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
+
+clean:
+	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
+	 -delete -o -name '\.*.d' -delete
+	$(call QUIET_CLEAN, liblinux.a)$(RM) $(OUTPUT)/liblinux.a
+	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
+
+FORCE: ;
+.PHONY: all clean FORCE
+.IGNORE: clean
+.NOTPARALLEL : lib/linux.o
tools/um/Targets new file +4 -0
new file mode 100644
index 000000000000..a4711f1ef422
@@ -0,0 +1,4 @@
+progs-y += uml/linux
+LDLIBS_linux-y := -lrt -lpthread -lutil
+LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
+LDFLAGS_linux-$(UML_STATIC) += -static
tools/um/uml/Build new file +0 -0
new file mode 100644
index 000000000000..e69de29bb2d1

Hi Hajime, hi Octavian,

I am going to have a bit more time to look at UML now. Can you rebase/update the patch versus the latest kernel so I can look at it.

It does not apply at present.

Thanks in advance,

Signature · 8 lines
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/

_______________________________________________
linux-um mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-um
#75 lore raw

Re: [RFC v5 01/21] um: split build in kernel and host parts

Quoted 5 lines · Anton Ivanov

Hi Hajime, hi Octavian,

I am going to have a bit more time to look at UML now. Can you rebase/update the patch versus the latest kernel so I can look at it.

It does not apply at present.

Okay, I'll do it.

Thanks,
-- Hajime

#76 lore raw

RFC v5 2/21 um: add os init and exit calls

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

These calls are added in preparation for moving some of the code from the kernel to the host build.

---
 arch/um/include/shared/init.h | 14 ++++----------
 arch/um/kernel/reboot.c       |  5 +++++
 arch/um/kernel/um_arch.c      | 11 +++++++++++
 3 files changed, 20 insertions(+), 10 deletions(-)
arch/um/include/shared/init.h +4 -10
index c66de434a983..d09308330ca5 100644
@@ -109,19 +109,13 @@ extern struct uml_param __uml_setup_start, __uml_setup_end;
 
 #ifdef __UM_HOST__
 
-#define __define_initcall(level,fn) \
-	static initcall_t __initcall_##fn __used \
-	__attribute__((__section__(".initcall" level ".init"))) = fn
-
-/* Userspace initcalls shouldn't depend on anything in the kernel, so we'll
- * make them run first.
- */
-#define __initcall(fn) __define_initcall("1", fn)
+#undef __uml_exit_call
+#define __uml_exit_call		__used __section(os_exitcalls)
+#define __init_call		__used __section(os_initcalls)
 
+#define __initcall(fn) static initcall_t __initcall_##fn __init_call = fn
 #define __exitcall(fn) static exitcall_t __exitcall_##fn __exit_call = fn
 
-#define __init_call	__used __section(.initcall.init)
-
 #endif
 
 #endif /* _LINUX_UML_INIT_H */
arch/um/kernel/reboot.c +5 -0
index 48c0610d506e..5420aec411f4 100644
@@ -35,10 +35,15 @@ static void kill_off_processes(void)
 	read_unlock(&tasklist_lock);
 }
 
+void __weak os_exitcalls(void)
+{
+}
+
 void uml_cleanup(void)
 {
 	kmalloc_ok = 0;
 	do_uml_exitcalls();
+	os_exitcalls();
 	kill_off_processes();
 }
 
arch/um/kernel/um_arch.c +11 -0
index 0f40eccbd759..78d6042fd2e6 100644
@@ -362,3 +362,14 @@ void __init check_bugs(void)
 void apply_alternatives(struct alt_instr *start, struct alt_instr *end)
 {
 }
+
+int __weak os_initcalls(void)
+{
+	return 0;
+}
+
+int __init run_os_initcalls(void)
+{
+	return os_initcalls();
+}
+__initcall(run_os_initcalls);
Signature · 1 line
2.21.0 (Apple Git-122.2)
#77 lore raw

RFC v5 3/21 um: move arch/um/os-Linux dir to tools/um/uml

In reply to Hajime Tazaki

This patch moves underlying OS dependent part under arch/um to tools/um directory so that arch/um code does not need to build host build facilities (e.g., libc).

---
 arch/um/Makefile                              |  3 +-
 arch/um/drivers/Makefile                      |  2 +
 .../um/{os-Linux => }/drivers/ethertap_kern.c |  0
 arch/um/{os-Linux => }/drivers/tuntap_kern.c  |  0
 .../drivers => include/shared}/etap.h         |  0
 arch/um/include/shared/init.h                 |  4 +-
 .../drivers => include/shared}/tuntap.h       |  0
 arch/um/os-Linux/Makefile                     | 19 --------
 arch/um/os-Linux/drivers/Makefile             | 13 -----
 tools/um/Makefile                             |  6 ++-
 tools/um/uml/Build                            | 48 +++++++++++++++++++
 tools/um/uml/drivers/Build                    | 10 ++++
 .../um/uml}/drivers/ethertap_user.c           |  0
 .../um/uml}/drivers/tuntap_user.c             |  0
 {arch/um/os-Linux => tools/um/uml}/elf_aux.c  |  0
 {arch/um/os-Linux => tools/um/uml}/execvp.c   |  4 --
 {arch/um/os-Linux => tools/um/uml}/file.c     |  0
 {arch/um/os-Linux => tools/um/uml}/helper.c   |  0
 {arch/um/os-Linux => tools/um/uml}/irq.c      |  0
 {arch/um/os-Linux => tools/um/uml}/main.c     |  0
 {arch/um/os-Linux => tools/um/uml}/mem.c      |  0
 {arch/um/os-Linux => tools/um/uml}/process.c  |  0
 .../um/os-Linux => tools/um/uml}/registers.c  |  0
 {arch/um/os-Linux => tools/um/uml}/sigio.c    |  0
 {arch/um/os-Linux => tools/um/uml}/signal.c   |  0
 .../skas/Makefile => tools/um/uml/skas/Build  |  6 +--
 {arch/um/os-Linux => tools/um/uml}/skas/mem.c |  0
 .../os-Linux => tools/um/uml}/skas/process.c  |  3 +-
 {arch/um/os-Linux => tools/um/uml}/start_up.c |  0
 {arch/um/os-Linux => tools/um/uml}/time.c     |  0
 {arch/um/os-Linux => tools/um/uml}/tty.c      |  0
 {arch/um/os-Linux => tools/um/uml}/umid.c     |  0
 .../um/os-Linux => tools/um/uml}/user_syms.c  |  1 +
 {arch/um/os-Linux => tools/um/uml}/util.c     |  0
 34 files changed, 72 insertions(+), 47 deletions(-)
 rename arch/um/{os-Linux => }/drivers/ethertap_kern.c (100%)
 rename arch/um/{os-Linux => }/drivers/tuntap_kern.c (100%)
 rename arch/um/{os-Linux/drivers => include/shared}/etap.h (100%)
 rename arch/um/{os-Linux/drivers => include/shared}/tuntap.h (100%)
 delete mode 100644 arch/um/os-Linux/Makefile
 delete mode 100644 arch/um/os-Linux/drivers/Makefile
 create mode 100644 tools/um/uml/drivers/Build
 rename {arch/um/os-Linux => tools/um/uml}/drivers/ethertap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/drivers/tuntap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/elf_aux.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/execvp.c (98%)
 rename {arch/um/os-Linux => tools/um/uml}/file.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/helper.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/irq.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/main.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/mem.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/process.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/registers.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/sigio.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/signal.c (100%)
 rename arch/um/os-Linux/skas/Makefile => tools/um/uml/skas/Build (56%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/mem.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/process.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/start_up.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/time.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/tty.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/umid.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/user_syms.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/util.c (100%)
arch/um/Makefile +1 -2
index c952ddefcc1c..af0cf64c1428 100644
@@ -24,8 +24,7 @@ OS := $(shell uname -s)
 SHELL := /bin/bash
 
 core-y			+= $(ARCH_DIR)/kernel/		\
-			   $(ARCH_DIR)/drivers/		\
-			   $(ARCH_DIR)/os-$(OS)/
+			   $(ARCH_DIR)/drivers/
 
 MODE_INCLUDE	+= -I$(srctree)/$(ARCH_DIR)/include/shared/skas
 
arch/um/drivers/Makefile +2 -0
index 2a249f619467..ae96c83e312d 100644
@@ -48,6 +48,8 @@ obj-$(CONFIG_UML_NET_VECTOR) += vector.o
 obj-$(CONFIG_UML_NET_VDE) += vde.o
 obj-$(CONFIG_UML_NET_MCAST) += umcast.o
 obj-$(CONFIG_UML_NET_PCAP) += pcap.o
+obj-$(CONFIG_UML_NET_ETHERTAP) += ethertap_kern.o
+obj-$(CONFIG_UML_NET_TUNTAP) += tuntap_kern.o
 obj-$(CONFIG_UML_NET) += net.o 
 obj-$(CONFIG_MCONSOLE) += mconsole.o
 obj-$(CONFIG_MMAPPER) += mmapper_kern.o 
arch/um/drivers/ethertap_kern.c renamed from arch/um/os-Linux/drivers/ethertap_kern.c +0 -0
similarity index 100%
rename from arch/um/os-Linux/drivers/ethertap_kern.c
rename to arch/um/drivers/ethertap_kern.c
arch/um/drivers/tuntap_kern.c renamed from arch/um/os-Linux/drivers/tuntap_kern.c +0 -0
similarity index 100%
rename from arch/um/os-Linux/drivers/tuntap_kern.c
rename to arch/um/drivers/tuntap_kern.c
arch/um/include/shared/etap.h renamed from arch/um/os-Linux/drivers/etap.h +0 -0
similarity index 100%
rename from arch/um/os-Linux/drivers/etap.h
rename to arch/um/include/shared/etap.h
arch/um/include/shared/init.h +3 -1
index d09308330ca5..15b52328e7aa 100644
@@ -41,7 +41,9 @@
 typedef int (*initcall_t)(void);
 typedef void (*exitcall_t)(void);
 
-#include <linux/compiler_types.h>
+//#include <linux/compiler_types.h>
+#define __used                          __attribute__((__used__))
+#define __section(S)                    __attribute__((__section__(#S)))
 
 /* These are for everybody (although not all archs will actually
    discard it in modules) */
arch/um/include/shared/tuntap.h renamed from arch/um/os-Linux/drivers/tuntap.h +0 -0
similarity index 100%
rename from arch/um/os-Linux/drivers/tuntap.h
rename to arch/um/include/shared/tuntap.h
arch/um/os-Linux/Makefile deleted +0 -19
deleted file mode 100644
index 839915b8c31c..000000000000
@@ -1,19 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-# 
-# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
-#
-
-# Don't instrument UML-specific code
-KCOV_INSTRUMENT                := n
-
-obj-y = execvp.o file.o helper.o irq.o main.o mem.o process.o \
-	registers.o sigio.o signal.o start_up.o time.o tty.o \
-	umid.o user_syms.o util.o drivers/ skas/
-
-obj-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
-
-USER_OBJS := $(user-objs-y) elf_aux.o execvp.o file.o helper.o irq.o \
-	main.o mem.o process.o registers.o sigio.o signal.o start_up.o time.o \
-	tty.o umid.o util.o
-
-include arch/um/scripts/Makefile.rules
arch/um/os-Linux/drivers/Makefile deleted +0 -13
deleted file mode 100644
index d79e75f1b69a..000000000000
@@ -1,13 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-# 
-# Copyright (C) 2000, 2002 Jeff Dike ([email protected])
-#
-
-ethertap-objs := ethertap_kern.o ethertap_user.o
-tuntap-objs := tuntap_kern.o tuntap_user.o
-
-obj-y = 
-obj-$(CONFIG_UML_NET_ETHERTAP) += ethertap.o
-obj-$(CONFIG_UML_NET_TUNTAP) += tuntap.o
-
-include arch/um/scripts/Makefile.rules
tools/um/Makefile +5 -1
index a63466ef7ef5..07b9b93ed817 100644
@@ -50,7 +50,7 @@ $(OUTPUT)lib/linux.o:
 	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) $(KCONFIG)
 	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) install INSTALL_PATH=$(OUTPUT)
 
-$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o
+$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o $(OUTPUT)lib/liblinux-in.o
 	$(QUIET_AR)$(AR) -rc $@ $^
 
 # rule to link programs
@@ -60,6 +60,10 @@ $(OUTPUT)%: $(OUTPUT)%-in.o $(OUTPUT)liblinux.a
 $(OUTPUT)%-in.o: FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
+# rule to build objects
+$(OUTPUT)%-in.o: $(OUTPUT)lib/linux.o
+	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
+
 clean:
 	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
 	 -delete -o -name '\.*.d' -delete
tools/um/uml/Build +48 -0

48 lines not on this page · raw message

tools/um/uml/drivers/Build new file +10 -0
new file mode 100644
index 000000000000..c7a8eaa97d72
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (C) 2000, 2002 Jeff Dike ([email protected])
+#
+
+include $(objtree)/include/config/auto.conf
+
+
+liblinux-$(CONFIG_UML_NET_ETHERTAP) += ethertap_user.o
+liblinux-$(CONFIG_UML_NET_TUNTAP) += tuntap_user.o
tools/um/uml/drivers/ethertap_user.c renamed from arch/um/os-Linux/drivers/ethertap_user.c +0 -0
similarity index 100%
rename from arch/um/os-Linux/drivers/ethertap_user.c
rename to tools/um/uml/drivers/ethertap_user.c
tools/um/uml/drivers/tuntap_user.c renamed from arch/um/os-Linux/drivers/tuntap_user.c +0 -0
similarity index 100%
rename from arch/um/os-Linux/drivers/tuntap_user.c
rename to tools/um/uml/drivers/tuntap_user.c
tools/um/uml/elf_aux.c renamed from arch/um/os-Linux/elf_aux.c +0 -0
similarity index 100%
rename from arch/um/os-Linux/elf_aux.c
rename to tools/um/uml/elf_aux.c
tools/um/uml/execvp.c renamed from arch/um/os-Linux/execvp.c +0 -4
similarity index 98%
rename from arch/um/os-Linux/execvp.c
rename to tools/um/uml/execvp.c
index 84a0777c2a45..c105e0e7b778 100644
@@ -26,12 +26,8 @@
 #include <errno.h>
 #include <limits.h>
 
-#ifndef TEST
-#include <um_malloc.h>
-#else
 #include <stdio.h>
 #define um_kmalloc malloc
-#endif
 #include <os.h>
 
 /* Execute FILE, searching in the `PATH' environment variable if it contains
tools/um/uml/file.c renamed from arch/um/os-Linux/file.c +0 -0
similarity index 100%
rename from arch/um/os-Linux/file.c
rename to tools/um/uml/file.c
tools/um/uml/helper.c renamed from arch/um/os-Linux/helper.c +0 -0
similarity index 100%
rename from arch/um/os-Linux/helper.c
rename to tools/um/uml/helper.c
tools/um/uml/irq.c renamed from arch/um/os-Linux/irq.c +0 -0
similarity index 100%
rename from arch/um/os-Linux/irq.c
rename to tools/um/uml/irq.c
tools/um/uml/main.c renamed from arch/um/os-Linux/main.c +0 -0
similarity index 100%
rename from arch/um/os-Linux/main.c
rename to tools/um/uml/main.c
tools/um/uml/mem.c renamed from arch/um/os-Linux/mem.c +0 -0
similarity index 100%
rename from arch/um/os-Linux/mem.c
rename to tools/um/uml/mem.c
tools/um/uml/process.c renamed from arch/um/os-Linux/process.c +0 -0
similarity index 100%
rename from arch/um/os-Linux/process.c
rename to tools/um/uml/process.c
tools/um/uml/registers.c renamed from arch/um/os-Linux/registers.c +0 -0
similarity index 100%
rename from arch/um/os-Linux/registers.c
rename to tools/um/uml/registers.c
tools/um/uml/sigio.c renamed from arch/um/os-Linux/sigio.c +0 -0
similarity index 100%
rename from arch/um/os-Linux/sigio.c
rename to tools/um/uml/sigio.c
tools/um/uml/signal.c renamed from arch/um/os-Linux/signal.c +0 -0
similarity index 100%
rename from arch/um/os-Linux/signal.c
rename to tools/um/uml/signal.c
tools/um/uml/skas/Build renamed from arch/um/os-Linux/skas/Makefile +1 -5
similarity index 56%
rename from arch/um/os-Linux/skas/Makefile
rename to tools/um/uml/skas/Build
index c4566e788815..5fc9d62df863 100644
@@ -3,8 +3,4 @@
 # Copyright (C) 2002 - 2007 Jeff Dike (jdike@{linux.intel,addtoit}.com)
 #
 
-obj-y := mem.o process.o
-
-USER_OBJS := $(obj-y)
-
-include arch/um/scripts/Makefile.rules
+liblinux-y += mem.o process.o
tools/um/uml/skas/mem.c renamed from arch/um/os-Linux/skas/mem.c +0 -0
similarity index 100%
rename from arch/um/os-Linux/skas/mem.c
rename to tools/um/uml/skas/mem.c
tools/um/uml/skas/process.c renamed from arch/um/os-Linux/skas/process.c +1 -2
similarity index 99%
rename from arch/um/os-Linux/skas/process.c
rename to tools/um/uml/skas/process.c
index 4fb877b99dde..fca6a08e81bd 100644
@@ -21,7 +21,6 @@
 #include <registers.h>
 #include <skas.h>
 #include <sysdep/stub.h>
-#include <linux/threads.h>
 
 int is_skas_winch(int pid, int fd, void *data)
 {
@@ -248,7 +247,7 @@ static int userspace_tramp(void *stack)
 	return 0;
 }
 
-int userspace_pid[NR_CPUS];
+int userspace_pid[UM_NR_CPUS];
 
 /**
  * start_userspace() - prepare a new userspace process
tools/um/uml/start_up.c renamed from arch/um/os-Linux/start_up.c +0 -0
similarity index 100%
rename from arch/um/os-Linux/start_up.c
rename to tools/um/uml/start_up.c
tools/um/uml/time.c renamed from arch/um/os-Linux/time.c +0 -0
similarity index 100%
rename from arch/um/os-Linux/time.c
rename to tools/um/uml/time.c
tools/um/uml/tty.c renamed from arch/um/os-Linux/tty.c +0 -0
similarity index 100%
rename from arch/um/os-Linux/tty.c
rename to tools/um/uml/tty.c
tools/um/uml/umid.c renamed from arch/um/os-Linux/umid.c +0 -0
similarity index 100%
rename from arch/um/os-Linux/umid.c
rename to tools/um/uml/umid.c
tools/um/uml/user_syms.c renamed from arch/um/os-Linux/user_syms.c +1 -0
similarity index 99%
rename from arch/um/os-Linux/user_syms.c
rename to tools/um/uml/user_syms.c
index 715594fe5719..4047fd7bfdb7 100644
@@ -13,6 +13,7 @@
 #undef strstr
 #undef memcpy
 #undef memset
+#define EXPORT_SYMBOL(x)
 
 extern size_t strlen(const char *);
 extern void *memmove(void *, const void *, size_t);
tools/um/uml/util.c renamed from arch/um/os-Linux/util.c +0 -0
similarity index 100%
rename from arch/um/os-Linux/util.c
rename to tools/um/uml/util.c
Signature · 1 line
2.21.0 (Apple Git-122.2)
#78 lore raw

RFC v5 4/21 um: host: implement os_initcalls and os_exitcalls

In reply to Hajime Tazaki

From: Octavian Purdila <[email protected]>

This patch implements the init and exit calls for host code. It uses the automatic __start_<section> and __stop_<section> variables that are defined by gcc / ld when using custom sections.

Note that this patch should be merged with "um: move arch/um/os-Linux dir to tools/um" but for now it is separate to make the review easier.

---
 tools/um/uml/util.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)
tools/um/uml/util.c +26 -0
index ecf2f390fad2..4011b36fee7e 100644
@@ -186,3 +186,29 @@ void os_warn(const char *fmt, ...)
 	vfprintf(stderr, fmt, list);
 	va_end(list);
 }
+
+extern void (*__start_os_exitcalls)(void);
+extern void (*__stop_os_exitcalls)(void);
+
+void os_exitcalls(void)
+{
+	exitcall_t *call;
+
+	call = &__stop_os_exitcalls;
+	while (--call >= &__start_os_exitcalls)
+		(*call)();
+}
+
+extern int (*__start_os_initcalls)(void);
+extern int (*__stop_os_initcalls)(void);
+
+int os_initcalls(void)
+{
+	initcall_t *call;
+
+	call = &__stop_os_initcalls;
+	while (--call >= &__start_os_initcalls)
+		(*call)();
+
+	return 0;
+}
Signature · 1 line
2.21.0 (Apple Git-122.2)
#79 lore raw

RFC v5 5/21 um: move arch/x86/um/os-Linux to tools/um/uml/

In reply to Hajime Tazaki

Similar to arch/um/os-Linux code, arch/x86 also contains OS dependent part as well. This patch moves underlying OS dependent part to tools/um directory so that arch/x86 code does not need to build host build facilities (e.g., libc).

---
 arch/x86/um/Makefile                                |  2 +-
 arch/x86/um/os-Linux/Makefile                       | 13 -------------
 tools/um/Makefile                                   |  2 +-
 tools/um/uml/Build                                  |  2 ++
 tools/um/uml/x86/Build                              | 11 +++++++++++
 .../x86/um/os-Linux => tools/um/uml/x86}/mcontext.c |  0
 {arch/x86/um/os-Linux => tools/um/uml/x86}/prctl.c  |  0
 .../um/os-Linux => tools/um/uml/x86}/registers.c    |  0
 .../um/os-Linux => tools/um/uml/x86}/task_size.c    |  0
 {arch/x86/um/os-Linux => tools/um/uml/x86}/tls.c    |  0
 10 files changed, 15 insertions(+), 15 deletions(-)
 delete mode 100644 arch/x86/um/os-Linux/Makefile
 create mode 100644 tools/um/uml/x86/Build
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/mcontext.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/prctl.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/registers.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/task_size.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/tls.c (100%)
arch/x86/um/Makefile +1 -1
index 77f70b969d14..25e9846b36a2 100644
@@ -13,7 +13,7 @@ obj-y = bugs_$(BITS).o delay.o fault.o ldt.o \
 	ptrace_$(BITS).o ptrace_user.o setjmp_$(BITS).o signal.o \
 	stub_$(BITS).o stub_segv.o \
 	sys_call_table_$(BITS).o sysrq_$(BITS).o tls_$(BITS).o \
-	mem_$(BITS).o subarch.o os-$(OS)/
+	mem_$(BITS).o subarch.o
 
 ifeq ($(CONFIG_X86_32),y)
 
arch/x86/um/os-Linux/Makefile deleted +0 -13
deleted file mode 100644
index 253bfb8cb702..000000000000
@@ -1,13 +0,0 @@
-#
-# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
-# Licensed under the GPL
-#
-
-obj-y = registers.o task_size.o mcontext.o
-
-obj-$(CONFIG_X86_32) += tls.o
-obj-$(CONFIG_64BIT) += prctl.o
-
-USER_OBJS := $(obj-y)
-
-include arch/um/scripts/Makefile.rules
tools/um/Makefile +1 -1
index 07b9b93ed817..552ed5f1edae 100644
@@ -61,7 +61,7 @@ $(OUTPUT)%-in.o: FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
 # rule to build objects
-$(OUTPUT)%-in.o: $(OUTPUT)lib/linux.o
+$(OUTPUT)%-in.o: $(OUTPUT)lib/linux.o FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
 clean:
tools/um/uml/Build +2 -0
index 8e0607ee8b0a..435a9670d3ff 100644
@@ -12,6 +12,8 @@ liblinux-y += execvp.o file.o helper.o irq.o main.o mem.o process.o \
 	registers.o sigio.o signal.o start_up.o time.o tty.o \
 	umid.o user_syms.o util.o drivers/ skas/
 
+liblinux-y += x86/
+
 liblinux-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
 
 export O = $(srctree)
tools/um/uml/x86/Build new file +11 -0
new file mode 100644
index 000000000000..16bbc7eb19d1
@@ -0,0 +1,11 @@
+#
+# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
+# Licensed under the GPL
+#
+
+include $(objtree)/include/config/auto.conf
+
+liblinux-y = registers.o task_size.o mcontext.o
+
+liblinux-$(CONFIG_X86_32) += tls.o
+liblinux-$(CONFIG_64BIT) += prctl.o
tools/um/uml/x86/mcontext.c renamed from arch/x86/um/os-Linux/mcontext.c +0 -0
similarity index 100%
rename from arch/x86/um/os-Linux/mcontext.c
rename to tools/um/uml/x86/mcontext.c
tools/um/uml/x86/prctl.c renamed from arch/x86/um/os-Linux/prctl.c +0 -0
similarity index 100%
rename from arch/x86/um/os-Linux/prctl.c
rename to tools/um/uml/x86/prctl.c
tools/um/uml/x86/registers.c renamed from arch/x86/um/os-Linux/registers.c +0 -0
similarity index 100%
rename from arch/x86/um/os-Linux/registers.c
rename to tools/um/uml/x86/registers.c
tools/um/uml/x86/task_size.c renamed from arch/x86/um/os-Linux/task_size.c +0 -0
similarity index 100%
rename from arch/x86/um/os-Linux/task_size.c
rename to tools/um/uml/x86/task_size.c
tools/um/uml/x86/tls.c renamed from arch/x86/um/os-Linux/tls.c +0 -0
similarity index 100%
rename from arch/x86/um/os-Linux/tls.c
rename to tools/um/uml/x86/tls.c
Signature · 1 line
2.21.0 (Apple Git-122.2)
#80 lore raw

RFC v5 6/21 scritps: um: suppress warnings if SRCARCH=um

In reply to Hajime Tazaki

This commit fixes numbers of warning messages about leaked CONFIG options. nommu mode of UML requires copies of kernel headers to offer syscall-like API for the library users. Thus, the warnings are to be avoided to function this exposure of API.

---
 scripts/headers_install.sh | 3 +++
 1 file changed, 3 insertions(+)
scripts/headers_install.sh +3 -0
index a07668a5c36b..59966f3b4ebe 100755
@@ -99,6 +99,9 @@ include/uapi/linux/raw.h:CONFIG_MAX_RAW_DEVS
 for c in $configs
 do
 	warn=1
+	if [ "$SRCARCH" = "um" ] ; then
+		break
+	fi
 
 	for ignore in $config_leak_ignores
 	do
Signature · 1 line
2.21.0 (Apple Git-122.2)
#81 lore raw

RFC v5 7/21 um: extend arch_switch_to for alternate SUBARCH

In reply to Hajime Tazaki

This commit introduces additional argument of previous task when context switch happens. New SUBARCH can use the new information to switch tasks in a subarch-specific manner.

The patch is particularly required by nommu mode implemented as a SUBARCH of UML.

---
 arch/um/kernel/process.c  | 6 +++---
 arch/x86/um/ptrace_32.c   | 2 +-
 arch/x86/um/syscalls_64.c | 2 +-
 3 files changed, 5 insertions(+), 5 deletions(-)
arch/um/kernel/process.c +3 -3
index cbe33af2a880..e5238a42ea17 100644
@@ -81,7 +81,7 @@ static inline void set_current(struct task_struct *task)
 		{ external_pid(), task });
 }
 
-extern void arch_switch_to(struct task_struct *to);
+extern void arch_switch_to(struct task_struct *from, struct task_struct *to);
 
 void *__switch_to(struct task_struct *from, struct task_struct *to)
 {
@@ -89,7 +89,7 @@ void *__switch_to(struct task_struct *from, struct task_struct *to)
 	set_current(to);
 
 	switch_threads(&from->thread.switch_buf, &to->thread.switch_buf);
-	arch_switch_to(current);
+	arch_switch_to(from, to);
 
 	return current->thread.prev_sched;
 }
@@ -146,7 +146,7 @@ void fork_handler(void)
 	 * arch_switch_to isn't needed. We could want to apply this to
 	 * improve performance. -bb
 	 */
-	arch_switch_to(current);
+	arch_switch_to(NULL, current);
 
 	current->thread.prev_sched = NULL;
 
arch/x86/um/ptrace_32.c +1 -1
index 2497bac56066..0f184710d4ca 100644
@@ -11,7 +11,7 @@
 
 extern int arch_switch_tls(struct task_struct *to);
 
-void arch_switch_to(struct task_struct *to)
+void arch_switch_to(struct task_struct *from, struct task_struct *to)
 {
 	int err = arch_switch_tls(to);
 	if (!err)
arch/x86/um/syscalls_64.c +1 -1
index 58f51667e2e4..2ef9474d2bd2 100644
@@ -80,7 +80,7 @@ SYSCALL_DEFINE2(arch_prctl, int, option, unsigned long, arg2)
 	return arch_prctl(current, option, (unsigned long __user *) arg2);
 }
 
-void arch_switch_to(struct task_struct *to)
+void arch_switch_to(struct task_struct *from, struct task_struct *to)
 {
 	if ((to->thread.arch.fs == 0) || (to->mm == NULL))
 		return;
Signature · 1 line
2.21.0 (Apple Git-122.2)