Popcorn Linux Distributed Thread Execution
Subsystems (14)
cover Popcorn Linux Distributed Thread Execution
This patch set adds the Popcorn Distributed Thread Execution support to the kernel. It is based off of Linux 5.2 commit 72a20ce. We are looking for feedback on design and implementation from the community.
Background
==========
Popcorn Linux is a collaborative work by the Software and Systems Research Group at Virginia Tech. It is based on the original thesis by David Katz. Principal contributors since include, Sang-Hoon Kim, Maria Sadini Ajith Saya, Vicnent Legout, Antonio Barbalace and Binoy Ravindran.
Popcorn Linux is a Linux kernel-based software stack that enables applications to execute, with a shared code base, on distributed hosts. Popcorn allows applications to start execution on a particular host and migrate, at run-time, to a remote host. Multi-threaded applications may migrate any particular thread to any remote host.
Unlike userspace checkpoint-restart solutions (e.g., CRIU), Popcorn enables seamless and dynamic migration across hosts during execution (no user interaction), and ensures coherent virtual memory across hosts for concurrent thread execution.
Popcorn Linux implements a software-based distributed shared memory by extending Linux's virtual memory subsystem and it enables processes on different machines to observe a common and coherent virtual address space. Coherency of virtual memory pages of different hosts is ensured using a reader-replicate/writer-invalidate, page-level consistency protocol.
The version of Popcorn Linux presented in this RFC supports only x86 configurations. The stack in this RFC includes a modified kernel and a userspace run-time library.
There is a more advanced version of Popcorn Linux that allows applications to concurrently execute across ISA-different cores (.e.g. X86, ARM). This feature-rich version adds a customized LLVM-compiler toolchain to the stack. Nevertheless, this RFC focuses on a simpler single-architecture form of Popcorn that does not require compiler modifications.
Both the compiler toolchain and the heterogeneous implementation of Popcorn Linux may be found at these github locations. It should be noted the heterogeneous version is under active development and is likely less stable than the one provided in this patch.
Heterogeneous Popcorn Linux
https://github.com/ssrg-vt/popcorn-kernel
LLVM Compiler
https://github.com/ssrg-vt/popcorn-compiler
More information on the Popcorn Linux research team and their current work may be found here:
Popcorn Library
===============
The Popcorn kernel library is a suite of light tests that showcase the use of Popcorn's core system calls. These tests allow for a basic migration of processes and threads between 2 or more nodes.
The test suite is being expanded and may be found in at
https://github.com/ssrg-vt/popcorn-kernel-lib. The branch matching this RFC kernel code base is called "upstream".
Reverting L1TF protections
==========================
This initial iteration of code currently reverts L1TF side channel protections on x86 systems. Future iterations of code will comply with the L1TF patches and current pagetable walking algorithms.
Security Disclaimer
===================
Popcorn Linux assumes that it is operating across equally-trusted host systems. This patch-set is intended to initiate discussions and should not be built and loaded onto public (internet-connected) machines. Popcorn Linux, as-is, runs a kernel-based daemon that listens for messages passed to it via a TCP socket. This IP-based message layer is intended for Popcorn testing and development purposes only, given obvious latency and security issues stemming from passing pages and kernel structures over TCP.
Andrew Hughes (9):
Core Popcorn Changes Popcorn Linux
Add x86 specifc files for Popcorn
Temporarily revert L1TF mitigation for Popcorn Popcorn system call additions
Popcorn Utility
Process Server for Popcorn Distributed Thread Execution Virtual Memory Address Server for Distributed Thread Execution Page Server for Distributed Thread Execution
Add Popcorn Message Layer and socket support
arch/x86/Kconfig | 3 + arch/x86/entry/syscalls/syscall_64.tbl | 3 + arch/x86/include/asm/pgtable-2level.h | 17 - arch/x86/include/asm/pgtable-3level.h | 2 - arch/x86/include/asm/pgtable.h | 52 +- arch/x86/include/asm/pgtable_64.h | 2 - arch/x86/kernel/Makefile | 1 + arch/x86/kernel/process_server.c | 250 +++ arch/x86/mm/fault.c | 18 + arch/x86/mm/mmap.c | 21 - drivers/msg_layer/Kconfig | 28 + drivers/msg_layer/Makefile | 2 + drivers/msg_layer/common.h | 63 + drivers/msg_layer/socket.c | 710 +++++++++ fs/proc/base.c | 9 + fs/read_write.c | 15 +- include/linux/mm_types.h | 12 + include/linux/sched.h | 27 +- include/linux/syscalls.h | 9 + include/popcorn/bundle.h | 38 + include/popcorn/debug.h | 38 + include/popcorn/page_server.h | 34 + include/popcorn/pcn_kmsg.h | 205 +++ include/popcorn/process_server.h | 18 + include/popcorn/regset.h | 96 ++ include/popcorn/stat.h | 16 + include/popcorn/types.h | 20 + include/popcorn/vma_server.h | 33 + include/uapi/asm-generic/mman-common.h | 4 + include/uapi/asm-generic/unistd.h | 11 +- kernel/Kconfig.popcorn | 54 + kernel/Makefile | 1 + kernel/exit.c | 9 + kernel/fork.c | 51 +- kernel/futex.c | 32 + kernel/popcorn/Makefile | 7 + kernel/popcorn/bundle.c | 115 ++ kernel/popcorn/fh_action.c | 207 +++ kernel/popcorn/fh_action.h | 34 + kernel/popcorn/init.c | 58 + kernel/popcorn/page_server.c | 2019 ++++++++++++++++++++++++ kernel/popcorn/page_server.h | 16 + kernel/popcorn/pcn_kmsg.c | 231 +++ kernel/popcorn/pgtable.h | 31 + kernel/popcorn/process_server.c | 1037 ++++++++++++ kernel/popcorn/process_server.h | 21 + kernel/popcorn/stat.c | 165 ++ kernel/popcorn/trace_events.h | 76 + kernel/popcorn/types.h | 358 +++++ kernel/popcorn/util.c | 121 ++ kernel/popcorn/util.h | 14 + kernel/popcorn/vma_server.c | 818 ++++++++++ kernel/popcorn/vma_server.h | 24 + kernel/popcorn/wait_station.c | 84 + kernel/popcorn/wait_station.h | 27 + kernel/sched/core.c | 106 +- kernel/sys_ni.c | 3 + mm/gup.c | 18 + mm/internal.h | 4 + mm/madvise.c | 51 + mm/memory.c | 236 ++- mm/mmap.c | 53 + mm/mprotect.c | 70 +- mm/mremap.c | 20 + 64 files changed, 7763 insertions(+), 165 deletions(-) create mode 100644 arch/x86/kernel/process_server.c create mode 100644 drivers/msg_layer/Kconfig create mode 100644 drivers/msg_layer/Makefile create mode 100644 drivers/msg_layer/common.h create mode 100644 drivers/msg_layer/socket.c create mode 100644 include/popcorn/bundle.h create mode 100644 include/popcorn/debug.h create mode 100644 include/popcorn/page_server.h create mode 100644 include/popcorn/pcn_kmsg.h create mode 100644 include/popcorn/process_server.h create mode 100644 include/popcorn/regset.h create mode 100644 include/popcorn/stat.h create mode 100644 include/popcorn/types.h create mode 100644 include/popcorn/vma_server.h create mode 100644 kernel/Kconfig.popcorn create mode 100644 kernel/popcorn/Makefile create mode 100644 kernel/popcorn/bundle.c create mode 100644 kernel/popcorn/fh_action.c create mode 100644 kernel/popcorn/fh_action.h create mode 100644 kernel/popcorn/init.c create mode 100644 kernel/popcorn/page_server.c create mode 100644 kernel/popcorn/page_server.h create mode 100644 kernel/popcorn/pcn_kmsg.c create mode 100644 kernel/popcorn/pgtable.h create mode 100644 kernel/popcorn/process_server.c create mode 100644 kernel/popcorn/process_server.h create mode 100644 kernel/popcorn/stat.c create mode 100644 kernel/popcorn/trace_events.h create mode 100644 kernel/popcorn/types.h create mode 100644 kernel/popcorn/util.c create mode 100644 kernel/popcorn/util.h create mode 100644 kernel/popcorn/vma_server.c create mode 100644 kernel/popcorn/vma_server.h create mode 100644 kernel/popcorn/wait_station.c create mode 100644 kernel/popcorn/wait_station.h
Signature · 1 line
2.17.1
RFC 1/9 Core Popcorn Changes
Popcorn Linux is a Linux kernel-based software stack that enables applications to execute, with a shared source base, on distributed hosts.
To achieve its goal of distributed multi-threaded applications, Popcorn introduces certain core modifications to the Linux kernel. These include the addition of several system calls, a message layer and the Popcorn implementation itself.
System Calls
In this RFC there are three system calls associated with the migration mechanism.
popcorn_migrate: allows user to migrate a thread to another popcorn registered node.
popcorn_get_node_info: get status information on Popcorn registered nodes.
popcorn_get_thread_status: get status information of current distributed thread.
Message Layer
Popcorn views compute resources as nodes. Each Popcorn node may be multiple cores running an instance of the Linux kernel. Each node registers itself in the network via the message layer TCP/IP socket. Popcorn processes communicate with each other using Popcorn specific messages. Popcorn messages are used for the VMA coherency protocol, managing the necessary distributed locks, signaling process migration and exit.
Popcorn Implementation
The heart of Popcorn's implementation resides in kernel/popcorn. Popcorn implements a main kernel thread to execute process migration from its origin node to remote nodes. A pair of work queues are used to process incoming messages and requests.
Work is tracked via a remote_context struct introduced by Popcorn Linux. This struct also contains necessary information to implement the VMA coherency protocol. As such, this struct is embedded in task_struct and mm_struct and forms part of the memory management modifications necessary to achieve dynamic thread migration.
We welcome feedback to these core modifications. And look forward to an open and productive discussion of the complete Popcorn Linux work.
--- fs/proc/base.c | 9 ++ fs/read_write.c | 15 +- include/linux/mm_types.h | 12 ++ include/linux/sched.h | 27 +++- include/uapi/asm-generic/mman-common.h | 4 + kernel/Kconfig.popcorn | 54 +++++++ kernel/Makefile | 1 + kernel/exit.c | 9 ++ kernel/fork.c | 51 ++++++- kernel/futex.c | 32 ++++ kernel/sched/core.c | 106 +++++++++++++- kernel/sys_ni.c | 3 + mm/gup.c | 18 +++ mm/internal.h | 4 + mm/madvise.c | 51 +++++++ mm/memory.c | 195 ++++++++++++++++++++++++- mm/mmap.c | 53 +++++++ mm/mprotect.c | 21 ++- mm/mremap.c | 20 +++ 19 files changed, 679 insertions(+), 6 deletions(-) create mode 100644 kernel/Kconfig.popcorn
fs/proc/base.c +9 -0
@@ -88,6 +88,9 @@#include <linux/user_namespace.h> #include <linux/fs_struct.h> #include <linux/slab.h> +#ifdef CONFIG_POPCORN +#include <popcorn/types.h> +#endif #include <linux/sched/autogroup.h> #include <linux/sched/mm.h> #include <linux/sched/coredump.h>
@@ -345,6 +348,12 @@ static ssize_t proc_pid_cmdline_read(struct file *file, char __user *buf, tsk =get_proc_task (file_inode (file));if (!tsk)return -ESRCH ; +#ifdef CONFIG_POPCORN +if (distributed_remote_process (tsk)) { +put_task_struct (tsk); +return 0 ; + } +#endif ret =get_task_cmdline (tsk, buf, count, pos);put_task_struct (tsk);if (ret >0 )
fs/read_write.c +14 -1
@@ -573,11 +573,18 @@ static inline loff_t *file_ppos(struct file *file)return file->f_mode &FMODE_STREAM ?NULL : &file->f_pos; } +#ifdef CONFIG_POPCORN_CHECK_SANITY +#include <popcorn/types.h> +#endif ssize_t ksys_read (unsigned int fd,char __user *buf,size_t count) {struct fd f =fdget_pos (fd);ssize_t ret = -EBADF ; - +#ifdef CONFIG_POPCORN_CHECK_SANITY +if (WARN_ON (distributed_remote_process (current))) { +printk (" file read at remote thread is not supported yet\n" ); + } +#endif if (f.file) {loff_t pos, *ppos =file_ppos (f.file);if (ppos) {
@@ -602,6 +609,12 @@ ssize_t ksys_write(unsigned int fd, const char __user *buf, size_t count)struct fd f =fdget_pos (fd);ssize_t ret = -EBADF ; +#ifdef CONFIG_POPCORN_CHECK_SANITY +if (WARN_ON (distributed_remote_process (current))) { +printk (" file write at remote thread is not supported yet\n" ); + } +#endif +if (f.file) {loff_t pos, *ppos =file_ppos (f.file);if (ppos) {
include/linux/mm_types.h +12 -0
@@ -17,6 +17,10 @@#include <asm/mmu.h> +#ifdef CONFIG_POPCORN +struct remote_context ; +#endif +#ifndef AT_VECTOR_SIZE_ARCH #define AT_VECTOR_SIZE_ARCH 0#endif
@@ -505,6 +509,10 @@ struct mm_struct {/* HMM needs to track a few things per mm */ struct hmm *hmm;#endif +#ifdef CONFIG_POPCORN +struct remote_context *remote; +#endif + } __randomize_layout;/*
@@ -670,6 +678,10 @@ enum vm_fault_reason {VM_FAULT_DONE_COW = (__force vm_fault_t )0x001000 ,VM_FAULT_NEEDDSYNC = (__force vm_fault_t )0x002000 ,VM_FAULT_HINDEX_MASK = (__force vm_fault_t )0x0f0000 , +#ifdef CONFIG_POPCORN +VM_FAULT_CONTINUE = (__force vm_fault_t )0x004000 , + VM_FAULT_KILLED = (__force vm_fault_t)0x008000, +#endif };/* Encode hstate index for a hwpoisoned large page */
include/linux/sched.h +26 -1
@@ -29,7 +29,9 @@#include <linux/mm_types_task.h> #include <linux/task_io_accounting.h> #include <linux/rseq.h> - +#ifdef CONFIG_POPCORN +#include <linux/completion.h> +#endif /* task_struct member predeclarations (sorted alphabetically): */ struct audit_context ;struct backing_dev_info ;
@@ -1177,6 +1179,29 @@ struct task_struct {unsigned long task_state_change;#endif int pagefault_disabled; +#ifdef CONFIG_POPCORN +struct remote_context *remote; +union { +int peer_nid; +int remote_nid; +int origin_nid; + }; +union { +pid_t peer_pid; +pid_t remote_pid; +pid_t origin_pid; + }; + +bool is_worker;/* kernel thread that manages the process*/ +bool at_remote;/* Is executing on behalf of another node? */ + +volatile void *remote_work; +struct completion remote_work_pended; + +int migration_target_nid; +int backoff_weight; +#endif +#ifdef CONFIG_MMU struct task_struct *oom_reaper_list;#endif
include/uapi/asm-generic/mman-common.h +4 -0
@@ -64,6 +64,10 @@#define MADV_WIPEONFORK 18/* Zero memory on fork, child only */ #define MADV_KEEPONFORK 19/* Undo MADV_WIPEONFORK */ +#ifdef CONFIG_POPCORN +#define MADV_RELEASE 20 +#endif +/* compatibility flags */ #define MAP_FILE 0
kernel/Kconfig.popcorn new file +54 -0
54 lines not on this page · raw message
kernel/Makefile +1 -0
@@ -109,6 +109,7 @@ obj-$(CONFIG_CRASH_DUMP) += crash_dump.o obj-$(CONFIG_JUMP_LABEL) += jump_label.o obj-$(CONFIG_CONTEXT_TRACKING) += context_tracking.o obj-$(CONFIG_TORTURE_TEST) += torture.o +obj-$(CONFIG_POPCORN) += popcorn/ obj-$(CONFIG_HAS_IOMEM) += iomem.o obj-$(CONFIG_ZONE_DEVICE) += memremap.o
kernel/exit.c +9 -0
@@ -69,6 +69,10 @@#include <asm/pgtable.h> #include <asm/mmu_context.h> +#ifdef CONFIG_POPCORN +#include <popcorn/process_server.h> +#endif +static void __unhash_process (struct task_struct *p,bool group_dead) { nr_threads--;
@@ -503,6 +507,11 @@ static void exit_mm(void)if (!mm)return ;sync_mm_rss (mm); + +#ifdef CONFIG_POPCORN +process_server_task_exit (current); +#endif +/* * Serialize with any possible pending coredump. * We must hold mmap_sem around checking core_state
kernel/fork.c +50 -1
81 lines not on this page · raw message
kernel/futex.c +32 -0
@@ -59,6 +59,12 @@#include <asm/futex.h> +#ifdef CONFIG_POPCORN +#include <popcorn/types.h> +#include <popcorn/process_server.h> +#include <popcorn/page_server.h> +#endif +#include "locking/rtmutex_common.h" /*
@@ -2684,6 +2690,9 @@ static int futex_wait(u32 __user *uaddr, unsigned int flags, u32 val,struct futex_hash_bucket *hb;struct futex_q q = futex_q_init;int ret; +#ifdef CONFIG_POPCORN +struct fault_handle *fh =NULL ; +#endif if (!bitset)return -EINVAL ;
@@ -2701,11 +2710,19 @@ static int futex_wait(u32 __user *uaddr, unsigned int flags, u32 val, } retry: +#ifdef CONFIG_POPCORN + ret =page_server_get_userpage (uaddr, &fh,"wait" ); +if (ret <0 ) + goto out; +#endif /* * Prepare to wait on uaddr. On success, holds hb lock and increments * q.key refs. */ ret =futex_wait_setup (uaddr, val, flags, &q, &hb); +#ifdef CONFIG_POPCORN +page_server_put_userpage (fh,"wait" ); +#endif if (ret) goto out;
@@ -3629,6 +3646,15 @@ long do_futex(u32 __user *uaddr, int op, u32 val, ktime_t *timeout,return -ENOSYS ; } +#ifdef CONFIG_POPCORN +if (distributed_process (current)) { +WARN_ON (cmd !=FUTEX_WAIT && + cmd !=FUTEX_WAIT_BITSET && + cmd !=FUTEX_WAKE && + cmd !=FUTEX_WAKE_BITSET ); + } +#endif +switch (cmd) {case FUTEX_WAIT : val3 =FUTEX_BITSET_MATCH_ANY ;
@@ -3695,6 +3721,12 @@ SYSCALL_DEFINE6(futex, u32 __user *, uaddr, int, op, u32, val, cmd == FUTEX_CMP_REQUEUE_PI || cmd == FUTEX_WAKE_OP) val2 = (u32 ) (unsigned long ) utime; +#ifdef CONFIG_POPCORN +if (distributed_remote_process (current)) { +return process_server_do_futex_at_remote ( + uaddr, op, val, tp ? true : false, &ts, uaddr2, val2, val3); + } +#endif return do_futex (uaddr, op, val, tp, uaddr2, val2, val3); }
kernel/sched/core.c +105 -1
124 lines not on this page · raw message
kernel/sys_ni.c +3 -0
@@ -166,6 +166,9 @@ COND_SYSCALL(syslog);/* kernel/ptrace.c */ /* kernel/sched/core.c */ +COND_SYSCALL (popcorn_migrate); +COND_SYSCALL (popcorn_get_node_info); +COND_SYSCALL (popcorn_get_thread_status);/* kernel/sys.c */ COND_SYSCALL (setregid);
mm/gup.c +18 -0
@@ -22,6 +22,11 @@#include <asm/pgtable.h> #include <asm/tlbflush.h> +#ifdef CONFIG_POPCORN +#include <popcorn/process_server.h> +#include <popcorn/vma_server.h> +#endif +#include "internal.h" struct follow_page_context {
@@ -969,6 +974,19 @@ int fixup_user_fault(struct task_struct *tsk, struct mm_struct *mm, retry: vma =find_extend_vma (mm, address); +#ifdef CONFIG_POPCORN +if (distributed_remote_process (tsk)) { +if (!vma || address < vma->vm_start) { +if (vma_server_fetch_vma (tsk, address) ==0 ) { +/* Replace with updated VMA */ + vma =find_extend_vma (mm, address); + }else { +return -ENOMEM ; + } + } + } +#endif +if (!vma || address < vma->vm_start)return -EFAULT ;
mm/internal.h +4 -0
@@ -9,6 +9,10 @@#include <linux/fs.h> #include <linux/mm.h> + +#ifdef CONFIG_POPCORN +#include <popcorn/types.h> +#endif #include <linux/pagemap.h> #include <linux/tracepoint-defs.h>
mm/madvise.c +51 -0
86 lines not on this page · raw message
mm/memory.c +194 -1
227 lines not on this page · raw message
mm/mmap.c +53 -0
98 lines not on this page · raw message
mm/mprotect.c +20 -1
@@ -24,6 +24,11 @@#include <linux/mmu_notifier.h> #include <linux/migrate.h> #include <linux/perf_event.h> +#ifdef CONFIG_POPCORN +#include <popcorn/types.h> +#include <popcorn/vma_server.h> +#endif +#include <linux/pkeys.h> #include <linux/ksm.h> #include <linux/uaccess.h>
@@ -479,7 +484,13 @@ static int do_mprotect_pkey(unsigned long start, size_t len,return -ENOMEM ;if (!arch_validate_prot (prot, start))return -EINVAL ; - +#ifdef CONFIG_POPCORN +if (distributed_remote_process (current)) { + error =vma_server_mprotect_remote (start, len, prot); +if (error) +return error; + } +#endif reqprot = prot;if (down_write_killable (¤t->mm->mmap_sem))
@@ -582,6 +593,14 @@ SYSCALL_DEFINE3(mprotect, unsigned long, start, size_t, len,return do_mprotect_pkey (start, len, prot,-1 ); } +#ifdef CONFIG_POPCORN +long ksys_mprotect (unsigned long start,size_t len, +unsigned long prot) +{ +return __do_sys_mprotect (start, len, prot); +} +#endif +#ifdef CONFIG_ARCH_HAS_PKEYS SYSCALL_DEFINE4(pkey_mprotect, unsigned long, start, size_t, len,
mm/mremap.c +20 -0
@@ -30,6 +30,11 @@#include "internal.h" +#ifdef CONFIG_POPCORN +#include <popcorn/types.h> +#include <popcorn/vma_server.h> +#endif +static pmd_t *get_old_pmd (struct mm_struct *mm,unsigned long addr) {pgd_t *pgd;
@@ -617,6 +622,12 @@ SYSCALL_DEFINE5(mremap, unsigned long, addr, unsigned long, old_len, old_len =PAGE_ALIGN (old_len); new_len =PAGE_ALIGN (new_len); +#ifdef CONFIG_POPCORN +if (distributed_remote_process (current)) { +vma_server_mremap_remote (addr, old_len, new_len, flags, new_addr); + } +#endif +/* * We allow a zero old-len as a special case
@@ -727,3 +738,12 @@ SYSCALL_DEFINE5(mremap, unsigned long, addr, unsigned long, old_len,userfaultfd_unmap_complete (mm, &uf_unmap);return ret; } + +#ifdef CONFIG_POPCORN +long ksys_mremap (unsigned long addr, +unsigned long old_len,unsigned long new_len, +unsigned long flags,unsigned long new_addr) +{ +return __do_sys_mremap (addr, old_len, new_len, flags, new_addr); +} +#endif
Signature · 1 line
2.17.1
RFC 2/9 Add x86 specifc files for Popcorn
In reply to Javier MalavePopcorn Linux is a Linux kernel-based software stack that enables applications to execute, with a shared source base, on distributed hosts.
This Popcorn patch adds x86_64 functionality only. Future iterations of Popcorn will support heterogeneous architectures.
--- arch/x86/Kconfig | 3 +++ arch/x86/include/asm/pgtable.h | 11 +++++++++++ arch/x86/kernel/Makefile | 1 + arch/x86/mm/fault.c | 18 ++++++++++++++++++ 4 files changed, 33 insertions(+)
arch/x86/Kconfig +3 -0
@@ -88,6 +88,7 @@ config X86 select ARCH_SUPPORTS_ACPI select ARCH_SUPPORTS_ATOMIC_RMW select ARCH_SUPPORTS_NUMA_BALANCING if X86_64 + select ARCH_SUPPORTS_POPCORN if X86_64 select ARCH_USE_BUILTIN_BSWAP select ARCH_USE_QUEUED_RWLOCKS select ARCH_USE_QUEUED_SPINLOCKS
@@ -353,6 +354,8 @@ config PGTABLE_LEVELS default 3 if X86_PAE default 2 +source "kernel/Kconfig.popcorn" + config CC_HAS_SANE_STACKPROTECTOR bool default $(success,$(srctree)/scripts/gcc-x86_64-has-stack-protector.sh $(CC)) if 64BIT
arch/x86/include/asm/pgtable.h +11 -0
@@ -210,9 +210,13 @@ static inline u64 protnone_mask(u64 val);static inline unsigned long pte_pfn (pte_t pte) { +#ifdef CONFIG_POPCORN +return (pte_val (pte) &PTE_PFN_MASK ) >>PAGE_SHIFT ; +#else phys_addr_t pfn =pte_val (pte); pfn ^=protnone_mask (pfn);return (pfn &PTE_PFN_MASK ) >>PAGE_SHIFT ; +#endif }static inline unsigned long pmd_pfn (pmd_t pmd)
@@ -602,7 +606,11 @@ static inline u64 flip_protnone_guard(u64 oldval, u64 val, u64 mask);static inline pte_t pte_modify (pte_t pte,pgprot_t newprot) { +#ifndef CONFIG_POPCORN pteval_t val =pte_val (pte), oldval = val; +#else +pteval_t val =pte_val (pte); +#endif /* * Chop off the NX bit (if present), and add the NX portion of
@@ -610,8 +618,11 @@ static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)*/ val &= _PAGE_CHG_MASK; val |=check_pgprot (newprot) & ~_PAGE_CHG_MASK; +#ifndef CONFIG_POPCORN val =flip_protnone_guard (oldval, val,PTE_PFN_MASK ); +#endif return __pte (val); + }static inline pmd_t pmd_modify (pmd_t pmd,pgprot_t newprot)
arch/x86/kernel/Makefile +1 -0
@@ -133,6 +133,7 @@ obj-$(CONFIG_EFI) += sysfb_efi.o obj-$(CONFIG_PERF_EVENTS) += perf_regs.o obj-$(CONFIG_TRACING) += tracepoint.o +obj-$(CONFIG_POPCORN) += process_server.o obj-$(CONFIG_SCHED_MC_PRIO) += itmt.o obj-$(CONFIG_X86_INTEL_UMIP) += umip.o
arch/x86/mm/fault.c +18 -0
@@ -33,6 +33,11 @@#define CREATE_TRACE_POINTS #include <asm/trace/exceptions.h> +#ifdef CONFIG_POPCORN +#include <popcorn/types.h> +#include <popcorn/vma_server.h> +#endif +/* * Returns 0 if mmiotrace is disabled, or if the fault is not * handled by mmiotrace:
@@ -1416,6 +1421,19 @@ void do_user_addr_fault(struct pt_regs *regs, } vma =find_vma (mm, address); +#ifdef CONFIG_POPCORN +/* vma worker should not fault */ +BUG_ON (tsk->is_worker); + +if (distributed_remote_process (tsk)) { +if (!vma || vma->vm_start > address) { +if (vma_server_fetch_vma (tsk, address) ==0 ) { +/* Replace with updated VMA */ + vma =find_vma (mm, address); + } + } + } +#endif if (unlikely (!vma)) {bad_area (regs, hw_error_code, address);return ;
Signature · 1 line
2.17.1
RFC 3/9 Temporary revert L1TF mitigation for Popcorn
In reply to Javier MalavePopcorn Linux is a Linux kernel-based software stack that enables applications to execute, with a shared source base, on distributed hosts.
To ensure correct functionality across hosts and focus feedback during the RFC process we have reverted L1TF mitigations for x86 mitigations temporarily. Future iterations of Popcorn will comply with the L1TF mitigations.
--- arch/x86/include/asm/pgtable-2level.h | 17 -------- arch/x86/include/asm/pgtable-3level.h | 2 - arch/x86/include/asm/pgtable.h | 59 +++++---------------------- arch/x86/include/asm/pgtable_64.h | 2 - arch/x86/mm/mmap.c | 21 ---------- mm/memory.c | 41 +++++++------------ mm/mprotect.c | 49 ---------------------- 7 files changed, 25 insertions(+), 166 deletions(-)
arch/x86/include/asm/pgtable-2level.h +0 -17
@@ -95,21 +95,4 @@ static inline unsigned long pte_bitop(unsigned long value, unsigned int rightshi#define __pte_to_swp_entry (pte) ((swp_entry_t) { (pte).pte_low })#define __swp_entry_to_pte (x) ((pte_t) { .pte = (x).val }) -/* No inverted PFNs on 2 level page tables */ - -static inline u64 protnone_mask (u64 val) -{ -return 0 ; -} - -static inline u64 flip_protnone_guard (u64 oldval,u64 val,u64 mask) -{ -return val; -} - -static inline bool __pte_needs_invert (u64 val) -{ -return false; -} -#endif /* _ASM_X86_PGTABLE_2LEVEL_H */
arch/x86/include/asm/pgtable-3level.h +0 -2
@@ -332,6 +332,4 @@ static inline pte_t gup_get_pte(pte_t *ptep) return pte; } -#include <asm/pgtable-invert.h> -#endif /* _ASM_X86_PGTABLE_3LEVEL_H */
arch/x86/include/asm/pgtable.h +11 -48
119 lines not on this page · raw message
arch/x86/include/asm/pgtable_64.h +0 -2
@@ -272,7 +272,5 @@ static inline bool gup_fast_permitted(unsigned long start, int nr_pages)return true; } -#include <asm/pgtable-invert.h> -#endif /* !__ASSEMBLY__ */ #endif /* _ASM_X86_PGTABLE_64_H */
arch/x86/mm/mmap.c +0 -21
@@ -227,24 +227,3 @@ int valid_mmap_phys_addr_range(unsigned long pfn, size_t count)return phys_addr_valid (addr + count -1 ); } - -/* -* Only allow root to set high MMIO mappings to PROT_NONE. -* This prevents an unpriv. user to set them to PROT_NONE and invert -* them, then pointing to valid memory for L1TF speculation. -* -* Note: for locked down kernels may want to disable the root override. -*/ -bool pfn_modify_allowed (unsigned long pfn,pgprot_t prot) -{ -if (!boot_cpu_has_bug (X86_BUG_L1TF )) -return true; -if (!__pte_needs_invert (pgprot_val (prot))) -return true; -/* If it's real memory always allow */ -if (pfn_valid (pfn)) -return true; -if (pfn >=l1tf_pfn_limit () && !capable (CAP_SYS_ADMIN )) -return false; -return true; -}
mm/memory.c +14 -27
115 lines not on this page · raw message
mm/mprotect.c +0 -49
61 lines not on this page · raw message
Signature · 1 line
2.17.1
RFC 4/9 Popcorn system call additions
In reply to Javier MalaveThe Popcorn system calls are a core component of Popcorn Linux. All system calls have been added to kernel/sched/core.c
The main system call is popcorn_migrate. User applications may invoke this system call to trigger a migration from their origin node to a remote node; and vice-versa. The system call relies on several "server modules" (process server, vma server, page server) to perform the migration and maintain VMA coherency. A message layer for IPC has been also added to communicate Popcorn messages across the distributed threads. You may find a basic example of the system call at the Popcorn kernel library. All three system calls are showcased in the Popcorn kernel library.
--- arch/x86/entry/syscalls/syscall_64.tbl | 3 +++ include/linux/syscalls.h | 9 +++++++++ include/uapi/asm-generic/unistd.h | 11 +++++++++-- 3 files changed, 21 insertions(+), 2 deletions(-)
arch/x86/entry/syscalls/syscall_64.tbl +3 -0
@@ -355,6 +355,9 @@ 431 common fsconfig __x64_sys_fsconfig 432 common fsmount __x64_sys_fsmount 433 common fspick __x64_sys_fspick +434 64 popcorn_migrate __x64_sys_popcorn_migrate +435 64 popcorn_get_thread_status __x64_sys_popcorn_get_thread_status +436 64 popcorn_get_node_info __x64_sys_popcorn_get_node_info # # x32-specific system call numbers start at 512 to avoid cache impact
include/linux/syscalls.h +9 -0
@@ -1250,6 +1250,15 @@ ssize_t ksys_pread64(unsigned int fd, char __user *buf, size_t count,ssize_t ksys_pwrite64 (unsigned int fd,const char __user *buf,size_t count,loff_t pos);int ksys_fallocate (int fd,int mode,loff_t offset,loff_t len); +#ifdef CONFIG_POPCORN +long ksys_brk (unsigned long brk); +long ksys_mremap (unsigned long addr, +unsigned long old_len,unsigned long new_len, +unsigned long flags,unsigned long new_addr); +long ksys_madvise (unsigned long start,size_t len,int behavior); +long ksys_mprotect (unsigned long start,size_t len, +unsigned long prot); +#endif #ifdef CONFIG_ADVISE_SYSCALLS int ksys_fadvise64_64 (int fd,loff_t offset,loff_t len,int advice);#else
include/uapi/asm-generic/unistd.h +9 -2
@@ -844,9 +844,16 @@ __SYSCALL(__NR_fsconfig, sys_fsconfig) __SYSCALL(__NR_fsmount, sys_fsmount)#define __NR_fspick 433 __SYSCALL(__NR_fspick, sys_fspick) - +#ifdef CONFIG_POPCORN +#define __NR_popcorn_migrate 434 +__SYSCALL(__NR_popcorn_migrate, sys_popcorn_migrate) +#define __NR_popcorn_get_thread_status 435 +__SYSCALL(__NR_popcorn_get_thread_status, sys_popcorn_get_thread_status) +#define __NR_popcorn_get_node_info 436 +__SYSCALL(__NR_popcorn_get_node_info, sys_popcorn_get_node_info) +#endif #undef __NR_syscalls -#define __NR_syscalls 434 +#define __NR_syscalls 437/* * 32 bit systems traditionally used different
Signature · 1 line
2.17.1
RFC 5/9 Popcorn Utility
In reply to Javier MalaveThis patch contains utility and other
Popcorn supplementary modules, including
initialization routines.
--- include/popcorn/bundle.h | 38 ++++ include/popcorn/debug.h | 38 ++++ include/popcorn/regset.h | 96 +++++++++ include/popcorn/stat.h | 16 ++ include/popcorn/types.h | 20 ++ kernel/popcorn/Makefile | 7 + kernel/popcorn/bundle.c | 115 +++++++++++ kernel/popcorn/init.c | 58 ++++++ kernel/popcorn/stat.c | 165 ++++++++++++++++ kernel/popcorn/trace_events.h | 76 ++++++++ kernel/popcorn/types.h | 358 ++++++++++++++++++++++++++++++++++ kernel/popcorn/util.c | 121 ++++++++++++ kernel/popcorn/util.h | 14 ++ kernel/popcorn/wait_station.c | 84 ++++++++ kernel/popcorn/wait_station.h | 27 +++ 15 files changed, 1233 insertions(+) create mode 100644 include/popcorn/bundle.h create mode 100644 include/popcorn/debug.h create mode 100644 include/popcorn/regset.h create mode 100644 include/popcorn/stat.h create mode 100644 include/popcorn/types.h create mode 100644 kernel/popcorn/Makefile create mode 100644 kernel/popcorn/bundle.c create mode 100644 kernel/popcorn/init.c create mode 100644 kernel/popcorn/stat.c create mode 100644 kernel/popcorn/trace_events.h create mode 100644 kernel/popcorn/types.h create mode 100644 kernel/popcorn/util.c create mode 100644 kernel/popcorn/util.h create mode 100644 kernel/popcorn/wait_station.c create mode 100644 kernel/popcorn/wait_station.h
include/popcorn/bundle.h new file +38 -0
@@ -0,0 +1,38 @@ +// SPDX-License-Identifier: GPL-2.0, 3-clause BSD +#ifndef __POPCORN_RACK_H__ +#define __POPCORN_RACK_H__ + +#define MAX_POPCORN_NODES 32 +#if (MAX_POPCORN_NODES >62 ) +#error Currently support up to 62 nodes +#endif + +enum popcorn_arch { +POPCORN_ARCH_UNKNOWN =-1 , +POPCORN_ARCH_X86 =1 , +POPCORN_ARCH_PPC =2 , +POPCORN_ARCH_MAX , +}; + +extern int my_nid; +extern const enum popcorn_arch my_arch; + +bool get_popcorn_node_online (int nid); +int get_popcorn_node_arch (int nid); +int popcorn_nodes_init (void ); +void set_popcorn_node_online (int nid,bool online); +void broadcast_my_node_info (int nr_nodes); + +struct popcorn_thread_status { +int current_nid; +int peer_nid; +pid_t peer_pid; +}; + +struct popcorn_node_info { +unsigned int status; +int arch; +int distance; +}; + +#endif
include/popcorn/debug.h new file +38 -0
@@ -0,0 +1,38 @@ +// SPDX-License-Identifier: GPL-2.0, 3-clause BSD +#ifndef __INCLUDE_POPCORN_DEBUG_H__ +#define __INCLUDE_POPCORN_DEBUG_H__ + +#define PCNPRINTK (...) printk(KERN_INFO "popcorn: " __VA_ARGS__) +#define PCNPRINTK_ERR (...) printk(KERN_ERR "popcorn: " __VA_ARGS__) + +#ifdef CONFIG_POPCORN_DEBUG +#define PRINTK (...) printk(KERN_INFO __VA_ARGS__) +#else +#define PRINTK (...) +#endif + +#ifdef CONFIG_POPCORN_DEBUG_PROCESS_SERVER +#define PSPRINTK (...) printk(KERN_INFO __VA_ARGS__) +#else +#define PSPRINTK (...) +#endif + +#ifdef CONFIG_POPCORN_DEBUG_VMA_SERVER +#define VSPRINTK (...) printk(KERN_INFO __VA_ARGS__) +#else +#define VSPRINTK (...) +#endif + +#ifdef CONFIG_POPCORN_DEBUG_PAGE_SERVER +#define PGPRINTK (...) printk(KERN_INFO __VA_ARGS__) +#else +#define PGPRINTK (...) +#endif + +#ifdef CONFIG_POPCORN_DEBUG_MSG_LAYER +#define MSGPRINTK (...) printk(KERN_INFO __VA_ARGS__) +#else +#define MSGPRINTK (...) +#endif + +#endif /* __INCLUDE_POPCORN_DEBUG_H__ */
include/popcorn/regset.h new file +96 -0
96 lines not on this page · raw message
include/popcorn/stat.h new file +16 -0
@@ -0,0 +1,16 @@ +// SPDX-License-Identifier: GPL-2.0, 3-clause BSD +#ifndef __KERNEL_POPCORN_STAT_H__ +#define __KERNEL_POPCORN_STAT_H__ + +struct pcn_kmsg_message ; + +void account_pcn_message_sent (struct pcn_kmsg_message *msg); +void account_pcn_message_recv (struct pcn_kmsg_message *msg); + +void account_pcn_rdma_write (size_t size); +void account_pcn_rdma_read (size_t size); + +#define POPCORN_STAT_FMT "%12llu %12llu %s\n" +#define POPCORN_STAT_FMT2 "%8llu.%03llu %8llu.%03llu %s\n" + +#endif /* KERNEL_POPCORN_STAT_H_ */
include/popcorn/types.h new file +20 -0
@@ -0,0 +1,20 @@ +// SPDX-License-Identifier: GPL-2.0, 3-clause BSD +#ifndef __INCLUDE_POPCORN_TYPES_H__ +#define __INCLUDE_POPCORN_TYPES_H__ + +#include <linux/sched.h> + +static inline bool distributed_process (struct task_struct *tsk) +{ +if (!tsk->mm)return false; +return !!tsk->mm->remote; +} + +static inline bool distributed_remote_process (struct task_struct *tsk) +{ +return distributed_process (tsk) && tsk->at_remote; +} + +#include <popcorn/debug.h> + +#endif /* __INCLUDE_POPCORN_TYPES_H__ */
kernel/popcorn/Makefile new file +7 -0
@@ -0,0 +1,7 @@ +obj-$(CONFIG_POPCORN) += init.o util.o +obj-$(CONFIG_POPCORN) += wait_station.o +obj-$(CONFIG_POPCORN) += process_server.o vma_server.o +obj-$(CONFIG_POPCORN) += page_server.o fh_action.o +obj-$(CONFIG_POPCORN) += bundle.o +obj-$(CONFIG_POPCORN) += pcn_kmsg.o +obj-$(CONFIG_POPCORN) += stat.o
kernel/popcorn/bundle.c new file +115 -0
115 lines not on this page · raw message
kernel/popcorn/init.c new file +58 -0
58 lines not on this page · raw message
kernel/popcorn/stat.c new file +165 -0
165 lines not on this page · raw message
kernel/popcorn/trace_events.h new file +76 -0
76 lines not on this page · raw message
kernel/popcorn/types.h new file +358 -0
358 lines not on this page · raw message
kernel/popcorn/util.c new file +121 -0
121 lines not on this page · raw message
kernel/popcorn/util.h new file +14 -0
@@ -0,0 +1,14 @@ +// SPDX-License-Identifier: GPL-2.0, BSD +#ifndef __POPCORN_KERNEL_UTIL_H__ +#define __POPCORN_KERNEL_UTIL_H__ +struct page ; + +void print_page_data (unsigned char *addr); +void print_page_signature (unsigned char *addr); +void print_page_signature_pid (pid_t pid,unsigned char *addr); +void print_page_owner (unsigned long addr,unsigned long *owners,pid_t pid); + +int get_file_path (struct file *file,char *sz,size_t size); + +void trace_task_status (void ); +#endif
kernel/popcorn/wait_station.c new file +84 -0
84 lines not on this page · raw message
kernel/popcorn/wait_station.h new file +27 -0
@@ -0,0 +1,27 @@ +// SPDX-License-Identifier: GPL-2.0, BSD +#ifndef _POPCORN_WAIT_STATION_H_ +#define _POPCORN_WAIT_STATION_H_ + +#include <linux/completion.h> +#include <linux/atomic.h> + +struct wait_station { +int id; +pid_t pid; +volatile void *private; +struct completion pendings; +atomic_t pendings_count; +}; + +struct task_struct ; + +struct wait_station *get_wait_station_multiple (struct task_struct *tsk, +int count); +static inline struct wait_station *get_wait_station (struct task_struct *tsk) +{ +return get_wait_station_multiple (tsk,1 ); +} +struct wait_station *wait_station (int id); +void put_wait_station (struct wait_station *ws); +void *wait_at_station (struct wait_station *ws); +#endif
Signature · 1 line
2.17.1
RFC 6/9 Process Server for Popcorn Distributed Thread Execution
In reply to Javier MalaveThis is the core migration and back migration processor. It contains handlers for migration, back migration and distributed futex messages. It also handles remote process exits.
This module is triggered by popcorn_migrate syscall via the function process_server_do_migration. If the process that triggers the system call is its origin node, then __do_migration triggers the forward migration process to the user specified destination node. The forward migration process uses a kernel thread called remote_thread_main.
Otherwise, if the process is in a remote node, it already has been migrated away, so __do_back_migration triggers the back migration process to the specified destination node (which in this case should be the origin node).
Process Server is in charge of processing additional functionality. When nodes receive Popcorn messages (see RFC 9/9 Message Layer) specific handlers to each Popcorn server post work to the Popcorn work queues.
The Process Server calls upon the appropriate "process" function to complete the work. There are two routines in charge of this: process_remote_works and run_remote_worker.
--- arch/x86/kernel/process_server.c | 250 +++++++ include/popcorn/process_server.h | 18 + kernel/popcorn/process_server.c | 1037 ++++++++++++++++++++++++++++++ kernel/popcorn/process_server.h | 21 + 4 files changed, 1326 insertions(+) create mode 100644 arch/x86/kernel/process_server.c create mode 100644 include/popcorn/process_server.h create mode 100644 kernel/popcorn/process_server.c create mode 100644 kernel/popcorn/process_server.h
arch/x86/kernel/process_server.c new file +250 -0
@@ -0,0 +1,250 @@ +/* +* File: +* process_server.c +* +* Description: +* this file implements the x86 architecture specific +* helper functionality of the process server +* +* Created on: +* Sep 19, 2014 +* +* Author: +* Sharath Kumar Bhat, SSRG, VirginiaTech +* +*/ + +/* File includes */ +#include <linux/sched.h> +#include <linux/kdebug.h> +#include <linux/ptrace.h> +#include <asm/uaccess.h> +#include <asm/prctl.h> +#include <asm/proto.h> +#include <asm/desc.h> +#include <asm/fpu/internal.h> + +#include <popcorn/types.h> +#include <popcorn/regset.h> + +/* +* Function: +* save_thread_info +* +* Description: +* this function saves the architecture specific info of the task +* to the struct struct field_arch structure passed +* +* Input: +* regs, pointer to the pt_regs field of the task +* +* Output: +* arch, pointer to the struct field_arch structure type where the +* architecture specific information of the task has to be +* saved +* +* Return value: +* on success, returns 0 +* on failure, returns negative integer +*/ +int save_thread_info (struct field_arch *arch) +{ +unsigned short fsindex, gsindex; +unsigned long ds, es, fs, gs; +int cpu; + +BUG_ON (!arch); + + cpu =get_cpu (); + +/* +* Segments +* CS and SS are set during the user/kernel mode switch. +* Thus, nothing to do with them. +*/ + + ds = current->thread.ds; + es = current->thread.es; + +savesegment (fs, fsindex); +if (fsindex) { + fs =get_desc_base (current->thread.tls_array + current->thread.fsbase); + }else { +rdmsrl (MSR_FS_BASE , fs); + } + +savesegment (gs, gsindex); +if (gsindex) { + gs =get_desc_base (current->thread.tls_array + current->thread.gsbase); + }else { +rdmsrl (MSR_KERNEL_GS_BASE , gs); + } + +WARN_ON (ds); +WARN_ON (es); +WARN_ON (gs); + arch->tls = fs; + +put_cpu (); + +/* +PSPRINTK("%s [%d] tls %lx\n", __func__, current->pid, arch->tls); +*/ + +return 0 ; +} + + +/* +* Function: +* restore_thread_info +* +* Description: +* this function restores the architecture specific info of the +* task from the struct field_arch structure passed +* +* Input: +* arch, pointer to the struct field_arch structure type from which the +* architecture specific information of the task has to be +* restored +* +* restore_segments, +* restore segmentations as well if segs is true. Unless, do +* not restore the segmentation units (for back migration) +* +* Output: +* none +* +* Return value: +* on success, returns 0 +* on failure, returns negative integer +*/ +int restore_thread_info (struct field_arch *arch,bool restore_segments) +{ +struct pt_regs *regs =current_pt_regs (); +struct regset_x86_64 *regset = &arch->regs_x86; +int cpu; + + cpu =get_cpu (); + + regs->r15 = regset->r15; + regs->r14 = regset->r14; + regs->r13 = regset->r13; + regs->r12 = regset->r12; + regs->bp = regset->rbp; + regs->bx = regset->rbx; + + regs->r11 = regset->r11; + regs->r10 = regset->r10; + regs->r9 = regset->r9; + regs->r8 = regset->r8; + regs->ax = regset->rax; + regs->cx = regset->rcx; + regs->dx = regset->rdx; + regs->si = regset->rsi; + regs->di = regset->rdi; + + regs->ip = regset->rip; + regs->sp = regset->rsp; + regs->flags = regset->rflags; + +if (restore_segments) { + regs->cs = __USER_CS; + regs->ss = __USER_DS; + +/* +current->thread.ds = regset->ds; +current->thread.es = regset->es; +*/ + +if (arch->tls) { +do_arch_prctl_64 (current,ARCH_SET_FS , arch->tls); + } +/* +if (arch->thread_gs) { +do_arch_prctl_64(current, ARCH_SET_GS, arch->thread_gs); +} +*/ + } + +put_cpu (); + +#ifdef CONFIG_POPCORN_DEBUG_VERBOSE +PSPRINTK ("%s [%d] ip %lx\n" , __func__, current->pid, regs->ip); +PSPRINTK ("%s [%d] sp %lx bp %lx\n" , __func__, current->pid, regs->sp, regs->bp); +#endif +return 0 ; +} + +#include <asm/stacktrace.h> +noinline_for_stack void update_frame_pointer(void) +{ +unsigned long *rbp; + rbp =__builtin_frame_address (0 );/* update_frame_pointer */ + +/* User rbp is at one stack frames below */ + *rbp =current_pt_regs ()->bp;/* sched_migrate */ +} + + +/* +* Function: +* dump_processor_regs +* +* Description: +* this function prints the architecture specific registers specified +* in the input argument +* +* Input: +* task, pointer to the architecture specific registers +* +* Output: +* none +* +* Return value: +* void +* +* Why don't use show_all() for x86? +*/ +void dump_processor_regs (struct pt_regs * regs) +{ +unsigned long fs, gs; +unsigned long fsindex, gsindex; + +dump_stack (); +if (!regs)return ; +printk (KERN_ALERT "DUMP REGS %s\n" , __func__); + +if (NULL != regs) { +printk (KERN_ALERT "r15{%lx}\n" ,regs->r15); +printk (KERN_ALERT "r14{%lx}\n" ,regs->r14); +printk (KERN_ALERT "r13{%lx}\n" ,regs->r13); +printk (KERN_ALERT "r12{%lx}\n" ,regs->r12); +printk (KERN_ALERT "r11{%lx}\n" ,regs->r11); +printk (KERN_ALERT "r10{%lx}\n" ,regs->r10); +printk (KERN_ALERT "r9{%lx}\n" ,regs->r9); +printk (KERN_ALERT "r8{%lx}\n" ,regs->r8); +printk (KERN_ALERT "bp{%lx}\n" ,regs->bp); +printk (KERN_ALERT "bx{%lx}\n" ,regs->bx); +printk (KERN_ALERT "ax{%lx}\n" ,regs->ax); +printk (KERN_ALERT "cx{%lx}\n" ,regs->cx); +printk (KERN_ALERT "dx{%lx}\n" ,regs->dx); +printk (KERN_ALERT "di{%lx}\n" ,regs->di); +printk (KERN_ALERT "orig_ax{%lx}\n" ,regs->orig_ax); +printk (KERN_ALERT "ip{%lx}\n" ,regs->ip); +printk (KERN_ALERT "cs{%lx}\n" ,regs->cs); +printk (KERN_ALERT "flags{%lx}\n" ,regs->flags); +printk (KERN_ALERT "sp{%lx}\n" ,regs->sp); +printk (KERN_ALERT "ss{%lx}\n" ,regs->ss); + } +rdmsrl (MSR_FS_BASE , fs); +rdmsrl (MSR_GS_BASE , gs); +printk (KERN_ALERT "fs{%lx} - %lx content %lx\n" ,fs, current->thread.fsbase, fs ? * (unsigned long *)fs :0x1234567l ); +printk (KERN_ALERT "gs{%lx} - %lx content %lx\n" ,gs, current->thread.gsbase, fs ? * (unsigned long *)gs :0x1234567l ); + +savesegment (fs, fsindex); +savesegment (gs, gsindex); +printk (KERN_ALERT "fsindex{%lx} - %x\n" ,fsindex, current->thread.fsindex); +printk (KERN_ALERT "gsindex{%lx} - %x\n" ,gsindex, current->thread.gsindex); +printk (KERN_ALERT "REGS DUMP COMPLETE\n" ); +}
include/popcorn/process_server.h new file +18 -0
@@ -0,0 +1,18 @@ +// SPDX-License-Identifier: GPL-2.0, 3-clause BSD +#ifndef __POPCORN_PROCESS_SERVER_H +#define __POPCORN_PROCESS_SERVER_H + + +int process_server_do_migration (struct task_struct *tsk,unsigned int dst_nid, +void __user *uregs); +int process_server_task_exit (struct task_struct *tsk); +int update_frame_pointer (void ); + +long process_server_do_futex_at_remote (u32 __user *uaddr,int op,u32 val, +bool valid_ts,struct timespec64 *ts, +u32 __user *uaddr2,u32 val2,u32 val3); + +struct remote_context ; +void free_remote_context (struct remote_context *rc); + +#endif /* __POPCORN_PROCESS_SERVER_H */
kernel/popcorn/process_server.c new file +1037 -0
1,037 lines not on this page · raw message
kernel/popcorn/process_server.h new file +21 -0
@@ -0,0 +1,21 @@ +// SPDX-License-Identifier: GPL-2.0, BSD +#ifndef __INTERNAL_PROCESS_SERVER_H__ +#define __INTERNAL_PROCESS_SERVER_H__ + +#include <popcorn/process_server.h> + +enum { +INDEX_OUTBOUND =0 , +INDEX_INBOUND =1 , +}; + +struct task_struct ; +struct field_arch ; + +inline void __lock_remote_contexts (spinlock_t *remote_contexts_lock,int index); +inline void __unlock_remote_contexts (spinlock_t *remote_contexts_lock, +int index); + +int save_thread_info (struct field_arch *arch); +int restore_thread_info (struct field_arch *arch,bool restore_segments); +#endif /* __INTERNAL_PROCESS_SERVER_H_ */
Signature · 1 line
2.17.1
RFC 7/9 Virtual Memory Address Server for Distributed Thread Execution
In reply to Javier MalavePopcorn Linux implements software-based distributed shared memory by extending Linux's virtual memory subsystem, and enables processes on different machines to observe a common and coherent virtual address space. Coherency of virtual memory pages of different hosts is ensured using a reader-replicate/writer-invalidate, page-level consistency protocol.
The VMA server's job is to implement the aforementioned protocol. Currently this work is fully managed at the origin node. Remote delegation of VMA operations is not supported except for MUNMAP.
--- include/popcorn/vma_server.h | 33 ++ kernel/popcorn/vma_server.c | 818 +++++++++++++++++++++++++++++++++++ kernel/popcorn/vma_server.h | 24 + 3 files changed, 875 insertions(+) create mode 100644 include/popcorn/vma_server.h create mode 100644 kernel/popcorn/vma_server.c create mode 100644 kernel/popcorn/vma_server.h
include/popcorn/vma_server.h new file +33 -0
@@ -0,0 +1,33 @@ +// SPDX-License-Identifier: GPL-2.0, 3-clause BSD +#ifndef INCLUDE_POPCORN_VMA_SERVER_H_ +#define INCLUDE_POPCORN_VMA_SERVER_H_ + + +/* +* VMA operation handlers for origin +*/ +int vma_server_munmap_origin (unsigned long start,size_t len,int nid_except); + + +/* +* Retrieve VMAs from origin +*/ +int vma_server_fetch_vma (struct task_struct *tsk,unsigned long address); + +/* +* VMA operation handler for remote +*/ +unsigned long vma_server_mmap_remote (struct file *file,unsigned long addr, +unsigned long len,unsigned long prot, +unsigned long flags,unsigned long pgoff); + +int vma_server_munmap_remote (unsigned long start,size_t len); +int vma_server_brk_remote (unsigned long oldbrk,unsigned long brk); +int vma_server_madvise_remote (unsigned long start,size_t len,int behavior); +int vma_server_mprotect_remote (unsigned long start,size_t len, +unsigned long prot); +int vma_server_mremap_remote (unsigned long addr,unsigned long old_len, +unsigned long new_len,unsigned long flags, +unsigned long new_addr); + +#endif /* INCLUDE_POPCORN_VMA_SERVER_H_ */
kernel/popcorn/vma_server.c new file +818 -0
818 lines not on this page · raw message
kernel/popcorn/vma_server.h new file +24 -0
@@ -0,0 +1,24 @@ +// SPDX-License-Identifier: GPL-2.0, BSD +#ifndef __INTERNAL_VMA_SERVER_H_ +#define __INTERNAL_VMA_SERVER_H_ + +#include <popcorn/vma_server.h> + +enum vma_op_code { +VMA_OP_NOP =-1 , +VMA_OP_MMAP , +VMA_OP_MUNMAP , +VMA_OP_MPROTECT , +VMA_OP_MREMAP , +VMA_OP_MADVISE , +VMA_OP_BRK , +VMA_OP_MAX , +}; + +struct remote_context ; + +void process_vma_info_request (vma_info_request_t *req); + +void process_vma_op_request (vma_op_request_t *req); + +#endif /* __INTERNAL_VMA_SERVER_H_ */
Signature · 1 line
2.17.1
RFC 8/9 Page Server for Distributed Thread Execution
In reply to Javier MalavePopcorn threads execute in a distributed manner,
thus pages may fall into an inconsistent state.
A few examples may be,
A) A new page is created in a remote node, and is needed at the origin node, after back migration.
B) A page has been changed in one node, the thread migrates to a previous node where it had executed code, and attempts to read the remotely modified page.
As part of the coherent VMA space, the page server takes care of managing page consistency. Page server must mark pages invalid when applicable, handle faults and verify ownership.
--- include/popcorn/page_server.h | 34 + kernel/popcorn/fh_action.c | 207 ++++ kernel/popcorn/fh_action.h | 34 + kernel/popcorn/page_server.c | 2019 +++++++++++++++++++++++++++++++++ kernel/popcorn/page_server.h | 16 + kernel/popcorn/pgtable.h | 31 + 6 files changed, 2341 insertions(+) create mode 100644 include/popcorn/page_server.h create mode 100644 kernel/popcorn/fh_action.c create mode 100644 kernel/popcorn/fh_action.h create mode 100644 kernel/popcorn/page_server.c create mode 100644 kernel/popcorn/page_server.h create mode 100644 kernel/popcorn/pgtable.h
include/popcorn/page_server.h new file +34 -0
@@ -0,0 +1,34 @@ +// SPDX-License-Identifier: GPL-2.0, 3-clause BSD +#ifndef INCLUDE_POPCORN_PAGE_SERVER_H_ +#define INCLUDE_POPCORN_PAGE_SERVER_H_ + +struct fault_handle ; + +/* +* Entry points for dealing with page fault in Popcorn Rack +*/ +int page_server_handle_pte_fault (struct vm_fault *vmf); + +void page_server_zap_pte (struct vm_area_struct *vma,unsigned long addr, +pte_t *pte,pte_t *pteval); + +int page_server_get_userpage (u32 __user *uaddr,struct fault_handle **handle, +char *mode); +void page_server_put_userpage (struct fault_handle *fh,char *mode); + +void page_server_panic (bool condition,struct mm_struct *mm, +unsigned long address,pte_t *pte, +pte_t pte_val); + +int page_server_release_page_ownership (struct vm_area_struct *vma, +unsigned long addr); + +/* Implemented in mm/memory.c */ +int handle_pte_fault_origin (struct mm_struct *,struct vm_area_struct *, +unsigned long ,pte_t *,pmd_t *,unsigned int ); +struct page *get_normal_page (struct vm_area_struct *vma,unsigned long addr, +pte_t *pte); +int cow_file_at_origin (struct mm_struct *mm,struct vm_area_struct *vma, +unsigned long addr,pte_t *pte); + +#endif /* INCLUDE_POPCORN_PAGE_SERVER_H_ */
kernel/popcorn/fh_action.c new file +207 -0
207 lines not on this page · raw message
kernel/popcorn/fh_action.h new file +34 -0
@@ -0,0 +1,34 @@ +// SPDX-License-Identifier: GPL-2.0, BSD +#ifndef __POPCORN_FAULT_HANDLING_ACTION_H__ +#define __POPCORN_FAULT_HANDLING_ACTION_H__ + +#include <linux/mm.h> + +enum { +FH_ACTION_INVALID =0x00 , +FH_ACTION_FOLLOW =0x10 , +FH_ACTION_RETRY =0x20 , +FH_ACTION_WAIT =0x01 , +FH_ACTION_LOCAL =0x02 , +FH_ACTION_DELAY =0x04 , + +PC_FAULT_FLAG_REMOTE =0x200 , + +FH_ACTION_MAX_FOLLOWER =8 , +}; + +static inline bool fault_for_write (unsigned long flags) +{ +return !!(flags &FAULT_FLAG_WRITE ); +} + +static inline bool fault_for_read (unsigned long flags) +{ +return !fault_for_write (flags); +} + +unsigned short get_fh_action (bool at_remote, +unsigned long fh_flags, +unsigned fault_flags + ); +#endif
kernel/popcorn/page_server.c new file +2019 -0
2,019 lines not on this page · raw message
kernel/popcorn/page_server.h new file +16 -0
@@ -0,0 +1,16 @@ +// SPDX-License-Identifier: GPL-2.0, BSD +#ifndef __INTERNAL_PAGE_SERVER_H__ +#define __INTERNAL_PAGE_SERVER_H__ + +#include <popcorn/page_server.h> + +/* +* Flush pages in remote to the origin +*/ +int page_server_flush_remote_pages (struct remote_context *rc); + +void free_remote_context_pages (struct remote_context *rc); +int process_madvise_release_from_remote (int from_nid,unsigned long start, +unsigned long end); + +#endif /* __INTERNAL_PAGE_SERVER_H_ */
kernel/popcorn/pgtable.h new file +31 -0
@@ -0,0 +1,31 @@ +// SPDX-License-Identifier: GPL-2.0, BSD +#ifndef __KERNEL_POPCORN_PGTABLE_H__ +#define __KERNEL_POPCORN_PGTABLE_H__ + +#include <asm/pgtable.h> + +#ifdef CONFIG_X86 +static inline pte_t pte_make_invalid (pte_t entry) +{ + entry =pte_modify (entry,__pgprot (pte_flags (entry) & ~_PAGE_PRESENT)); + +return entry; +} + +static inline pte_t pte_make_valid (pte_t entry) +{ + entry =pte_modify (entry,__pgprot (pte_flags (entry) | _PAGE_PRESENT)); + +return entry; +} + +static inline bool pte_is_present (pte_t entry) +{ +return (pte_val (entry) & _PAGE_PRESENT) == _PAGE_PRESENT; +} + +#else +#error "The architecture is not supported yet." +#endif + +#endif
Signature · 1 line
2.17.1
RFC 9/9 Add Popcorn Message Layer and socket support
In reply to Javier MalaveThis patch adds the Popcorn Message layer. It allows for Popcorn functionality over a TCP/IP network. The network is created when the module is inserted on all nodes, using a list of IP addresses located in /etc/popcorn/nodes. Each node ID is defined by the order of these IP addresses.
All Popcorn work is handled via work queues. Popcorn uses two work queues:popcorn_ordered_wq and popcorn_wq. Each Popcorn server module (vma server, page server, process server) registers handlers with the Popcorn kernel message module. These handlers are in charge of posting work to the work queues.
The message socket receive handler delegates the Popcorn kernel message module to call upon the proper work queue handlers. Moreover, the Popcorn kernel message module is responsible for posting and processing Popcorn messages. It forwards requests and responses for the different Popcorn server modules invoking the appropriate handlers for each message type.
--- drivers/msg_layer/Kconfig | 28 ++ drivers/msg_layer/Makefile | 2 + drivers/msg_layer/common.h | 63 ++++ drivers/msg_layer/socket.c | 710 +++++++++++++++++++++++++++++++++++++ include/popcorn/pcn_kmsg.h | 205 +++++++++++ kernel/popcorn/pcn_kmsg.c | 231 ++++++++++++ 6 files changed, 1239 insertions(+) create mode 100644 drivers/msg_layer/Kconfig create mode 100644 drivers/msg_layer/Makefile create mode 100644 drivers/msg_layer/common.h create mode 100644 drivers/msg_layer/socket.c create mode 100644 include/popcorn/pcn_kmsg.h create mode 100644 kernel/popcorn/pcn_kmsg.c
drivers/msg_layer/Kconfig new file +28 -0
@@ -0,0 +1,28 @@ +if POPCORN + +config POPCORN_KMSG + bool "Inter-kernel messaging layer" + default y + help + Enable support for various inter-kernel message passing + implementations + +if POPCORN_KMSG + +# Socket over Ethernet +config POPCORN_KMSG_SOCKET + tristate "Over TCP/IP (DANGEROUS)" + depends on INET && m + default m + help + Send Popcorn messages through TCP/IP sockets + +# Debuging +config POPCORN_DEBUG_MSG_LAYER + bool "Print debug messages" + depends on POPCORN_DEBUG + default n + +endif # POPCORN_KMSG + +endif # POPCORN
drivers/msg_layer/Makefile new file +2 -0
@@ -0,0 +1,2 @@ +obj-$(CONFIG_POPCORN_KMSG_SOCKET) := msg_socket.o +msg_socket-y := socket.o ring_buffer.o
drivers/msg_layer/common.h new file +63 -0
63 lines not on this page · raw message
drivers/msg_layer/socket.c new file +710 -0
710 lines not on this page · raw message
include/popcorn/pcn_kmsg.h new file +205 -0
205 lines not on this page · raw message
kernel/popcorn/pcn_kmsg.c new file +231 -0
231 lines not on this page · raw message
Signature · 1 line
2.17.1
Quoted 3 lines · Javier Malave
This patch set adds the Popcorn Distributed Thread Execution support to the kernel. It is based off of Linux 5.2 commit 72a20ce. We are looking for feedback on design and implementation from the community.
You may want to cc linux-api mailing list...?
Quoted 5 lines
Popcorn Linux is a Linux kernel-based software stack that enables applications to execute, with a shared code base, on distributed hosts. Popcorn allows applications to start execution on a particular host and migrate, at run-time, to a remote host. Multi-threaded applications may migrate any particular thread to any remote host.
Sounds like a lot of fun.
Quoted 5 lines
Popcorn Linux implements a software-based distributed shared memory by extending Linux's virtual memory subsystem and it enables processes on different machines to observe a common and coherent virtual address space. Coherency of virtual memory pages of different hosts is ensured using a reader-replicate/writer-invalidate, page-level consistency protocol.
Sounds interesting. I guess this needs very, very fast network. Do you have some performance numbers somewhere?
Best regards,
Pavel
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html