Dovetail integration - the finals
cover Dovetail integration - the finals
Now the final bits to git dovetail / 5.10 support into mainline. The last missing piece was the avoidance xnmalloc in critical paths when submitting inband work. I've used Hongzhan's patch for xnthread_signal and also Philippe's patches for the remaining cases. The outcome seems to work fine for both I-pipe and Dovetail now - but please review!
Not included are necessary RTDM extensions and the addition of latmus as currently still queued in wip/dovetail. There were some open issues, but those could not be easily addressed on top of next.
On top, I would also still like to understand - and ideally address prior to releasing a new major version - if we cannot align the ARM syscall ABI of Xenomai to the kernel /wrt argument passing. Now is a good chance to do that.
Jan
Jan Kiszka (9):
cobalt/thread: Rework kthread check for xnthread_signal cobalt/thread: Move xnthread_signal work off the stack cobalt/thread: Privatize xnthread_map to map_kthread cobalt/thread: Pull kthread inband work onto creator stack cobalt/thread: Make sure relax inband work is on a stack outliving the
wakeup
lib/cobalt: Reorder low_init for having cobalt_use_legacy_tsc earlier
available
cobalt/tick: dovetail: Drop pointless inline specifier from
pipeline_must_force_program_tick
cobalt/syscall: Account for different syscall argument marshaling
ci: Add arm, arm64 and x86 dovetail targets for kernel 5.10
Philippe Gerum (8):
cobalt/thread: dovetail: keep hard irqs off on transition to in-band cobalt/sched: dovetail: prevent early scheduling on uninit runqueues cobalt/arm: dovetail: add architecture bits lib/cobalt/arm: dovetail: skip detection of KUSER_TSC support lib/cobalt/arm64: dovetail: skip detection of KUSER_TSC support cobalt/tick: dovetail: improve accuracy of the host tick delay cobalt/clock: dovetail: abstract interface to hardware TSC cobalt/arm64: dovetail: add architecture bits
.gitlab-ci.yml | 26 ++
.../kernel/dovetail/pipeline/pipeline.h | 11 +
.../cobalt/kernel/dovetail/pipeline/sched.h | 12 +
.../cobalt/kernel/dovetail/pipeline/tick.h | 2 +-
.../cobalt/kernel/ipipe/pipeline/pipeline.h | 12 +
include/cobalt/kernel/ipipe/pipeline/sched.h | 20 +
include/cobalt/kernel/thread.h | 24 +-
include/cobalt/sys/cobalt.h | 2 +
include/copperplate/clockobj.h | 12 +-
kernel/cobalt/arch/arm/Kconfig | 4 +
kernel/cobalt/arch/arm/dovetail/Makefile | 5 +
.../include/asm/xenomai/calibration.h | 39 ++
.../dovetail/include/asm/xenomai/features.h | 30 ++
.../arm/dovetail/include/asm/xenomai/fptest.h | 52 +++
.../dovetail}/include/asm/xenomai/machine.h | 67 +--
.../dovetail/include/asm/xenomai/syscall.h | 101 +++++
.../dovetail/include/asm/xenomai/syscall32.h | 24 ++
.../arm/dovetail/include/asm/xenomai/thread.h | 32 ++
.../dovetail/include/asm/xenomai/wrappers.h | 27 ++
kernel/cobalt/arch/arm/dovetail/machine.c | 44 ++
kernel/cobalt/arch/arm64/dovetail/Makefile | 5 +
.../include/asm/xenomai/calibration.h | 24 ++
.../dovetail/include/asm/xenomai/features.h | 15 +
.../dovetail/include/asm/xenomai/fptest.h | 35 ++
.../dovetail/include/asm/xenomai/machine.h | 33 ++
.../dovetail/include/asm/xenomai/syscall.h | 63 +++
.../dovetail/include/asm/xenomai/syscall32.h | 12 +
.../dovetail/include/asm/xenomai/thread.h | 22 +
.../dovetail/include/asm/xenomai/wrappers.h | 15 +
kernel/cobalt/arch/arm64/dovetail/machine.c | 41 ++
.../arm64/ipipe/include/asm/xenomai/machine.h | 3 +
kernel/cobalt/arch/arm64/ipipe/machine.c | 3 +
.../dovetail/include/asm/xenomai/syscall.h | 5 -
kernel/cobalt/dovetail/kevents.c | 2 +-
kernel/cobalt/dovetail/sched.c | 1 +
kernel/cobalt/dovetail/tick.c | 14 +-
.../include/asm-generic/xenomai/syscall.h | 7 -
kernel/cobalt/ipipe/kevents.c | 2 +-
kernel/cobalt/posix/syscall.c | 14 +-
kernel/cobalt/sched.c | 10 +-
kernel/cobalt/synch.c | 4 +-
kernel/cobalt/thread.c | 402 ++++++++----------
lib/cobalt/arch/arm/features.c | 4 +
lib/cobalt/arch/arm/include/asm/xenomai/tsc.h | 2 +-
lib/cobalt/arch/arm64/features.c | 4 +
.../arch/arm64/include/asm/xenomai/tsc.h | 2 +-
.../arch/powerpc/include/asm/xenomai/tsc.h | 2 +-
lib/cobalt/arch/x86/include/asm/xenomai/tsc.h | 2 +-
lib/cobalt/clock.c | 6 +-
lib/cobalt/init.c | 2 +-
lib/cobalt/internal.c | 18 +-
lib/copperplate/clockobj.c | 12 +-
testsuite/smokey/tsc/tsc.c | 2 +-
53 files changed, 1030 insertions(+), 304 deletions(-)
create mode 100644 kernel/cobalt/arch/arm/dovetail/Makefile
create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/calibration.h
create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/features.h
create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/fptest.h
copy kernel/cobalt/arch/{arm64/ipipe => arm/dovetail}/include/asm/xenomai/machine.h (59%)
create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/syscall.h
create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/syscall32.h
create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/thread.h
create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/wrappers.h
create mode 100644 kernel/cobalt/arch/arm/dovetail/machine.c
create mode 100644 kernel/cobalt/arch/arm64/dovetail/Makefile
create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/calibration.h
create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/features.h
create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/fptest.h
create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/machine.h
create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/syscall.h
create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/syscall32.h
create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/thread.h
create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/wrappers.h
create mode 100644 kernel/cobalt/arch/arm64/dovetail/machine.cSignature · 1 line
2.26.2
PATCH 1/17 cobalt/thread: Rework kthread check for xnthread_signal
From: Jan Kiszka <[email protected]>
We only reach lostage_task_signal for a kthread if xnthread_signal has sent us. So move the check to the caller, making debugging easier.
--- kernel/cobalt/thread.c | 19 +++---------------- 1 file changed, 3 insertions(+), 16 deletions(-)
kernel/cobalt/thread.c +3 -16
@@ -2089,19 +2089,9 @@ struct lostage_signal {int signo, sigval; }; -static inline void do_kthread_signal (struct task_struct *p, -struct xnthread *thread, -struct lostage_signal *rq) -{ -printk (XENO_WARNING -"kernel shadow %s received unhandled signal %d (action=0x%x)\n" , - thread->name, rq->signo, rq->sigval); -} -static void lostage_task_signal (struct pipeline_inband_work *inband_work) {struct lostage_signal *rq; -struct xnthread *thread;struct task_struct *p;kernel_siginfo_t si;int signo;
@@ -2109,12 +2099,6 @@ static void lostage_task_signal(struct pipeline_inband_work *inband_work) rq = container_of(inband_work, struct lostage_signal, inband_work); p = rq->task; - thread =xnthread_from_task (p); -if (thread && !xnthread_test_state (thread,XNUSER )) { -do_kthread_signal (p, thread, rq); -return ; - } - signo = rq->signo;trace_cobalt_lostage_signal (p, signo);
@@ -2298,6 +2282,9 @@ void xnthread_signal(struct xnthread *thread, int sig, int arg) .sigval = sig == SIGDEBUG ? arg | sigdebug_marker : arg, }; +if (XENO_WARN_ON (COBALT , !xnthread_test_state (thread,XNUSER ))) +return ; +trace_cobalt_lostage_request ("signal" , sigwork.task);pipeline_post_inband_work (&sigwork);
Signature · 1 line
2.26.2
PATCH 2/17 cobalt/thread: Move xnthread_signal work off the stack
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
Unlike the I-pipe, Dovetail does not copy the work descriptor but merely hands over the request to the common irq_work() mechanism. We must guarantee that such descriptor lives in a portion of memory which won't go stale until the handler has run, which by design can only happen once the calling out-of-band context unwinds.
Therefore, we have to create signal slots per possible cause in the thread's control block in order to overcome sigwork in xnthread_signal.
For SIGDEBUG, we are only interested in the very first event coming in, so one slot is enough. All SIGSHADOW_* events need their own slot:SIGSHADOW_ACTION_HARDEN and SIGSHADOW_ACTION_HOME can be raised by remote threads asynchronously to the target thread. SIGSHADOW_ACTION_BACKTRACE comes in addition to SIGDEBUG_MIGRATE_*. For the latter reason, SIGSHADOW_ACTION_BACKTRACE cannot pile up though.
Including SIGTERM, we have totally 5 slots.
To ensure that multiple asynchronous signals on the same slot do not overwrite each other (e.g. SIGSHADOW_ACTION_HOME or SIGDEBUG reasons), synchronize the slot usage under nklock. Create a __xnthread_signal variant that can be called from already locked contexts (the majority of xnthread_signal users).
Based on original patch by Hongzhan Chen.
--- include/cobalt/kernel/thread.h | 21 ++++++- kernel/cobalt/dovetail/kevents.c | 2 +- kernel/cobalt/ipipe/kevents.c | 2 +- kernel/cobalt/synch.c | 4 +- kernel/cobalt/thread.c | 103 +++++++++++++++++++++++-------- 5 files changed, 101 insertions(+), 31 deletions(-)
include/cobalt/kernel/thread.h +19 -2
@@ -23,6 +23,7 @@#include <linux/sched.h> #include <linux/sched/rt.h> #include <pipeline/thread.h> +#include <pipeline/inband_work.h> #include <cobalt/kernel/list.h> #include <cobalt/kernel/stat.h> #include <cobalt/kernel/timer.h>
@@ -42,6 +43,13 @@#define XNTHREAD_BLOCK_BITS (XNSUSP|XNPEND|XNDELAY|XNDORMANT|XNRELAX|XNHELD|XNDBGSTOP)#define XNTHREAD_MODE_BITS (XNRRB|XNWARN|XNTRAPLB) +#define XNTHREAD_SIGDEBUG 0 +#define XNTHREAD_SIGSHADOW_HARDEN 1 +#define XNTHREAD_SIGSHADOW_BACKTRACE 2 +#define XNTHREAD_SIGSHADOW_HOME 3 +#define XNTHREAD_SIGTERM 4 +#define XNTHREAD_MAX_SIGNALS 5 +struct xnthread ;struct xnsched ;struct xnselector ;
@@ -50,6 +58,13 @@ struct xnsched_tpslot;struct xnthread_personality ;struct completion ; +struct lostage_signal { +struct pipeline_inband_work inband_work;/* Must be first. */ +struct task_struct *task; +int signo, sigval; +struct lostage_signal *self;/* Revisit: I-pipe requirement */ +}; +struct xnthread_init_attr {struct xnthread_personality *personality;cpumask_t affinity;
@@ -199,6 +214,7 @@ struct xnthread {const char *exe_path;/* Executable path */ u32 proghash;/* Hash value for exe_path */ #endif +struct lostage_signal sigarray[XNTHREAD_MAX_SIGNALS ]; };static inline int xnthread_get_state (const struct xnthread *thread)
@@ -492,8 +508,9 @@ void __xnthread_demote(struct xnthread *thread);void xnthread_demote (struct xnthread *thread); -void xnthread_signal (struct xnthread *thread, -int sig,int arg); +void __xnthread_signal (struct xnthread *thread,int sig,int arg); + +void xnthread_signal (struct xnthread *thread,int sig,int arg);void xnthread_pin_initial (struct xnthread *thread);
kernel/cobalt/dovetail/kevents.c +1 -1
@@ -188,7 +188,7 @@ static int handle_setaffinity_event(struct dovetail_migration_data *d)xnlock_get_irqsave (&nklock, s);if (xnthread_test_state (thread,XNTHREAD_BLOCK_BITS & ~XNRELAX )) -xnthread_signal (thread,SIGSHADOW ,SIGSHADOW_ACTION_HARDEN ); +__xnthread_signal (thread,SIGSHADOW ,SIGSHADOW_ACTION_HARDEN );xnlock_put_irqrestore (&nklock, s);
kernel/cobalt/ipipe/kevents.c +1 -1
@@ -261,7 +261,7 @@ static int handle_setaffinity_event(struct ipipe_cpu_migration_data *d)xnlock_get_irqsave (&nklock, s);if (xnthread_test_state (thread,XNTHREAD_BLOCK_BITS & ~XNRELAX )) -xnthread_signal (thread,SIGSHADOW ,SIGSHADOW_ACTION_HARDEN ); +__xnthread_signal (thread,SIGSHADOW ,SIGSHADOW_ACTION_HARDEN );xnlock_put_irqrestore (&nklock, s);
kernel/cobalt/synch.c +2 -2
@@ -1147,7 +1147,7 @@ void xnsynch_detect_relaxed_owner(struct xnsynch *synch, !xnthread_test_info(sleeper, XNPIALERT) && xnthread_test_state(synch->owner, XNRELAX)) {xnthread_set_info (sleeper,XNPIALERT ); -xnthread_signal (sleeper,SIGDEBUG , +__xnthread_signal (sleeper,SIGDEBUG ,SIGDEBUG_MIGRATE_PRIOINV ); } elsexnthread_clear_info (sleeper ,XNPIALERT );
@@ -1171,7 +1171,7 @@ void xnsynch_detect_boosted_relax(struct xnthread *owner)xnsynch_for_each_sleeper (sleeper, synch) {if (xnthread_test_state (sleeper,XNWARN )) {xnthread_set_info (sleeper,XNPIALERT ); -xnthread_signal (sleeper,SIGDEBUG , +__xnthread_signal (sleeper,SIGDEBUG ,SIGDEBUG_MIGRATE_PRIOINV ); } }
kernel/cobalt/thread.c +78 -25
168 lines not on this page · raw message
Signature · 1 line
2.26.2
PATCH 3/17 cobalt/thread: Privatize xnthread_map to map_kthread
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
No user outside of the core, no need to export it. Move the code around so that we can easily use it without forward declarations.
--- include/cobalt/kernel/thread.h | 3 - kernel/cobalt/thread.c | 239 ++++++++++++++------------------- 2 files changed, 98 insertions(+), 144 deletions(-)
include/cobalt/kernel/thread.h +0 -3
@@ -514,9 +514,6 @@ void xnthread_signal(struct xnthread *thread, int sig, int arg);void xnthread_pin_initial (struct xnthread *thread); -int xnthread_map (struct xnthread *thread, -struct completion *done); -void xnthread_call_mayday (struct xnthread *thread,int reason);static inline void xnthread_get_resource (struct xnthread *curr)
kernel/cobalt/thread.c +98 -141
257 lines not on this page · raw message
Signature · 1 line
2.26.2
PATCH 4/17 cobalt/thread: Pull kthread inband work onto creator stack
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
As dovetail does not copy the work passed to pipeline_post_inband_work, we must ensure that the one used in xnthread_relax lives as long as the wakeup takes. Therefore, embed the pipeline_inband_work struct needed for signaling the kthread creator into kthread_arg struct. This lives on the creator stack, as long as it takes to signal completion to it.
--- kernel/cobalt/thread.c | 38 +++++++++++++------------------------- 1 file changed, 13 insertions(+), 25 deletions(-)
kernel/cobalt/thread.c +13 -25
@@ -84,30 +84,18 @@ static inline void enlist_new_thread(struct xnthread *thread)xnvfile_touch_tag (&nkthreadlist_tag); } -struct parent_wakeup_request { +struct kthread_arg {struct pipeline_inband_work inband_work;/* Must be first. */ +struct xnthread *thread;struct completion *done; };static void do_parent_wakeup (struct pipeline_inband_work *inband_work) { -struct parent_wakeup_request *rq; +struct kthread_arg *ka; - rq = container_of(inband_work, struct parent_wakeup_request, inband_work); -complete (rq->done); -} - -static inline void wakeup_parent (struct completion *done) -{ -struct parent_wakeup_request wakework = { - .inband_work =PIPELINE_INBAND_WORK_INITIALIZER (wakework, - do_parent_wakeup), - .done = done, - }; - -trace_cobalt_lostage_request ("wakeup" , current); - -pipeline_post_inband_work (&wakework); + ka = container_of(inband_work, struct kthread_arg, inband_work); +complete (ka->done); }static inline void init_kthread_info (struct xnthread *thread)
@@ -119,7 +107,7 @@ static inline void init_kthread_info(struct xnthread *thread) p->process =NULL ; } -static int map_kthread (struct xnthread *thread,struct completion *done) +static int map_kthread (struct xnthread *thread,struct kthread_arg *ka) {int ret;spl_t s;
@@ -158,7 +146,12 @@ static int map_kthread(struct xnthread *thread, struct completion *done)*/ xnthread_resume (thread,XNDORMANT ); ret =xnthread_harden (); -wakeup_parent (done); + +trace_cobalt_lostage_request ("wakeup" , current); + + ka->inband_work = (struct pipeline_inband_work ) +PIPELINE_INBAND_WORK_INITIALIZER (*ka, do_parent_wakeup); +pipeline_post_inband_work (ka);xnlock_get_irqsave (&nklock, s);
@@ -181,11 +174,6 @@ static int map_kthread(struct xnthread *thread, struct completion *done)return ret; } -struct kthread_arg { -struct xnthread *thread; -struct completion *done; -}; -static int kthread_trampoline (void *arg) {struct kthread_arg *ka = arg;
@@ -210,7 +198,7 @@ static int kthread_trampoline(void *arg) param.sched_priority = prio;sched_setscheduler (current, policy, ¶m); - ret =map_kthread (thread, ka->done); + ret =map_kthread (thread, ka);if (ret) {printk (XENO_WARNING "failed to create kernel shadow %s\n" , thread->name);
Signature · 1 line
2.26.2
PATCH 5/17 cobalt/thread: Make sure relax inband work is on a stack outliving the wakeup
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
As dovetail does not copy the work passed to pipeline_post_inband_work, we must ensure that the one used in xnthread_relax lives as long as the wakeup takes. Therefore, fold post_wakeup into the only caller so that the stack frame of xnthread_relax is guaranteed to be used which fulfills this requirement.
--- kernel/cobalt/thread.c | 23 ++++++++--------------- 1 file changed, 8 insertions(+), 15 deletions(-)
kernel/cobalt/thread.c +8 -15
@@ -1993,19 +1993,6 @@ static void lostage_task_wakeup(struct pipeline_inband_work *inband_work)wake_up_process (p); } -static void post_wakeup (struct task_struct *p) -{ -struct lostage_wakeup wakework = { - .inband_work =PIPELINE_INBAND_WORK_INITIALIZER (wakework, - lostage_task_wakeup), - .task = p, - }; - -trace_cobalt_lostage_request ("wakeup" , wakework.task); - -pipeline_post_inband_work (&wakework); -} -void __xnthread_propagate_schedparam (struct xnthread *curr) {int kpolicy =SCHED_FIFO , kprio = curr->bprio, ret;
@@ -2065,9 +2052,14 @@ void __xnthread_propagate_schedparam(struct xnthread *curr)*/ void xnthread_relax (int notify,int reason) { +struct task_struct *p = current; +struct lostage_wakeup wakework = { + .inband_work =PIPELINE_INBAND_WORK_INITIALIZER (wakework, + lostage_task_wakeup), + .task = p, + };struct xnthread *thread =xnthread_current ();int cpu__maybe_unused , suspension; -struct task_struct *p = current;kernel_siginfo_t si;primary_mode_only ();
@@ -2090,7 +2082,8 @@ void xnthread_relax(int notify, int reason)* xnthread_suspend() has an interrupts-on section built in. */ splmax (); -post_wakeup (p); +trace_cobalt_lostage_request ("wakeup" , p); +pipeline_post_inband_work (&wakework);/* * Grab the nklock to synchronize the Linux task state * manipulation with handle_sigwake_event. This lock will be
Signature · 1 line
2.26.2
PATCH 6/17 cobalt/thread: dovetail: keep hard irqs off on transition to in-band
In reply to Jan KiszkaFrom: Philippe Gerum <[email protected]>
Dovetail provides a fast service to escalate the caller to out-of-band mode for executing a routine (run_oob_call()), which we use to enforce primary mode in ___xnsched_run() to schedule out the relaxing thread.
Due to the way run_oob_call() works, enabling hardirqs during this transition can trigger a subtle bug caused by the relaxing thread to be switched out, as a result of taking an interrupt during the irqs on section, before __xnsched_run() actually runs on behalf of xnthread_relax() -> xnthread_suspend().
This may lead to a breakage of the inband interrupt state, revealed by lockdep complaining about a HARDIRQ-IN-W -> HARDIRQ-ON-W situation, when finalize_task_switch() runs for reconciling both the in-band and Xenomai scheduler states.
Re-enabling hard irqs before switching out the relaxing thread was throught as a mean to reduce the scope of the interrupt-free section while relaxing a thread with the I-pipe, which unlike Dovetail requires us to open code an escalation service based on triggering a synthetic interrupt.
We differentiate the behavior between the I-pipe and Dovetail by abstracting the related bits as pipeline_leave_oob_unlock(), keeping the current implementation unchanged for the I-pipe.
--- .../cobalt/kernel/dovetail/pipeline/sched.h | 12 +++++++ include/cobalt/kernel/ipipe/pipeline/sched.h | 20 ++++++++++++ kernel/cobalt/thread.c | 32 +++++++------------ 3 files changed, 43 insertions(+), 21 deletions(-)
include/cobalt/kernel/dovetail/pipeline/sched.h +12 -0
@@ -7,6 +7,8 @@#ifndef _COBALT_KERNEL_DOVETAIL_SCHED_H#define _COBALT_KERNEL_DOVETAIL_SCHED_H +#include <cobalt/kernel/lock.h> +struct xnthread ;struct xnsched ;struct task_struct ;
@@ -35,6 +37,16 @@ int pipeline_leave_inband(void);int pipeline_leave_oob_prepare (void ); +static inline void pipeline_leave_oob_unlock (void ) +{ +/* +* We may not re-enable hard irqs due to the specifics of +* stage escalation via run_oob_call(), to prevent breaking +* the (virtual) interrupt state. +*/ +xnlock_put (&nklock); +} +void pipeline_leave_oob_finish (void ); static inline
include/cobalt/kernel/ipipe/pipeline/sched.h +20 -0
@@ -7,6 +7,8 @@#ifndef _COBALT_KERNEL_IPIPE_SCHED_H#define _COBALT_KERNEL_IPIPE_SCHED_H +#include <cobalt/kernel/lock.h> +struct xnthread ;struct xnsched ;struct task_struct ;
@@ -27,6 +29,24 @@ int pipeline_leave_inband(void);int pipeline_leave_oob_prepare (void ); +static inline void pipeline_leave_oob_unlock (void ) +{ +/* +* Introduce an opportunity for interrupt delivery right +* before switching context, which shortens the +* uninterruptible code path. +* +* We have to shut irqs off before __xnsched_run() is called +* next though: if an interrupt could preempt us right after +* xnarch_escalate() is passed but before the nklock is +* grabbed, we would enter the critical section in +* ___xnsched_run() from the root domain, which would defeat +* the purpose of escalating the request. +*/ +xnlock_clear_irqon (&nklock); +splmax (); +} +void pipeline_leave_oob_finish (void );void pipeline_finalize_thread (struct xnthread *thread);
kernel/cobalt/thread.c +11 -21
@@ -955,27 +955,17 @@ void xnthread_suspend(struct xnthread *thread, int mask,if (wchan) thread->wchan = wchan; -/* -* If the current thread is being relaxed, we must have been -* called from xnthread_relax(), in which case we introduce an -* opportunity for interrupt delivery right before switching -* context, which shortens the uninterruptible code path. -* -* We have to shut irqs off before calling __xnsched_run() -* though: if an interrupt could preempt us right after -* xnarch_escalate() is passed but before the nklock is -* grabbed, we would enter the critical section in -* ___xnsched_run() from the root domain, which would defeat -* the purpose of escalating the request. -* -* NOTE: using __xnsched_run() for rescheduling allows us to -* break the scheduler lock temporarily. -*/ if (likely (thread == sched->curr)) {xnsched_set_resched (sched); +/* +* Transition to secondary mode (XNRELAX) is a +* separate path which is only available to +* xnthread_relax(). Using __xnsched_run() there for +* rescheduling allows us to break the scheduler lock +* temporarily. +*/ if (unlikely (mask &XNRELAX )) { -xnlock_clear_irqon (&nklock); -splmax (); +pipeline_leave_oob_unlock ();__xnsched_run (sched);return ; }
@@ -1683,7 +1673,7 @@ int xnthread_join(struct xnthread *thread, bool uninterruptible)xnthread_set_state (thread,XNJOINED ); tpid =xnthread_host_pid (thread); - +if (curr && !xnthread_test_state (curr,XNRELAX )) {xnlock_put_irqrestore (&nklock, s);xnthread_relax (0 ,0 );
@@ -1889,7 +1879,7 @@ int __xnthread_set_schedparam(struct xnthread *thread,/* Ask the target thread to call back if relaxed. */ if (xnthread_test_state (thread,XNRELAX ))__xnthread_signal (thread,SIGSHADOW ,SIGSHADOW_ACTION_HOME ); - +return ret; }
@@ -2081,7 +2071,6 @@ void xnthread_relax(int notify, int reason)* We disable interrupts during the migration sequence, but * xnthread_suspend() has an interrupts-on section built in. */ -splmax ();trace_cobalt_lostage_request ("wakeup" , p);pipeline_post_inband_work (&wakework);/*
@@ -2089,6 +2078,7 @@ void xnthread_relax(int notify, int reason)* manipulation with handle_sigwake_event. This lock will be * dropped by xnthread_suspend(). */ +splmax ();xnlock_get (&nklock);xnthread_run_handler_stack (thread, relax_thread); suspension =pipeline_leave_oob_prepare ();
Signature · 1 line
2.26.2
Re: [PATCH 06/17] cobalt/thread: dovetail: keep hard irqs off on transition to in-band
Quoted 39 lines · Jan Kiszka via Xenomai
From: Philippe Gerum <[email protected]>
Dovetail provides a fast service to escalate the caller to out-of-band mode for executing a routine (
run_oob_call()), which we use to enforce primary mode in___xnsched_run()to schedule out the relaxing thread.Due to the way
run_oob_call()works, enabling hardirqs during this transition can trigger a subtle bug caused by the relaxing thread to be switched out, as a result of taking an interrupt during the irqs on section, before__xnsched_run()actually runs on behalf ofxnthread_relax()->xnthread_suspend().This may lead to a breakage of the inband interrupt state, revealed by lockdep complaining about a HARDIRQ-IN-W -> HARDIRQ-ON-W situation, when
finalize_task_switch()runs for reconciling both the in-band and Xenomai scheduler states.Re-enabling hard irqs before switching out the relaxing thread was throught as a mean to reduce the scope of the interrupt-free section while relaxing a thread with the I-pipe, which unlike Dovetail requires us to open code an escalation service based on triggering a synthetic interrupt.
We differentiate the behavior between the I-pipe and Dovetail by abstracting the related bits as
pipeline_leave_oob_unlock(), keeping the current implementation unchanged for the I-pipe.Signed-off-by Philippe Gerum <[email protected]>Signed-off-by Jan Kiszka <[email protected]>--- .../cobalt/kernel/dovetail/pipeline/sched.h | 12 +++++++ include/cobalt/kernel/ipipe/pipeline/sched.h | 20 ++++++++++++ kernel/cobalt/thread.c | 32 +++++++------------ 3 files changed, 43 insertions(+), 21 deletions(-)include/cobalt/kernel/dovetail/pipeline/sched.h +0
-0
...
include/cobalt/kernel/dovetail/pipeline/sched.h void xnthread_relax(int notify, int reason) · line 2081
Quoted 13 lines
include/cobalt/kernel/dovetail/pipeline/sched.h +1
-1@@ -2081,7 +2071,6 @@ void xnthread_relax(int notify, int reason)* We disable interrupts during the migration sequence, but * xnthread_suspend() has an interrupts-on section built in. */ -splmax ();trace_cobalt_lostage_request ("wakeup" , p);pipeline_post_inband_work (&wakework);/* @@ -2089,6 +2078,7 @@ void xnthread_relax(int notify, int reason)* manipulation with handle_sigwake_event. This lock will be * dropped by xnthread_suspend(). */ +splmax ();
This moves pipeline_post_inband_work() outside of the irqs-off section, no? I think this destabilizes our migration to in-band, allowing the injected wake-event to be consumed by Linux prior to the oob thread suspension. I have a test here that seems to trigger this reliably. Currently validating of moving splmax up again helps. Also
Can you recall why you changed it this way?
Jan
Quoted 3 lines
include/cobalt/kernel/dovetail/pipeline/sched.h +0
-0xnlock_get (&nklock);xnthread_run_handler_stack (thread, relax_thread); suspension =pipeline_leave_oob_prepare ();
Signature · 2 lines
Siemens AG, Technology Competence Center Embedded Linux
Re: [PATCH 06/17] cobalt/thread: dovetail: keep hard irqs off on transition to in-band
Quoted 59 lines · Jan Kiszka
Quoted 35 lines · Jan Kiszka via Xenomai
From: Philippe Gerum <[email protected]>
Dovetail provides a fast service to escalate the caller to out-of-band
mode for executing a routine (run_oob_call()), which we use to enforce primary mode in___xnsched_run()to schedule out the relaxing thread. Due to the wayrun_oob_call()works, enabling hardirqs during this transition can trigger a subtle bug caused by the relaxing thread to be switched out, as a result of taking an interrupt during the irqs on section, before__xnsched_run()actually runs on behalf ofxnthread_relax()->xnthread_suspend().
This may lead to a breakage of the inband interrupt state, revealed by
lockdep complaining about a HARDIRQ-IN-W -> HARDIRQ-ON-W situation, whenfinalize_task_switch()runs for reconciling both the in-band and Xenomai scheduler states.
Re-enabling hard irqs before switching out the relaxing thread was throught as a mean to reduce the scope of the interrupt-free section while relaxing a thread with the I-pipe, which unlike Dovetail requires us to open code an escalation service based on triggering a synthetic interrupt.
We differentiate the behavior between the I-pipe and Dovetail by abstracting the related bits aspipeline_leave_oob_unlock(), keeping the current implementation unchanged for the I-pipe.Signed-off-by Philippe Gerum <[email protected]>Signed-off-by Jan Kiszka <[email protected]>--- .../cobalt/kernel/dovetail/pipeline/sched.h | 12 +++++++ include/cobalt/kernel/ipipe/pipeline/sched.h | 20 ++++++++++++ kernel/cobalt/thread.c | 32 +++++++------------ 3 files changed, 43 insertions(+), 21 deletions(-)include/cobalt/kernel/dovetail/pipeline/sched.h +0
-0...
include/cobalt/kernel/dovetail/pipeline/sched.h void xnthread_relax(int notify, int reason) · line 2081
Quoted 13 lines
include/cobalt/kernel/dovetail/pipeline/sched.h +1
-1@@ -2081,7 +2071,6 @@ void xnthread_relax(int notify, int reason)* We disable interrupts during the migration sequence, but * xnthread_suspend() has an interrupts-on section built in. */ -splmax ();trace_cobalt_lostage_request ("wakeup" , p);pipeline_post_inband_work (&wakework);/* @@ -2089,6 +2078,7 @@ void xnthread_relax(int notify, int reason)* manipulation with handle_sigwake_event. This lock will be * dropped by xnthread_suspend(). */ +splmax ();This moves
pipeline_post_inband_work()outside of the irqs-off section, no? I think this destabilizes our migration to in-band, allowing the injected wake-event to be consumed by Linux prior to the oob thread suspension. I have a test here that seems to trigger this reliably. Currently validating of moving splmax up again helps. Also
Quoted 2 lines
Can you recall why you changed it this way?
The rationale was that current had to be running oob at this point in xnthread_relax(), so there would be no way for an in-band irq to be handled, causing the per-cpu work to run on the same CPU.
Now, the tricky part is that this assumption might be wrong if an oob irq wakes up a thread which preempts on the same CPU, then switches back to secondary mode when it resumes. In this case, the irq work would run earlier than expected. Worth testing indeed.
Signature · 1 line
Philippe.
Re: [PATCH 06/17] cobalt/thread: dovetail: keep hard irqs off on transition to in-band
Quoted 64 lines · Philippe Gerum
Quoted 59 lines · Jan Kiszka
Quoted 35 lines · Jan Kiszka via Xenomai
From: Philippe Gerum <[email protected]>
Dovetail provides a fast service to escalate the caller to out-of-band
mode for executing a routine (run_oob_call()), which we use to enforce primary mode in___xnsched_run()to schedule out the relaxing thread. Due to the wayrun_oob_call()works, enabling hardirqs during this transition can trigger a subtle bug caused by the relaxing thread to be switched out, as a result of taking an interrupt during the irqs on section, before__xnsched_run()actually runs on behalf ofxnthread_relax()->xnthread_suspend().
This may lead to a breakage of the inband interrupt state, revealed by
lockdep complaining about a HARDIRQ-IN-W -> HARDIRQ-ON-W situation, whenfinalize_task_switch()runs for reconciling both the in-band and Xenomai scheduler states.
Re-enabling hard irqs before switching out the relaxing thread was throught as a mean to reduce the scope of the interrupt-free section while relaxing a thread with the I-pipe, which unlike Dovetail requires us to open code an escalation service based on triggering a synthetic interrupt.
We differentiate the behavior between the I-pipe and Dovetail by abstracting the related bits aspipeline_leave_oob_unlock(), keeping the current implementation unchanged for the I-pipe.Signed-off-by Philippe Gerum <[email protected]>Signed-off-by Jan Kiszka <[email protected]>--- .../cobalt/kernel/dovetail/pipeline/sched.h | 12 +++++++ include/cobalt/kernel/ipipe/pipeline/sched.h | 20 ++++++++++++ kernel/cobalt/thread.c | 32 +++++++------------ 3 files changed, 43 insertions(+), 21 deletions(-)include/cobalt/kernel/dovetail/pipeline/sched.h +0
-0...
include/cobalt/kernel/dovetail/pipeline/sched.h void xnthread_relax(int notify, int reason) · line 2081
Quoted 13 lines
include/cobalt/kernel/dovetail/pipeline/sched.h +1
-1@@ -2081,7 +2071,6 @@ void xnthread_relax(int notify, int reason)* We disable interrupts during the migration sequence, but * xnthread_suspend() has an interrupts-on section built in. */ -splmax ();trace_cobalt_lostage_request ("wakeup" , p);pipeline_post_inband_work (&wakework);/* @@ -2089,6 +2078,7 @@ void xnthread_relax(int notify, int reason)* manipulation with handle_sigwake_event. This lock will be * dropped by xnthread_suspend(). */ +splmax ();This moves
pipeline_post_inband_work()outside of the irqs-off section, no? I think this destabilizes our migration to in-band, allowing the injected wake-event to be consumed by Linux prior to the oob thread suspension. I have a test here that seems to trigger this reliably. Currently validating of moving splmax up again helps. AlsoQuoted 2 lines
Can you recall why you changed it this way?
Quoted 8 lines
The rationale was that current had to be running oob at this point in
xnthread_relax(), so there would be no way for an in-band irq to be handled, causing the per-cpu work to run on the same CPU.Now, the tricky part is that this assumption might be wrong if an oob irq wakes up a thread which preempts on the same CPU, then switches back to secondary mode when it resumes. In this case, the irq work would run earlier than expected. Worth testing indeed.
As a matter of fact, EVL keeps the section atomic, Cobalt should align on this:
void evl_switch_inband(int cause)
...
hard_local_irq_disable();
irq_work_queue(&curr->inband_work);Signature · 1 line
Philippe.
Re: [PATCH 06/17] cobalt/thread: dovetail: keep hard irqs off on transition to in-band
Quoted 67 lines · Philippe Gerum
Quoted 64 lines · Philippe Gerum
Quoted 61 lines · Jan Kiszka
Quoted 35 lines · Jan Kiszka via Xenomai
From: Philippe Gerum <[email protected]>
Dovetail provides a fast service to escalate the caller to out-of-band
mode for executing a routine (run_oob_call()), which we use to enforce primary mode in___xnsched_run()to schedule out the relaxing thread. Due to the wayrun_oob_call()works, enabling hardirqs during this transition can trigger a subtle bug caused by the relaxing thread to be switched out, as a result of taking an interrupt during the irqs on section, before__xnsched_run()actually runs on behalf ofxnthread_relax()->xnthread_suspend().
This may lead to a breakage of the inband interrupt state, revealed by
lockdep complaining about a HARDIRQ-IN-W -> HARDIRQ-ON-W situation, whenfinalize_task_switch()runs for reconciling both the in-band and Xenomai scheduler states.
Re-enabling hard irqs before switching out the relaxing thread was throught as a mean to reduce the scope of the interrupt-free section while relaxing a thread with the I-pipe, which unlike Dovetail requires us to open code an escalation service based on triggering a synthetic interrupt.
We differentiate the behavior between the I-pipe and Dovetail by abstracting the related bits aspipeline_leave_oob_unlock(), keeping the current implementation unchanged for the I-pipe.Signed-off-by Philippe Gerum <[email protected]>Signed-off-by Jan Kiszka <[email protected]>--- .../cobalt/kernel/dovetail/pipeline/sched.h | 12 +++++++ include/cobalt/kernel/ipipe/pipeline/sched.h | 20 ++++++++++++ kernel/cobalt/thread.c | 32 +++++++------------ 3 files changed, 43 insertions(+), 21 deletions(-)include/cobalt/kernel/dovetail/pipeline/sched.h +0
-0...
include/cobalt/kernel/dovetail/pipeline/sched.h void xnthread_relax(int notify, int reason) · line 2081
Quoted 13 lines
include/cobalt/kernel/dovetail/pipeline/sched.h +1
-1@@ -2081,7 +2071,6 @@ void xnthread_relax(int notify, int reason)* We disable interrupts during the migration sequence, but * xnthread_suspend() has an interrupts-on section built in. */ -splmax ();trace_cobalt_lostage_request ("wakeup" , p);pipeline_post_inband_work (&wakework);/* @@ -2089,6 +2078,7 @@ void xnthread_relax(int notify, int reason)* manipulation with handle_sigwake_event. This lock will be * dropped by xnthread_suspend(). */ +splmax ();This moves
pipeline_post_inband_work()outside of the irqs-off section, no? I think this destabilizes our migration to in-band, allowing the injected wake-event to be consumed by Linux prior to the oob thread suspension. I have a test here that seems to trigger this reliably. Currently validating of moving splmax up again helps. AlsoCan you recall why you changed it this way?
Quoted 8 lines
Quoted 8 lines
The rationale was that current had to be running oob at this point in
xnthread_relax(), so there would be no way for an in-band irq to be handled, causing the per-cpu work to run on the same CPU.Now, the tricky part is that this assumption might be wrong if an oob irq wakes up a thread which preempts on the same CPU, then switches back to secondary mode when it resumes. In this case, the irq work would run earlier than expected. Worth testing indeed.
Yeah, the test actually involves remote suspending. That may explain why only we saw it so far with a rather special setup, luckily.
Quoted 11 lines
As a matter of fact, EVL keeps the section atomic, Cobalt should align on this:
void
evl_switch_inband(int cause)... hard_local_irq_disable(); irq_work_queue(&curr->inband_work);
OK, will send a patch the reverts that change above.
Thanks,
Jan
Signature · 2 lines
Siemens AG, Technology Competence Center Embedded Linux
PATCH 7/17 cobalt/sched: dovetail: prevent early scheduling on uninit runqueues
In reply to Jan KiszkaFrom: Philippe Gerum <[email protected]>
Make sure not to raise XNSCHED when setting up the root thread, so that we can't start rescheduling from irq_exit_pipeline() before all CPUs have their runqueue fully built.
At this chance, add the missing tracepoint reporting a remote scheduling request.
--- kernel/cobalt/dovetail/sched.c | 1 + kernel/cobalt/sched.c | 10 +++++++++- 2 files changed, 10 insertions(+), 1 deletion(-)
kernel/cobalt/dovetail/sched.c +1 -0
@@ -91,6 +91,7 @@ void pipeline_clear_mayday(void) /* May solely affect current. */irqreturn_t pipeline_reschedule_ipi_handler (int irq,void *dev_id) { +trace_cobalt_schedule_remote (xnsched_current ());/* Will reschedule from irq_exit_pipeline(). */
kernel/cobalt/sched.c +9 -1
@@ -465,7 +465,15 @@ int xnsched_set_policy(struct xnthread *thread,if (xnthread_test_state (thread,XNREADY ))xnsched_enqueue (thread); -if (!xnthread_test_state (thread,XNDORMANT )) +/* +* Make sure not to raise XNSCHED when setting up the root +* thread, so that we can't start rescheduling on interrupt +* exit before all CPUs have their runqueue fully +* built. Filtering on XNROOT here is correct because the root +* thread enters the idle class once as part of the runqueue +* setup process and never leaves it afterwards. +*/ +if (!xnthread_test_state (thread,XNDORMANT |XNROOT ))xnsched_set_resched (thread->sched);return 0 ;
Signature · 1 line
2.26.2
PATCH 8/17 cobalt/arm: dovetail: add architecture bits
In reply to Jan KiszkaFrom: Philippe Gerum <[email protected]>
The license notice for arch/arm/dovetail/machine.c is fixed in order to match the remaining code, which is [1] for the prefaulting handler, and myself for the rest.
[1] from Xenomai v2.5-rc1:
commit 8421cc62502db56b90d2912ea5f6049536177204
Author: Gilles Chanteperdrix <[email protected]> Date: Sat Nov 15 23:45:24 2008 +0000
Fault heaps memory on ARM
--- kernel/cobalt/arch/arm/Kconfig | 4 + kernel/cobalt/arch/arm/dovetail/Makefile | 5 + .../include/asm/xenomai/calibration.h | 39 ++++++++ .../dovetail/include/asm/xenomai/features.h | 30 ++++++ .../arm/dovetail/include/asm/xenomai/fptest.h | 52 ++++++++++ .../dovetail/include/asm/xenomai/machine.h | 72 ++++++++++++++ .../dovetail/include/asm/xenomai/syscall.h | 94 +++++++++++++++++++ .../dovetail/include/asm/xenomai/syscall32.h | 24 +++++ .../arm/dovetail/include/asm/xenomai/thread.h | 32 +++++++ .../dovetail/include/asm/xenomai/wrappers.h | 27 ++++++ kernel/cobalt/arch/arm/dovetail/machine.c | 44 +++++++++ 11 files changed, 423 insertions(+) create mode 100644 kernel/cobalt/arch/arm/dovetail/Makefile create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/calibration.h create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/features.h create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/fptest.h create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/machine.h create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/syscall.h create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/syscall32.h create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/thread.h create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/wrappers.h create mode 100644 kernel/cobalt/arch/arm/dovetail/machine.c
kernel/cobalt/arch/arm/Kconfig +4 -0
@@ -10,3 +10,7 @@ config XENO_ARCH_SYS3264 config XENO_ARCH_OUTOFLINE_XNLOCK bool default y + +config XENO_ARCH_IPIPE_COMPAT + def_bool DOVETAIL + select IPIPE_COMPAT
kernel/cobalt/arch/arm/dovetail/Makefile new file +5 -0
@@ -0,0 +1,5 @@ + +obj-$(CONFIG_XENOMAI) += xenomai.o +xenomai-y := machine.o + +ccflags-y := -I$(srctree)/arch/arm/xenomai/include -I$(srctree)/include/xenomai
kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/calibration.h new file +39 -0
@@ -0,0 +1,39 @@ +/* +* Copyright (C) 2001-2021 Philippe Gerum <[email protected]>. +* +* ARM port +* Copyright (C) 2005 Stelian Pop +* +* Xenomai is free software; you can redistribute it and/or modify it +* under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. +* +* Xenomai is distributed in the hope that it will be useful, but +* WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with Xenomai; if not, write to the Free Software +* Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA +* 02111-1307, USA. +*/ +#ifndef _COBALT_ARM_DOVETAIL_CALIBRATION_H +#define _COBALT_ARM_DOVETAIL_CALIBRATION_H + +static inline void xnarch_get_latencies (struct xnclock_gravity *p) +{ +unsigned int sched_latency; + +#if CONFIG_XENO_OPT_TIMING_SCHEDLAT !=0 + sched_latency =CONFIG_XENO_OPT_TIMING_SCHEDLAT ; +#else + sched_latency =5000 ; +#endif + p->user = sched_latency; + p->kernel =CONFIG_XENO_OPT_TIMING_KSCHEDLAT ; + p->irq =CONFIG_XENO_OPT_TIMING_IRQLAT ; +} + +#endif /* !_COBALT_ARM_DOVETAIL_CALIBRATION_H */
kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/features.h new file +30 -0
@@ -0,0 +1,30 @@ +/* +* Copyright (C) 2005 Philippe Gerum <[email protected]>. +* +* ARM port +* Copyright (C) 2005 Stelian Pop +* +* Xenomai is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation; either version 2 of the License, +* or (at your option) any later version. +* +* Xenomai is distributed in the hope that it will be useful, but +* WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with Xenomai; if not, write to the Free Software +* Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA +* 02111-1307, USA. +*/ +#ifndef _COBALT_ARM_DOVETAIL_FEATURES_H +#define _COBALT_ARM_DOVETAIL_FEATURES_H + +struct cobalt_featinfo ; +static inline void collect_arch_features (struct cobalt_featinfo *p) { } + +#include <asm/xenomai/uapi/features.h> + +#endif /* !_COBALT_ARM_DOVETAIL_FEATURES_H */
kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/fptest.h new file +52 -0
52 lines not on this page · raw message
kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/machine.h new file +72 -0
72 lines not on this page · raw message
kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/syscall.h new file +94 -0
94 lines not on this page · raw message
kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/syscall32.h new file +24 -0
@@ -0,0 +1,24 @@ +/* +* Copyright (C) 2014 Philippe Gerum <[email protected]>. +* +* Xenomai is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation; either version 2 of the License, +* or (at your option) any later version. +* +* Xenomai is distributed in the hope that it will be useful, but +* WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with Xenomai; if not, write to the Free Software +* Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA +* 02111-1307, USA. +*/ +#ifndef _COBALT_ARM_ASM_SYSCALL32_H +#define _COBALT_ARM_ASM_SYSCALL32_H + +#include <asm-generic/xenomai/syscall32.h> + +#endif /* !_COBALT_ARM_ASM_SYSCALL32_H */
kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/thread.h new file +32 -0
@@ -0,0 +1,32 @@ +/* +* Copyright (C) 2005 Stelian Pop +* +* Xenomai is free software; you can redistribute it and/or modify it +* under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. +* +* Xenomai is distributed in the hope that it will be useful, but +* WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with Xenomai; if not, write to the Free Software +* Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA +* 02111-1307, USA. +*/ +#ifndef _COBALT_ARM_DOVETAIL_THREAD_H +#define _COBALT_ARM_DOVETAIL_THREAD_H + +#include <asm-generic/xenomai/dovetail/thread.h> +#include <asm/traps.h> + +#define xnarch_fault_pc (__regs) ((__regs)->ARM_pc - (thumb_mode(__regs) ? 2 : 4)) +#define xnarch_fault_pf_p (__nr) ((__nr) == ARM_TRAP_ACCESS) +#define xnarch_fault_bp_p (__nr) ((current->ptrace & PT_PTRACED) && \ + ((__nr) == ARM_TRAP_BREAK || \ + (__nr) == ARM_TRAP_UNDEFINSTR)) +#define xnarch_fault_notify (__nr) (!xnarch_fault_bp_p(__nr)) + +#endif /* !_COBALT_ARM_DOVETAIL_THREAD_H */
kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/wrappers.h new file +27 -0
@@ -0,0 +1,27 @@ +/* +* Copyright (C) 2005 Philippe Gerum <[email protected]>. +* +* Xenomai is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation; either version 2 of the License, +* or (at your option) any later version. +* +* Xenomai is distributed in the hope that it will be useful, but +* WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with Xenomai; if not, write to the Free Software +* Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA +* 02111-1307, USA. +*/ +#ifndef _COBALT_ARM_ASM_WRAPPERS_H +#define _COBALT_ARM_ASM_WRAPPERS_H + +#include <asm-generic/xenomai/wrappers.h> /* Read the generic portion. */ + +#define __put_user_inatomic __put_user +#define __get_user_inatomic __get_user + +#endif /* _COBALT_ARM_ASM_WRAPPERS_H */
kernel/cobalt/arch/arm/dovetail/machine.c new file +44 -0
44 lines not on this page · raw message
Signature · 1 line
2.26.2
PATCH 9/17 lib/cobalt: Reorder low_init for having cobalt_use_legacy_tsc earlier available
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
This is needed as cobalt_arch_check_features() will make use of __cobalt_tsc_clockfreq via cobalt_use_legacy_tsc().
--- lib/cobalt/init.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/cobalt/init.c +1 -1
@@ -177,11 +177,11 @@ static void low_init(void)early_panic ("mlockall: %s" ,strerror (errno));trace_me ("memory locked" ); +cobalt_ticks_init (f->clock_freq);cobalt_features_init (f);cobalt_init_umm (f->vdso_offset);trace_me ("memory heaps mapped" );cobalt_init_current_keys (); -cobalt_ticks_init (f->clock_freq); }static int cobalt_init_2 (void );
Signature · 1 line
2.26.2
PATCH 10/17 lib/cobalt/arm: dovetail: skip detection of KUSER_TSC support
In reply to Jan KiszkaFrom: Philippe Gerum <[email protected]>
When Dovetail is enabled in the kernel, we get timestamps for common clocks (CLOCK_REALTIME, CLOCK_MONOTONIC) from the vDSO-based gettime() routine as usual. Do not expect KUSER_TSC support to be there in such a case, it is not.
--- lib/cobalt/arch/arm/features.c | 4 ++++ 1 file changed, 4 insertions(+)
lib/cobalt/arch/arm/features.c +4 -0
@@ -25,6 +25,7 @@#include <unistd.h> #include <limits.h> #include <cobalt/wrappers.h> +#include <cobalt/ticks.h> #include <asm/xenomai/syscall.h> #include <asm/xenomai/tsc.h> #include <asm/xenomai/features.h>
@@ -44,6 +45,9 @@ void cobalt_arch_check_features(struct cobalt_featinfo *finfo)int err, fd;void *addr; +if (!cobalt_use_legacy_tsc ()) +return ; +if (__xn_tscinfo.kinfo.counter !=NULL )return ;
Signature · 1 line
2.26.2
PATCH 11/17 lib/cobalt/arm64: dovetail: skip detection of KUSER_TSC support
In reply to Jan KiszkaFrom: Philippe Gerum <[email protected]>
When Dovetail is enabled in the kernel, we get timestamps for common clocks (CLOCK_REALTIME, CLOCK_MONOTONIC) from the vDSO-based gettime() routine as usual. Do not expect KUSER_TSC support to be there in such a case, it is not.
--- lib/cobalt/arch/arm64/features.c | 4 ++++ 1 file changed, 4 insertions(+)
lib/cobalt/arch/arm64/features.c +4 -0
@@ -25,6 +25,7 @@#include <unistd.h> #include <limits.h> #include <cobalt/wrappers.h> +#include <cobalt/ticks.h> #include <asm/xenomai/syscall.h> #include <asm/xenomai/tsc.h> #include <asm/xenomai/features.h>
@@ -44,6 +45,9 @@ void cobalt_arch_check_features(struct cobalt_featinfo *finfo)int err, fd;void *addr; +if (!cobalt_use_legacy_tsc ()) +return ; +if (__xn_tscinfo.kinfo.counter !=NULL )return ;
Signature · 1 line
2.26.2
PATCH 12/17 cobalt/tick: dovetail: improve accuracy of the host tick delay
In reply to Jan KiszkaFrom: Philippe Gerum <[email protected]>
Expiration dates of in-band timers are based on the monotonic time base read by ktime_get(), which is ok for us to use from non-NMI context.
--- kernel/cobalt/dovetail/tick.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-)
kernel/cobalt/dovetail/tick.c +8 -6
@@ -55,19 +55,21 @@ void pipeline_set_timer_shot(unsigned long delay) /* ns */ }static int proxy_set_next_ktime (ktime_t expires, -struct clock_event_device *proxy_dev) +struct clock_event_device *proxy_dev)/* hard irqs on/off */ {struct xnsched *sched; -ktime_t delta;unsigned long flags; +ktime_t delta;int ret;/* -* When Negative delta have been observed, we set delta zero. -* Or else exntimer_start() will return -ETIMEDOUT and do not -* trigger shot +* Expiration dates of in-band timers are based on the common +* monotonic time base. If the timeout date has already +* elapsed, make sure xntimer_start() does not fail with +* -ETIMEDOUT but programs the hardware for ticking +* immediately instead. */ - delta =ktime_sub (expires,ktime_get_mono_fast_ns ()); + delta =ktime_sub (expires,ktime_get ());if (delta <0 ) delta =0 ;
Signature · 1 line
2.26.2
PATCH 13/17 cobalt/tick: dovetail: Drop pointless inline specifier from pipeline_must_force_program_tick
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
--- include/cobalt/kernel/dovetail/pipeline/tick.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
include/cobalt/kernel/dovetail/pipeline/tick.h +1 -1
@@ -11,6 +11,6 @@ void pipeline_uninstall_tick_proxy(void);struct xnsched ; -inline bool pipeline_must_force_program_tick (struct xnsched *sched); +bool pipeline_must_force_program_tick (struct xnsched *sched);#endif /* !_COBALT_KERNEL_IPIPE_TICK_H */
Signature · 1 line
2.26.2
PATCH 14/17 cobalt/clock: dovetail: abstract interface to hardware TSC
In reply to Jan KiszkaFrom: Philippe Gerum <[email protected]>
Unlike the I-pipe, Dovetail does not need to expose the underlying timestamp counter to applications. The best equivalent to this counter would be the common monotonic clock, expressing time as a count of nanoseconds.
Fork the implementation accordingly between Dovetail and I-pipe configurations, depending on which pipeline flavour was detected at init.
--- include/cobalt/sys/cobalt.h | 2 ++ include/copperplate/clockobj.h | 12 ++++++++++-- lib/cobalt/arch/arm/include/asm/xenomai/tsc.h | 2 +- .../arch/arm64/include/asm/xenomai/tsc.h | 2 +- .../arch/powerpc/include/asm/xenomai/tsc.h | 2 +- lib/cobalt/arch/x86/include/asm/xenomai/tsc.h | 2 +- lib/cobalt/clock.c | 6 +++--- lib/cobalt/internal.c | 18 ++++++++++++++++-- lib/copperplate/clockobj.c | 12 +----------- testsuite/smokey/tsc/tsc.c | 2 +- 10 files changed, 37 insertions(+), 23 deletions(-)
include/cobalt/sys/cobalt.h +2 -0
@@ -134,6 +134,8 @@ void cobalt_register_tsd_hook(struct cobalt_tsd_hook *th);void cobalt_assert_nrt (void ); +unsigned long long cobalt_read_tsc (void ); +extern int __cobalt_control_bind;#ifdef __cplusplus
include/copperplate/clockobj.h +10 -2
@@ -70,8 +70,6 @@ void clockobj_get_distance(struct clockobj *clkobj,const struct itimerspec *itm, struct timespec *delta); -ticks_t clockobj_get_tsc (void ); -void clockobj_caltime_to_timeout (struct clockobj *clkobj,const struct tm *tm,unsigned long rticks,struct timespec *ts);
@@ -112,6 +110,7 @@ void __clockobj_ticks_to_timespec(struct clockobj *clkobj,#ifdef CONFIG_XENO_COBALT #include <cobalt/ticks.h> +#include <cobalt/sys/cobalt.h> /* * The Cobalt core exclusively deals with aperiodic timings, so a
@@ -121,6 +120,13 @@ void __clockobj_ticks_to_timespec(struct clockobj *clkobj,* equivalent to Copperplate TSC units, and Copperplate ticks are * periods of the reference clockobj which Cobalt does not know about. */ + +static inline ticks_t clockobj_get_tsc (void ) +{ +/* Guaranteed to be the source of CLOCK_COPPERPLATE. */ +return cobalt_read_tsc (); +} +static inline sticks_t clockobj_ns_to_tsc (sticks_t ns) {return cobalt_ns_to_ticks (ns);
@@ -142,6 +148,8 @@ void clockobj_ns_to_timespec(ticks_t ns, struct timespec *ts)#else /* CONFIG_XENO_MERCURY */ +ticks_t clockobj_get_tsc (void ); +static inline sticks_t clockobj_ns_to_tsc (sticks_t ns) {return ns;
lib/cobalt/arch/arm/include/asm/xenomai/tsc.h +1 -1
@@ -40,7 +40,7 @@ struct __xn_full_tscinfo {extern struct __xn_full_tscinfo __xn_tscinfo;static inline __attribute__((always_inline)) -unsigned long long cobalt_read_tsc (void ) +unsigned long long cobalt_read_legacy_tsc (void ) {return __xn_tscinfo.kuser_tsc_get (__xn_tscinfo.kinfo.counter); }
lib/cobalt/arch/arm64/include/asm/xenomai/tsc.h +1 -1
@@ -46,7 +46,7 @@ static inline uint64_t get_counter(void) }static inline __attribute__((always_inline)) -unsigned long long cobalt_read_tsc (void ) +unsigned long long cobalt_read_legacy_tsc (void ) {return get_counter (); }
lib/cobalt/arch/powerpc/include/asm/xenomai/tsc.h +1 -1
@@ -18,7 +18,7 @@ #ifndef _LIB_COBALT_POWERPC_TSC_H #define _LIB_COBALT_POWERPC_TSC_H -static inline unsigned long long cobalt_read_tsc(void) +static inline unsigned long long cobalt_read_legacy_tsc(void) { union { unsigned long long t;
lib/cobalt/arch/x86/include/asm/xenomai/tsc.h +1 -1
@@ -19,7 +19,7 @@#ifndef _LIB_COBALT_X86_TSC_H#define _LIB_COBALT_X86_TSC_H -static inline unsigned long long cobalt_read_tsc (void ) +static inline unsigned long long cobalt_read_legacy_tsc (void ) {#ifdef __i386__unsigned long long t;
lib/cobalt/clock.c +3 -3
@@ -136,7 +136,7 @@ static int __do_clock_host_realtime(struct timespec *ts)* mechanism in the kernel. */ unsynced_read_block(&tmp, &hostrt_data->lock) { - now =cobalt_read_tsc (); + now =cobalt_read_legacy_tsc (); base = hostrt_data->cycle_last; mask = hostrt_data->mask; mult = hostrt_data->mult;
@@ -175,12 +175,12 @@ static int gettime_via_tsc(clockid_t clock_id, struct timespec *tp)break ;case CLOCK_MONOTONIC :case CLOCK_MONOTONIC_RAW : - ns =cobalt_ticks_to_ns (cobalt_read_tsc ()); + ns =cobalt_ticks_to_ns (cobalt_read_legacy_tsc ()); tp->tv_sec =cobalt_divrem_billion (ns, &rem); tp->tv_nsec = rem;return 0 ;case CLOCK_REALTIME : - ns =cobalt_ticks_to_ns (cobalt_read_tsc ()); + ns =cobalt_ticks_to_ns (cobalt_read_legacy_tsc ()); ns += cobalt_vdso->wallclock_offset; tp->tv_sec =cobalt_divrem_billion (ns, &rem); tp->tv_nsec = rem;
lib/cobalt/internal.c +16 -2
@@ -31,6 +31,8 @@#include <stdarg.h> #include <pthread.h> #include <asm/xenomai/syscall.h> +#include <asm/xenomai/tsc.h> +#include <cobalt/ticks.h> #include <cobalt/sys/cobalt.h> #include "internal.h"
@@ -553,7 +555,7 @@ int cobalt_xlate_schedparam(int policy,if (priority <0 ) priority = -priority; - +memset (param,0 ,sizeof (*param)); param->sched_priority = priority;
@@ -566,6 +568,18 @@ void cobalt_assert_nrt(void)pthread_kill (pthread_self (),SIGDEBUG ); } +unsigned long long cobalt_read_tsc (void ) +{ +struct timespec ts; + +if (cobalt_use_legacy_tsc ()) +return cobalt_read_legacy_tsc (); + +__cobalt_vdso_gettime (CLOCK_MONOTONIC , &ts); + +return ts.tv_sec *1000000000ULL + ts.tv_nsec; +} +unsigned int cobalt_features;void cobalt_features_init (struct cobalt_featinfo *f)
@@ -574,4 +588,4 @@ void cobalt_features_init(struct cobalt_featinfo *f)/* Trigger arch specific feature initialization */ cobalt_arch_check_features (f); -} +}
lib/copperplate/clockobj.c +1 -11
@@ -229,22 +229,12 @@ int clockobj_set_resolution(struct clockobj *clkobj, unsigned int resolution_ns)#ifdef CONFIG_XENO_COBALT #include <cobalt/arith.h> -#include <asm/xenomai/tsc.h> +#include <cobalt/sys/cobalt.h> #ifdef CONFIG_XENO_COPPERPLATE_CLOCK_RESTRICTED #error "restricted CLOCK_COPPERPLATE not available"#endif -/* -* NOTE: we can't inline this routine, as we don't want to expose -* lib/cobalt/arch/.../include/asm/xenomai/tsc.h. -*/ -ticks_t clockobj_get_tsc (void ) -{ -/* Guaranteed to be the source of CLOCK_COPPERPLATE. */ -return cobalt_read_tsc (); -} -ticks_t clockobj_get_time (struct clockobj *clkobj) {ticks_t ns =cobalt_ticks_to_ns_rounded (cobalt_read_tsc ());
testsuite/smokey/tsc/tsc.c +1 -1
@@ -30,7 +30,7 @@#include <smokey/smokey.h> -#include <asm/xenomai/tsc.h> +#include <cobalt/sys/cobalt.h> #define DURATION 10000000
Signature · 1 line
2.26.2
PATCH 15/17 cobalt/arm64: dovetail: add architecture bits
In reply to Jan KiszkaFrom: Philippe Gerum <[email protected]>
At this chance, fix stale and missing copyright notices. Specifically, Dmitriy Cherkasov <[email protected]> is the original author of the Xenomai/arm64 port, based on the ARM implementation; add credit where it is due.
In the same move, introduce SPDX tags as replacement for GPL boilerplate for the altered files.
--- kernel/cobalt/arch/arm64/dovetail/Makefile | 5 ++ .../include/asm/xenomai/calibration.h | 24 +++++++ .../dovetail/include/asm/xenomai/features.h | 15 ++++ .../dovetail/include/asm/xenomai/fptest.h | 35 ++++++++++ .../dovetail/include/asm/xenomai/machine.h | 33 +++++++++ .../dovetail/include/asm/xenomai/syscall.h | 68 +++++++++++++++++++ .../dovetail/include/asm/xenomai/syscall32.h | 12 ++++ .../dovetail/include/asm/xenomai/thread.h | 22 ++++++ .../dovetail/include/asm/xenomai/wrappers.h | 15 ++++ kernel/cobalt/arch/arm64/dovetail/machine.c | 41 +++++++++++ .../arm64/ipipe/include/asm/xenomai/machine.h | 3 + kernel/cobalt/arch/arm64/ipipe/machine.c | 3 + 12 files changed, 276 insertions(+) create mode 100644 kernel/cobalt/arch/arm64/dovetail/Makefile create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/calibration.h create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/features.h create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/fptest.h create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/machine.h create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/syscall.h create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/syscall32.h create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/thread.h create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/wrappers.h create mode 100644 kernel/cobalt/arch/arm64/dovetail/machine.c
kernel/cobalt/arch/arm64/dovetail/Makefile new file +5 -0
@@ -0,0 +1,5 @@ +obj-$(CONFIG_XENOMAI) += xenomai.o + +xenomai-y := machine.o + +ccflags-y := -I$(srctree)/arch/arm64/xenomai/include -I$(srctree)/include/xenomai
kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/calibration.h new file +24 -0
@@ -0,0 +1,24 @@ +/* +* SPDX-License-Identifier: GPL-2.0 +* +* Copyright (C) 2015 Philippe Gerum <[email protected]> +*/ + +#ifndef _COBALT_ARM64_DOVETAIL_CALIBRATION_H +#define _COBALT_ARM64_DOVETAIL_CALIBRATION_H + +static inline void xnarch_get_latencies (struct xnclock_gravity *p) +{ +unsigned int sched_latency; + +#if CONFIG_XENO_OPT_TIMING_SCHEDLAT !=0 + sched_latency =CONFIG_XENO_OPT_TIMING_SCHEDLAT ; +#else + sched_latency =5000 ; +#endif + p->user = sched_latency; + p->kernel =CONFIG_XENO_OPT_TIMING_KSCHEDLAT ; + p->irq =CONFIG_XENO_OPT_TIMING_IRQLAT ; +} + +#endif /* !_COBALT_ARM64_DOVETAIL_CALIBRATION_H */
kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/features.h new file +15 -0
@@ -0,0 +1,15 @@ +/* +* SPDX-License-Identifier: GPL-2.0 +* +* Copyright (C) 2015 Philippe Gerum <[email protected]> +*/ + +#ifndef _COBALT_ARM64_DOVETAIL_FEATURES_H +#define _COBALT_ARM64_DOVETAIL_FEATURES_H + +struct cobalt_featinfo ; +static inline void collect_arch_features (struct cobalt_featinfo *p) { } + +#include <asm/xenomai/uapi/features.h> + +#endif /* !_COBALT_ARM64_DOVETAIL_FEATURES_H */
kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/fptest.h new file +35 -0
@@ -0,0 +1,35 @@ +/* +* SPDX-License-Identifier: GPL-2.0 +* +* Copyright (C) 2006 Gilles Chanteperdrix <[email protected]> +*/ + +#ifndef _COBALT_ARM64_DOVETAIL_FPTEST_H +#define _COBALT_ARM64_DOVETAIL_FPTEST_H + +#include <linux/errno.h> +#include <asm/xenomai/uapi/fptest.h> +#include <asm/hwcap.h> + +#define have_fp (ELF_HWCAP & HWCAP_FP) + +static inline int fp_kernel_supported (void ) +{ +return 0 ; +} + +static inline int fp_linux_begin (void ) +{ +return -ENOSYS ; +} + +static inline void fp_linux_end (void ) +{ +} + +static inline int fp_detect (void ) +{ +return have_fp ? __COBALT_HAVE_FPU :0 ; +} + +#endif /* !_COBALT_ARM64_DOVETAIL_FPTEST_H */
kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/machine.h new file +33 -0
@@ -0,0 +1,33 @@ +/* +* SPDX-License-Identifier: GPL-2.0 +* +* Copyright (C) 2015 Dmitriy Cherkasov <[email protected]> +*/ + +#ifndef _COBALT_ARM64_DOVETAIL_MACHINE_H +#define _COBALT_ARM64_DOVETAIL_MACHINE_H + +#include <linux/version.h> +#include <asm/byteorder.h> +#include <cobalt/kernel/assert.h> + +/* D-side always behaves as PIPT on AArch64 (see arch/arm64/include/asm/cachetype.h) */ +#define xnarch_cache_aliasing () 0 + +static inline __attribute_const__ unsigned long ffnz (unsigned long ul) +{ +int __r; + +/* zero input is not valid */ +XENO_WARN_ON (COBALT , ul ==0 ); + + __asm__ ("rbit\t%0, %1\n" +"clz\t%0, %0\n" + :"=r" (__r) :"r" (ul) :"cc" ); + +return __r; +} + +#include <asm-generic/xenomai/machine.h> + +#endif /* !_COBALT_ARM64_DOVETAIL_MACHINE_H */
kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/syscall.h new file +68 -0
68 lines not on this page · raw message
kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/syscall32.h new file +12 -0
@@ -0,0 +1,12 @@ +/* +* SPDX-License-Identifier: GPL-2.0 +* +* Copyright (C) 2014 Philippe Gerum <[email protected]> +*/ + +#ifndef _COBALT_ARM64_DOVETAIL_SYSCALL32_H +#define _COBALT_ARM64_DOVETAIL_SYSCALL32_H + +#include <asm-generic/xenomai/syscall32.h> + +#endif /* !_COBALT_ARM64_DOVETAIL_SYSCALL32_H */
kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/thread.h new file +22 -0
@@ -0,0 +1,22 @@ +/* +* SPDX-License-Identifier: GPL-2.0 +* +* Copyright (C) 2015 Dmitriy Cherkasov <[email protected]> +* Copyright (C) 2021 Philippe Gerum <[email protected]> +*/ + +#ifndef _COBALT_ARM64_DOVETAIL_THREAD_H +#define _COBALT_ARM64_DOVETAIL_THREAD_H + +#include <asm-generic/xenomai/dovetail/thread.h> +#include <asm/dovetail.h> + +#define xnarch_fault_pc (__regs) ((unsigned long)((__regs)->pc - 4))/* XXX ? */ + +#define xnarch_fault_pf_p (__nr) ((__nr) == ARM64_TRAP_ACCESS) +#define xnarch_fault_bp_p (__nr) ((current->ptrace & PT_PTRACED) && \ + ((__nr) == ARM64_TRAP_DEBUG || (__nr) == ARM64_TRAP_UNDI)) + +#define xnarch_fault_notify (__nr) (!xnarch_fault_bp_p(__nr)) + +#endif /* !_COBALT_ARM64_DOVETAIL_THREAD_H */
kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/wrappers.h new file +15 -0
@@ -0,0 +1,15 @@ +/* +* SPDX-License-Identifier: GPL-2.0 +* +* Copyright (C) 2005 Philippe Gerum <[email protected]> +*/ + +#ifndef _COBALT_ARM64_ASM_WRAPPERS_H +#define _COBALT_ARM64_ASM_WRAPPERS_H + +#include <asm-generic/xenomai/wrappers.h> /* Read the generic portion. */ + +#define __put_user_inatomic __put_user +#define __get_user_inatomic __get_user + +#endif /* _COBALT_ARM64_ASM_WRAPPERS_H */
kernel/cobalt/arch/arm64/dovetail/machine.c new file +41 -0
41 lines not on this page · raw message
kernel/cobalt/arch/arm64/ipipe/include/asm/xenomai/machine.h +3 -0
@@ -4,6 +4,9 @@* ARM port * Copyright (C) 2005 Stelian Pop * +* ARM64 port +* Copyright (C) 2015 Dmitriy Cherkasov <[email protected]> +* * Xenomai is free software; you can redistribute it and/or * modify it under the terms of the GNU General Public License as * published by the Free Software Foundation, Inc., 675 Mass Ave,
kernel/cobalt/arch/arm64/ipipe/machine.c +3 -0
@@ -1,6 +1,9 @@/** * Copyright (C) 2005 Stelian Pop * +* ARM64 port +* Copyright (C) 2015 Dmitriy Cherkasov <[email protected]> +* * Xenomai is free software; you can redistribute it and/or * modify it under the terms of the GNU General Public License as * published by the Free Software Foundation, Inc., 675 Mass Ave,
Signature · 1 line
2.26.2
PATCH 16/17 cobalt/syscall: Account for different syscall argument marshaling
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
I-pipe makes sure that arguments of compat calls are ordered just like native calls. Dovetail does not. But it is better to use the kernel's syscall_get_arguments for retrieving the arguments anyway. Introduce pipeline_get_syscall_args to abstract that difference.
Xenomai/ARM syscalls have an unconventional argument marshalling, with ARM_ORIG_r0 containing the Xenomai syscall number (ORed with __COBALT_SYSCALL_BIT), followed by the first actual argument to the syscall in ARM_r1.
For this reason, using syscall_get_arguments() to collect the syscall parameters won't yield the expected result, with regs[0] matching __xn_reg_sys(regs) instead of __xn_reg_arg1(regs).
Allow Dovetail-enabled architectures to override the generic implementation of pipeline_get_syscall_args(), provide a working one for ARM.
--- .../cobalt/kernel/dovetail/pipeline/pipeline.h | 11 +++++++++++ include/cobalt/kernel/ipipe/pipeline/pipeline.h | 12 ++++++++++++ .../arm/dovetail/include/asm/xenomai/syscall.h | 17 ++++++++++++----- .../dovetail/include/asm/xenomai/syscall.h | 5 ----- .../x86/dovetail/include/asm/xenomai/syscall.h | 5 ----- .../include/asm-generic/xenomai/syscall.h | 7 ------- kernel/cobalt/posix/syscall.c | 14 ++++++++++---- 7 files changed, 45 insertions(+), 26 deletions(-)
include/cobalt/kernel/dovetail/pipeline/pipeline.h +11 -0
@@ -9,6 +9,8 @@#include <linux/cpumask.h> #include <cobalt/kernel/assert.h> #include <asm/xenomai/features.h> +#include <asm/xenomai/syscall.h> +#include <asm/syscall.h> #include <pipeline/machine.h> typedef unsigned long spl_t ;
@@ -95,4 +97,13 @@ static inline void pipeline_collect_features(struct cobalt_featinfo *f) f->clock_freq = 0; /* N/A */ } +#ifndef pipeline_get_syscall_args +static inline void pipeline_get_syscall_args (struct task_struct *task, +struct pt_regs *regs, +unsigned long *args) +{ +syscall_get_arguments (task, regs, args); +} +#endif /* !pipeline_get_syscall_args */ +#endif /* !_COBALT_KERNEL_DOVETAIL_PIPELINE_H */
include/cobalt/kernel/ipipe/pipeline/pipeline.h +12 -0
@@ -11,6 +11,7 @@#include <pipeline/machine.h> #include <asm/xenomai/features.h> +#include <asm/xenomai/syscall.h> #define xnsched_primary_domain cobalt_pipeline.domain
@@ -81,4 +82,15 @@ static inline void pipeline_collect_features(struct cobalt_featinfo *f) f->clock_freq = cobalt_pipeline.clock_freq; } +static inline void pipeline_get_syscall_args (struct task_struct *task, +struct pt_regs *regs, +unsigned long *args) +{ + *args++ =__xn_reg_arg1 (regs); + *args++ =__xn_reg_arg2 (regs); + *args++ =__xn_reg_arg3 (regs); + *args++ =__xn_reg_arg4 (regs); + *args =__xn_reg_arg5 (regs); +} +#endif /* !_COBALT_KERNEL_IPIPE_PIPELINE_H */
kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/syscall.h +12 -5
@@ -59,11 +59,6 @@ })#define __xn_reg_rval (__regs) ((__regs)->ARM_r0) -#define __xn_reg_arg1 (__regs) ((__regs)->ARM_r1) -#define __xn_reg_arg2 (__regs) ((__regs)->ARM_r2) -#define __xn_reg_arg3 (__regs) ((__regs)->ARM_r3) -#define __xn_reg_arg4 (__regs) ((__regs)->ARM_r4) -#define __xn_reg_arg5 (__regs) ((__regs)->ARM_r5)#define __xn_reg_pc (__regs) ((__regs)->ARM_ip)#define __xn_reg_sp (__regs) ((__regs)->ARM_sp)
@@ -91,4 +86,16 @@ int xnarch_local_syscall(unsigned long a1, unsigned long a2, return -ENOSYS; } +#define pipeline_get_syscall_args pipeline_get_syscall_args +static inline void pipeline_get_syscall_args (struct task_struct *task, +struct pt_regs *regs, +unsigned long *args) +{ + args[0 ] = regs->ARM_r1; + args[1 ] = regs->ARM_r2; + args[2 ] = regs->ARM_r3; + args[3 ] = regs->ARM_r4; + args[4 ] = regs->ARM_r5; +} +#endif /* !_COBALT_ARM_DOVETAIL_SYSCALL_H */
kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/syscall.h +0 -5
@@ -23,11 +23,6 @@#define __xn_syscall (__regs) ((unsigned long)(__xn_reg_sys(__regs) & ~__COBALT_SYSCALL_BIT))#define __xn_reg_rval (__regs) ((__regs)->regs[0]) -#define __xn_reg_arg1 (__regs) ((__regs)->regs[0]) -#define __xn_reg_arg2 (__regs) ((__regs)->regs[1]) -#define __xn_reg_arg3 (__regs) ((__regs)->regs[2]) -#define __xn_reg_arg4 (__regs) ((__regs)->regs[3]) -#define __xn_reg_arg5 (__regs) ((__regs)->regs[4])#define __xn_reg_pc (__regs) ((__regs)->pc)#define __xn_reg_sp (__regs) ((__regs)->sp)
kernel/cobalt/arch/x86/dovetail/include/asm/xenomai/syscall.h +0 -5
@@ -29,11 +29,6 @@*/ #define __xn_reg_sys (regs) ((regs)->orig_ax)#define __xn_reg_rval (regs) ((regs)->ax) -#define __xn_reg_arg1 (regs) ((regs)->di) -#define __xn_reg_arg2 (regs) ((regs)->si) -#define __xn_reg_arg3 (regs) ((regs)->dx) -#define __xn_reg_arg4 (regs) ((regs)->r10) -#define __xn_reg_arg5 (regs) ((regs)->r8)#define __xn_reg_pc (regs) ((regs)->ip)#define __xn_reg_sp (regs) ((regs)->sp)
kernel/cobalt/include/asm-generic/xenomai/syscall.h +0 -7
@@ -36,13 +36,6 @@#define access_wok (addr, size) access_ok(VERIFY_WRITE, (addr), (size))#endif -#define __xn_reg_arglist (regs) \ - __xn_reg_arg1(regs), \ - __xn_reg_arg2(regs), \ - __xn_reg_arg3(regs), \ - __xn_reg_arg4(regs), \ - __xn_reg_arg5(regs) -#define __xn_copy_from_user (dstP, srcP, n) raw_copy_from_user(dstP, srcP, n)#define __xn_copy_to_user (dstP, srcP, n) raw_copy_to_user(dstP, srcP, n)#define __xn_put_user (src, dstP) __put_user(src, dstP)
kernel/cobalt/posix/syscall.c +10 -4
@@ -486,6 +486,7 @@ int handle_head_syscall(bool caller_is_relaxed, struct pt_regs *regs)struct xnthread *thread;cobalt_syshand handler;struct task_struct *p; +unsigned long args[6 ];unsigned int nr, code;long ret;
@@ -592,7 +593,10 @@ restart:* handler (lostage ones), or rejected by allowed_syscall(). */ - ret =handler (__xn_reg_arglist (regs)); + p = current; +pipeline_get_syscall_args (p, regs, args); + + ret =handler (args[0 ], args[1 ], args[2 ], args[3 ], args[4 ]);if (ret == -ENOSYS && (sysflags & __xn_exec_adaptive)) {if (switched) { ret =xnthread_harden ();
@@ -611,7 +615,6 @@ done:__xn_status_return (regs, ret); sigs =0 ;if (!xnsched_root_p ()) { - p = current;if (signal_pending (p) ||xnthread_test_info (thread,XNKICKED )) { sigs =1 ;
@@ -677,6 +680,7 @@ int handle_root_syscall(struct pt_regs *regs)struct xnthread *thread;cobalt_syshand handler;struct task_struct *p; +unsigned long args[6 ];unsigned int nr, code;long ret;
@@ -735,7 +739,10 @@ restart:xnthread_propagate_schedparam (thread); } - ret =handler (__xn_reg_arglist (regs)); + p = current; +pipeline_get_syscall_args (p, regs, args); + + ret =handler (args[0 ], args[1 ], args[2 ], args[3 ], args[4 ]);if (ret == -ENOSYS && (sysflags & __xn_exec_adaptive)) { sysflags ^= __xn_exec_histage;if (switched) {
@@ -756,7 +763,6 @@ restart:* just invoked, so make sure to fetch it. */ thread =xnthread_current (); - p = current;if (signal_pending (p)) { sigs =1 ;prepare_for_signal (p, thread, regs, sysflags);
Signature · 1 line
2.26.2
PATCH 17/17 ci: Add arm, arm64 and x86 dovetail targets for kernel 5.10
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
--- .gitlab-ci.yml | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+)
.gitlab-ci.yml +26 -0
@@ -21,6 +21,7 @@ variables: USE_CCACHE: "1" CCACHE_MAXSIZE: "1G" IPIPE_MIRROR_URL: "https://source.denx.de/Xenomai" + DOVETAIL_URL: "https://source.denx.de/Xenomai/linux-dovetail.git" PACKAGES: "gcc-8 u-boot-tools git make bc bison libelf-dev autotools-dev \ autoconf autogen libtool pkg-config ccache flex libssl-dev" PACKAGES_arm: "gcc-8-arm-linux-gnueabihf libc6-dev-armhf-cross"
@@ -72,6 +73,7 @@ variables: - ./scripts/config -d DEBUG_INFO # I-pipe and Xenomai settings - ./scripts/config -e IPIPE + - ./scripts/config -e DOVETAIL - ./scripts/config -e XENOMAI - ./scripts/config -e XENO_OPT_SCHED_CLASSES - ./scripts/config -e XENO_OPT_SCHED_WEAK
@@ -170,6 +172,14 @@ variables: paths: - .ccache +arm-5.10-head: + extends: .build + variables: + ARCH: arm + PIPELINE_REV: v5.10.y-dovetail-rebase + PIPELINE_KERNEL: ${DOVETAIL_URL} + KERNEL_DEFCONFIG: multi_v7_defconfig + arm-5.4-release: extends: .build variables:
@@ -186,6 +196,14 @@ arm-4.19-release: PIPELINE_KERNEL: ${IPIPE_MIRROR_URL}/ipipe-arm.git KERNEL_DEFCONFIG: multi_v7_defconfig +arm64-5.10-head: + extends: .build + variables: + ARCH: arm64 + PIPELINE_REV: v5.10.y-dovetail-rebase + PIPELINE_KERNEL: ${DOVETAIL_URL} + KERNEL_DEFCONFIG: defconfig + arm64-5.4-release: extends: .build variables:
@@ -202,6 +220,14 @@ arm64-4.19-cip-release: PIPELINE_KERNEL: ${IPIPE_MIRROR_URL}/ipipe-arm64.git KERNEL_DEFCONFIG: defconfig +x86-5.10-head: + extends: .build + variables: + ARCH: x86 + PIPELINE_REV: v5.10.y-dovetail-rebase + PIPELINE_KERNEL: ${DOVETAIL_URL} + KERNEL_DEFCONFIG: x86_64_defconfig + x86-5.4-head: extends: .build variables:
Signature · 1 line
2.26.2
Quoted 4 lines · Jan Kiszka
On top, I would also still like to understand - and ideally address prior to releasing a new major version - if we cannot align the ARM syscall ABI of Xenomai to the kernel /wrt argument passing. Now is a good chance to do that.
That would be fairly simple implementation-wise, problem though: this would require to add this change to the next I-pipe/arm releases, invalidating all previous I-pipe releases for 3.2 in the same move. In theory, all existing I-pipe/arm patches starting from kernel 3.10 can run the -next branch (although it may be better to stick with 4.14+ in practice).
Signature · 1 line
Philippe.
Quoted 8 lines · Philippe Gerum
Quoted 4 lines · Jan Kiszka
On top, I would also still like to understand - and ideally address prior to releasing a new major version - if we cannot align the ARM syscall ABI of Xenomai to the kernel /wrt argument passing. Now is a good chance to do that.
Quoted 7 lines
That would be fairly simple implementation-wise, problem though: this would require to add this change to the next I-pipe/arm releases, invalidating all previous I-pipe releases for 3.2 in the same move. In theory, all existing I-pipe/arm patches starting from kernel 3.10 can run the -next branch (although it may be better to stick with 4.14+ in practice).
4.14 is long dead, 4.19 could be the lowest target. Yes, that might involve some some complications. Maybe we could add a feature flag for the new ABI, start with dovetail and then look into lifting 4.19 and 5.4 I-pipe.
Jan
Signature · 2 lines
Siemens AG, T RDA IOT Corporate Competence Center Embedded Linux
Quoted 17 lines · Jan Kiszka
Quoted 8 lines · Philippe Gerum
Quoted 4 lines · Jan Kiszka
On top, I would also still like to understand - and ideally address prior to releasing a new major version - if we cannot align the ARM syscall ABI of Xenomai to the kernel /wrt argument passing. Now is a good chance to do that.
Quoted 7 lines
That would be fairly simple implementation-wise, problem though: this would require to add this change to the next I-pipe/arm releases, invalidating all previous I-pipe releases for 3.2 in the same move. In theory, all existing I-pipe/arm patches starting from kernel 3.10 can run the -next branch (although it may be better to stick with 4.14+ in practice).
Quoted 5 lines
4.14 is long dead, 4.19 could be the lowest target. Yes, that might involve some some complications. Maybe we could add a feature flag for the new ABI, start with dovetail and then look into lifting 4.19 and 5.4 I-pipe.
Dovetail already follows the ARM EABI, only using a marker to distinguish in-band syscalls from foreign ones in r7, on all archs. This said, I plan to work on another option for conveying oob syscalls based on prctl() asap, this would be saner than invalidating a whole range of syscall numbers for the regular kernel. In addition, this might help enabling tools like valgrind at little/no cost. The earlier the better to investigate this, I'd rather not have to maintain multiple ABIs in Dovetail.
Signature · 1 line
Philippe.