Linear temporal logic monitors for RT application
Subsystems (8)
v13 cover Linear temporal logic monitors for RT application
Real-time applications may have design flaws causing them to have unexpected latency. For example, the applications may raise page faults, or may be blocked trying to take a mutex without priority inheritance.
However, while attempting to implement DA monitors for these real-time rules, deterministic automaton is found to be inappropriate as the specification language. The automaton is complicated, hard to understand, and error-prone.
For these cases, linear temporal logic is found to be more suitable. The LTL is more concise and intuitive.
This series adds support for LTL RV monitor, and use it to implement two monitors for reporting problems with real-time tasks.
v12->v13 https://lore.kernel.org/linux-trace-kernel/[email protected]
- keep consistent format around __printf(1, 0)
v11->v12 https://lore.kernel.org/linux-trace-kernel/[email protected]
- drop the applied patches
- fixup build failure when CONFIG_PRINTK=n
v10->v11 https://lore.kernel.org/linux-trace-kernel/[email protected]/
- Split the patch adding LTL support into three smaller patches
- Keep comments and document within 80 columns
- Add a '&' to memset's argument to make it look more natural
Note: all changed patches are explicitly marked so below the --- line
v9->v10 https://lore.kernel.org/lkml/[email protected]
- rebase onto v6.16-rc1 (only the vpanic patch is changed)
- riscv: move page fault tracepoints back to be after kprobe pagefault handling
- drop x86 tracepoint patches, they have already been applied
- drop arm64 tracepoint patch. It requires further discussion and will be sent separately.
- pagefault monitor: set dependency on x86 and riscv, only these archs have the required tracepoints right now.
v8->v9 https://lore.kernel.org/lkml/[email protected]/
- Move page faults tracepoints to be before kprobe pagefault handling
- Add guidance into Kconfig entries whether they should be enabled
- Replace TRACE_EVENT with DECLARE_EVENT_CLASS for the tracepoint classes, so that the class names are not wrongly exposed to userspace
v7->v8 https://lore.kernel.org/lkml/[email protected]/
- Fix some pylint warnings
- Fix some bugs with some currently-unused operators in the ltl2ba.py script
- sleep monitor: Allow all FUTEX_WAIT_* as valid sleep reason
v6->v7 https://lore.kernel.org/lkml/[email protected]/
- Add missing parameter description for vpanic()
- Remove the now-redundant CFLAGS_fault.o for x86
- Change #if to #ifdef to resolve a build warning
- rtapp/sleep monitor:
+ Handle the case where an RT task "aborts" the sleep by setting state to TASK_RUNNING. This case previously caused a false positive. Fix it by adding "ABORT_SLEEP" as an RT-safe wake.
+ Also allow CLOCK_TAI for real-time tasks.
v5->v6 https://lore.kernel.org/lkml/[email protected]
- sleep monitor: Drop the block_on_rt_mutex tracepoints. The contention tracepoints are sufficient.
v4->v5 https://lore.kernel.org/lkml/[email protected]
- sleep monitor: Fix a false positive due to a race with waking and scheduling.
- sleep monitor: Add block_on_rt_mutex tracepoints and use them for BLOCK_ON_RT_MUTEX, instead of trace_sched_pi_setprio
- sleep monitor: tighten the rule on nanosleep: only clock_nanosleep() with TIMER_ABSTIME and CLOCK_MONOTONIC is allowed
- add comments explaining why it is correct to treat PI-boosted tasks as real-time tasks.
It should be noted that due to the changes in v5, 'perf' does not work as well as before, because sometimes the errors happen out of the real-time tasks' contexts. Fixing this is left for future work.
stress-ng is also far noisier in v5, because the rule on nanosleep is tightened.
v3->v4 https://lore.kernel.org/lkml/[email protected]
- support deadline tasks
- rtapp_sleep: use sched_pi_setprio tracepoint instead of contention tracepoints for BLOCK_ON_RT_MUTEX, so that proxy lock is covered.
- fix the scripts generating an "slightly" incorrect verification automaton
- makes rtapp monitor depends on RV_PER_TASK_MONITORS >= 2
- make the event tracepoint output a bit more readable
- some documentation's format fixes
v2->v3 https://lore.kernel.org/lkml/[email protected]/
- fix a problem with sleep monitor's specification (around KTHREAD_SHOULD_STOP)
- merge the patches that move the dot2k/rvgen scripts around
- pull panic/printk changes into separate patches
- fixup some build errors
- fixup monitor's init function return code
- fix some flake8 warnings with the scripts
- add some references to LTL documentation
- fixup some mistakes with rtapp documentation
- fixup capitalization mistake with monitor_synthesis.rst
- remove the now-redundant macro RV_PER_TASK_MONITORS
v1->v2 https://lore.kernel.org/lkml/[email protected]/
- Integrate the LTL scripts into the existing dot2k tool, taking advantage of the existing monitor generation scripts.
- Switch the struct ltl_monitor to use bitmap instead of an array, to optimize memory usage.
- Correct the generated code to be non-deterministic state machine, instead of deterministic state machine
- Put common code for all LTL monitors into a single file (include/rv/ltl_monitor.h), reducing code duplication
- Change the LTL monitors to make user of container. Add a bug fix to container while at it.
- Make the number of per-task monitor configurable
Nam Cao (12):
rv: Add #undef TRACE_INCLUDE_FILE
printk: Make vprintk_deferred() public
panic: Add vpanic()
rv: Let the reactors take care of buffers
rv: rename CONFIG_DA_MON_EVENTS to CONFIG_RV_MON_EVENTS
rv: Add support for LTL monitors
rv: Add rtapp container monitor
riscv: mm: Add page fault trace points
rv: Add rtapp_pagefault monitor
rv: Add rtapp_sleep monitor
rv: Add documentation for rtapp monitor
rv: Allow to configure the number of per-task monitor
Documentation/trace/rv/index.rst | 1 + Documentation/trace/rv/monitor_rtapp.rst | 133 +++++++++ arch/riscv/mm/fault.c | 8 + include/linux/panic.h | 3 + include/linux/printk.h | 7 + include/linux/rv.h | 75 ++++- include/linux/sched.h | 8 +- include/rv/da_monitor.h | 45 +-- include/rv/ltl_monitor.h | 184 +++++++++++++ kernel/fork.c | 5 +- kernel/panic.c | 16 +- kernel/printk/internal.h | 1 - kernel/trace/rv/Kconfig | 27 +- kernel/trace/rv/Makefile | 3 + kernel/trace/rv/monitors/pagefault/Kconfig | 20 ++ .../trace/rv/monitors/pagefault/pagefault.c | 88 ++++++ .../trace/rv/monitors/pagefault/pagefault.h | 64 +++++ .../rv/monitors/pagefault/pagefault_trace.h | 14 + kernel/trace/rv/monitors/rtapp/Kconfig | 11 + kernel/trace/rv/monitors/rtapp/rtapp.c | 33 +++ kernel/trace/rv/monitors/rtapp/rtapp.h | 3 + kernel/trace/rv/monitors/sleep/Kconfig | 22 ++ kernel/trace/rv/monitors/sleep/sleep.c | 237 ++++++++++++++++ kernel/trace/rv/monitors/sleep/sleep.h | 257 ++++++++++++++++++ kernel/trace/rv/monitors/sleep/sleep_trace.h | 14 + kernel/trace/rv/reactor_panic.c | 8 +- kernel/trace/rv/reactor_printk.c | 8 +- kernel/trace/rv/rv.c | 10 +- kernel/trace/rv/rv_reactors.c | 2 +- kernel/trace/rv/rv_trace.h | 52 +++- tools/verification/models/rtapp/pagefault.ltl | 1 + tools/verification/models/rtapp/sleep.ltl | 22 ++ 32 files changed, 1307 insertions(+), 75 deletions(-) create mode 100644 Documentation/trace/rv/monitor_rtapp.rst create mode 100644 include/rv/ltl_monitor.h create mode 100644 kernel/trace/rv/monitors/pagefault/Kconfig create mode 100644 kernel/trace/rv/monitors/pagefault/pagefault.c create mode 100644 kernel/trace/rv/monitors/pagefault/pagefault.h create mode 100644 kernel/trace/rv/monitors/pagefault/pagefault_trace.h create mode 100644 kernel/trace/rv/monitors/rtapp/Kconfig create mode 100644 kernel/trace/rv/monitors/rtapp/rtapp.c create mode 100644 kernel/trace/rv/monitors/rtapp/rtapp.h create mode 100644 kernel/trace/rv/monitors/sleep/Kconfig create mode 100644 kernel/trace/rv/monitors/sleep/sleep.c create mode 100644 kernel/trace/rv/monitors/sleep/sleep.h create mode 100644 kernel/trace/rv/monitors/sleep/sleep_trace.h create mode 100644 tools/verification/models/rtapp/pagefault.ltl create mode 100644 tools/verification/models/rtapp/sleep.ltl
Signature · 1 line
2.39.5
PATCH v13 1/12 Add #undef TRACE_INCLUDE_FILE
Without "#undef TRACE_INCLUDE_FILE", there could be a build error due to TRACE_INCLUDE_FILE being redefined. Therefore add it.
Also fix a typo while at it.
--- kernel/trace/rv/rv_trace.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
kernel/trace/rv/rv_trace.h +2 -1
@@ -129,8 +129,9 @@ DECLARE_EVENT_CLASS(error_da_monitor_id,#endif /* CONFIG_DA_MON_EVENTS_ID */ #endif /* _TRACE_RV_H */ -/* This part ust be outside protection */ +/* This part must be outside protection */ #undef TRACE_INCLUDE_PATH#define TRACE_INCLUDE_PATH . +#undef TRACE_INCLUDE_FILE#define TRACE_INCLUDE_FILE rv_trace#include <trace/define_trace.h>
Signature · 1 line
2.39.5
PATCH v13 2/12 printk: Make vprintk_deferred() public
In reply to Nam Caovprintk_deferred() is useful for implementing runtime verification reactors. Make it public.
---
Cc: John Ogness <[email protected]>
Cc: Sergey Senozhatsky <[email protected]>
---
v13: keep consistent format and put __printf(1, 0) in another line
--- include/linux/printk.h | 7 +++++++ kernel/printk/internal.h | 1 - 2 files changed, 7 insertions(+), 1 deletion(-)
include/linux/printk.h +7 -0
@@ -154,6 +154,8 @@ int vprintk_emit(int facility, int level, asmlinkage __printf(1, 0) int vprintk(const char *fmt, va_list args); +__printf(1, 0) +int vprintk_deferred(const char *fmt, va_list args); asmlinkage __printf(1, 2) __cold int _printk(const char *fmt, ...);
@@ -214,6 +216,11 @@ int vprintk(const char *s, va_list args) {return 0 ; } +static inline __printf(1, 0) +int vprintk_deferred(const char *fmt, va_list args) +{ +return 0 ; +} static inline __printf(1, 2) __cold int _printk(const char *s, ...) {
kernel/printk/internal.h +0 -1
@@ -72,7 +72,6 @@ int vprintk_store(int facility, int level, const char *fmt, va_list args); __printf(1, 0) int vprintk_default(const char *fmt, va_list args); -__printf(1, 0) int vprintk_deferred(const char *fmt, va_list args);void __printk_safe_enter (void );void __printk_safe_exit (void );
Signature · 1 line
2.39.5
PATCH v13 3/12 panic: Add vpanic()
In reply to Nam Caovpanic() is useful for implementing runtime verification reactors. Add it.
---
Cc: John Ogness <[email protected]>
Cc: Sergey Senozhatsky <[email protected]>
--- include/linux/panic.h | 3 +++ kernel/panic.c | 16 ++++++++++++---- 2 files changed, 15 insertions(+), 4 deletions(-)
include/linux/panic.h +3 -0
@@ -3,6 +3,7 @@#define _LINUX_PANIC_H#include <linux/compiler_attributes.h> +#include <linux/stdarg.h> #include <linux/types.h> struct pt_regs ;
@@ -10,6 +11,8 @@ struct pt_regs;extern long (*panic_blink)(int state); __printf(1, 2)void panic (const char *fmt, ...)__noreturn __cold ; +__printf(1, 0) +void vpanic (const char *fmt,va_list args)__noreturn __cold ;void nmi_panic (struct pt_regs *regs,const char *msg);void check_panic_on_warn (const char *origin);extern void oops_enter (void );
kernel/panic.c +12 -4
@@ -309,13 +309,13 @@ static void panic_other_cpus_shutdown(bool crash_kexec)/** * panic - halt the system * @fmt: The text string to print +* @args: Arguments for the format string * * Display a message, then perform cleanups. This function never returns. */ -void panic (const char *fmt, ...) +void vpanic (const char *fmt,va_list args) {static char buf[1024 ]; -va_list args;long i, i_next =0 , len;int state =0 ;int old_cpu, this_cpu;
@@ -366,9 +366,7 @@ void panic(const char *fmt, ...)console_verbose ();bust_spinlocks (1 ); -va_start (args, fmt); len =vscnprintf (buf,sizeof (buf), fmt, args); -va_end (args);if (len && buf[len -1 ] =='\n' ) buf[len -1 ] ='\0' ;
@@ -505,7 +503,17 @@ void panic(const char *fmt, ...)mdelay (PANIC_TIMER_STEP ); } } +EXPORT_SYMBOL (vpanic); +/* Identical to vpanic(), except it takes variadic arguments instead of va_list */ +void panic (const char *fmt, ...) +{ +va_list args; + +va_start (args, fmt); +vpanic (fmt, args); +va_end (args); +}EXPORT_SYMBOL (panic);#define TAINT_FLAG (taint, _c_true, _c_false, _module) \
Signature · 1 line
2.39.5
PATCH v13 4/12 Let the reactors take care of buffers
In reply to Nam CaoEach RV monitor has one static buffer to send to the reactors. If multiple errors are detected simultaneously, the one buffer could be overwritten.
Instead, leave it to the reactors to handle buffering.
---
Cc: Petr Mladek <[email protected]>
Cc: John Ogness <[email protected]>
Cc: Sergey Senozhatsky <[email protected]>
--- include/linux/rv.h | 9 +++++-- include/rv/da_monitor.h | 45 +++++++------------------------- kernel/trace/rv/reactor_panic.c | 8 ++++-- kernel/trace/rv/reactor_printk.c | 8 ++++-- kernel/trace/rv/rv_reactors.c | 2 +- 5 files changed, 30 insertions(+), 42 deletions(-)
include/linux/rv.h +7 -2
@@ -38,7 +38,7 @@ union rv_task_monitor {struct rv_reactor {const char *name;const char *description; -void (*react)(char *msg); + __printf(1, 2) void (*react)(const char *msg, ...); };#endif
@@ -50,7 +50,7 @@ struct rv_monitor {void (*disable)(void );void (*reset)(void );#ifdef CONFIG_RV_REACTORS -void (*react)(char *msg); + __printf(1, 2) void (*react)(const char *msg, ...);#endif };
@@ -64,6 +64,11 @@ void rv_put_task_monitor_slot(int slot);bool rv_reacting_on (void );int rv_unregister_reactor (struct rv_reactor *reactor);int rv_register_reactor (struct rv_reactor *reactor); +#else +static inline bool rv_reacting_on (void ) +{ +return false; +}#endif /* CONFIG_RV_REACTORS */ #endif /* CONFIG_RV */
include/rv/da_monitor.h +10 -35
71 lines not on this page · raw message
kernel/trace/rv/reactor_panic.c +6 -2
@@ -13,9 +13,13 @@#include <linux/init.h> #include <linux/rv.h> -static void rv_panic_reaction (char *msg) +__printf(1, 2) static void rv_panic_reaction(const char *msg, ...) { -panic (msg); +va_list args; + +va_start (args, msg); +vpanic (msg, args); +va_end (args); }static struct rv_reactor rv_panic = {
kernel/trace/rv/reactor_printk.c +6 -2
@@ -12,9 +12,13 @@#include <linux/init.h> #include <linux/rv.h> -static void rv_printk_reaction (char *msg) +__printf(1, 2) static void rv_printk_reaction(const char *msg, ...) { -printk_deferred (msg); +va_list args; + +va_start (args, msg); +vprintk_deferred (msg, args); +va_end (args); }static struct rv_reactor rv_printk = {
kernel/trace/rv/rv_reactors.c +1 -1
@@ -490,7 +490,7 @@ void reactor_cleanup_monitor(struct rv_monitor_def *mdef)/* * Nop reactor register */ -static void rv_nop_reaction (char *msg) +__printf(1, 2) static void rv_nop_reaction(const char *msg, ...) { }
Signature · 1 line
2.39.5
kernel test robot
build botRe: [PATCH v13 04/12] rv: Let the reactors take care of buffers
kernel test robot noticed the following build warnings:
Hi Nam,
kernel test robot noticed the following build warnings:
[auto build test WARNING on trace/for-next]
[also build test WARNING on tip/sched/core akpm-mm/mm-everything linus/master v6.16-rc5 next-20250710] [cannot apply to kees/for-next/execve]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Nam-Cao/rv-Add-undef-TRACE_INCLUDE_FILE/20250710-032354 base: https://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace for-next patch link: https://lore.kernel.org/r/620a25d4b2e0f73701c4f255063b60c8d8feb7e7.1752088709.git.namcao%40linutronix.de patch subject: [PATCH v13 04/12] rv: Let the reactors take care of buffers config: x86_64-rhel-9.4-func (https://download.01.org/0day-ci/archive/20250711/[email protected]/config) compiler: gcc-12 (Debian 12.2.0-14+deb12u1) 12.2.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250711/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
>> kernel/trace/rv/reactor_panic.o: warning: objtool: rv_panic_reaction(): unexpected end of section .text
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
PATCH v13 5/12 rename CONFIG_DA_MON_EVENTS to CONFIG_RV_MON_EVENTS
In reply to Nam CaoCONFIG_DA_MON_EVENTS is not specific to deterministic automaton. It could be used for other monitor types. Therefore rename it to CONFIG_RV_MON_EVENTS.
This prepares for the introduction of linear temporal logic monitor.
--- kernel/trace/rv/Kconfig | 6 +++--- kernel/trace/rv/rv.c | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-)
kernel/trace/rv/Kconfig +3 -3
@@ -1,14 +1,14 @@ # SPDX-License-Identifier: GPL-2.0-only # -config DA_MON_EVENTS +config RV_MON_EVENTS bool config DA_MON_EVENTS_IMPLICIT - select DA_MON_EVENTS + select RV_MON_EVENTS bool config DA_MON_EVENTS_ID - select DA_MON_EVENTS + select RV_MON_EVENTS bool menuconfig RV
kernel/trace/rv/rv.c +1 -1
@@ -143,7 +143,7 @@#include <linux/init.h> #include <linux/slab.h> -#ifdef CONFIG_DA_MON_EVENTS +#ifdef CONFIG_RV_MON_EVENTS #define CREATE_TRACE_POINTS #include <rv_trace.h> #endif
Signature · 1 line
2.39.5
PATCH v13 6/12 Add support for LTL monitors
In reply to Nam CaoWhile attempting to implement DA monitors for some complex specifications, deterministic automaton is found to be inappropriate as the specification language. The automaton is complicated, hard to understand, and error-prone.
For these cases, linear temporal logic is more suitable as the specification language.
Add support for linear temporal logic runtime verification monitor.
--- include/linux/rv.h | 63 ++++++++++++- include/rv/ltl_monitor.h | 184 +++++++++++++++++++++++++++++++++++++ kernel/fork.c | 5 +- kernel/trace/rv/Kconfig | 7 ++ kernel/trace/rv/rv_trace.h | 47 ++++++++++ 5 files changed, 298 insertions(+), 8 deletions(-) create mode 100644 include/rv/ltl_monitor.h
include/linux/rv.h +59 -4
@@ -10,6 +10,10 @@#define MAX_DA_NAME_LEN 32#ifdef CONFIG_RV +#include <linux/bitops.h> +#include <linux/types.h> +#include <linux/array_size.h> +/* * Deterministic automaton per-object variables. */
@@ -18,6 +22,59 @@ struct da_monitor {unsigned int curr_state; }; +#ifdef CONFIG_RV_LTL_MONITOR + +/* +* In the future, if the number of atomic propositions or the size of Buchi +* automaton is larger, we can switch to dynamic allocation. For now, the code +* is simpler this way. +*/ +#define RV_MAX_LTL_ATOM 32 +#define RV_MAX_BA_STATES 32 + +/** +* struct ltl_monitor - A linear temporal logic runtime verification monitor +* @states: States in the Buchi automaton. As Buchi automaton is a +* non-deterministic state machine, the monitor can be in multiple +* states simultaneously. This is a bitmask of all possible states. +* If this is zero, that means either: +* - The monitor has not started yet (e.g. because not all +* atomic propositions are known). +* - There is no possible state to be in. In other words, a +* violation of the LTL property is detected. +* @atoms: The values of atomic propositions. +* @unknown_atoms: Atomic propositions which are still unknown. +*/ +struct ltl_monitor { + DECLARE_BITMAP(states, RV_MAX_BA_STATES); + DECLARE_BITMAP(atoms, RV_MAX_LTL_ATOM); + DECLARE_BITMAP(unknown_atoms, RV_MAX_LTL_ATOM); +}; + +static inline bool rv_ltl_valid_state (struct ltl_monitor *mon) +{ +for (int i =0 ; i <ARRAY_SIZE (mon->states); ++i) { +if (mon->states[i]) +return true; + } +return false; +} + +static inline bool rv_ltl_all_atoms_known (struct ltl_monitor *mon) +{ +for (int i =0 ; i <ARRAY_SIZE (mon->unknown_atoms); ++i) { +if (mon->unknown_atoms[i]) +return false; + } +return true; +} + +#else + +struct ltl_monitor {}; + +#endif /* CONFIG_RV_LTL_MONITOR */ +/* * Per-task RV monitors count. Nowadays fixed in RV_PER_TASK_MONITORS. * If we find justification for more monitors, we can think about
@@ -27,11 +84,9 @@ struct da_monitor {#define RV_PER_TASK_MONITORS 1#define RV_PER_TASK_MONITOR_INIT (RV_PER_TASK_MONITORS) -/* -* Futher monitor types are expected, so make this a union. -*/ union rv_task_monitor { -struct da_monitor da_mon; +struct da_monitor da_mon; +struct ltl_monitor ltl_mon; };#ifdef CONFIG_RV_REACTORS
include/rv/ltl_monitor.h new file +184 -0
184 lines not on this page · raw message
kernel/fork.c +1 -4
@@ -1886,10 +1886,7 @@ static void copy_oom_score_adj(u64 clone_flags, struct task_struct *tsk)#ifdef CONFIG_RV static void rv_task_fork (struct task_struct *p) { -int i; - -for (i =0 ; i <RV_PER_TASK_MONITORS ; i++) - p->rv[i].da_mon.monitoring = false; +memset (&p->rv,0 ,sizeof (p->rv)); }#else #define rv_task_fork (p) do {} while (0)
kernel/trace/rv/Kconfig +7 -0
@@ -11,6 +11,13 @@ config DA_MON_EVENTS_ID select RV_MON_EVENTS bool +config LTL_MON_EVENTS_ID + select RV_MON_EVENTS + bool + +config RV_LTL_MONITOR + bool + menuconfig RV bool "Runtime Verification" depends on TRACING
kernel/trace/rv/rv_trace.h +47 -0
53 lines not on this page · raw message
Signature · 1 line
2.39.5
PATCH v13 7/12 Add rtapp container monitor
In reply to Nam CaoAdd the container "rtapp" which is the monitor collection for detecting problems with real-time applications. The monitors will be added in the follow-up commits.
--- kernel/trace/rv/Kconfig | 1 + kernel/trace/rv/Makefile | 1 + kernel/trace/rv/monitors/rtapp/Kconfig | 10 ++++++++ kernel/trace/rv/monitors/rtapp/rtapp.c | 33 ++++++++++++++++++++++++++ kernel/trace/rv/monitors/rtapp/rtapp.h | 3 +++ 5 files changed, 48 insertions(+) create mode 100644 kernel/trace/rv/monitors/rtapp/Kconfig create mode 100644 kernel/trace/rv/monitors/rtapp/rtapp.c create mode 100644 kernel/trace/rv/monitors/rtapp/rtapp.h
kernel/trace/rv/Kconfig +1 -0
@@ -41,6 +41,7 @@ source "kernel/trace/rv/monitors/snroc/Kconfig" source "kernel/trace/rv/monitors/scpd/Kconfig" source "kernel/trace/rv/monitors/snep/Kconfig" source "kernel/trace/rv/monitors/sncid/Kconfig" +source "kernel/trace/rv/monitors/rtapp/Kconfig" # Add new monitors here config RV_REACTORS
kernel/trace/rv/Makefile +1 -0
@@ -12,6 +12,7 @@ obj-$(CONFIG_RV_MON_SNROC) += monitors/snroc/snroc.o obj-$(CONFIG_RV_MON_SCPD) += monitors/scpd/scpd.o obj-$(CONFIG_RV_MON_SNEP) += monitors/snep/snep.o obj-$(CONFIG_RV_MON_SNCID) += monitors/sncid/sncid.o +obj-$(CONFIG_RV_MON_RTAPP) += monitors/rtapp/rtapp.o # Add new monitors here obj-$(CONFIG_RV_REACTORS) += rv_reactors.o obj-$(CONFIG_RV_REACT_PRINTK) += reactor_printk.o
kernel/trace/rv/monitors/rtapp/Kconfig new file +10 -0
@@ -0,0 +1,10 @@ +config RV_MON_RTAPP + depends on RV + bool "rtapp monitor" + help + Collection of monitors to check for common problems with real-time + application that may cause unexpected latency. + + If you are developing a real-time system and not entirely sure whether + the applications are designed correctly for real-time, you want to say + Y here.
kernel/trace/rv/monitors/rtapp/rtapp.c new file +33 -0
@@ -0,0 +1,33 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <linux/kernel.h> +#include <linux/module.h> +#include <linux/init.h> +#include <linux/rv.h> + +#define MODULE_NAME "rtapp" + +#include "rtapp.h" + +struct rv_monitor rv_rtapp; + +struct rv_monitor rv_rtapp = { + .name ="rtapp" , + .description ="Collection of monitors for detecting problems with real-time applications" , +}; + +static int __init register_rtapp (void ) +{ +return rv_register_monitor (&rv_rtapp,NULL ); +} + +static void __exit unregister_rtapp(void) +{ +rv_unregister_monitor (&rv_rtapp); +} + +module_init (register_rtapp); +module_exit (unregister_rtapp); + +MODULE_LICENSE ("GPL" ); +MODULE_AUTHOR ("Nam Cao <[email protected]>" ); +MODULE_DESCRIPTION ("Collection of monitors for detecting problems with real-time applications" );
kernel/trace/rv/monitors/rtapp/rtapp.h new file +3 -0
@@ -0,0 +1,3 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +extern struct rv_monitor rv_rtapp;
Signature · 1 line
2.39.5
PATCH v13 8/12 riscv: mm: Add page fault trace points
In reply to Nam CaoAdd page fault trace points, which are useful to implement RV monitor that watches page faults.
---
Cc: Paul Walmsley <[email protected]>
Cc: Palmer Dabbelt <[email protected]>
Cc: Albert Ou <[email protected]>
Cc: [email protected]
--- arch/riscv/mm/fault.c | 8 ++++++++ 1 file changed, 8 insertions(+)
arch/riscv/mm/fault.c +8 -0
@@ -20,6 +20,9 @@#include <asm/ptrace.h> #include <asm/tlbflush.h> +#define CREATE_TRACE_POINTS +#include <trace/events/exceptions.h> +#include "../kernel/head.h" static void show_pte (unsigned long addr)
@@ -291,6 +294,11 @@ void handle_page_fault(struct pt_regs *regs)if (kprobe_page_fault (regs, cause))return ; +if (user_mode (regs)) +trace_page_fault_user (addr, regs, cause); +else +trace_page_fault_kernel (addr, regs, cause); +/* * Fault-in kernel-space virtual memory on-demand. * The 'reference' page table is init_mm.pgd.
Signature · 1 line
2.39.5
PATCH v13 9/12 Add rtapp_pagefault monitor
In reply to Nam CaoUserspace real-time applications may have design flaws that they raise page faults in real-time threads, and thus have unexpected latencies.
Add an linear temporal logic monitor to detect this scenario.
--- kernel/trace/rv/Kconfig | 1 + kernel/trace/rv/Makefile | 1 + kernel/trace/rv/monitors/pagefault/Kconfig | 20 +++++ .../trace/rv/monitors/pagefault/pagefault.c | 88 +++++++++++++++++++ .../trace/rv/monitors/pagefault/pagefault.h | 64 ++++++++++++++ .../rv/monitors/pagefault/pagefault_trace.h | 14 +++ kernel/trace/rv/rv_trace.h | 1 + tools/verification/models/rtapp/pagefault.ltl | 1 + 8 files changed, 190 insertions(+) create mode 100644 kernel/trace/rv/monitors/pagefault/Kconfig create mode 100644 kernel/trace/rv/monitors/pagefault/pagefault.c create mode 100644 kernel/trace/rv/monitors/pagefault/pagefault.h create mode 100644 kernel/trace/rv/monitors/pagefault/pagefault_trace.h create mode 100644 tools/verification/models/rtapp/pagefault.ltl
kernel/trace/rv/Kconfig +1 -0
@@ -42,6 +42,7 @@ source "kernel/trace/rv/monitors/scpd/Kconfig" source "kernel/trace/rv/monitors/snep/Kconfig" source "kernel/trace/rv/monitors/sncid/Kconfig" source "kernel/trace/rv/monitors/rtapp/Kconfig" +source "kernel/trace/rv/monitors/pagefault/Kconfig" # Add new monitors here config RV_REACTORS
kernel/trace/rv/Makefile +1 -0
@@ -13,6 +13,7 @@ obj-$(CONFIG_RV_MON_SCPD) += monitors/scpd/scpd.o obj-$(CONFIG_RV_MON_SNEP) += monitors/snep/snep.o obj-$(CONFIG_RV_MON_SNCID) += monitors/sncid/sncid.o obj-$(CONFIG_RV_MON_RTAPP) += monitors/rtapp/rtapp.o +obj-$(CONFIG_RV_MON_PAGEFAULT) += monitors/pagefault/pagefault.o # Add new monitors here obj-$(CONFIG_RV_REACTORS) += rv_reactors.o obj-$(CONFIG_RV_REACT_PRINTK) += reactor_printk.o
kernel/trace/rv/monitors/pagefault/Kconfig new file +20 -0
@@ -0,0 +1,20 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +config RV_MON_PAGEFAULT + depends on RV + select RV_LTL_MONITOR + depends on RV_MON_RTAPP + depends on X86 || RISCV + default y + select LTL_MON_EVENTS_ID + bool "pagefault monitor" + help + Monitor that real-time tasks do not raise page faults, causing + undesirable latency. + + If you are developing a real-time system and not entirely sure whether + the applications are designed correctly for real-time, you want to say + Y here. + + This monitor does not affect execution speed while it is not running, + therefore it is safe to enable this in production kernel.
kernel/trace/rv/monitors/pagefault/pagefault.c new file +88 -0
88 lines not on this page · raw message
kernel/trace/rv/monitors/pagefault/pagefault.h new file +64 -0
64 lines not on this page · raw message
kernel/trace/rv/monitors/pagefault/pagefault_trace.h new file +14 -0
@@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* +* Snippet to be included in rv_trace.h +*/ + +#ifdef CONFIG_RV_MON_PAGEFAULT +DEFINE_EVENT (event_ltl_monitor_id, event_pagefault, + TP_PROTO(struct task_struct *task, char *states, char *atoms, char *next), +TP_ARGS (task, states, atoms, next)); +DEFINE_EVENT (error_ltl_monitor_id, error_pagefault, + TP_PROTO(struct task_struct *task), +TP_ARGS (task)); +#endif /* CONFIG_RV_MON_PAGEFAULT */
kernel/trace/rv/rv_trace.h +1 -0
@@ -172,6 +172,7 @@ DECLARE_EVENT_CLASS(error_ltl_monitor_id,TP_printk ("%s[%d]: violation detected" ,__get_str (comm), __entry->pid) ); +#include <monitors/pagefault/pagefault_trace.h> // Add new monitors based on CONFIG_LTL_MON_EVENTS_ID here #endif /* CONFIG_LTL_MON_EVENTS_ID */ #endif /* _TRACE_RV_H */
tools/verification/models/rtapp/pagefault.ltl new file +1 -0
@@ -0,0 +1 @@ +RULE = always (RT imply not PAGEFAULT)
Signature · 1 line
2.39.5
PATCH v13 10/12 Add rtapp_sleep monitor
In reply to Nam CaoAdd a monitor for checking that real-time tasks do not go to sleep in a manner that may cause undesirable latency.
Also change
RV depends on TRACING
to
RV select TRACING
to avoid the following recursive dependency:
error: recursive dependency detected!
symbol TRACING is selected by PREEMPTIRQ_TRACEPOINTS symbol PREEMPTIRQ_TRACEPOINTS depends on TRACE_IRQFLAGS symbol TRACE_IRQFLAGS is selected by RV_MON_SLEEP symbol RV_MON_SLEEP depends on RV
symbol RV depends on TRACING
--- kernel/trace/rv/Kconfig | 3 +- kernel/trace/rv/Makefile | 1 + kernel/trace/rv/monitors/sleep/Kconfig | 22 ++ kernel/trace/rv/monitors/sleep/sleep.c | 237 +++++++++++++++++ kernel/trace/rv/monitors/sleep/sleep.h | 257 +++++++++++++++++++ kernel/trace/rv/monitors/sleep/sleep_trace.h | 14 + kernel/trace/rv/rv_trace.h | 1 + tools/verification/models/rtapp/sleep.ltl | 22 ++ 8 files changed, 556 insertions(+), 1 deletion(-) create mode 100644 kernel/trace/rv/monitors/sleep/Kconfig create mode 100644 kernel/trace/rv/monitors/sleep/sleep.c create mode 100644 kernel/trace/rv/monitors/sleep/sleep.h create mode 100644 kernel/trace/rv/monitors/sleep/sleep_trace.h create mode 100644 tools/verification/models/rtapp/sleep.ltl
kernel/trace/rv/Kconfig +2 -1
@@ -20,7 +20,7 @@ config RV_LTL_MONITOR menuconfig RV bool "Runtime Verification" - depends on TRACING + select TRACING help Enable the kernel runtime verification infrastructure. RV is a lightweight (yet rigorous) method that complements classical
@@ -43,6 +43,7 @@ source "kernel/trace/rv/monitors/snep/Kconfig" source "kernel/trace/rv/monitors/sncid/Kconfig" source "kernel/trace/rv/monitors/rtapp/Kconfig" source "kernel/trace/rv/monitors/pagefault/Kconfig" +source "kernel/trace/rv/monitors/sleep/Kconfig" # Add new monitors here config RV_REACTORS
kernel/trace/rv/Makefile +1 -0
@@ -14,6 +14,7 @@ obj-$(CONFIG_RV_MON_SNEP) += monitors/snep/snep.o obj-$(CONFIG_RV_MON_SNCID) += monitors/sncid/sncid.o obj-$(CONFIG_RV_MON_RTAPP) += monitors/rtapp/rtapp.o obj-$(CONFIG_RV_MON_PAGEFAULT) += monitors/pagefault/pagefault.o +obj-$(CONFIG_RV_MON_SLEEP) += monitors/sleep/sleep.o # Add new monitors here obj-$(CONFIG_RV_REACTORS) += rv_reactors.o obj-$(CONFIG_RV_REACT_PRINTK) += reactor_printk.o
kernel/trace/rv/monitors/sleep/Kconfig new file +22 -0
@@ -0,0 +1,22 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +config RV_MON_SLEEP + depends on RV + select RV_LTL_MONITOR + depends on HAVE_SYSCALL_TRACEPOINTS + depends on RV_MON_RTAPP + select TRACE_IRQFLAGS + default y + select LTL_MON_EVENTS_ID + bool "sleep monitor" + help + Monitor that real-time tasks do not sleep in a manner that may + cause undesirable latency. + + If you are developing a real-time system and not entirely sure whether + the applications are designed correctly for real-time, you want to say + Y here. + + Enabling this monitor may have performance impact (due to select + TRACE_IRQFLAGS). Therefore, you probably should say N for + production kernel.
kernel/trace/rv/monitors/sleep/sleep.c new file +237 -0
237 lines not on this page · raw message
kernel/trace/rv/monitors/sleep/sleep.h new file +257 -0
257 lines not on this page · raw message
kernel/trace/rv/monitors/sleep/sleep_trace.h new file +14 -0
@@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* +* Snippet to be included in rv_trace.h +*/ + +#ifdef CONFIG_RV_MON_SLEEP +DEFINE_EVENT (event_ltl_monitor_id, event_sleep, + TP_PROTO(struct task_struct *task, char *states, char *atoms, char *next), +TP_ARGS (task, states, atoms, next)); +DEFINE_EVENT (error_ltl_monitor_id, error_sleep, + TP_PROTO(struct task_struct *task), +TP_ARGS (task)); +#endif /* CONFIG_RV_MON_SLEEP */
kernel/trace/rv/rv_trace.h +1 -0
@@ -173,6 +173,7 @@ DECLARE_EVENT_CLASS(error_ltl_monitor_id,TP_printk ("%s[%d]: violation detected" ,__get_str (comm), __entry->pid) );#include <monitors/pagefault/pagefault_trace.h> +#include <monitors/sleep/sleep_trace.h> // Add new monitors based on CONFIG_LTL_MON_EVENTS_ID here #endif /* CONFIG_LTL_MON_EVENTS_ID */ #endif /* _TRACE_RV_H */
tools/verification/models/rtapp/sleep.ltl new file +22 -0
@@ -0,0 +1,22 @@ +RULE = always ((RT and SLEEP) imply (RT_FRIENDLY_SLEEP or ALLOWLIST)) + +RT_FRIENDLY_SLEEP = (RT_VALID_SLEEP_REASON or KERNEL_THREAD) + and ((not WAKE) until RT_FRIENDLY_WAKE) + +RT_VALID_SLEEP_REASON = FUTEX_WAIT + or RT_FRIENDLY_NANOSLEEP + +RT_FRIENDLY_NANOSLEEP = CLOCK_NANOSLEEP + and NANOSLEEP_TIMER_ABSTIME + and (NANOSLEEP_CLOCK_MONOTONIC or NANOSLEEP_CLOCK_TAI) + +RT_FRIENDLY_WAKE = WOKEN_BY_EQUAL_OR_HIGHER_PRIO + or WOKEN_BY_HARDIRQ + or WOKEN_BY_NMI + or ABORT_SLEEP + or KTHREAD_SHOULD_STOP + +ALLOWLIST = BLOCK_ON_RT_MUTEX + or FUTEX_LOCK_PI + or TASK_IS_RCU + or TASK_IS_MIGRATION
Signature · 1 line
2.39.5
PATCH v13 11/12 Add documentation for rtapp monitor
In reply to Nam CaoAdd documentation describing the rtapp monitor.
--- Documentation/trace/rv/index.rst | 1 + Documentation/trace/rv/monitor_rtapp.rst | 133 +++++++++++++++++++++++ 2 files changed, 134 insertions(+) create mode 100644 Documentation/trace/rv/monitor_rtapp.rst
Documentation/trace/rv/index.rst +1 -0
@@ -13,3 +13,4 @@ Runtime Verification monitor_wip.rst monitor_wwnr.rst monitor_sched.rst + monitor_rtapp.rst
Documentation/trace/rv/monitor_rtapp.rst new file +133 -0
133 lines not on this page · raw message
Signature · 1 line
2.39.5
PATCH v13 12/12 Allow to configure the number of per-task monitor
In reply to Nam CaoNow that there are 2 monitors for real-time applications, users may want to enable both of them simultaneously. Make the number of per-task monitor configurable. Default it to 2 for now.
--- include/linux/rv.h | 9 +-------- include/linux/sched.h | 8 +++----- kernel/trace/rv/Kconfig | 9 +++++++++ kernel/trace/rv/monitors/rtapp/Kconfig | 1 + kernel/trace/rv/rv.c | 8 ++++---- 5 files changed, 18 insertions(+), 17 deletions(-)
include/linux/rv.h +1 -8
@@ -75,14 +75,7 @@ struct ltl_monitor {};#endif /* CONFIG_RV_LTL_MONITOR */ -/* -* Per-task RV monitors count. Nowadays fixed in RV_PER_TASK_MONITORS. -* If we find justification for more monitors, we can think about -* adding more or developing a dynamic method. So far, none of -* these are justified. -*/ -#define RV_PER_TASK_MONITORS 1 -#define RV_PER_TASK_MONITOR_INIT (RV_PER_TASK_MONITORS) +#define RV_PER_TASK_MONITOR_INIT (CONFIG_RV_PER_TASK_MONITORS)union rv_task_monitor {struct da_monitor da_mon;
include/linux/sched.h +3 -5
@@ -1642,12 +1642,10 @@ struct task_struct {#ifdef CONFIG_RV /* -* Per-task RV monitor. Nowadays fixed in RV_PER_TASK_MONITORS. -* If we find justification for more monitors, we can think -* about adding more or developing a dynamic method. So far, -* none of these are justified. +* Per-task RV monitor, fixed in CONFIG_RV_PER_TASK_MONITORS. +* If memory becomes a concern, we can think about a dynamic method. */ -union rv_task_monitor rv[RV_PER_TASK_MONITORS ]; +union rv_task_monitor rv[CONFIG_RV_PER_TASK_MONITORS ];#endif #ifdef CONFIG_USER_EVENTS
kernel/trace/rv/Kconfig +9 -0
@@ -32,6 +32,15 @@ menuconfig RV For further information, see: Documentation/trace/rv/runtime-verification.rst +config RV_PER_TASK_MONITORS + int "Maximum number of per-task monitor" + depends on RV + range 1 8 + default 2 + help + This option configures the maximum number of per-task RV monitors that can run + simultaneously. + source "kernel/trace/rv/monitors/wip/Kconfig" source "kernel/trace/rv/monitors/wwnr/Kconfig" source "kernel/trace/rv/monitors/sched/Kconfig"
kernel/trace/rv/monitors/rtapp/Kconfig +1 -0
@@ -1,5 +1,6 @@ config RV_MON_RTAPP depends on RV + depends on RV_PER_TASK_MONITORS >= 2 bool "rtapp monitor" help Collection of monitors to check for common problems with real-time
kernel/trace/rv/rv.c +4 -4
@@ -165,7 +165,7 @@ struct dentry *get_monitors_root(void)LIST_HEAD (rv_monitors_list);static int task_monitor_count; -static bool task_monitor_slots[RV_PER_TASK_MONITORS ]; +static bool task_monitor_slots[CONFIG_RV_PER_TASK_MONITORS ];int rv_get_task_monitor_slot (void ) {
@@ -173,12 +173,12 @@ int rv_get_task_monitor_slot(void)lockdep_assert_held (&rv_interface_lock); -if (task_monitor_count ==RV_PER_TASK_MONITORS ) +if (task_monitor_count ==CONFIG_RV_PER_TASK_MONITORS )return -EBUSY ; task_monitor_count++; -for (i =0 ; i <RV_PER_TASK_MONITORS ; i++) { +for (i =0 ; i <CONFIG_RV_PER_TASK_MONITORS ; i++) {if (task_monitor_slots[i] == false) { task_monitor_slots[i] = true;return i;
@@ -194,7 +194,7 @@ void rv_put_task_monitor_slot(int slot) {lockdep_assert_held (&rv_interface_lock); -if (slot <0 || slot >=RV_PER_TASK_MONITORS ) { +if (slot <0 || slot >=CONFIG_RV_PER_TASK_MONITORS ) {WARN_ONCE (1 ,"RV releasing an invalid slot!: %d\n" , slot);return ; }
Signature · 1 line
2.39.5
This patch was applied to riscv/linux.git (fixes) by Steven Rostedt (Google) <[email protected]>:
Hello:
This patch was applied to riscv/linux.git (fixes)
by Steven Rostedt (Google) <[email protected]>:
Quoted 10 lines · On Wed, 9 Jul 2025 21:21:11 +0200 you wrote:
Real-time applications may have design flaws causing them to have unexpected latency. For example, the applications may raise page faults, or may be blocked trying to take a mutex without priority inheritance.
However, while attempting to implement DA monitors for these real-time rules, deterministic automaton is found to be inappropriate as the specification language. The automaton is complicated, hard to understand, and error-prone.
[...]
Here is the summary with links:
- [v13,08/12] riscv: mm: Add page fault trace points
https://git.kernel.org/riscv/c/a37c71ca412d
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html