Skip to contentSkip to conversation

The Runtime Verification (RV) interface

PATCH Superseded Daniel Bristot de Oliveira 10 participants 82 messages Jun 16, 2022 – Jul 13, 2022 1 message not in the archive

Subsystems (7)

v4 of 9 Superseded Latest is v9 →

lore
Share & tools
lore
#0 lore raw

v4 cover The Runtime Verification (RV) interface

Over the last years, I've been exploring the possibility of verifying the Linux kernel behavior using Runtime Verification.

Runtime Verification (RV) is a lightweight (yet rigorous) method that complements classical exhaustive verification techniques (such as model checking and theorem proving) with a more practical approach for complex systems.

Instead of relying on a fine-grained model of a system (e.g., a re-implementation a instruction level), RV works by analyzing the trace of the system's actual execution, comparing it against a formal specification of the system behavior.

The usage of deterministic automaton for RV is a well-established approach. In the specific case of the Linux kernel, you can check how to model complex behavior of the Linux kernel with this paper:

DE OLIVEIRA, Daniel Bristot; CUCINOTTA, Tommaso; DE OLIVEIRA, Romulo Silva. *Efficient formal verification for the Linux kernel.* In: International Conference on Software Engineering and Formal Methods. Springer, Cham, 2019. p. 315-332.

And how efficient is this approach here:

DE OLIVEIRA, Daniel B.; DE OLIVEIRA, Romulo S.; CUCINOTTA, Tommaso. *A thread synchronization model for the PREEMPT_RT Linux kernel.* Journal of Systems Architecture, 2020, 107: 101729.

tlrd: it is possible to model complex behaviors in a modular way, with an acceptable overhead (even for production systems). See this presentation at 2019's ELCE: https://www.youtube.com/watch?v=BfTuEHafNgg

Here I am proposing a more practical approach for the usage of deterministic automata for runtime verification, and it includes:

- An interface for controlling the verification;
- A tool and set of headers that enables the automatic code generation of the RV monitor (Monitor Synthesis);
- Sample monitors to evaluate the interface;
- A sample monitor developed in the context of the Elisa Project demonstrating how to use RV in the context of safety-critical systems.

Given that RV is a tracing consumer, the code is being placed inside the tracing subsystem (Steven and I have been talking about it for a while).

Changes from v3:
- Rebased on 5.19
(rostedt's request were made on 1x1 meetings)
- Moved monitors to monitors/$name/ (Rostedt)
- Consolidate the tracepoints into a single include file in the default directory (trave/events/rv.h) (Rostedt)
- The tracepoints now record the entire string to the buffer.
- Change the enable_monitors to disable monitors with ! (instead of -). (Rostedt)
- Add a suffix to the state/events enums, to avoid conflict in the vmlinux.h used by eBPF.
- The models are now placed in the $name.h (it used to store the tracepoints, but they are now consolidated in a single file)
- dot2c and dot2k updated to the changes
- models re-generated with these new standards.
- user-space tools moved to an directory outside of tools/tracing as other methods of verification/log sources are planned.
Changes from v2:
- Tons of checkpatch and kernel test robot
- Moved files to better places
- Adjusted watchdog tracepoints patch (Guenter Roeck)
- Added pretimeout watchdog events (Peter Enderborg)
- Used task struct to store per-task monitors (Peter Zijlstra)
- Changed the instrumentation to use internal definition of tracepoint and check the callback signature (Steven Rostedt)
- Used printk_deferred() and removed the comment about deadlocks (Shuah Khan/John Ogness)
- Some simplifications:
- Removed the safe watchdog nowayout for now (myself)
- Removed export symbols for now (myself)
Changes from V1:
- rebased to the latest kernel;
- code cleanup;
- the watchdog dev monitor;
- safety app;

Things kept for a second moment (after this patchset):
- Add a reactor tha enables the visualization of the visited states via KCOV (Marco Elver & Dmitry Vyukov)
- Add a CRC method to check from user-space if the values exported by the monitor were not corrupted by any other kernel task (Gabriele Paoloni)
- Export symbols for external modules
- dot2bpf

Daniel Bristot de Oliveira (20):
rv: Add Runtime Verification (RV) interface
rv: Add runtime reactors interface
rv/include: Add helper functions for deterministic automata rv/include: Add deterministic automata monitor definition via C macros rv/include: Add instrumentation helper functions
tools/rv: Add dot2c
tools/rv: Add dot2k
rv/monitor: Add the wip monitor skeleton created by dot2k rv/monitor: wip instrumentation and Makefile/Kconfig entries rv/monitor: Add the wwnr monitor skeleton created by dot2k rv/monitor: wwnr instrumentation and Makefile/Kconfig entries rv/reactor: Add the printk reactor
rv/reactor: Add the panic reactor
Documentation/rv: Add a basic documentation Documentation/rv: Add deterministic automata monitor synthesis
documentation
Documentation/rv: Add deterministic automata instrumentation
documentation
watchdog/dev: Add tracepoints
rv/monitor: Add safe watchdog monitor
rv/safety_app: Add a safety_app sample
Documentation/rv: Add watchdog-monitor documentation

 Documentation/trace/index.rst                 |   1 +
 .../trace/rv/da_monitor_instrumentation.rst   | 223 ++++++
 .../trace/rv/da_monitor_synthesis.rst         | 284 +++++++
 Documentation/trace/rv/index.rst              |   9 +
 .../trace/rv/runtime-verification.rst         | 233 ++++++
 Documentation/trace/rv/watchdog-monitor.rst   | 250 ++++++
 drivers/watchdog/watchdog_dev.c               |  43 +-
 drivers/watchdog/watchdog_pretimeout.c        |   2 +
 include/linux/rv.h                            |  38 +
 include/linux/sched.h                         |  11 +
 include/linux/watchdog.h                      |   7 +-
 include/rv/automata.h                         |  49 ++
 include/rv/da_monitor.h                       | 419 ++++++++++
 include/rv/instrumentation.h                  |  23 +
 include/rv/rv.h                               |  32 +
 include/trace/events/rv.h                     | 153 ++++
 include/trace/events/watchdog.h               | 101 +++
 kernel/fork.c                                 |  14 +
 kernel/trace/Kconfig                          |   2 +
 kernel/trace/Makefile                         |   2 +
 kernel/trace/rv/Kconfig                       |  84 ++
 kernel/trace/rv/Makefile                      |   9 +
 kernel/trace/rv/monitors/safe_wtd/safe_wtd.c  | 300 +++++++
 kernel/trace/rv/monitors/safe_wtd/safe_wtd.h  |  84 ++
 kernel/trace/rv/monitors/wip/wip.c            | 110 +++
 kernel/trace/rv/monitors/wip/wip.h            |  38 +
 kernel/trace/rv/monitors/wwnr/wwnr.c          | 109 +++
 kernel/trace/rv/monitors/wwnr/wwnr.h          |  38 +
 kernel/trace/rv/reactor_panic.c               |  44 +
 kernel/trace/rv/reactor_printk.c              |  43 +
 kernel/trace/rv/rv.c                          | 757 ++++++++++++++++++
 kernel/trace/rv/rv.h                          |  54 ++
 kernel/trace/rv/rv_reactors.c                 | 476 +++++++++++
 kernel/trace/trace.c                          |   4 +
 kernel/trace/trace.h                          |   2 +
 tools/verification/dot2/Makefile              |  26 +
 tools/verification/dot2/automata.py           | 179 +++++
 tools/verification/dot2/dot2c                 |  30 +
 tools/verification/dot2/dot2c.py              | 244 ++++++
 tools/verification/dot2/dot2k                 |  50 ++
 tools/verification/dot2/dot2k.py              | 177 ++++
 .../dot2/dot2k_templates/main_global.c        |  94 +++
 .../dot2/dot2k_templates/main_per_cpu.c       |  94 +++
 .../dot2/dot2k_templates/main_per_task.c      |  94 +++
 tools/verification/safety_app/Makefile        |  51 ++
 tools/verification/safety_app/safety_app.c    | 614 ++++++++++++++
 46 files changed, 5691 insertions(+), 10 deletions(-)
 create mode 100644 Documentation/trace/rv/da_monitor_instrumentation.rst
 create mode 100644 Documentation/trace/rv/da_monitor_synthesis.rst
 create mode 100644 Documentation/trace/rv/index.rst
 create mode 100644 Documentation/trace/rv/runtime-verification.rst
 create mode 100644 Documentation/trace/rv/watchdog-monitor.rst
 create mode 100644 include/linux/rv.h
 create mode 100644 include/rv/automata.h
 create mode 100644 include/rv/da_monitor.h
 create mode 100644 include/rv/instrumentation.h
 create mode 100644 include/rv/rv.h
 create mode 100644 include/trace/events/rv.h
 create mode 100644 include/trace/events/watchdog.h
 create mode 100644 kernel/trace/rv/Kconfig
 create mode 100644 kernel/trace/rv/Makefile
 create mode 100644 kernel/trace/rv/monitors/safe_wtd/safe_wtd.c
 create mode 100644 kernel/trace/rv/monitors/safe_wtd/safe_wtd.h
 create mode 100644 kernel/trace/rv/monitors/wip/wip.c
 create mode 100644 kernel/trace/rv/monitors/wip/wip.h
 create mode 100644 kernel/trace/rv/monitors/wwnr/wwnr.c
 create mode 100644 kernel/trace/rv/monitors/wwnr/wwnr.h
 create mode 100644 kernel/trace/rv/reactor_panic.c
 create mode 100644 kernel/trace/rv/reactor_printk.c
 create mode 100644 kernel/trace/rv/rv.c
 create mode 100644 kernel/trace/rv/rv.h
 create mode 100644 kernel/trace/rv/rv_reactors.c
 create mode 100644 tools/verification/dot2/Makefile
 create mode 100644 tools/verification/dot2/automata.py
 create mode 100644 tools/verification/dot2/dot2c
 create mode 100644 tools/verification/dot2/dot2c.py
 create mode 100644 tools/verification/dot2/dot2k
 create mode 100644 tools/verification/dot2/dot2k.py
 create mode 100644 tools/verification/dot2/dot2k_templates/main_global.c
 create mode 100644 tools/verification/dot2/dot2k_templates/main_per_cpu.c
 create mode 100644 tools/verification/dot2/dot2k_templates/main_per_task.c
 create mode 100644 tools/verification/safety_app/Makefile
 create mode 100644 tools/verification/safety_app/safety_app.c
Signature · 1 line
2.35.1
#1 lore raw

PATCH v4 1/20 Add Runtime Verification (RV) interface

RV is a lightweight (yet rigorous) method that complements classical exhaustive verification techniques (such as model checking and theorem proving) with a more practical approach to complex systems.

RV works by analyzing the trace of the system's actual execution, comparing it against a formal specification of the system behavior. RV can give precise information on the runtime behavior of the monitored system while enabling the reaction for unexpected events, avoiding, for example, the propagation of a failure on safety-critical systems.

The development of this interface roots in the development of the paper:

DE OLIVEIRA, Daniel Bristot; CUCINOTTA, Tommaso; DE OLIVEIRA, Romulo Silva. Efficient formal verification for the Linux kernel. In:
International Conference on Software Engineering and Formal Methods. Springer, Cham, 2019. p. 315-332.

And:

DE OLIVEIRA, Daniel Bristot, et al. Automata-based formal analysis and verification of the real-time Linux kernel. PhD Thesis, 2020.

The RV interface resembles the tracing/ interface on purpose. The current path for the RV interface is /sys/kernel/tracing/rv/.

It presents these files:

"available_monitors"
- List the available monitors, one per line.

For example:
[root@f32 rv]# cat available_monitors
wip
wwnr

"enabled_monitors"
- Lists the enabled monitors, one per line;
- Writing to it enables a given monitor;
- Writing a monitor name with a '-' prefix disables it;
- Truncating the file disables all enabled monitors.

For example:
[root@f32 rv]# cat enabled_monitors
[root@f32 rv]# echo wip > enabled_monitors
[root@f32 rv]# echo wwnr >> enabled_monitors
[root@f32 rv]# cat enabled_monitors
wip
wwnr
[root@f32 rv]# echo !wip >> enabled_monitors
[root@f32 rv]# cat enabled_monitors
wwnr
[root@f32 rv]# echo > enabled_monitors
[root@f32 rv]# cat enabled_monitors
[root@f32 rv]#

Note that more than one monitor can be enabled concurrently.

"monitoring_on"
- It is an on/off general switcher for monitoring. Note that it does not disable enabled monitors, but stop the per-entity monitors of monitoring the events received from the system. It resambles the "tracing_on" switcher.

"monitors/"
Each monitor will have its one directory inside "monitors/". There the monitor specific files will be presented.
The "monitors/" directory resambles the "events" directory on tracefs.

For example:
[root@f32 rv]# cd monitors/wip/
[root@f32 wip]# ls
desc enable
[root@f32 wip]# cat desc
auto-generated wakeup in preemptive monitor.
[root@f32 wip]# cat enable
0

For further information, see the comments in the header of kernel/trace/rv/rv.c from this patch.

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 include/linux/rv.h       |  23 ++
 include/linux/sched.h    |  11 +
 include/rv/rv.h          |  23 ++
 kernel/fork.c            |  14 +
 kernel/trace/Kconfig     |   2 +
 kernel/trace/Makefile    |   2 +
 kernel/trace/rv/Kconfig  |  12 +
 kernel/trace/rv/Makefile |   3 +
 kernel/trace/rv/rv.c     | 738 +++++++++++++++++++++++++++++++++++++++
 kernel/trace/rv/rv.h     |  34 ++
 kernel/trace/trace.c     |   4 +
 kernel/trace/trace.h     |   2 +
 12 files changed, 868 insertions(+)
 create mode 100644 include/linux/rv.h
 create mode 100644 include/rv/rv.h
 create mode 100644 kernel/trace/rv/Kconfig
 create mode 100644 kernel/trace/rv/Makefile
 create mode 100644 kernel/trace/rv/rv.c
 create mode 100644 kernel/trace/rv/rv.h
include/linux/rv.h new file +23 -0
new file mode 100644
index 000000000000..205e65f57637
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Runtime Verification.
+ *
+ * For futher information, see: kernel/trace/rv/rv.c.
+ *
+ * Copyright (C) 2019-2022 Daniel Bristot de Oliveira <[email protected]>
+ */
+#ifndef _LINUX_RV_H
+#define _LINUX_RV_H
+struct rv_monitor {
+	const char		*name;
+	const char		*description;
+	bool			enabled;
+	int			(*start)(void);
+	void			(*stop)(void);
+	void			(*reset)(void);
+};
+
+extern bool monitoring_on;
+int rv_unregister_monitor(struct rv_monitor *monitor);
+int rv_register_monitor(struct rv_monitor *monitor);
+#endif /* _LINUX_RV_H */
include/linux/sched.h +11 -0
index c46f3a63b758..b037f364efdc 100644
@@ -35,6 +35,7 @@
 #include <linux/seqlock.h>
 #include <linux/kcsan.h>
 #include <asm/kmap_size.h>
+#include <rv/rv.h>
 
 /* task_struct member predeclarations (sorted alphabetically): */
 struct audit_context;
@@ -1500,6 +1501,16 @@ struct task_struct {
 	struct callback_head		l1d_flush_kill;
 #endif
 
+#ifdef CONFIG_RV
+	/*
+	 * Per-task RV monitor. Nowadays fixed in RV_PER_TASK_MONITORS.
+	 * If we find justification for more monitors, we can think
+	 * about adding more or developing a dynamic method. So far,
+	 * none of these are justified.
+	 */
+	union rv_task_monitor		rv[RV_PER_TASK_MONITORS];
+#endif
+
 	/*
 	 * New fields for task_struct should be added above here, so that
 	 * they are included in the randomized portion of task_struct.
include/rv/rv.h new file +23 -0
new file mode 100644
index 000000000000..27a108881d35
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef _RV_RV_H
+#define _RV_RV_H
+
+/*
+ * Per-task RV monitors count. Nowadays fixed in RV_PER_TASK_MONITORS.
+ * If we find justification for more monitors, we can think about
+ * adding more or developing a dynamic method. So far, none of
+ * these are justified.
+ */
+#define RV_PER_TASK_MONITORS		1
+#define RV_PER_TASK_MONITOR_INIT	(RV_PER_TASK_MONITORS)
+
+/*
+ * Futher monitor types are expected, so make this a union.
+ */
+union rv_task_monitor {
+};
+
+int get_task_monitor_slot(void);
+void put_task_monitor_slot(int slot);
+#endif /* _RV_RV_H */
kernel/fork.c +14 -0
index 9d44f2d46c69..5e40e58ef83d 100644
@@ -1964,6 +1964,18 @@ static void copy_oom_score_adj(u64 clone_flags, struct task_struct *tsk)
 	mutex_unlock(&oom_adj_mutex);
 }
 
+#ifdef CONFIG_RV
+static void rv_task_fork(struct task_struct *p)
+{
+	int i;
+
+	for (i = 0; i < RV_PER_TASK_MONITORS; i++)
+		;
+}
+#else
+#define rv_task_fork(p) do {} while (0)
+#endif
+
 /*
  * This creates a new process as a copy of the old one,
  * but does not actually start it yet.
@@ -2399,6 +2411,8 @@ static __latent_entropy struct task_struct *copy_process(
 	 */
 	copy_seccomp(p);
 
+	rv_task_fork(p);
+
 	rseq_fork(p, clone_flags);
 
 	/* Don't start children in a dying pid namespace */
kernel/trace/Kconfig +2 -0
index debbbb083286..b415690748bf 100644
@@ -1105,4 +1105,6 @@ config HIST_TRIGGERS_DEBUG
 
           If unsure, say N.
 
+source "kernel/trace/rv/Kconfig"
+
 endif # FTRACE
kernel/trace/Makefile +2 -0
index 0d261774d6f3..b2670fff6e94 100644
@@ -108,3 +108,5 @@ obj-$(CONFIG_RETHOOK) += rethook.o
 obj-$(CONFIG_TRACEPOINT_BENCHMARK) += trace_benchmark.o
 
 libftrace-y := ftrace.o
+
+obj-$(CONFIG_RV) += rv/
kernel/trace/rv/Kconfig new file +12 -0
new file mode 100644
index 000000000000..6d127cdb00dd
@@ -0,0 +1,12 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+menuconfig RV
+	bool "Runtime Verification"
+	depends on TRACING
+	help
+	  Enable the kernel runtime verification infrastructure. RV is a
+	  lightweight (yet rigorous) method that complements classical
+	  exhaustive verification techniques (such as model checking and
+	  theorem proving). RV works by analyzing the trace of the system's
+	  actual execution, comparing it against a formal specification of
+	  the system behavior.
kernel/trace/rv/Makefile new file +3 -0
new file mode 100644
index 000000000000..fd995379df67
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_RV) += rv.o
kernel/trace/rv/rv.c new file +738 -0

738 lines not on this page · raw message

kernel/trace/rv/rv.h new file +34 -0
new file mode 100644
index 000000000000..0796867a7b1e
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <linux/mutex.h>
+
+struct rv_interface {
+	struct dentry *root_dir;
+	struct dentry *monitors_dir;
+};
+
+#include "../trace.h"
+#include <linux/tracefs.h>
+#include <linux/rv.h>
+
+#define rv_create_dir			tracefs_create_dir
+#define rv_create_file			tracefs_create_file
+#define rv_remove			tracefs_remove
+
+#define MAX_RV_MONITOR_NAME_SIZE	32
+
+extern struct mutex rv_interface_lock;
+
+struct rv_monitor_def {
+	struct list_head list;
+	struct rv_monitor *monitor;
+	struct dentry *root_d;
+	bool enabled;
+	bool task_monitor;
+};
+
+extern bool monitoring_on;
+struct dentry *get_monitors_root(void);
+void reset_all_monitors(void);
+int init_rv_monitors(struct dentry *root_dir);
+int get_task_monitor_slot(void);
+void put_task_monitor_slot(int slot);
kernel/trace/trace.c +4 -0
index 2c95992e2c71..60e357c3120b 100644
@@ -9774,6 +9774,10 @@ static __init int tracer_init_tracefs(void)
 		tracer_init_tracefs_work_func(NULL);
 	}
 
+#ifdef CONFIG_RV
+	rv_init_interface();
+#endif
+
 	return 0;
 }
 
kernel/trace/trace.h +2 -0
index ff816fb41e48..becc03c0a45e 100644
@@ -2005,4 +2005,6 @@ struct trace_min_max_param {
 
 extern const struct file_operations trace_min_max_fops;
 
+extern int rv_init_interface(void);
+
 #endif /* _LINUX_KERNEL_TRACE_H */
Signature · 1 line
2.35.1
#2 lore raw

Re: [PATCH V4 01/20] rv: Add Runtime Verification (RV) interface

Hi Daniel,

A few flyby issues I noticed while going through the patches to understand what RV offers.

For the typos, I wonder if it isn't better to use a spellcheck - they unnecessarily detract from the review. Sorry for the annoyance!

Quoted 57 lines · Daniel Bristot de Oliveira

RV is a lightweight (yet rigorous) method that complements classical exhaustive verification techniques (such as model checking and theorem proving) with a more practical approach to complex systems.

RV works by analyzing the trace of the system's actual execution, comparing it against a formal specification of the system behavior. RV can give precise information on the runtime behavior of the monitored system while enabling the reaction for unexpected events, avoiding, for example, the propagation of a failure on safety-critical systems.

The development of this interface roots in the development of the paper:

DE OLIVEIRA, Daniel Bristot; CUCINOTTA, Tommaso; DE OLIVEIRA, Romulo Silva. Efficient formal verification for the Linux kernel. In:
International Conference on Software Engineering and Formal Methods. Springer, Cham, 2019. p. 315-332.

And:

DE OLIVEIRA, Daniel Bristot, et al. Automata-based formal analysis and verification of the real-time Linux kernel. PhD Thesis, 2020.

The RV interface resembles the tracing/ interface on purpose. The current path for the RV interface is /sys/kernel/tracing/rv/.

It presents these files:

"available_monitors"
- List the available monitors, one per line.

For example:
[root@f32 rv]# cat available_monitors
wip
wwnr

"enabled_monitors"
- Lists the enabled monitors, one per line;
- Writing to it enables a given monitor;
- Writing a monitor name with a '-' prefix disables it;
- Truncating the file disables all enabled monitors.

For example:
[root@f32 rv]# cat enabled_monitors
[root@f32 rv]# echo wip > enabled_monitors
[root@f32 rv]# echo wwnr >> enabled_monitors
[root@f32 rv]# cat enabled_monitors
wip
wwnr
[root@f32 rv]# echo !wip >> enabled_monitors
[root@f32 rv]# cat enabled_monitors
wwnr
[root@f32 rv]# echo > enabled_monitors
[root@f32 rv]# cat enabled_monitors
[root@f32 rv]#

Quoted 7 lines

Note that more than one monitor can be enabled concurrently.

"monitoring_on"
- It is an on/off general switcher for monitoring. Note that it does not disable enabled monitors, but stop the per-entity monitors of monitoring the events received from the system. It resambles the "tracing_on" switcher.

resembles

Quoted 5 lines

"monitors/"
Each monitor will have its one directory inside "monitors/". There the monitor specific files will be presented.
The "monitors/" directory resambles the "events" directory on

resembles

... and in the rest of the file.

Quoted 54 lines

tracefs.

For example:
[root@f32 rv]# cd monitors/wip/
[root@f32 wip]# ls
desc enable
[root@f32 wip]# cat desc
auto-generated wakeup in preemptive monitor.
[root@f32 wip]# cat enable
0

For further information, see the comments in the header of kernel/trace/rv/rv.c from this patch.

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 include/linux/rv.h       |  23 ++
 include/linux/sched.h    |  11 +
 include/rv/rv.h          |  23 ++
 kernel/fork.c            |  14 +
 kernel/trace/Kconfig     |   2 +
 kernel/trace/Makefile    |   2 +
 kernel/trace/rv/Kconfig  |  12 +
 kernel/trace/rv/Makefile |   3 +
 kernel/trace/rv/rv.c     | 738 +++++++++++++++++++++++++++++++++++++++
 kernel/trace/rv/rv.h     |  34 ++
 kernel/trace/trace.c     |   4 +
 kernel/trace/trace.h     |   2 +
 12 files changed, 868 insertions(+)
 create mode 100644 include/linux/rv.h
 create mode 100644 include/rv/rv.h
 create mode 100644 kernel/trace/rv/Kconfig
 create mode 100644 kernel/trace/rv/Makefile
 create mode 100644 kernel/trace/rv/rv.c
 create mode 100644 kernel/trace/rv/rv.h

[...]

Quoted 183 lines
kernel/trace/rv/rv.c new file +177 -0
new file mode 100644
index 000000000000..43af7b13187e
@@ -0,0 +1,738 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * This is the online Runtime Verification (RV) interface.
+ *
+ * RV is a lightweight (yet rigorous) method that complements classical
+ * exhaustive verification techniques (such as model checking and
+ * theorem proving) with a more practical approach to complex systems.
+ *
+ * RV works by analyzing the trace of the system's actual execution,
+ * comparing it against a formal specification of the system behavior.
+ * RV can give precise information on the runtime behavior of the
+ * monitored system while enabling the reaction for unexpected
+ * events, avoiding, for example, the propagation of a failure on
+ * safety-critical systems.
+ *
+ * The development of this interface roots in the development of the
+ * paper:
+ *
+ * DE OLIVEIRA, Daniel Bristot; CUCINOTTA, Tommaso; DE OLIVEIRA, Romulo
+ * Silva. Efficient formal verification for the Linux kernel. In:
+ * International Conference on Software Engineering and Formal Methods.
+ * Springer, Cham, 2019. p. 315-332.
+ *
+ * And:
+ *
+ * DE OLIVEIRA, Daniel Bristot, et al. Automata-based formal analysis
+ * and verification of the real-time Linux kernel. PhD Thesis, 2020.
+ *
+ * == Runtime monitor interface ==
+ *
+ * A monitor is the central part of the runtime verification of a system.
+ *
+ * The monitor stands in between the formal specification of the desired
+ * (or undesired) behavior, and the trace of the actual system.
+ *
+ * In Linux terms, the runtime verification monitors are encapsulated
+ * inside the "RV monitor" abstraction. A RV monitor includes a reference
+ * model of the system, a set of instances of the monitor (per-cpu monitor,
+ * per-task monitor, and so on), and the helper functions that glue the
+ * monitor to the system via trace. Generally, a monitor includes some form
+ * of trace output as a reaction for event parsing and exceptions,
+ * as depicted bellow:
+ *
+ * Linux  +----- RV Monitor ----------------------------------+ Formal
+ *  Realm |                                                   |  Realm
+ *  +-------------------+     +----------------+     +-----------------+
+ *  |   Linux kernel    |     |     Monitor    |     |     Reference   |
+ *  |     Tracing       |  -> |   Instance(s)  | <-  |       Model     |
+ *  | (instrumentation) |     | (verification) |     | (specification) |
+ *  +-------------------+     +----------------+     +-----------------+
+ *         |                          |                       |
+ *         |                          V                       |
+ *         |                     +----------+                 |
+ *         |                     | Reaction |                 |
+ *         |                     +--+--+--+-+                 |
+ *         |                        |  |  |                   |
+ *         |                        |  |  +-> trace output ?  |
+ *         +------------------------|--|----------------------+
+ *                                  |  +----> panic ?
+ *                                  +-------> <user-specified>
+ *
+ * This file implements the interface for loading RV monitors, and
+ * to control the verification session.
+ *
+ * == Registering monitors ==
+ *
+ * The struct rv_monitor defines a set of callback functions to control
+ * a verification session. For instance, when a given monitor is enabled,
+ * the "start" callback function is called to hook the instrumentation
+ * functions to the kernel trace events. The "stop" function is called
+ * when disabling the verification session.
+ *
+ * A RV monitor is registered via:
+ *   int rv_register_monitor(struct rv_monitor *monitor);
+ * And unregistered via:
+ *   int rv_unregister_monitor(struct rv_monitor *monitor);
+ *
+ * These functions are exported to modules, enabling verification monitors
+ * to be dynamically loaded.
+ *
+ * == User interface ==
+ *
+ * The user interface resembles kernel tracing interface. It presents
+ * these files:
+ *
+ *  "available_monitors"
+ *    - List the available monitors, one per line.
+ *
+ *    For example:
+ *    [root@f32 rv]# cat available_monitors
+ *    wip
+ *    wwnr
+ *
+ *  "enabled_monitors"
+ *    - Lists the enabled monitors, one per line;
+ *    - Writing to it enables a given monitor;
+ *    - Writing a monitor name with a '-' prefix disables it;
+ *    - Truncating the file disables all enabled monitors.
+ *
+ *    For example:
+ *    [root@f32 rv]# cat enabled_monitors
+ *    [root@f32 rv]# echo wip > enabled_monitors
+ *    [root@f32 rv]# echo wwnr >> enabled_monitors
+ *    [root@f32 rv]# cat enabled_monitors
+ *    wip
+ *    wwnr
+ *    [root@f32 rv]# echo !wip >> enabled_monitors
+ *    [root@f32 rv]# cat enabled_monitors
+ *    wwnr
+ *    [root@f32 rv]# echo > enabled_monitors
+ *    [root@f32 rv]# cat enabled_monitors
+ *    [root@f32 rv]#
+ *
+ *    Note that more than one monitor can be enabled concurrently.
+ *
+ *  "monitoring_on"
+ *    - It is an on/off general switcher for monitoring. Note
+ *    that it does not disable enabled monitors, but stop the per-entity
+ *    monitors of monitoring the events received from the system.
+ *    It resambles the "tracing_on" switcher.
+ *
+ *  "monitors/"
+ *    Each monitor will have its one directory inside "monitors/". There
+ *    the monitor specific files will be presented.
+ *    The "monitors/" directory resambles the "events" directory on
+ *    tracefs.
+ *
+ *    For example:
+ *    [root@f32 rv]# cd monitors/wip/
+ *    [root@f32 wip]# ls
+ *    desc  enable
+ *    [root@f32 wip]# cat desc
+ *    auto-generated wakeup in preemptive monitor.
+ *    [root@f32 wip]# cat enable
+ *    0
+ *
+ * Copyright (C) 2019-2022 Daniel Bristot de Oliveira <[email protected]>
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/slab.h>
+#include <rv/rv.h>
+
+#include "rv.h"
+
+DEFINE_MUTEX(rv_interface_lock);
+struct rv_interface rv_root;
+
+struct dentry *get_monitors_root(void)
+{
+	return rv_root.monitors_dir;
+}
+
+/*
+ * Monitoring on global switcher!
+ */
+bool __read_mostly monitoring_on;
+
+/*
+ * Interface for the monitor register.
+ */
+LIST_HEAD(rv_monitors_list);
+
+static int task_monitor_count;
+static bool task_monitor_slots[RV_PER_TASK_MONITORS];
+
+int get_task_monitor_slot(void)
+{
+	int i;
+
+	lockdep_assert_held(&rv_interface_lock);
+
+	if (task_monitor_count == RV_PER_TASK_MONITORS)
+		return -EBUSY;
+

kernel/trace/rv/rv.c line 187

Quoted 10 lines
kernel/trace/rv/rv.c new file +10 -0
+	task_monitor_count++;
+
+	for (i = 0; i < RV_PER_TASK_MONITORS; i++) {
+		if (task_monitor_slots[i] == false) {
+			task_monitor_slots[i] = true;
+			return i;
+		}
+	}
+
+	WARN_ONCE(1, "RV task_monitor_cout and slots are out of sync\n");

task_monitor_count

kernel/trace/rv/rv.c line 197

Quoted 10 lines
kernel/trace/rv/rv.c new file +10 -0
+
+	return -EINVAL;
+}
+
+void put_task_monitor_slot(int slot)
+{
+	lockdep_assert_held(&rv_interface_lock);
+
+	if (slot < 0 || slot > RV_PER_TASK_MONITORS) {
+		WARN_ONCE(1, "RV releasing an invlid slot!: %d\n", slot);

invalid

kernel/trace/rv/rv.c line 201

Quoted 4 lines
kernel/trace/rv/rv.c new file +4 -0
+		return;
+	}
+
+	WARN_ONCE(!task_monitor_slots[slot], "RV releasing unsused task_monitor_slots: %d\n",

unused

Quoted 7 lines
kernel/trace/rv/rv.c new file +7 -0
+		  slot);
+
+	task_monitor_count--;
+	task_monitor_slots[slot] = false;
+}
+
+/*

kernel/trace/rv/rv.c line 218

Quoted 10 lines
kernel/trace/rv/rv.c new file +10 -0
+ * This section collects the monitor/ files and folders.
+ */
+static ssize_t monitor_enable_read_data(struct file *filp,
+					char __user *user_buf,
+					size_t count, loff_t *ppos)
+{
+	struct rv_monitor_def *mdef = filp->private_data;
+	char buff[4];
+
+	memset(buff, 0, sizeof(buff));

Not sure if the generated code is different, but the memset can be rolled into buff declaration as -

char buff[4] = { 0 };

Quoted 48 lines
kernel/trace/rv/rv.c new file +48 -0
+
+	mutex_lock(&rv_interface_lock);
+	sprintf(buff, "%x\n", mdef->monitor->enabled);
+	mutex_unlock(&rv_interface_lock);
+
+	return simple_read_from_buffer(user_buf, count, ppos,
+				       buff, strlen(buff)+1);
+}
+
+/*
+ * Disable a given runtime monitor.
+ */
+static int disable_monitor(struct rv_monitor_def *mdef)
+{
+	if (mdef->monitor->enabled) {
+		mdef->monitor->enabled = 0;
+		mdef->monitor->stop();
+	}
+
+	mdef->enabled = 0;
+	return 0;
+}
+
+/*
+ * Enable a given monitor.
+ */
+static int enable_monitor(struct rv_monitor_def *mdef)
+{
+	int retval;
+
+	/*
+	 * Reset all internal monitors before starting.
+	 */
+	mdef->monitor->reset();
+	if (!mdef->monitor->enabled) {
+		retval = mdef->monitor->start();
+		if (retval)
+			return retval;
+	}
+
+	mdef->monitor->enabled = 1;
+	mdef->enabled = 1;
+
+	return 0;
+}
+
+/*
+ * interface for enabling/disabling a monitor.

kernel/trace/rv/rv.c line 276

Quoted 10 lines
kernel/trace/rv/rv.c new file +10 -0
+ */
+static ssize_t monitor_enable_write_data(struct file *filp,
+					 const char __user *user_buf,
+					 size_t count, loff_t *ppos)
+{
+	struct rv_monitor_def *mdef = filp->private_data;
+	int retval;
+	u64 val;
+
+	retval = kstrtoull_from_user(user_buf, count, 10, &val);

IIUC, kstrtobool_from_user() is better suited here.

Thanks,
Punit

Quoted 23 lines
kernel/trace/rv/rv.c new file +23 -0
+	if (retval)
+		return retval;
+
+	retval = count;
+
+	mutex_lock(&rv_interface_lock);
+
+	switch (val) {
+	case 0:
+		retval = disable_monitor(mdef);
+		break;
+	case 1:
+		retval = enable_monitor(mdef);
+		break;
+	default:
+		retval = -EINVAL;
+	}
+
+	mutex_unlock(&rv_interface_lock);
+
+	return retval;
+}
+

[...]

#3 lore raw

Re: [PATCH V4 01/20] rv: Add Runtime Verification (RV) interface

Hi Punit!

Quoted 7 lines · Punit Agrawal

Hi Daniel,

A few flyby issues I noticed while going through the patches to understand what RV offers.

For the typos, I wonder if it isn't better to use a spellcheck - they unnecessarily detract from the review. Sorry for the annoyance!

I do appreciate this kind of review. I run a spell checker, but sometimes I forget a patch here or there. I will add your changes and check the other patches.

Thanks!
-- Daniel

maintainer
#4 lore raw

Re: [PATCH V4 01/20] rv: Add Runtime Verification (RV) interface

In reply to Daniel Bristot de Oliveira
Quoted 11 lines · On Thu, 16 Jun 2022 10:44:43 +0200 Daniel Bristot de Oliveira…

It presents these files:

"available_monitors"
- List the available monitors, one per line.

For example:
[root@f32 rv]# cat available_monitors
wip
wwnr

Quoted 4 lines

"enabled_monitors"
- Lists the enabled monitors, one per line;
- Writing to it enables a given monitor;
- Writing a monitor name with a '-' prefix disables it;

You forgot to update this. You want '!' not '-'.

Quoted 10 lines

- Truncating the file disables all enabled monitors.

For example:
[root@f32 rv]# cat enabled_monitors
[root@f32 rv]# echo wip > enabled_monitors
[root@f32 rv]# echo wwnr >> enabled_monitors
[root@f32 rv]# cat enabled_monitors
wip
wwnr
[root@f32 rv]# echo !wip >> enabled_monitors

You need to add quotes around the ! because that's a bash special character.

echo '!wip' >> enabled_monitors

Quoted 6 lines

[root@f32 rv]# cat enabled_monitors
wwnr
[root@f32 rv]# echo > enabled_monitors
[root@f32 rv]# cat enabled_monitors
[root@f32 rv]#

Quoted 7 lines

Note that more than one monitor can be enabled concurrently.

"monitoring_on"
- It is an on/off general switcher for monitoring. Note that it does not disable enabled monitors, but stop the per-entity monitors of monitoring the events received from the system. It resambles the "tracing_on" switcher.

You mean that the tracepoints are still attached, but the process of monitoring isn't doing anything?

Quoted 66 lines

"monitors/"
Each monitor will have its one directory inside "monitors/". There the monitor specific files will be presented.
The "monitors/" directory resambles the "events" directory on tracefs.

For example:
[root@f32 rv]# cd monitors/wip/
[root@f32 wip]# ls
desc enable
[root@f32 wip]# cat desc
auto-generated wakeup in preemptive monitor.
[root@f32 wip]# cat enable
0

For further information, see the comments in the header of kernel/trace/rv/rv.c from this patch.

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 include/linux/rv.h       |  23 ++
 include/linux/sched.h    |  11 +
 include/rv/rv.h          |  23 ++
 kernel/fork.c            |  14 +
 kernel/trace/Kconfig     |   2 +
 kernel/trace/Makefile    |   2 +
 kernel/trace/rv/Kconfig  |  12 +
 kernel/trace/rv/Makefile |   3 +
 kernel/trace/rv/rv.c     | 738 +++++++++++++++++++++++++++++++++++++++
 kernel/trace/rv/rv.h     |  34 ++
 kernel/trace/trace.c     |   4 +
 kernel/trace/trace.h     |   2 +
 12 files changed, 868 insertions(+)
 create mode 100644 include/linux/rv.h
 create mode 100644 include/rv/rv.h
 create mode 100644 kernel/trace/rv/Kconfig
 create mode 100644 kernel/trace/rv/Makefile
 create mode 100644 kernel/trace/rv/rv.c
 create mode 100644 kernel/trace/rv/rv.h
include/linux/rv.h new file +0 -0
new file mode 100644
index 000000000000..205e65f57637

include/linux/rv.h line 10

Quoted 10 lines
include/linux/rv.h new file +10 -0
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Runtime Verification.
+ *
+ * For futher information, see: kernel/trace/rv/rv.c.
+ *
+ * Copyright (C) 2019-2022 Daniel Bristot de Oliveira <[email protected]>
+ */
+#ifndef _LINUX_RV_H
+#define _LINUX_RV_H

Add a space here.

Quoted 77 lines
include/linux/rv.h new file +13 -0
+struct rv_monitor {
+	const char		*name;
+	const char		*description;
+	bool			enabled;
+	int			(*start)(void);
+	void			(*stop)(void);
+	void			(*reset)(void);
+};
+
+extern bool monitoring_on;
+int rv_unregister_monitor(struct rv_monitor *monitor);
+int rv_register_monitor(struct rv_monitor *monitor);
+#endif /* _LINUX_RV_H */
include/linux/sched.h +11 -0
index c46f3a63b758..b037f364efdc 100644
@@ -35,6 +35,7 @@
 #include <linux/seqlock.h>
 #include <linux/kcsan.h>
 #include <asm/kmap_size.h>
+#include <rv/rv.h>
 
 /* task_struct member predeclarations (sorted alphabetically): */
 struct audit_context;
@@ -1500,6 +1501,16 @@ struct task_struct {
 	struct callback_head		l1d_flush_kill;
 #endif
 
+#ifdef CONFIG_RV
+	/*
+	 * Per-task RV monitor. Nowadays fixed in RV_PER_TASK_MONITORS.
+	 * If we find justification for more monitors, we can think
+	 * about adding more or developing a dynamic method. So far,
+	 * none of these are justified.
+	 */
+	union rv_task_monitor		rv[RV_PER_TASK_MONITORS];
+#endif
+
 	/*
 	 * New fields for task_struct should be added above here, so that
 	 * they are included in the randomized portion of task_struct.
include/rv/rv.h new file +23 -0
new file mode 100644
index 000000000000..27a108881d35
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef _RV_RV_H
+#define _RV_RV_H
+
+/*
+ * Per-task RV monitors count. Nowadays fixed in RV_PER_TASK_MONITORS.
+ * If we find justification for more monitors, we can think about
+ * adding more or developing a dynamic method. So far, none of
+ * these are justified.
+ */
+#define RV_PER_TASK_MONITORS		1
+#define RV_PER_TASK_MONITOR_INIT	(RV_PER_TASK_MONITORS)
+
+/*
+ * Futher monitor types are expected, so make this a union.
+ */
+union rv_task_monitor {
+};
+
+int get_task_monitor_slot(void);
+void put_task_monitor_slot(int slot);
+#endif /* _RV_RV_H */
kernel/fork.c +0 -0
index 9d44f2d46c69..5e40e58ef83d 100644
@@ -1964,6 +1964,18 @@ static void copy_oom_score_adj(u64 clone_flags, struct task_struct *tsk)
 	mutex_unlock(&oom_adj_mutex);

kernel/fork.c static void copy_oom_score_adj(u64 clone_flags, struct task_struct *tsk) · line 1974

Quoted 10 lines
kernel/fork.c +8 -0
 }
 
+#ifdef CONFIG_RV
+static void rv_task_fork(struct task_struct *p)
+{
+	int i;
+
+	for (i = 0; i < RV_PER_TASK_MONITORS; i++)
+		;
+}

Please move this to the next patch. It's useless for the review process to see a loop that does nothing.

Quoted 32 lines
kernel/fork.c +6 -0
+#else
+#define rv_task_fork(p) do {} while (0)
+#endif
+
 /*
  * This creates a new process as a copy of the old one,
  * but does not actually start it yet.
@@ -2399,6 +2411,8 @@ static __latent_entropy struct task_struct *copy_process(
 	 */
 	copy_seccomp(p);
 
+	rv_task_fork(p);
+
 	rseq_fork(p, clone_flags);
 
 	/* Don't start children in a dying pid namespace */
kernel/trace/Kconfig +2 -0
index debbbb083286..b415690748bf 100644
@@ -1105,4 +1105,6 @@ config HIST_TRIGGERS_DEBUG
 
           If unsure, say N.
 
+source "kernel/trace/rv/Kconfig"
+
 endif # FTRACE
kernel/trace/Makefile +0 -0
index 0d261774d6f3..b2670fff6e94 100644

kernel/trace/Makefile obj-$(CONFIG_RETHOOK) += rethook.o · line 112

Quoted 5 lines
kernel/trace/Makefile +2 -0
@@ -108,3 +108,5 @@ obj-$(CONFIG_RETHOOK) += rethook.o
 obj-$(CONFIG_TRACEPOINT_BENCHMARK) += trace_benchmark.o
 
 libftrace-y := ftrace.o
+
+obj-$(CONFIG_RV) += rv/

Please move this before the libftrace-y line. The libftrace-y is best at the end of the file.

---
obj-$(CONFIG_RV) += rv/

libftrace-y := ftrace.o
---

Quoted 120 lines
kernel/trace/rv/Kconfig new file +12 -0
new file mode 100644
index 000000000000..6d127cdb00dd
@@ -0,0 +1,12 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+menuconfig RV
+	bool "Runtime Verification"
+	depends on TRACING
+	help
+	  Enable the kernel runtime verification infrastructure. RV is a
+	  lightweight (yet rigorous) method that complements classical
+	  exhaustive verification techniques (such as model checking and
+	  theorem proving). RV works by analyzing the trace of the system's
+	  actual execution, comparing it against a formal specification of
+	  the system behavior.
kernel/trace/rv/Makefile new file +3 -0
new file mode 100644
index 000000000000..fd995379df67
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_RV) += rv.o
kernel/trace/rv/rv.c new file +87 -0
new file mode 100644
index 000000000000..43af7b13187e
@@ -0,0 +1,738 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * This is the online Runtime Verification (RV) interface.
+ *
+ * RV is a lightweight (yet rigorous) method that complements classical
+ * exhaustive verification techniques (such as model checking and
+ * theorem proving) with a more practical approach to complex systems.
+ *
+ * RV works by analyzing the trace of the system's actual execution,
+ * comparing it against a formal specification of the system behavior.
+ * RV can give precise information on the runtime behavior of the
+ * monitored system while enabling the reaction for unexpected
+ * events, avoiding, for example, the propagation of a failure on
+ * safety-critical systems.
+ *
+ * The development of this interface roots in the development of the
+ * paper:
+ *
+ * DE OLIVEIRA, Daniel Bristot; CUCINOTTA, Tommaso; DE OLIVEIRA, Romulo
+ * Silva. Efficient formal verification for the Linux kernel. In:
+ * International Conference on Software Engineering and Formal Methods.
+ * Springer, Cham, 2019. p. 315-332.
+ *
+ * And:
+ *
+ * DE OLIVEIRA, Daniel Bristot, et al. Automata-based formal analysis
+ * and verification of the real-time Linux kernel. PhD Thesis, 2020.
+ *
+ * == Runtime monitor interface ==
+ *
+ * A monitor is the central part of the runtime verification of a system.
+ *
+ * The monitor stands in between the formal specification of the desired
+ * (or undesired) behavior, and the trace of the actual system.
+ *
+ * In Linux terms, the runtime verification monitors are encapsulated
+ * inside the "RV monitor" abstraction. A RV monitor includes a reference
+ * model of the system, a set of instances of the monitor (per-cpu monitor,
+ * per-task monitor, and so on), and the helper functions that glue the
+ * monitor to the system via trace. Generally, a monitor includes some form
+ * of trace output as a reaction for event parsing and exceptions,
+ * as depicted bellow:
+ *
+ * Linux  +----- RV Monitor ----------------------------------+ Formal
+ *  Realm |                                                   |  Realm
+ *  +-------------------+     +----------------+     +-----------------+
+ *  |   Linux kernel    |     |     Monitor    |     |     Reference   |
+ *  |     Tracing       |  -> |   Instance(s)  | <-  |       Model     |
+ *  | (instrumentation) |     | (verification) |     | (specification) |
+ *  +-------------------+     +----------------+     +-----------------+
+ *         |                          |                       |
+ *         |                          V                       |
+ *         |                     +----------+                 |
+ *         |                     | Reaction |                 |
+ *         |                     +--+--+--+-+                 |
+ *         |                        |  |  |                   |
+ *         |                        |  |  +-> trace output ?  |
+ *         +------------------------|--|----------------------+
+ *                                  |  +----> panic ?
+ *                                  +-------> <user-specified>
+ *
+ * This file implements the interface for loading RV monitors, and
+ * to control the verification session.
+ *
+ * == Registering monitors ==
+ *
+ * The struct rv_monitor defines a set of callback functions to control
+ * a verification session. For instance, when a given monitor is enabled,
+ * the "start" callback function is called to hook the instrumentation
+ * functions to the kernel trace events. The "stop" function is called
+ * when disabling the verification session.
+ *
+ * A RV monitor is registered via:
+ *   int rv_register_monitor(struct rv_monitor *monitor);
+ * And unregistered via:
+ *   int rv_unregister_monitor(struct rv_monitor *monitor);
+ *
+ * These functions are exported to modules, enabling verification monitors
+ * to be dynamically loaded.
+ *
+ * == User interface ==
+ *
+ * The user interface resembles kernel tracing interface. It presents
+ * these files:
+ *
+ *  "available_monitors"
+ *    - List the available monitors, one per line.

kernel/trace/rv/rv.c line 97

Quoted 10 lines
kernel/trace/rv/rv.c new file +10 -0
+ *
+ *    For example:
+ *    [root@f32 rv]# cat available_monitors
+ *    wip
+ *    wwnr
+ *
+ *  "enabled_monitors"
+ *    - Lists the enabled monitors, one per line;
+ *    - Writing to it enables a given monitor;
+ *    - Writing a monitor name with a '-' prefix disables it;

Need to update here too.

Quoted 30 lines
kernel/trace/rv/rv.c new file +30 -0
+ *    - Truncating the file disables all enabled monitors.
+ *
+ *    For example:
+ *    [root@f32 rv]# cat enabled_monitors
+ *    [root@f32 rv]# echo wip > enabled_monitors
+ *    [root@f32 rv]# echo wwnr >> enabled_monitors
+ *    [root@f32 rv]# cat enabled_monitors
+ *    wip
+ *    wwnr
+ *    [root@f32 rv]# echo !wip >> enabled_monitors
+ *    [root@f32 rv]# cat enabled_monitors
+ *    wwnr
+ *    [root@f32 rv]# echo > enabled_monitors
+ *    [root@f32 rv]# cat enabled_monitors
+ *    [root@f32 rv]#
+ *
+ *    Note that more than one monitor can be enabled concurrently.
+ *
+ *  "monitoring_on"
+ *    - It is an on/off general switcher for monitoring. Note
+ *    that it does not disable enabled monitors, but stop the per-entity
+ *    monitors of monitoring the events received from the system.
+ *    It resambles the "tracing_on" switcher.
+ *
+ *  "monitors/"
+ *    Each monitor will have its one directory inside "monitors/". There
+ *    the monitor specific files will be presented.
+ *    The "monitors/" directory resambles the "events" directory on
+ *    tracefs.
+ *

kernel/trace/rv/rv.c line 137

Quoted 10 lines
kernel/trace/rv/rv.c new file +10 -0
+ *    For example:
+ *    [root@f32 rv]# cd monitors/wip/
+ *    [root@f32 wip]# ls
+ *    desc  enable
+ *    [root@f32 wip]# cat desc
+ *    auto-generated wakeup in preemptive monitor.
+ *    [root@f32 wip]# cat enable
+ *    0
+ *
+ * Copyright (C) 2019-2022 Daniel Bristot de Oliveira <[email protected]>

Copyright should be at the beginning of the comment.

Quoted 74 lines
kernel/trace/rv/rv.c new file +74 -0
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/slab.h>
+#include <rv/rv.h>
+
+#include "rv.h"
+
+DEFINE_MUTEX(rv_interface_lock);
+struct rv_interface rv_root;
+
+struct dentry *get_monitors_root(void)
+{
+	return rv_root.monitors_dir;
+}
+
+/*
+ * Monitoring on global switcher!
+ */
+bool __read_mostly monitoring_on;
+
+/*
+ * Interface for the monitor register.
+ */
+LIST_HEAD(rv_monitors_list);
+
+static int task_monitor_count;
+static bool task_monitor_slots[RV_PER_TASK_MONITORS];
+
+int get_task_monitor_slot(void)
+{
+	int i;
+
+	lockdep_assert_held(&rv_interface_lock);
+
+	if (task_monitor_count == RV_PER_TASK_MONITORS)
+		return -EBUSY;
+
+	task_monitor_count++;
+
+	for (i = 0; i < RV_PER_TASK_MONITORS; i++) {
+		if (task_monitor_slots[i] == false) {
+			task_monitor_slots[i] = true;
+			return i;
+		}
+	}
+
+	WARN_ONCE(1, "RV task_monitor_cout and slots are out of sync\n");
+
+	return -EINVAL;
+}
+
+void put_task_monitor_slot(int slot)
+{
+	lockdep_assert_held(&rv_interface_lock);
+
+	if (slot < 0 || slot > RV_PER_TASK_MONITORS) {
+		WARN_ONCE(1, "RV releasing an invlid slot!: %d\n", slot);
+		return;
+	}
+
+	WARN_ONCE(!task_monitor_slots[slot], "RV releasing unsused task_monitor_slots: %d\n",
+		  slot);
+
+	task_monitor_count--;
+	task_monitor_slots[slot] = false;
+}
+
+/*
+ * This section collects the monitor/ files and folders.
+ */
+static ssize_t monitor_enable_read_data(struct file *filp,

kernel/trace/rv/rv.c line 221

Quoted 10 lines
kernel/trace/rv/rv.c new file +10 -0
+					char __user *user_buf,
+					size_t count, loff_t *ppos)
+{
+	struct rv_monitor_def *mdef = filp->private_data;
+	char buff[4];
+
+	memset(buff, 0, sizeof(buff));
+
+	mutex_lock(&rv_interface_lock);
+	sprintf(buff, "%x\n", mdef->monitor->enabled);

Just make this:

	const char *buff;

	buff = mdef->monitor->enabled ? "1\n" : "0\n";
Quoted 7 lines
kernel/trace/rv/rv.c new file +7 -0
+	mutex_unlock(&rv_interface_lock);
+
+	return simple_read_from_buffer(user_buf, count, ppos,
+				       buff, strlen(buff)+1);
+}
+
+/*

kernel/trace/rv/rv.c line 238

Quoted 10 lines
kernel/trace/rv/rv.c new file +10 -0
+ * Disable a given runtime monitor.
+ */
+static int disable_monitor(struct rv_monitor_def *mdef)
+{
+	if (mdef->monitor->enabled) {
+		mdef->monitor->enabled = 0;
+		mdef->monitor->stop();
+	}
+
+	mdef->enabled = 0;

What's the difference between mdef->enabled and mdef->monitor->enabled?

Quoted 67 lines
kernel/trace/rv/rv.c new file +67 -0
+	return 0;
+}
+
+/*
+ * Enable a given monitor.
+ */
+static int enable_monitor(struct rv_monitor_def *mdef)
+{
+	int retval;
+
+	/*
+	 * Reset all internal monitors before starting.
+	 */
+	mdef->monitor->reset();
+	if (!mdef->monitor->enabled) {
+		retval = mdef->monitor->start();
+		if (retval)
+			return retval;
+	}
+
+	mdef->monitor->enabled = 1;
+	mdef->enabled = 1;
+
+	return 0;
+}
+
+/*
+ * interface for enabling/disabling a monitor.
+ */
+static ssize_t monitor_enable_write_data(struct file *filp,
+					 const char __user *user_buf,
+					 size_t count, loff_t *ppos)
+{
+	struct rv_monitor_def *mdef = filp->private_data;
+	int retval;
+	u64 val;
+
+	retval = kstrtoull_from_user(user_buf, count, 10, &val);
+	if (retval)
+		return retval;
+
+	retval = count;
+
+	mutex_lock(&rv_interface_lock);
+
+	switch (val) {
+	case 0:
+		retval = disable_monitor(mdef);
+		break;
+	case 1:
+		retval = enable_monitor(mdef);
+		break;
+	default:
+		retval = -EINVAL;
+	}
+
+	mutex_unlock(&rv_interface_lock);
+
+	return retval;
+}
+
+static const struct file_operations interface_enable_fops = {
+	.open   = simple_open,
+	.llseek = no_llseek,
+	.write  = monitor_enable_write_data,
+	.read   = monitor_enable_read_data,
+};

kernel/trace/rv/rv.c line 315

Quoted 10 lines
kernel/trace/rv/rv.c new file +10 -0
+
+/*
+ * Interface to read the enable/disable status of a monitor.
+ */
+static ssize_t
+monitor_desc_read_data(struct file *filp, char __user *user_buf,
+		       size_t count, loff_t *ppos)
+{
+	struct rv_monitor_def *mdef = filp->private_data;
+	char buf[MAX_RV_MONITOR_NAME_SIZE];

I think you want to add +2 here. "\n\0"

kernel/trace/rv/rv.c line 320

Quoted 5 lines
kernel/trace/rv/rv.c new file +5 -0
+
+	memset(buf, 0, sizeof(buf));
+
+	mutex_lock(&rv_interface_lock);
+	sprintf(buf, "%s\n", mdef->monitor->description);

Use snprintf().

Quoted 20 lines
kernel/trace/rv/rv.c new file +20 -0
+	mutex_unlock(&rv_interface_lock);
+
+	return simple_read_from_buffer(user_buf, count, ppos,
+					buf, strlen(buf)+1);
+}
+
+static const struct file_operations interface_desc_fops = {
+	.open   = simple_open,
+	.llseek	= no_llseek,
+	.read	= monitor_desc_read_data,
+};
+
+/*
+ * During the registration of a monitor, this function creates
+ * the monitor dir, where the specific options of the monitor
+ * is exposed.
+ */
+static int create_monitor_dir(struct rv_monitor_def *mdef)
+{
+	struct dentry *root = get_monitors_root();

kernel/trace/rv/rv.c line 350

Quoted 10 lines
kernel/trace/rv/rv.c new file +10 -0
+	struct dentry *tmp;
+	const char *name = mdef->monitor->name;
+	int retval = 0;
+
+	mdef->root_d = rv_create_dir(name, root);
+
+	if (!mdef->root_d)
+		return -ENOMEM;
+
+	tmp = rv_create_file("enable", 0600,

I'd recommend make the modes (0600) into macros. I recently changed these for tracing, and having them hard coded was a pain.

#define RV_FILE_READ 0600

kernel/trace/rv/rv.c line 358

Quoted 8 lines
kernel/trace/rv/rv.c new file +8 -0
+			     mdef->root_d, mdef,
+			     &interface_enable_fops);
+	if (!tmp) {
+		retval = -ENOMEM;
+		goto out_remove_root;
+	}
+
+	tmp = rv_create_file("desc", 0400,

Same here, and in all other cases.

Quoted 157 lines
kernel/trace/rv/rv.c new file +157 -0
+			      mdef->root_d, mdef,
+			      &interface_desc_fops);
+	if (!tmp) {
+		retval = -ENOMEM;
+		goto out_remove_root;
+	}
+
+	return retval;
+
+out_remove_root:
+	rv_remove(mdef->root_d);
+	return retval;
+}
+
+/*
+ * Available/Enable monitor shared seq functions.
+ */
+static int monitors_show(struct seq_file *m, void *p)
+{
+	struct rv_monitor_def *mon_def = p;
+
+	seq_printf(m, "%s\n", mon_def->monitor->name);
+	return 0;
+}
+
+/*
+ * Used by the seq file operations at the end of a read
+ * operation.
+ */
+static void monitors_stop(struct seq_file *m, void *p)
+{
+	mutex_unlock(&rv_interface_lock);
+}
+
+/*
+ * Available monitor seq functions:
+ */
+static void *available_monitors_start(struct seq_file *m, loff_t *pos)
+{
+	mutex_lock(&rv_interface_lock);
+	return seq_list_start(&rv_monitors_list, *pos);
+}
+
+static void *available_monitors_next(struct seq_file *m, void *p, loff_t *pos)
+{
+	return seq_list_next(p, &rv_monitors_list, pos);
+}
+
+/*
+ * Enable monitor seq functions:
+ */
+
+static void *enabled_monitors_next(struct seq_file *m, void *p, loff_t *pos)
+{
+	struct rv_monitor_def *m_def = p;
+
+	(*pos)++;
+
+	list_for_each_entry_continue(m_def, &rv_monitors_list, list) {
+		if (m_def->monitor->enabled)
+			return m_def;
+	}
+
+	return NULL;
+}
+
+static void *enabled_monitors_start(struct seq_file *m, loff_t *pos)
+{
+	struct rv_monitor_def *m_def;
+	loff_t l;
+
+	mutex_lock(&rv_interface_lock);
+	m_def = list_entry(&rv_monitors_list, struct rv_monitor_def, list);
+
+	for (l = 0; l <= *pos; ) {
+		m_def = enabled_monitors_next(m, m_def, &l);
+		if (!m_def)
+			break;
+	}
+
+	return m_def;
+}
+
+/*
+ * available/enabled monitors seq definition.
+ */
+static const struct seq_operations available_monitors_seq_ops = {
+	.start	= available_monitors_start,
+	.next	= available_monitors_next,
+	.stop	= monitors_stop,
+	.show	= monitors_show
+};
+
+static const struct seq_operations enabled_monitors_seq_ops = {
+	.start  = enabled_monitors_start,
+	.next   = enabled_monitors_next,
+	.stop   = monitors_stop,
+	.show   = monitors_show
+};
+
+/*
+ * available_monitors interface.
+ */
+static int available_monitors_open(struct inode *inode, struct file *file)
+{
+	return seq_open(file, &available_monitors_seq_ops);
+};
+
+static const struct file_operations available_monitors_ops = {
+	.open    = available_monitors_open,
+	.read    = seq_read,
+	.llseek  = seq_lseek,
+	.release = seq_release
+};
+
+/*
+ * enabled_monitors interface
+ */
+static void disable_all_monitors(void)
+{
+	struct rv_monitor_def *mdef;
+
+	list_for_each_entry(mdef, &rv_monitors_list, list)
+		disable_monitor(mdef);
+}
+
+static int enabled_monitors_open(struct inode *inode, struct file *file)
+{
+	if ((file->f_mode & FMODE_WRITE) && (file->f_flags & O_TRUNC))
+		disable_all_monitors();
+
+	return seq_open(file, &enabled_monitors_seq_ops);
+};
+
+static ssize_t
+enabled_monitors_write(struct file *filp, const char __user *user_buf,
+		      size_t count, loff_t *ppos)
+{
+	char buff[MAX_RV_MONITOR_NAME_SIZE+1];
+	struct rv_monitor_def *mdef;
+	int retval = -EINVAL;
+	bool enable = true;
+	char *ptr = buff;
+	int len;
+
+	if (count < 1 || count > MAX_RV_MONITOR_NAME_SIZE+1)
+		return -EINVAL;
+
+	memset(buff, 0, sizeof(buff));
+
+	retval = simple_write_to_buffer(buff, sizeof(buff)-1, ppos, user_buf,
+					count);
+	if (!retval)
+		return -EFAULT;
+
+	if (buff[0] == '!') {
+		enable = false;

kernel/trace/rv/rv.c line 525

Quoted 10 lines
kernel/trace/rv/rv.c new file +10 -0
+		ptr++;
+	}
+
+	len = strlen(ptr);
+	if (!len)
+		return count;
+	/*
+	 * remove \n
+	 */
+	ptr[len-1] = '\0';

Are you sure there's an '\n' here?

One could just do "write(fd, "monitor", 7)"

Perhaps use strim()

kernel/trace/rv/rv.c line 532

Quoted 7 lines
kernel/trace/rv/rv.c new file +7 -0
+
+	mutex_lock(&rv_interface_lock);
+
+	retval = -EINVAL;
+
+	list_for_each_entry(mdef, &rv_monitors_list, list) {
+		if (strcmp(ptr, mdef->monitor->name) == 0) {

BTW, you could do:

		if (strcmp(ptr, mdef->monitor->name) != 0)
			continue;

And then get rid of an extra indent below.

kernel/trace/rv/rv.c line 542

Quoted 10 lines
kernel/trace/rv/rv.c new file +10 -0
+			/*
+			 * Monitor found!
+			 */
+			if (enable)
+				retval = enable_monitor(mdef);
+			else
+				retval = disable_monitor(mdef);
+
+			if (retval)
+				goto out;

Why not just break?

In fact, you could just do:

			if (!retval)
				retval = count;
			break;
Quoted 25 lines
kernel/trace/rv/rv.c new file +25 -0
+
+			/*
+			 * Success!
+			 */
+			retval = count;
+			break;
+		}
+	}
+
+out:
+	mutex_unlock(&rv_interface_lock);
+	return retval;
+}
+
+static const struct file_operations enabled_monitors_ops = {
+	.open		= enabled_monitors_open,
+	.read		= seq_read,
+	.write		= enabled_monitors_write,
+	.llseek		= seq_lseek,
+	.release	= seq_release,
+};
+
+/*
+ * monitoring_on general switcher
+ */

kernel/trace/rv/rv.c line 577

Quoted 10 lines
kernel/trace/rv/rv.c new file +10 -0
+static ssize_t monitoring_on_read_data(struct file *filp,
+					char __user *user_buf,
+					size_t count, loff_t *ppos)
+{
+	char buff[4];
+
+	memset(buff, 0, sizeof(buff));
+
+	mutex_lock(&rv_interface_lock);
+	sprintf(buff, "%d\n", monitoring_on);

Same here with just hard coding "1\n" : "0\n"

Quoted 90 lines
kernel/trace/rv/rv.c new file +90 -0
+	mutex_unlock(&rv_interface_lock);
+
+	return simple_read_from_buffer(user_buf, count, ppos,
+				       buff, strlen(buff)+1);
+}
+
+static void turn_monitoring_off(void)
+{
+	monitoring_on = false;
+}
+
+static void turn_monitoring_on(void)
+{
+	reset_all_monitors();
+	monitoring_on = true;
+}
+
+static ssize_t monitoring_on_write_data(struct file *filp,
+					 const char __user *user_buf,
+					 size_t count, loff_t *ppos)
+{
+	int retval;
+	u64 val;
+
+	retval = kstrtoull_from_user(user_buf, count, 10, &val);
+	if (retval)
+		return retval;
+
+	retval = count;
+
+	mutex_lock(&rv_interface_lock);
+
+	switch (val) {
+	case 0:
+		turn_monitoring_off();
+		break;
+	case 1:
+		turn_monitoring_on();
+		break;
+	default:
+		retval = -EINVAL;
+	}
+
+	mutex_unlock(&rv_interface_lock);
+
+	return retval;
+}
+
+static const struct file_operations monitoring_on_fops = {
+	.open   = simple_open,
+	.llseek = no_llseek,
+	.write  = monitoring_on_write_data,
+	.read   = monitoring_on_read_data,
+};
+
+/*
+ * Monitor API.
+ */
+static void destroy_monitor_dir(struct rv_monitor_def *mdef)
+{
+	rv_remove(mdef->root_d);
+}
+
+/**
+ * rv_register_monitor - register a rv monitor.
+ * @monitor:    The rv_monitor to be registered.
+ *
+ * Returns 0 if successful, error otherwise.
+ */
+int rv_register_monitor(struct rv_monitor *monitor)
+{
+	struct rv_monitor_def *r;
+	int retval = 0;
+
+	if (strlen(monitor->name) >= MAX_RV_MONITOR_NAME_SIZE) {
+		pr_info("Monitor %s has a name longer than %d\n",
+			monitor->name, MAX_RV_MONITOR_NAME_SIZE);
+		return -1;
+	}
+
+	mutex_lock(&rv_interface_lock);
+
+	list_for_each_entry(r, &rv_monitors_list, list) {
+		if (strcmp(monitor->name, r->monitor->name) == 0) {
+			pr_info("Monitor %s is already registered\n",
+				monitor->name);
+			retval = -1;
+			goto out_unlock;
+		}
+	}

kernel/trace/rv/rv.c line 677

Quoted 10 lines
kernel/trace/rv/rv.c new file +10 -0
+
+	r = kzalloc(sizeof(struct rv_monitor_def), GFP_KERNEL);
+	if (!r) {
+		retval = -ENOMEM;
+		goto out_unlock;
+	}
+
+	r->monitor = monitor;
+
+	create_monitor_dir(r);

Should you check for success in creating the dir?

Quoted 83 lines
kernel/trace/rv/rv.c new file +61 -0
+
+	list_add_tail(&r->list, &rv_monitors_list);
+
+out_unlock:
+	mutex_unlock(&rv_interface_lock);
+	return retval;
+}
+
+/**
+ * rv_unregister_monitor - unregister a rv monitor.
+ * @monitor:    The rv_monitor to be unregistered.
+ *
+ * Returns 0 if successful, error otherwise.
+ */
+int rv_unregister_monitor(struct rv_monitor *monitor)
+{
+	struct rv_monitor_def *ptr, *next;
+
+	mutex_lock(&rv_interface_lock);
+
+	list_for_each_entry_safe(ptr, next, &rv_monitors_list, list) {
+		if (strcmp(monitor->name, ptr->monitor->name) == 0) {
+			list_del(&ptr->list);
+			destroy_monitor_dir(ptr);
+		}
+	}
+
+	mutex_unlock(&rv_interface_lock);
+	return 0;
+}
+
+void reset_all_monitors(void)
+{
+	struct rv_monitor_def *mdef;
+
+	/*
+	 * Reset all monitors before re-enabling monitoring.
+	 */
+	list_for_each_entry(mdef, &rv_monitors_list, list) {
+		if (mdef->monitor->enabled)
+			mdef->monitor->reset();
+	}
+
+}
+
+int __init rv_init_interface(void)
+{
+	rv_root.root_dir = rv_create_dir("rv", NULL);
+	rv_root.monitors_dir = rv_create_dir("monitors", rv_root.root_dir);
+
+	rv_create_file("available_monitors", 0400, rv_root.root_dir, NULL,
+		       &available_monitors_ops);
+	rv_create_file("enabled_monitors", 0600, rv_root.root_dir, NULL,
+		       &enabled_monitors_ops);
+	rv_create_file("monitoring_on", 0600, rv_root.root_dir, NULL,
+		       &monitoring_on_fops);
+
+	monitoring_on = true;
+
+	return 0;
+}
kernel/trace/rv/rv.h new file +16 -0
new file mode 100644
index 000000000000..0796867a7b1e
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <linux/mutex.h>
+
+struct rv_interface {
+	struct dentry *root_dir;
+	struct dentry *monitors_dir;
+};
+
+#include "../trace.h"
+#include <linux/tracefs.h>
+#include <linux/rv.h>
+
+#define rv_create_dir			tracefs_create_dir
+#define rv_create_file			tracefs_create_file
+#define rv_remove			tracefs_remove
+

kernel/trace/rv/rv.h line 26

Quoted 10 lines
kernel/trace/rv/rv.h new file +10 -0
+#define MAX_RV_MONITOR_NAME_SIZE	32
+
+extern struct mutex rv_interface_lock;
+
+struct rv_monitor_def {
+	struct list_head list;
+	struct rv_monitor *monitor;
+	struct dentry *root_d;
+	bool enabled;
+	bool task_monitor;

Can you tab these to make it easier to read?

struct rv_monitor_def {
	struct list_head	list;
	struct rv_monitor	*monitor;
	struct dentry		*root_d;
	bool			enabled;
	bool			task_monitor;
};

-- Steve

Quoted 34 lines
kernel/trace/rv/rv.h new file +8 -0
+};
+
+extern bool monitoring_on;
+struct dentry *get_monitors_root(void);
+void reset_all_monitors(void);
+int init_rv_monitors(struct dentry *root_dir);
+int get_task_monitor_slot(void);
+void put_task_monitor_slot(int slot);
kernel/trace/trace.c +4 -0
index 2c95992e2c71..60e357c3120b 100644
@@ -9774,6 +9774,10 @@ static __init int tracer_init_tracefs(void)
 		tracer_init_tracefs_work_func(NULL);
 	}
 
+#ifdef CONFIG_RV
+	rv_init_interface();
+#endif
+
 	return 0;
 }
 
kernel/trace/trace.h +2 -0
index ff816fb41e48..becc03c0a45e 100644
@@ -2005,4 +2005,6 @@ struct trace_min_max_param {
 
 extern const struct file_operations trace_min_max_fops;
 
+extern int rv_init_interface(void);
+
 #endif /* _LINUX_KERNEL_TRACE_H */
#5 lore raw

Re: [PATCH V4 01/20] rv: Add Runtime Verification (RV) interface

Quoted 2 lines · Steven Rostedt

On Thu, 16 Jun 2022 10:44:43 +0200
Daniel Bristot de Oliveira <[email protected]> wrote:

[ removing comments that I agreed and changed the code/log accordingly ]

Quoted 9 lines
Quoted 6 lines

"monitoring_on"
- It is an on/off general switcher for monitoring. Note that it does not disable enabled monitors, but stop the per-entity monitors of monitoring the events received from the system. It resambles the "tracing_on" switcher.

You mean that the tracepoints are still attached, but the process of monitoring isn't doing anything?

correct, I am now mentioning it in the comment.

[...]

Quoted 10 lines

kernel/trace/rv/rv.c line 238

Quoted 8 lines
kernel/trace/rv/rv.c new file +8 -0
+static int disable_monitor(struct rv_monitor_def *mdef)
+{
+	if (mdef->monitor->enabled) {
+		mdef->monitor->enabled = 0;
+		mdef->monitor->stop();
+	}
+
+	mdef->enabled = 0;

What's the difference between mdef->enabled and mdef->monitor->enabled?

Ooops, the mdef->enabled is a leftover... removing mdef->enabled.

Quoted 3 lines
Quoted 3 lines
kernel/trace/rv/rv.c new file +3 -0
+	return 0;
+}
+

[...]

Quoted 17 lines

kernel/trace/rv/rv.c line 350

Quoted 13 lines
kernel/trace/rv/rv.c new file +13 -0
+static int create_monitor_dir(struct rv_monitor_def *mdef)
+{
+	struct dentry *root = get_monitors_root();
+	struct dentry *tmp;
+	const char *name = mdef->monitor->name;
+	int retval = 0;
+
+	mdef->root_d = rv_create_dir(name, root);
+
+	if (!mdef->root_d)
+		return -ENOMEM;
+
+	tmp = rv_create_file("enable", 0600,

I'd recommend make the modes (0600) into macros. I recently changed these for tracing, and having them hard coded was a pain.

Quoted 2 lines

#define RV_FILE_READ 0600

Added:
#define RV_MODE_WRITE TRACE_MODE_WRITE
#define RV_MODE_READ TRACE_MODE_READ

Quoted 24 lines

kernel/trace/rv/rv.c line 358

Quoted 8 lines
kernel/trace/rv/rv.c new file +8 -0
+			     mdef->root_d, mdef,
+			     &interface_enable_fops);
+	if (!tmp) {
+		retval = -ENOMEM;
+		goto out_remove_root;
+	}
+
+	tmp = rv_create_file("desc", 0400,

Same here, and in all other cases.

Quoted 13 lines
kernel/trace/rv/rv.c new file +13 -0
+			      mdef->root_d, mdef,
+			      &interface_desc_fops);
+	if (!tmp) {
+		retval = -ENOMEM;
+		goto out_remove_root;
+	}
+
+	return retval;
+
+out_remove_root:
+	rv_remove(mdef->root_d);
+	return retval;
+}

[...]

Quoted 36 lines
Quoted 23 lines
kernel/trace/rv/rv.c new file +23 -0
+static ssize_t
+enabled_monitors_write(struct file *filp, const char __user *user_buf,
+		      size_t count, loff_t *ppos)
+{
+	char buff[MAX_RV_MONITOR_NAME_SIZE+1];
+	struct rv_monitor_def *mdef;
+	int retval = -EINVAL;
+	bool enable = true;
+	char *ptr = buff;
+	int len;
+
+	if (count < 1 || count > MAX_RV_MONITOR_NAME_SIZE+1)
+		return -EINVAL;
+
+	memset(buff, 0, sizeof(buff));
+
+	retval = simple_write_to_buffer(buff, sizeof(buff)-1, ppos, user_buf,
+					count);
+	if (!retval)
+		return -EFAULT;
+
+	if (buff[0] == '!') {
+		enable = false;

kernel/trace/rv/rv.c line 525

Quoted 10 lines
kernel/trace/rv/rv.c new file +10 -0
+		ptr++;
+	}
+
+	len = strlen(ptr);
+	if (!len)
+		return count;
+	/*
+	 * remove \n
+	 */
+	ptr[len-1] = '\0';

Are you sure there's an '\n' here?

Quoted 3 lines

One could just do "write(fd, "monitor", 7)"

Perhaps use strim()

ack.

Quoted 29 lines

kernel/trace/rv/rv.c line 532

Quoted 8 lines
kernel/trace/rv/rv.c new file +7 -0
+
+	mutex_lock(&rv_interface_lock);
+
+	retval = -EINVAL;
+
+	list_for_each_entry(mdef, &rv_monitors_list, list) {
+		if (strcmp(ptr, mdef->monitor->name) == 0) {

BTW, you could do:

		if (strcmp(ptr, mdef->monitor->name) != 0)
			continue;

And then get rid of an extra indent below.

kernel/trace/rv/rv.c line 542

Quoted 10 lines
kernel/trace/rv/rv.c new file +10 -0
+			/*
+			 * Monitor found!
+			 */
+			if (enable)
+				retval = enable_monitor(mdef);
+			else
+				retval = disable_monitor(mdef);
+
+			if (retval)
+				goto out;

Why not just break?

Quoted 5 lines

In fact, you could just do:

			if (!retval)
				retval = count;
			break;

yep, it looks better.

-- Daniel

#6 lore raw

Re: [PATCH V4 01/20] rv: Add Runtime Verification (RV) interface

In reply to Daniel Bristot de Oliveira
Quoted 134 lines · Daniel Bristot de Oliveira

RV is a lightweight (yet rigorous) method that complements classical exhaustive verification techniques (such as model checking and theorem proving) with a more practical approach to complex systems.

RV works by analyzing the trace of the system's actual execution, comparing it against a formal specification of the system behavior. RV can give precise information on the runtime behavior of the monitored system while enabling the reaction for unexpected events, avoiding, for example, the propagation of a failure on safety-critical systems.

The development of this interface roots in the development of the paper:

DE OLIVEIRA, Daniel Bristot; CUCINOTTA, Tommaso; DE OLIVEIRA, Romulo Silva. Efficient formal verification for the Linux kernel. In:
International Conference on Software Engineering and Formal Methods. Springer, Cham, 2019. p. 315-332.

And:

DE OLIVEIRA, Daniel Bristot, et al. Automata-based formal analysis and verification of the real-time Linux kernel. PhD Thesis, 2020.

The RV interface resembles the tracing/ interface on purpose. The current path for the RV interface is /sys/kernel/tracing/rv/.

It presents these files:

"available_monitors"
- List the available monitors, one per line.

For example:
[root@f32 rv]# cat available_monitors
wip
wwnr

"enabled_monitors"
- Lists the enabled monitors, one per line;
- Writing to it enables a given monitor;
- Writing a monitor name with a '-' prefix disables it;
- Truncating the file disables all enabled monitors.

For example:
[root@f32 rv]# cat enabled_monitors
[root@f32 rv]# echo wip > enabled_monitors
[root@f32 rv]# echo wwnr >> enabled_monitors
[root@f32 rv]# cat enabled_monitors
wip
wwnr
[root@f32 rv]# echo !wip >> enabled_monitors
[root@f32 rv]# cat enabled_monitors
wwnr
[root@f32 rv]# echo > enabled_monitors
[root@f32 rv]# cat enabled_monitors
[root@f32 rv]#

Note that more than one monitor can be enabled concurrently.

"monitoring_on"
- It is an on/off general switcher for monitoring. Note that it does not disable enabled monitors, but stop the per-entity monitors of monitoring the events received from the system. It resambles the "tracing_on" switcher.

"monitors/"
Each monitor will have its one directory inside "monitors/". There the monitor specific files will be presented.
The "monitors/" directory resambles the "events" directory on tracefs.

For example:
[root@f32 rv]# cd monitors/wip/
[root@f32 wip]# ls
desc enable
[root@f32 wip]# cat desc
auto-generated wakeup in preemptive monitor.
[root@f32 wip]# cat enable
0

For further information, see the comments in the header of kernel/trace/rv/rv.c from this patch.

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 include/linux/rv.h       |  23 ++
 include/linux/sched.h    |  11 +
 include/rv/rv.h          |  23 ++
 kernel/fork.c            |  14 +
 kernel/trace/Kconfig     |   2 +
 kernel/trace/Makefile    |   2 +
 kernel/trace/rv/Kconfig  |  12 +
 kernel/trace/rv/Makefile |   3 +
 kernel/trace/rv/rv.c     | 738 +++++++++++++++++++++++++++++++++++++++
 kernel/trace/rv/rv.h     |  34 ++
 kernel/trace/trace.c     |   4 +
 kernel/trace/trace.h     |   2 +
 12 files changed, 868 insertions(+)
 create mode 100644 include/linux/rv.h
 create mode 100644 include/rv/rv.h
 create mode 100644 kernel/trace/rv/Kconfig
 create mode 100644 kernel/trace/rv/Makefile
 create mode 100644 kernel/trace/rv/rv.c
 create mode 100644 kernel/trace/rv/rv.h
include/linux/rv.h new file +4 -0
new file mode 100644
index 000000000000..205e65f57637
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Runtime Verification.
+ *

include/linux/rv.h line 14

Quoted 10 lines
include/linux/rv.h new file +10 -0
+ * For futher information, see: kernel/trace/rv/rv.c.
+ *
+ * Copyright (C) 2019-2022 Daniel Bristot de Oliveira <[email protected]>
+ */
+#ifndef _LINUX_RV_H
+#define _LINUX_RV_H
+struct rv_monitor {
+	const char		*name;
+	const char		*description;
+	bool			enabled;

Can the 'bool enabled;' be put at the end like the definition of structure rv_monitor_def. If '8+8+sizeof(bool)+8+8+8' not the same as '8+8+8+8+8+sizeof(bool)', I mean is it possible that after the end of stucture there is a int or char not require to align to 8 as an example from my nonsense.

Quoted 574 lines
include/linux/rv.h new file +9 -0
+	int			(*start)(void);
+	void			(*stop)(void);
+	void			(*reset)(void);
+};
+
+extern bool monitoring_on;
+int rv_unregister_monitor(struct rv_monitor *monitor);
+int rv_register_monitor(struct rv_monitor *monitor);
+#endif /* _LINUX_RV_H */
include/linux/sched.h +11 -0
index c46f3a63b758..b037f364efdc 100644
@@ -35,6 +35,7 @@
 #include <linux/seqlock.h>
 #include <linux/kcsan.h>
 #include <asm/kmap_size.h>
+#include <rv/rv.h>
 
 /* task_struct member predeclarations (sorted alphabetically): */
 struct audit_context;
@@ -1500,6 +1501,16 @@ struct task_struct {
 	struct callback_head		l1d_flush_kill;
 #endif
 
+#ifdef CONFIG_RV
+	/*
+	 * Per-task RV monitor. Nowadays fixed in RV_PER_TASK_MONITORS.
+	 * If we find justification for more monitors, we can think
+	 * about adding more or developing a dynamic method. So far,
+	 * none of these are justified.
+	 */
+	union rv_task_monitor		rv[RV_PER_TASK_MONITORS];
+#endif
+
 	/*
 	 * New fields for task_struct should be added above here, so that
 	 * they are included in the randomized portion of task_struct.
include/rv/rv.h new file +23 -0
new file mode 100644
index 000000000000..27a108881d35
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef _RV_RV_H
+#define _RV_RV_H
+
+/*
+ * Per-task RV monitors count. Nowadays fixed in RV_PER_TASK_MONITORS.
+ * If we find justification for more monitors, we can think about
+ * adding more or developing a dynamic method. So far, none of
+ * these are justified.
+ */
+#define RV_PER_TASK_MONITORS		1
+#define RV_PER_TASK_MONITOR_INIT	(RV_PER_TASK_MONITORS)
+
+/*
+ * Futher monitor types are expected, so make this a union.
+ */
+union rv_task_monitor {
+};
+
+int get_task_monitor_slot(void);
+void put_task_monitor_slot(int slot);
+#endif /* _RV_RV_H */
kernel/fork.c +14 -0
index 9d44f2d46c69..5e40e58ef83d 100644
@@ -1964,6 +1964,18 @@ static void copy_oom_score_adj(u64 clone_flags, struct task_struct *tsk)
 	mutex_unlock(&oom_adj_mutex);
 }
 
+#ifdef CONFIG_RV
+static void rv_task_fork(struct task_struct *p)
+{
+	int i;
+
+	for (i = 0; i < RV_PER_TASK_MONITORS; i++)
+		;
+}
+#else
+#define rv_task_fork(p) do {} while (0)
+#endif
+
 /*
  * This creates a new process as a copy of the old one,
  * but does not actually start it yet.
@@ -2399,6 +2411,8 @@ static __latent_entropy struct task_struct *copy_process(
 	 */
 	copy_seccomp(p);
 
+	rv_task_fork(p);
+
 	rseq_fork(p, clone_flags);
 
 	/* Don't start children in a dying pid namespace */
kernel/trace/Kconfig +2 -0
index debbbb083286..b415690748bf 100644
@@ -1105,4 +1105,6 @@ config HIST_TRIGGERS_DEBUG
 
           If unsure, say N.
 
+source "kernel/trace/rv/Kconfig"
+
 endif # FTRACE
kernel/trace/Makefile +2 -0
index 0d261774d6f3..b2670fff6e94 100644
@@ -108,3 +108,5 @@ obj-$(CONFIG_RETHOOK) += rethook.o
 obj-$(CONFIG_TRACEPOINT_BENCHMARK) += trace_benchmark.o
 
 libftrace-y := ftrace.o
+
+obj-$(CONFIG_RV) += rv/
kernel/trace/rv/Kconfig new file +12 -0
new file mode 100644
index 000000000000..6d127cdb00dd
@@ -0,0 +1,12 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+menuconfig RV
+	bool "Runtime Verification"
+	depends on TRACING
+	help
+	  Enable the kernel runtime verification infrastructure. RV is a
+	  lightweight (yet rigorous) method that complements classical
+	  exhaustive verification techniques (such as model checking and
+	  theorem proving). RV works by analyzing the trace of the system's
+	  actual execution, comparing it against a formal specification of
+	  the system behavior.
kernel/trace/rv/Makefile new file +3 -0
new file mode 100644
index 000000000000..fd995379df67
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_RV) += rv.o
kernel/trace/rv/rv.c new file +421 -0
new file mode 100644
index 000000000000..43af7b13187e
@@ -0,0 +1,738 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * This is the online Runtime Verification (RV) interface.
+ *
+ * RV is a lightweight (yet rigorous) method that complements classical
+ * exhaustive verification techniques (such as model checking and
+ * theorem proving) with a more practical approach to complex systems.
+ *
+ * RV works by analyzing the trace of the system's actual execution,
+ * comparing it against a formal specification of the system behavior.
+ * RV can give precise information on the runtime behavior of the
+ * monitored system while enabling the reaction for unexpected
+ * events, avoiding, for example, the propagation of a failure on
+ * safety-critical systems.
+ *
+ * The development of this interface roots in the development of the
+ * paper:
+ *
+ * DE OLIVEIRA, Daniel Bristot; CUCINOTTA, Tommaso; DE OLIVEIRA, Romulo
+ * Silva. Efficient formal verification for the Linux kernel. In:
+ * International Conference on Software Engineering and Formal Methods.
+ * Springer, Cham, 2019. p. 315-332.
+ *
+ * And:
+ *
+ * DE OLIVEIRA, Daniel Bristot, et al. Automata-based formal analysis
+ * and verification of the real-time Linux kernel. PhD Thesis, 2020.
+ *
+ * == Runtime monitor interface ==
+ *
+ * A monitor is the central part of the runtime verification of a system.
+ *
+ * The monitor stands in between the formal specification of the desired
+ * (or undesired) behavior, and the trace of the actual system.
+ *
+ * In Linux terms, the runtime verification monitors are encapsulated
+ * inside the "RV monitor" abstraction. A RV monitor includes a reference
+ * model of the system, a set of instances of the monitor (per-cpu monitor,
+ * per-task monitor, and so on), and the helper functions that glue the
+ * monitor to the system via trace. Generally, a monitor includes some form
+ * of trace output as a reaction for event parsing and exceptions,
+ * as depicted bellow:
+ *
+ * Linux  +----- RV Monitor ----------------------------------+ Formal
+ *  Realm |                                                   |  Realm
+ *  +-------------------+     +----------------+     +-----------------+
+ *  |   Linux kernel    |     |     Monitor    |     |     Reference   |
+ *  |     Tracing       |  -> |   Instance(s)  | <-  |       Model     |
+ *  | (instrumentation) |     | (verification) |     | (specification) |
+ *  +-------------------+     +----------------+     +-----------------+
+ *         |                          |                       |
+ *         |                          V                       |
+ *         |                     +----------+                 |
+ *         |                     | Reaction |                 |
+ *         |                     +--+--+--+-+                 |
+ *         |                        |  |  |                   |
+ *         |                        |  |  +-> trace output ?  |
+ *         +------------------------|--|----------------------+
+ *                                  |  +----> panic ?
+ *                                  +-------> <user-specified>
+ *
+ * This file implements the interface for loading RV monitors, and
+ * to control the verification session.
+ *
+ * == Registering monitors ==
+ *
+ * The struct rv_monitor defines a set of callback functions to control
+ * a verification session. For instance, when a given monitor is enabled,
+ * the "start" callback function is called to hook the instrumentation
+ * functions to the kernel trace events. The "stop" function is called
+ * when disabling the verification session.
+ *
+ * A RV monitor is registered via:
+ *   int rv_register_monitor(struct rv_monitor *monitor);
+ * And unregistered via:
+ *   int rv_unregister_monitor(struct rv_monitor *monitor);
+ *
+ * These functions are exported to modules, enabling verification monitors
+ * to be dynamically loaded.
+ *
+ * == User interface ==
+ *
+ * The user interface resembles kernel tracing interface. It presents
+ * these files:
+ *
+ *  "available_monitors"
+ *    - List the available monitors, one per line.
+ *
+ *    For example:
+ *    [root@f32 rv]# cat available_monitors
+ *    wip
+ *    wwnr
+ *
+ *  "enabled_monitors"
+ *    - Lists the enabled monitors, one per line;
+ *    - Writing to it enables a given monitor;
+ *    - Writing a monitor name with a '-' prefix disables it;
+ *    - Truncating the file disables all enabled monitors.
+ *
+ *    For example:
+ *    [root@f32 rv]# cat enabled_monitors
+ *    [root@f32 rv]# echo wip > enabled_monitors
+ *    [root@f32 rv]# echo wwnr >> enabled_monitors
+ *    [root@f32 rv]# cat enabled_monitors
+ *    wip
+ *    wwnr
+ *    [root@f32 rv]# echo !wip >> enabled_monitors
+ *    [root@f32 rv]# cat enabled_monitors
+ *    wwnr
+ *    [root@f32 rv]# echo > enabled_monitors
+ *    [root@f32 rv]# cat enabled_monitors
+ *    [root@f32 rv]#
+ *
+ *    Note that more than one monitor can be enabled concurrently.
+ *
+ *  "monitoring_on"
+ *    - It is an on/off general switcher for monitoring. Note
+ *    that it does not disable enabled monitors, but stop the per-entity
+ *    monitors of monitoring the events received from the system.
+ *    It resambles the "tracing_on" switcher.
+ *
+ *  "monitors/"
+ *    Each monitor will have its one directory inside "monitors/". There
+ *    the monitor specific files will be presented.
+ *    The "monitors/" directory resambles the "events" directory on
+ *    tracefs.
+ *
+ *    For example:
+ *    [root@f32 rv]# cd monitors/wip/
+ *    [root@f32 wip]# ls
+ *    desc  enable
+ *    [root@f32 wip]# cat desc
+ *    auto-generated wakeup in preemptive monitor.
+ *    [root@f32 wip]# cat enable
+ *    0
+ *
+ * Copyright (C) 2019-2022 Daniel Bristot de Oliveira <[email protected]>
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/slab.h>
+#include <rv/rv.h>
+
+#include "rv.h"
+
+DEFINE_MUTEX(rv_interface_lock);
+struct rv_interface rv_root;
+
+struct dentry *get_monitors_root(void)
+{
+	return rv_root.monitors_dir;
+}
+
+/*
+ * Monitoring on global switcher!
+ */
+bool __read_mostly monitoring_on;
+
+/*
+ * Interface for the monitor register.
+ */
+LIST_HEAD(rv_monitors_list);
+
+static int task_monitor_count;
+static bool task_monitor_slots[RV_PER_TASK_MONITORS];
+
+int get_task_monitor_slot(void)
+{
+	int i;
+
+	lockdep_assert_held(&rv_interface_lock);
+
+	if (task_monitor_count == RV_PER_TASK_MONITORS)
+		return -EBUSY;
+
+	task_monitor_count++;
+
+	for (i = 0; i < RV_PER_TASK_MONITORS; i++) {
+		if (task_monitor_slots[i] == false) {
+			task_monitor_slots[i] = true;
+			return i;
+		}
+	}
+
+	WARN_ONCE(1, "RV task_monitor_cout and slots are out of sync\n");
+
+	return -EINVAL;
+}
+
+void put_task_monitor_slot(int slot)
+{
+	lockdep_assert_held(&rv_interface_lock);
+
+	if (slot < 0 || slot > RV_PER_TASK_MONITORS) {
+		WARN_ONCE(1, "RV releasing an invlid slot!: %d\n", slot);
+		return;
+	}
+
+	WARN_ONCE(!task_monitor_slots[slot], "RV releasing unsused task_monitor_slots: %d\n",
+		  slot);
+
+	task_monitor_count--;
+	task_monitor_slots[slot] = false;
+}
+
+/*
+ * This section collects the monitor/ files and folders.
+ */
+static ssize_t monitor_enable_read_data(struct file *filp,
+					char __user *user_buf,
+					size_t count, loff_t *ppos)
+{
+	struct rv_monitor_def *mdef = filp->private_data;
+	char buff[4];
+
+	memset(buff, 0, sizeof(buff));
+
+	mutex_lock(&rv_interface_lock);
+	sprintf(buff, "%x\n", mdef->monitor->enabled);
+	mutex_unlock(&rv_interface_lock);
+
+	return simple_read_from_buffer(user_buf, count, ppos,
+				       buff, strlen(buff)+1);
+}
+
+/*
+ * Disable a given runtime monitor.
+ */
+static int disable_monitor(struct rv_monitor_def *mdef)
+{
+	if (mdef->monitor->enabled) {
+		mdef->monitor->enabled = 0;
+		mdef->monitor->stop();
+	}
+
+	mdef->enabled = 0;
+	return 0;
+}
+
+/*
+ * Enable a given monitor.
+ */
+static int enable_monitor(struct rv_monitor_def *mdef)
+{
+	int retval;
+
+	/*
+	 * Reset all internal monitors before starting.
+	 */
+	mdef->monitor->reset();
+	if (!mdef->monitor->enabled) {
+		retval = mdef->monitor->start();
+		if (retval)
+			return retval;
+	}
+
+	mdef->monitor->enabled = 1;
+	mdef->enabled = 1;
+
+	return 0;
+}
+
+/*
+ * interface for enabling/disabling a monitor.
+ */
+static ssize_t monitor_enable_write_data(struct file *filp,
+					 const char __user *user_buf,
+					 size_t count, loff_t *ppos)
+{
+	struct rv_monitor_def *mdef = filp->private_data;
+	int retval;
+	u64 val;
+
+	retval = kstrtoull_from_user(user_buf, count, 10, &val);
+	if (retval)
+		return retval;
+
+	retval = count;
+
+	mutex_lock(&rv_interface_lock);
+
+	switch (val) {
+	case 0:
+		retval = disable_monitor(mdef);
+		break;
+	case 1:
+		retval = enable_monitor(mdef);
+		break;
+	default:
+		retval = -EINVAL;
+	}
+
+	mutex_unlock(&rv_interface_lock);
+
+	return retval;
+}
+
+static const struct file_operations interface_enable_fops = {
+	.open   = simple_open,
+	.llseek = no_llseek,
+	.write  = monitor_enable_write_data,
+	.read   = monitor_enable_read_data,
+};
+
+/*
+ * Interface to read the enable/disable status of a monitor.
+ */
+static ssize_t
+monitor_desc_read_data(struct file *filp, char __user *user_buf,
+		       size_t count, loff_t *ppos)
+{
+	struct rv_monitor_def *mdef = filp->private_data;
+	char buf[MAX_RV_MONITOR_NAME_SIZE];
+
+	memset(buf, 0, sizeof(buf));
+
+	mutex_lock(&rv_interface_lock);
+	sprintf(buf, "%s\n", mdef->monitor->description);
+	mutex_unlock(&rv_interface_lock);
+
+	return simple_read_from_buffer(user_buf, count, ppos,
+					buf, strlen(buf)+1);
+}
+
+static const struct file_operations interface_desc_fops = {
+	.open   = simple_open,
+	.llseek	= no_llseek,
+	.read	= monitor_desc_read_data,
+};
+
+/*
+ * During the registration of a monitor, this function creates
+ * the monitor dir, where the specific options of the monitor
+ * is exposed.
+ */
+static int create_monitor_dir(struct rv_monitor_def *mdef)
+{
+	struct dentry *root = get_monitors_root();
+	struct dentry *tmp;
+	const char *name = mdef->monitor->name;
+	int retval = 0;
+
+	mdef->root_d = rv_create_dir(name, root);
+
+	if (!mdef->root_d)
+		return -ENOMEM;
+
+	tmp = rv_create_file("enable", 0600,
+			     mdef->root_d, mdef,
+			     &interface_enable_fops);
+	if (!tmp) {
+		retval = -ENOMEM;
+		goto out_remove_root;
+	}
+
+	tmp = rv_create_file("desc", 0400,
+			      mdef->root_d, mdef,
+			      &interface_desc_fops);
+	if (!tmp) {
+		retval = -ENOMEM;
+		goto out_remove_root;
+	}
+
+	return retval;
+
+out_remove_root:
+	rv_remove(mdef->root_d);
+	return retval;
+}
+
+/*
+ * Available/Enable monitor shared seq functions.
+ */
+static int monitors_show(struct seq_file *m, void *p)
+{
+	struct rv_monitor_def *mon_def = p;
+
+	seq_printf(m, "%s\n", mon_def->monitor->name);
+	return 0;
+}
+
+/*
+ * Used by the seq file operations at the end of a read
+ * operation.
+ */
+static void monitors_stop(struct seq_file *m, void *p)
+{
+	mutex_unlock(&rv_interface_lock);
+}
+
+/*
+ * Available monitor seq functions:
+ */
+static void *available_monitors_start(struct seq_file *m, loff_t *pos)
+{
+	mutex_lock(&rv_interface_lock);
+	return seq_list_start(&rv_monitors_list, *pos);
+}
+
+static void *available_monitors_next(struct seq_file *m, void *p, loff_t *pos)
+{
+	return seq_list_next(p, &rv_monitors_list, pos);
+}
+
+/*
+ * Enable monitor seq functions:
+ */
+
+static void *enabled_monitors_next(struct seq_file *m, void *p, loff_t *pos)
+{
+	struct rv_monitor_def *m_def = p;
+
+	(*pos)++;
+
+	list_for_each_entry_continue(m_def, &rv_monitors_list, list) {
+		if (m_def->monitor->enabled)
+			return m_def;
+	}
+

kernel/trace/rv/rv.c line 431

Quoted 10 lines
kernel/trace/rv/rv.c new file +10 -0
+	return NULL;
+}
+
+static void *enabled_monitors_start(struct seq_file *m, loff_t *pos)
+{
+	struct rv_monitor_def *m_def;
+	loff_t l;
+
+	mutex_lock(&rv_interface_lock);
+	m_def = list_entry(&rv_monitors_list, struct rv_monitor_def, list);

I realized this m_def is not real but vain. Is it possible the loop is skiped and just return m_def that is not valid.

Quoted 375 lines
kernel/trace/rv/rv.c new file +306 -0
+	for (l = 0; l <= *pos; ) {
+		m_def = enabled_monitors_next(m, m_def, &l);
+		if (!m_def)
+			break;
+	}
+
+	return m_def;
+}
+
+/*
+ * available/enabled monitors seq definition.
+ */
+static const struct seq_operations available_monitors_seq_ops = {
+	.start	= available_monitors_start,
+	.next	= available_monitors_next,
+	.stop	= monitors_stop,
+	.show	= monitors_show
+};
+
+static const struct seq_operations enabled_monitors_seq_ops = {
+	.start  = enabled_monitors_start,
+	.next   = enabled_monitors_next,
+	.stop   = monitors_stop,
+	.show   = monitors_show
+};
+
+/*
+ * available_monitors interface.
+ */
+static int available_monitors_open(struct inode *inode, struct file *file)
+{
+	return seq_open(file, &available_monitors_seq_ops);
+};
+
+static const struct file_operations available_monitors_ops = {
+	.open    = available_monitors_open,
+	.read    = seq_read,
+	.llseek  = seq_lseek,
+	.release = seq_release
+};
+
+/*
+ * enabled_monitors interface
+ */
+static void disable_all_monitors(void)
+{
+	struct rv_monitor_def *mdef;
+
+	list_for_each_entry(mdef, &rv_monitors_list, list)
+		disable_monitor(mdef);
+}
+
+static int enabled_monitors_open(struct inode *inode, struct file *file)
+{
+	if ((file->f_mode & FMODE_WRITE) && (file->f_flags & O_TRUNC))
+		disable_all_monitors();
+
+	return seq_open(file, &enabled_monitors_seq_ops);
+};
+
+static ssize_t
+enabled_monitors_write(struct file *filp, const char __user *user_buf,
+		      size_t count, loff_t *ppos)
+{
+	char buff[MAX_RV_MONITOR_NAME_SIZE+1];
+	struct rv_monitor_def *mdef;
+	int retval = -EINVAL;
+	bool enable = true;
+	char *ptr = buff;
+	int len;
+
+	if (count < 1 || count > MAX_RV_MONITOR_NAME_SIZE+1)
+		return -EINVAL;
+
+	memset(buff, 0, sizeof(buff));
+
+	retval = simple_write_to_buffer(buff, sizeof(buff)-1, ppos, user_buf,
+					count);
+	if (!retval)
+		return -EFAULT;
+
+	if (buff[0] == '!') {
+		enable = false;
+		ptr++;
+	}
+
+	len = strlen(ptr);
+	if (!len)
+		return count;
+	/*
+	 * remove \n
+	 */
+	ptr[len-1] = '\0';
+
+	mutex_lock(&rv_interface_lock);
+
+	retval = -EINVAL;
+
+	list_for_each_entry(mdef, &rv_monitors_list, list) {
+		if (strcmp(ptr, mdef->monitor->name) == 0) {
+			/*
+			 * Monitor found!
+			 */
+			if (enable)
+				retval = enable_monitor(mdef);
+			else
+				retval = disable_monitor(mdef);
+
+			if (retval)
+				goto out;
+
+			/*
+			 * Success!
+			 */
+			retval = count;
+			break;
+		}
+	}
+
+out:
+	mutex_unlock(&rv_interface_lock);
+	return retval;
+}
+
+static const struct file_operations enabled_monitors_ops = {
+	.open		= enabled_monitors_open,
+	.read		= seq_read,
+	.write		= enabled_monitors_write,
+	.llseek		= seq_lseek,
+	.release	= seq_release,
+};
+
+/*
+ * monitoring_on general switcher
+ */
+static ssize_t monitoring_on_read_data(struct file *filp,
+					char __user *user_buf,
+					size_t count, loff_t *ppos)
+{
+	char buff[4];
+
+	memset(buff, 0, sizeof(buff));
+
+	mutex_lock(&rv_interface_lock);
+	sprintf(buff, "%d\n", monitoring_on);
+	mutex_unlock(&rv_interface_lock);
+
+	return simple_read_from_buffer(user_buf, count, ppos,
+				       buff, strlen(buff)+1);
+}
+
+static void turn_monitoring_off(void)
+{
+	monitoring_on = false;
+}
+
+static void turn_monitoring_on(void)
+{
+	reset_all_monitors();
+	monitoring_on = true;
+}
+
+static ssize_t monitoring_on_write_data(struct file *filp,
+					 const char __user *user_buf,
+					 size_t count, loff_t *ppos)
+{
+	int retval;
+	u64 val;
+
+	retval = kstrtoull_from_user(user_buf, count, 10, &val);
+	if (retval)
+		return retval;
+
+	retval = count;
+
+	mutex_lock(&rv_interface_lock);
+
+	switch (val) {
+	case 0:
+		turn_monitoring_off();
+		break;
+	case 1:
+		turn_monitoring_on();
+		break;
+	default:
+		retval = -EINVAL;
+	}
+
+	mutex_unlock(&rv_interface_lock);
+
+	return retval;
+}
+
+static const struct file_operations monitoring_on_fops = {
+	.open   = simple_open,
+	.llseek = no_llseek,
+	.write  = monitoring_on_write_data,
+	.read   = monitoring_on_read_data,
+};
+
+/*
+ * Monitor API.
+ */
+static void destroy_monitor_dir(struct rv_monitor_def *mdef)
+{
+	rv_remove(mdef->root_d);
+}
+
+/**
+ * rv_register_monitor - register a rv monitor.
+ * @monitor:    The rv_monitor to be registered.
+ *
+ * Returns 0 if successful, error otherwise.
+ */
+int rv_register_monitor(struct rv_monitor *monitor)
+{
+	struct rv_monitor_def *r;
+	int retval = 0;
+
+	if (strlen(monitor->name) >= MAX_RV_MONITOR_NAME_SIZE) {
+		pr_info("Monitor %s has a name longer than %d\n",
+			monitor->name, MAX_RV_MONITOR_NAME_SIZE);
+		return -1;
+	}
+
+	mutex_lock(&rv_interface_lock);
+
+	list_for_each_entry(r, &rv_monitors_list, list) {
+		if (strcmp(monitor->name, r->monitor->name) == 0) {
+			pr_info("Monitor %s is already registered\n",
+				monitor->name);
+			retval = -1;
+			goto out_unlock;
+		}
+	}
+
+	r = kzalloc(sizeof(struct rv_monitor_def), GFP_KERNEL);
+	if (!r) {
+		retval = -ENOMEM;
+		goto out_unlock;
+	}
+
+	r->monitor = monitor;
+
+	create_monitor_dir(r);
+
+	list_add_tail(&r->list, &rv_monitors_list);
+
+out_unlock:
+	mutex_unlock(&rv_interface_lock);
+	return retval;
+}
+
+/**
+ * rv_unregister_monitor - unregister a rv monitor.
+ * @monitor:    The rv_monitor to be unregistered.
+ *
+ * Returns 0 if successful, error otherwise.
+ */
+int rv_unregister_monitor(struct rv_monitor *monitor)
+{
+	struct rv_monitor_def *ptr, *next;
+
+	mutex_lock(&rv_interface_lock);
+
+	list_for_each_entry_safe(ptr, next, &rv_monitors_list, list) {
+		if (strcmp(monitor->name, ptr->monitor->name) == 0) {
+			list_del(&ptr->list);
+			destroy_monitor_dir(ptr);
+		}
+	}
+
+	mutex_unlock(&rv_interface_lock);
+	return 0;
+}
+
+void reset_all_monitors(void)
+{
+	struct rv_monitor_def *mdef;
+
+	/*
+	 * Reset all monitors before re-enabling monitoring.
+	 */
+	list_for_each_entry(mdef, &rv_monitors_list, list) {
+		if (mdef->monitor->enabled)
+			mdef->monitor->reset();
+	}
+
+}
+
+int __init rv_init_interface(void)
+{
+	rv_root.root_dir = rv_create_dir("rv", NULL);
+	rv_root.monitors_dir = rv_create_dir("monitors", rv_root.root_dir);
+
+	rv_create_file("available_monitors", 0400, rv_root.root_dir, NULL,
+		       &available_monitors_ops);
+	rv_create_file("enabled_monitors", 0600, rv_root.root_dir, NULL,
+		       &enabled_monitors_ops);
+	rv_create_file("monitoring_on", 0600, rv_root.root_dir, NULL,
+		       &monitoring_on_fops);
+
+	monitoring_on = true;
+
+	return 0;
+}
kernel/trace/rv/rv.h new file +34 -0
new file mode 100644
index 000000000000..0796867a7b1e
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <linux/mutex.h>
+
+struct rv_interface {
+	struct dentry *root_dir;
+	struct dentry *monitors_dir;
+};
+
+#include "../trace.h"
+#include <linux/tracefs.h>
+#include <linux/rv.h>
+
+#define rv_create_dir			tracefs_create_dir
+#define rv_create_file			tracefs_create_file
+#define rv_remove			tracefs_remove
+
+#define MAX_RV_MONITOR_NAME_SIZE	32
+
+extern struct mutex rv_interface_lock;
+
+struct rv_monitor_def {
+	struct list_head list;
+	struct rv_monitor *monitor;
+	struct dentry *root_d;
+	bool enabled;
+	bool task_monitor;
+};
+
+extern bool monitoring_on;
+struct dentry *get_monitors_root(void);
+void reset_all_monitors(void);
+int init_rv_monitors(struct dentry *root_dir);
+int get_task_monitor_slot(void);
+void put_task_monitor_slot(int slot);
kernel/trace/trace.c +4 -0
index 2c95992e2c71..60e357c3120b 100644
@@ -9774,6 +9774,10 @@ static __init int tracer_init_tracefs(void)
 		tracer_init_tracefs_work_func(NULL);
 	}
 
+#ifdef CONFIG_RV
+	rv_init_interface();
+#endif
+
 	return 0;
 }
 
kernel/trace/trace.h +2 -0
index ff816fb41e48..becc03c0a45e 100644
@@ -2005,4 +2005,6 @@ struct trace_min_max_param {
 
 extern const struct file_operations trace_min_max_fops;
 
+extern int rv_init_interface(void);
+
 #endif /* _LINUX_KERNEL_TRACE_H */
Signature · 1 line
2.35.1
#7 lore raw

Re: [PATCH V4 01/20] rv: Add Runtime Verification (RV) interface

Quoted 10 lines · Tao Zhou

include/linux/rv.h line 14

Quoted 4 lines
include/linux/rv.h new file +4 -0
+struct rv_monitor {
+	const char		*name;
+	const char		*description;
+	bool			enabled;

Can the 'bool enabled;' be put at the end like the definition of structure rv_monitor_def. If '8+8+sizeof(bool)+8+8+8' not the same as '8+8+8+8+8+sizeof(bool)', I mean is it possible that after the end of stucture there is a int or char not require to align to 8 as an example from my nonsense.

That will make no difference at all. C doesn't allow other variables to "fill in the hole" at the end of the structure like that. For example, one could legitimately do 'memset(&rvm, sizeof(rvm))', and that would wipe out those other variables as well.

#8 lore raw

Re: [PATCH V4 01/20] rv: Add Runtime Verification (RV) interface

Quoted 12 lines · Matthew Wilcox
Quoted 10 lines · Tao Zhou

include/linux/rv.h line 14

Quoted 4 lines
include/linux/rv.h new file +4 -0
+struct rv_monitor {
+	const char		*name;
+	const char		*description;
+	bool			enabled;

Can the 'bool enabled;' be put at the end like the definition of structure rv_monitor_def. If '8+8+sizeof(bool)+8+8+8' not the same as '8+8+8+8+8+sizeof(bool)', I mean is it possible that after the end of stucture there is a int or char not require to align to 8 as an example from my nonsense.

Quoted 4 lines

That will make no difference at all. C doesn't allow other variables to "fill in the hole" at the end of the structure like that. For example, one could legitimately do 'memset(&rvm, sizeof(rvm))', and that would wipe out those other variables as well.

I mean if it is possible that if @enabled placed at the end of the structure rv_monitor will save some bytes.
If @enabled place in between, the next function pointer which is 8 bytes will align to be in x8 address and the size of structure rv_monitor is larger than been placed at the end of the structure.
Or the compiler can do magic that I can not guess.

Sorry for my late reply. I am not sure about this. But your reply is not about what I mean. You say that the size of structure is the same(I doute about this). But what my concert is that the other data next to the structure rv_monitor how to align if placing the @enabled at the end.

Place in between, bytes:
8+8+8(padd)+8+8+8=48

Place at the end, bytes:
8+8+8+8+8+sizeof(bool)=?

? is small than 48 and the data next to the data structure rv_monitor can use the saved byte which is the result of placing @enabled at the end of structure to place the data.

#9 lore raw

Re: [PATCH V4 01/20] rv: Add Runtime Verification (RV) interface

Quoted 34 lines · Tao Zhou
Quoted 12 lines · Matthew Wilcox
Quoted 10 lines · Tao Zhou

include/linux/rv.h line 14

Quoted 4 lines
include/linux/rv.h new file +4 -0
+struct rv_monitor {
+	const char		*name;
+	const char		*description;
+	bool			enabled;

Can the 'bool enabled;' be put at the end like the definition of structure rv_monitor_def. If '8+8+sizeof(bool)+8+8+8' not the same as '8+8+8+8+8+sizeof(bool)', I mean is it possible that after the end of stucture there is a int or char not require to align to 8 as an example from my nonsense.

Quoted 4 lines

That will make no difference at all. C doesn't allow other variables to "fill in the hole" at the end of the structure like that. For example, one could legitimately do 'memset(&rvm, sizeof(rvm))', and that would wipe out those other variables as well.

I mean if it is possible that if @enabled placed at the end of the structure rv_monitor will save some bytes.
If @enabled place in between, the next function pointer which is 8 bytes will align to be in x8 address and the size of structure rv_monitor is larger than been placed at the end of the structure.
Or the compiler can do magic that I can not guess.

Sorry for my late reply. I am not sure about this. But your reply is not about what I mean. You say that the size of structure is the same(I doute about this). But what my concert is that the other data next to the structure rv_monitor how to align if placing the @enabled at the end.

Place in between, bytes:
8+8+8(padd)+8+8+8=48

Quoted 6 lines

Place at the end, bytes:
8+8+8+8+8+sizeof(bool)=?

? is small than 48 and the data next to the data structure rv_monitor can use the saved byte which is the result of placing @enabled at the end of structure to place the data.

You don't need to take my word for it. You can try it yourself.

#10 lore raw

Re: [PATCH V4 01/20] rv: Add Runtime Verification (RV) interface

In reply to Tao Zhou

Hey Tao!

Quoted 9 lines · Tao Zhou

kernel/trace/rv/rv.c line 431

Quoted 7 lines
kernel/trace/rv/rv.c new file +7 -0
+static void *enabled_monitors_start(struct seq_file *m, loff_t *pos)
+{
+	struct rv_monitor_def *m_def;
+	loff_t l;
+
+	mutex_lock(&rv_interface_lock);
+	m_def = list_entry(&rv_monitors_list, struct rv_monitor_def, list);

I realized this m_def is not real but vain. Is it possible the loop is skiped and just return m_def that is not valid.

that is empty... not a problem.

I am not seeing (the possible) problem here. Could you simulate/reproduce the problem?

Btw, this code is "inspired" (iow stolen) from trace_events.c.

Am I missing something? steve?

Quoted 8 lines

kernel/trace/rv/rv.c line 440

Quoted 8 lines
kernel/trace/rv/rv.c new file +8 -0
+	for (l = 0; l <= *pos; ) {
+		m_def = enabled_monitors_next(m, m_def, &l);
+		if (!m_def)
+			break;
+	}
+
+	return m_def;
+}

-- Daniel

#11 lore raw

Re: [PATCH V4 01/20] rv: Add Runtime Verification (RV) interface

Quoted 13 lines · Daniel Bristot de Oliveira

Hey Tao!

Quoted 9 lines · Tao Zhou

kernel/trace/rv/rv.c line 431

Quoted 7 lines
kernel/trace/rv/rv.c new file +7 -0
+static void *enabled_monitors_start(struct seq_file *m, loff_t *pos)
+{
+	struct rv_monitor_def *m_def;
+	loff_t l;
+
+	mutex_lock(&rv_interface_lock);
+	m_def = list_entry(&rv_monitors_list, struct rv_monitor_def, list);

I realized this m_def is not real but vain. Is it possible the loop is skiped and just return m_def that is not valid.

Quoted 3 lines

that is empty... not a problem.

I am not seeing (the possible) problem here. Could you simulate/reproduce the problem?

The @*pos of enable_monitors_start() can not be -1 or other negative value. And I checked that the *pos is 0(right?). That is safe. Sorry for not being that ture and maybe this is a notice here. Because if it is a negative value, the returned m_def is a point to a data place 16 bytes before &rv_monitors_list. That is a not ture rv_monitors_list stucture data. But it is not possiable now. Maybe "inspired" from your question. Look it more, I image this simulation. If the monitor(and all is enabled) is more enough to let the *pos to increase to -1. And the returned m_def is last monitor that returned from enable_monitors_start(). The enable_monitors_next() check from the last monitor and return NULL. Only show the last monitor. This will not really happen I think.
But I am not focus enough to the seq file code or others now, so this may be more possible to be not right. Late reply continued from me..

Thanks,
Tao

maintainer
#12 lore raw

Re: [PATCH V4 01/20] rv: Add Runtime Verification (RV) interface

Quoted 4 lines · On Sun, 10 Jul 2022 23:11:43 +0800 Tao Zhou <[email protected]> wrote:

The @*pos of enable_monitors_start() can not be -1 or other negative value. And I checked that the *pos is 0(right?). That is safe. Sorry for not being that ture and maybe this is a notice here. Because if it is a negative value, the returned m_def is a point to a data place 16 bytes before &rv_monitors_list.

Quoted 8 lines

That is a not ture rv_monitors_list stucture data. But it is not possiable now. Maybe "inspired" from your question. Look it more, I image this simulation. If the monitor(and all is enabled) is more enough to let the *pos to increase to -1. And the returned m_def is last monitor that returned from enable_monitors_start(). The enable_monitors_next() check from the last monitor and return NULL. Only show the last monitor. This will not really happen I think.
But I am not focus enough to the seq file code or others now, so this may be more possible to be not right. Late reply continued from me..

So basically you are saying we should have:

kernel/trace/rv/rv.c line 430

Quoted 6 lines
kernel/trace/rv/rv.c new file +6 -0
+static void *enabled_monitors_start(struct seq_file *m, loff_t *pos)
+{
+       struct rv_monitor_def *m_def;
+       loff_t l;
+
+       mutex_lock(&rv_interface_lock);
	if (list_empty(&rv_monitors_list->list))
		return NULL;

?

Probably safer to have that.

-- Steve

Quoted 10 lines
kernel/trace/rv/rv.c new file +10 -0
+       m_def = list_entry(&rv_monitors_list, struct rv_monitor_def, list);
+
+       for (l = 0; l <= *pos; ) {
+               m_def = enabled_monitors_next(m, m_def, &l);
+               if (!m_def)
+                       break;
+       }
+
+       return m_def;
+}
#13 lore raw

Re: [PATCH V4 01/20] rv: Add Runtime Verification (RV) interface

Quoted 28 lines · Steven Rostedt
Quoted 4 lines · On Sun, 10 Jul 2022 23:11:43 +0800 Tao Zhou <[email protected]> wrote:

The @*pos of enable_monitors_start() can not be -1 or other negative value. And I checked that the *pos is 0(right?). That is safe. Sorry for not being that ture and maybe this is a notice here. Because if it is a negative value, the returned m_def is a point to a data place 16 bytes before &rv_monitors_list.

Quoted 8 lines

That is a not ture rv_monitors_list stucture data. But it is not possiable now. Maybe "inspired" from your question. Look it more, I image this simulation. If the monitor(and all is enabled) is more enough to let the *pos to increase to -1. And the returned m_def is last monitor that returned from enable_monitors_start(). The enable_monitors_next() check from the last monitor and return NULL. Only show the last monitor. This will not really happen I think.
But I am not focus enough to the seq file code or others now, so this may be more possible to be not right. Late reply continued from me..

So basically you are saying we should have:

kernel/trace/rv/rv.c line 430

Quoted 6 lines
kernel/trace/rv/rv.c new file +6 -0
+static void *enabled_monitors_start(struct seq_file *m, loff_t *pos)
+{
+       struct rv_monitor_def *m_def;
+       loff_t l;
+
+       mutex_lock(&rv_interface_lock);
	if (list_empty(&rv_monitors_list->list))
		return NULL;
Quoted 1 line

?

From the function itself, this looks safer. Thanks.

Quoted 15 lines

Probably safer to have that.

-- Steve

Quoted 10 lines
kernel/trace/rv/rv.c new file +10 -0
+       m_def = list_entry(&rv_monitors_list, struct rv_monitor_def, list);
+
+       for (l = 0; l <= *pos; ) {
+               m_def = enabled_monitors_next(m, m_def, &l);
+               if (!m_def)
+                       break;
+       }
+
+       return m_def;
+}
#14 lore raw

PATCH v4 2/20 Add runtime reactors interface

In reply to Daniel Bristot de Oliveira

A runtime monitor can cause a reaction to the detection of an exception on the model's execution. By default, the monitors have tracing reactions, printing the monitor output via tracepoints. But other reactions can be added (on-demand) via this interface.

The user interface resembles the kernel tracing interface and presents these files:

"available_reactors"
- Reading shows the available reactors, one per line.

For example:
[root@f32 rv]# cat available_reactors
nop
panic
printk

"reacting_on"
- It is an on/off general switch for reactors, disabling all reactions.

"monitors/MONITOR/reactors"
- List available reactors, with the select reaction for the given MONITOR inside []. The default one is the nop (no operation) reactor.
- Writing the name of a reactor enables it to the given MONITOR.

For example:
[root@f32 rv]# cat monitors/wip/reactors [nop]
panic
printk
[root@f32 rv]# echo panic > monitors/wip/reactors [root@f32 rv]# cat monitors/wip/reactors nop
[panic]
printk

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 include/linux/rv.h            |  13 +
 kernel/trace/rv/Kconfig       |  14 +
 kernel/trace/rv/Makefile      |   1 +
 kernel/trace/rv/rv.c          |  18 +-
 kernel/trace/rv/rv.h          |  20 ++
 kernel/trace/rv/rv_reactors.c | 476 ++++++++++++++++++++++++++++++++++
 6 files changed, 540 insertions(+), 2 deletions(-)
 create mode 100644 kernel/trace/rv/rv_reactors.c
include/linux/rv.h +13 -0
index 205e65f57637..1e48c6bb74bf 100644
@@ -8,6 +8,13 @@
  */
 #ifndef _LINUX_RV_H
 #define _LINUX_RV_H
+
+struct rv_reactor {
+	char			*name;
+	char			*description;
+	void			(*react)(char *msg);
+};
+
 struct rv_monitor {
 	const char		*name;
 	const char		*description;
@@ -15,9 +22,15 @@ struct rv_monitor {
 	int			(*start)(void);
 	void			(*stop)(void);
 	void			(*reset)(void);
+	void			(*react)(char *msg);
+
 };
 
 extern bool monitoring_on;
 int rv_unregister_monitor(struct rv_monitor *monitor);
 int rv_register_monitor(struct rv_monitor *monitor);
+
+extern bool reacting_on;
+int rv_unregister_reactor(struct rv_reactor *reactor);
+int rv_register_reactor(struct rv_reactor *reactor);
 #endif /* _LINUX_RV_H */
kernel/trace/rv/Kconfig +14 -0
index 6d127cdb00dd..560408fec0c8 100644
@@ -10,3 +10,17 @@ menuconfig RV
 	  theorem proving). RV works by analyzing the trace of the system's
 	  actual execution, comparing it against a formal specification of
 	  the system behavior.
+
+if RV
+
+config RV_REACTORS
+	bool "Runtime verification reactors"
+	default y if RV
+	help
+	  Enables the online runtime verification reactors. A runtime
+	  monitor can cause a reaction to the detection of an exception
+	  on the model's execution. By default, the monitors have
+	  tracing reactions, printing the monitor output via tracepoints,
+	  but other reactions can be added (on-demand) via this interface.
+
+endif # RV
kernel/trace/rv/Makefile +1 -0
index fd995379df67..8944274d9b41 100644
@@ -1,3 +1,4 @@
 # SPDX-License-Identifier: GPL-2.0
 
 obj-$(CONFIG_RV) += rv.o
+obj-$(CONFIG_RV_REACTORS) += rv_reactors.o
kernel/trace/rv/rv.c +16 -2
index 43af7b13187e..7576d492a974 100644
@@ -362,8 +362,13 @@ static int create_monitor_dir(struct rv_monitor_def *mdef)
 		retval = -ENOMEM;
 		goto out_remove_root;
 	}
+#ifdef CONFIG_RV_REACTORS
+	retval = reactor_create_monitor_files(mdef);
+	if (retval)
+		goto out_remove_root;
+#endif
 
-	return retval;
+	return 0;
 
 out_remove_root:
 	rv_remove(mdef->root_d);
@@ -674,7 +679,11 @@ int rv_register_monitor(struct rv_monitor *monitor)
 
 	r->monitor = monitor;
 
-	create_monitor_dir(r);
+	retval = create_monitor_dir(r);
+	if (retval) {
+		kfree(r);
+		goto out_unlock;
+	}
 
 	list_add_tail(&r->list, &rv_monitors_list);
 
@@ -732,6 +741,11 @@ int __init rv_init_interface(void)
 	rv_create_file("monitoring_on", 0600, rv_root.root_dir, NULL,
 		       &monitoring_on_fops);
 
+#ifdef CONFIG_RV_REACTORS
+	init_rv_reactors(rv_root.root_dir);
+	reacting_on = true;
+#endif
+
 	monitoring_on = true;
 
 	return 0;
kernel/trace/rv/rv.h +20 -0
index 0796867a7b1e..6d43f52d72a9 100644
@@ -15,14 +15,28 @@ struct rv_interface {
 #define rv_remove			tracefs_remove
 
 #define MAX_RV_MONITOR_NAME_SIZE	32
+#define MAX_RV_REACTOR_NAME_SIZE	32
 
 extern struct mutex rv_interface_lock;
 
+#ifdef CONFIG_RV_REACTORS
+struct rv_reactor_def {
+	struct list_head list;
+	struct rv_reactor *reactor;
+	/* protected by the monitor interface lock */
+	int counter;
+};
+#endif
+
 struct rv_monitor_def {
 	struct list_head list;
 	struct rv_monitor *monitor;
+#ifdef CONFIG_RV_REACTORS
+	struct rv_reactor_def *rdef;
+#endif
 	struct dentry *root_d;
 	bool enabled;
+	bool reacting;
 	bool task_monitor;
 };
 
@@ -32,3 +46,9 @@ void reset_all_monitors(void);
 int init_rv_monitors(struct dentry *root_dir);
 int get_task_monitor_slot(void);
 void put_task_monitor_slot(int slot);
+
+#ifdef CONFIG_RV_REACTORS
+extern bool reacting_on;
+int reactor_create_monitor_files(struct rv_monitor_def *mdef);
+int init_rv_reactors(struct dentry *root_dir);
+#endif
kernel/trace/rv/rv_reactors.c new file +476 -0

476 lines not on this page · raw message

Signature · 1 line
2.35.1
maintainer
#15 lore raw

Re: [PATCH V4 02/20] rv: Add runtime reactors interface

Quoted 39 lines · On Thu, 16 Jun 2022 10:44:44 +0200 Daniel Bristot de Oliveira…
include/linux/rv.h +13 -0
index 205e65f57637..1e48c6bb74bf 100644
@@ -8,6 +8,13 @@
  */
 #ifndef _LINUX_RV_H
 #define _LINUX_RV_H
+
+struct rv_reactor {
+	char			*name;
+	char			*description;
+	void			(*react)(char *msg);
+};
+
 struct rv_monitor {
 	const char		*name;
 	const char		*description;
@@ -15,9 +22,15 @@ struct rv_monitor {
 	int			(*start)(void);
 	void			(*stop)(void);
 	void			(*reset)(void);
+	void			(*react)(char *msg);
+
 };
 
 extern bool monitoring_on;
 int rv_unregister_monitor(struct rv_monitor *monitor);
 int rv_register_monitor(struct rv_monitor *monitor);
+
+extern bool reacting_on;
+int rv_unregister_reactor(struct rv_reactor *reactor);
+int rv_register_reactor(struct rv_reactor *reactor);
 #endif /* _LINUX_RV_H */
kernel/trace/rv/Kconfig +0 -0
index 6d127cdb00dd..560408fec0c8 100644

kernel/trace/rv/Kconfig menuconfig RV · line 14

Quoted 5 lines
kernel/trace/rv/Kconfig +2 -0
@@ -10,3 +10,17 @@ menuconfig RV
 	  theorem proving). RV works by analyzing the trace of the system's
 	  actual execution, comparing it against a formal specification of
 	  the system behavior.
+
+if RV

Remove the above.

kernel/trace/rv/Kconfig menuconfig RV · line 18

Quoted 4 lines
kernel/trace/rv/Kconfig +4 -0
+
+config RV_REACTORS
+	bool "Runtime verification reactors"
+	default y if RV

default y
DEPENDS ON RV

Quoted 22 lines
kernel/trace/rv/Kconfig +8 -0
+	help
+	  Enables the online runtime verification reactors. A runtime
+	  monitor can cause a reaction to the detection of an exception
+	  on the model's execution. By default, the monitors have
+	  tracing reactions, printing the monitor output via tracepoints,
+	  but other reactions can be added (on-demand) via this interface.
+
+endif # RV
kernel/trace/rv/Makefile +1 -0
index fd995379df67..8944274d9b41 100644
@@ -1,3 +1,4 @@
 # SPDX-License-Identifier: GPL-2.0
 
 obj-$(CONFIG_RV) += rv.o
+obj-$(CONFIG_RV_REACTORS) += rv_reactors.o
kernel/trace/rv/rv.c +0 -0
index 43af7b13187e..7576d492a974 100644

kernel/trace/rv/rv.c static int create_monitor_dir(struct rv_monitor_def *mdef) · line 365

Quoted 4 lines
kernel/trace/rv/rv.c +1 -0
@@ -362,8 +362,13 @@ static int create_monitor_dir(struct rv_monitor_def *mdef)
 		retval = -ENOMEM;
 		goto out_remove_root;
 	}
+#ifdef CONFIG_RV_REACTORS

Could you move the ifdefs to a header or above, and then just make theses functions into nops when not defined. Keeps the actual code cleaner.

Quoted 24 lines
kernel/trace/rv/rv.c +10 -2
+	retval = reactor_create_monitor_files(mdef);
+	if (retval)
+		goto out_remove_root;
+#endif
 
-	return retval;
+	return 0;
 
 out_remove_root:
 	rv_remove(mdef->root_d);
@@ -674,7 +679,11 @@ int rv_register_monitor(struct rv_monitor *monitor)
 
 	r->monitor = monitor;
 
-	create_monitor_dir(r);
+	retval = create_monitor_dir(r);
+	if (retval) {
+		kfree(r);
+		goto out_unlock;
+	}
 
 	list_add_tail(&r->list, &rv_monitors_list);
 

kernel/trace/rv/rv.c int __init rv_init_interface(void) · line 746

Quoted 6 lines
kernel/trace/rv/rv.c +3 -0
@@ -732,6 +741,11 @@ int __init rv_init_interface(void)
 	rv_create_file("monitoring_on", 0600, rv_root.root_dir, NULL,
 		       &monitoring_on_fops);
 
+#ifdef CONFIG_RV_REACTORS
+	init_rv_reactors(rv_root.root_dir);
+	reacting_on = true;

Same here.

Could have init_rv_reactors() return a value and then:

	if (init_rv_reactors(...))
		reacting_on = true;
Quoted 235 lines
kernel/trace/rv/rv.c +2 -0
+#endif
+
 	monitoring_on = true;
 
 	return 0;
kernel/trace/rv/rv.h +20 -0
index 0796867a7b1e..6d43f52d72a9 100644
@@ -15,14 +15,28 @@ struct rv_interface {
 #define rv_remove			tracefs_remove
 
 #define MAX_RV_MONITOR_NAME_SIZE	32
+#define MAX_RV_REACTOR_NAME_SIZE	32
 
 extern struct mutex rv_interface_lock;
 
+#ifdef CONFIG_RV_REACTORS
+struct rv_reactor_def {
+	struct list_head list;
+	struct rv_reactor *reactor;
+	/* protected by the monitor interface lock */
+	int counter;
+};
+#endif
+
 struct rv_monitor_def {
 	struct list_head list;
 	struct rv_monitor *monitor;
+#ifdef CONFIG_RV_REACTORS
+	struct rv_reactor_def *rdef;
+#endif
 	struct dentry *root_d;
 	bool enabled;
+	bool reacting;
 	bool task_monitor;
 };
 
@@ -32,3 +46,9 @@ void reset_all_monitors(void);
 int init_rv_monitors(struct dentry *root_dir);
 int get_task_monitor_slot(void);
 void put_task_monitor_slot(int slot);
+
+#ifdef CONFIG_RV_REACTORS
+extern bool reacting_on;
+int reactor_create_monitor_files(struct rv_monitor_def *mdef);
+int init_rv_reactors(struct dentry *root_dir);
+#endif
kernel/trace/rv/rv_reactors.c new file +181 -0
new file mode 100644
index 000000000000..bfe54d6996cc
@@ -0,0 +1,476 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Runtime reactor interface.
+ *
+ * A runtime monitor can cause a reaction to the detection of an
+ * exception on the model's execution. By default, the monitors have
+ * tracing reactions, printing the monitor output via tracepoints.
+ * But other reactions can be added (on-demand) via this interface.
+ *
+ * == Registering reactors ==
+ *
+ * The struct rv_reactor defines a callback function to be executed
+ * in case of a model exception happens. The callback function
+ * receives a message to be (optionally) printed before executing
+ * the reaction.
+ *
+ * A RV reactor is registered via:
+ *   int rv_register_reactor(struct rv_reactor *reactor)
+ * And unregistered via:
+ *   int rv_unregister_reactor(struct rv_reactor *reactor)
+ *
+ * These functions are exported to modules, enabling reactors to be
+ * dynamically loaded.
+ *
+ * == User interface ==
+ *
+ * The user interface resembles the kernel tracing interface and
+ * presents these files:
+ *
+ *  "available_reactors"
+ *    - List the available reactors, one per line.
+ *
+ *    For example:
+ *    [root@f32 rv]# cat available_reactors
+ *    nop
+ *    panic
+ *    printk
+ *
+ *  "reacting_on"
+ *    - It is an on/off general switch for reactors, disabling
+ *    all reactions.
+ *
+ *  "monitors/MONITOR/reactors"
+ *    - List available reactors, with the select reaction for the given
+ *    MONITOR inside []. The defaul one is the nop (no operation)
+ *    reactor.
+ *    - Writing the name of an reactor enables it to the given
+ *    MONITOR.
+ *
+ *    For example:
+ *    [root@f32 rv]# cat monitors/wip/reactors
+ *    [nop]
+ *    panic
+ *    printk
+ *    [root@f32 rv]# echo panic > monitors/wip/reactors
+ *    [root@f32 rv]# cat monitors/wip/reactors
+ *    nop
+ *    [panic]
+ *    printk
+ *
+ * Copyright (C) 2019-2022 Daniel Bristot de Oliveira <[email protected]>
+ */
+
+#include <linux/slab.h>
+
+#include "rv.h"
+
+bool __read_mostly reacting_on;
+
+/*
+ * Interface for the reactor register.
+ */
+LIST_HEAD(rv_reactors_list);
+
+static struct rv_reactor_def *get_reactor_rdef_by_name(char *name)
+{
+	struct rv_reactor_def *r;
+
+	list_for_each_entry(r, &rv_reactors_list, list) {
+		if (strcmp(name, r->reactor->name) == 0)
+			return r;
+	}
+	return NULL;
+}
+
+/*
+ * Available reactors seq functions.
+ */
+static int reactors_show(struct seq_file *m, void *p)
+{
+	struct rv_reactor_def *rea_def = p;
+
+	seq_printf(m, "%s\n", rea_def->reactor->name);
+	return 0;
+}
+
+static void reactors_stop(struct seq_file *m, void *p)
+{
+	mutex_unlock(&rv_interface_lock);
+}
+
+static void *reactors_start(struct seq_file *m, loff_t *pos)
+{
+	mutex_lock(&rv_interface_lock);
+	return seq_list_start(&rv_reactors_list, *pos);
+}
+
+static void *reactors_next(struct seq_file *m, void *p, loff_t *pos)
+{
+	return seq_list_next(p, &rv_reactors_list, pos);
+}
+
+/*
+ * available reactors seq definition.
+ */
+static const struct seq_operations available_reactors_seq_ops = {
+	.start	= reactors_start,
+	.next	= reactors_next,
+	.stop	= reactors_stop,
+	.show	= reactors_show
+};
+
+/*
+ * available_reactors interface.
+ */
+static int available_reactors_open(struct inode *inode, struct file *file)
+{
+	return seq_open(file, &available_reactors_seq_ops);
+};
+
+static const struct file_operations available_reactors_ops = {
+	.open    = available_reactors_open,
+	.read    = seq_read,
+	.llseek  = seq_lseek,
+	.release = seq_release
+};
+
+/*
+ * Monitor reactor file.
+ */
+static int monitor_reactor_show(struct seq_file *m, void *p)
+{
+	struct rv_monitor_def *mdef = m->private;
+	struct rv_reactor_def *rdef = p;
+
+	if (mdef->rdef == rdef)
+		seq_printf(m, "[%s]\n", rdef->reactor->name);
+	else
+		seq_printf(m, "%s\n", rdef->reactor->name);
+	return 0;
+}
+
+/*
+ * available reactors seq definition.
+ */
+static const struct seq_operations monitor_reactors_seq_ops = {
+	.start	= reactors_start,
+	.next	= reactors_next,
+	.stop	= reactors_stop,
+	.show	= monitor_reactor_show
+};
+
+static ssize_t
+monitor_reactors_write(struct file *file, const char __user *user_buf,
+		      size_t count, loff_t *ppos)
+{
+	char buff[MAX_RV_REACTOR_NAME_SIZE+1];
+	struct rv_monitor_def *mdef;
+	struct rv_reactor_def *rdef;
+	struct seq_file *seq_f;
+	int retval = -EINVAL;
+	char *ptr = buff;
+	int len;
+
+	if (count < 1 || count > MAX_RV_REACTOR_NAME_SIZE+1)
+		return -EINVAL;
+
+	memset(buff, 0, sizeof(buff));
+
+	retval = simple_write_to_buffer(buff, sizeof(buff)-1, ppos, user_buf,
+					count);

kernel/trace/rv/rv_reactors.c line 191

Quoted 10 lines
kernel/trace/rv/rv_reactors.c new file +10 -0
+	if (!retval)
+		return -EFAULT;
+
+	len = strlen(ptr);
+	if (!len)
+		return count;
+	/*
+	 * remove the \n
+	 */
+	ptr[len-1] = '\0';

Again, use strim().

Quoted 22 lines
kernel/trace/rv/rv_reactors.c new file +22 -0
+
+	/*
+	 * See monitor_reactors_open()
+	 */
+	seq_f = file->private_data;
+	mdef = seq_f->private;
+
+	mutex_lock(&rv_interface_lock);
+
+	retval = -EINVAL;
+
+	/*
+	 * nop special case: disable reacting.
+	 */
+	if (strcmp(ptr, "nop") == 0) {
+
+		if (mdef->monitor->enabled)
+			mdef->monitor->stop();
+
+		mdef->rdef = get_reactor_rdef_by_name("nop");
+		mdef->reacting = false;
+		mdef->monitor->react = NULL;

kernel/trace/rv/rv_reactors.c line 223

Quoted 10 lines
kernel/trace/rv/rv_reactors.c new file +10 -0
+
+		if (mdef->monitor->enabled)
+			mdef->monitor->start();
+
+		retval = count;
+		goto unlock;
+	}
+
+	list_for_each_entry(rdef, &rv_reactors_list, list) {
+		if (strcmp(ptr, rdef->reactor->name) == 0) {

Again,

		if (strcmp(ptr, rdef->reactor->name) != 0)
			continue;

Then we can remove the extra indent.

-- Steve

Quoted 25 lines
kernel/trace/rv/rv_reactors.c new file +25 -0
+			/*
+			 * found!
+			 */
+			if (mdef->monitor->enabled)
+				mdef->monitor->stop();
+
+			mdef->rdef = rdef;
+			mdef->reacting = true;
+			mdef->monitor->react = rdef->reactor->react;
+
+			if (mdef->monitor->enabled)
+				mdef->monitor->start();
+
+			retval = count;
+			break;
+		}
+	}
+
+unlock:
+	mutex_unlock(&rv_interface_lock);
+
+	return retval;
+}
+
+/*
#16 lore raw

PATCH v4 3/20 rv/include: Add helper functions for deterministic automata

In reply to Daniel Bristot de Oliveira

Formally, a deterministic automaton, denoted by G, is defined as a quintuple:

G = { X, E, f, x_0, X_m }

where:
- X is the set of states;
- E is the finite set of events;
- x_0 is the initial state;
- X_m (subset of X) is the set of marked states.
- f : X x E -> X $ is the transition function. It defines the state transition in the occurrence of a event from E in the state X. In the special case of deterministic automata, the occurrence of the event in E in a state in X has a deterministic next state from X.

An automaton can also be represented using a graphical format of vertices (nodes) and edges. The open-source tool Graphviz can produce this graphic format using the (textual) DOT language as the source code.

The dot2c tool presented in this paper:

DE OLIVEIRA, Daniel Bristot; CUCINOTTA, Tommaso; DE OLIVEIRA, Romulo Silva. Efficient formal verification for the Linux kernel. In:
International Conference on Software Engineering and Formal Methods. Springer, Cham, 2019. p. 315-332.

Translates a deterministic automaton in the DOT format into a C source code representation that to be used for monitoring.

This header file implements helper functions to facilitate the usage of the C output from dot2c for monitoring.

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 include/rv/automata.h | 49 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)
 create mode 100644 include/rv/automata.h
include/rv/automata.h new file +49 -0
new file mode 100644
index 000000000000..0c0aa54bd820
@@ -0,0 +1,49 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Deterministic automata helper functions, to be used with the automata
+ * models in C generated by the dot2k tool.
+ *
+ * Copyright (C) 2019-2022 Daniel Bristot de Oliveira <[email protected]>
+ */
+
+#define DECLARE_AUTOMATA_HELPERS(name, type)					\
+										\
+static inline void *model_get_model_##name(void)				\
+{										\
+	return (void *) &automaton_##name;					\
+}										\
+										\
+static char *model_get_state_name_##name(enum states_##name state)		\
+{										\
+	return automaton_##name.state_names[state];				\
+}										\
+										\
+static char *model_get_event_name_##name(enum events_##name event)		\
+{										\
+	return automaton_##name.event_names[event];				\
+}										\
+										\
+static inline type model_get_init_state_##name(void)				\
+{										\
+	return automaton_##name.initial_state;					\
+}										\
+										\
+static inline type model_get_next_state_##name(enum states_##name curr_state,	\
+					       enum events_##name event)	\
+{										\
+	if ((curr_state < 0) || (curr_state > state_max_##name))		\
+		return -1;							\
+										\
+	if ((event < 0) || (event > event_max_##name))				\
+		return -1;							\
+										\
+	return automaton_##name.function[curr_state][event];			\
+}										\
+										\
+static inline bool model_is_final_state_##name(enum states_##name state)	\
+{										\
+	if ((state < 0) || (state > state_max_##name))				\
+		return 0;							\
+										\
+	return !!automaton_##name.final_states[state];				\
+}
Signature · 1 line
2.35.1
maintainer
#17 lore raw

Re: [PATCH V4 03/20] rv/include: Add helper functions for deterministic automata

Quoted 64 lines · On Thu, 16 Jun 2022 10:44:45 +0200 Daniel Bristot de Oliveira…

Formally, a deterministic automaton, denoted by G, is defined as a quintuple:

G = { X, E, f, x_0, X_m }

where:
- X is the set of states;
- E is the finite set of events;
- x_0 is the initial state;
- X_m (subset of X) is the set of marked states.
- f : X x E -> X $ is the transition function. It defines the state transition in the occurrence of a event from E in the state X. In the special case of deterministic automata, the occurrence of the event in E in a state in X has a deterministic next state from X.

An automaton can also be represented using a graphical format of vertices (nodes) and edges. The open-source tool Graphviz can produce this graphic format using the (textual) DOT language as the source code.

The dot2c tool presented in this paper:

DE OLIVEIRA, Daniel Bristot; CUCINOTTA, Tommaso; DE OLIVEIRA, Romulo Silva. Efficient formal verification for the Linux kernel. In:
International Conference on Software Engineering and Formal Methods. Springer, Cham, 2019. p. 315-332.

Translates a deterministic automaton in the DOT format into a C source code representation that to be used for monitoring.

This header file implements helper functions to facilitate the usage of the C output from dot2c for monitoring.

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 include/rv/automata.h | 49 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)
 create mode 100644 include/rv/automata.h
include/rv/automata.h new file +0 -0
new file mode 100644
index 000000000000..0c0aa54bd820

include/rv/automata.h line 6

Quoted 6 lines
include/rv/automata.h new file +6 -0
@@ -0,0 +1,49 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Deterministic automata helper functions, to be used with the automata
+ * models in C generated by the dot2k tool.
+ *
+ * Copyright (C) 2019-2022 Daniel Bristot de Oliveira <[email protected]>

Copyright before the description.

include/rv/automata.h line 8

Quoted 2 lines
include/rv/automata.h new file +2 -0
+ */
+

There should be comments here describing how the helper functions are used.

-- Steve

Quoted 41 lines
include/rv/automata.h new file +41 -0
+#define DECLARE_AUTOMATA_HELPERS(name, type)					\
+										\
+static inline void *model_get_model_##name(void)				\
+{										\
+	return (void *) &automaton_##name;					\
+}										\
+										\
+static char *model_get_state_name_##name(enum states_##name state)		\
+{										\
+	return automaton_##name.state_names[state];				\
+}										\
+										\
+static char *model_get_event_name_##name(enum events_##name event)		\
+{										\
+	return automaton_##name.event_names[event];				\
+}										\
+										\
+static inline type model_get_init_state_##name(void)				\
+{										\
+	return automaton_##name.initial_state;					\
+}										\
+										\
+static inline type model_get_next_state_##name(enum states_##name curr_state,	\
+					       enum events_##name event)	\
+{										\
+	if ((curr_state < 0) || (curr_state > state_max_##name))		\
+		return -1;							\
+										\
+	if ((event < 0) || (event > event_max_##name))				\
+		return -1;							\
+										\
+	return automaton_##name.function[curr_state][event];			\
+}										\
+										\
+static inline bool model_is_final_state_##name(enum states_##name state)	\
+{										\
+	if ((state < 0) || (state > state_max_##name))				\
+		return 0;							\
+										\
+	return !!automaton_##name.final_states[state];				\
+}
#18 lore raw

Re: [PATCH V4 03/20] rv/include: Add helper functions for deterministic automata

In reply to Daniel Bristot de Oliveira
Quoted 89 lines · Daniel Bristot de Oliveira

Formally, a deterministic automaton, denoted by G, is defined as a quintuple:

G = { X, E, f, x_0, X_m }

where:
- X is the set of states;
- E is the finite set of events;
- x_0 is the initial state;
- X_m (subset of X) is the set of marked states.
- f : X x E -> X $ is the transition function. It defines the state transition in the occurrence of a event from E in the state X. In the special case of deterministic automata, the occurrence of the event in E in a state in X has a deterministic next state from X.

An automaton can also be represented using a graphical format of vertices (nodes) and edges. The open-source tool Graphviz can produce this graphic format using the (textual) DOT language as the source code.

The dot2c tool presented in this paper:

DE OLIVEIRA, Daniel Bristot; CUCINOTTA, Tommaso; DE OLIVEIRA, Romulo Silva. Efficient formal verification for the Linux kernel. In:
International Conference on Software Engineering and Formal Methods. Springer, Cham, 2019. p. 315-332.

Translates a deterministic automaton in the DOT format into a C surce code representation that to be used for monitoring.

This header file implements helper functions to facilitate the usage of the C output from dot2c for monitoring.

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 include/rv/automata.h | 49 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)
 create mode 100644 include/rv/automata.h
include/rv/automata.h new file +25 -0
new file mode 100644
index 000000000000..0c0aa54bd820
@@ -0,0 +1,49 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Deterministic automata helper functions, to be used with the automata
+ * models in C generated by the dot2k tool.
+ *
+ * Copyright (C) 2019-2022 Daniel Bristot de Oliveira <[email protected]>
+ */
+
+#define DECLARE_AUTOMATA_HELPERS(name, type)					\
+										\
+static inline void *model_get_model_##name(void)				\
+{										\
+	return (void *) &automaton_##name;					\
+}									\
+										\
+static char *model_get_state_name_##name(enum states_##name state)		\
+{										\
+	return automaton_##name.state_names[state];				\
+}										\
+										\
+static char *model_get_event_name_##name(enum events_##name event)		\
+{										\
+	return automaton_##name.event_names[event];				\
+}										\
+										\

include/rv/automata.h line 35

Quoted 10 lines
include/rv/automata.h new file +10 -0
+static inline type model_get_init_state_##name(void)				\
+{										\
+	return automaton_##name.initial_state;					\
+}										\
+										\
+static inline type model_get_next_state_##name(enum states_##name curr_state,	\
+					       enum events_##name event)	\
+{										\
+	if ((curr_state < 0) || (curr_state > state_max_##name))		\
+		return -1;							\

curr_state can not be state_max_xxx. curr_state must be not bigger than state_max_xxx. Or am I miss something?

include/rv/automata.h line 39

Quoted 3 lines
include/rv/automata.h new file +3 -0
+	if ((event < 0) || (event > event_max_##name))				\
+		return -1;							\
+										\

Same here for the event boundary check.

include/rv/automata.h line 47

Quoted 8 lines
include/rv/automata.h new file +8 -0
+	return automaton_##name.function[curr_state][event];			\
+}										\
+										\
+static inline bool model_is_final_state_##name(enum states_##name state)	\
+{										\
+	if ((state < 0) || (state > state_max_##name))				\
+		return 0;							\
+										\

Same here.

include/rv/automata.h line 48

Quoted 1 line
include/rv/automata.h new file +1 -0
+	return !!automaton_##name.final_states[state];				\

If the value of .final_states[state] is 0 or 1, can the type of .final_states[state] be befined to bool. Or not need to use !! to explicitly transfer the type to bool. I remember that you define this as char array and the matrix model value of this array is 0 or 1 see from the next patche. 1 delegate the state it is the initial state. 0 for others.

Quoted 4 lines
include/rv/automata.h new file +1 -0
+}
Signature · 1 line
2.35.1
#19 lore raw

Re: [PATCH V4 03/20] rv/include: Add helper functions for deterministic automata

Quoted 122 lines · Tao Zhou
Quoted 89 lines · Daniel Bristot de Oliveira

Formally, a deterministic automaton, denoted by G, is defined as a quintuple:

G = { X, E, f, x_0, X_m }

where:
- X is the set of states;
- E is the finite set of events;
- x_0 is the initial state;
- X_m (subset of X) is the set of marked states.
- f : X x E -> X $ is the transition function. It defines the state transition in the occurrence of a event from E in the state X. In the special case of deterministic automata, the occurrence of the event in E in a state in X has a deterministic next state from X.

An automaton can also be represented using a graphical format of vertices (nodes) and edges. The open-source tool Graphviz can produce this graphic format using the (textual) DOT language as the source code.

The dot2c tool presented in this paper:

DE OLIVEIRA, Daniel Bristot; CUCINOTTA, Tommaso; DE OLIVEIRA, Romulo Silva. Efficient formal verification for the Linux kernel. In:
International Conference on Software Engineering and Formal Methods. Springer, Cham, 2019. p. 315-332.

Translates a deterministic automaton in the DOT format into a C surce code representation that to be used for monitoring.

This header file implements helper functions to facilitate the usage of the C output from dot2c for monitoring.

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 include/rv/automata.h | 49 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)
 create mode 100644 include/rv/automata.h
include/rv/automata.h new file +25 -0
new file mode 100644
index 000000000000..0c0aa54bd820
@@ -0,0 +1,49 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Deterministic automata helper functions, to be used with the automata
+ * models in C generated by the dot2k tool.
+ *
+ * Copyright (C) 2019-2022 Daniel Bristot de Oliveira <[email protected]>
+ */
+
+#define DECLARE_AUTOMATA_HELPERS(name, type)					\
+										\
+static inline void *model_get_model_##name(void)				\
+{										\
+	return (void *) &automaton_##name;					\
+}									\
+										\
+static char *model_get_state_name_##name(enum states_##name state)		\
+{										\
+	return automaton_##name.state_names[state];				\
+}										\
+										\
+static char *model_get_event_name_##name(enum events_##name event)		\
+{										\
+	return automaton_##name.event_names[event];				\
+}										\
+										\

include/rv/automata.h line 35

Quoted 10 lines
include/rv/automata.h new file +10 -0
+static inline type model_get_init_state_##name(void)				\
+{										\
+	return automaton_##name.initial_state;					\
+}										\
+										\
+static inline type model_get_next_state_##name(enum states_##name curr_state,	\
+					       enum events_##name event)	\
+{										\
+	if ((curr_state < 0) || (curr_state > state_max_##name))		\
+		return -1;							\

curr_state can not be state_max_xxx. curr_state must be not bigger than state_max_xxx. Or am I miss something?

include/rv/automata.h line 39

Quoted 3 lines
include/rv/automata.h new file +3 -0
+	if ((event < 0) || (event > event_max_##name))				\
+		return -1;							\
+										\

Same here for the event boundary check.

include/rv/automata.h line 47

Quoted 8 lines
include/rv/automata.h new file +8 -0
+	return automaton_##name.function[curr_state][event];			\
+}										\
+										\
+static inline bool model_is_final_state_##name(enum states_##name state)	\
+{										\
+	if ((state < 0) || (state > state_max_##name))				\
+		return 0;							\
+										\

Same here.

Quoted 8 lines

include/rv/automata.h line 48

Quoted 1 line
include/rv/automata.h new file +1 -0
+	return !!automaton_##name.final_states[state];				\

If the value of .final_states[state] is 0 or 1, can the type of .final_states[state] be befined to bool. Or not need to use !! to explicitly transfer the type to bool. I remember that you define this as char array and the matrix model value of this array is 0 or 1 see from the next patche. 1 delegate the state it is the initial state. 0 for others.

All points addressed!
-- Daniel

#20 lore raw

PATCH v4 4/20 rv/include: Add deterministic automata monitor definition via C macros

In reply to Daniel Bristot de Oliveira

In Linux terms, the runtime verification monitors are encapsulated inside the "RV monitor" abstraction. The "RV monitor" includes a set of instances of the monitor (per-cpu monitor, per-task monitor, and so on), the helper functions that glue the monitor to the system reference model, and the trace output as a reaction for event parsing and exceptions, as depicted below:

Linux +----- RV Monitor ----------------------------------+ Formal
Realm | | Realm

 +-------------------+     +----------------+     +-----------------+
 |   Linux kernel    |     |     Monitor    |     |     Reference   |
 |     Tracing       |  -> |   Instance(s)  | <-  |       Model     |
 | (instrumentation) |     | (verification) |     | (specification) |
 +-------------------+     +----------------+     +-----------------+
        |                          |                       |
        |                          V                       |
        |                     +----------+                 |
        |                     | Reaction |                 |
        |                     +--+--+--+-+                 |
        |                        |  |  |                   |
        |                        |  |  +-> trace output ?  |
        +------------------------|--|----------------------+
                                 |  +----> panic ?
                                 +-------> <user-specified>

The dot2c tool presented in this paper:

DE OLIVEIRA, Daniel Bristot; CUCINOTTA, Tommaso; DE OLIVEIRA, Romulo Silva. Efficient formal verification for the Linux kernel. In:
International Conference on Software Engineering and Formal Methods. Springer, Cham, 2019. p. 315-332.

Translates a deterministic automaton in the DOT format into a C source code representation that to be used for monitoring connecting the Formal Reaml to Linux-like code.

This header file goes beyond, extending the code generation to the verification stage, generating the code to the Monitor Instance(s) level using C macros. The trace event code inspires this approach.

The benefits of the usage of macro for monitor synthesis is 3-fold:

- Reduces the code duplication;
- Facilitates the bug fix/improvement;
(but mainly:)
- Avoids the case of developers changing the core of the monitor code to manipulate the model in a (let's say) non-standard way.

This initial implementation presents three different types of monitor instances:

- #define DECLARE_DA_MON_GLOBAL(name, type)
- #define DECLARE_DA_MON_PER_CPU(name, type)
- #define DECLARE_DA_MON_PER_TASK(name, type)

The first declares the functions for a global deterministic automata monitor, the second with per-cpu instances, and the third with per-task instances.

In all cases, the name is a string that identifies the monitor, and the type is the data type used by dot2c/k on the representation of the model.

For example, the model "wip" below:

preempt_disable sched_waking
+############+ >------------------> +################+ >------------+
-># preemptive # # non-preemptive # |
+############+ <-----------------< +################+ <------------+
preempt_enable

with two states and three events can be stored in a 'char' type. Considering that the preemption control is a per-cpu behavior, the monitor declaration will be:

  DECLARE_DA_MON_PER_CPU(wip, char);

The monitor is executed by sending events to be processed via the functions presented below:

  da_handle_event_$(MONITOR_NAME)($(event from event enum));
  da_handle_init_event_$(MONITOR_NAME)($(event from event enum));

The function da_handle_event_$(MONITOR_NAME) is the regular case, while the function da_handle_init_event_$(MONITOR_NAME)() is a special case used to synchronize the system with the model.

When a monitor is enabled, it is placed in the initial state of the automata. However, the monitor does not know if the system is in the initial state. Hence, the monitor ignores events sent by sent by da_handle_event_$(MONITOR_NAME) until the function da_handle_init_event_$(MONITOR_NAME)() is called.

The function da_handle_init_event_$(MONITOR_NAME)() should be used for the case in which the system generates the event is the one that returns the automata to the initial state.

After receiving a da_handle_init_event_$(MONITOR_NAME)() event, the monitor will know that it is in sync with the system and hence will start processing the next events.

Using the wip model as example, the events "preempt_disable" and "sched_waking" should be sent to monitor, respectively, via:

        da_handle_event_wip(preempt_disable);
        da_handle_event_wip(sched_waking);

While the event "preempt_enabled" will use:

        da_handle_init_event_wip(preempt_enable);

To notify the monitor that the system will be returning to the initial state, so the system and the monitor should be in sync.

With the monitor synthesis in place, using these headers and dot2k, the developer's work should be limited to the instrumentation of the system, increasing the confidence in the overall approach.

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 include/linux/rv.h        |   2 +
 include/rv/da_monitor.h   | 419 ++++++++++++++++++++++++++++++++++++++
 include/rv/rv.h           |   9 +
 include/trace/events/rv.h | 120 +++++++++++
 kernel/fork.c             |   2 +-
 kernel/trace/rv/Kconfig   |  14 ++
 kernel/trace/rv/rv.c      |   5 +
 7 files changed, 570 insertions(+), 1 deletion(-)
 create mode 100644 include/rv/da_monitor.h
 create mode 100644 include/trace/events/rv.h
include/linux/rv.h +2 -0
index 1e48c6bb74bf..af2081671219 100644
@@ -9,6 +9,8 @@
 #ifndef _LINUX_RV_H
 #define _LINUX_RV_H
 
+#define MAX_DA_NAME_LEN         24
+
 struct rv_reactor {
 	char			*name;
 	char			*description;
include/rv/da_monitor.h new file +419 -0

419 lines not on this page · raw message

include/rv/rv.h +9 -0
index 27a108881d35..b0658cdc53d9 100644
@@ -3,6 +3,14 @@
 #ifndef _RV_RV_H
 #define _RV_RV_H
 
+/*
+ * Deterministic automaton per-object variables.
+ */
+struct da_monitor {
+	bool	monitoring;
+	int	curr_state;
+};
+
 /*
  * Per-task RV monitors count. Nowadays fixed in RV_PER_TASK_MONITORS.
  * If we find justification for more monitors, we can think about
@@ -16,6 +24,7 @@
  * Futher monitor types are expected, so make this a union.
  */
 union rv_task_monitor {
+	struct da_monitor da_mon;
 };
 
 int get_task_monitor_slot(void);
include/trace/events/rv.h new file +120 -0

120 lines not on this page · raw message

kernel/fork.c +1 -1
index 5e40e58ef83d..6f1f82ccd5f2 100644
@@ -1970,7 +1970,7 @@ static void rv_task_fork(struct task_struct *p)
 	int i;
 
 	for (i = 0; i < RV_PER_TASK_MONITORS; i++)
-		;
+		p->rv[i].da_mon.monitoring = false;
 }
 #else
 #define rv_task_fork(p) do {} while (0)
kernel/trace/rv/Kconfig +14 -0
index 560408fec0c8..1eafb5adcfcb 100644
@@ -1,5 +1,19 @@
 # SPDX-License-Identifier: GPL-2.0-only
 #
+config DA_MON_EVENTS
+	default n
+	bool
+
+config DA_MON_EVENTS_IMPLICIT
+	select DA_MON_EVENTS
+	default n
+	bool
+
+config DA_MON_EVENTS_ID
+	select DA_MON_EVENTS
+	default n
+	bool
+
 menuconfig RV
 	bool "Runtime Verification"
 	depends on TRACING
kernel/trace/rv/rv.c +5 -0
index 7576d492a974..51a610227341 100644
@@ -143,6 +143,11 @@
 #include <linux/slab.h>
 #include <rv/rv.h>
 
+#ifdef CONFIG_DA_MON_EVENTS
+#define CREATE_TRACE_POINTS
+#include <trace/events/rv.h>
+#endif
+
 #include "rv.h"
 
 DEFINE_MUTEX(rv_interface_lock);
Signature · 1 line
2.35.1
#21 lore raw

Re: [PATCH V4 04/20] rv/include: Add deterministic automata monitor definition via C macros

Quoted 179 lines · Daniel Bristot de Oliveira

In Linux terms, the runtime verification monitors are encapsulated inside the "RV monitor" abstraction. The "RV monitor" includes a set of instances of the monitor (per-cpu monitor, per-task monitor, and so on), the helper functions that glue the monitor to the system reference model, and the trace output as a reaction for event parsing and exceptions, as depicted below:

Linux +----- RV Monitor ----------------------------------+ Formal
Realm | | Realm

 +-------------------+     +----------------+     +-----------------+
 |   Linux kernel    |     |     Monitor    |     |     Reference   |
 |     Tracing       |  -> |   Instance(s)  | <-  |       Model     |
 | (instrumentation) |     | (verification) |     | (specification) |
 +-------------------+     +----------------+     +-----------------+
        |                          |                       |
        |                          V                       |
        |                     +----------+                 |
        |                     | Reaction |                 |
        |                     +--+--+--+-+                 |
        |                        |  |  |                   |
        |                        |  |  +-> trace output ?  |
        +------------------------|--|----------------------+
                                 |  +----> panic ?
                                 +-------> <user-specified>

The dot2c tool presented in this paper:

DE OLIVEIRA, Daniel Bristot; CUCINOTTA, Tommaso; DE OLIVEIRA, Romulo Silva. Efficient formal verification for the Linux kernel. In:
International Conference on Software Engineering and Formal Methods. Springer, Cham, 2019. p. 315-332.

Translates a deterministic automaton in the DOT format into a C source code representation that to be used for monitoring connecting the Formal Reaml to Linux-like code.

This header file goes beyond, extending the code generation to the verification stage, generating the code to the Monitor Instance(s) level using C macros. The trace event code inspires this approach.

The benefits of the usage of macro for monitor synthesis is 3-fold:

- Reduces the code duplication;
- Facilitates the bug fix/improvement;
(but mainly:)
- Avoids the case of developers changing the core of the monitor code to manipulate the model in a (let's say) non-standard way.

This initial implementation presents three different types of monitor instances:

- #define DECLARE_DA_MON_GLOBAL(name, type)
- #define DECLARE_DA_MON_PER_CPU(name, type)
- #define DECLARE_DA_MON_PER_TASK(name, type)

The first declares the functions for a global deterministic automata monitor, the second with per-cpu instances, and the third with per-task instances.

In all cases, the name is a string that identifies the monitor, and the type is the data type used by dot2c/k on the representation of the model.

For example, the model "wip" below:

preempt_disable sched_waking
+############+ >------------------> +################+ >------------+
-># preemptive # # non-preemptive # |
+############+ <-----------------< +################+ <------------+
preempt_enable

with two states and three events can be stored in a 'char' type. Considering that the preemption control is a per-cpu behavior, the monitor declaration will be:

  DECLARE_DA_MON_PER_CPU(wip, char);

The monitor is executed by sending events to be processed via the functions presented below:

  da_handle_event_$(MONITOR_NAME)($(event from event enum));
  da_handle_init_event_$(MONITOR_NAME)($(event from event enum));

The function da_handle_event_$(MONITOR_NAME) is the regular case, while the function da_handle_init_event_$(MONITOR_NAME)() is a special case used to synchronize the system with the model.

When a monitor is enabled, it is placed in the initial state of the automata. However, the monitor does not know if the system is in the initial state. Hence, the monitor ignores events sent by sent by da_handle_event_$(MONITOR_NAME) until the function da_handle_init_event_$(MONITOR_NAME)() is called.

The function da_handle_init_event_$(MONITOR_NAME)() should be used for the case in which the system generates the event is the one that returns the automata to the initial state.

After receiving a da_handle_init_event_$(MONITOR_NAME)() event, the monitor will know that it is in sync with the system and hence will start processing the next events.

Using the wip model as example, the events "preempt_disable" and "sched_waking" should be sent to monitor, respectively, via:

        da_handle_event_wip(preempt_disable);
        da_handle_event_wip(sched_waking);

While the event "preempt_enabled" will use:

        da_handle_init_event_wip(preempt_enable);

To notify the monitor that the system will be returning to the initial state, so the system and the monitor should be in sync.

With the monitor synthesis in place, using these headers and dot2k, the developer's work should be limited to the instrumentation of the system, increasing the confidence in the overall approach.

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 include/linux/rv.h        |   2 +
 include/rv/da_monitor.h   | 419 ++++++++++++++++++++++++++++++++++++++
 include/rv/rv.h           |   9 +
 include/trace/events/rv.h | 120 +++++++++++
 kernel/fork.c             |   2 +-
 kernel/trace/rv/Kconfig   |  14 ++
 kernel/trace/rv/rv.c      |   5 +
 7 files changed, 570 insertions(+), 1 deletion(-)
 create mode 100644 include/rv/da_monitor.h
 create mode 100644 include/trace/events/rv.h
include/linux/rv.h +2 -0
index 1e48c6bb74bf..af2081671219 100644
@@ -9,6 +9,8 @@
 #ifndef _LINUX_RV_H
 #define _LINUX_RV_H
 
+#define MAX_DA_NAME_LEN         24
+
 struct rv_reactor {
 	char			*name;
 	char			*description;
include/rv/da_monitor.h new file +11 -0
new file mode 100644
index 000000000000..043660429659
@@ -0,0 +1,419 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Deterministic automata (DA) monitor functions, to be used togheter
+ * with automata models in C generated by the dot2k tool.
+ *
+ * The dot2k tool is available at tools/tracing/rv/
+ *
+ * Copyright (C) 2019-2022 Daniel Bristot de Oliveira <[email protected]>
+ */
+
+#include <rv/automata.h>

include/rv/da_monitor.h line 21

Quoted 10 lines
include/rv/da_monitor.h new file +10 -0
+#include <linux/rv.h>
+
+/*
+ * Generic helpers for all types of deterministic automata monitors.
+ */
+#define DECLARE_DA_MON_GENERIC_HELPERS(name, type)				\
+static char REACT_MSG[1024];							\
+										\
+static inline char								\
+*format_react_msg(type curr_state, type event)					\

Not seperate char * into tow lines seems to be comfortable to me.

Quoted 354 lines
include/rv/da_monitor.h new file +354 -0
+{										\
+	snprintf(REACT_MSG, 1024,						\
+		 "rv: monitor %s does not allow event %s on state %s\n",	\
+		 MODULE_NAME,							\
+		 model_get_event_name_##name(event),				\
+		 model_get_state_name_##name(curr_state));			\
+	return REACT_MSG;							\
+}										\
+										\
+static void cond_react(char *msg)						\
+{										\
+	if (rv_##name.react)							\
+		rv_##name.react(msg);						\
+}										\
+										\
+static inline void da_monitor_reset_##name(struct da_monitor *da_mon)		\
+{										\
+	da_mon->monitoring = 0;							\
+	da_mon->curr_state = model_get_init_state_##name();			\
+}										\
+										\
+static inline type da_monitor_curr_state_##name(struct da_monitor *da_mon)	\
+{										\
+	return da_mon->curr_state;						\
+}										\
+										\
+static inline void								\
+da_monitor_set_state_##name(struct da_monitor *da_mon, enum states_##name state)\
+{										\
+	da_mon->curr_state = state;						\
+}										\
+static inline void da_monitor_start_##name(struct da_monitor *da_mon)		\
+{										\
+	da_mon->monitoring = 1;							\
+}										\
+										\
+static inline bool da_monitoring_##name(struct da_monitor *da_mon)		\
+{										\
+	return da_mon->monitoring;						\
+}
+
+
+/*
+ * Event handler for implict monitors. Implicity monitor is the one which the
+ * handler does not need to specify which da_monitor to manilupulate. Examples
+ * of implicit monitor are the per_cpu or the global ones.
+ */
+#define DECLARE_DA_MON_MODEL_HANDLER_IMPLICIT(name, type)			\
+static inline bool								\
+da_event_##name(struct da_monitor *da_mon, enum events_##name event)		\
+{										\
+	type curr_state = da_monitor_curr_state_##name(da_mon);			\
+	type next_state = model_get_next_state_##name(curr_state, event);	\
+										\
+	if (next_state >= 0) {							\
+		da_monitor_set_state_##name(da_mon, next_state);		\
+										\
+		trace_event_##name(model_get_state_name_##name(curr_state),	\
+				model_get_event_name_##name(event),		\
+				model_get_state_name_##name(next_state),	\
+				model_is_final_state_##name(next_state));	\
+										\
+		return true;							\
+	}									\
+										\
+	if (reacting_on)							\
+		cond_react(format_react_msg(curr_state, event));		\
+										\
+	trace_error_##name(model_get_state_name_##name(curr_state),		\
+			   model_get_event_name_##name(event));			\
+										\
+	return false;								\
+}										\
+
+/*
+ * Event handler for per_task monitors.
+ */
+#define DECLARE_DA_MON_MODEL_HANDLER_PER_TASK(name, type)			\
+static inline type								\
+da_event_##name(struct da_monitor *da_mon, struct task_struct *tsk,		\
+		enum events_##name event)					\
+{										\
+	type curr_state = da_monitor_curr_state_##name(da_mon);			\
+	type next_state = model_get_next_state_##name(curr_state, event);	\
+										\
+	if (next_state >= 0) {							\
+		da_monitor_set_state_##name(da_mon, next_state);		\
+										\
+		trace_event_##name(tsk->pid,					\
+				   model_get_state_name_##name(curr_state),	\
+				   model_get_event_name_##name(event),		\
+				   model_get_state_name_##name(next_state),	\
+				   model_is_final_state_##name(next_state));	\
+										\
+		return true;							\
+	}									\
+										\
+	if (reacting_on)							\
+		cond_react(format_react_msg(curr_state, event));		\
+										\
+	trace_error_##name(tsk->pid,						\
+			   model_get_state_name_##name(curr_state),		\
+			   model_get_event_name_##name(event));			\
+										\
+	return false;								\
+}
+
+/*
+ * Functions to define, init and get a global monitor.
+ */
+#define DECLARE_DA_MON_INIT_GLOBAL(name, type)					\
+										\
+static struct da_monitor da_mon_##name;						\
+										\
+static struct da_monitor *da_get_monitor_##name(void)				\
+{										\
+	return &da_mon_##name;							\
+}										\
+										\
+static void da_monitor_reset_all_##name(void)					\
+{										\
+	da_monitor_reset_##name(da_mon_##name);					\
+}										\
+										\
+static inline int da_monitor_init_##name(void)					\
+{										\
+	struct da_monitor *da_mon = &da_mon_##name				\
+	da_mon->curr_state = model_get_init_state_##name();			\
+	da_mon->monitoring = 0;							\
+	return 0;								\
+}										\
+										\
+static inline void da_monitor_destroy_##name(void)				\
+{										\
+	return;									\
+}
+
+/*
+ * Functions to define, init and get a per-cpu monitor.
+ */
+#define DECLARE_DA_MON_INIT_PER_CPU(name, type)					\
+										\
+DEFINE_PER_CPU(struct da_monitor, da_mon_##name);				\
+										\
+static struct da_monitor *da_get_monitor_##name(void)				\
+{										\
+	return this_cpu_ptr(&da_mon_##name);					\
+}										\
+										\
+static void da_monitor_reset_all_##name(void)					\
+{										\
+	struct da_monitor *da_mon;						\
+	int cpu;								\
+	for_each_cpu(cpu, cpu_online_mask) {					\
+		da_mon = per_cpu_ptr(&da_mon_##name, cpu);			\
+		da_monitor_reset_##name(da_mon);				\
+	}									\
+}										\
+										\
+static inline int da_monitor_init_##name(void)					\
+{										\
+	struct da_monitor *da_mon;						\
+	int cpu;								\
+	for_each_cpu(cpu, cpu_online_mask) {					\
+		da_mon = per_cpu_ptr(&da_mon_##name, cpu);			\
+		da_mon->curr_state = model_get_init_state_##name();		\
+		da_mon->monitoring = 0;						\
+	}									\
+	return 0;								\
+}										\
+										\
+static inline void da_monitor_destroy_##name(void)				\
+{										\
+	return;									\
+}
+
+/*
+ * Functions to define, init and get a per-task monitor.
+ */
+#define DECLARE_DA_MON_INIT_PER_TASK(name, type)				\
+										\
+static int task_mon_slot_##name = RV_PER_TASK_MONITOR_INIT;			\
+										\
+static inline struct da_monitor *da_get_monitor_##name(struct task_struct *tsk)	\
+{										\
+	return &tsk->rv[task_mon_slot_##name].da_mon;				\
+}										\
+										\
+static void da_monitor_reset_all_##name(void)					\
+{										\
+	struct task_struct *g, *p;						\
+										\
+	read_lock(&tasklist_lock);						\
+	for_each_process_thread(g, p)						\
+		da_monitor_reset_##name(da_get_monitor_##name(p));		\
+	read_unlock(&tasklist_lock);						\
+}										\
+										\
+static int da_monitor_init_##name(void)						\
+{										\
+	struct da_monitor *da_mon;						\
+	struct task_struct *g, *p;						\
+	int retval;								\
+										\
+	retval = get_task_monitor_slot();					\
+	if (retval < 0)								\
+		return retval;							\
+										\
+	task_mon_slot_##name = retval;						\
+										\
+	read_lock(&tasklist_lock);						\
+	for_each_process_thread(g, p) {						\
+		da_mon = da_get_monitor_##name(p);				\
+		da_mon->curr_state = model_get_init_state_##name();		\
+		da_mon->monitoring = 0;						\
+	}									\
+	read_unlock(&tasklist_lock);						\
+										\
+	return 0;								\
+}										\
+										\
+static inline void da_monitor_destroy_##name(void)				\
+{										\
+	if (task_mon_slot_##name == RV_PER_TASK_MONITOR_INIT) {			\
+		WARN_ONCE(1, "Disabling a disabled monitor: " #name);		\
+		return;								\
+	}									\
+	put_task_monitor_slot(task_mon_slot_##name);				\
+	return;									\
+}
+
+/*
+ * Handle event for implicit monitor: da_get_monitor_##name() will figure out
+ * the monitor.
+ */
+#define DECLARE_DA_MON_MONITOR_HANDLER_IMPLICIT(name, type)			\
+										\
+static inline void __da_handle_event_##name(struct da_monitor *da_mon,		\
+				     enum events_##name event)			\
+{										\
+	int retval;								\
+										\
+	if (unlikely(!monitoring_on))						\
+		return;								\
+										\
+	if (unlikely(!rv_##name.enabled))					\
+		return;								\
+										\
+	if (unlikely(!da_monitoring_##name(da_mon)))				\
+		return;								\
+										\
+	retval = da_event_##name(da_mon, event);				\
+										\
+	if (!retval)								\
+		da_monitor_reset_##name(da_mon);				\
+}										\
+										\
+static inline void da_handle_event_##name(enum events_##name event)		\
+{										\
+	struct da_monitor *da_mon = da_get_monitor_##name();			\
+	__da_handle_event_##name(da_mon, event);				\
+}										\
+										\
+static inline bool da_handle_init_event_##name(enum events_##name event)	\
+{										\
+	struct da_monitor *da_mon;						\
+										\
+	if (unlikely(!rv_##name.enabled))					\
+		return false;							\
+										\
+	da_mon = da_get_monitor_##name();					\
+										\
+	if (unlikely(!da_monitoring_##name(da_mon))) {				\
+		da_monitor_start_##name(da_mon);				\
+		return false;							\
+	}									\
+										\
+	__da_handle_event_##name(da_mon, event);				\
+										\
+	return true;								\
+}										\
+										\
+static inline bool da_handle_init_run_event_##name(enum events_##name event)	\
+{										\
+	struct da_monitor *da_mon;						\
+										\
+	if (unlikely(!rv_##name.enabled))					\
+		return false;							\
+										\
+	da_mon = da_get_monitor_##name();					\
+										\
+	if (unlikely(!da_monitoring_##name(da_mon)))				\
+		da_monitor_start_##name(da_mon);				\
+										\
+	__da_handle_event_##name(da_mon, event);				\
+										\
+	return true;								\
+}
+
+/*
+ * Handle event for per task.
+ */
+#define DECLARE_DA_MON_MONITOR_HANDLER_PER_TASK(name, type)			\
+										\
+static inline void								\
+__da_handle_event_##name(struct da_monitor *da_mon, struct task_struct *tsk,	\
+			 enum events_##name event)				\
+{										\
+	int retval;								\
+										\
+	if (unlikely(!monitoring_on))						\
+		return;								\
+										\
+	if (unlikely(!rv_##name.enabled))					\
+		return;								\
+										\
+	if (unlikely(!da_monitoring_##name(da_mon)))				\
+		return;								\
+										\
+	retval = da_event_##name(da_mon, tsk, event);				\
+										\
+	if (!retval)								\
+		da_monitor_reset_##name(da_mon);				\
+}										\
+										\
+static inline void								\
+da_handle_event_##name(struct task_struct *tsk, enum events_##name event)	\
+{										\
+	struct da_monitor *da_mon = da_get_monitor_##name(tsk);			\
+	__da_handle_event_##name(da_mon, tsk, event);				\
+}										\
+										\
+static inline bool								\
+da_handle_init_event_##name(struct task_struct *tsk, enum events_##name event)	\
+{										\
+	struct da_monitor *da_mon;						\
+										\
+	if (unlikely(!rv_##name.enabled))					\
+		return false;							\
+										\
+	da_mon = da_get_monitor_##name(tsk);					\
+										\
+	if (unlikely(!da_monitoring_##name(da_mon))) {				\
+		da_monitor_start_##name(da_mon);				\
+		return false;							\
+	}									\
+										\
+	__da_handle_event_##name(da_mon, tsk, event);				\
+										\
+	return true;								\
+}
+
+/*
+ * Entry point for the global monitor.

include/rv/da_monitor.h line 385

Quoted 10 lines
include/rv/da_monitor.h new file +10 -0
+ */
+#define DECLARE_DA_MON_GLOBAL(name, type)					\
+										\
+DECLARE_AUTOMATA_HELPERS(name, type);						\
+										\
+DECLARE_DA_MON_GENERIC_HELPERS(name, type);					\
+										\
+DECLARE_DA_MON_MODEL_HANDLER_IMPLICIT(name, type);				\
+										\
+DECLARE_DA_MON_INIT_PER_CPU(name, type);					\

Why the global monitor declaration use the per-cpu monitor macro. Global monitor has its own DECLARE_DA_MON_INIT_GLOBAL(name, type); Or am I miss something?

Quoted 243 lines
include/rv/da_monitor.h new file +34 -0
+										\
+DECLARE_DA_MON_MONITOR_HANDLER_IMPLICIT(name, type);
+
+
+/*
+ * Entry point for the per-cpu monitor.
+ */
+#define DECLARE_DA_MON_PER_CPU(name, type)					\
+										\
+DECLARE_AUTOMATA_HELPERS(name, type);						\
+										\
+DECLARE_DA_MON_GENERIC_HELPERS(name, type);					\
+										\
+DECLARE_DA_MON_MODEL_HANDLER_IMPLICIT(name, type);				\
+										\
+DECLARE_DA_MON_INIT_PER_CPU(name, type);					\
+										\
+DECLARE_DA_MON_MONITOR_HANDLER_IMPLICIT(name, type);
+
+
+/*
+ * Entry point for the per-task monitor.
+ */
+#define DECLARE_DA_MON_PER_TASK(name, type)					\
+										\
+DECLARE_AUTOMATA_HELPERS(name, type);						\
+										\
+DECLARE_DA_MON_GENERIC_HELPERS(name, type);					\
+										\
+DECLARE_DA_MON_MODEL_HANDLER_PER_TASK(name, type);				\
+										\
+DECLARE_DA_MON_INIT_PER_TASK(name, type);					\
+										\
+DECLARE_DA_MON_MONITOR_HANDLER_PER_TASK(name, type);
include/rv/rv.h +9 -0
index 27a108881d35..b0658cdc53d9 100644
@@ -3,6 +3,14 @@
 #ifndef _RV_RV_H
 #define _RV_RV_H
 
+/*
+ * Deterministic automaton per-object variables.
+ */
+struct da_monitor {
+	bool	monitoring;
+	int	curr_state;
+};
+
 /*
  * Per-task RV monitors count. Nowadays fixed in RV_PER_TASK_MONITORS.
  * If we find justification for more monitors, we can think about
@@ -16,6 +24,7 @@
  * Futher monitor types are expected, so make this a union.
  */
 union rv_task_monitor {
+	struct da_monitor da_mon;
 };
 
 int get_task_monitor_slot(void);
include/trace/events/rv.h new file +120 -0
new file mode 100644
index 000000000000..9f40f2a49f84
@@ -0,0 +1,120 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM rv
+
+#if !defined(_TRACE_RV_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_RV_H
+
+#include <linux/rv.h>
+#include <linux/tracepoint.h>
+
+#ifdef CONFIG_DA_MON_EVENTS_IMPLICIT
+DECLARE_EVENT_CLASS(event_da_monitor,
+
+	TP_PROTO(char *state, char *event, char *next_state, bool safe),
+
+	TP_ARGS(state, event, next_state, safe),
+
+	TP_STRUCT__entry(
+		__array(	char,	state,		MAX_DA_NAME_LEN	)
+		__array(	char,	event,		MAX_DA_NAME_LEN	)
+		__array(	char,	next_state,	MAX_DA_NAME_LEN	)
+		__field(	bool,	safe				)
+	),
+
+	TP_fast_assign(
+		memcpy(__entry->state,		state,		MAX_DA_NAME_LEN);
+		memcpy(__entry->event,		event,		MAX_DA_NAME_LEN);
+		memcpy(__entry->next_state,	next_state,	MAX_DA_NAME_LEN);
+		__entry->safe			= safe;
+	),
+
+	TP_printk("%s x %s -> %s %s",
+		__entry->state,
+		__entry->event,
+		__entry->next_state,
+		__entry->safe ? "(safe)" : "")
+);
+
+DECLARE_EVENT_CLASS(error_da_monitor,
+
+	TP_PROTO(char *state, char *event),
+
+	TP_ARGS(state, event),
+
+	TP_STRUCT__entry(
+		__array(	char,	state,		MAX_DA_NAME_LEN	)
+		__array(	char,	event,		MAX_DA_NAME_LEN	)
+	),
+
+	TP_fast_assign(
+		memcpy(__entry->state,		state,		MAX_DA_NAME_LEN);
+		memcpy(__entry->event,		event,		MAX_DA_NAME_LEN);
+	),
+
+	TP_printk("event %s not expected in the state %s",
+		__entry->event,
+		__entry->state)
+);
+#endif /* CONFIG_DA_MON_EVENTS_IMPLICIT */
+
+#ifdef CONFIG_DA_MON_EVENTS_ID
+DECLARE_EVENT_CLASS(event_da_monitor_id,
+
+	TP_PROTO(int id, char *state, char *event, char *next_state, bool safe),
+
+	TP_ARGS(id, state, event, next_state, safe),
+
+	TP_STRUCT__entry(
+		__field(      	 int,	id				)
+		__array(	char,	state,		MAX_DA_NAME_LEN	)
+		__array(	char,	event,		MAX_DA_NAME_LEN	)
+		__array(	char,	next_state,	MAX_DA_NAME_LEN	)
+		__field(	bool,	safe				)
+	),
+
+	TP_fast_assign(
+		memcpy(__entry->state,		state,		MAX_DA_NAME_LEN);
+		memcpy(__entry->event,		event,		MAX_DA_NAME_LEN);
+		memcpy(__entry->next_state,	next_state,	MAX_DA_NAME_LEN);
+		__entry->id			= id;
+		__entry->safe			= safe;
+	),
+
+	TP_printk("%d: %s x %s -> %s %s",
+		__entry->id,
+		__entry->state,
+		__entry->event,
+		__entry->next_state,
+		__entry->safe ? "(safe)" : "")
+);
+
+DECLARE_EVENT_CLASS(error_da_monitor_id,
+
+	TP_PROTO(int id, char *state, char *event),
+
+	TP_ARGS(id, state, event),
+
+	TP_STRUCT__entry(
+		__field(      	 int,	id				)
+		__array(	char,	state,		MAX_DA_NAME_LEN	)
+		__array(	char,	event,		MAX_DA_NAME_LEN	)
+	),
+
+	TP_fast_assign(
+		memcpy(__entry->state,		state,		MAX_DA_NAME_LEN);
+		memcpy(__entry->event,		event,		MAX_DA_NAME_LEN);
+		__entry->id			= id;
+	),
+
+	TP_printk("%d: event %s not expected in the state %s",
+		__entry->id,
+		__entry->event,
+		__entry->state)
+);
+#endif /* CONFIG_DA_MON_EVENTS_ID */
+#endif /* _TRACE_RV_H */
+
+/* This part ust be outside protection */
+#undef TRACE_INCLUDE_PATH
+#include <trace/define_trace.h>
kernel/fork.c +1 -1
index 5e40e58ef83d..6f1f82ccd5f2 100644
@@ -1970,7 +1970,7 @@ static void rv_task_fork(struct task_struct *p)
 	int i;
 
 	for (i = 0; i < RV_PER_TASK_MONITORS; i++)
-		;
+		p->rv[i].da_mon.monitoring = false;
 }
 #else
 #define rv_task_fork(p) do {} while (0)
kernel/trace/rv/Kconfig +14 -0
index 560408fec0c8..1eafb5adcfcb 100644
@@ -1,5 +1,19 @@
 # SPDX-License-Identifier: GPL-2.0-only
 #
+config DA_MON_EVENTS
+	default n
+	bool
+
+config DA_MON_EVENTS_IMPLICIT
+	select DA_MON_EVENTS
+	default n
+	bool
+
+config DA_MON_EVENTS_ID
+	select DA_MON_EVENTS
+	default n
+	bool
+
 menuconfig RV
 	bool "Runtime Verification"
 	depends on TRACING
kernel/trace/rv/rv.c +5 -0
index 7576d492a974..51a610227341 100644
@@ -143,6 +143,11 @@
 #include <linux/slab.h>
 #include <rv/rv.h>
 
+#ifdef CONFIG_DA_MON_EVENTS
+#define CREATE_TRACE_POINTS
+#include <trace/events/rv.h>
+#endif
+
 #include "rv.h"
 
 DEFINE_MUTEX(rv_interface_lock);
Signature · 1 line
2.35.1
#22 lore raw

Re: [PATCH V4 04/20] rv/include: Add deterministic automata monitor definition via C macros

Quoted 7 lines · Tao Zhou
Quoted 7 lines
include/rv/da_monitor.h new file +7 -0
+/*
+ * Entry point for the global monitor.
+ */
+#define DECLARE_DA_MON_GLOBAL(name, type)					\
+										\
+DECLARE_AUTOMATA_HELPERS(name, type);						\
+										\
Quoted 9 lines

include/rv/da_monitor.h line 385

Quoted 5 lines
include/rv/da_monitor.h new file +5 -0
+DECLARE_DA_MON_GENERIC_HELPERS(name, type);					\
+										\
+DECLARE_DA_MON_MODEL_HANDLER_IMPLICIT(name, type);				\
+										\
+DECLARE_DA_MON_INIT_PER_CPU(name, type);					\

Why the global monitor declaration use the per-cpu monitor macro. Global monitor has its own DECLARE_DA_MON_INIT_GLOBAL(name, type); Or am I miss something?

Pure miss attention from my side.

Fixed
-- Daniel

#23 lore raw

PATCH v4 5/20 rv/include: Add instrumentation helper functions

In reply to Daniel Bristot de Oliveira

Instrumentation helper functions to facilitate the instrumentation of auto-generated RV monitors create by dot2k.

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 include/rv/instrumentation.h | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
 create mode 100644 include/rv/instrumentation.h
include/rv/instrumentation.h new file +23 -0
new file mode 100644
index 000000000000..88dd33f10827
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Helper functions to facilitate the instrumentation of auto-generated
+ * RV monitors create by dot2k.
+ *
+ * The dot2k tool is available at tools/tracing/rv/dot2/
+ *
+ * Copyright (C) 2019-2022 Daniel Bristot de Oliveira <[email protected]>
+ */
+
+#include <linux/ftrace.h>
+
+#define rv_attach_trace_probe(monitor, tp, rv_handler)					\
+	do {										\
+		check_trace_callback_type_##tp(rv_handler);				\
+		WARN_ONCE(register_trace_##tp(rv_handler, NULL),			\
+				"fail attaching " #monitor " " #tp "handler");		\
+	} while (0)
+
+#define rv_detach_trace_probe(monitor, tp, rv_handler)					\
+	do {										\
+		unregister_trace_##tp(rv_handler, NULL);				\
+	} while (0)
Signature · 1 line
2.35.1
#24 lore raw

PATCH v4 6/20 tools/rv: Add dot2c

In reply to Daniel Bristot de Oliveira

dot2c is a tool that transforms an automata in the graphiviz .dot file into an C representation of the automata.

usage: dot2c [-h] dot_file

dot2c: converts a .dot file into a C structure

positional arguments:
dot_file The dot file to be converted

optional arguments:
-h, --help show this help message and exit

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 tools/verification/dot2/Makefile    |  21 +++
 tools/verification/dot2/automata.py | 179 ++++++++++++++++++++
 tools/verification/dot2/dot2c       |  30 ++++
 tools/verification/dot2/dot2c.py    | 244 ++++++++++++++++++++++++++++
 4 files changed, 474 insertions(+)
 create mode 100644 tools/verification/dot2/Makefile
 create mode 100644 tools/verification/dot2/automata.py
 create mode 100644 tools/verification/dot2/dot2c
 create mode 100644 tools/verification/dot2/dot2c.py
tools/verification/dot2/Makefile new file +21 -0
new file mode 100644
index 000000000000..235d182f6b2c
@@ -0,0 +1,21 @@
+INSTALL=install
+
+prefix  ?= /usr
+bindir  ?= $(prefix)/bin
+mandir  ?= $(prefix)/share/man
+miscdir ?= $(prefix)/share/dot2
+srcdir  ?= $(prefix)/src
+
+PYLIB  ?= $(shell python3 -c 'import sysconfig;  print (sysconfig.get_path("purelib"))')
+
+.PHONY: all
+all:
+
+.PHONY: clean
+clean:
+
+.PHONY: install
+install:
+	$(INSTALL) automata.py -D -m 644 $(DESTDIR)$(PYLIB)/dot2/automata.py
+	$(INSTALL) dot2c.py -D -m 644 $(DESTDIR)$(PYLIB)/dot2/dot2c.py
+	$(INSTALL) dot2c -D -m 755 $(DESTDIR)$(bindir)/
tools/verification/dot2/automata.py new file +179 -0

179 lines not on this page · raw message

tools/verification/dot2/dot2c new file +30 -0
new file mode 100644
index 000000000000..0165f203dedc
@@ -0,0 +1,30 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# dot2m: transform dot files into C structures.
+# For more information, see:
+#   https://bristot.me/efficient-formal-verification-for-the-linux-kernel/
+#
+# This program was written in the development of this paper:
+#  de Oliveira, D. B. and Cucinotta, T. and de Oliveira, R. S.
+#  "Efficient Formal Verification for the Linux Kernel." International
+#  Conference on Software Engineering and Formal Methods. Springer, Cham, 2019.
+#
+# Copyright 2018-2020 Red Hat, Inc.
+#
+# Author:
+#  Daniel Bristot de Oliveira <[email protected]>
+
+if __name__ == '__main__':
+    from dot2 import dot2c
+    import argparse
+    import ntpath
+    import sys
+
+    parser = argparse.ArgumentParser(description='dot2c: converts a .dot file into a C structure')
+    parser.add_argument('dot_file',  help='The dot file to be converted')
+
+
+    args = parser.parse_args()
+    d=dot2c.Dot2c(args.dot_file)
+    d.print_model_classic()
tools/verification/dot2/dot2c.py new file +244 -0

244 lines not on this page · raw message

Signature · 1 line
2.35.1
maintainer
#25 lore raw

Re: [PATCH V4 06/20] tools/rv: Add dot2c

Quoted 6 lines · On Thu, 16 Jun 2022 10:44:48 +0200 Daniel Bristot de Oliveira…
tools/verification/dot2/automata.py new file +6 -0
+    def __create_matrix(self):
+        # transform the array into a dictionary
+        events = self.events
+        states = self.states
+        events_dict = {}
+        states_dict = {}

tools/verification/dot2/automata.py line 162

Quoted 10 lines
tools/verification/dot2/automata.py new file +10 -0
+        nr_event = 0
+        for event in events:
+            events_dict[event] = nr_event
+            nr_event += 1
+
+        nr_state = 0
+        for state in states:
+            states_dict[state] = nr_state
+            nr_state = nr_state + 1
+

Hmm, do you just like inconsistency?

		nr_event += 1

		nr_state = nr_state + 1

??

-- Steve

maintainer
#26 lore raw

Re: [PATCH V4 06/20] tools/rv: Add dot2c

In reply to Daniel Bristot de Oliveira

tools/verification/dot2/dot2c.py line 94

Quoted 8 lines · On Thu, 16 Jun 2022 10:44:48 +0200 Daniel Bristot de Oliveira…
tools/verification/dot2/dot2c.py new file +8 -0
+    def get_minimun_type(self):
+        min_type="char"
+
+        if self.states.__len__() > 255:
+            min_type="short"
+
+        if self.states.__len__() > 65535:
+            min_type="int"

Should these be "unsigned"? As char goes from -128 <-> 127 and short goes from -32768 <-> 32767

And are you sure you're not going to have more than 2,147,483,647 states ;-) (or 4,294,967,296 if it's unsigned).

-- Steve

Quoted 3 lines
tools/verification/dot2/dot2c.py new file +3 -0
+
+        return min_type
+
#27 lore raw

Re: [PATCH V4 06/20] tools/rv: Add dot2c

Quoted 14 lines · Steven Rostedt

tools/verification/dot2/dot2c.py line 94

Quoted 8 lines · On Thu, 16 Jun 2022 10:44:48 +0200 Daniel Bristot de Oliveira…
tools/verification/dot2/dot2c.py new file +8 -0
+    def get_minimun_type(self):
+        min_type="char"
+
+        if self.states.__len__() > 255:
+            min_type="short"
+
+        if self.states.__len__() > 65535:
+            min_type="int"

Should these be "unsigned"? As char goes from -128 <-> 127 and short goes from -32768 <-> 32767

Quoted 2 lines

And are you sure you're not going to have more than 2,147,483,647 states ;-) (or 4,294,967,296 if it's unsigned).

Fixed! now I am using the "state_max" as the invalid state, so I can use unsigned.

I also revisited all python code for consistency.

-- Daniel

#28 lore raw

PATCH v4 7/20 tools/rv: Add dot2k

In reply to Daniel Bristot de Oliveira

transform .dot file into kernel rv monitor

usage: dot2k [-h] -d DOT_FILE -t MONITOR_TYPE [-n MODEL_NAME] [-D DESCRIPTION]

optional arguments:
-h, --help show this help message and exit
-d DOT_FILE, --dot DOT_FILE
-t MONITOR_TYPE, --monitor_type MONITOR_TYPE
-n MODEL_NAME, --model_name MODEL_NAME
-D DESCRIPTION, --description DESCRIPTION

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 tools/verification/dot2/Makefile              |   5 +
 tools/verification/dot2/dot2k                 |  50 +++++
 tools/verification/dot2/dot2k.py              | 177 ++++++++++++++++++
 .../dot2/dot2k_templates/main_global.c        |  94 ++++++++++
 .../dot2/dot2k_templates/main_per_cpu.c       |  94 ++++++++++
 .../dot2/dot2k_templates/main_per_task.c      |  94 ++++++++++
 6 files changed, 514 insertions(+)
 create mode 100644 tools/verification/dot2/dot2k
 create mode 100644 tools/verification/dot2/dot2k.py
 create mode 100644 tools/verification/dot2/dot2k_templates/main_global.c
 create mode 100644 tools/verification/dot2/dot2k_templates/main_per_cpu.c
 create mode 100644 tools/verification/dot2/dot2k_templates/main_per_task.c
tools/verification/dot2/Makefile +5 -0
index 235d182f6b2c..021beb07a521 100644
@@ -19,3 +19,8 @@ install:
 	$(INSTALL) automata.py -D -m 644 $(DESTDIR)$(PYLIB)/dot2/automata.py
 	$(INSTALL) dot2c.py -D -m 644 $(DESTDIR)$(PYLIB)/dot2/dot2c.py
 	$(INSTALL) dot2c -D -m 755 $(DESTDIR)$(bindir)/
+	$(INSTALL) dot2k.py -D -m 644 $(DESTDIR)$(PYLIB)/dot2/dot2k.py
+	$(INSTALL) dot2k -D -m 755 $(DESTDIR)$(bindir)/
+
+	mkdir -p ${miscdir}/
+	cp -rp dot2k_templates $(DESTDIR)$(miscdir)/
tools/verification/dot2/dot2k new file +50 -0

50 lines not on this page · raw message

tools/verification/dot2/dot2k.py new file +177 -0

177 lines not on this page · raw message

tools/verification/dot2/dot2k_templates/main_global.c new file +94 -0

94 lines not on this page · raw message

tools/verification/dot2/dot2k_templates/main_per_cpu.c new file +94 -0

94 lines not on this page · raw message

tools/verification/dot2/dot2k_templates/main_per_task.c new file +94 -0

94 lines not on this page · raw message

Signature · 1 line
2.35.1
#29 lore raw

PATCH v4 8/20 rv/monitor: Add the wip monitor skeleton created by dot2k

In reply to Daniel Bristot de Oliveira

This is the direct output this command line:

  $ dot2k -d ~/wip.dot -t per_cpu

with wip.dot as:
----- %< -----

digraph state_automaton {
	center = true;
	size = "7,11";
	rankdir = LR;
	{node [shape = circle] "non_preemptive"};
	{node [shape = plaintext, style=invis, label=""] "__init_preemptive"};
	{node [shape = doublecircle] "preemptive"};
	{node [shape = circle] "preemptive"};
	"__init_preemptive" -> "preemptive";
	"non_preemptive" [label = "non_preemptive"];
	"non_preemptive" -> "non_preemptive" [ label = "sched_waking" ];
	"non_preemptive" -> "preemptive" [ label = "preempt_enable" ];
	"preemptive" [label = "preemptive"];
	"preemptive" -> "non_preemptive" [ label = "preempt_disable" ];
	{ rank = min ;
		"__init_preemptive";
		"preemptive";
	}
}

----- >% -----

This model is broken because preempt_disable_notrace(). It is broken on purpose to test the reactors.

It does not compile because it lacks the instrumentation, which will be add next.

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 kernel/trace/rv/monitors/wip/wip.c | 112 +++++++++++++++++++++++++++++
 kernel/trace/rv/monitors/wip/wip.h |  38 ++++++++++
 2 files changed, 150 insertions(+)
 create mode 100644 kernel/trace/rv/monitors/wip/wip.c
 create mode 100644 kernel/trace/rv/monitors/wip/wip.h
kernel/trace/rv/monitors/wip/wip.c new file +112 -0
new file mode 100644
index 000000000000..08950e09b823
@@ -0,0 +1,112 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/ftrace.h>
+#include <linux/tracepoint.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/rv.h>
+#include <rv/instrumentation.h>
+#include <rv/da_monitor.h>
+
+#define MODULE_NAME "wip"
+
+/*
+ * XXX: include required tracepoint headers, e.g.,
+ * #include <linux/trace/events/sched.h>
+ */
+#include <trace/events/rv.h>
+
+/*
+ * This is the self-generated part of the monitor. Generally, there is no need
+ * to touch this section.
+ */
+#include "wip.h"
+
+/*
+ * Declare the deterministic automata monitor.
+ *
+ * The rv monitor reference is needed for the monitor declaration.
+ */
+struct rv_monitor rv_wip;
+DECLARE_DA_MON_PER_CPU(wip, char);
+
+/*
+ * This is the instrumentation part of the monitor.
+ *
+ * This is the section where manual work is required. Here the kernel events
+ * are translated into model's event.
+ *
+ */
+static void handle_preempt_disable(void *data, /* XXX: fill header */)
+{
+	da_handle_event_wip(preempt_disable_wip);
+}
+
+static void handle_preempt_enable(void *data, /* XXX: fill header */)
+{
+	da_handle_event_wip(preempt_enable_wip);
+}
+
+static void handle_sched_waking(void *data, /* XXX: fill header */)
+{
+	da_handle_event_wip(sched_waking_wip);
+}
+
+static int start_wip(void)
+{
+	int retval;
+
+	retval = da_monitor_init_wip();
+	if (retval)
+		return retval;
+
+	rv_attach_trace_probe("wip", /* XXX: tracepoint */, handle_preempt_disable);
+	rv_attach_trace_probe("wip", /* XXX: tracepoint */, handle_preempt_enable);
+	rv_attach_trace_probe("wip", /* XXX: tracepoint */, handle_sched_waking);
+
+	return 0;
+}
+
+static void stop_wip(void)
+{
+	rv_wip.enabled = 0;
+
+	rv_detach_trace_probe("wip", /* XXX: tracepoint */, handle_preempt_disable);
+	rv_detach_trace_probe("wip", /* XXX: tracepoint */, handle_preempt_enable);
+	rv_detach_trace_probe("wip", /* XXX: tracepoint */, handle_sched_waking);
+
+	da_monitor_destroy_wip();
+}
+
+/*
+ * This is the monitor register section.
+ */
+struct rv_monitor rv_wip = {
+	.name = "wip",
+	.description = "auto-generated wip",
+	.start = start_wip,
+	.stop = stop_wip,
+	.reset = da_monitor_reset_all_wip,
+	.enabled = 0,
+};
+
+int register_wip(void)
+{
+	rv_register_monitor(&rv_wip);
+	return 0;
+}
+
+void unregister_wip(void)
+{
+	if (rv_wip.enabled)
+		stop_wip();
+
+	rv_unregister_monitor(&rv_wip);
+}
+
+module_init(register_wip);
+module_exit(unregister_wip);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("dot2k: auto-generated");
+MODULE_DESCRIPTION("wip");
kernel/trace/rv/monitors/wip/wip.h new file +38 -0
new file mode 100644
index 000000000000..3055ea331706
@@ -0,0 +1,38 @@
+enum states_wip {
+	preemptive_wip = 0,
+	non_preemptive_wip,
+	state_max_wip
+};
+
+enum events_wip {
+	preempt_disable_wip = 0,
+	preempt_enable_wip,
+	sched_waking_wip,
+	event_max_wip
+};
+
+struct automaton_wip {
+	char *state_names[state_max_wip];
+	char *event_names[event_max_wip];
+	char function[state_max_wip][event_max_wip];
+	char initial_state;
+	char final_states[state_max_wip];
+};
+
+struct automaton_wip automaton_wip = {
+	.state_names = {
+		"preemptive",
+		"non_preemptive"
+	},
+	.event_names = {
+		"preempt_disable",
+		"preempt_enable",
+		"sched_waking"
+	},
+	.function = {
+		{ non_preemptive_wip,                 -1,                 -1 },
+		{                 -1,     preemptive_wip, non_preemptive_wip },
+	},
+	.initial_state = preemptive_wip,
+	.final_states = { 1, 0 },
+};
Signature · 1 line
2.35.1
maintainer
#30 lore raw

Re: [PATCH V4 08/20] rv/monitor: Add the wip monitor skeleton created by dot2k

Quoted 4 lines · On Thu, 16 Jun 2022 10:44:50 +0200 Daniel Bristot de Oliveira…

This is the direct output this command line:

  $ dot2k -d ~/wip.dot -t per_cpu

with wip.dot as:

So you created the "Work In Progress" monitor?

Is that what wip stands for?

-- Steve

#31 lore raw

PATCH v4 9/20 rv/monitor: wip instrumentation and Makefile/Kconfig entries

In reply to Daniel Bristot de Oliveira

Adds the instrumentation to the previously created wip monitor, as an example of the developer work. It also adds a Makefile, Kconfig and tracepoint entries.

This is a good example of the manual work that is left for the developer to do.

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 include/trace/events/rv.h          | 10 ++++++++++
 kernel/trace/rv/Kconfig            |  7 +++++++
 kernel/trace/rv/Makefile           |  1 +
 kernel/trace/rv/monitors/wip/wip.c | 30 ++++++++++++++----------------
 4 files changed, 32 insertions(+), 16 deletions(-)
include/trace/events/rv.h +10 -0
index 9f40f2a49f84..4e0dabffcf29 100644
@@ -56,6 +56,16 @@ DECLARE_EVENT_CLASS(error_da_monitor,
 		__entry->event,
 		__entry->state)
 );
+
+#ifdef CONFIG_RV_MON_WIP
+DEFINE_EVENT(event_da_monitor, event_wip,
+	    TP_PROTO(char *state, char *event, char *next_state, bool safe),
+	    TP_ARGS(state, event, next_state, safe));
+
+DEFINE_EVENT(error_da_monitor, error_wip,
+	     TP_PROTO(char *state, char *event),
+	     TP_ARGS(state, event));
+#endif /* CONFIG_RV_MON_WIP */
 #endif /* CONFIG_DA_MON_EVENTS_IMPLICIT */
 
 #ifdef CONFIG_DA_MON_EVENTS_ID
kernel/trace/rv/Kconfig +7 -0
index 1eafb5adcfcb..e9246b0bec9d 100644
@@ -26,6 +26,13 @@ menuconfig RV
 	  the system behavior.
 
 if RV
+config RV_MON_WIP
+	depends on PREEMPTIRQ_TRACEPOINTS
+	select DA_MON_EVENTS_IMPLICIT
+	bool "WIP monitor"
+	help
+	  Enable WIP sample monitor, this is a sample monitor that
+	  illustrates the usage of per-cpu monitors.
 
 config RV_REACTORS
 	bool "Runtime verification reactors"
kernel/trace/rv/Makefile +1 -0
index 8944274d9b41..b41109d2750a 100644
@@ -2,3 +2,4 @@
 
 obj-$(CONFIG_RV) += rv.o
 obj-$(CONFIG_RV_REACTORS) += rv_reactors.o
+obj-$(CONFIG_RV_MON_WIP) += monitors/wip/wip.o
kernel/trace/rv/monitors/wip/wip.c +14 -16
index 08950e09b823..123827db0a03 100644
@@ -10,11 +10,9 @@
 
 #define MODULE_NAME "wip"
 
-/*
- * XXX: include required tracepoint headers, e.g.,
- * #include <linux/trace/events/sched.h>
- */
 #include <trace/events/rv.h>
+#include <trace/events/sched.h>
+#include <trace/events/preemptirq.h>
 
 /*
  * This is the self-generated part of the monitor. Generally, there is no need
@@ -37,17 +35,17 @@ DECLARE_DA_MON_PER_CPU(wip, char);
  * are translated into model's event.
  *
  */
-static void handle_preempt_disable(void *data, /* XXX: fill header */)
+static void handle_preempt_disable(void *data, unsigned long ip, unsigned long parent_ip)
 {
 	da_handle_event_wip(preempt_disable_wip);
 }
 
-static void handle_preempt_enable(void *data, /* XXX: fill header */)
+static void handle_preempt_enable(void *data, unsigned long ip, unsigned long parent_ip)
 {
-	da_handle_event_wip(preempt_enable_wip);
+	da_handle_init_event_wip(preempt_enable_wip);
 }
 
-static void handle_sched_waking(void *data, /* XXX: fill header */)
+static void handle_sched_waking(void *data, struct task_struct *task)
 {
 	da_handle_event_wip(sched_waking_wip);
 }
@@ -60,9 +58,9 @@ static int start_wip(void)
 	if (retval)
 		return retval;
 
-	rv_attach_trace_probe("wip", /* XXX: tracepoint */, handle_preempt_disable);
-	rv_attach_trace_probe("wip", /* XXX: tracepoint */, handle_preempt_enable);
-	rv_attach_trace_probe("wip", /* XXX: tracepoint */, handle_sched_waking);
+	rv_attach_trace_probe("wip", preempt_disable, handle_preempt_disable);
+	rv_attach_trace_probe("wip", preempt_enable, handle_preempt_enable);
+	rv_attach_trace_probe("wip", sched_waking, handle_sched_waking);
 
 	return 0;
 }
@@ -71,9 +69,9 @@ static void stop_wip(void)
 {
 	rv_wip.enabled = 0;
 
-	rv_detach_trace_probe("wip", /* XXX: tracepoint */, handle_preempt_disable);
-	rv_detach_trace_probe("wip", /* XXX: tracepoint */, handle_preempt_enable);
-	rv_detach_trace_probe("wip", /* XXX: tracepoint */, handle_sched_waking);
+	rv_detach_trace_probe("wip", preempt_disable, handle_preempt_disable);
+	rv_detach_trace_probe("wip", preempt_enable, handle_preempt_enable);
+	rv_detach_trace_probe("wip", sched_waking, handle_sched_waking);
 
 	da_monitor_destroy_wip();
 }
@@ -108,5 +106,5 @@ module_init(register_wip);
 module_exit(unregister_wip);
 
 MODULE_LICENSE("GPL");
-MODULE_AUTHOR("dot2k: auto-generated");
-MODULE_DESCRIPTION("wip");
+MODULE_AUTHOR("Daniel Bristot de Oliveira <[email protected]>");
+MODULE_DESCRIPTION("wip: wakeup in preemptive - per-cpu sample monitor.");
Signature · 1 line
2.35.1
#32 lore raw

Re: [PATCH V4 09/20] rv/monitor: wip instrumentation and Makefile/Kconfig entries

I love your patch! Perhaps something to improve:

Hi Daniel,

I love your patch! Perhaps something to improve:

[auto build test WARNING on rostedt-trace/for-next]
[also build test WARNING on tip/sched/core]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/intel-lab-lkp/linux/commits/Daniel-Bristot-de-Oliveira/The-Runtime-Verification-RV-interface/20220616-164837
base:   https://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace.git for-next
config: riscv-randconfig-r004-20220616 (https://download.01.org/0day-ci/archive/20220616/[email protected]/config)
compiler: riscv64-linux-gcc (GCC) 11.3.0

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/a76032130de277ff6f39fc08277a70823fd85bf3
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Daniel-Bristot-de-Oliveira/The-Runtime-Verification-RV-interface/20220616-164837 git checkout a76032130de277ff6f39fc08277a70823fd85bf3
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.3.0 make.cross W=1 O=build_dir ARCH=riscv SHELL=/bin/bash kernel/trace/rv/

If you fix the issue, kindly add following tag where applicable

All warnings (new ones prefixed by >>):

In file included from kernel/trace/rv/monitors/wip/wip.c:8:
kernel/trace/rv/monitors/wip/wip.c: In function 'start_wip':
include/rv/instrumentation.h:15:17: error: implicit declaration of function 'check_trace_callback_type_preempt_disable'; did you mean 'check_trace_callback_type_irq_disable'? [-Werror=implicit-function-declaration]

      15 |                 check_trace_callback_type_##tp(rv_handler);                             \
         |                 ^~~~~~~~~~~~~~~~~~~~~~~~~~
   kernel/trace/rv/monitors/wip/wip.c:61:9: note: in expansion of macro 'rv_attach_trace_probe'
      61 |         rv_attach_trace_probe("wip", preempt_disable, handle_preempt_disable);
         |         ^~~~~~~~~~~~~~~~~~~~~
   In file included from include/linux/printk.h:11,
                    from include/linux/kernel.h:29,
                    from include/linux/interrupt.h:6,
                    from include/linux/trace_recursion.h:5,
                    from include/linux/ftrace.h:10,
                    from kernel/trace/rv/monitors/wip/wip.c:2:
   include/rv/instrumentation.h:16:27: error: implicit declaration of function 'register_trace_preempt_disable'; did you mean 'register_trace_prio_irq_disable'? [-Werror=implicit-function-declaration]
      16 |                 WARN_ONCE(register_trace_##tp(rv_handler, NULL),                        \
         |                           ^~~~~~~~~~~~~~~
   include/linux/once_lite.h:15:41: note: in definition of macro 'DO_ONCE_LITE_IF'
      15 |                 bool __ret_do_once = !!(condition);                     \
         |                                         ^~~~~~~~~
   include/rv/instrumentation.h:16:17: note: in expansion of macro 'WARN_ONCE'
      16 |                 WARN_ONCE(register_trace_##tp(rv_handler, NULL),                        \
         |                 ^~~~~~~~~
   kernel/trace/rv/monitors/wip/wip.c:61:9: note: in expansion of macro 'rv_attach_trace_probe'
      61 |         rv_attach_trace_probe("wip", preempt_disable, handle_preempt_disable);
         |         ^~~~~~~~~~~~~~~~~~~~~
   In file included from kernel/trace/rv/monitors/wip/wip.c:8:
   include/rv/instrumentation.h:15:17: error: implicit declaration of function 'check_trace_callback_type_preempt_enable'; did you mean 'check_trace_callback_type_irq_enable'? [-Werror=implicit-function-declaration]
      15 |                 check_trace_callback_type_##tp(rv_handler);                             \
         |                 ^~~~~~~~~~~~~~~~~~~~~~~~~~
   kernel/trace/rv/monitors/wip/wip.c:62:9: note: in expansion of macro 'rv_attach_trace_probe'
      62 |         rv_attach_trace_probe("wip", preempt_enable, handle_preempt_enable);
         |         ^~~~~~~~~~~~~~~~~~~~~
   In file included from include/linux/printk.h:11,
                    from include/linux/kernel.h:29,
                    from include/linux/interrupt.h:6,
                    from include/linux/trace_recursion.h:5,
                    from include/linux/ftrace.h:10,
                    from kernel/trace/rv/monitors/wip/wip.c:2:
   include/rv/instrumentation.h:16:27: error: implicit declaration of function 'register_trace_preempt_enable'; did you mean 'register_trace_prio_irq_enable'? [-Werror=implicit-function-declaration]
      16 |                 WARN_ONCE(register_trace_##tp(rv_handler, NULL),                        \
         |                           ^~~~~~~~~~~~~~~
   include/linux/once_lite.h:15:41: note: in definition of macro 'DO_ONCE_LITE_IF'
      15 |                 bool __ret_do_once = !!(condition);                     \
         |                                         ^~~~~~~~~
   include/rv/instrumentation.h:16:17: note: in expansion of macro 'WARN_ONCE'
      16 |                 WARN_ONCE(register_trace_##tp(rv_handler, NULL),                        \
         |                 ^~~~~~~~~
   kernel/trace/rv/monitors/wip/wip.c:62:9: note: in expansion of macro 'rv_attach_trace_probe'
      62 |         rv_attach_trace_probe("wip", preempt_enable, handle_preempt_enable);
         |         ^~~~~~~~~~~~~~~~~~~~~
   In file included from kernel/trace/rv/monitors/wip/wip.c:8:
   kernel/trace/rv/monitors/wip/wip.c: In function 'stop_wip':
   include/rv/instrumentation.h:22:17: error: implicit declaration of function 'unregister_trace_preempt_disable'; did you mean 'unregister_trace_irq_disable'? [-Werror=implicit-function-declaration]
      22 |                 unregister_trace_##tp(rv_handler, NULL);                                \
         |                 ^~~~~~~~~~~~~~~~~
   kernel/trace/rv/monitors/wip/wip.c:72:9: note: in expansion of macro 'rv_detach_trace_probe'
      72 |         rv_detach_trace_probe("wip", preempt_disable, handle_preempt_disable);
         |         ^~~~~~~~~~~~~~~~~~~~~
   include/rv/instrumentation.h:22:17: error: implicit declaration of function 'unregister_trace_preempt_enable'; did you mean 'unregister_trace_irq_enable'? [-Werror=implicit-function-declaration]
      22 |                 unregister_trace_##tp(rv_handler, NULL);                                \
         |                 ^~~~~~~~~~~~~~~~~
   kernel/trace/rv/monitors/wip/wip.c:73:9: note: in expansion of macro 'rv_detach_trace_probe'
      73 |         rv_detach_trace_probe("wip", preempt_enable, handle_preempt_enable);
         |         ^~~~~~~~~~~~~~~~~~~~~
   kernel/trace/rv/monitors/wip/wip.c: At top level:
>> kernel/trace/rv/monitors/wip/wip.c:91:5: warning: no previous prototype for 'register_wip' [-Wmissing-prototypes]
      91 | int register_wip(void)
         |     ^~~~~~~~~~~~
>> kernel/trace/rv/monitors/wip/wip.c:97:6: warning: no previous prototype for 'unregister_wip' [-Wmissing-prototypes]

97 | void unregister_wip(void)
| ^~~~~~~~~~~~~~
cc1: some warnings being treated as errors

vim +/register_wip +91 kernel/trace/rv/monitors/wip/wip.c

15679ffb99664f Daniel Bristot de Oliveira 2022-06-16 90

15679ffb99664f Daniel Bristot de Oliveira 2022-06-16  @91  int register_wip(void)
15679ffb99664f Daniel Bristot de Oliveira 2022-06-16   92  {
15679ffb99664f Daniel Bristot de Oliveira 2022-06-16   93  	rv_register_monitor(&rv_wip);
15679ffb99664f Daniel Bristot de Oliveira 2022-06-16   94  	return 0;
15679ffb99664f Daniel Bristot de Oliveira 2022-06-16   95  }
15679ffb99664f Daniel Bristot de Oliveira 2022-06-16   96  
15679ffb99664f Daniel Bristot de Oliveira 2022-06-16  @97  void unregister_wip(void)
15679ffb99664f Daniel Bristot de Oliveira 2022-06-16   98  {
15679ffb99664f Daniel Bristot de Oliveira 2022-06-16   99  	if (rv_wip.enabled)
15679ffb99664f Daniel Bristot de Oliveira 2022-06-16  100  		stop_wip();
15679ffb99664f Daniel Bristot de Oliveira 2022-06-16  101  
15679ffb99664f Daniel Bristot de Oliveira 2022-06-16  102  	rv_unregister_monitor(&rv_wip);
15679ffb99664f Daniel Bristot de Oliveira 2022-06-16  103  }
15679ffb99664f Daniel Bristot de Oliveira 2022-06-16  104  
Signature · 2 lines
0-DAY CI Kernel Test Service
https://01.org/lkp
reviewer
#33 lore raw

Re: [PATCH V4 09/20] rv/monitor: wip instrumentation and Makefile/Kconfig entries

In reply to Daniel Bristot de Oliveira

kernel/trace/rv/Kconfig menuconfig RV · line 32

Quoted 12 lines · Daniel Bristot de Oliveira
kernel/trace/rv/Kconfig +4 -0
index 1eafb5adcfcb..e9246b0bec9d 100644
@@ -26,6 +26,13 @@ menuconfig RV
 	  the system behavior.
 
 if RV
+config RV_MON_WIP
+	depends on PREEMPTIRQ_TRACEPOINTS
+	select DA_MON_EVENTS_IMPLICIT
+	bool "WIP monitor"

Does WIP mean work-in-progress? (It does to me.)

If not, please explain what it means in the help text.

kernel/trace/rv/Kconfig menuconfig RV · line 35

Quoted 3 lines
kernel/trace/rv/Kconfig +3 -0
+	help
+	  Enable WIP sample monitor, this is a sample monitor that
+	  illustrates the usage of per-cpu monitors.

thanks.

Signature · 1 line
~Randy
#34 lore raw

Re: [PATCH V4 09/20] rv/monitor: wip instrumentation and Makefile/Kconfig entries

Quoted 9 lines · Randy Dunlap
Quoted 7 lines · Daniel Bristot de Oliveira
kernel/trace/rv/Kconfig +0 -0
index 1eafb5adcfcb..e9246b0bec9d 100644
@@ -26,6 +26,13 @@ menuconfig RV
 	  the system behavior.
 
Quoted 9 lines

kernel/trace/rv/Kconfig menuconfig RV · line 32

Quoted 5 lines
kernel/trace/rv/Kconfig +4 -0
 if RV
+config RV_MON_WIP
+	depends on PREEMPTIRQ_TRACEPOINTS
+	select DA_MON_EVENTS_IMPLICIT
+	bool "WIP monitor"

Does WIP mean work-in-progress? (It does to me.)

If not, please explain what it means in the help text.

Wakeup in preemptive. I will add it to the description.

Thanks Randy!
-- Daniel

maintainer
#35 lore raw

Re: [PATCH V4 09/20] rv/monitor: wip instrumentation and Makefile/Kconfig entries

In reply to Randy Dunlap
Quoted 16 lines · On Thu, 16 Jun 2022 14:00:10 -0700 Randy Dunlap <[email protected]> wrote:

kernel/trace/rv/Kconfig menuconfig RV · line 32

Quoted 12 lines · Daniel Bristot de Oliveira
kernel/trace/rv/Kconfig +4 -0
index 1eafb5adcfcb..e9246b0bec9d 100644
@@ -26,6 +26,13 @@ menuconfig RV
 	  the system behavior.
 
 if RV
+config RV_MON_WIP
+	depends on PREEMPTIRQ_TRACEPOINTS
+	select DA_MON_EVENTS_IMPLICIT
+	bool "WIP monitor"  

Does WIP mean work-in-progress? (It does to me.)

Quoted 1 line

If not, please explain what it means in the help text.

Yeah, that's exactly what I thought too ;-)

-- Steve

Quoted 6 lines

kernel/trace/rv/Kconfig menuconfig RV · line 35

Quoted 4 lines
kernel/trace/rv/Kconfig +3 -0
+	help
+	  Enable WIP sample monitor, this is a sample monitor that
+	  illustrates the usage of per-cpu monitors.  

thanks.

#36 lore raw

PATCH v4 10/20 rv/monitor: Add the wwnr monitor skeleton created by dot2k

In reply to Daniel Bristot de Oliveira

Per task wakeup while not running (wwnr) monitor, generated by dot2k with this command line:

  $ dot2k -d wwnr.dot -t per_task

The model is:
----- %< -----

digraph state_automaton {
	center = true;
	size = "7,11";
	{node [shape = plaintext, style=invis, label=""] "__init_not_running"};
	{node [shape = ellipse] "not_running"};
	{node [shape = plaintext] "not_running"};
	{node [shape = plaintext] "running"};
	"__init_not_running" -> "not_running";
	"not_running" [label = "not_running", color = green3];
	"not_running" -> "not_running" [ label = "wakeup" ];
	"not_running" -> "running" [ label = "switch_in" ];
	"running" [label = "running"];
	"running" -> "not_running" [ label = "switch_out" ];
	{ rank = min ;
		"__init_not_running";
		"not_running";
	}
}

----- >% -----

This model is broken, the reason is that a task can be running in the processor without being set as RUNNABLE. Think about a task about to sleep:

1: set_current_state(TASK_UNINTERRUPTIBLE);
2: schedule();

And then imagine an IRQ happening in between the lines one and two, waking the task up. BOOM, the wakeup will happen while the task is running.

Q: Why do we need this model, so?
A: To test the reactors.

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 kernel/trace/rv/monitors/wwnr/wwnr.c | 115 +++++++++++++++++++++++++++
 kernel/trace/rv/monitors/wwnr/wwnr.h |  38 +++++++++
 2 files changed, 153 insertions(+)
 create mode 100644 kernel/trace/rv/monitors/wwnr/wwnr.c
 create mode 100644 kernel/trace/rv/monitors/wwnr/wwnr.h
kernel/trace/rv/monitors/wwnr/wwnr.c new file +115 -0
new file mode 100644
index 000000000000..8ba01f0f0df8
@@ -0,0 +1,115 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/ftrace.h>
+#include <linux/tracepoint.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/rv.h>
+#include <rv/instrumentation.h>
+#include <rv/da_monitor.h>
+
+#define MODULE_NAME "wwnr"
+
+/*
+ * XXX: include required tracepoint headers, e.g.,
+ * #include <linux/trace/events/sched.h>
+ */
+#include <trace/events/rv.h>
+
+/*
+ * This is the self-generated part of the monitor. Generally, there is no need
+ * to touch this section.
+ */
+#include "wwnr.h"
+
+/*
+ * Declare the deterministic automata monitor.
+ *
+ * The rv monitor reference is needed for the monitor declaration.
+ */
+struct rv_monitor rv_wwnr;
+DECLARE_DA_MON_PER_TASK(wwnr, char);
+
+/*
+ * This is the instrumentation part of the monitor.
+ *
+ * This is the section where manual work is required. Here the kernel events
+ * are translated into model's event.
+ *
+ */
+static void handle_switch_in(void *data, /* XXX: fill header */)
+{
+	struct task_struct *p = /* XXX: how do I get p? */;
+	da_handle_event_wwnr(p, switch_in_wwnr);
+}
+
+static void handle_switch_out(void *data, /* XXX: fill header */)
+{
+	struct task_struct *p = /* XXX: how do I get p? */;
+	da_handle_event_wwnr(p, switch_out_wwnr);
+}
+
+static void handle_wakeup(void *data, /* XXX: fill header */)
+{
+	struct task_struct *p = /* XXX: how do I get p? */;
+	da_handle_event_wwnr(p, wakeup_wwnr);
+}
+
+static int start_wwnr(void)
+{
+	int retval;
+
+	retval = da_monitor_init_wwnr();
+	if (retval)
+		return retval;
+
+	rv_attach_trace_probe("wwnr", /* XXX: tracepoint */, handle_switch_in);
+	rv_attach_trace_probe("wwnr", /* XXX: tracepoint */, handle_switch_out);
+	rv_attach_trace_probe("wwnr", /* XXX: tracepoint */, handle_wakeup);
+
+	return 0;
+}
+
+static void stop_wwnr(void)
+{
+	rv_wwnr.enabled = 0;
+
+	rv_detach_trace_probe("wwnr", /* XXX: tracepoint */, handle_switch_in);
+	rv_detach_trace_probe("wwnr", /* XXX: tracepoint */, handle_switch_out);
+	rv_detach_trace_probe("wwnr", /* XXX: tracepoint */, handle_wakeup);
+
+	da_monitor_destroy_wwnr();
+}
+
+/*
+ * This is the monitor register section.
+ */
+struct rv_monitor rv_wwnr = {
+	.name = "wwnr",
+	.description = "auto-generated wwnr",
+	.start = start_wwnr,
+	.stop = stop_wwnr,
+	.reset = da_monitor_reset_all_wwnr,
+	.enabled = 0,
+};
+
+int register_wwnr(void)
+{
+	rv_register_monitor(&rv_wwnr);
+	return 0;
+}
+
+void unregister_wwnr(void)
+{
+	if (rv_wwnr.enabled)
+		stop_wwnr();
+
+	rv_unregister_monitor(&rv_wwnr);
+}
+
+module_init(register_wwnr);
+module_exit(unregister_wwnr);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("dot2k: auto-generated");
+MODULE_DESCRIPTION("wwnr");
kernel/trace/rv/monitors/wwnr/wwnr.h new file +38 -0
new file mode 100644
index 000000000000..f3dc160642bf
@@ -0,0 +1,38 @@
+enum states_wwnr {
+	not_running_wwnr = 0,
+	running_wwnr,
+	state_max_wwnr
+};
+
+enum events_wwnr {
+	switch_in_wwnr = 0,
+	switch_out_wwnr,
+	wakeup_wwnr,
+	event_max_wwnr
+};
+
+struct automaton_wwnr {
+	char *state_names[state_max_wwnr];
+	char *event_names[event_max_wwnr];
+	char function[state_max_wwnr][event_max_wwnr];
+	char initial_state;
+	char final_states[state_max_wwnr];
+};
+
+struct automaton_wwnr automaton_wwnr = {
+	.state_names = {
+		"not_running",
+		"running"
+	},
+	.event_names = {
+		"switch_in",
+		"switch_out",
+		"wakeup"
+	},
+	.function = {
+		{     running_wwnr,               -1, not_running_wwnr },
+		{               -1, not_running_wwnr,               -1 },
+	},
+	.initial_state = not_running_wwnr,
+	.final_states = { 1, 0 },
+};
Signature · 1 line
2.35.1
#37 lore raw

Re: [PATCH V4 10/20] rv/monitor: Add the wwnr monitor skeleton created by dot2k

Quoted 216 lines · Daniel Bristot de Oliveira

Per task wakeup while not running (wwnr) monitor, generated by dot2k with this command line:

  $ dot2k -d wwnr.dot -t per_task

The model is:
----- %< -----

digraph state_automaton {
	center = true;
	size = "7,11";
	{node [shape = plaintext, style=invis, label=""] "__init_not_running"};
	{node [shape = ellipse] "not_running"};
	{node [shape = plaintext] "not_running"};
	{node [shape = plaintext] "running"};
	"__init_not_running" -> "not_running";
	"not_running" [label = "not_running", color = green3];
	"not_running" -> "not_running" [ label = "wakeup" ];
	"not_running" -> "running" [ label = "switch_in" ];
	"running" [label = "running"];
	"running" -> "not_running" [ label = "switch_out" ];
	{ rank = min ;
		"__init_not_running";
		"not_running";
	}
}

----- >% -----

This model is broken, the reason is that a task can be running in the processor without being set as RUNNABLE. Think about a task about to sleep:

1: set_current_state(TASK_UNINTERRUPTIBLE);
2: schedule();

And then imagine an IRQ happening in between the lines one and two, waking the task up. BOOM, the wakeup will happen while the task is running.

Q: Why do we need this model, so?
A: To test the reactors.

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 kernel/trace/rv/monitors/wwnr/wwnr.c | 115 +++++++++++++++++++++++++++
 kernel/trace/rv/monitors/wwnr/wwnr.h |  38 +++++++++
 2 files changed, 153 insertions(+)
 create mode 100644 kernel/trace/rv/monitors/wwnr/wwnr.c
 create mode 100644 kernel/trace/rv/monitors/wwnr/wwnr.h
kernel/trace/rv/monitors/wwnr/wwnr.c new file +115 -0
new file mode 100644
index 000000000000..8ba01f0f0df8
@@ -0,0 +1,115 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/ftrace.h>
+#include <linux/tracepoint.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/rv.h>
+#include <rv/instrumentation.h>
+#include <rv/da_monitor.h>
+
+#define MODULE_NAME "wwnr"
+
+/*
+ * XXX: include required tracepoint headers, e.g.,
+ * #include <linux/trace/events/sched.h>
+ */
+#include <trace/events/rv.h>
+
+/*
+ * This is the self-generated part of the monitor. Generally, there is no need
+ * to touch this section.
+ */
+#include "wwnr.h"
+
+/*
+ * Declare the deterministic automata monitor.
+ *
+ * The rv monitor reference is needed for the monitor declaration.
+ */
+struct rv_monitor rv_wwnr;
+DECLARE_DA_MON_PER_TASK(wwnr, char);
+
+/*
+ * This is the instrumentation part of the monitor.
+ *
+ * This is the section where manual work is required. Here the kernel events
+ * are translated into model's event.
+ *
+ */
+static void handle_switch_in(void *data, /* XXX: fill header */)
+{
+	struct task_struct *p = /* XXX: how do I get p? */;
+	da_handle_event_wwnr(p, switch_in_wwnr);
+}
+
+static void handle_switch_out(void *data, /* XXX: fill header */)
+{
+	struct task_struct *p = /* XXX: how do I get p? */;
+	da_handle_event_wwnr(p, switch_out_wwnr);
+}
+
+static void handle_wakeup(void *data, /* XXX: fill header */)
+{
+	struct task_struct *p = /* XXX: how do I get p? */;
+	da_handle_event_wwnr(p, wakeup_wwnr);
+}
+
+static int start_wwnr(void)
+{
+	int retval;
+
+	retval = da_monitor_init_wwnr();
+	if (retval)
+		return retval;
+
+	rv_attach_trace_probe("wwnr", /* XXX: tracepoint */, handle_switch_in);
+	rv_attach_trace_probe("wwnr", /* XXX: tracepoint */, handle_switch_out);
+	rv_attach_trace_probe("wwnr", /* XXX: tracepoint */, handle_wakeup);
+
+	return 0;
+}
+
+static void stop_wwnr(void)
+{
+	rv_wwnr.enabled = 0;
+
+	rv_detach_trace_probe("wwnr", /* XXX: tracepoint */, handle_switch_in);
+	rv_detach_trace_probe("wwnr", /* XXX: tracepoint */, handle_switch_out);
+	rv_detach_trace_probe("wwnr", /* XXX: tracepoint */, handle_wakeup);
+
+	da_monitor_destroy_wwnr();
+}
+
+/*
+ * This is the monitor register section.
+ */
+struct rv_monitor rv_wwnr = {
+	.name = "wwnr",
+	.description = "auto-generated wwnr",
+	.start = start_wwnr,
+	.stop = stop_wwnr,
+	.reset = da_monitor_reset_all_wwnr,
+	.enabled = 0,
+};
+
+int register_wwnr(void)
+{
+	rv_register_monitor(&rv_wwnr);
+	return 0;
+}
+
+void unregister_wwnr(void)
+{
+	if (rv_wwnr.enabled)
+		stop_wwnr();
+
+	rv_unregister_monitor(&rv_wwnr);
+}
+
+module_init(register_wwnr);
+module_exit(unregister_wwnr);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("dot2k: auto-generated");
+MODULE_DESCRIPTION("wwnr");
kernel/trace/rv/monitors/wwnr/wwnr.h new file +21 -0
new file mode 100644
index 000000000000..f3dc160642bf
@@ -0,0 +1,38 @@
+enum states_wwnr {
+	not_running_wwnr = 0,
+	running_wwnr,
+	state_max_wwnr
+};
+
+enum events_wwnr {
+	switch_in_wwnr = 0,
+	switch_out_wwnr,
+	wakeup_wwnr,
+	event_max_wwnr
+};
+
+struct automaton_wwnr {
+	char *state_names[state_max_wwnr];
+	char *event_names[event_max_wwnr];
+	char function[state_max_wwnr][event_max_wwnr];
+	char initial_state;
+	char final_states[state_max_wwnr];
+};
+

kernel/trace/rv/monitors/wwnr/wwnr.h line 31

Quoted 10 lines
kernel/trace/rv/monitors/wwnr/wwnr.h new file +10 -0
+struct automaton_wwnr automaton_wwnr = {
+	.state_names = {
+		"not_running",
+		"running"
+	},
+	.event_names = {
+		"switch_in",
+		"switch_out",
+		"wakeup"
+	},

The .state_names and .event_names lack the wwnr postfix. If they generated aotomatically, something need to fix, if not, can manually change to the consistent name.

If the state change from running to non-running(initial state), in the next patch in handle_switch() where there checked TASK_INTERRUPTABLE to set initial state. Why can not check TASK_UNINTERURPTABLE to set initial state there, confused. This model is not completed for me now.

Quoted 10 lines
kernel/trace/rv/monitors/wwnr/wwnr.h new file +7 -0
+	.function = {
+		{     running_wwnr,               -1, not_running_wwnr },
+		{               -1, not_running_wwnr,               -1 },
+	},
+	.initial_state = not_running_wwnr,
+	.final_states = { 1, 0 },
+};
Signature · 1 line
2.35.1
#38 lore raw

PATCH v4 11/20 rv/monitor: wwnr instrumentation and Makefile/Kconfig entries

In reply to Daniel Bristot de Oliveira

Adds the instrumentation to the previously created wwnr monitor, as an example of the developer work. It also adds a Makefile, Kconfig and tracepoint entries.

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 include/trace/events/rv.h            | 12 +++++++++
 kernel/trace/rv/Kconfig              |  8 ++++++
 kernel/trace/rv/Makefile             |  1 +
 kernel/trace/rv/monitors/wwnr/wwnr.c | 38 ++++++++++++----------------
 4 files changed, 37 insertions(+), 22 deletions(-)
include/trace/events/rv.h +12 -0
index 4e0dabffcf29..00f11a8dac3b 100644
@@ -122,6 +122,18 @@ DECLARE_EVENT_CLASS(error_da_monitor_id,
 		__entry->event,
 		__entry->state)
 );
+
+#ifdef CONFIG_RV_MON_WWNR
+/* id is the pid of the task */
+DEFINE_EVENT(event_da_monitor_id, event_wwnr,
+	     TP_PROTO(int id, char *state, char *event, char *next_state, bool safe),
+	     TP_ARGS(id, state, event, next_state, safe));
+
+DEFINE_EVENT(error_da_monitor_id, error_wwnr,
+	     TP_PROTO(int id, char *state, char *event),
+	     TP_ARGS(id, state, event));
+#endif /* CONFIG_RV_MON_WWNR */
+
 #endif /* CONFIG_DA_MON_EVENTS_ID */
 #endif /* _TRACE_RV_H */
 
kernel/trace/rv/Kconfig +8 -0
index e9246b0bec9d..fba2ace2a22b 100644
@@ -34,6 +34,14 @@ config RV_MON_WIP
 	  Enable WIP sample monitor, this is a sample monitor that
 	  illustrates the usage of per-cpu monitors.
 
+config RV_MON_WWNR
+	select DA_MON_EVENTS_ID
+	bool "WWNR monitor"
+	help
+	  Enable WWNR sample monitor, this is a sample monitor that
+	  illustrates the usage of per-task monitor. The model is
+	  broken on purpose: it serves to test reactors.
+
 config RV_REACTORS
 	bool "Runtime verification reactors"
 	default y if RV
kernel/trace/rv/Makefile +1 -0
index b41109d2750a..af0ff9a46418 100644
@@ -3,3 +3,4 @@
 obj-$(CONFIG_RV) += rv.o
 obj-$(CONFIG_RV_REACTORS) += rv_reactors.o
 obj-$(CONFIG_RV_MON_WIP) += monitors/wip/wip.o
+obj-$(CONFIG_RV_MON_WWNR) += monitors/wwnr/wwnr.o
kernel/trace/rv/monitors/wwnr/wwnr.c +16 -22
index 8ba01f0f0df8..3fe1ad9125d3 100644
@@ -10,11 +10,8 @@
 
 #define MODULE_NAME "wwnr"
 
-/*
- * XXX: include required tracepoint headers, e.g.,
- * #include <linux/trace/events/sched.h>
- */
 #include <trace/events/rv.h>
+#include <trace/events/sched.h>
 
 /*
  * This is the self-generated part of the monitor. Generally, there is no need
@@ -37,21 +34,20 @@ DECLARE_DA_MON_PER_TASK(wwnr, char);
  * are translated into model's event.
  *
  */
-static void handle_switch_in(void *data, /* XXX: fill header */)
+static void handle_switch(void *data, bool preempt, struct task_struct *p,
+			  struct task_struct *n, unsigned int prev_state)
 {
-	struct task_struct *p = /* XXX: how do I get p? */;
-	da_handle_event_wwnr(p, switch_in_wwnr);
-}
+	/* start monitoring only after the first suspension */
+	if (prev_state == TASK_INTERRUPTIBLE)
+		da_handle_init_event_wwnr(p, switch_out_wwnr);
+	else
+		da_handle_event_wwnr(p, switch_out_wwnr);
 
-static void handle_switch_out(void *data, /* XXX: fill header */)
-{
-	struct task_struct *p = /* XXX: how do I get p? */;
-	da_handle_event_wwnr(p, switch_out_wwnr);
+	da_handle_event_wwnr(n, switch_in_wwnr);
 }
 
-static void handle_wakeup(void *data, /* XXX: fill header */)
+static void handle_wakeup(void *data, struct task_struct *p)
 {
-	struct task_struct *p = /* XXX: how do I get p? */;
 	da_handle_event_wwnr(p, wakeup_wwnr);
 }
 
@@ -63,9 +59,8 @@ static int start_wwnr(void)
 	if (retval)
 		return retval;
 
-	rv_attach_trace_probe("wwnr", /* XXX: tracepoint */, handle_switch_in);
-	rv_attach_trace_probe("wwnr", /* XXX: tracepoint */, handle_switch_out);
-	rv_attach_trace_probe("wwnr", /* XXX: tracepoint */, handle_wakeup);
+	rv_attach_trace_probe("wwnr", sched_switch, handle_switch);
+	rv_attach_trace_probe("wwnr", sched_wakeup, handle_wakeup);
 
 	return 0;
 }
@@ -74,9 +69,8 @@ static void stop_wwnr(void)
 {
 	rv_wwnr.enabled = 0;
 
-	rv_detach_trace_probe("wwnr", /* XXX: tracepoint */, handle_switch_in);
-	rv_detach_trace_probe("wwnr", /* XXX: tracepoint */, handle_switch_out);
-	rv_detach_trace_probe("wwnr", /* XXX: tracepoint */, handle_wakeup);
+	rv_detach_trace_probe("wwnr", sched_switch, handle_switch);
+	rv_detach_trace_probe("wwnr", sched_wakeup, handle_wakeup);
 
 	da_monitor_destroy_wwnr();
 }
@@ -111,5 +105,5 @@ module_init(register_wwnr);
 module_exit(unregister_wwnr);
 
 MODULE_LICENSE("GPL");
-MODULE_AUTHOR("dot2k: auto-generated");
-MODULE_DESCRIPTION("wwnr");
+MODULE_AUTHOR("Daniel Bristot de Oliveira <[email protected]>");
+MODULE_DESCRIPTION("wwnr: wakeup while not running monitor.");
Signature · 1 line
2.35.1
#39 lore raw

Re: [PATCH V4 11/20] rv/monitor: wwnr instrumentation and Makefile/Kconfig entries

I love your patch! Perhaps something to improve:

Hi Daniel,

I love your patch! Perhaps something to improve:

[auto build test WARNING on rostedt-trace/for-next]
[also build test WARNING on tip/sched/core]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/intel-lab-lkp/linux/commits/Daniel-Bristot-de-Oliveira/The-Runtime-Verification-RV-interface/20220616-164837
base:   https://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace.git for-next
config: x86_64-allyesconfig (https://download.01.org/0day-ci/archive/20220616/[email protected]/config)
compiler: gcc-11 (Debian 11.3.0-3) 11.3.0
reproduce (this is a W=1 build):
        # https://github.com/intel-lab-lkp/linux/commit/31dad6685057c10f6301fbc4018b6586fce0757e
        git remote add linux-review https://github.com/intel-lab-lkp/linux
        git fetch --no-tags linux-review Daniel-Bristot-de-Oliveira/The-Runtime-Verification-RV-interface/20220616-164837
        git checkout 31dad6685057c10f6301fbc4018b6586fce0757e
        # save the config file
        mkdir build_dir && cp config build_dir/.config
        make W=1 O=build_dir ARCH=x86_64 SHELL=/bin/bash kernel/trace/rv/

If you fix the issue, kindly add following tag where applicable

All warnings (new ones prefixed by >>):

In file included from kernel/trace/rv/monitors/wwnr/wwnr.c:8:
kernel/trace/rv/monitors/wwnr/wwnr.c: In function 'start_wwnr':
kernel/trace/rv/monitors/wwnr/wwnr.c:62:53: error: passing argument 1 of 'check_trace_callback_type_sched_switch' from incompatible pointer type [-Werror=incompatible-pointer-types]

      62 |         rv_attach_trace_probe("wwnr", sched_switch, handle_switch);
         |                                                     ^~~~~~~~~~~~~
         |                                                     |
         |                                                     void (*)(void *, bool,  struct task_struct *, struct task_struct *, unsigned int) {aka void (*)(void *, _Bool,  struct task_struct *, struct task_struct *, unsigned int)}
   include/rv/instrumentation.h:15:48: note: in definition of macro 'rv_attach_trace_probe'
      15 |                 check_trace_callback_type_##tp(rv_handler);                             \
         |                                                ^~~~~~~~~~
   In file included from kernel/trace/rv/monitors/wwnr/wwnr.c:3:
   include/linux/tracepoint.h:279:49: note: expected 'void (*)(void *, bool,  unsigned int,  struct task_struct *, struct task_struct *)' {aka 'void (*)(void *, _Bool,  unsigned int,  struct task_struct *, struct task_struct *)'} but argument is of type 'void (*)(void *, bool,  struct task_struct *, struct task_struct *, unsigned int)' {aka 'void (*)(void *, _Bool,  struct task_struct *, struct task_struct *, unsigned int)'}
     279 |         check_trace_callback_type_##name(void (*cb)(data_proto))        \
         |                                          ~~~~~~~^~~~~~~~~~~~~~~
   include/linux/tracepoint.h:419:9: note: in expansion of macro '__DECLARE_TRACE'
     419 |         __DECLARE_TRACE(name, PARAMS(proto), PARAMS(args),              \
         |         ^~~~~~~~~~~~~~~
   include/linux/tracepoint.h:553:9: note: in expansion of macro 'DECLARE_TRACE'
     553 |         DECLARE_TRACE(name, PARAMS(proto), PARAMS(args))
         |         ^~~~~~~~~~~~~
   include/trace/events/sched.h:222:1: note: in expansion of macro 'TRACE_EVENT'
     222 | TRACE_EVENT(sched_switch,
         | ^~~~~~~~~~~
   In file included from include/linux/printk.h:11,
                    from include/linux/kernel.h:29,
                    from include/linux/interrupt.h:6,
                    from include/linux/trace_recursion.h:5,
                    from include/linux/ftrace.h:10,
                    from kernel/trace/rv/monitors/wwnr/wwnr.c:2:
   kernel/trace/rv/monitors/wwnr/wwnr.c:62:53: error: passing argument 1 of 'register_trace_sched_switch' from incompatible pointer type [-Werror=incompatible-pointer-types]
      62 |         rv_attach_trace_probe("wwnr", sched_switch, handle_switch);
         |                                                     ^~~~~~~~~~~~~
         |                                                     |
         |                                                     void (*)(void *, bool,  struct task_struct *, struct task_struct *, unsigned int) {aka void (*)(void *, _Bool,  struct task_struct *, struct task_struct *, unsigned int)}
   include/linux/once_lite.h:15:41: note: in definition of macro 'DO_ONCE_LITE_IF'
      15 |                 bool __ret_do_once = !!(condition);                     \
         |                                         ^~~~~~~~~
   include/rv/instrumentation.h:16:17: note: in expansion of macro 'WARN_ONCE'
      16 |                 WARN_ONCE(register_trace_##tp(rv_handler, NULL),                        \
         |                 ^~~~~~~~~
   kernel/trace/rv/monitors/wwnr/wwnr.c:62:9: note: in expansion of macro 'rv_attach_trace_probe'
      62 |         rv_attach_trace_probe("wwnr", sched_switch, handle_switch);
         |         ^~~~~~~~~~~~~~~~~~~~~
   In file included from kernel/trace/rv/monitors/wwnr/wwnr.c:3:
   include/linux/tracepoint.h:260:38: note: expected 'void (*)(void *, bool,  unsigned int,  struct task_struct *, struct task_struct *)' {aka 'void (*)(void *, _Bool,  unsigned int,  struct task_struct *, struct task_struct *)'} but argument is of type 'void (*)(void *, bool,  struct task_struct *, struct task_struct *, unsigned int)' {aka 'void (*)(void *, _Bool,  struct task_struct *, struct task_struct *, unsigned int)'}
     260 |         register_trace_##name(void (*probe)(data_proto), void *data)    \
         |                               ~~~~~~~^~~~~~~~~~~~~~~~~~
   include/linux/tracepoint.h:419:9: note: in expansion of macro '__DECLARE_TRACE'
     419 |         __DECLARE_TRACE(name, PARAMS(proto), PARAMS(args),              \
         |         ^~~~~~~~~~~~~~~
   include/linux/tracepoint.h:553:9: note: in expansion of macro 'DECLARE_TRACE'
     553 |         DECLARE_TRACE(name, PARAMS(proto), PARAMS(args))
         |         ^~~~~~~~~~~~~
   include/trace/events/sched.h:222:1: note: in expansion of macro 'TRACE_EVENT'
     222 | TRACE_EVENT(sched_switch,
         | ^~~~~~~~~~~
   In file included from kernel/trace/rv/monitors/wwnr/wwnr.c:8:
   kernel/trace/rv/monitors/wwnr/wwnr.c: In function 'stop_wwnr':
   kernel/trace/rv/monitors/wwnr/wwnr.c:72:53: error: passing argument 1 of 'unregister_trace_sched_switch' from incompatible pointer type [-Werror=incompatible-pointer-types]
      72 |         rv_detach_trace_probe("wwnr", sched_switch, handle_switch);
         |                                                     ^~~~~~~~~~~~~
         |                                                     |
         |                                                     void (*)(void *, bool,  struct task_struct *, struct task_struct *, unsigned int) {aka void (*)(void *, _Bool,  struct task_struct *, struct task_struct *, unsigned int)}
   include/rv/instrumentation.h:22:39: note: in definition of macro 'rv_detach_trace_probe'
      22 |                 unregister_trace_##tp(rv_handler, NULL);                                \
         |                                       ^~~~~~~~~~
   In file included from kernel/trace/rv/monitors/wwnr/wwnr.c:3:
   include/linux/tracepoint.h:273:40: note: expected 'void (*)(void *, bool,  unsigned int,  struct task_struct *, struct task_struct *)' {aka 'void (*)(void *, _Bool,  unsigned int,  struct task_struct *, struct task_struct *)'} but argument is of type 'void (*)(void *, bool,  struct task_struct *, struct task_struct *, unsigned int)' {aka 'void (*)(void *, _Bool,  struct task_struct *, struct task_struct *, unsigned int)'}
     273 |         unregister_trace_##name(void (*probe)(data_proto), void *data)  \
         |                                 ~~~~~~~^~~~~~~~~~~~~~~~~~
   include/linux/tracepoint.h:419:9: note: in expansion of macro '__DECLARE_TRACE'
     419 |         __DECLARE_TRACE(name, PARAMS(proto), PARAMS(args),              \
         |         ^~~~~~~~~~~~~~~
   include/linux/tracepoint.h:553:9: note: in expansion of macro 'DECLARE_TRACE'
     553 |         DECLARE_TRACE(name, PARAMS(proto), PARAMS(args))
         |         ^~~~~~~~~~~~~
   include/trace/events/sched.h:222:1: note: in expansion of macro 'TRACE_EVENT'
     222 | TRACE_EVENT(sched_switch,
         | ^~~~~~~~~~~
   kernel/trace/rv/monitors/wwnr/wwnr.c: At top level:
>> kernel/trace/rv/monitors/wwnr/wwnr.c:90:5: warning: no previous prototype for 'register_wwnr' [-Wmissing-prototypes]
      90 | int register_wwnr(void)
         |     ^~~~~~~~~~~~~
>> kernel/trace/rv/monitors/wwnr/wwnr.c:96:6: warning: no previous prototype for 'unregister_wwnr' [-Wmissing-prototypes]

96 | void unregister_wwnr(void)
| ^~~~~~~~~~~~~~~
cc1: some warnings being treated as errors

vim +/register_wwnr +90 kernel/trace/rv/monitors/wwnr/wwnr.c

13d11b21732323 Daniel Bristot de Oliveira 2022-06-16 89

13d11b21732323 Daniel Bristot de Oliveira 2022-06-16  @90  int register_wwnr(void)
13d11b21732323 Daniel Bristot de Oliveira 2022-06-16   91  {
13d11b21732323 Daniel Bristot de Oliveira 2022-06-16   92  	rv_register_monitor(&rv_wwnr);
13d11b21732323 Daniel Bristot de Oliveira 2022-06-16   93  	return 0;
13d11b21732323 Daniel Bristot de Oliveira 2022-06-16   94  }
13d11b21732323 Daniel Bristot de Oliveira 2022-06-16   95  
13d11b21732323 Daniel Bristot de Oliveira 2022-06-16  @96  void unregister_wwnr(void)
13d11b21732323 Daniel Bristot de Oliveira 2022-06-16   97  {
13d11b21732323 Daniel Bristot de Oliveira 2022-06-16   98  	if (rv_wwnr.enabled)
13d11b21732323 Daniel Bristot de Oliveira 2022-06-16   99  		stop_wwnr();
13d11b21732323 Daniel Bristot de Oliveira 2022-06-16  100  
13d11b21732323 Daniel Bristot de Oliveira 2022-06-16  101  	rv_unregister_monitor(&rv_wwnr);
13d11b21732323 Daniel Bristot de Oliveira 2022-06-16  102  }
13d11b21732323 Daniel Bristot de Oliveira 2022-06-16  103  
Signature · 2 lines
0-DAY CI Kernel Test Service
https://01.org/lkp
maintainer
#40 lore raw

Re: [PATCH V4 11/20] rv/monitor: wwnr instrumentation and Makefile/Kconfig entries

In reply to Daniel Bristot de Oliveira
Quoted 18 lines · On Thu, 16 Jun 2022 10:44:53 +0200 Daniel Bristot de Oliveira…
kernel/trace/rv/monitors/wwnr/wwnr.c +1 -4
index 8ba01f0f0df8..3fe1ad9125d3 100644
@@ -10,11 +10,8 @@
 
 #define MODULE_NAME "wwnr"
 
-/*
- * XXX: include required tracepoint headers, e.g.,
- * #include <linux/trace/events/sched.h>
- */
 #include <trace/events/rv.h>
+#include <trace/events/sched.h>
 
 /*
  * This is the self-generated part of the monitor. Generally, there is no need

kernel/trace/rv/monitors/wwnr/wwnr.c DECLARE_DA_MON_PER_TASK(wwnr, char); · line 39

Quoted 7 lines
kernel/trace/rv/monitors/wwnr/wwnr.c +2 -1
@@ -37,21 +34,20 @@ DECLARE_DA_MON_PER_TASK(wwnr, char);
  * are translated into model's event.
  *
  */
-static void handle_switch_in(void *data, /* XXX: fill header */)
+static void handle_switch(void *data, bool preempt, struct task_struct *p,
+			  struct task_struct *n, unsigned int prev_state)
 {

Patch 8 was the "educational" patch. There's no reason to split 10 and 11 up too.

-- Steve

Quoted 22 lines
kernel/trace/rv/monitors/wwnr/wwnr.c +7 -9
-	struct task_struct *p = /* XXX: how do I get p? */;
-	da_handle_event_wwnr(p, switch_in_wwnr);
-}
+	/* start monitoring only after the first suspension */
+	if (prev_state == TASK_INTERRUPTIBLE)
+		da_handle_init_event_wwnr(p, switch_out_wwnr);
+	else
+		da_handle_event_wwnr(p, switch_out_wwnr);
 
-static void handle_switch_out(void *data, /* XXX: fill header */)
-{
-	struct task_struct *p = /* XXX: how do I get p? */;
-	da_handle_event_wwnr(p, switch_out_wwnr);
+	da_handle_event_wwnr(n, switch_in_wwnr);
 }
 
-static void handle_wakeup(void *data, /* XXX: fill header */)
+static void handle_wakeup(void *data, struct task_struct *p)
 {
-	struct task_struct *p = /* XXX: how do I get p? */;
 	da_handle_event_wwnr(p, wakeup_wwnr);
 }
#41 lore raw

PATCH v4 12/20 rv/reactor: Add the printk reactor

In reply to Daniel Bristot de Oliveira

Sample reactor that printks the reaction message.

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 kernel/trace/rv/Kconfig          |  8 ++++++
 kernel/trace/rv/Makefile         |  3 ++-
 kernel/trace/rv/reactor_printk.c | 43 ++++++++++++++++++++++++++++++++
 3 files changed, 53 insertions(+), 1 deletion(-)
 create mode 100644 kernel/trace/rv/reactor_printk.c
kernel/trace/rv/Kconfig +8 -0
index fba2ace2a22b..be8e3dab0a52 100644
@@ -52,4 +52,12 @@ config RV_REACTORS
 	  tracing reactions, printing the monitor output via tracepoints,
 	  but other reactions can be added (on-demand) via this interface.
 
+config RV_REACT_PRINTK
+	bool "Printk reactor"
+	depends on RV_REACTORS
+	default y if RV_REACTORS
+	help
+	  Enables the printk reactor. The printk reactor emmits a printk()
+	  message if an exception is found.
+
 endif # RV
kernel/trace/rv/Makefile +2 -1
index af0ff9a46418..a13c750a35c1 100644
@@ -1,6 +1,7 @@
 # SPDX-License-Identifier: GPL-2.0
 
 obj-$(CONFIG_RV) += rv.o
-obj-$(CONFIG_RV_REACTORS) += rv_reactors.o
 obj-$(CONFIG_RV_MON_WIP) += monitors/wip/wip.o
 obj-$(CONFIG_RV_MON_WWNR) += monitors/wwnr/wwnr.o
+obj-$(CONFIG_RV_REACTORS) += rv_reactors.o
+obj-$(CONFIG_RV_REACT_PRINTK) += reactor_printk.o
kernel/trace/rv/reactor_printk.c new file +43 -0

43 lines not on this page · raw message

Signature · 1 line
2.35.1
#42 lore raw

PATCH v4 13/20 rv/reactor: Add the panic reactor

In reply to Daniel Bristot de Oliveira

Sample reactor that panics the system when an exception is found. This is useful both to capture a vmcore, or to fail-safe a critical system.

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 kernel/trace/rv/Kconfig         |  8 ++++++
 kernel/trace/rv/Makefile        |  1 +
 kernel/trace/rv/reactor_panic.c | 44 +++++++++++++++++++++++++++++++++
 3 files changed, 53 insertions(+)
 create mode 100644 kernel/trace/rv/reactor_panic.c
kernel/trace/rv/Kconfig +8 -0
index be8e3dab0a52..91a17b13a080 100644
@@ -60,4 +60,12 @@ config RV_REACT_PRINTK
 	  Enables the printk reactor. The printk reactor emmits a printk()
 	  message if an exception is found.
 
+config RV_REACT_PANIC
+	bool "Panic reactor"
+	depends on RV_REACTORS
+	default y if RV_REACTORS
+	help
+	  Enables the panic reactor. The panic reactor emmits a printk()
+	  message if an exception is found and panic()s the system.
+
 endif # RV
kernel/trace/rv/Makefile +1 -0
index a13c750a35c1..963d14875b45 100644
@@ -5,3 +5,4 @@ obj-$(CONFIG_RV_MON_WIP) += monitors/wip/wip.o
 obj-$(CONFIG_RV_MON_WWNR) += monitors/wwnr/wwnr.o
 obj-$(CONFIG_RV_REACTORS) += rv_reactors.o
 obj-$(CONFIG_RV_REACT_PRINTK) += reactor_printk.o
+obj-$(CONFIG_RV_REACT_PANIC) += reactor_panic.o
kernel/trace/rv/reactor_panic.c new file +44 -0

44 lines not on this page · raw message

Signature · 1 line
2.35.1
#43 lore raw

Re: [PATCH V4 13/20] rv/reactor: Add the panic reactor

I love your patch! Perhaps something to improve:

Hi Daniel,

I love your patch! Perhaps something to improve:

[auto build test WARNING on rostedt-trace/for-next]
[also build test WARNING on tip/sched/core]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/intel-lab-lkp/linux/commits/Daniel-Bristot-de-Oliveira/The-Runtime-Verification-RV-interface/20220616-164837
base:   https://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace.git for-next
config: x86_64-allyesconfig (https://download.01.org/0day-ci/archive/20220616/[email protected]/config)
compiler: gcc-11 (Debian 11.3.0-3) 11.3.0
reproduce (this is a W=1 build):
        # https://github.com/intel-lab-lkp/linux/commit/c32f84c1978c0e0b8526da45a2ab87e191246f68
        git remote add linux-review https://github.com/intel-lab-lkp/linux
        git fetch --no-tags linux-review Daniel-Bristot-de-Oliveira/The-Runtime-Verification-RV-interface/20220616-164837
        git checkout c32f84c1978c0e0b8526da45a2ab87e191246f68
        # save the config file
        mkdir build_dir && cp config build_dir/.config
        make W=1 O=build_dir ARCH=x86_64 SHELL=/bin/bash kernel/trace/rv/

If you fix the issue, kindly add following tag where applicable

All warnings (new ones prefixed by >>):

>> kernel/trace/rv/reactor_panic.c:28:5: warning: no previous prototype for 'register_react_panic' [-Wmissing-prototypes]
      28 | int register_react_panic(void)
         |     ^~~~~~~~~~~~~~~~~~~~
>> kernel/trace/rv/reactor_panic.c:34:6: warning: no previous prototype for 'unregister_react_panic' [-Wmissing-prototypes]

34 | void unregister_react_panic(void)
| ^~~~~~~~~~~~~~~~~~~~~~

vim +/register_react_panic +28 kernel/trace/rv/reactor_panic.c

    27	
  > 28	int register_react_panic(void)
    29	{
    30		rv_register_reactor(&rv_panic);
    31		return 0;
    32	}
    33	
  > 34	void unregister_react_panic(void)
    35	{
    36		rv_unregister_reactor(&rv_panic);
    37	}
    38	
Signature · 2 lines
0-DAY CI Kernel Test Service
https://01.org/lkp
reviewer
#44 lore raw

Re: [PATCH V4 13/20] rv/reactor: Add the panic reactor

In reply to Daniel Bristot de Oliveira

kernel/trace/rv/Kconfig config RV_REACT_PRINTK · line 60

Quoted 6 lines · Daniel Bristot de Oliveira
kernel/trace/rv/Kconfig +0 -0
index be8e3dab0a52..91a17b13a080 100644
@@ -60,4 +60,12 @@ config RV_REACT_PRINTK
 	  Enables the printk reactor. The printk reactor emmits a printk()

emits

kernel/trace/rv/Kconfig config RV_REACT_PRINTK · line 68

Quoted 8 lines
kernel/trace/rv/Kconfig +6 -0
 	  message if an exception is found.
 
+config RV_REACT_PANIC
+	bool "Panic reactor"
+	depends on RV_REACTORS
+	default y if RV_REACTORS
+	help
+	  Enables the panic reactor. The panic reactor emmits a printk()

emits

Quoted 1 line
kernel/trace/rv/Kconfig +1 -0
+	  message if an exception is found and panic()s the system.
Signature · 1 line
~Randy
#45 lore raw

Re: [PATCH V4 13/20] rv/reactor: Add the panic reactor

Quoted 11 lines · Randy Dunlap

kernel/trace/rv/Kconfig config RV_REACT_PRINTK · line 60

Quoted 6 lines · Daniel Bristot de Oliveira
kernel/trace/rv/Kconfig +0 -0
index be8e3dab0a52..91a17b13a080 100644
@@ -60,4 +60,12 @@ config RV_REACT_PRINTK
 	  Enables the printk reactor. The printk reactor emmits a printk()

emits

oops!

Quoted 10 lines

kernel/trace/rv/Kconfig config RV_REACT_PRINTK · line 68

Quoted 8 lines
kernel/trace/rv/Kconfig +6 -0
 	  message if an exception is found.
 
+config RV_REACT_PANIC
+	bool "Panic reactor"
+	depends on RV_REACTORS
+	default y if RV_REACTORS
+	help
+	  Enables the panic reactor. The panic reactor emmits a printk()

emits

and Oops again.

Quoted 2 lines

kernel/trace/rv/Kconfig config RV_REACT_PRINTK · line 69

Quoted 1 line
kernel/trace/rv/Kconfig +1 -0
+	  message if an exception is found and panic()s the system.

I will fix that, thanks Randy!
-- Daniel

maintainer
#47 lore raw

Re: [PATCH V4 13/20] rv/reactor: Add the panic reactor

In reply to Daniel Bristot de Oliveira
Quoted 35 lines · On Thu, 16 Jun 2022 10:44:55 +0200 Daniel Bristot de Oliveira…

Sample reactor that panics the system when an exception is found. This is useful both to capture a vmcore, or to fail-safe a critical system.

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 kernel/trace/rv/Kconfig         |  8 ++++++
 kernel/trace/rv/Makefile        |  1 +
 kernel/trace/rv/reactor_panic.c | 44 +++++++++++++++++++++++++++++++++
 3 files changed, 53 insertions(+)
 create mode 100644 kernel/trace/rv/reactor_panic.c
kernel/trace/rv/Kconfig +0 -0
index be8e3dab0a52..91a17b13a080 100644

kernel/trace/rv/Kconfig config RV_REACT_PRINTK · line 66

Quoted 7 lines
kernel/trace/rv/Kconfig +4 -0
@@ -60,4 +60,12 @@ config RV_REACT_PRINTK
 	  Enables the printk reactor. The printk reactor emmits a printk()
 	  message if an exception is found.
 
+config RV_REACT_PANIC
+	bool "Panic reactor"
+	depends on RV_REACTORS
+	default y if RV_REACTORS

Just put default y if it already depends on RV_REACTORS.

-- Steve

Quoted 6 lines
kernel/trace/rv/Kconfig +4 -0
+	help
+	  Enables the panic reactor. The panic reactor emmits a printk()
+	  message if an exception is found and panic()s the system.
+
 endif # RV
kernel/trace/rv/Makefile +0 -0
#48 lore raw

PATCH v4 14/20 Documentation/rv: Add a basic documentation

In reply to Daniel Bristot de Oliveira

Add the runtime-verification.rst document, explaining the basics of RV and how to use the interface.

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 Documentation/trace/index.rst                 |   1 +
 Documentation/trace/rv/index.rst              |   9 +
 .../trace/rv/runtime-verification.rst         | 233 ++++++++++++++++++
 kernel/trace/rv/Kconfig                       |   3 +
 4 files changed, 246 insertions(+)
 create mode 100644 Documentation/trace/rv/index.rst
 create mode 100644 Documentation/trace/rv/runtime-verification.rst
Documentation/trace/index.rst +1 -0
index f9b7bcb5a630..2d73e8697523 100644
@@ -32,3 +32,4 @@ Linux Tracing Technologies
    sys-t
    coresight/index
    user_events
+   rv/index
Documentation/trace/rv/index.rst new file +9 -0
new file mode 100644
index 000000000000..92338dceffab
@@ -0,0 +1,9 @@
+===================================
+RV - Runtime Verification Interface
+===================================
+
+.. toctree::
+   :maxdepth: 2
+   :glob:
+
+   *
Documentation/trace/rv/runtime-verification.rst new file +233 -0

233 lines not on this page · raw message

kernel/trace/rv/Kconfig +3 -0
index 91a17b13a080..21f03fb3101a 100644
@@ -25,6 +25,9 @@ menuconfig RV
 	  actual execution, comparing it against a formal specification of
 	  the system behavior.
 
+	  For further information, see:
+	    Documentation/trace/rv/runtime-verification.rst
+
 if RV
 config RV_MON_WIP
 	depends on PREEMPTIRQ_TRACEPOINTS
Signature · 1 line
2.35.1
#49 lore raw

Re: [PATCH V4 14/20] Documentation/rv: Add a basic documentation

Documentation/trace/rv/runtime-verification.rst line 77

Quoted 13 lines · Daniel Bristot de Oliveira
Documentation/trace/rv/runtime-verification.rst new file +13 -0
+For further information about the principles of runtime verification and
+RV applied to Linux:
+
+  BARTOCCI, Ezio, et al. *Introduction to runtime verification.* In: Lectures on
+  Runtime Verification. Springer, Cham, 2018. p. 1-33.
+
+  FALCONE, Ylies, et al. *A taxonomy for classifying runtime verification tools.*
+  In: International Conference on Runtime Verification. Springer, Cham, 2018. p.
+  241-262.
+
+  DE OLIVEIRA, Daniel Bristot, et al. *Automata-based formal analysis and
+  verification of the real-time Linux kernel.* Ph.D. Thesis, 2020.
+

Shouldn't these references be formatted citations reference syntax?

Documentation/trace/rv/runtime-verification.rst line 136

Quoted 6 lines
Documentation/trace/rv/runtime-verification.rst new file +6 -0
+For example::
+
+   [root@f32 rv]# cat available_monitors
+   wip
+   wwnr
+

I think the prompt should be just # (without username and host).

Thanks.

Signature · 1 line
An old man doll... just what I always wanted! - Clara
#50 lore raw

Re: [PATCH V4 14/20] Documentation/rv: Add a basic documentation

Quoted 4 lines · Bagas Sanjaya

Documentation/trace/rv/runtime-verification.rst line 77

Quoted 3 lines
Documentation/trace/rv/runtime-verification.rst new file +3 -0
+  DE OLIVEIRA, Daniel Bristot, et al. *Automata-based formal analysis and
+  verification of the real-time Linux kernel.* Ph.D. Thesis, 2020.
+

Shouldn't these references be formatted citations reference syntax?

I improved them, but there is no single syntax. In practice, people find the BibTeX "citation" from the place where the paper was published and let the venue template do its work, formatting it properly (don't ask me what people using word use... I just know that many people have a bad first impression of papers made on word haha).

Quoted 8 lines

Documentation/trace/rv/runtime-verification.rst line 136

Quoted 7 lines
Documentation/trace/rv/runtime-verification.rst new file +6 -0
+For example::
+
+   [root@f32 rv]# cat available_monitors
+   wip
+   wwnr
+

I think the prompt should be just # (without username and host).

Fixed!

Thanks.
-- Daniel

Quoted 1 line

Thanks.

#51 lore raw

PATCH v4 15/20 Documentation/rv: Add deterministic automata monitor synthesis documentation

In reply to Daniel Bristot de Oliveira

Add the da_monitor_synthesis.rst introduces some concepts behind the Deterministic Automata (DA) monitor synthesis and interface.

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 .../trace/rv/da_monitor_synthesis.rst         | 284 ++++++++++++++++++
 1 file changed, 284 insertions(+)
 create mode 100644 Documentation/trace/rv/da_monitor_synthesis.rst
Documentation/trace/rv/da_monitor_synthesis.rst new file +284 -0
new file mode 100644
index 000000000000..1e1c857d7bbd
@@ -0,0 +1,284 @@
+Deterministic Automata Monitor Synthesis
+========================================
+
+The starting point for the application of runtime verification (RV) technics is
+the *specification* or *modeling* of the desired (or undesired) behavior of the
+system under scrutiny.
+
+The formal representation needs to be then *synthesized* into a *monitor* that
+can then be used in the analysis of the trace of the system. The *monitor*
+conects to the system via an *instrumentation* layer, that converts the events
+from the *system* to the events of the *specification*.
+
+This document introduces some concepts behind the **Deterministic Automata
+(DA)** monitor synthesis.
+
+DA monitor synthesis in a nutshell
+------------------------------------------------------
+
+The synthesis of automata-based models into the Linux *RV monitor* abstraction
+is automated by a tool named "dot2k", and the "rv/da_monitor.h" provided
+by the RV interface.
+
+Given a file "wip.dot", representing a per-cpu monitor, with this content::
+
+  digraph state_automaton {
+	center = true;
+	size = "7,11";
+	rankdir = LR;
+	{node [shape = circle] "non_preemptive"};
+	{node [shape = plaintext, style=invis, label=""] "__init_preemptive"};
+	{node [shape = doublecircle] "preemptive"};
+	{node [shape = circle] "preemptive"};
+	"__init_preemptive" -> "preemptive";
+	"non_preemptive" [label = "non_preemptive"];
+	"non_preemptive" -> "non_preemptive" [ label = "sched_waking" ];
+	"non_preemptive" -> "preemptive" [ label = "preempt_enable" ];
+	"preemptive" [label = "preemptive"];
+	"preemptive" -> "non_preemptive" [ label = "preempt_disable" ];
+	{ rank = min ;
+		"__init_preemptive";
+		"preemptive";
+	}
+  }
+
+Run the dot2k tool with the model, specifying that it is a "per-cpu"
+model::
+
+  $ dot2k -d ~/wip.dot -t per_cpu
+
+This will create a directory named "wip/" with the following files:
+
+- wip.h: the wip in C
+- wip.c: the RV monitor
+
+The following line in the "wip.c" file is responsible for the monitor
+synthesis::
+
+  DECLARE_DA_MON_PER_CPU(wip, char);
+
+With that in place, the work left to be done is the *instrumentation* of
+the monitor, which is already initialized by dot2k.
+
+DA: Introduction and representation formats
+---------------------------------------------------------------
+
+Formally, a deterministic automaton, denoted by G, is defined as a quintuple:
+
+        *G* = { *X*, *E*, *f*, x\ :subscript:`0`, X\ :subscript:`m` }
+
+where:
+
+- *X* is the set of states;
+- *E* is the finite set of events;
+- x\ :subscript:`0` is the initial state;
+- X\ :subscript:`m` (subset of *X*) is the set of marked states.
+- *f* : *X* x *E* -> *X* $ is the transition function. It defines the state
+  transition in the occurrence of an event from *E* in the state *X*. In the
+  special case of deterministic automata, the occurrence of the event in *E*
+  in a state in *X* has a deterministic next state from *X*.
+
+One of the most evident benefits for the practical application of the automata
+formalism is its *graphic representation*, represented using vertices (nodes)
+and edges, which is very intuitive for *operating system* practitioners.
+
+For example, given an automata wip, with a regular representation of:
+
+- *X* = { ``preemptive``, ``non_preemptive``}
+- *E* = { ``preempt_enable``, ``preempt_disable``, ``sched_waking``}
+- x\ :subscript:`0` = ``preemptive``
+- X\ :subscript:`m` = {``preemptive``}
+- *f* =
+   - *f*\ (``preemptive``, ``preempt_disable``) = ``non_preemptive``
+   - *f*\ (``non_preemptive``, ``sched_waking``) = ``non_preemptive``
+   - *f*\ (``non_preemptive``, ``preempt_enable``) = ``preemptive``
+
+
+It can also be represented in a graphic format, without any loss, using this
+format::
+
+                       preempt_enable
+          +---------------------------------+
+          v                                 |
+        #============#  preempt_disable   +------------------+
+    --> H preemptive H -----------------> |  non_preemptive  |
+        #============#                    +------------------+
+                                            ^ sched_waking |
+                                            +--------------+
+
+The Graphviz open-source tool can produce this graphic format using the
+(textual) DOT language as the source code. The DOT format is widely
+used and can be converted to many other formats, including the ASCII art above.
+
+The dot2c tool presented in:
+
+  DE OLIVEIRA, Daniel Bristot; CUCINOTTA, Tommaso; DE OLIVEIRA, Romulo
+  Silva. Efficient formal verification for the Linux kernel. In:
+  International Conference on Software Engineering and Formal Methods.
+  Springer, Cham, 2019. p. 315-332.
+
+Translates a deterministic automaton in the DOT format into a C source
+code. For instance, using the wip model as input for dot2c results in
+the following C representation::
+
+  enum states_wip {
+	preemptive_wip = 0,
+	non_preemptive_wip,
+	state_max_wip
+  };
+
+  enum events_wip {
+	preempt_disable_wip = 0,
+	preempt_enable_wip,
+	sched_waking_wip,
+	event_max_wip
+  };
+
+  struct automaton_wip {
+	char *state_names[state_max_wip];
+	char *event_names[event_max_wip];
+	char function[state_max_wip][event_max_wip];
+	char initial_state;
+	char final_states[state_max_wip];
+  };
+
+  struct automaton_wip automaton_wip = {
+	.state_names = {
+		"preemptive",
+		"non_preemptive"
+	},
+	.event_names = {
+		"preempt_disable",
+		"preempt_enable",
+		"sched_waking"
+	},
+	.function = {
+		{ non_preemptive_wip,                 -1,                 -1 },
+		{                 -1,     preemptive_wip, non_preemptive_wip },
+	},
+	.initial_state = preemptive_wip,
+	.final_states = { 1, 0 },
+  };
+
+DA monitor synthesis for Linux
+------------------------------
+
+In Linux terms, the runtime verification monitors are encapsulated
+inside the "RV monitor" abstraction. The "RV monitor" includes a set
+of instances of the monitor (per-cpu monitor, per-task monitor, and
+so on), the helper functions that glue the monitor to the system
+reference model, and the trace output as a reaction for event parsing
+and exceptions, as depicted below::
+
+ Linux  +----- RV Monitor ----------------------------------+ Formal
+  Realm |                                                   |  Realm
+  +-------------------+     +----------------+     +-----------------+
+  |   Linux kernel    |     |     Monitor    |     |     Reference   |
+  |     Tracing       |  -> |   Instance(s)  | <-  |       Model     |
+  | (instrumentation) |     | (verification) |     | (specification) |
+  +-------------------+     +----------------+     +-----------------+
+         |                          |                       |
+         |                          V                       |
+         |                     +----------+                 |
+         |                     | Reaction |                 |
+         |                     +--+--+--+-+                 |
+         |                        |  |  |                   |
+         |                        |  |  +-> trace output ?  |
+         +------------------------|--|----------------------+
+                                  |  +----> panic ?
+                                  +-------> <user-specified>
+
+
+The dot2c tool works connecting the *Reference Model* to the *RV Monitor*
+abstraction by translating the *formal notation* into *code*.
+
+The "rv/da_monitor.h" header goes beyond dot2c, extending the code
+generation to the verification stage, generating the code to the *Monitor
+Instance(s)* level using C macros. The trace event code inspires this
+approach.
+
+The benefits of the usage of macro for monitor synthesis is 3-fold:
+
+- Reduces the code duplication;
+- Facilitates the bug fix/improvement;
+- Avoids the case of developers changing the core of the monitor code
+  to manipulate the model in a (let's say) non-standard way.
+
+This initial implementation presents two different types of monitor instances:
+
+- ``#define DECLARE_DA_MON_PER_CPU(name, type)``
+- ``#define DECLARE_DA_MON_PER_TASK(name, type)``
+
+The first declares the functions for deterministic automata monitor with
+per-cpu instances, and the second with per-task instances.
+
+In both cases, the name is a string that identifies the monitor, and the type
+is the data type used by dot2c/k on the representation of the model.
+
+For example, the "wip" model with two states and three events can be
+stored in a "char" type. Considering that the preemption control is a
+per-cpu behavior, the monitor declaration will be::
+
+  DECLARE_DA_MON_PER_CPU(wip, char);
+
+The monitor is executed by sending events to be processed via the functions
+presented below::
+
+  da_handle_event_$(MONITOR_NAME)($(event from event enum));
+  da_handle_init_event_$(MONITOR_NAME)($(event from event enum));
+
+The function ``da_handle_event_$(MONITOR_NAME)()`` is the regular case,
+while the function ``da_handle_init_event_$(MONITOR_NAME)()`` is a special
+case used to synchronize the system with the model.
+
+When a monitor is enabled, it is placed in the initial state of the automata.
+However, the monitor does not know if the system is in the *initial state*.
+Hence, the monitor ignores events sent by sent by
+``da_handle_event_$(MONITOR_NAME)()`` until the function
+``da_handle_init_event_$(MONITOR_NAME)()`` is called.
+
+The function ``da_handle_init_event_$(MONITOR_NAME)()`` should be used for
+the case in which the system generates the event is the one that returns
+the automata to the initial state.
+
+After receiving a ``da_handle_init_event_$(MONITOR_NAME)()`` event, the
+monitor will know that it is in sync with the system and hence will
+start processing the next events.
+
+Using the wip model as example, the events "preempt_disable" and
+"sched_waking" should be sent to monitor, respectively, via::
+
+  da_handle_event_wip(preempt_disable);
+  da_handle_event_wip(sched_waking);
+
+While the event "preempt_enabled" will use::
+
+  da_handle_init_event_wip(preempt_enable);
+
+To notify the monitor that the system will be returning to the initial state,
+so the system and the monitor should be in sync.
+
+rv/da_monitor.h
+-------------------------------------------
+
+The "rv/da_monitor.h" is, mostly, a set of C macros that create function
+definitions based on the paremeters passed via ``DECLARE_DA_MON_*``.
+
+In fewer words, the declaration of a monitor generates:
+
+- Helper functions for getting information from the automata model generated
+  by dot2k.
+- Helper functions for the analysis of a deterministic automata model
+- Functions for the initialization of the monitor instances
+- The definition of the structure to store the monitor instances' data
+
+One important aspect is that the monitor does not call external functions
+for the handling of the events sent by the instrumentation, except for
+generating *tracing events* or *reactions*.
+
+Final remarks
+-------------
+
+With the monitor synthesis in place using, the "rv/da_monitor.h" and
+dot2k, the developer's work should be limited to the instrumentation
+of the system, increasing the confidence in the overall approach.
Signature · 1 line
2.35.1
maintainer
#52 lore raw

Re: [PATCH V4 15/20] Documentation/rv: Add deterministic automata monitor synthesis documentation

Documentation/trace/rv/da_monitor_synthesis.rst line 23

Quoted 8 lines · On Thu, 16 Jun 2022 10:44:57 +0200 Daniel Bristot de Oliveira…
Documentation/trace/rv/da_monitor_synthesis.rst new file +8 -0
+DA monitor synthesis in a nutshell
+------------------------------------------------------
+
+The synthesis of automata-based models into the Linux *RV monitor* abstraction
+is automated by a tool named "dot2k", and the "rv/da_monitor.h" provided
+by the RV interface.
+
+Given a file "wip.dot", representing a per-cpu monitor, with this content::

Specify what "wip" is here too.

-- Steve

Quoted 21 lines
Documentation/trace/rv/da_monitor_synthesis.rst new file +21 -0
+
+  digraph state_automaton {
+	center = true;
+	size = "7,11";
+	rankdir = LR;
+	{node [shape = circle] "non_preemptive"};
+	{node [shape = plaintext, style=invis, label=""] "__init_preemptive"};
+	{node [shape = doublecircle] "preemptive"};
+	{node [shape = circle] "preemptive"};
+	"__init_preemptive" -> "preemptive";
+	"non_preemptive" [label = "non_preemptive"];
+	"non_preemptive" -> "non_preemptive" [ label = "sched_waking" ];
+	"non_preemptive" -> "preemptive" [ label = "preempt_enable" ];
+	"preemptive" [label = "preemptive"];
+	"preemptive" -> "non_preemptive" [ label = "preempt_disable" ];
+	{ rank = min ;
+		"__init_preemptive";
+		"preemptive";
+	}
+  }
+
#53 lore raw

PATCH v4 16/20 Documentation/rv: Add deterministic automata instrumentation documentation

In reply to Daniel Bristot de Oliveira

Add the da_monitor_instrumentation.rst. It describes the basics of RV monitor instrumentation.

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 .../trace/rv/da_monitor_instrumentation.rst   | 223 ++++++++++++++++++
 1 file changed, 223 insertions(+)
 create mode 100644 Documentation/trace/rv/da_monitor_instrumentation.rst
Documentation/trace/rv/da_monitor_instrumentation.rst new file +223 -0
new file mode 100644
index 000000000000..994110050d93
@@ -0,0 +1,223 @@
+Deterministic Automata Instrumentation
+========================================
+
+This document introduces some concepts behind the **Deterministic Automata
+(DA)** monitor instrumentation.
+
+The synthesis of automata-based models into the Linux *RV monitor* abstraction
+is automated by a tool named dot2k, and the "rv/da_monitor.h" provided
+by the RV interface.
+
+For example, given a file "wip.dot", representing a per-cpu monitor, with
+this content::
+
+  digraph state_automaton {
+	center = true;
+	size = "7,11";
+	rankdir = LR;
+	{node [shape = circle] "non_preemptive"};
+	{node [shape = plaintext, style=invis, label=""] "__init_preemptive"};
+	{node [shape = doublecircle] "preemptive"};
+	{node [shape = circle] "preemptive"};
+	"__init_preemptive" -> "preemptive";
+	"non_preemptive" [label = "non_preemptive"];
+	"non_preemptive" -> "non_preemptive" [ label = "sched_waking" ];
+	"non_preemptive" -> "preemptive" [ label = "preempt_enable" ];
+	"preemptive" [label = "preemptive"];
+	"preemptive" -> "non_preemptive" [ label = "preempt_disable" ];
+	{ rank = min ;
+		"__init_preemptive";
+		"preemptive";
+	}
+  }
+
+That is the "DOT" representation of this automata model::
+
+                       preempt_enable
+          +---------------------------------+
+          v                                 |
+        #============#  preempt_disable   +------------------+
+    --> H preemptive H -----------------> |  non_preemptive  |
+        #============#                    +------------------+
+                                            ^ sched_waking |
+                                            +--------------+
+
+
+Run the dot2k tool with the model, specifying that it is a "per-cpu"
+model::
+
+  $ dot2k -d ~/wip.dot -t per_cpu
+
+This will create a directory named "wip/" with the following files:
+
+- model.h: the wip in C
+- wip.h: tracepoints that report the execution of the events by the
+  monitor
+- wip.c: the RV monitor
+
+The monitor instrumentation should be done entirely in the RV monitor,
+in the example above, in the wip.c file.
+
+The RV monitor instrumentation section
+--------------------------------------
+
+The RV monitor file created by dot2k, with the name "$MODEL_NAME.c"
+will include a section dedicated to instrumentation.
+
+In the example of the wip.dot above, it will look like::
+
+  /*
+   * This is the instrumentation part of the monitor.
+   *
+   * This is the section where manual work is required. Here the kernel events
+   * are translated into model's event.
+   *
+   */
+  static void handle_preempt_disable(void *data, /* XXX: fill header */)
+  {
+	da_handle_event_wip(preempt_disable_wip);
+  }
+
+  static void handle_preempt_enable(void *data, /* XXX: fill header */)
+  {
+	da_handle_event_wip(preempt_enable_wip);
+  }
+
+  static void handle_sched_waking(void *data, /* XXX: fill header */)
+  {
+	da_handle_event_wip(sched_waking_wip);
+  }
+
+  static int start_wip(void)
+  {
+	int retval;
+
+	retval = da_monitor_init_wip();
+	if (retval)
+		return retval;
+
+	rv_attach_trace_probe("wip", /* XXX: tracepoint */, handle_preempt_disable);
+	rv_attach_trace_probe("wip", /* XXX: tracepoint */, handle_preempt_enable);
+	rv_attach_trace_probe("wip", /* XXX: tracepoint */, handle_sched_waking);
+
+	return 0;
+  }
+
+The comment at the top of the section explains the general idea: the
+instrumentation section translates *kernel events* into the *events
+accepted by the model*.
+
+Tracing callback functions
+-----------------------------
+
+The first three functions are skeletons for callback *handler functions* for
+each of the three events from the wip model. The developer does not
+necessarily need to use them: they are just starting points.
+
+Using the example of::
+
+ void handle_preempt_disable(void *data, /* XXX: fill header */)
+ {
+        da_handle_event_wip(preempt_disable_wip);
+ }
+
+The "preempt_disable" event from the model conects directly to the
+"preemptirq:preempt_disable". The "preemptirq:preempt_disable" event
+has the following signature, from "include/trace/events/preemptirq.h"::
+
+  TP_PROTO(unsigned long ip, unsigned long parent_ip)
+
+Hence, the "handle_preempt_disable()" function will look like::
+
+  void handle_preempt_disable(void *data, unsigned long ip, unsigned long parent_ip)
+
+In this case, the kernel even translates one to one with the automata event,
+and indeed, no other change is needed for this function.
+
+The next handler function, "handle_preempt_enable()" has the same argument
+list from the "handle_preempt_disable()". The difference is that the
+"preempt_enable" event will be used to synchronize the system to the model.
+
+Initially, the *model* is placed in the initial state. However, the *system*
+might, or might not be in the initial state. The monitor cannot start
+processing events until it knows that the system reached the initial state.
+Otherwise the monitor and the system could be out-of-sync.
+
+Looking at the automata definition, it is possible to see that the system
+and the model are expected to return to the initial state after the
+"preempt_enable" execution. Hence, it can be used to synchronize the
+system and the model at the initialization of the monitoring section.
+
+The initialization is informed via an special handle function, the
+"da_handle_init_event_$(MONITOR)(event)", in this case::
+
+  da_handle_event_wip(preempt_disable_wip);
+
+So, the callback function will look like::
+
+  void handle_preempt_enable(void *data, unsigned long ip, unsigned long parent_ip)
+  {
+        da_handle_init_event_wip(preempt_enable_wip);
+  }
+
+Finally, the "handle_sched_waking()" will look like::
+
+  void handle_sched_waking(void *data, struct task_struct *task)
+  {
+        da_handle_event_wip(sched_waking_wip);
+  }
+
+And the explanation is left for the reader as an exercise.
+
+Start and Stop functions
+------------------------
+
+dot2k automatically creates two special functions::
+
+  start_$MODELNAME()
+  stop_$MODELNAME()
+
+These functions are called when the monitor is enabled and disabled,
+respectivelly.
+
+They should be used to *attach* and *detach* the instrumentation to the running
+system. The developer must add to the relative function all that is needed to
+*attach* and *detach* its monitor to the system.
+
+For the wip case, these functions were named::
+
+ start_wip()
+ stop_wip()
+
+But no change was required because: by default, these functions *attach* and
+*detach* the tracepoints_to_attach, which was enough for this case.
+
+Instrumentation helpers
+--------------------------
+
+To complete the instrumentation, the *handler functions* need to be attached to a
+kernel event, at the monitoring start phase.
+
+The RV interface also facilitates this step. For example, the macro "rv_attach_trace_probe()"
+is used to connect the wip model events to the relative kernel event. dot2k automatically
+adds "rv_attach_trace_probe()" function call for each model event in the start phase, as
+a suggestion.
+
+For example, from the wip sample model::
+
+  static int start_wip(void)
+  {
+        int retval;
+
+        retval = da_monitor_init_wip();
+        if (retval)
+                return retval;
+
+        rv_attach_trace_probe("wip", /* XXX: tracepoint */, handle_preempt_disable);
+        rv_attach_trace_probe("wip", /* XXX: tracepoint */, handle_preempt_enable);
+        rv_attach_trace_probe("wip", /* XXX: tracepoint */, handle_sched_waking);
+
+        return 0;
+  }
+
+The probes then need to be detached at the stop phase.
Signature · 1 line
2.35.1
#54 lore raw

PATCH v4 17/20 watchdog/dev: Add tracepoints

In reply to Daniel Bristot de Oliveira

Add a set of tracepoints, enabling the observability of the watchdog device interactions with user-space.

The events are:
watchdog:watchdog_open
watchdog:watchdog_close
watchdog:watchdog_start
watchdog:watchdog_stop
watchdog:watchdog_set_timeout
watchdog:watchdog_ping
watchdog:watchdog_nowayout
watchdog:watchdog_set_keep_alive
watchdog:watchdog_keep_alive
watchdog:watchdog_set_pretimeout
watchdog:watchdog_pretimeout

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 drivers/watchdog/watchdog_dev.c        |  43 ++++++++++-
 drivers/watchdog/watchdog_pretimeout.c |   2 +
 include/linux/watchdog.h               |   7 +-
 include/trace/events/watchdog.h        | 101 +++++++++++++++++++++++++
 4 files changed, 143 insertions(+), 10 deletions(-)
 create mode 100644 include/trace/events/watchdog.h
drivers/watchdog/watchdog_dev.c +39 -4
index 54903f3c851e..2f28dc5ab763 100644
@@ -44,6 +44,9 @@
 #include <linux/watchdog.h>	/* For watchdog specific items */
 #include <linux/uaccess.h>	/* For copy_to_user/put_user/... */
 
+#define CREATE_TRACE_POINTS
+#include <trace/events/watchdog.h>
+
 #include "watchdog_core.h"
 #include "watchdog_pretimeout.h"
 
@@ -130,9 +133,11 @@ static inline void watchdog_update_worker(struct watchdog_device *wdd)
 	if (watchdog_need_worker(wdd)) {
 		ktime_t t = watchdog_next_keepalive(wdd);
 
-		if (t > 0)
+		if (t > 0) {
 			hrtimer_start(&wd_data->timer, t,
 				      HRTIMER_MODE_REL_HARD);
+			trace_watchdog_set_keep_alive(wdd, ktime_to_ms(t));
+		}
 	} else {
 		hrtimer_cancel(&wd_data->timer);
 	}
@@ -141,7 +146,7 @@ static inline void watchdog_update_worker(struct watchdog_device *wdd)
 static int __watchdog_ping(struct watchdog_device *wdd)
 {
 	struct watchdog_core_data *wd_data = wdd->wd_data;
-	ktime_t earliest_keepalive, now;
+	ktime_t earliest_keepalive, now, next_keepalive;
 	int err;
 
 	earliest_keepalive = ktime_add(wd_data->last_hw_keepalive,
@@ -149,14 +154,16 @@ static int __watchdog_ping(struct watchdog_device *wdd)
 	now = ktime_get();
 
 	if (ktime_after(earliest_keepalive, now)) {
-		hrtimer_start(&wd_data->timer,
-			      ktime_sub(earliest_keepalive, now),
+		next_keepalive = ktime_sub(earliest_keepalive, now);
+		hrtimer_start(&wd_data->timer, next_keepalive,
 			      HRTIMER_MODE_REL_HARD);
+		trace_watchdog_set_keep_alive(wdd, ktime_to_ms(next_keepalive));
 		return 0;
 	}
 
 	wd_data->last_hw_keepalive = now;
 
+	trace_watchdog_ping(wdd);
 	if (wdd->ops->ping)
 		err = wdd->ops->ping(wdd);  /* ping the watchdog */
 	else
@@ -215,6 +222,7 @@ static void watchdog_ping_work(struct kthread_work *work)
 	wd_data = container_of(work, struct watchdog_core_data, work);
 
 	mutex_lock(&wd_data->lock);
+	trace_watchdog_keep_alive(wd_data->wdd);
 	if (watchdog_worker_should_ping(wd_data))
 		__watchdog_ping(wd_data->wdd);
 	mutex_unlock(&wd_data->lock);
@@ -250,6 +258,8 @@ static int watchdog_start(struct watchdog_device *wdd)
 
 	set_bit(_WDOG_KEEPALIVE, &wd_data->status);
 
+	trace_watchdog_start(wdd);
+
 	started_at = ktime_get();
 	if (watchdog_hw_running(wdd) && wdd->ops->ping) {
 		err = __watchdog_ping(wdd);
@@ -294,6 +304,7 @@ static int watchdog_stop(struct watchdog_device *wdd)
 		return -EBUSY;
 	}
 
+	trace_watchdog_stop(wdd);
 	if (wdd->ops->stop) {
 		clear_bit(WDOG_HW_RUNNING, &wdd->status);
 		err = wdd->ops->stop(wdd);
@@ -367,6 +378,7 @@ static int watchdog_set_timeout(struct watchdog_device *wdd,
 	if (watchdog_timeout_invalid(wdd, timeout))
 		return -EINVAL;
 
+	trace_watchdog_set_timeout(wdd, timeout);
 	if (wdd->ops->set_timeout) {
 		err = wdd->ops->set_timeout(wdd, timeout);
 	} else {
@@ -399,6 +411,8 @@ static int watchdog_set_pretimeout(struct watchdog_device *wdd,
 	if (watchdog_pretimeout_invalid(wdd, timeout))
 		return -EINVAL;
 
+	trace_watchdog_set_pretimeout(wdd, timeout);
+
 	if (wdd->ops->set_pretimeout && (wdd->info->options & WDIOF_PRETIMEOUT))
 		err = wdd->ops->set_pretimeout(wdd, timeout);
 	else
@@ -430,6 +444,23 @@ static int watchdog_get_timeleft(struct watchdog_device *wdd,
 	return 0;
 }
 
+/**
+ * watchdog_set_nowayout - set nowaout bit
+ * @wdd:	The watchdog device to set nowayoutbit
+ * @nowayout	A boolean on/off switcher
+ *
+ * If nowayout boolean is true, the nowayout option is set. No action is
+ * taken if nowayout is false.
+ */
+void watchdog_set_nowayout(struct watchdog_device *wdd, bool nowayout)
+{
+	if (nowayout) {
+		set_bit(WDOG_NO_WAY_OUT, &wdd->status);
+		trace_watchdog_nowayout(wdd);
+	}
+}
+EXPORT_SYMBOL(watchdog_set_nowayout);
+
 #ifdef CONFIG_WATCHDOG_SYSFS
 static ssize_t nowayout_show(struct device *dev, struct device_attribute *attr,
 				char *buf)
@@ -861,6 +892,8 @@ static int watchdog_open(struct inode *inode, struct file *file)
 		goto out_clear;
 	}
 
+	trace_watchdog_open(wdd);
+
 	err = watchdog_start(wdd);
 	if (err < 0)
 		goto out_mod;
@@ -883,6 +916,7 @@ static int watchdog_open(struct inode *inode, struct file *file)
 	return stream_open(inode, file);
 
 out_mod:
+	trace_watchdog_close(wdd);
 	module_put(wd_data->wdd->ops->owner);
 out_clear:
 	clear_bit(_WDOG_DEV_OPEN, &wd_data->status);
@@ -944,6 +978,7 @@ static int watchdog_release(struct inode *inode, struct file *file)
 	/* make sure that /dev/watchdog can be re-opened */
 	clear_bit(_WDOG_DEV_OPEN, &wd_data->status);
 
+	trace_watchdog_close(wdd);
 done:
 	running = wdd && watchdog_hw_running(wdd);
 	mutex_unlock(&wd_data->lock);
drivers/watchdog/watchdog_pretimeout.c +2 -0
index 376a495ab80c..58c391ed2205 100644
@@ -8,6 +8,7 @@
 #include <linux/spinlock.h>
 #include <linux/string.h>
 #include <linux/watchdog.h>
+#include <trace/events/watchdog.h>
 
 #include "watchdog_core.h"
 #include "watchdog_pretimeout.h"
@@ -107,6 +108,7 @@ void watchdog_notify_pretimeout(struct watchdog_device *wdd)
 		return;
 	}
 
+	trace_watchdog_pretimeout(wdd);
 	wdd->gov->pretimeout(wdd);
 	spin_unlock_irqrestore(&pretimeout_lock, flags);
 }
include/linux/watchdog.h +1 -6
index 99660197a36c..11d93407e492 100644
@@ -139,12 +139,7 @@ static inline bool watchdog_hw_running(struct watchdog_device *wdd)
 	return test_bit(WDOG_HW_RUNNING, &wdd->status);
 }
 
-/* Use the following function to set the nowayout feature */
-static inline void watchdog_set_nowayout(struct watchdog_device *wdd, bool nowayout)
-{
-	if (nowayout)
-		set_bit(WDOG_NO_WAY_OUT, &wdd->status);
-}
+void watchdog_set_nowayout(struct watchdog_device *wdd, bool nowayout);
 
 /* Use the following function to stop the watchdog on reboot */
 static inline void watchdog_stop_on_reboot(struct watchdog_device *wdd)
include/trace/events/watchdog.h new file +101 -0

101 lines not on this page · raw message

Signature · 1 line
2.35.1
maintainer
#55 lore raw

Re: [PATCH V4 17/20] watchdog/dev: Add tracepoints

Quoted 125 lines · Daniel Bristot de Oliveira

Add a set of tracepoints, enabling the observability of the watchdog device interactions with user-space.

The events are:
watchdog:watchdog_open
watchdog:watchdog_close
watchdog:watchdog_start
watchdog:watchdog_stop
watchdog:watchdog_set_timeout
watchdog:watchdog_ping
watchdog:watchdog_nowayout
watchdog:watchdog_set_keep_alive
watchdog:watchdog_keep_alive
watchdog:watchdog_set_pretimeout
watchdog:watchdog_pretimeout

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
  drivers/watchdog/watchdog_dev.c        |  43 ++++++++++-
  drivers/watchdog/watchdog_pretimeout.c |   2 +
  include/linux/watchdog.h               |   7 +-
  include/trace/events/watchdog.h        | 101 +++++++++++++++++++++++++
  4 files changed, 143 insertions(+), 10 deletions(-)

create mode 100644 include/trace/events/watchdog.h

drivers/watchdog/watchdog_dev.c +15 -4
index 54903f3c851e..2f28dc5ab763 100644
@@ -44,6 +44,9 @@
  #include <linux/watchdog.h>	/* For watchdog specific items */
  #include <linux/uaccess.h>	/* For copy_to_user/put_user/... */
 
+#define CREATE_TRACE_POINTS
+#include <trace/events/watchdog.h>
+
  #include "watchdog_core.h"
  #include "watchdog_pretimeout.h"
 
@@ -130,9 +133,11 @@ static inline void watchdog_update_worker(struct watchdog_device *wdd)
  	if (watchdog_need_worker(wdd)) {
  		ktime_t t = watchdog_next_keepalive(wdd);
 
-		if (t > 0)
+		if (t > 0) {
  			hrtimer_start(&wd_data->timer, t,
  				      HRTIMER_MODE_REL_HARD);
+			trace_watchdog_set_keep_alive(wdd, ktime_to_ms(t));
+		}
  	} else {
  		hrtimer_cancel(&wd_data->timer);
  	}
@@ -141,7 +146,7 @@ static inline void watchdog_update_worker(struct watchdog_device *wdd)
  static int __watchdog_ping(struct watchdog_device *wdd)
  {
  	struct watchdog_core_data *wd_data = wdd->wd_data;
-	ktime_t earliest_keepalive, now;
+	ktime_t earliest_keepalive, now, next_keepalive;
  	int err;
 
  	earliest_keepalive = ktime_add(wd_data->last_hw_keepalive,
@@ -149,14 +154,16 @@ static int __watchdog_ping(struct watchdog_device *wdd)
  	now = ktime_get();
 
  	if (ktime_after(earliest_keepalive, now)) {
-		hrtimer_start(&wd_data->timer,
-			      ktime_sub(earliest_keepalive, now),
+		next_keepalive = ktime_sub(earliest_keepalive, now);
+		hrtimer_start(&wd_data->timer, next_keepalive,
  			      HRTIMER_MODE_REL_HARD);
+		trace_watchdog_set_keep_alive(wdd, ktime_to_ms(next_keepalive));
  		return 0;
  	}
 
  	wd_data->last_hw_keepalive = now;
 
+	trace_watchdog_ping(wdd);
  	if (wdd->ops->ping)
  		err = wdd->ops->ping(wdd);  /* ping the watchdog */
  	else
@@ -215,6 +222,7 @@ static void watchdog_ping_work(struct kthread_work *work)
  	wd_data = container_of(work, struct watchdog_core_data, work);
 
  	mutex_lock(&wd_data->lock);
+	trace_watchdog_keep_alive(wd_data->wdd);
  	if (watchdog_worker_should_ping(wd_data))
  		__watchdog_ping(wd_data->wdd);
  	mutex_unlock(&wd_data->lock);
@@ -250,6 +258,8 @@ static int watchdog_start(struct watchdog_device *wdd)
 
  	set_bit(_WDOG_KEEPALIVE, &wd_data->status);
 
+	trace_watchdog_start(wdd);
+
  	started_at = ktime_get();
  	if (watchdog_hw_running(wdd) && wdd->ops->ping) {
  		err = __watchdog_ping(wdd);
@@ -294,6 +304,7 @@ static int watchdog_stop(struct watchdog_device *wdd)
  		return -EBUSY;
  	}
 
+	trace_watchdog_stop(wdd);
  	if (wdd->ops->stop) {
  		clear_bit(WDOG_HW_RUNNING, &wdd->status);
  		err = wdd->ops->stop(wdd);

drivers/watchdog/watchdog_dev.c static int watchdog_set_timeout(struct watchdog_device *wdd, · line 381

Quoted 4 lines
drivers/watchdog/watchdog_dev.c +1 -0
@@ -367,6 +378,7 @@ static int watchdog_set_timeout(struct watchdog_device *wdd,
  	if (watchdog_timeout_invalid(wdd, timeout))
  		return -EINVAL;
 
+	trace_watchdog_set_timeout(wdd, timeout);

The driver has no obligation to set the timeout to the requested value. It might be more valuable to report both the requested and the actual values.

drivers/watchdog/watchdog_dev.c static int watchdog_set_pretimeout(struct watchdog_device *wdd, · line 415

Quoted 9 lines
drivers/watchdog/watchdog_dev.c +2 -0
  	if (wdd->ops->set_timeout) {
  		err = wdd->ops->set_timeout(wdd, timeout);
  	} else {
@@ -399,6 +411,8 @@ static int watchdog_set_pretimeout(struct watchdog_device *wdd,
  	if (watchdog_pretimeout_invalid(wdd, timeout))
  		return -EINVAL;
 
+	trace_watchdog_set_pretimeout(wdd, timeout);
+

Again, the driver has no obligation to set the timeout to the requested value.

Quoted 160 lines
drivers/watchdog/watchdog_dev.c +21 -0
  	if (wdd->ops->set_pretimeout && (wdd->info->options & WDIOF_PRETIMEOUT))
  		err = wdd->ops->set_pretimeout(wdd, timeout);
  	else
@@ -430,6 +444,23 @@ static int watchdog_get_timeleft(struct watchdog_device *wdd,
  	return 0;
  }
 
+/**
+ * watchdog_set_nowayout - set nowaout bit
+ * @wdd:	The watchdog device to set nowayoutbit
+ * @nowayout	A boolean on/off switcher
+ *
+ * If nowayout boolean is true, the nowayout option is set. No action is
+ * taken if nowayout is false.
+ */
+void watchdog_set_nowayout(struct watchdog_device *wdd, bool nowayout)
+{
+	if (nowayout) {
+		set_bit(WDOG_NO_WAY_OUT, &wdd->status);
+		trace_watchdog_nowayout(wdd);
+	}
+}
+EXPORT_SYMBOL(watchdog_set_nowayout);
+
  #ifdef CONFIG_WATCHDOG_SYSFS
  static ssize_t nowayout_show(struct device *dev, struct device_attribute *attr,
  				char *buf)
@@ -861,6 +892,8 @@ static int watchdog_open(struct inode *inode, struct file *file)
  		goto out_clear;
  	}
 
+	trace_watchdog_open(wdd);
+
  	err = watchdog_start(wdd);
  	if (err < 0)
  		goto out_mod;
@@ -883,6 +916,7 @@ static int watchdog_open(struct inode *inode, struct file *file)
  	return stream_open(inode, file);
 
  out_mod:
+	trace_watchdog_close(wdd);
  	module_put(wd_data->wdd->ops->owner);
  out_clear:
  	clear_bit(_WDOG_DEV_OPEN, &wd_data->status);
@@ -944,6 +978,7 @@ static int watchdog_release(struct inode *inode, struct file *file)
  	/* make sure that /dev/watchdog can be re-opened */
  	clear_bit(_WDOG_DEV_OPEN, &wd_data->status);
 
+	trace_watchdog_close(wdd);
  done:
  	running = wdd && watchdog_hw_running(wdd);
  	mutex_unlock(&wd_data->lock);
drivers/watchdog/watchdog_pretimeout.c +2 -0
index 376a495ab80c..58c391ed2205 100644
@@ -8,6 +8,7 @@
  #include <linux/spinlock.h>
  #include <linux/string.h>
  #include <linux/watchdog.h>
+#include <trace/events/watchdog.h>
 
  #include "watchdog_core.h"
  #include "watchdog_pretimeout.h"
@@ -107,6 +108,7 @@ void watchdog_notify_pretimeout(struct watchdog_device *wdd)
  		return;
  	}
 
+	trace_watchdog_pretimeout(wdd);
  	wdd->gov->pretimeout(wdd);
  	spin_unlock_irqrestore(&pretimeout_lock, flags);
  }
include/linux/watchdog.h +1 -6
index 99660197a36c..11d93407e492 100644
@@ -139,12 +139,7 @@ static inline bool watchdog_hw_running(struct watchdog_device *wdd)
  	return test_bit(WDOG_HW_RUNNING, &wdd->status);
  }
 
-/* Use the following function to set the nowayout feature */
-static inline void watchdog_set_nowayout(struct watchdog_device *wdd, bool nowayout)
-{
-	if (nowayout)
-		set_bit(WDOG_NO_WAY_OUT, &wdd->status);
-}
+void watchdog_set_nowayout(struct watchdog_device *wdd, bool nowayout);
 
  /* Use the following function to stop the watchdog on reboot */
  static inline void watchdog_stop_on_reboot(struct watchdog_device *wdd)
include/trace/events/watchdog.h new file +64 -0
new file mode 100644
index 000000000000..145cd6cfaa02
@@ -0,0 +1,101 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM watchdog
+
+#if !defined(_TRACE_WATCHDOG_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_WATCHDOG_H
+
+#include <linux/tracepoint.h>
+
+/*
+ * These are all events whose sole argument is the watchdog id.
+ */
+DECLARE_EVENT_CLASS(dev_operations_template,
+
+	TP_PROTO(struct watchdog_device *wdd),
+
+	TP_ARGS(wdd),
+
+	TP_STRUCT__entry(
+		__field(__u32, id)
+	),
+
+	TP_fast_assign(
+		__entry->id = wdd->id;
+	),
+
+	TP_printk("id=%d",
+		  __entry->id)
+);
+
+DEFINE_EVENT(dev_operations_template, watchdog_open,
+	     TP_PROTO(struct watchdog_device *wdd),
+	     TP_ARGS(wdd));
+
+DEFINE_EVENT(dev_operations_template, watchdog_close,
+	     TP_PROTO(struct watchdog_device *wdd),
+	     TP_ARGS(wdd));
+
+DEFINE_EVENT(dev_operations_template, watchdog_start,
+	     TP_PROTO(struct watchdog_device *wdd),
+	     TP_ARGS(wdd));
+
+DEFINE_EVENT(dev_operations_template, watchdog_stop,
+	     TP_PROTO(struct watchdog_device *wdd),
+	     TP_ARGS(wdd));
+
+DEFINE_EVENT(dev_operations_template, watchdog_ping,
+	     TP_PROTO(struct watchdog_device *wdd),
+	     TP_ARGS(wdd));
+
+DEFINE_EVENT(dev_operations_template, watchdog_nowayout,
+	     TP_PROTO(struct watchdog_device *wdd),
+	     TP_ARGS(wdd));
+
+DEFINE_EVENT(dev_operations_template, watchdog_keep_alive,
+	     TP_PROTO(struct watchdog_device *wdd),
+	     TP_ARGS(wdd));
+
+DEFINE_EVENT(dev_operations_template, watchdog_pretimeout,
+	     TP_PROTO(struct watchdog_device *wdd),
+	     TP_ARGS(wdd));
+
+/*
+ * These are all events with a device ID and a given timeout.

include/trace/events/watchdog.h line 74

Quoted 10 lines
include/trace/events/watchdog.h new file +10 -0
+ */
+DECLARE_EVENT_CLASS(watchdog_timeout_template,
+
+	TP_PROTO(struct watchdog_device *wdd, u64 timeout),
+
+	TP_ARGS(wdd, timeout),
+
+	TP_STRUCT__entry(
+		__field(__u32, id)
+		__field(__u64, timeout)

Why u64 ? timeout is unsigned long.

Quoted 27 lines
include/trace/events/watchdog.h new file +27 -0
+	),
+
+	TP_fast_assign(
+		__entry->id		= wdd->id;
+		__entry->timeout	= timeout;
+	),
+
+	TP_printk("id=%d timeout=%llus",
+		  __entry->id, __entry->timeout)
+);
+
+DEFINE_EVENT(watchdog_timeout_template, watchdog_set_timeout,
+	     TP_PROTO(struct watchdog_device *wdd, u64 timeout),
+	     TP_ARGS(wdd, timeout));
+
+DEFINE_EVENT(watchdog_timeout_template, watchdog_set_pretimeout,
+	     TP_PROTO(struct watchdog_device *wdd, u64 timeout),
+	     TP_ARGS(wdd, timeout));
+
+DEFINE_EVENT(watchdog_timeout_template, watchdog_set_keep_alive,
+	     TP_PROTO(struct watchdog_device *wdd, u64 timeout),
+	     TP_ARGS(wdd, timeout));
+
+#endif /* _TRACE_WATCHDOG_H */
+
+/* This part must be outside protection */
+#include <trace/define_trace.h>
#56 lore raw

Re: [PATCH V4 17/20] watchdog/dev: Add tracepoints

Quoted 116 lines · Guenter Roeck
Quoted 115 lines · Daniel Bristot de Oliveira

Add a set of tracepoints, enabling the observability of the watchdog device interactions with user-space.

The events are:
    watchdog:watchdog_open
    watchdog:watchdog_close
    watchdog:watchdog_start
    watchdog:watchdog_stop
    watchdog:watchdog_set_timeout
    watchdog:watchdog_ping
    watchdog:watchdog_nowayout
    watchdog:watchdog_set_keep_alive
    watchdog:watchdog_keep_alive
    watchdog:watchdog_set_pretimeout
    watchdog:watchdog_pretimeout

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>

---
  drivers/watchdog/watchdog_dev.c        |  43 ++++++++++- drivers/watchdog/watchdog_pretimeout.c |   2 + include/linux/watchdog.h               |   7 +- include/trace/events/watchdog.h        | 101 +++++++++++++++++++++++++

  4 files changed, 143 insertions(+), 10 deletions(-)

  create mode 100644 include/trace/events/watchdog.h

drivers/watchdog/watchdog_dev.c +0 -0
index 54903f3c851e..2f28dc5ab763 100644
@@ -44,6 +44,9 @@
   #include <linux/watchdog.h>    /* For watchdog specific items */
   #include <linux/uaccess.h>    /* For copy_to_user/put_user/... */

  +#define CREATE_TRACE_POINTS

drivers/watchdog/watchdog_dev.c +2 -0
+#include <trace/events/watchdog.h>
+

  #include "watchdog_core.h"
  #include "watchdog_pretimeout.h"
  @@ -130,9 +133,11 @@ static inline void watchdog_update_worker(struct watchdog_device *wdd)
      if (watchdog_need_worker(wdd)) {
          ktime_t t = watchdog_next_keepalive(wdd);
  -        if (t > 0)
+        if (t > 0) {
              hrtimer_start(&wd_data->timer, t, HRTIMER_MODE_REL_HARD);
+            trace_watchdog_set_keep_alive(wdd, ktime_to_ms(t)); +        }
      } else {
          hrtimer_cancel(&wd_data->timer);
      }

drivers/watchdog/watchdog_dev.c +4 -3
@@ -141,7 +146,7 @@ static inline void watchdog_update_worker(struct
 watchdog_device *wdd)
   static int __watchdog_ping(struct watchdog_device *wdd)
   {
       struct watchdog_core_data *wd_data = wdd->wd_data;
-    ktime_t earliest_keepalive, now;
+    ktime_t earliest_keepalive, now, next_keepalive;
       int err;
         earliest_keepalive = ktime_add(wd_data->last_hw_keepalive,
@@ -149,14 +154,16 @@ static int __watchdog_ping(struct watchdog_device *wdd)
       now = ktime_get();
         if (ktime_after(earliest_keepalive, now)) {
-        hrtimer_start(&wd_data->timer,
-                  ktime_sub(earliest_keepalive, now),
+        next_keepalive = ktime_sub(earliest_keepalive, now);
+        hrtimer_start(&wd_data->timer, next_keepalive,
                     HRTIMER_MODE_REL_HARD);
+        trace_watchdog_set_keep_alive(wdd, ktime_to_ms(next_keepalive));
           return 0;
       }
         wd_data->last_hw_keepalive = now;
   +    trace_watchdog_ping(wdd);
       if (wdd->ops->ping)

          err = wdd->ops->ping(wdd);  /* ping the watchdog */ else

drivers/watchdog/watchdog_dev.c +2 -0
@@ -215,6 +222,7 @@ static void watchdog_ping_work(struct kthread_work *work)
       wd_data = container_of(work, struct watchdog_core_data, work);
         mutex_lock(&wd_data->lock);
+    trace_watchdog_keep_alive(wd_data->wdd);
       if (watchdog_worker_should_ping(wd_data))
           __watchdog_ping(wd_data->wdd);
       mutex_unlock(&wd_data->lock);
@@ -250,6 +258,8 @@ static int watchdog_start(struct watchdog_device *wdd)
         set_bit(_WDOG_KEEPALIVE, &wd_data->status);
   +    trace_watchdog_start(wdd);
+
       started_at = ktime_get();
       if (watchdog_hw_running(wdd) && wdd->ops->ping) {
           err = __watchdog_ping(wdd);
@@ -294,6 +304,7 @@ static int watchdog_stop(struct watchdog_device *wdd)
           return -EBUSY;
       }
   +    trace_watchdog_stop(wdd);
       if (wdd->ops->stop) {
           clear_bit(WDOG_HW_RUNNING, &wdd->status);
           err = wdd->ops->stop(wdd);
Quoted 10 lines

drivers/watchdog/watchdog_dev.c static int watchdog_set_timeout(struct watchdog_device *wdd, · line 380

Quoted 4 lines
drivers/watchdog/watchdog_dev.c +0 -0
@@ -367,6 +378,7 @@ static int watchdog_set_timeout(struct watchdog_device *wdd,
       if (watchdog_timeout_invalid(wdd, timeout))
           return -EINVAL;
   +    trace_watchdog_set_timeout(wdd, timeout);

The driver has no obligation to set the timeout to the requested value. It might be more valuable to report both the requested and the actual values.

Ack! how do I get the actual value?

Quoted 4 lines
Quoted 4 lines
drivers/watchdog/watchdog_dev.c +0 -0
       if (wdd->ops->set_timeout) {
           err = wdd->ops->set_timeout(wdd, timeout);
       } else {
@@ -399,6 +411,8 @@ static int watchdog_set_pretimeout(struct watchdog_device
Quoted 8 lines

drivers/watchdog/watchdog_dev.c static int watchdog_set_pretimeout(struct watchdog_device *wdd, · line 415

Quoted 5 lines
drivers/watchdog/watchdog_dev.c +1 -0
 *wdd,
       if (watchdog_pretimeout_invalid(wdd, timeout))
           return -EINVAL;
   +    trace_watchdog_set_pretimeout(wdd, timeout);
+

Again, the driver has no obligation to set the timeout to the requested value.

/me takes note.

Quoted 165 lines
Quoted 165 lines
drivers/watchdog/watchdog_dev.c +0 -0
       if (wdd->ops->set_pretimeout && (wdd->info->options & WDIOF_PRETIMEOUT))
           err = wdd->ops->set_pretimeout(wdd, timeout);

      else

drivers/watchdog/watchdog_dev.c +0 -0
@@ -430,6 +444,23 @@ static int watchdog_get_timeleft(struct watchdog_device
 *wdd,
       return 0;
   }

  +/**
+ * watchdog_set_nowayout - set nowaout bit
+ * @wdd:    The watchdog device to set nowayoutbit
+ * @nowayout    A boolean on/off switcher
+ *
+ * If nowayout boolean is true, the nowayout option is set. No action is
+ * taken if nowayout is false.
+ */

drivers/watchdog/watchdog_dev.c +9 -0
+void watchdog_set_nowayout(struct watchdog_device *wdd, bool nowayout)
+{
+    if (nowayout) {
+        set_bit(WDOG_NO_WAY_OUT, &wdd->status);
+        trace_watchdog_nowayout(wdd);
+    }
+}
+EXPORT_SYMBOL(watchdog_set_nowayout);
+

  #ifdef CONFIG_WATCHDOG_SYSFS
  static ssize_t nowayout_show(struct device *dev, struct device_attribute *attr, char *buf)

drivers/watchdog/watchdog_dev.c +1 -0
@@ -861,6 +892,8 @@ static int watchdog_open(struct inode *inode, struct file
 *file)
           goto out_clear;
       }
   +    trace_watchdog_open(wdd);
+
       err = watchdog_start(wdd);
       if (err < 0)

          goto out_mod;

drivers/watchdog/watchdog_dev.c +0 -0
@@ -883,6 +916,7 @@ static int watchdog_open(struct inode *inode, struct file
 *file)
       return stream_open(inode, file);

    out_mod:
+    trace_watchdog_close(wdd); module_put(wd_data->wdd->ops->owner); out_clear:
      clear_bit(_WDOG_DEV_OPEN, &wd_data->status);

drivers/watchdog/watchdog_dev.c +0 -0
@@ -944,6 +978,7 @@ static int watchdog_release(struct inode *inode, struct
 file *file)
       /* make sure that /dev/watchdog can be re-opened */
       clear_bit(_WDOG_DEV_OPEN, &wd_data->status);
   +    trace_watchdog_close(wdd);

  done:
      running = wdd && watchdog_hw_running(wdd);
      mutex_unlock(&wd_data->lock);

drivers/watchdog/watchdog_pretimeout.c +1 -0
index 376a495ab80c..58c391ed2205 100644
@@ -8,6 +8,7 @@
   #include <linux/spinlock.h>
   #include <linux/string.h>
   #include <linux/watchdog.h>
+#include <trace/events/watchdog.h>
     #include "watchdog_core.h"
   #include "watchdog_pretimeout.h"
@@ -107,6 +108,7 @@ void watchdog_notify_pretimeout(struct watchdog_device *wdd)
           return;
       }
   +    trace_watchdog_pretimeout(wdd);
       wdd->gov->pretimeout(wdd);
       spin_unlock_irqrestore(&pretimeout_lock, flags);
   }
include/linux/watchdog.h +0 -0
index 99660197a36c..11d93407e492 100644
@@ -139,12 +139,7 @@ static inline bool watchdog_hw_running(struct
 watchdog_device *wdd)
       return test_bit(WDOG_HW_RUNNING, &wdd->status);
   }

  -/* Use the following function to set the nowayout feature */ -static inline void watchdog_set_nowayout(struct watchdog_device *wdd, bool nowayout)

include/linux/watchdog.h +1 -4
-{
-    if (nowayout)
-        set_bit(WDOG_NO_WAY_OUT, &wdd->status);
-}
+void watchdog_set_nowayout(struct watchdog_device *wdd, bool nowayout);

    /* Use the following function to stop the watchdog on reboot */ static inline void watchdog_stop_on_reboot(struct watchdog_device *wdd)

include/trace/events/watchdog.h new file +69 -0
new file mode 100644
index 000000000000..145cd6cfaa02
@@ -0,0 +1,101 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM watchdog
+
+#if !defined(_TRACE_WATCHDOG_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_WATCHDOG_H
+
+#include <linux/tracepoint.h>
+
+/*
+ * These are all events whose sole argument is the watchdog id.
+ */
+DECLARE_EVENT_CLASS(dev_operations_template,
+
+    TP_PROTO(struct watchdog_device *wdd),
+
+    TP_ARGS(wdd),
+
+    TP_STRUCT__entry(
+        __field(__u32, id)
+    ),
+
+    TP_fast_assign(
+        __entry->id = wdd->id;
+    ),
+
+    TP_printk("id=%d",
+          __entry->id)
+);
+
+DEFINE_EVENT(dev_operations_template, watchdog_open,
+         TP_PROTO(struct watchdog_device *wdd),
+         TP_ARGS(wdd));
+
+DEFINE_EVENT(dev_operations_template, watchdog_close,
+         TP_PROTO(struct watchdog_device *wdd),
+         TP_ARGS(wdd));
+
+DEFINE_EVENT(dev_operations_template, watchdog_start,
+         TP_PROTO(struct watchdog_device *wdd),
+         TP_ARGS(wdd));
+
+DEFINE_EVENT(dev_operations_template, watchdog_stop,
+         TP_PROTO(struct watchdog_device *wdd),
+         TP_ARGS(wdd));
+
+DEFINE_EVENT(dev_operations_template, watchdog_ping,
+         TP_PROTO(struct watchdog_device *wdd),
+         TP_ARGS(wdd));
+
+DEFINE_EVENT(dev_operations_template, watchdog_nowayout,
+         TP_PROTO(struct watchdog_device *wdd),
+         TP_ARGS(wdd));
+
+DEFINE_EVENT(dev_operations_template, watchdog_keep_alive,
+         TP_PROTO(struct watchdog_device *wdd),
+         TP_ARGS(wdd));
+
+DEFINE_EVENT(dev_operations_template, watchdog_pretimeout,
+         TP_PROTO(struct watchdog_device *wdd),
+         TP_ARGS(wdd));
+
+/*
+ * These are all events with a device ID and a given timeout.
+ */
+DECLARE_EVENT_CLASS(watchdog_timeout_template,
+
+    TP_PROTO(struct watchdog_device *wdd, u64 timeout),
+
Quoted 8 lines

include/trace/events/watchdog.h line 74

Quoted 5 lines
include/trace/events/watchdog.h new file +5 -0
+    TP_ARGS(wdd, timeout),
+
+    TP_STRUCT__entry(
+        __field(__u32, id)
+        __field(__u64, timeout)

Why u64 ? timeout is unsigned long.

ack! I will change it. (I am seeing unsigned int, am I missing something?).

Thanks!
-- Daniel

maintainer
#57 lore raw

Re: [PATCH V4 17/20] watchdog/dev: Add tracepoints

Quoted 128 lines · Daniel Bristot de Oliveira
Quoted 116 lines · Guenter Roeck
Quoted 115 lines · Daniel Bristot de Oliveira

Add a set of tracepoints, enabling the observability of the watchdog device interactions with user-space.

The events are:
    watchdog:watchdog_open
    watchdog:watchdog_close
    watchdog:watchdog_start
    watchdog:watchdog_stop
    watchdog:watchdog_set_timeout
    watchdog:watchdog_ping
    watchdog:watchdog_nowayout
    watchdog:watchdog_set_keep_alive
    watchdog:watchdog_keep_alive
    watchdog:watchdog_set_pretimeout
    watchdog:watchdog_pretimeout

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
   drivers/watchdog/watchdog_dev.c        |  43 ++++++++++-
   drivers/watchdog/watchdog_pretimeout.c |   2 +
   include/linux/watchdog.h               |   7 +-
   include/trace/events/watchdog.h        | 101 +++++++++++++++++++++++++
   4 files changed, 143 insertions(+), 10 deletions(-)

  create mode 100644 include/trace/events/watchdog.h

drivers/watchdog/watchdog_dev.c +2 -0
index 54903f3c851e..2f28dc5ab763 100644
@@ -44,6 +44,9 @@
   #include <linux/watchdog.h>    /* For watchdog specific items */
   #include <linux/uaccess.h>    /* For copy_to_user/put_user/... */
   +#define CREATE_TRACE_POINTS
+#include <trace/events/watchdog.h>
+
   #include "watchdog_core.h"
   #include "watchdog_pretimeout.h"
   @@ -130,9 +133,11 @@ static inline void watchdog_update_worker(struct

watchdog_device *wdd)
      if (watchdog_need_worker(wdd)) {
          ktime_t t = watchdog_next_keepalive(wdd);
  -        if (t > 0)
+        if (t > 0) {
              hrtimer_start(&wd_data->timer, t, HRTIMER_MODE_REL_HARD);
+            trace_watchdog_set_keep_alive(wdd, ktime_to_ms(t)); +        }
      } else {
          hrtimer_cancel(&wd_data->timer);
      }

drivers/watchdog/watchdog_dev.c +6 -3
@@ -141,7 +146,7 @@ static inline void watchdog_update_worker(struct
 watchdog_device *wdd)
   static int __watchdog_ping(struct watchdog_device *wdd)
   {
       struct watchdog_core_data *wd_data = wdd->wd_data;
-    ktime_t earliest_keepalive, now;
+    ktime_t earliest_keepalive, now, next_keepalive;
       int err;
         earliest_keepalive = ktime_add(wd_data->last_hw_keepalive,
@@ -149,14 +154,16 @@ static int __watchdog_ping(struct watchdog_device *wdd)
       now = ktime_get();
         if (ktime_after(earliest_keepalive, now)) {
-        hrtimer_start(&wd_data->timer,
-                  ktime_sub(earliest_keepalive, now),
+        next_keepalive = ktime_sub(earliest_keepalive, now);
+        hrtimer_start(&wd_data->timer, next_keepalive,
                     HRTIMER_MODE_REL_HARD);
+        trace_watchdog_set_keep_alive(wdd, ktime_to_ms(next_keepalive));
           return 0;
       }
         wd_data->last_hw_keepalive = now;
   +    trace_watchdog_ping(wdd);
       if (wdd->ops->ping)
           err = wdd->ops->ping(wdd);  /* ping the watchdog */
       else
@@ -215,6 +222,7 @@ static void watchdog_ping_work(struct kthread_work *work)
       wd_data = container_of(work, struct watchdog_core_data, work);
         mutex_lock(&wd_data->lock);
+    trace_watchdog_keep_alive(wd_data->wdd);
       if (watchdog_worker_should_ping(wd_data))
           __watchdog_ping(wd_data->wdd);
       mutex_unlock(&wd_data->lock);
@@ -250,6 +258,8 @@ static int watchdog_start(struct watchdog_device *wdd)
         set_bit(_WDOG_KEEPALIVE, &wd_data->status);
   +    trace_watchdog_start(wdd);
+
       started_at = ktime_get();
       if (watchdog_hw_running(wdd) && wdd->ops->ping) {
           err = __watchdog_ping(wdd);
@@ -294,6 +304,7 @@ static int watchdog_stop(struct watchdog_device *wdd)
           return -EBUSY;
       }
   +    trace_watchdog_stop(wdd);
       if (wdd->ops->stop) {
           clear_bit(WDOG_HW_RUNNING, &wdd->status);
           err = wdd->ops->stop(wdd);
Quoted 10 lines

drivers/watchdog/watchdog_dev.c static int watchdog_set_timeout(struct watchdog_device *wdd, · line 380

Quoted 4 lines
drivers/watchdog/watchdog_dev.c +0 -0
@@ -367,6 +378,7 @@ static int watchdog_set_timeout(struct watchdog_device *wdd,
       if (watchdog_timeout_invalid(wdd, timeout))
           return -EINVAL;
   +    trace_watchdog_set_timeout(wdd, timeout);

The driver has no obligation to set the timeout to the requested value. It might be more valuable to report both the requested and the actual values.

Quoted 2 lines

Ack! how do I get the actual value?

Read it from the data structure after the driver function returned.

Quoted 189 lines
Quoted 4 lines
Quoted 4 lines
drivers/watchdog/watchdog_dev.c +0 -0
       if (wdd->ops->set_timeout) {
           err = wdd->ops->set_timeout(wdd, timeout);
       } else {
@@ -399,6 +411,8 @@ static int watchdog_set_pretimeout(struct watchdog_device
Quoted 8 lines

drivers/watchdog/watchdog_dev.c static int watchdog_set_pretimeout(struct watchdog_device *wdd, · line 415

Quoted 5 lines
drivers/watchdog/watchdog_dev.c +1 -0
 *wdd,
       if (watchdog_pretimeout_invalid(wdd, timeout))
           return -EINVAL;
   +    trace_watchdog_set_pretimeout(wdd, timeout);
+

Again, the driver has no obligation to set the timeout to the requested value.

/me takes note.

Quoted 165 lines
Quoted 165 lines
drivers/watchdog/watchdog_dev.c +18 -0
       if (wdd->ops->set_pretimeout && (wdd->info->options & WDIOF_PRETIMEOUT))
           err = wdd->ops->set_pretimeout(wdd, timeout);
       else
@@ -430,6 +444,23 @@ static int watchdog_get_timeleft(struct watchdog_device
 *wdd,
       return 0;
   }
   +/**
+ * watchdog_set_nowayout - set nowaout bit
+ * @wdd:    The watchdog device to set nowayoutbit
+ * @nowayout    A boolean on/off switcher
+ *
+ * If nowayout boolean is true, the nowayout option is set. No action is
+ * taken if nowayout is false.
+ */
+void watchdog_set_nowayout(struct watchdog_device *wdd, bool nowayout)
+{
+    if (nowayout) {
+        set_bit(WDOG_NO_WAY_OUT, &wdd->status);
+        trace_watchdog_nowayout(wdd);
+    }
+}
+EXPORT_SYMBOL(watchdog_set_nowayout);
+
   #ifdef CONFIG_WATCHDOG_SYSFS
   static ssize_t nowayout_show(struct device *dev, struct device_attribute *attr,
                   char *buf)
@@ -861,6 +892,8 @@ static int watchdog_open(struct inode *inode, struct file
 *file)
           goto out_clear;
       }
   +    trace_watchdog_open(wdd);
+
       err = watchdog_start(wdd);
       if (err < 0)
           goto out_mod;
@@ -883,6 +916,7 @@ static int watchdog_open(struct inode *inode, struct file
 *file)
       return stream_open(inode, file);
     out_mod:
+    trace_watchdog_close(wdd);
       module_put(wd_data->wdd->ops->owner);
   out_clear:
       clear_bit(_WDOG_DEV_OPEN, &wd_data->status);
@@ -944,6 +978,7 @@ static int watchdog_release(struct inode *inode, struct
 file *file)
       /* make sure that /dev/watchdog can be re-opened */
       clear_bit(_WDOG_DEV_OPEN, &wd_data->status);
   +    trace_watchdog_close(wdd);
   done:
       running = wdd && watchdog_hw_running(wdd);
       mutex_unlock(&wd_data->lock);
drivers/watchdog/watchdog_pretimeout.c +1 -0
index 376a495ab80c..58c391ed2205 100644
@@ -8,6 +8,7 @@
   #include <linux/spinlock.h>
   #include <linux/string.h>
   #include <linux/watchdog.h>
+#include <trace/events/watchdog.h>
     #include "watchdog_core.h"
   #include "watchdog_pretimeout.h"
@@ -107,6 +108,7 @@ void watchdog_notify_pretimeout(struct watchdog_device *wdd)
           return;
       }
   +    trace_watchdog_pretimeout(wdd);
       wdd->gov->pretimeout(wdd);
       spin_unlock_irqrestore(&pretimeout_lock, flags);
   }
include/linux/watchdog.h +1 -5
index 99660197a36c..11d93407e492 100644
@@ -139,12 +139,7 @@ static inline bool watchdog_hw_running(struct
 watchdog_device *wdd)
       return test_bit(WDOG_HW_RUNNING, &wdd->status);
   }
   -/* Use the following function to set the nowayout feature */
-static inline void watchdog_set_nowayout(struct watchdog_device *wdd, bool
 nowayout)
-{
-    if (nowayout)
-        set_bit(WDOG_NO_WAY_OUT, &wdd->status);
-}
+void watchdog_set_nowayout(struct watchdog_device *wdd, bool nowayout);
     /* Use the following function to stop the watchdog on reboot */
   static inline void watchdog_stop_on_reboot(struct watchdog_device *wdd)
include/trace/events/watchdog.h new file +69 -0
new file mode 100644
index 000000000000..145cd6cfaa02
@@ -0,0 +1,101 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM watchdog
+
+#if !defined(_TRACE_WATCHDOG_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_WATCHDOG_H
+
+#include <linux/tracepoint.h>
+
+/*
+ * These are all events whose sole argument is the watchdog id.
+ */
+DECLARE_EVENT_CLASS(dev_operations_template,
+
+    TP_PROTO(struct watchdog_device *wdd),
+
+    TP_ARGS(wdd),
+
+    TP_STRUCT__entry(
+        __field(__u32, id)
+    ),
+
+    TP_fast_assign(
+        __entry->id = wdd->id;
+    ),
+
+    TP_printk("id=%d",
+          __entry->id)
+);
+
+DEFINE_EVENT(dev_operations_template, watchdog_open,
+         TP_PROTO(struct watchdog_device *wdd),
+         TP_ARGS(wdd));
+
+DEFINE_EVENT(dev_operations_template, watchdog_close,
+         TP_PROTO(struct watchdog_device *wdd),
+         TP_ARGS(wdd));
+
+DEFINE_EVENT(dev_operations_template, watchdog_start,
+         TP_PROTO(struct watchdog_device *wdd),
+         TP_ARGS(wdd));
+
+DEFINE_EVENT(dev_operations_template, watchdog_stop,
+         TP_PROTO(struct watchdog_device *wdd),
+         TP_ARGS(wdd));
+
+DEFINE_EVENT(dev_operations_template, watchdog_ping,
+         TP_PROTO(struct watchdog_device *wdd),
+         TP_ARGS(wdd));
+
+DEFINE_EVENT(dev_operations_template, watchdog_nowayout,
+         TP_PROTO(struct watchdog_device *wdd),
+         TP_ARGS(wdd));
+
+DEFINE_EVENT(dev_operations_template, watchdog_keep_alive,
+         TP_PROTO(struct watchdog_device *wdd),
+         TP_ARGS(wdd));
+
+DEFINE_EVENT(dev_operations_template, watchdog_pretimeout,
+         TP_PROTO(struct watchdog_device *wdd),
+         TP_ARGS(wdd));
+
+/*
+ * These are all events with a device ID and a given timeout.
+ */
+DECLARE_EVENT_CLASS(watchdog_timeout_template,
+
+    TP_PROTO(struct watchdog_device *wdd, u64 timeout),
+
Quoted 8 lines

include/trace/events/watchdog.h line 74

Quoted 5 lines
include/trace/events/watchdog.h new file +5 -0
+    TP_ARGS(wdd, timeout),
+
+    TP_STRUCT__entry(
+        __field(__u32, id)
+        __field(__u64, timeout)

Why u64 ? timeout is unsigned long.

Quoted 1 line

ack! I will change it. (I am seeing unsigned int, am I missing something?).

Yes, you are correct.

Guenter

#58 lore raw

Re: [PATCH V4 17/20] watchdog/dev: Add tracepoints

Quoted 130 lines · Guenter Roeck
Quoted 129 lines · Daniel Bristot de Oliveira
Quoted 127 lines · Guenter Roeck
Quoted 117 lines · Daniel Bristot de Oliveira

Add a set of tracepoints, enabling the observability of the watchdog device interactions with user-space.

The events are:
     watchdog:watchdog_open watchdog:watchdog_close watchdog:watchdog_start watchdog:watchdog_stop watchdog:watchdog_set_timeout watchdog:watchdog_ping watchdog:watchdog_nowayout watchdog:watchdog_set_keep_alive watchdog:watchdog_keep_alive watchdog:watchdog_set_pretimeout watchdog:watchdog_pretimeout

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>

---
   drivers/watchdog/watchdog_dev.c        |  43 ++++++++++- drivers/watchdog/watchdog_pretimeout.c |   2 + include/linux/watchdog.h               |   7 +- include/trace/events/watchdog.h        | 101 +++++++++++++++++++++++++ 4 files changed, 143 insertions(+), 10 deletions(-)
   create mode 100644 include/trace/events/watchdog.h

diff --git a/drivers/watchdog/watchdog_dev.c b/drivers/watchdog/watchdog_dev.c index 54903f3c851e..2f28dc5ab763 100644
--- a/drivers/watchdog/watchdog_dev.c
+++ b/drivers/watchdog/watchdog_dev.c
@@ -44,6 +44,9 @@
   #include <linux/watchdog.h>    /* For watchdog specific items */ #include <linux/uaccess.h>    /* For copy_to_user/put_user/... */ +#define CREATE_TRACE_POINTS

drivers/watchdog/watchdog_dev.c +2 -0
+#include <trace/events/watchdog.h>
+

   #include "watchdog_core.h"
   #include "watchdog_pretimeout.h"
   @@ -130,9 +133,11 @@ static inline void watchdog_update_worker(struct watchdog_device *wdd)
       if (watchdog_need_worker(wdd)) {
           ktime_t t = watchdog_next_keepalive(wdd);
   -        if (t > 0)
+        if (t > 0) {
               hrtimer_start(&wd_data->timer, t, HRTIMER_MODE_REL_HARD);
+            trace_watchdog_set_keep_alive(wdd, ktime_to_ms(t));
+        }
       } else {
           hrtimer_cancel(&wd_data->timer);
       }
@@ -141,7 +146,7 @@ static inline void watchdog_update_worker(struct watchdog_device *wdd)
   static int __watchdog_ping(struct watchdog_device *wdd)
   {
       struct watchdog_core_data *wd_data = wdd->wd_data;
-    ktime_t earliest_keepalive, now;
+    ktime_t earliest_keepalive, now, next_keepalive;
       int err;
         earliest_keepalive = ktime_add(wd_data->last_hw_keepalive, @@ -149,14 +154,16 @@ static int __watchdog_ping(struct watchdog_device *wdd) now = ktime_get();
         if (ktime_after(earliest_keepalive, now)) {
-        hrtimer_start(&wd_data->timer,
-                  ktime_sub(earliest_keepalive, now),
+        next_keepalive = ktime_sub(earliest_keepalive, now);
+        hrtimer_start(&wd_data->timer, next_keepalive, HRTIMER_MODE_REL_HARD);
+        trace_watchdog_set_keep_alive(wdd, ktime_to_ms(next_keepalive)); return 0;
       }
         wd_data->last_hw_keepalive = now;
   +    trace_watchdog_ping(wdd);
       if (wdd->ops->ping)
           err = wdd->ops->ping(wdd);  /* ping the watchdog */
       else
@@ -215,6 +222,7 @@ static void watchdog_ping_work(struct kthread_work *work) wd_data = container_of(work, struct watchdog_core_data, work); mutex_lock(&wd_data->lock);
+    trace_watchdog_keep_alive(wd_data->wdd);
       if (watchdog_worker_should_ping(wd_data)) __watchdog_ping(wd_data->wdd); mutex_unlock(&wd_data->lock);
@@ -250,6 +258,8 @@ static int watchdog_start(struct watchdog_device *wdd) set_bit(_WDOG_KEEPALIVE, &wd_data->status);
   +    trace_watchdog_start(wdd);
+
       started_at = ktime_get();
       if (watchdog_hw_running(wdd) && wdd->ops->ping) {
           err = __watchdog_ping(wdd);
@@ -294,6 +304,7 @@ static int watchdog_stop(struct watchdog_device *wdd) return -EBUSY;
       }
   +    trace_watchdog_stop(wdd);
       if (wdd->ops->stop) {
           clear_bit(WDOG_HW_RUNNING, &wdd->status);
           err = wdd->ops->stop(wdd);
@@ -367,6 +378,7 @@ static int watchdog_set_timeout(struct watchdog_device *wdd,

Quoted 3 lines

       if (watchdog_timeout_invalid(wdd, timeout)) return -EINVAL;
   +    trace_watchdog_set_timeout(wdd, timeout);

The driver has no obligation to set the timeout to the requested value. It might be more valuable to report both the requested and the actual values.

Quoted 3 lines
Quoted 2 lines

Ack! how do I get the actual value?

Read it from the data structure after the driver function returned.

Got it! I will add this field too.

-- Daniel

#59 lore raw

Re: [PATCH V4 17/20] watchdog/dev: Add tracepoints

In reply to Guenter Roeck
Quoted 4 lines · Guenter Roeck
Quoted 2 lines

ack! I will change it. (I am seeing unsigned int, am I missing something?).

Yes, you are correct.

Fixed!

-- Daniel

#60 lore raw

PATCH v4 18/20 rv/monitor: Add safe watchdog monitor

In reply to Daniel Bristot de Oliveira

The watchdog is an essential building block for the usage of Linux in safety-critical systems because it allows the system to be monitored from an external element - the watchdog hardware, acting as a safety-monitor.

A user-space application controls the watchdog device via the watchdog interface. This application, hereafter safety_app, enables the watchdog and periodically pets the watchdog upon correct completion of the safety related processing.

If the safety_app, for any reason, stops pinging the watchdog, the watchdog hardware can set the system in a fail-safe state. For example, shutting the system down.

Given the importance of the safety_app / watchdog hardware couple, the interaction between these software pieces also needs some sort of monitoring. In other words, "who monitors the monitor?"

The safe watchdog (safe_wtd) RV monitor monitors the interaction between the safety_app and the watchdog device, enforcing the correct sequence of events that leads the system to a safe state.

Furthermore, the safety_app can monitor the RV monitor by collecting the events generated by the RV monitor itself via tracing interface. In this way, closing the monitoring loop with the safety_app.

To reach a safe state, the safe_wtd RV monitor requires the safety_app to:

- Open the watchdog device
- Start the watchdog
- Set a timeout
- ping at least once

The RV monitor also avoids some undesired actions. For example, to have other threads to touch the watchdog.

The monitor also has a set of options, enabled via kernel command line/module options. They are:

- watchdog_id: the device id to monitor (default 0).
- dont_stop: once enabled, do not allow the RV monitor to be stopped (default off);
- safe_timeout: define a maximum safe value that an user-space application can set as the watchdog timeout (default unlimited).
- check_timeout: After every ping, check if the time left in the watchdog is less than or equal to the last timeout set for the watchdog. It only works for watchdog devices that provide the get_timeleft() function (default off).

For further information, please refer to:
Documentation/trace/rv/watchdog-monitor.rst

The monitor specification was developed together with Gabriele Paoloni, in the context of the Linux Foundation Elisa Project.

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 include/trace/events/rv.h                    |  11 +
 kernel/trace/rv/Kconfig                      |  10 +
 kernel/trace/rv/Makefile                     |   1 +
 kernel/trace/rv/monitors/safe_wtd/safe_wtd.c | 300 +++++++++++++++++++
 kernel/trace/rv/monitors/safe_wtd/safe_wtd.h |  84 ++++++
 5 files changed, 406 insertions(+)
 create mode 100644 kernel/trace/rv/monitors/safe_wtd/safe_wtd.c
 create mode 100644 kernel/trace/rv/monitors/safe_wtd/safe_wtd.h
include/trace/events/rv.h +11 -0
index 00f11a8dac3b..895eb3435ed7 100644
@@ -66,6 +66,17 @@ DEFINE_EVENT(error_da_monitor, error_wip,
 	     TP_PROTO(char *state, char *event),
 	     TP_ARGS(state, event));
 #endif /* CONFIG_RV_MON_WIP */
+
+#ifdef CONFIG_RV_MON_SAFE_WTD
+DEFINE_EVENT(event_da_monitor, event_safe_wtd,
+	     TP_PROTO(char *state, char *event, char *next_state, bool safe),
+	     TP_ARGS(state, event, next_state, safe));
+
+DEFINE_EVENT(error_da_monitor, error_safe_wtd,
+	     TP_PROTO(char *state, char *event),
+	     TP_ARGS(state, event));
+#endif /* CONFIG_RV_MON_SAFE_WTD */
+
 #endif /* CONFIG_DA_MON_EVENTS_IMPLICIT */
 
 #ifdef CONFIG_DA_MON_EVENTS_ID
kernel/trace/rv/Kconfig +10 -0
index 21f03fb3101a..b14ae63e792b 100644
@@ -45,6 +45,16 @@ config RV_MON_WWNR
 	  illustrates the usage of per-task monitor. The model is
 	  broken on purpose: it serves to test reactors.
 
+config RV_MON_SAFE_WTD
+	select DA_MON_EVENTS_IMPLICIT
+	bool "Safety watchdog"
+	help
+	  Enable safe_wtd, this monitor observes the interaction
+	  between a user-space safety monitor and a watchdog device.
+
+	  For futher information see:
+	    Documentation/trace/rv/safety-monitor.rst
+
 config RV_REACTORS
 	bool "Runtime verification reactors"
 	default y if RV
kernel/trace/rv/Makefile +1 -0
index 963d14875b45..904db96c7eae 100644
@@ -3,6 +3,7 @@
 obj-$(CONFIG_RV) += rv.o
 obj-$(CONFIG_RV_MON_WIP) += monitors/wip/wip.o
 obj-$(CONFIG_RV_MON_WWNR) += monitors/wwnr/wwnr.o
+obj-$(CONFIG_RV_MON_SAFE_WTD) += monitors/safe_wtd/safe_wtd.o
 obj-$(CONFIG_RV_REACTORS) += rv_reactors.o
 obj-$(CONFIG_RV_REACT_PRINTK) += reactor_printk.o
 obj-$(CONFIG_RV_REACT_PANIC) += reactor_panic.o
kernel/trace/rv/monitors/safe_wtd/safe_wtd.c new file +300 -0

300 lines not on this page · raw message

kernel/trace/rv/monitors/safe_wtd/safe_wtd.h new file +84 -0

84 lines not on this page · raw message

Signature · 1 line
2.35.1
maintainer
#61 lore raw

Re: [PATCH V4 18/20] rv/monitor: Add safe watchdog monitor

Quoted 215 lines · Daniel Bristot de Oliveira

The watchdog is an essential building block for the usage of Linux in safety-critical systems because it allows the system to be monitored from an external element - the watchdog hardware, acting as a safety-monitor.

A user-space application controls the watchdog device via the watchdog interface. This application, hereafter safety_app, enables the watchdog and periodically pets the watchdog upon correct completion of the safety related processing.

If the safety_app, for any reason, stops pinging the watchdog, the watchdog hardware can set the system in a fail-safe state. For example, shutting the system down.

Given the importance of the safety_app / watchdog hardware couple, the interaction between these software pieces also needs some sort of monitoring. In other words, "who monitors the monitor?"

The safe watchdog (safe_wtd) RV monitor monitors the interaction between the safety_app and the watchdog device, enforcing the correct sequence of events that leads the system to a safe state.

Furthermore, the safety_app can monitor the RV monitor by collecting the events generated by the RV monitor itself via tracing interface. In this way, closing the monitoring loop with the safety_app.

To reach a safe state, the safe_wtd RV monitor requires the safety_app to:

- Open the watchdog device
- Start the watchdog
- Set a timeout
- ping at least once

The RV monitor also avoids some undesired actions. For example, to have other threads to touch the watchdog.

The monitor also has a set of options, enabled via kernel command line/module options. They are:

- watchdog_id: the device id to monitor (default 0).
- dont_stop: once enabled, do not allow the RV monitor to be stopped (default off);
- safe_timeout: define a maximum safe value that an user-space application can set as the watchdog timeout (default unlimited).
- check_timeout: After every ping, check if the time left in the watchdog is less than or equal to the last timeout set for the watchdog. It only works for watchdog devices that provide the get_timeleft() function (default off).

For further information, please refer to:
Documentation/trace/rv/watchdog-monitor.rst

The monitor specification was developed together with Gabriele Paoloni, in the context of the Linux Foundation Elisa Project.

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
  include/trace/events/rv.h                    |  11 +
  kernel/trace/rv/Kconfig                      |  10 +
  kernel/trace/rv/Makefile                     |   1 +
  kernel/trace/rv/monitors/safe_wtd/safe_wtd.c | 300 +++++++++++++++++++
  kernel/trace/rv/monitors/safe_wtd/safe_wtd.h |  84 ++++++
  5 files changed, 406 insertions(+)

create mode 100644 kernel/trace/rv/monitors/safe_wtd/safe_wtd.c create mode 100644 kernel/trace/rv/monitors/safe_wtd/safe_wtd.h

include/trace/events/rv.h +11 -0
index 00f11a8dac3b..895eb3435ed7 100644
@@ -66,6 +66,17 @@ DEFINE_EVENT(error_da_monitor, error_wip,
  	     TP_PROTO(char *state, char *event),
  	     TP_ARGS(state, event));
  #endif /* CONFIG_RV_MON_WIP */
+
+#ifdef CONFIG_RV_MON_SAFE_WTD
+DEFINE_EVENT(event_da_monitor, event_safe_wtd,
+	     TP_PROTO(char *state, char *event, char *next_state, bool safe),
+	     TP_ARGS(state, event, next_state, safe));
+
+DEFINE_EVENT(error_da_monitor, error_safe_wtd,
+	     TP_PROTO(char *state, char *event),
+	     TP_ARGS(state, event));
+#endif /* CONFIG_RV_MON_SAFE_WTD */
+
  #endif /* CONFIG_DA_MON_EVENTS_IMPLICIT */
 
  #ifdef CONFIG_DA_MON_EVENTS_ID
kernel/trace/rv/Kconfig +10 -0
index 21f03fb3101a..b14ae63e792b 100644
@@ -45,6 +45,16 @@ config RV_MON_WWNR
  	  illustrates the usage of per-task monitor. The model is
  	  broken on purpose: it serves to test reactors.
 
+config RV_MON_SAFE_WTD
+	select DA_MON_EVENTS_IMPLICIT
+	bool "Safety watchdog"
+	help
+	  Enable safe_wtd, this monitor observes the interaction
+	  between a user-space safety monitor and a watchdog device.
+
+	  For futher information see:
+	    Documentation/trace/rv/safety-monitor.rst
+
  config RV_REACTORS
  	bool "Runtime verification reactors"
  	default y if RV
kernel/trace/rv/Makefile +1 -0
index 963d14875b45..904db96c7eae 100644
@@ -3,6 +3,7 @@
  obj-$(CONFIG_RV) += rv.o
  obj-$(CONFIG_RV_MON_WIP) += monitors/wip/wip.o
  obj-$(CONFIG_RV_MON_WWNR) += monitors/wwnr/wwnr.o
+obj-$(CONFIG_RV_MON_SAFE_WTD) += monitors/safe_wtd/safe_wtd.o
  obj-$(CONFIG_RV_REACTORS) += rv_reactors.o
  obj-$(CONFIG_RV_REACT_PRINTK) += reactor_printk.o
  obj-$(CONFIG_RV_REACT_PANIC) += reactor_panic.o
kernel/trace/rv/monitors/safe_wtd/safe_wtd.c new file +68 -0
new file mode 100644
index 000000000000..9856e0770d0d
@@ -0,0 +1,300 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/ftrace.h>
+#include <linux/tracepoint.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/rv.h>
+#include <rv/instrumentation.h>
+#include <rv/da_monitor.h>
+
+#include <linux/watchdog.h>
+#include <linux/moduleparam.h>
+
+#include <trace/events/rv.h>
+#include <trace/events/watchdog.h>
+
+#define MODULE_NAME "safe_wtd"
+
+/*
+ * This is the self-generated part of the monitor. Generally, there is no need
+ * to touch this section.
+ */
+#include "safe_wtd.h"
+
+/*
+ * Declare the deterministic automata monitor.
+ *
+ * The rv monitor reference is needed for the monitor declaration.
+ */
+struct rv_monitor rv_safe_wtd;
+DECLARE_DA_MON_GLOBAL(safe_wtd, char);
+
+/*
+ * custom: safe_timeout is the maximum value a watchdog monitor
+ * can set. This value is registered here to duplicate the information.
+ * In this way, a miss-behaving monitor can be detected.
+ */
+static int safe_timeout = ~0;
+module_param(safe_timeout, int, 0444);
+
+/*
+ * custom: if check_timeout is set, the monitor will check if the time left
+ * in the watchdog is less than or equals to the last safe timeout set by
+ * user-space. This check is done after each ping. In this way, if any
+ * code by-passed the watchdog dev interface setting a higher (so unsafe)
+ * timeout, this monitor will catch the side effect and react.
+ */
+static int last_timeout_set = 0;
+static int check_timeout = 0;
+module_param(check_timeout, int, 0444);
+
+/*
+ * custom: if dont_stop is set the monitor will react if stopped.
+ */
+static int dont_stop = 0;
+module_param(dont_stop, int, 0444);
+
+/*
+ * custom: there are some states that are kept after the watchdog is closed.
+ * For example, the nowayout state.
+ *
+ * Thus, the RV monitor needs to keep track of these states after a start/stop
+ * of the RV monitor itself, and should not reset after each restart - keeping the
+ * know state until the system shutdown.
+ *
+ * If for an unknown reason an RV monitor would like to reset the RV monitor at each
+ * RV monitor start, set it to one.
+ */

kernel/trace/rv/monitors/safe_wtd/safe_wtd.c line 78

Quoted 10 lines
kernel/trace/rv/monitors/safe_wtd/safe_wtd.c new file +10 -0
+static int reset_on_restart = 0;
+module_param(reset_on_restart, int, 0444);
+
+/*
+ * open_pid takes note of the first thread that opened the watchdog.
+ *
+ * Any other thread that generates an event will cause an "other_threads"
+ * event in the monitor.
+ */
+static int open_pid = 0;

Userspace could open a watchdog, create a child process, and handle it from the child. That is perfectly valid.

kernel/trace/rv/monitors/safe_wtd/safe_wtd.c line 84

Quoted 6 lines
kernel/trace/rv/monitors/safe_wtd/safe_wtd.c new file +6 -0
+
+/*
+ * watchdog_id: the watchdog to monitor
+ */
+static int watchdog_id = 0;
+module_param(watchdog_id, int, 0444);

Limiting the watcher to a single watchdog sounds less than perfect. What if the system supports more than one, more than one is enabled, and the non-monitored watchdog misbehaves ?

Quoted 70 lines
kernel/trace/rv/monitors/safe_wtd/safe_wtd.c new file +70 -0
+
+static void handle_nowayout(void *data, struct watchdog_device *wdd)
+{
+	if (wdd->id != watchdog_id)
+		return;
+
+	da_handle_init_run_event_safe_wtd(nowayout_safe_wtd);
+}
+
+static void handle_close(void *data, struct watchdog_device *wdd)
+{
+	if (wdd->id != watchdog_id)
+		return;
+
+	if (open_pid && current->pid != open_pid) {
+		da_handle_init_run_event_safe_wtd(other_threads_safe_wtd);
+	} else {
+		da_handle_event_safe_wtd(close_safe_wtd);
+		open_pid = 0;
+	}
+}
+
+static void handle_open(void *data, struct watchdog_device *wdd)
+{
+	if (wdd->id != watchdog_id)
+		return;
+
+	if (open_pid && current->pid != open_pid) {
+		da_handle_init_run_event_safe_wtd(other_threads_safe_wtd);
+	} else {
+		da_handle_init_run_event_safe_wtd(open_safe_wtd);
+		open_pid = current->pid;
+	}
+}
+
+static void blocked_events(void *data, struct watchdog_device *wdd)
+{
+	if (wdd->id != watchdog_id)
+		return;
+
+	if (open_pid && current->pid != open_pid) {
+		da_handle_init_run_event_safe_wtd(other_threads_safe_wtd);
+		return;
+	}
+	da_handle_event_safe_wtd(other_threads_safe_wtd);
+}
+
+static void blocked_events_timeout(void *data, struct watchdog_device *wdd, u64 timeout)
+{
+	blocked_events(data, wdd);
+}
+
+static void handle_ping(void *data, struct watchdog_device *wdd)
+{
+	char msg[128];
+	unsigned int timeout;
+
+	if (wdd->id != watchdog_id)
+		return;
+
+	if (open_pid && current->pid != open_pid) {
+		da_handle_init_run_event_safe_wtd(other_threads_safe_wtd);
+		return;
+	}
+
+	da_handle_event_safe_wtd(ping_safe_wtd);
+
+	if (!check_timeout)
+		return;
+

kernel/trace/rv/monitors/safe_wtd/safe_wtd.c line 164

Quoted 10 lines
kernel/trace/rv/monitors/safe_wtd/safe_wtd.c new file +10 -0
+	if (wdd->ops->get_timeleft) {
+		timeout = wdd->ops->get_timeleft(wdd);
+		if (timeout > last_timeout_set) {
+			snprintf(msg, 128,
+				 "watchdog timeout is %u > than previously set (%d)\n",
+				 timeout, last_timeout_set);
+			cond_react(msg);
+		}
+	} else {
+		snprintf(msg, 128, "error getting timeout: option not supported\n");

This is not an error. The get_timeleft callback is optional.

Quoted 215 lines
kernel/trace/rv/monitors/safe_wtd/safe_wtd.c new file +136 -0
+		cond_react(msg);
+	}
+}
+
+static void handle_set_safe_timeout(void *data, struct watchdog_device *wdd, u64 timeout)
+{
+	char msg[128];
+
+	if (wdd->id != watchdog_id)
+		return;
+
+	if (open_pid && current->pid != open_pid) {
+		da_handle_init_run_event_safe_wtd(other_threads_safe_wtd);
+		return;
+	}
+
+	da_handle_event_safe_wtd(set_safe_timeout_safe_wtd);
+
+	if (timeout > safe_timeout) {
+		snprintf(msg, 128, "set safety timeout is too high: %d", (int) timeout);
+		cond_react(msg);
+	}
+
+	if (check_timeout)
+		last_timeout_set = timeout;
+}
+
+static void handle_start(void *data, struct watchdog_device *wdd)
+{
+	if (wdd->id != watchdog_id)
+		return;
+
+	if (open_pid && current->pid != open_pid) {
+		da_handle_init_run_event_safe_wtd(other_threads_safe_wtd);
+		return;
+	}
+
+	da_handle_event_safe_wtd(start_safe_wtd);
+}
+
+static void handle_stop(void *data, struct watchdog_device *wdd)
+{
+	if (wdd->id != watchdog_id)
+		return;
+
+	if (open_pid && current->pid != open_pid) {
+		da_handle_init_run_event_safe_wtd(other_threads_safe_wtd);
+		return;
+	}
+
+	da_handle_event_safe_wtd(stop_safe_wtd);
+}
+
+static int mon_started = 0;
+
+static int start_safe_wtd_monitor(void)
+{
+	int retval;
+
+	if (!mon_started || reset_on_restart) {
+		retval = da_monitor_init_safe_wtd();
+		if (retval)
+			return retval;
+
+		mon_started = 1;
+	}
+
+	rv_attach_trace_probe("safe_wtd", watchdog_close, handle_close);
+	rv_attach_trace_probe("safe_wtd", watchdog_nowayout, handle_nowayout);
+	rv_attach_trace_probe("safe_wtd", watchdog_open, handle_open);
+	rv_attach_trace_probe("safe_wtd", watchdog_ping, handle_ping);
+	rv_attach_trace_probe("safe_wtd", watchdog_set_timeout, handle_set_safe_timeout);
+	rv_attach_trace_probe("safe_wtd", watchdog_start, handle_start);
+	rv_attach_trace_probe("safe_wtd", watchdog_stop, handle_stop);
+	rv_attach_trace_probe("safe_wtd", watchdog_set_keep_alive, blocked_events_timeout);
+	rv_attach_trace_probe("safe_wtd", watchdog_keep_alive, blocked_events);
+	rv_attach_trace_probe("safe_wtd", watchdog_set_pretimeout, blocked_events_timeout);
+	rv_attach_trace_probe("safe_wtd", watchdog_pretimeout, blocked_events);
+
+	return 0;
+}
+
+static void stop_safe_wtd_monitor(void)
+{
+	if (dont_stop)
+		cond_react("dont_stop safe_wtd is set.");
+
+	rv_safe_wtd.enabled = 0;
+
+	rv_detach_trace_probe("safe_wtd", watchdog_close, handle_close);
+	rv_detach_trace_probe("safe_wtd", watchdog_nowayout, handle_nowayout);
+	rv_detach_trace_probe("safe_wtd", watchdog_open, handle_open);
+	rv_detach_trace_probe("safe_wtd", watchdog_ping, handle_ping);
+	rv_detach_trace_probe("safe_wtd", watchdog_set_timeout, handle_set_safe_timeout);
+	rv_detach_trace_probe("safe_wtd", watchdog_start, handle_start);
+	rv_detach_trace_probe("safe_wtd", watchdog_stop, handle_stop);
+	rv_detach_trace_probe("safe_wtd", watchdog_set_keep_alive, blocked_events_timeout);
+	rv_detach_trace_probe("safe_wtd", watchdog_keep_alive, blocked_events);
+	rv_detach_trace_probe("safe_wtd", watchdog_set_pretimeout, blocked_events_timeout);
+	rv_detach_trace_probe("safe_wtd", watchdog_pretimeout, blocked_events);
+
+	da_monitor_destroy_safe_wtd();
+}
+
+/*
+ * This is the monitor register section.
+ */
+struct rv_monitor rv_safe_wtd = {
+	.name = "safe_wtd",
+	.description = "A watchdog monitor guarding a safety monitor actions",
+	.start = start_safe_wtd_monitor,
+	.stop = stop_safe_wtd_monitor,
+	.reset = da_monitor_reset_all_safe_wtd,
+	.enabled = 0,
+};
+
+int register_safe_wtd(void)
+{
+	rv_register_monitor(&rv_safe_wtd);
+	return 0;
+}
+
+void unregister_safe_wtd(void)
+{
+	if (rv_safe_wtd.enabled)
+		stop_safe_wtd_monitor();
+
+	rv_unregister_monitor(&rv_safe_wtd);
+}
+
+module_init(register_safe_wtd);
+module_exit(unregister_safe_wtd);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Daniel Bristot de Oliveira <[email protected]>");
+MODULE_DESCRIPTION("Safe watchdog RV monitor");
kernel/trace/rv/monitors/safe_wtd/safe_wtd.h new file +73 -0
new file mode 100644
index 000000000000..835c9d0979f6
@@ -0,0 +1,84 @@
+enum states_safe_wtd {
+	init_safe_wtd = 0,
+	closed_running_safe_wtd,
+	closed_running_nwo_safe_wtd,
+	nwo_safe_wtd,
+	opened_safe_wtd,
+	opened_nwo_safe_wtd,
+	reopened_safe_wtd,
+	safe_safe_wtd,
+	safe_nwo_safe_wtd,
+	set_safe_wtd,
+	set_nwo_safe_wtd,
+	started_safe_wtd,
+	started_nwo_safe_wtd,
+	stoped_safe_wtd,
+	state_max_safe_wtd
+};
+
+enum events_safe_wtd {
+	close_safe_wtd = 0,
+	nowayout_safe_wtd,
+	open_safe_wtd,
+	other_threads_safe_wtd,
+	ping_safe_wtd,
+	set_safe_timeout_safe_wtd,
+	start_safe_wtd,
+	stop_safe_wtd,
+	event_max_safe_wtd
+};
+
+struct automaton_safe_wtd {
+	char *state_names[state_max_safe_wtd];
+	char *event_names[event_max_safe_wtd];
+	char function[state_max_safe_wtd][event_max_safe_wtd];
+	char initial_state;
+	char final_states[state_max_safe_wtd];
+};
+
+struct automaton_safe_wtd automaton_safe_wtd = {
+	.state_names = {
+		"init",
+		"closed_running",
+		"closed_running_nwo",
+		"nwo",
+		"opened",
+		"opened_nwo",
+		"reopened",
+		"safe",
+		"safe_nwo",
+		"set",
+		"set_nwo",
+		"started",
+		"started_nwo",
+		"stoped"
+	},
+	.event_names = {
+		"close",
+		"nowayout",
+		"open",
+		"other_threads",
+		"ping",
+		"set_safe_timeout",
+		"start",
+		"stop"
+	},
+	.function = {
+		{                          -1,                nwo_safe_wtd,             opened_safe_wtd,               init_safe_wtd,                          -1,                          -1,                          -1,                          -1 },
+		{                          -1, closed_running_nwo_safe_wtd,           reopened_safe_wtd,     closed_running_safe_wtd,                          -1,                          -1,                          -1,                          -1 },
+		{                          -1, closed_running_nwo_safe_wtd,        started_nwo_safe_wtd, closed_running_nwo_safe_wtd,                          -1,                          -1,                          -1,                          -1 },
+		{                          -1,                nwo_safe_wtd,         opened_nwo_safe_wtd,                nwo_safe_wtd,                          -1,                          -1,                          -1,                          -1 },
+		{               init_safe_wtd,                          -1,                          -1,                          -1,                          -1,                          -1,            started_safe_wtd,                          -1 },
+		{                nwo_safe_wtd,                          -1,                          -1,                          -1,                          -1,                          -1,        started_nwo_safe_wtd,                          -1 },
+		{     closed_running_safe_wtd,                          -1,                          -1,                          -1,                          -1,                set_safe_wtd,                          -1,             opened_safe_wtd },

kernel/trace/rv/monitors/safe_wtd/safe_wtd.h line 83

Quoted 10 lines
kernel/trace/rv/monitors/safe_wtd/safe_wtd.h new file +10 -0
+		{     closed_running_safe_wtd,                          -1,                          -1,                          -1,               safe_safe_wtd,                          -1,                          -1,             stoped_safe_wtd },
+		{ closed_running_nwo_safe_wtd,                          -1,                          -1,                          -1,           safe_nwo_safe_wtd,                          -1,                          -1,                          -1 },
+		{                          -1,                          -1,                          -1,                          -1,               safe_safe_wtd,                          -1,                          -1,                          -1 },
+		{                          -1,                          -1,                          -1,                          -1,           safe_nwo_safe_wtd,                          -1,                          -1,                          -1 },
+		{     closed_running_safe_wtd,                          -1,                          -1,                          -1,                          -1,                set_safe_wtd,                          -1,             stoped_safe_wtd },
+		{ closed_running_nwo_safe_wtd,                          -1,                          -1,                          -1,                          -1,            set_nwo_safe_wtd,                          -1,                          -1 },
+		{               init_safe_wtd,                          -1,                          -1,                          -1,                          -1,                          -1,                          -1,                          -1 },
+	},
+	.initial_state = init_safe_wtd,
+	.final_states = { 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },

I find this event table all but impossible to verify.

Quoted 1 line
kernel/trace/rv/monitors/safe_wtd/safe_wtd.h new file +1 -0
+};
#62 lore raw

Re: [PATCH V4 18/20] rv/monitor: Add safe watchdog monitor

Hi Guenter,

Quoted 222 lines · Guenter Roeck
Quoted 221 lines · Daniel Bristot de Oliveira

The watchdog is an essential building block for the usage of Linux in safety-critical systems because it allows the system to be monitored from an external element - the watchdog hardware, acting as a safety-monitor.

A user-space application controls the watchdog device via the watchdog interface. This application, hereafter safety_app, enables the watchdog and periodically pets the watchdog upon correct completion of the safety related processing.

If the safety_app, for any reason, stops pinging the watchdog, the watchdog hardware can set the system in a fail-safe state. For example, shutting the system down.

Given the importance of the safety_app / watchdog hardware couple, the interaction between these software pieces also needs some sort of monitoring. In other words, "who monitors the monitor?"

The safe watchdog (safe_wtd) RV monitor monitors the interaction between the safety_app and the watchdog device, enforcing the correct sequence of events that leads the system to a safe state.

Furthermore, the safety_app can monitor the RV monitor by collecting the events generated by the RV monitor itself via tracing interface. In this way, closing the monitoring loop with the safety_app.

To reach a safe state, the safe_wtd RV monitor requires the safety_app to:

    - Open the watchdog device
    - Start the watchdog
    - Set a timeout
    - ping at least once

The RV monitor also avoids some undesired actions. For example, to have other threads to touch the watchdog.

The monitor also has a set of options, enabled via kernel command line/module options. They are:

    - watchdog_id: the device id to monitor (default 0).
    - dont_stop: once enabled, do not allow the RV monitor to be stopped (default off);
    - safe_timeout: define a maximum safe value that an user-space application can set as the watchdog timeout
        (default unlimited).
    - check_timeout: After every ping, check if the time left in the watchdog is less than or equal to the last timeout set for the watchdog. It only works for watchdog devices that provide the get_timeleft() function (default off).

For further information, please refer to:
    Documentation/trace/rv/watchdog-monitor.rst

The monitor specification was developed together with Gabriele Paoloni, in the context of the Linux Foundation Elisa Project.

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>

---
  include/trace/events/rv.h                    |  11 + kernel/trace/rv/Kconfig                      |  10 + kernel/trace/rv/Makefile                     |   1 + kernel/trace/rv/monitors/safe_wtd/safe_wtd.c | 300 +++++++++++++++++++ kernel/trace/rv/monitors/safe_wtd/safe_wtd.h |  84 ++++++

  5 files changed, 406 insertions(+)

  create mode 100644 kernel/trace/rv/monitors/safe_wtd/safe_wtd.c create mode 100644 kernel/trace/rv/monitors/safe_wtd/safe_wtd.h

include/trace/events/rv.h +11 -0
index 00f11a8dac3b..895eb3435ed7 100644
@@ -66,6 +66,17 @@ DEFINE_EVENT(error_da_monitor, error_wip,
            TP_PROTO(char *state, char *event),
            TP_ARGS(state, event));
   #endif /* CONFIG_RV_MON_WIP */
+
+#ifdef CONFIG_RV_MON_SAFE_WTD
+DEFINE_EVENT(event_da_monitor, event_safe_wtd,
+         TP_PROTO(char *state, char *event, char *next_state, bool safe),
+         TP_ARGS(state, event, next_state, safe));
+
+DEFINE_EVENT(error_da_monitor, error_safe_wtd,
+         TP_PROTO(char *state, char *event),
+         TP_ARGS(state, event));
+#endif /* CONFIG_RV_MON_SAFE_WTD */
+
   #endif /* CONFIG_DA_MON_EVENTS_IMPLICIT */
     #ifdef CONFIG_DA_MON_EVENTS_ID
kernel/trace/rv/Kconfig +0 -0
index 21f03fb3101a..b14ae63e792b 100644
@@ -45,6 +45,16 @@ config RV_MON_WWNR

        illustrates the usage of per-task monitor. The model is broken on purpose: it serves to test reactors. +config RV_MON_SAFE_WTD
+    select DA_MON_EVENTS_IMPLICIT
+    bool "Safety watchdog"
+    help
+      Enable safe_wtd, this monitor observes the interaction +      between a user-space safety monitor and a watchdog device.

kernel/trace/rv/Kconfig +4 -0
+
+      For futher information see:
+        Documentation/trace/rv/safety-monitor.rst
+

  config RV_REACTORS
      bool "Runtime verification reactors"
      default y if RV

kernel/trace/rv/Makefile +0 -0
index 963d14875b45..904db96c7eae 100644
@@ -3,6 +3,7 @@

  obj-$(CONFIG_RV) += rv.o
  obj-$(CONFIG_RV_MON_WIP) += monitors/wip/wip.o obj-$(CONFIG_RV_MON_WWNR) += monitors/wwnr/wwnr.o +obj-$(CONFIG_RV_MON_SAFE_WTD) += monitors/safe_wtd/safe_wtd.o obj-$(CONFIG_RV_REACTORS) += rv_reactors.o obj-$(CONFIG_RV_REACT_PRINTK) += reactor_printk.o obj-$(CONFIG_RV_REACT_PANIC) += reactor_panic.o

kernel/trace/rv/monitors/safe_wtd/safe_wtd.c new file +63 -0
new file mode 100644
index 000000000000..9856e0770d0d
@@ -0,0 +1,300 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/ftrace.h>
+#include <linux/tracepoint.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/rv.h>
+#include <rv/instrumentation.h>
+#include <rv/da_monitor.h>
+
+#include <linux/watchdog.h>
+#include <linux/moduleparam.h>
+
+#include <trace/events/rv.h>
+#include <trace/events/watchdog.h>
+
+#define MODULE_NAME "safe_wtd"
+
+/*
+ * This is the self-generated part of the monitor. Generally, there is no need
+ * to touch this section.
+ */
+#include "safe_wtd.h"
+
+/*
+ * Declare the deterministic automata monitor.
+ *
+ * The rv monitor reference is needed for the monitor declaration.
+ */
+struct rv_monitor rv_safe_wtd;
+DECLARE_DA_MON_GLOBAL(safe_wtd, char);
+
+/*
+ * custom: safe_timeout is the maximum value a watchdog monitor
+ * can set. This value is registered here to duplicate the information.
+ * In this way, a miss-behaving monitor can be detected.
+ */
+static int safe_timeout = ~0;
+module_param(safe_timeout, int, 0444);
+
+/*
+ * custom: if check_timeout is set, the monitor will check if the time left
+ * in the watchdog is less than or equals to the last safe timeout set by
+ * user-space. This check is done after each ping. In this way, if any
+ * code by-passed the watchdog dev interface setting a higher (so unsafe)
+ * timeout, this monitor will catch the side effect and react.
+ */
+static int last_timeout_set = 0;
+static int check_timeout = 0;
+module_param(check_timeout, int, 0444);
+
+/*
+ * custom: if dont_stop is set the monitor will react if stopped.
+ */
+static int dont_stop = 0;
+module_param(dont_stop, int, 0444);
+
+/*
+ * custom: there are some states that are kept after the watchdog is closed.
+ * For example, the nowayout state.
+ *
+ * Thus, the RV monitor needs to keep track of these states after a start/stop
+ * of the RV monitor itself, and should not reset after each restart -

keeping the
+ * know state until the system shutdown.
+ *
+ * If for an unknown reason an RV monitor would like to reset the RV monitor at each
+ * RV monitor start, set it to one.
+ */

kernel/trace/rv/monitors/safe_wtd/safe_wtd.c new file +5 -0
+static int reset_on_restart = 0;
+module_param(reset_on_restart, int, 0444);
+
+/*
+ * open_pid takes note of the first thread that opened the watchdog.
Quoted 8 lines
Quoted 5 lines

+ *
+ * Any other thread that generates an event will cause an "other_threads"
+ * event in the monitor.
+ */
+static int open_pid = 0;

Userspace could open a watchdog, create a child process, and handle it from the child. That is perfectly valid.

Right! It is a correct usage of the watchdog subsystem.

However, the idea here is to allow a "restricted" set of operations based on the safety analysis made by people in the LF Elisa Workgroup (Gabriele Paoloni in Cc:).

One of the specifications says that: only one process should touch the watchdog.

There are details about it in the "watchdog-monitor.rst," section "RV monitor specification."

There could be another monitor, a less resticted one, in which the operation you mention would be allowed.

I will complement this commit log in the next version of the patch set, clarifying that it is not a "full representation of the watchdog operations" but a restricted set of operations specified by...

Quoted 10 lines

kernel/trace/rv/monitors/safe_wtd/safe_wtd.c line 84

Quoted 6 lines
kernel/trace/rv/monitors/safe_wtd/safe_wtd.c new file +6 -0
+
+/*
+ * watchdog_id: the watchdog to monitor
+ */
+static int watchdog_id = 0;
+module_param(watchdog_id, int, 0444);

Limiting the watcher to a single watchdog sounds less than perfect. What if the system supports more than one, more than one is enabled, and the non-monitored watchdog misbehaves ?

I can add one monitor per watchdog dev. The easiest way would be adding a "struct da_monitor" variable in the watchdog_device structure, e.g.,

struct watchdog_device {
...

	#ifdef CONFIG_RV_MON_SAFE_WTD
	struct da_monitor da_mon;
	#endif

...
}

A simplified version of the the "per task" monitor, in the patch 01, changes in include/linux/sched.h.

Quoted 75 lines
Quoted 75 lines
kernel/trace/rv/monitors/safe_wtd/safe_wtd.c new file +48 -0
+
+static void handle_nowayout(void *data, struct watchdog_device *wdd)
+{
+    if (wdd->id != watchdog_id)
+        return;
+
+    da_handle_init_run_event_safe_wtd(nowayout_safe_wtd);
+}
+
+static void handle_close(void *data, struct watchdog_device *wdd)
+{
+    if (wdd->id != watchdog_id)
+        return;
+
+    if (open_pid && current->pid != open_pid) {
+        da_handle_init_run_event_safe_wtd(other_threads_safe_wtd);
+    } else {
+        da_handle_event_safe_wtd(close_safe_wtd);
+        open_pid = 0;
+    }
+}
+
+static void handle_open(void *data, struct watchdog_device *wdd)
+{
+    if (wdd->id != watchdog_id)
+        return;
+
+    if (open_pid && current->pid != open_pid) {
+        da_handle_init_run_event_safe_wtd(other_threads_safe_wtd);
+    } else {
+        da_handle_init_run_event_safe_wtd(open_safe_wtd);
+        open_pid = current->pid;
+    }
+}
+
+static void blocked_events(void *data, struct watchdog_device *wdd)
+{
+    if (wdd->id != watchdog_id)
+        return;
+
+    if (open_pid && current->pid != open_pid) {
+        da_handle_init_run_event_safe_wtd(other_threads_safe_wtd);
+        return;
+    }
+    da_handle_event_safe_wtd(other_threads_safe_wtd);
+}
+
+static void blocked_events_timeout(void *data, struct watchdog_device *wdd,

u64 timeout)

kernel/trace/rv/monitors/safe_wtd/safe_wtd.c new file +26 -0
+{
+    blocked_events(data, wdd);
+}
+
+static void handle_ping(void *data, struct watchdog_device *wdd)
+{
+    char msg[128];
+    unsigned int timeout;
+
+    if (wdd->id != watchdog_id)
+        return;
+
+    if (open_pid && current->pid != open_pid) {
+        da_handle_init_run_event_safe_wtd(other_threads_safe_wtd);
+        return;
+    }
+
+    da_handle_event_safe_wtd(ping_safe_wtd);
+
+    if (!check_timeout)
+        return;
+
+    if (wdd->ops->get_timeleft) {
+        timeout = wdd->ops->get_timeleft(wdd);
+        if (timeout > last_timeout_set) {
+            snprintf(msg, 128,
Quoted 8 lines
Quoted 6 lines

+                 "watchdog timeout is %u > than previously set (%d)\n", +                 timeout, last_timeout_set);
+            cond_react(msg);
+        }
+    } else {
+        snprintf(msg, 128, "error getting timeout: option not supported\n");

This is not an error. The get_timeleft callback is optional.

Right... but this part of the code is only reachable if the user explicitly asked to check the timeout (if (!check_timeout)...return before this code).

So, if the user only considers the system safe if the monitor also checks the written timeout, but the watchdog is one of those that do not have the callback implemented (which is ok for a Linux watchdog), the monitor captures this "undesired" behavior.

This monitor is not checking if the watchdog subsystem is correct at its plenitude, it is checking if the watchdog usage is following a set of specifications (raised by people in the LF Elisa workgroup).

Quoted 4 lines
Quoted 4 lines

+        cond_react(msg);
+    }

kernel/trace/rv/monitors/safe_wtd/safe_wtd.c new file +2 -0
+}
+

[...]

Quoted 40 lines
Quoted 40 lines
kernel/trace/rv/monitors/safe_wtd/safe_wtd.c new file +40 -0
+
+struct automaton_safe_wtd automaton_safe_wtd = {
+    .state_names = {
+        "init",
+        "closed_running",
+        "closed_running_nwo",
+        "nwo",
+        "opened",
+        "opened_nwo",
+        "reopened",
+        "safe",
+        "safe_nwo",
+        "set",
+        "set_nwo",
+        "started",
+        "started_nwo",
+        "stoped"
+    },
+    .event_names = {
+        "close",
+        "nowayout",
+        "open",
+        "other_threads",
+        "ping",
+        "set_safe_timeout",
+        "start",
+        "stop"
+    },
+    .function = {
+        {                          -1,                nwo_safe_wtd,             opened_safe_wtd,               init_safe_wtd,                          -1,                          -1,                         -1,                          -1 },
+        {                          -1, closed_running_nwo_safe_wtd,           reopened_safe_wtd,     closed_running_safe_wtd,                          -1,                          -1,                         -1,                          -1 },
+        {                          -1, closed_running_nwo_safe_wtd,        started_nwo_safe_wtd, closed_running_nwo_safe_wtd,                          -1,                          -1,                         -1,                          -1 },
+        {                          -1,                nwo_safe_wtd,         opened_nwo_safe_wtd,                nwo_safe_wtd,                          -1,                          -1,                         -1,                          -1 },
+        {               init_safe_wtd,                          -1,                          -1,                          -1,                          -1,                          -1,           started_safe_wtd,                          -1 },
+        {                nwo_safe_wtd,                          -1,                          -1,                          -1,                          -1,                          -1,       started_nwo_safe_wtd,                          -1 },
+        {     closed_running_safe_wtd,                          -1,                          -1,                          -1,                          -1,                set_safe_wtd,                          1,             opened_safe_wtd },
+        {     closed_running_safe_wtd,                          -1,                          -1,                          -1,               safe_safe_wtd,                          -1,                          1,             stoped_safe_wtd },
+        { closed_running_nwo_safe_wtd,                          -1,                          -1,                          -1,           safe_nwo_safe_wtd,                          -1,                         -1,                          -1 },
+        {                          -1,                          -1,                          -1,                          -1,               safe_safe_wtd,                          -1,                         -1,                          -1 },
+        {                          -1,                          -1,                          -1,                          -1,           safe_nwo_safe_wtd,                          -1,                         -1,                          -1 },
Quoted 8 lines
Quoted 6 lines

+        {     closed_running_safe_wtd,                          -1,                          -1,                          -1,                          -1,                set_safe_wtd,                         -1,             stoped_safe_wtd }, +        { closed_running_nwo_safe_wtd,                          -1,                          -1,                          -1,                          -1,            set_nwo_safe_wtd,                         -1,                          -1 }, +        {               init_safe_wtd,                          -1,                          -1,                          -1,                          -1,                          -1,                         -1,                          -1 }, +    },
+    .initial_state = init_safe_wtd,
+    .final_states = { 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },

I find this event table all but impossible to verify.

It is a matrix. Lines are states, and columns are events.

On a given state/line, receiving a given event/column, the data is the next state/row.

For instance, let's say "init" (row 0), event "nwo" (column 1), and the next state is the "nwo" (row 3).

-1 means invalid/blocked state (yeah, maybe it is better to have an #define INVALID_STATE -1).

This is the C representation of an automaton, following the formal definition of a deterministic automaton. I've added an explanation of this representation in the documentation (patch 15, file da_monitor_synthesis.rst).

A deeper look into this subject is here (peer-reviewed conference paper at Software Engineer and Formal Methods 2019):
https://bristot.me/wp-content/uploads/2019/09/paper.pdf

One could translate it back to the automaton's graphical format... to a format of by a tool used to analyze automaton properties... that is the good point of using a well-established formalism. (The bad part is that they are often boring... c'est la vie :-)).

-- Daniel

Quoted 3 lines
Quoted 2 lines

+};

A message that is not in the archive (sent off-list or not yet received): CA+wEVJbvcMZbCroO2_rdVxLvYkUo-ePxCwsp5vbDpoqys4HGWQ@mail.gmail.com

maintainer
#64 lore raw

Re: [PATCH V4 18/20] rv/monitor: Add safe watchdog monitor

Quoted 280 lines · Gabriele Paoloni

On Thu, Jun 16, 2022 at 5:29 PM Daniel Bristot de Oliveira <[email protected] <mailto:[email protected]>> wrote:

Hi Guenter,

On 6/16/22 15:36, Guenter Roeck wrote:
> On 6/16/22 01:45, Daniel Bristot de Oliveira wrote:
>> The watchdog is an essential building block for the usage of Linux in
>> safety-critical systems because it allows the system to be monitored from >> an external element - the watchdog hardware, acting as a safety-monitor. >>
>> A user-space application controls the watchdog device via the watchdog
>> interface. This application, hereafter safety_app, enables the watchdog
>> and periodically pets the watchdog upon correct completion of the safety >> related processing.
>>
>> If the safety_app, for any reason, stops pinging the watchdog,
>> the watchdog hardware can set the system in a fail-safe state. For
>> example, shutting the system down.
>>
>> Given the importance of the safety_app / watchdog hardware couple,
>> the interaction between these software pieces also needs some
>> sort of monitoring. In other words, "who monitors the monitor?"
>>
>> The safe watchdog (safe_wtd) RV monitor monitors the interaction between >> the safety_app and the watchdog device, enforcing the correct sequence of >> events that leads the system to a safe state.
>>
>> Furthermore, the safety_app can monitor the RV monitor by collecting the >> events generated by the RV monitor itself via tracing interface. In this way, >> closing the monitoring loop with the safety_app.
>>
>> To reach a safe state, the safe_wtd RV monitor requires the
>> safety_app to:
>>
>>     - Open the watchdog device
>>     - Start the watchdog
>>     - Set a timeout
>>     - ping at least once
>>
>> The RV monitor also avoids some undesired actions. For example, to have
>> other threads to touch the watchdog.
>>
>> The monitor also has a set of options, enabled via kernel command
>> line/module options. They are:
>>
>>     - watchdog_id: the device id to monitor (default 0).
>>     - dont_stop: once enabled, do not allow the RV monitor to be stopped >>         (default off);
>>     - safe_timeout: define a maximum safe value that an user-space
>>         application can set as the watchdog timeout
>>         (default unlimited).
>>     - check_timeout: After every ping, check if the time left in the
>>         watchdog is less than or equal to the last timeout set
>>         for the watchdog. It only works for watchdog devices that
>>         provide the get_timeleft() function (default off).
>>
>> For further information, please refer to:
>>     Documentation/trace/rv/watchdog-monitor.rst
>>
>> The monitor specification was developed together with Gabriele Paoloni,
>> in the context of the Linux Foundation Elisa Project.
>>
>> Cc: Wim Van Sebroeck <[email protected] <mailto:[email protected]>> >> Cc: Guenter Roeck <[email protected] <mailto:[email protected]>>
>> Cc: Jonathan Corbet <[email protected] <mailto:[email protected]>>
>> Cc: Steven Rostedt <[email protected] <mailto:[email protected]>>
>> Cc: Ingo Molnar <[email protected] <mailto:[email protected]>>
>> Cc: Thomas Gleixner <[email protected] <mailto:[email protected]>>
>> Cc: Peter Zijlstra <[email protected] <mailto:[email protected]>>
>> Cc: Will Deacon <[email protected] <mailto:[email protected]>>
>> Cc: Catalin Marinas <[email protected] <mailto:[email protected]>> >> Cc: Marco Elver <[email protected] <mailto:[email protected]>>
>> Cc: Dmitry Vyukov <[email protected] <mailto:[email protected]>>
>> Cc: "Paul E. McKenney" <[email protected] <mailto:[email protected]>>
>> Cc: Shuah Khan <[email protected] <mailto:[email protected]>> >> Cc: Gabriele Paoloni <[email protected] <mailto:[email protected]>>
>> Cc: Juri Lelli <[email protected] <mailto:[email protected]>>
>> Cc: Clark Williams <[email protected] <mailto:[email protected]>>
>> Cc: [email protected] <mailto:[email protected]>
>> Cc: [email protected] <mailto:[email protected]>
>> Cc: [email protected] <mailto:[email protected]> >> Signed-off-by: Daniel Bristot de Oliveira <[email protected] <mailto:[email protected]>> >> ---
>>   include/trace/events/rv.h                    |  11 +
>>   kernel/trace/rv/Kconfig                      |  10 +
>>   kernel/trace/rv/Makefile                     |   1 +
>>   kernel/trace/rv/monitors/safe_wtd/safe_wtd.c | 300 +++++++++++++++++++ >>   kernel/trace/rv/monitors/safe_wtd/safe_wtd.h |  84 ++++++
>>   5 files changed, 406 insertions(+)
>>   create mode 100644 kernel/trace/rv/monitors/safe_wtd/safe_wtd.c
>>   create mode 100644 kernel/trace/rv/monitors/safe_wtd/safe_wtd.h
>>
>> diff --git a/include/trace/events/rv.h b/include/trace/events/rv.h
>> index 00f11a8dac3b..895eb3435ed7 100644
>> --- a/include/trace/events/rv.h
>> +++ b/include/trace/events/rv.h
>> @@ -66,6 +66,17 @@ DEFINE_EVENT(error_da_monitor, error_wip,
>>            TP_PROTO(char *state, char *event),
>>            TP_ARGS(state, event));
>>   #endif /* CONFIG_RV_MON_WIP */
>> +
>> +#ifdef CONFIG_RV_MON_SAFE_WTD
>> +DEFINE_EVENT(event_da_monitor, event_safe_wtd,
>> +         TP_PROTO(char *state, char *event, char *next_state, bool safe), >> +         TP_ARGS(state, event, next_state, safe));
>> +
>> +DEFINE_EVENT(error_da_monitor, error_safe_wtd,
>> +         TP_PROTO(char *state, char *event),
>> +         TP_ARGS(state, event));
>> +#endif /* CONFIG_RV_MON_SAFE_WTD */
>> +
>>   #endif /* CONFIG_DA_MON_EVENTS_IMPLICIT */
>>     #ifdef CONFIG_DA_MON_EVENTS_ID
>> diff --git a/kernel/trace/rv/Kconfig b/kernel/trace/rv/Kconfig
>> index 21f03fb3101a..b14ae63e792b 100644
>> --- a/kernel/trace/rv/Kconfig
>> +++ b/kernel/trace/rv/Kconfig
>> @@ -45,6 +45,16 @@ config RV_MON_WWNR
>>         illustrates the usage of per-task monitor. The model is
>>         broken on purpose: it serves to test reactors.
>>   +config RV_MON_SAFE_WTD
>> +    select DA_MON_EVENTS_IMPLICIT
>> +    bool "Safety watchdog"
>> +    help
>> +      Enable safe_wtd, this monitor observes the interaction
>> +      between a user-space safety monitor and a watchdog device.
>> +
>> +      For futher information see:
>> +        Documentation/trace/rv/safety-monitor.rst
>> +
>>   config RV_REACTORS
>>       bool "Runtime verification reactors"
>>       default y if RV
>> diff --git a/kernel/trace/rv/Makefile b/kernel/trace/rv/Makefile
>> index 963d14875b45..904db96c7eae 100644
>> --- a/kernel/trace/rv/Makefile
>> +++ b/kernel/trace/rv/Makefile
>> @@ -3,6 +3,7 @@
>>   obj-$(CONFIG_RV) += rv.o
>>   obj-$(CONFIG_RV_MON_WIP) += monitors/wip/wip.o
>>   obj-$(CONFIG_RV_MON_WWNR) += monitors/wwnr/wwnr.o
>> +obj-$(CONFIG_RV_MON_SAFE_WTD) += monitors/safe_wtd/safe_wtd.o
>>   obj-$(CONFIG_RV_REACTORS) += rv_reactors.o
>>   obj-$(CONFIG_RV_REACT_PRINTK) += reactor_printk.o
>>   obj-$(CONFIG_RV_REACT_PANIC) += reactor_panic.o
>> diff --git a/kernel/trace/rv/monitors/safe_wtd/safe_wtd.c
>> b/kernel/trace/rv/monitors/safe_wtd/safe_wtd.c
>> new file mode 100644
>> index 000000000000..9856e0770d0d
>> --- /dev/null
>> +++ b/kernel/trace/rv/monitors/safe_wtd/safe_wtd.c
>> @@ -0,0 +1,300 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +#include <linux/ftrace.h>
>> +#include <linux/tracepoint.h>
>> +#include <linux/kernel.h>
>> +#include <linux/module.h>
>> +#include <linux/init.h>
>> +#include <linux/rv.h>
>> +#include <rv/instrumentation.h>
>> +#include <rv/da_monitor.h>
>> +
>> +#include <linux/watchdog.h>
>> +#include <linux/moduleparam.h>
>> +
>> +#include <trace/events/rv.h>
>> +#include <trace/events/watchdog.h>
>> +
>> +#define MODULE_NAME "safe_wtd"
>> +
>> +/*
>> + * This is the self-generated part of the monitor. Generally, there is no need >> + * to touch this section.
>> + */
>> +#include "safe_wtd.h"
>> +
>> +/*
>> + * Declare the deterministic automata monitor.
>> + *
>> + * The rv monitor reference is needed for the monitor declaration.
>> + */
>> +struct rv_monitor rv_safe_wtd;
>> +DECLARE_DA_MON_GLOBAL(safe_wtd, char);
>> +
>> +/*
>> + * custom: safe_timeout is the maximum value a watchdog monitor
>> + * can set. This value is registered here to duplicate the information. >> + * In this way, a miss-behaving monitor can be detected.
>> + */
>> +static int safe_timeout = ~0;
>> +module_param(safe_timeout, int, 0444);
>> +
>> +/*
>> + * custom: if check_timeout is set, the monitor will check if the time left >> + * in the watchdog is less than or equals to the last safe timeout set by >> + * user-space. This check is done after each ping. In this way, if any
>> + * code by-passed the watchdog dev interface setting a higher (so unsafe) >> + * timeout, this monitor will catch the side effect and react.
>> + */
>> +static int last_timeout_set = 0;
>> +static int check_timeout = 0;
>> +module_param(check_timeout, int, 0444);
>> +
>> +/*
>> + * custom: if dont_stop is set the monitor will react if stopped.
>> + */
>> +static int dont_stop = 0;
>> +module_param(dont_stop, int, 0444);
>> +
>> +/*
>> + * custom: there are some states that are kept after the watchdog is closed. >> + * For example, the nowayout state.
>> + *
>> + * Thus, the RV monitor needs to keep track of these states after a start/stop >> + * of the RV monitor itself, and should not reset after each restart -
>> keeping the
>> + * know state until the system shutdown.
>> + *
>> + * If for an unknown reason an RV monitor would like to reset the RV monitor >> at each
>> + * RV monitor start, set it to one.
>> + */
>> +static int reset_on_restart = 0;
>> +module_param(reset_on_restart, int, 0444);
>> +
>> +/*
>> + * open_pid takes note of the first thread that opened the watchdog.
>> + *
>> + * Any other thread that generates an event will cause an "other_threads" >> + * event in the monitor.
>> + */
>> +static int open_pid = 0;
>
> Userspace could open a watchdog, create a child process, and handle it
> from the child. That is perfectly valid.

Right! It is a correct usage of the watchdog subsystem.

However, the idea here is to allow a "restricted" set of operations based on the safety analysis made by people in the LF Elisa Workgroup (Gabriele Paoloni in Cc:).

Yes this is correct. This model represents the instance of a specific monitor resulting from a specific use case that was analysed in ELISA. From my understanding nothing prevents extending the monitor to a more complex model in the future that may eventually result in a group of allowed processes for the WTD manipulation...

Kind Regards
Gab

One of the specifications says that: only one process should touch the watchdog.

There are details about it in the "watchdog-monitor.rst," section "RV monitor specification."

There could be another monitor, a less resticted one, in which the operation you mention would be allowed.

I will complement this commit log in the next version of the patch set, clarifying that it is not a "full representation of the watchdog operations" but a restricted set of operations specified by...

>> +
>> +/*
>> + * watchdog_id: the watchdog to monitor
>> + */
>> +static int watchdog_id = 0;
>> +module_param(watchdog_id, int, 0444);
>
> Limiting the watcher to a single watchdog sounds less than perfect. > What if the system supports more than one, more than one is enabled, > and the non-monitored watchdog misbehaves ?

I can add one monitor per watchdog dev. The easiest way would be adding a "struct da_monitor" variable in the watchdog_device structure, e.g.,

Quoted 8 lines

struct watchdog_device {
...
        #ifdef CONFIG_RV_MON_SAFE_WTD
        struct da_monitor da_mon;
        #endif
...
}

In my opinion shis should be dynamically allocated and not waste space in driver code if unused.

Quoted 90 lines
    A simplified version of the the "per task" monitor, in the patch 01, changes in
    include/linux/sched.h.

     >> +
     >> +static void handle_nowayout(void *data, struct watchdog_device *wdd)
     >> +{
     >> +    if (wdd->id != watchdog_id)
     >> +        return;
     >> +
     >> +    da_handle_init_run_event_safe_wtd(nowayout_safe_wtd);
     >> +}
     >> +
     >> +static void handle_close(void *data, struct watchdog_device *wdd)
     >> +{
     >> +    if (wdd->id != watchdog_id)
     >> +        return;
     >> +
     >> +    if (open_pid && current->pid != open_pid) {
     >> +        da_handle_init_run_event_safe_wtd(other_threads_safe_wtd);
     >> +    } else {
     >> +        da_handle_event_safe_wtd(close_safe_wtd);
     >> +        open_pid = 0;
     >> +    }
     >> +}
     >> +
     >> +static void handle_open(void *data, struct watchdog_device *wdd)
     >> +{
     >> +    if (wdd->id != watchdog_id)
     >> +        return;
     >> +
     >> +    if (open_pid && current->pid != open_pid) {
     >> +        da_handle_init_run_event_safe_wtd(other_threads_safe_wtd);
     >> +    } else {
     >> +        da_handle_init_run_event_safe_wtd(open_safe_wtd);
     >> +        open_pid = current->pid;
     >> +    }
     >> +}
     >> +
     >> +static void blocked_events(void *data, struct watchdog_device *wdd)
     >> +{
     >> +    if (wdd->id != watchdog_id)
     >> +        return;
     >> +
     >> +    if (open_pid && current->pid != open_pid) {
     >> +        da_handle_init_run_event_safe_wtd(other_threads_safe_wtd);
     >> +        return;
     >> +    }
     >> +    da_handle_event_safe_wtd(other_threads_safe_wtd);
     >> +}
     >> +
     >> +static void blocked_events_timeout(void *data, struct watchdog_device *wdd,
     >> u64 timeout)
     >> +{
     >> +    blocked_events(data, wdd);
     >> +}
     >> +
     >> +static void handle_ping(void *data, struct watchdog_device *wdd)
     >> +{
     >> +    char msg[128];
     >> +    unsigned int timeout;
     >> +
     >> +    if (wdd->id != watchdog_id)
     >> +        return;
     >> +
     >> +    if (open_pid && current->pid != open_pid) {
     >> +        da_handle_init_run_event_safe_wtd(other_threads_safe_wtd);
     >> +        return;
     >> +    }
     >> +
     >> +    da_handle_event_safe_wtd(ping_safe_wtd);
     >> +
     >> +    if (!check_timeout)
     >> +        return;
     >> +
     >> +    if (wdd->ops->get_timeleft) {
     >> +        timeout = wdd->ops->get_timeleft(wdd);
     >> +        if (timeout > last_timeout_set) {
     >> +            snprintf(msg, 128,
     >> +                 "watchdog timeout is %u > than previously set (%d)\n",
     >> +                 timeout, last_timeout_set);
     >> +            cond_react(msg);
     >> +        }
     >> +    } else {
     >> +        snprintf(msg, 128, "error getting timeout: option not supported\n");
     >
     > This is not an error. The get_timeleft callback is optional.

    Right... but this part of the code is only reachable if the user explicitly
    asked to check the timeout (if (!check_timeout)...return before this code).
Quoted 9 lines
    So, if the user only considers the system safe if the monitor also checks the
    written timeout, but the watchdog is one of those that do not have the callback
    implemented (which is ok for a Linux watchdog), the monitor captures this
    "undesired" behavior.

    This monitor is not checking if the watchdog subsystem is correct at its
    plenitude, it is checking if the watchdog usage is following a set of
    specifications (raised by people in the LF Elisa workgroup).

The kernel is not intended for special use cases. The callback is optional, period. The test for check_timeout is way too late. A check like this should be made when the check is requested, not when it is executed - in other words, when the user requests it. That request should fail.

Quoted 56 lines

>> +        cond_react(msg);
>> +    }
>> +}
>> +

[...]

>> +
>> +struct automaton_safe_wtd automaton_safe_wtd = {
>> +    .state_names = {
>> +        "init",
>> +        "closed_running",
>> +        "closed_running_nwo",
>> +        "nwo",
>> +        "opened",
>> +        "opened_nwo",
>> +        "reopened",
>> +        "safe",
>> +        "safe_nwo",
>> +        "set",
>> +        "set_nwo",
>> +        "started",
>> +        "started_nwo",
>> +        "stoped"
>> +    },
>> +    .event_names = {
>> +        "close",
>> +        "nowayout",
>> +        "open",
>> +        "other_threads",
>> +        "ping",
>> +        "set_safe_timeout",
>> +        "start",
>> +        "stop"
>> +    },
>> +    .function = {
>> +        {                          -1,                nwo_safe_wtd,             opened_safe_wtd,               init_safe_wtd,                          -1,                          -1,                         -1,                          -1 }, >> +        {                          -1, closed_running_nwo_safe_wtd,           reopened_safe_wtd,     closed_running_safe_wtd,                          -1,                          -1,                         -1,                          -1 }, >> +        {                          -1, closed_running_nwo_safe_wtd,        started_nwo_safe_wtd, closed_running_nwo_safe_wtd,                          -1,                          -1,                         -1,                          -1 }, >> +        {                          -1,                nwo_safe_wtd,         opened_nwo_safe_wtd,                nwo_safe_wtd,                          -1,                          -1,                         -1,                          -1 }, >> +        {               init_safe_wtd,                          -1,                          -1,                          -1,                          -1,                          -1,           started_safe_wtd,                          -1 }, >> +        {                nwo_safe_wtd,                          -1,                          -1,                          -1,                          -1,                          -1,       started_nwo_safe_wtd,                          -1 }, >> +        {     closed_running_safe_wtd,                          -1,                          -1,                          -1,                          -1,                set_safe_wtd,                          1,             opened_safe_wtd }, >> +        {     closed_running_safe_wtd,                          -1,                          -1,                          -1,               safe_safe_wtd,                          -1,                          1,             stoped_safe_wtd }, >> +        { closed_running_nwo_safe_wtd,                          -1,                          -1,                          -1,           safe_nwo_safe_wtd,                          -1,                         -1,                          -1 }, >> +        {                          -1,                          -1,                          -1,                          -1,               safe_safe_wtd,                          -1,                         -1,                          -1 }, >> +        {                          -1,                          -1,                          -1,                          -1,           safe_nwo_safe_wtd,                          -1,                         -1,                          -1 }, >> +        {     closed_running_safe_wtd,                          -1,                          -1,                          -1,                          -1,                set_safe_wtd,                         -1,             stoped_safe_wtd }, >> +        { closed_running_nwo_safe_wtd,                          -1,                          -1,                          -1,                          -1,            set_nwo_safe_wtd,                         -1,                          -1 }, >> +        {               init_safe_wtd,                          -1,                          -1,                          -1,                          -1,                          -1,                         -1,                          -1 }, >> +    },
>> +    .initial_state = init_safe_wtd,
>> +    .final_states = { 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
>
> I find this event table all but impossible to verify.

Quoted 5 lines

It is a matrix. Lines are states, and columns are events.

On a given state/line, receiving a given event/column, the data is the next state/row.

I am aware of that, and I did program state machines before.

Quoted 10 lines

For instance, let's say "init" (row 0), event "nwo" (column 1), and the next state is the "nwo" (row 3).

-1 means invalid/blocked state (yeah, maybe it is better to have an #define INVALID_STATE -1).

This is the C representation of an automaton, following the formal definition of a deterministic automaton. I've added an explanation of this representation in the documentation (patch 15, file da_monitor_synthesis.rst).

Quoted 9 lines

A deeper look into this subject is here (peer-reviewed conference paper at
Software Engineer and Formal Methods 2019):
https://bristot.me/wp-content/uploads/2019/09/paper.pdf <https://bristot.me/wp-content/uploads/2019/09/paper.pdf>

One could translate it back to the automaton's graphical format... to a format of by a tool used to analyze automaton properties... that is the good point of using a well-established formalism. (The bad part is that they are often boring... c'est la vie :-)).

If the above state machine fails, no one but the authors will be able to even remotely figure out what happened, and if the watchdog driver is at fault or its monitor. It is a state machine making assumptions about state transitions, sure, but who knows if those asssumptions are even remotely correct or match reality. For example, I have no idea if the lack of a 'ping' function is handled correctly, if the lack of a 'stop' function is handled correctly, or what happens if any of the driver functions returns an error.

I already found three assumptions which do not or not necessarily match reality:

- The function to read the remaining timeout is optional and must not be used unconditionally, and its lack is not an error.
- The requested timeout (and pretimeout) do not have to match the actually configured timeout, and userspace must not rely on the assumption that the values match.
- The code assumes that the process opening the watchdog and the process accessing it are the same. While that is in general the case, it might well be that some application opens the watchdog and then handles it from a child process.

And that is just after briefly browsing through the code.

I am open to suggestions from others, but at this point I have serious doubts that this code is maintainable in the kernel.

Guenter

#65 lore raw

Re: [PATCH V4 18/20] rv/monitor: Add safe watchdog monitor

Quoted 10 lines · Guenter Roeck
Quoted 9 lines

    struct watchdog_device {
    ...
             #ifdef CONFIG_RV_MON_SAFE_WTD
             struct da_monitor da_mon;
             #endif
    ...
    }

Quoted 2 lines

In my opinion shis should be dynamically allocated and not waste space in driver code if unused.

ack

Quoted 111 lines
Quoted 96 lines

    A simplified version of the the "per task" monitor, in the patch 01, changes in

    include/linux/sched.h.

     >> +
     >> +static void handle_nowayout(void *data, struct watchdog_device *wdd) >> +{
     >> +    if (wdd->id != watchdog_id)
     >> +        return;
     >> +
     >> +    da_handle_init_run_event_safe_wtd(nowayout_safe_wtd);
     >> +}
     >> +
     >> +static void handle_close(void *data, struct watchdog_device *wdd) >> +{
     >> +    if (wdd->id != watchdog_id)
     >> +        return;
     >> +
     >> +    if (open_pid && current->pid != open_pid) {
     >> +        da_handle_init_run_event_safe_wtd(other_threads_safe_wtd); >> +    } else {
     >> +        da_handle_event_safe_wtd(close_safe_wtd);
     >> +        open_pid = 0;
     >> +    }
     >> +}
     >> +
     >> +static void handle_open(void *data, struct watchdog_device *wdd) >> +{
     >> +    if (wdd->id != watchdog_id)
     >> +        return;
     >> +
     >> +    if (open_pid && current->pid != open_pid) {
     >> +        da_handle_init_run_event_safe_wtd(other_threads_safe_wtd); >> +    } else {
     >> +        da_handle_init_run_event_safe_wtd(open_safe_wtd);
     >> +        open_pid = current->pid;
     >> +    }
     >> +}
     >> +
     >> +static void blocked_events(void *data, struct watchdog_device *wdd) >> +{
     >> +    if (wdd->id != watchdog_id)
     >> +        return;
     >> +
     >> +    if (open_pid && current->pid != open_pid) {
     >> +        da_handle_init_run_event_safe_wtd(other_threads_safe_wtd); >> +        return;
     >> +    }
     >> +    da_handle_event_safe_wtd(other_threads_safe_wtd);
     >> +}
     >> +
     >> +static void blocked_events_timeout(void *data, struct watchdog_device *wdd,

     >> u64 timeout)
     >> +{
     >> +    blocked_events(data, wdd);
     >> +}
     >> +
     >> +static void handle_ping(void *data, struct watchdog_device *wdd) >> +{
     >> +    char msg[128];
     >> +    unsigned int timeout;
     >> +
     >> +    if (wdd->id != watchdog_id)
     >> +        return;
     >> +
     >> +    if (open_pid && current->pid != open_pid) {
     >> +        da_handle_init_run_event_safe_wtd(other_threads_safe_wtd); >> +        return;
     >> +    }
     >> +
     >> +    da_handle_event_safe_wtd(ping_safe_wtd);
     >> +
     >> +    if (!check_timeout)
     >> +        return;
     >> +
     >> +    if (wdd->ops->get_timeleft) {
     >> +        timeout = wdd->ops->get_timeleft(wdd);
     >> +        if (timeout > last_timeout_set) {
     >> +            snprintf(msg, 128,
     >> +                 "watchdog timeout is %u > than previously set (%d)\n", >> +                 timeout, last_timeout_set);
     >> +            cond_react(msg);
     >> +        }
     >> +    } else {
     >> +        snprintf(msg, 128, "error getting timeout: option not supported\n");

     >
     > This is not an error. The get_timeleft callback is optional.

    Right... but this part of the code is only reachable if the user explicitly asked to check the timeout (if (!check_timeout)...return before this code).

    So, if the user only considers the system safe if the monitor also checks the written timeout, but the watchdog is one of those that do not have the callback

Quoted 7 lines

    implemented (which is ok for a Linux watchdog), the monitor captures this "undesired" behavior.

    This monitor is not checking if the watchdog subsystem is correct at its plenitude, it is checking if the watchdog usage is following a set of specifications (raised by people in the LF Elisa workgroup).

The kernel is not intended for special use cases. The callback is optional, period. The test for check_timeout is way too late. A check like this should be made when the check is requested, not when it is executed - in other words, when the user requests it. That request should fail.

Quoted 2 lines

     >> +        cond_react(msg);
     >> +    }

[...]

Quoted 32 lines
Quoted 8 lines

     >
     > I find this event table all but impossible to verify.

    It is a matrix. Lines are states, and columns are events.

    On a given state/line, receiving a given event/column, the data is the next state/row.

I am aware of that, and I did program state machines before.

Quoted 21 lines

    For instance, let's say "init" (row 0), event "nwo" (column 1), and the next state is the "nwo" (row 3).

    -1 means invalid/blocked state (yeah, maybe it is better to have an #define INVALID_STATE -1).

    This is the C representation of an automaton, following the formal definition of

    a deterministic automaton. I've added an explanation of this representation in

    the documentation (patch 15, file da_monitor_synthesis.rst).

    A deeper look into this subject is here (peer-reviewed conference paper at Software Engineer and Formal Methods 2019):
    https://bristot.me/wp-content/uploads/2019/09/paper.pdf <https://bristot.me/wp-content/uploads/2019/09/paper.pdf>

    One could translate it back to the automaton's graphical format... to a format

    of by a tool used to analyze automaton properties... that is the good point of

Quoted 7 lines
Quoted 3 lines

    using a well-established formalism. (The bad part is that they are often boring... c'est la vie :-)).

If the above state machine fails, no one but the authors will be able to even remotely figure out what happened, and if the watchdog driver is at fault or its monitor.

That is a point we can improve. If we look only for the matrix, yes, it is not the best way to try to understand what is going on. I agree.

The idea here is that one can describe the expected behavior and check. But we fail to explain the reasons for these choices...

The patch "20/20 Documentation/rv: Add watchdog-monitor documentation" attempts to explain the idea behind the monitor, and includes the ASCII representation of the monitor. But indeed, it fails on explaining why those assumptions were made, what it is trying to monitor, and what it is not...

It is a state machine making assumptions about state transitions,

Quoted 10 lines

sure, but who knows if those asssumptions are even remotely correct or match reality. For example, I have no idea if the lack of a 'ping' function is handled correctly,  if the lack of a 'stop' function is handled correctly, or what happens if any of the driver functions returns an error.

I already found three assumptions which do not or not necessarily match reality:

- The function to read the remaining timeout is optional and must not be used unconditionally, and its lack is not an error.

Ack, that can be removed.

Quoted 3 lines

- The requested timeout (and pretimeout) do not have to match the actually configured timeout, and userspace must not rely on the assumption that the values match.

I agree, the model is lacking that, and it needs it.

Quoted 4 lines

- The code assumes that the process opening the watchdog and the process accessing it are the same. While that is in general the case, it might well be that some application opens the watchdog and then handles it from a child process.

What is not clear on our documentation are the reasons behind these choices. They are based on a hazard analysis made by people safety experts in the Elisa group, that need to be clarified.

/me looks at people from Elisa...

-- Daniel

Quoted 4 lines

And that is just after briefly browsing through the code> I am open to suggestions from others, but at this point I have serious doubts that this code is maintainable in the kernel.

Guenter

maintainer
#66 lore raw

Re: [PATCH V4 18/20] rv/monitor: Add safe watchdog monitor

In reply to Guenter Roeck
Quoted 21 lines · On Thu, 16 Jun 2022 16:53:54 -0700 Guenter Roeck <[email protected]> wrote:
Quoted 21 lines

>> +
>> +struct automaton_safe_wtd automaton_safe_wtd = { >> +    .state_names = {
>> +        "init",
>> +        "closed_running",
>> +        "closed_running_nwo",
>> +        "nwo",
>> +        "opened",
>> +        "opened_nwo",
>> +        "reopened",
>> +        "safe",
>> +        "safe_nwo",
>> +        "set",
>> +        "set_nwo",
>> +        "started",
>> +        "started_nwo",
>> +        "stoped"
>> +    },
>> +    .event_names = {
>> +        "close",
>> +        "nowayout",

Quoted 8 lines
Quoted 8 lines

>> +        "open",
>> +        "other_threads",
>> +        "ping",
>> +        "set_safe_timeout",
>> +        "start",
>> +        "stop"
>> +    },
>> +    .function = {

I think it could become much more readable if you added comments here that show what each column is. That way, we do not need to try to remember it.

And maybe even a diagram in the comments. If we can automate the generating of an ASCII DFA diagram then that would be awesome too :-)

Quoted 69 lines
Quoted 20 lines

>> +        {                          -1,                nwo_safe_wtd,             opened_safe_wtd,               init_safe_wtd,                          -1,                          -1,                         -1,                          -1 }, >> +        {                          -1, closed_running_nwo_safe_wtd,           reopened_safe_wtd,     closed_running_safe_wtd,                          -1,                          -1,                         -1,                          -1 }, >> +        {                          -1, closed_running_nwo_safe_wtd,        started_nwo_safe_wtd, closed_running_nwo_safe_wtd,                          -1,                          -1,                         -1,                          -1 }, >> +        {                          -1,                nwo_safe_wtd,         opened_nwo_safe_wtd,                nwo_safe_wtd,                          -1,                          -1,                         -1,                          -1 }, >> +        {               init_safe_wtd,                          -1,                          -1,                          -1,                          -1,                          -1,           started_safe_wtd,                          -1 }, >> +        {                nwo_safe_wtd,                          -1,                          -1,                          -1,                          -1,                          -1,       started_nwo_safe_wtd,                          -1 }, >> +        {     closed_running_safe_wtd,                          -1,                          -1,                          -1,                          -1,                set_safe_wtd,                          1,             opened_safe_wtd }, >> +        {     closed_running_safe_wtd,                          -1,                          -1,                          -1,               safe_safe_wtd,                          -1,                          1,             stoped_safe_wtd }, >> +        { closed_running_nwo_safe_wtd,                          -1,                          -1,                          -1,           safe_nwo_safe_wtd,                          -1,                         -1,                          -1 }, >> +        {                          -1,                          -1,                          -1,                          -1,               safe_safe_wtd,                          -1,                         -1,                          -1 }, >> +        {                          -1,                          -1,                          -1,                          -1,           safe_nwo_safe_wtd,                          -1,                         -1,                          -1 }, >> +        {     closed_running_safe_wtd,                          -1,                          -1,                          -1,                          -1,                set_safe_wtd,                         -1,             stoped_safe_wtd }, >> +        { closed_running_nwo_safe_wtd,                          -1,                          -1,                          -1,                          -1,            set_nwo_safe_wtd,                         -1,                          -1 }, >> +        {               init_safe_wtd,                          -1,                          -1,                          -1,                          -1,                          -1,                         -1,                          -1 }, >> +    },
>> +    .initial_state = init_safe_wtd,
>> +    .final_states = { 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
>
> I find this event table all but impossible to verify.

Quoted 5 lines

It is a matrix. Lines are states, and columns are events.

On a given state/line, receiving a given event/column, the data is the next state/row.

I am aware of that, and I did program state machines before.

Quoted 10 lines

For instance, let's say "init" (row 0), event "nwo" (column 1), and the next state is the "nwo" (row 3).

-1 means invalid/blocked state (yeah, maybe it is better to have an #define INVALID_STATE -1).

This is the C representation of an automaton, following the formal definition of a deterministic automaton. I've added an explanation of this representation in the documentation (patch 15, file da_monitor_synthesis.rst).

Quoted 9 lines

A deeper look into this subject is here (peer-reviewed conference paper at
Software Engineer and Formal Methods 2019):
https://bristot.me/wp-content/uploads/2019/09/paper.pdf <https://bristot.me/wp-content/uploads/2019/09/paper.pdf>

One could translate it back to the automaton's graphical format... to a format of by a tool used to analyze automaton properties... that is the good point of using a well-established formalism. (The bad part is that they are often boring... c'est la vie :-)).

If the above state machine fails, no one but the authors will be able to even remotely figure out what happened, and if the watchdog driver is at fault or its monitor. It is a state machine making assumptions about state transitions, sure, but who knows if those asssumptions are even remotely correct or match reality. For example, I have no idea if the lack of a 'ping' function is handled correctly, if the lack of a 'stop' function is handled correctly, or what happens if any of the driver functions returns an error.

I already found three assumptions which do not or not necessarily match reality:

- The function to read the remaining timeout is optional and must not be used unconditionally, and its lack is not an error.
- The requested timeout (and pretimeout) do not have to match the actually configured timeout, and userspace must not rely on the assumption that the values match.
- The code assumes that the process opening the watchdog and the process accessing it are the same. While that is in general the case, it might well be that some application opens the watchdog and then handles it from a child process.

Quoted 5 lines

And that is just after briefly browsing through the code.

I am open to suggestions from others, but at this point I have serious doubts that this code is maintainable in the kernel.

Let me give some background on this. Various safety critical users (automotive, power plants, etc) are looking at using Linux in the field. Obviously, Linux itself is too big to verify properly. In fact, even small OSs that say they are verifiable seldom are. Part of the solution is to add a way to check that Linux is running as expected, and be able to detect when it is not. When that happens, you fall into a "fail-safe" mode (for example, if you have an autonomous driving vehicle, it could sound an alarm and inform the driver to go into manual mode, or it is going to simply "pull over". Note, this is just a hypothetical example, not something I've seen in reality).

But to implement this, you need to verify the verifier. In this case, the watchdog timer is what verifies that Linux is running properly. It is possible to verify parts of the kernel when you limit the inputs of the kernel. This means we are not trying to verify all use cases of the watchdog timer. That would be pretty much impossible, or take decades to complete. We only need to verify the use cases used in safety critical systems.

Do not confuse this with static analyzers or other general purpose tooling to find bugs. This is not the purpose of the run time verify. It is just to prove that certain use cases will perform as expected, given a limited input.

Years ago there was an interview with Linus where he was asked if he would trust Linux in real-time safety critical projects. And Linus's response was something to the effect that he would, as those projects have a very limited use case, and the code that actually gets run is not as big as you would think. And that code can be verified.

That's exactly what we are doing here. There are very strict standards on what is considered safe for safety critical projects and by limiting the use cases of the code we are verifying, we can achieve those objectives.

But your complaint about not being able to read or understand the table is legitimate, and that needs to be addressed. And what the limiting input is must also be commented as well, that way people will not get confused when they see a legitimate use case not covered by the DFA.

Action Items:
1. Add column title to the matrix.
2. Ideally, also add an ASCII DFA diagram.
3. And finally, comment what the range of inputs are (what use cases are being covered).

-- Steve

maintainer
#67 lore raw

Re: [PATCH V4 18/20] rv/monitor: Add safe watchdog monitor

On 6/28/22 12:32, Steven Rostedt wrote:
[ ... ]

Quoted 15 lines
Quoted 3 lines

I am open to suggestions from others, but at this point I have serious doubts that this code is maintainable in the kernel.

Let me give some background on this. Various safety critical users (automotive, power plants, etc) are looking at using Linux in the field. Obviously, Linux itself is too big to verify properly. In fact, even small OSs that say they are verifiable seldom are. Part of the solution is to add a way to check that Linux is running as expected, and be able to detect when it is not. When that happens, you fall into a "fail-safe" mode (for example, if you have an autonomous driving vehicle, it could sound an alarm and inform the driver to go into manual mode, or it is going to simply "pull over". Note, this is just a hypothetical example, not something I've seen in reality).

Quoted 8 lines

But to implement this, you need to verify the verifier. In this case, the watchdog timer is what verifies that Linux is running properly. It is possible to verify parts of the kernel when you limit the inputs of the kernel. This means we are not trying to verify all use cases of the watchdog timer. That would be pretty much impossible, or take decades to complete. We only need to verify the use cases used in safety critical systems.

I understand the context. However, ...

Quoted 5 lines

Do not confuse this with static analyzers or other general purpose tooling to find bugs. This is not the purpose of the run time verify. It is just to prove that certain use cases will perform as expected, given a limited input.

... this is, unfortunately, not explained in the patch. I would have much less of a problem with the series if those details were included.

Not that I would mind such a verifier, if it was possible to define one, but it would have to be tested with a large number of watchdog drivers to ensure that it addresses all use cases, or at least with a substantial percentage of use cases. It would also require that the state machine is readable to give people a chance to fix it if turns out to be broken. It would also have to be robust, meaning it would have to reject invalid (unsupported) settings from the start and not only during runtime.

Thanks,
Guenter

maintainer
#68 lore raw

Re: [PATCH V4 18/20] rv/monitor: Add safe watchdog monitor

Quoted 8 lines · On Fri, 1 Jul 2022 07:45:50 -0700 Guenter Roeck <[email protected]> wrote:
Quoted 5 lines

Do not confuse this with static analyzers or other general purpose tooling to find bugs. This is not the purpose of the run time verify. It is just to prove that certain use cases will perform as expected, given a limited input.

... this is, unfortunately, not explained in the patch. I would have much less of a problem with the series if those details were included.

It's one of those cases where developers get so involved in their code that they leave out the things that are so obvious to them, but not obvious to others ;-)

My new saying is: "We work in a field where the obvious seldom is".

Hmm, I think I'll go tweet that :-)

Quoted 8 lines

Not that I would mind such a verifier, if it was possible to define one, but it would have to be tested with a large number of watchdog drivers to ensure that it addresses all use cases, or at least with a substantial percentage of use cases. It would also require that the state machine is readable to give people a chance to fix it if turns out to be broken. It would also have to be robust, meaning it would have to reject invalid (unsupported) settings from the start and not only during runtime.

I would agree than any module would need to state up front exactly what it is modeling. In safety critical systems, all the components that are used are defined up front. Not sure if we can have the model not load if the required drivers to test are not loaded or ones not part of the model are (Daniel?).

Anyway, thanks for the feedback.

-- Steve

#69 lore raw

Re: [PATCH V4 18/20] rv/monitor: Add safe watchdog monitor

Quoted 9 lines · Steven Rostedt
Quoted 6 lines · On Fri, 1 Jul 2022 07:45:50 -0700 Guenter Roeck <[email protected]> wrote:
Quoted 5 lines

Do not confuse this with static analyzers or other general purpose tooling to find bugs. This is not the purpose of the run time verify. It is just to prove that certain use cases will perform as expected, given a limited input.

Quoted 6 lines
Quoted 2 lines

... this is, unfortunately, not explained in the patch. I would have much less of a problem with the series if those details were included.

It's one of those cases where developers get so involved in their code that they leave out the things that are so obvious to them, but not obvious to others ;-)

I agree that the main point here is that the documentation [patch 20/20] needs to be extended and correctly linked to the code.

The goal of the model is to specify the minimum but obligatory steps to set a watchdog (start, set a safe timeout, ping...) so it can be used by "safety/heath" monitors in safety-critical systems.

Another goal is to reduce the amount of code/dependencies that will require deeper inspections to qualify the subsystem for usage in a given context via monitoring (as steven mentioned - more about it here [1]), without having to reduce the generic subsystem.

Although the method allows one to create a complete model of the watchdog device layer, covering all use cases, that is not the idea of this monitor. Moreover, a full model would not be the adequate model for this specific (but relevant) case raised and discussed in the Elisa workgroup [2].

The goal of the monitor (that uses the model) is to verify that the interaction between the watchdog device layer and the "safety/heath" monitors follows this established model, at runtime.

Quoted 3 lines

My new saying is: "We work in a field where the obvious seldom is".

Hmm, I think I'll go tweet that :-)

/me liked the tweet... and yep, you clarified well the context in which this is being applied.

Quoted 6 lines
Quoted 6 lines

Not that I would mind such a verifier, if it was possible to define one, but it would have to be tested with a large number of watchdog drivers to ensure that it addresses all use cases, or at least with a substantial percentage of use cases. It would also require that the state machine is readable to give people a chance to fix it if turns out to be broken.

The patch 20/20 has the automaton in the ASCII art format. Both C and ASCII models were extracted from the same .dot file. I am not including the .dot file because there was a previous discussion with the doc people that prefer the ASCII format in the documentation. But I can add as well, not linked with the rst file. By reading the model in the ASCII format, it is possible to see that it is broad enough to cover many watchdogs as it uses simple/generic operations.

The model (the .h) and the instrumentation (the .c) can be updated at any time. I am adding the tooling to facilitate that, like [patch 06/20].

Patch 19/20 adds a safety application that enables the RV monitor, uses the watchdog, and collects feedback from the monitor to see if the requested actions are occurring in the model - and it can be used to test the RV monitor with any watchdog.

The goal is the one I described above, so an exception generated by this monitor needs to be read accordingly: it does not imply that a watchdog driver is broken. It means the interaction between the safety/health monitor and the watchdog is not following specifications and must be checked at the development phase, or that something went really wrong at runtime phase.

I will add that to the documentation and emphasize the context of this monitor.

Quoted 8 lines
Quoted 2 lines

It would also have to be robust, meaning it would have to reject invalid (unsupported) settings from the start and not only during runtime.

I would agree than any module would need to state up front exactly what it is modeling. In safety critical systems, all the components that are used are defined up front. Not sure if we can have the model not load if the required drivers to test are not loaded or ones not part of the model are (Daniel?).

I can add a check in the time in which the model is being enabled. But for this specific case, removing that part might be better and adding it later if it becomes fundamental.

In practice, selecting a watchdog device can assume that the get_timeleft implementation is a requirement - so that option was just an additional check.

[1] This is one use case for the broader goal of the strategy discussed here:
https://www.youtube.com/watch?v=DkiwkAKOXNs

[2] https://www.youtube.com/watch?v=qFSYlCbHCYk

Quoted 3 lines

Anyway, thanks for the feedback.

-- Steve

reviewer
#70 lore raw

Re: [PATCH V4 18/20] rv/monitor: Add safe watchdog monitor

In reply to Daniel Bristot de Oliveira

Hi--

Quoted 7 lines · Daniel Bristot de Oliveira
kernel/trace/rv/Kconfig +0 -0
index 21f03fb3101a..b14ae63e792b 100644
@@ -45,6 +45,16 @@ config RV_MON_WWNR
 	  illustrates the usage of per-task monitor. The model is
 	  broken on purpose: it serves to test reactors.

kernel/trace/rv/Kconfig config RV_MON_WWNR · line 56

Quoted 10 lines
kernel/trace/rv/Kconfig +9 -0
 
+config RV_MON_SAFE_WTD
+	select DA_MON_EVENTS_IMPLICIT
+	bool "Safety watchdog"
+	help
+	  Enable safe_wtd, this monitor observes the interaction
+	  between a user-space safety monitor and a watchdog device.
+
+	  For futher information see:
+	    Documentation/trace/rv/safety-monitor.rst

I'm curious about what "WTD" means.

I see lots of WDT in drivers/watchdog/Kconfig
(where it means WatchDog Timer AFAIK).

thanks.

Signature · 1 line
~Randy
#71 lore raw

Re: [PATCH V4 18/20] rv/monitor: Add safe watchdog monitor

Quoted 22 lines · Randy Dunlap

Hi--

Quoted 7 lines · Daniel Bristot de Oliveira
kernel/trace/rv/Kconfig +0 -0
index 21f03fb3101a..b14ae63e792b 100644
@@ -45,6 +45,16 @@ config RV_MON_WWNR
 	  illustrates the usage of per-task monitor. The model is
 	  broken on purpose: it serves to test reactors.

kernel/trace/rv/Kconfig config RV_MON_WWNR · line 56

Quoted 10 lines
kernel/trace/rv/Kconfig +9 -0
 
+config RV_MON_SAFE_WTD
+	select DA_MON_EVENTS_IMPLICIT
+	bool "Safety watchdog"
+	help
+	  Enable safe_wtd, this monitor observes the interaction
+	  between a user-space safety monitor and a watchdog device.
+
+	  For futher information see:
+	    Documentation/trace/rv/safety-monitor.rst

I'm curious about what "WTD" means.

Quoted 2 lines

I see lots of WDT in drivers/watchdog/Kconfig
(where it means WatchDog Timer AFAIK).

Yep, watchdog. I will add the long description right after the first WTD appearance.

-- Daniel

Quoted 1 line

thanks.

#72 lore raw

Re: [PATCH V4 18/20] rv/monitor: Add safe watchdog monitor

In reply to Daniel Bristot de Oliveira
Quoted 3 lines · Daniel Bristot de Oliveira

The watchdog is an essential building block for the usage of Linux in safety-critical systems because it allows the system to be monitored from an external element - the watchdog hardware, acting as a safety-monitor.

Guenter and Steven,

I will move the watchdog monitor to a separated thread, as it will require further discussions specific to it, mainly to get feedback from watchdog maintainers.

Anyways, I am adding additional information for the sample monitors based on our discussions here.

They are:

I added documentation about the automata format and the translation between the formal <-> dot -> C.

I am adding the .dot file to the tools/verification/models/ so that one can get the .dot file and convert it to other formats, like, a png file. This will make it easy to read the automata model.

I am adding a .rst documentation for each model, including details about it.

I will send the v5 of RV without the safe_wtd monitor and then start a new one about the watchdog later, after getting the RV interface series ready.

-- Daniel

#73 lore raw

PATCH v4 19/20 rv/safety_app: Add a safety_app sample

In reply to Daniel Bristot de Oliveira

This is the sample code of a safety application that uses the watchdog as a safety monitor and the RV monitors to monitor this interaction/get feedback from kernel about the watchdog states.

This tool first creates a trace instance to follow the RV events and then enables RV monitor. After that, the tool configures the watchdog and starts running the main loop.

The main loop runs a use-case-specific function, like checking the system. If the system is running as expected, it pings the watchdog. After pinging the watchdog, the tool then collects trace information to see if the RV monitor received the expected events and is in a safe/safe_nwo state.

For further information, run safety_app --help

The safety-app specification was developed together with Gabriele Paoloni, in the context of the Linux Foundation Elisa Project.

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 tools/verification/safety_app/Makefile     |  51 ++
 tools/verification/safety_app/safety_app.c | 614 +++++++++++++++++++++
 2 files changed, 665 insertions(+)
 create mode 100644 tools/verification/safety_app/Makefile
 create mode 100644 tools/verification/safety_app/safety_app.c
tools/verification/safety_app/Makefile new file +51 -0
new file mode 100644
index 000000000000..002531022e45
@@ -0,0 +1,51 @@
+NAME	:=	safety_app
+VERSION	:=	0.1
+
+# From libtracefs:
+# Makefiles suck: This macro sets a default value of $(2) for the
+# variable named by $(1), unless the variable has been set by
+# environment or command line. This is necessary for CC and AR
+# because make sets default values, so the simpler ?= approach
+# won't work as expected.
+define allow-override
+  $(if $(or $(findstring environment,$(origin $(1))),\
+            $(findstring command line,$(origin $(1)))),,\
+    $(eval $(1) = $(2)))
+endef
+
+# Allow setting CC and AR, or setting CROSS_COMPILE as a prefix.
+$(call allow-override,CC,$(CROSS_COMPILE)gcc)
+$(call allow-override,AR,$(CROSS_COMPILE)ar)
+$(call allow-override,STRIP,$(CROSS_COMPILE)strip)
+$(call allow-override,PKG_CONFIG,pkg-config)
+$(call allow-override,LD_SO_CONF_PATH,/etc/ld.so.conf.d/)
+$(call allow-override,LDCONFIG,ldconfig)
+
+INSTALL	=	install
+FOPTS	:=	-flto=auto -ffat-lto-objects -fexceptions -fstack-protector-strong \
+		-fasynchronous-unwind-tables -fstack-clash-protection
+WOPTS	:= 	-Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -Wno-maybe-uninitialized
+
+TRACEFS_HEADERS	:= $$($(PKG_CONFIG) --cflags libtracefs)
+
+CFLAGS	:=	-O -g -DVERSION=\"$(VERSION)\" $(FOPTS) $(WOPTS) $(TRACEFS_HEADERS)
+LDFLAGS	:=	-ggdb
+LIBS	:=	$$($(PKG_CONFIG) --libs libtracefs)
+FILES	:=	Makefile
+BINDIR	:=	/usr/bin
+
+OBJ	:=	$(NAME).o
+
+.PHONY:	all
+all:	$(OBJ)
+	$(CC) -o $(NAME) $(LDFLAGS) $(OBJ) $(LIBS)
+
+.PHONY: install
+install:
+	$(INSTALL) -d -m 755 $(DESTDIR)$(BINDIR)
+	$(INSTALL) $(NAME) -m 755 $(DESTDIR)$(BINDIR)
+	$(STRIP) $(DESTDIR)$(BINDIR)/$(NAME)
+
+.PHONY: clean
+clean:
+	@rm -rf *~ $(OBJ) $(NAME)
tools/verification/safety_app/safety_app.c new file +614 -0

614 lines not on this page · raw message

Signature · 1 line
2.35.1
#74 lore raw

PATCH v4 20/20 Documentation/rv: Add watchdog-monitor documentation

In reply to Daniel Bristot de Oliveira

Adds documentation about the safe_wtd and safe_wtd_nwo RV monitors, and their usage via a safety application.

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 Documentation/trace/rv/watchdog-monitor.rst | 250 ++++++++++++++++++++
 1 file changed, 250 insertions(+)
 create mode 100644 Documentation/trace/rv/watchdog-monitor.rst
Documentation/trace/rv/watchdog-monitor.rst new file +250 -0
new file mode 100644
index 000000000000..2b142fb31572
@@ -0,0 +1,250 @@
+Watchdog monitor
+----------------
+
+The watchdog is an essential building block for the usage of Linux in
+safety-critical systems because it allows the system to be monitored from
+an external element - the watchdog hardware, acting as a safety-monitor.
+
+A user-space application controls the watchdog device via the watchdog
+interface. This application, hereafter safety_app, enables the watchdog
+and periodically pets the watchdog upon correct completion of the safety
+related processing.
+
+If the safety_app, for any reason, stops pinging the watchdog,
+the watchdog hardware can set the system in a fail-safe state. For
+example, shutting the system down.
+
+Given the importance of the safety_app / watchdog hardware couple,
+the interaction between these software pieces also needs some
+sort of monitoring. In other words, "who monitors the monitor?"
+
+The safe watchdog (safe_wtd) RV monitor monitors the interaction between
+the safety_app and the watchdog device, enforcing the correct sequence of
+events that leads the system to a safe state.
+
+Furthermore, the safety_app can monitor the RV monitor by collecting the
+events generated by the RV monitor itself via tracing interface. In this way,
+closing the monitoring loop with the safety_app.
+
+A diagram of the components and their interactions is::
+
+  user-space:
+                  +--------------------------------+
+                  | safety_app                     |-----------+
+                  +--------------------------------+           |
+                     |                    ^                    |
+                     | Configure          | Enable and         |
+                     |                    | check data         |
+  ===================+====================+===============     |
+  kernel-space:      |                    |                    |
+                     v                    v                    |
+                +----------+  instr.     +-------------+       |
+                | watchdog | ----------->| RV Monitor  |----+  |
+                | device   |             +-------------+    |  |
+                +----------+                                |  |
+                  |                                         |  |
+                  |                                         |  |
+  ================+======================================   |  |
+  hardware:       |                                         |  |
+                  v                                         |  +-> Bring the system
+                +--------------------+                      +----> to a safe state,
+                | watchdog hardware  |---------------------------> e.g., halt.
+                +--------------------+
+
+Sample safety_app
+-----------------
+
+The user-space safety_app sample code in ``tools/verification/safety_app/``
+serves to illustrate the usage of the RV monitors for this use-case, as
+well as the starting point to the development of a user-specific safety_app.
+
+Watchdog events
+---------------
+
+The RV monitor observes the watchdog by using instrumentation to
+process the events generated by the interaction between the
+safety_app and the watchdog device layer in kernel.
+
+The monitored events are:
+
+  - watchdog:watchdog_open: open the watchdog device;
+  - watchdog:watchdog_close: close the watchdog device;
+  - watchdog:watchdog_start: start the watchdog;
+  - watchdog:watchdog_stop: stop the watchdog;
+  - watchdog:watchdog_set_timeout: set the watchdog timeout;
+  - watchdog:watchdog_ping: reprogram the watchdog with the previously set
+    timeout;
+  - watchdog:watchdog_nowayout: prevents the watchdog from stopping;
+  - watchdog:watchdog_set_keep_alive: set an intermediary ping to overcome
+    the limitation of a hardware watchdog maximum timeout being shorter than
+    the timeout set by the user-space tool;
+  - watchdog:watchdog_keep_alive: the execution of the function that runs the
+    intermediary keep alive ping;
+
+RV monitor events
+-----------------
+
+The RV monitor monitors the relevant events as an outside observer,
+interpreting all the components (the hardware; the watchdog device
+interface; and the safety monitor) as an integrated component.
+
+The events selected for the monitor are:
+
+  - other_threads: an event generated by any thread other than the
+    one that set nowayout or open the watchdog the last time.
+  - open: a thread opens the watchdog to manipulate it;
+  - close: a thread closes the watchdog;
+  - start: starts the watchdog countdown;
+  - stop: stops the watchdog;
+  - set_safe_timeout: configures the watchdog with a given timeout;
+  - ping: resets the watchdog countdown with the previously configured timeout;
+  - nowayout: prevents the watchdog to be stopped until the system's shutdown;
+  - sched_keep_alive: schedules a kernel worker to ping the watchdog if the
+    timeout is longer than the watchdog hardware can handle.
+  - keep_alive: executes the previously scheduled watchdog ping;
+
+Noting that the events that does not appear in the automata models are
+considered blocked events, and their execution will always cause the
+RV monitor to react to an unexpected event.
+
+RV monitor specification
+------------------------
+
+The monitor's goal is to assess a set of specifications that conducts the
+system to a safe state.
+
+These specifications are:
+
+  - 1: Once open, only one process manipulates the watchdog;
+  - 2: Following 1, the keep-alive mechanisms will not be used;
+  - 3: If required, nowayout will be set before opening the watchdog;
+  - 4: A safe timeout must be set;
+  - 5: At least one ping must be made before entering the safe/safe_nwo states
+  - 6: The RV monitor does not react if the watchdog is closed without stopping.
+       But the hardware watchdog is expected to react.
+
+Deterministic automata monitors
+-------------------------------
+
+Following the specifications, a deterministic automata monitor
+was developed. The monitor is modeled as Deterministic Automata model.
+
+The deterministic automata model for safe_wtd is::
+
+              #==================================#   other_threads
+              H                                  H ----------------+
+ -----------> H               init               H                 |
+              H                                  H <---------------+
+              #==================================#
+                      |     |     ^
+                      |     |     |               close
+                      |     |     +----------------------------------------------------+
+                      |     |                                                          |
+                      |     |                     open                                 |
+                      |     +------------------------------------------------------+   |
+                      |                                                            |   |
+                      |  nowayout                                                  |   |
+                      v                                                            |   |
+    nowayout        +-------------------+                                          |   |
+    other_threads   |                   |          nowayout                        |   |
+  +---------------- |        nwo        |<-------------------------------------+   |   |
+  |                 |                   |                                      |   |   |
+  +---------------> |                   | <+                                   |   |   |
+                    +-------------------+  |                                   |   |   |
+                      |                    |                                   |   |   |
+                      | open               | close                             |   |   |
+                      v                    |                                   |   |   |
+                    +-------------------+  |                                   |   |   |
+                    |    opened_nwo     | -+                                   |   |   |
+                    +-------------------+                                      |   |   |
+                      |                                                        |   |   |
+                      | start                                                  |   |   |
+                      v                                                        |   |   |
+                    +-------------------+                                      |   |   |
+  +---------------> |    started_nwo    | -+                                   |   |   |
+  |                 +-------------------+  |                                   |   |   |
+  |                   |                    |                                   |   |   |
+  | open              | set_safe_timeout   |                                   |   |   |
+  |                   v                    |                                   |   |   |
+  |                 +-------------------+  |                                   |   |   |
+  |                 |      set_nwo      |  |                                   |   |   |
+  |                 +-------------------+  |                                   |   |   |
+  |                           |            |                                   |   |   |
+  |     +-------------+       | ping       |                                   |   |   |
+  |     |             |       |            |                                   |   |   |
+  |     | ping        v       v            |                                   |   |   |
+  |     |           +-------------------+  |                                   |   |   |
+  |     +-----------|     safe_nwo      |  |                                   |   |   |
+  |                 +-------------------+  |                                   |   |   |
+  |                   |                    |                                   |   |   |
+  |                   | close              | close                             |   |   |
+  |                   v                    v                                   |   |   |
+  |                 +----------------------------------+   nowayout            |   |   |
+  |                 |                                  |   other_threads       |   |   |
+  |                 |        closed_running_nwo        | ----------------+     |   |   |
+  |                 |                                  |                 |     |   |   |
+  +---------------- |                                  | <---------------+     |   |   |
+                    +----------------------------------+                       |   |   |
+                        |        nowayout             ^                        |   |   |
+                        +-----------------------------+                        |   |   |
+                                                                               |   |   |
+                                                                               |   |   |
+                               +-------------------+           +--------+      |   |   |
+                               |                   |           |        |------+---+   |
+                               |      started      |  start    | opened |      |       |
+             +---------------- |                   | <-------- |        |>-----+-------+
+             |                 +-------------------+           +--------+      |       ^
+             |                   |                                             |       |
+             |                   | set_safe_timeout              +-------------+-------+
+             |                   v                               |             |
+             |                 +-------------------+             |             |
+             |                 |                   |             |             |
+             |                 |        set        |             |             |
+  +----------+---------------> |                   |             |             |
+  |          |                 +-------------------+             |             |
+  |          |                   |                               |             |
+  |          |                   | ping                          |             |
+  |          |                   v                               |             |
+  |          |                 +-------------------+   ping      |             |
+  |          |                 |                   | -------+    |             |
+  |          |           +---- |       safe        |        |    |             |
+  |          |           |     |                   | <------+    |             |
+  |          |           |     +-------------------+             |             |
+  |          |           |       |                               |             |
+  |          | stop      |       | stop                          |             |
+  |          |           |       v                               |             |
+  |          |           |     +-------------------+   close     |             |
+  |          +-----------+---> |      stopped      |-------------+             |
+  |                      |     +-------------------+                           |
+  |                      +---+                                                 |
+  |                          | close                                           |
+  |                          v                                                 |
+  |     other_threads  +----------------------------------------+              |
+  |   +--------------> |                                        |              |
+  |   |                |             closed_running             |              |
+  |   +--------------- |                                        |--------------+
+  |                    +----------------------------------------+
+  |                               |          ^
+  |                         open  |          | close
+  |                               v          |
+  |    set_safe_timeout       +-------------------+
+  +-------------------------> |     reopened      |
+                                +-------------------+
+
+It is important to note that the events sched_keep_alive and keep_alive
+are not allowed in the monitor (they are said to be blocked events).
+The execution of any blocked events leads the RV monitor to react.
+
+Additional options
+------------------
+
+The RV monitor also has a set of options enabled via kernel command
+line/module options. They are:
+
+ - watchdog_id: the device id to monitor (default 0);
+ - dont_stop: once enabled, do not allow the RV monitor to be stopped (default off);
+ - safe_timeout: define a maximum safe value that a user-space application can
+   set as the watchdog timeout (default unlimited);
+ - check_timeout: After every ping, check if the time left in the watchdog is less
+   than or equal to the last timeout set for the watchdog. It only works for watchdog
+   devices that provide the get_timeleft() function (default off);
Signature · 1 line
2.35.1
#75 lore raw

Re: [PATCH V4 20/20] Documentation/rv: Add watchdog-monitor documentation

Quoted 256 lines · Daniel Bristot de Oliveira

Adds documentation about the safe_wtd and safe_wtd_nwo RV monitors, and their usage via a safety application.

Cc Wim Van Sebroeck <[email protected]>
Cc Guenter Roeck <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Marco Elver <[email protected]>
Cc Dmitry Vyukov <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Gabriele Paoloni <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Clark Williams <[email protected]>
---
 Documentation/trace/rv/watchdog-monitor.rst | 250 ++++++++++++++++++++
 1 file changed, 250 insertions(+)
 create mode 100644 Documentation/trace/rv/watchdog-monitor.rst
Documentation/trace/rv/watchdog-monitor.rst new file +222 -0
new file mode 100644
index 000000000000..2b142fb31572
@@ -0,0 +1,250 @@
+Watchdog monitor
+----------------
+
+The watchdog is an essential building block for the usage of Linux in
+safety-critical systems because it allows the system to be monitored from
+an external element - the watchdog hardware, acting as a safety-monitor.
+
+A user-space application controls the watchdog device via the watchdog
+interface. This application, hereafter safety_app, enables the watchdog
+and periodically pets the watchdog upon correct completion of the safety
+related processing.
+
+If the safety_app, for any reason, stops pinging the watchdog,
+the watchdog hardware can set the system in a fail-safe state. For
+example, shutting the system down.
+
+Given the importance of the safety_app / watchdog hardware couple,
+the interaction between these software pieces also needs some
+sort of monitoring. In other words, "who monitors the monitor?"
+
+The safe watchdog (safe_wtd) RV monitor monitors the interaction between
+the safety_app and the watchdog device, enforcing the correct sequence of
+events that leads the system to a safe state.
+
+Furthermore, the safety_app can monitor the RV monitor by collecting the
+events generated by the RV monitor itself via tracing interface. In this way,
+closing the monitoring loop with the safety_app.
+
+A diagram of the components and their interactions is::
+
+  user-space:
+                  +--------------------------------+
+                  | safety_app                     |-----------+
+                  +--------------------------------+           |
+                     |                    ^                    |
+                     | Configure          | Enable and         |
+                     |                    | check data         |
+  ===================+====================+===============     |
+  kernel-space:      |                    |                    |
+                     v                    v                    |
+                +----------+  instr.     +-------------+       |
+                | watchdog | ----------->| RV Monitor  |----+  |
+                | device   |             +-------------+    |  |
+                +----------+                                |  |
+                  |                                         |  |
+                  |                                         |  |
+  ================+======================================   |  |
+  hardware:       |                                         |  |
+                  v                                         |  +-> Bring the system
+                +--------------------+                      +----> to a safe state,
+                | watchdog hardware  |---------------------------> e.g., halt.
+                +--------------------+
+
+Sample safety_app
+-----------------
+
+The user-space safety_app sample code in ``tools/verification/safety_app/``
+serves to illustrate the usage of the RV monitors for this use-case, as
+well as the starting point to the development of a user-specific safety_app.
+
+Watchdog events
+---------------
+
+The RV monitor observes the watchdog by using instrumentation to
+process the events generated by the interaction between the
+safety_app and the watchdog device layer in kernel.
+
+The monitored events are:
+
+  - watchdog:watchdog_open: open the watchdog device;
+  - watchdog:watchdog_close: close the watchdog device;
+  - watchdog:watchdog_start: start the watchdog;
+  - watchdog:watchdog_stop: stop the watchdog;
+  - watchdog:watchdog_set_timeout: set the watchdog timeout;
+  - watchdog:watchdog_ping: reprogram the watchdog with the previously set
+    timeout;
+  - watchdog:watchdog_nowayout: prevents the watchdog from stopping;
+  - watchdog:watchdog_set_keep_alive: set an intermediary ping to overcome
+    the limitation of a hardware watchdog maximum timeout being shorter than
+    the timeout set by the user-space tool;
+  - watchdog:watchdog_keep_alive: the execution of the function that runs the
+    intermediary keep alive ping;
+
+RV monitor events
+-----------------
+
+The RV monitor monitors the relevant events as an outside observer,
+interpreting all the components (the hardware; the watchdog device
+interface; and the safety monitor) as an integrated component.
+
+The events selected for the monitor are:
+
+  - other_threads: an event generated by any thread other than the
+    one that set nowayout or open the watchdog the last time.
+  - open: a thread opens the watchdog to manipulate it;
+  - close: a thread closes the watchdog;
+  - start: starts the watchdog countdown;
+  - stop: stops the watchdog;
+  - set_safe_timeout: configures the watchdog with a given timeout;
+  - ping: resets the watchdog countdown with the previously configured timeout;
+  - nowayout: prevents the watchdog to be stopped until the system's shutdown;
+  - sched_keep_alive: schedules a kernel worker to ping the watchdog if the
+    timeout is longer than the watchdog hardware can handle.
+  - keep_alive: executes the previously scheduled watchdog ping;
+
+Noting that the events that does not appear in the automata models are
+considered blocked events, and their execution will always cause the
+RV monitor to react to an unexpected event.
+
+RV monitor specification
+------------------------
+
+The monitor's goal is to assess a set of specifications that conducts the
+system to a safe state.
+
+These specifications are:
+
+  - 1: Once open, only one process manipulates the watchdog;
+  - 2: Following 1, the keep-alive mechanisms will not be used;
+  - 3: If required, nowayout will be set before opening the watchdog;
+  - 4: A safe timeout must be set;
+  - 5: At least one ping must be made before entering the safe/safe_nwo states
+  - 6: The RV monitor does not react if the watchdog is closed without stopping.
+       But the hardware watchdog is expected to react.
+
+Deterministic automata monitors
+-------------------------------
+
+Following the specifications, a deterministic automata monitor
+was developed. The monitor is modeled as Deterministic Automata model.
+
+The deterministic automata model for safe_wtd is::
+
+              #==================================#   other_threads
+              H                                  H ----------------+
+ -----------> H               init               H                 |
+              H                                  H <---------------+
+              #==================================#
+                      |     |     ^
+                      |     |     |               close
+                      |     |     +----------------------------------------------------+
+                      |     |                                                          |
+                      |     |                     open                                 |
+                      |     +------------------------------------------------------+   |
+                      |                                                            |   |
+                      |  nowayout                                                  |   |
+                      v                                                            |   |
+    nowayout        +-------------------+                                          |   |
+    other_threads   |                   |          nowayout                        |   |
+  +---------------- |        nwo        |<-------------------------------------+   |   |
+  |                 |                   |                                      |   |   |
+  +---------------> |                   | <+                                   |   |   |
+                    +-------------------+  |                                   |   |   |
+                      |                    |                                   |   |   |
+                      | open               | close                             |   |   |
+                      v                    |                                   |   |   |
+                    +-------------------+  |                                   |   |   |
+                    |    opened_nwo     | -+                                   |   |   |
+                    +-------------------+                                      |   |   |
+                      |                                                        |   |   |
+                      | start                                                  |   |   |
+                      v                                                        |   |   |
+                    +-------------------+                                      |   |   |
+  +---------------> |    started_nwo    | -+                                   |   |   |
+  |                 +-------------------+  |                                   |   |   |
+  |                   |                    |                                   |   |   |
+  | open              | set_safe_timeout   |                                   |   |   |
+  |                   v                    |                                   |   |   |
+  |                 +-------------------+  |                                   |   |   |
+  |                 |      set_nwo      |  |                                   |   |   |
+  |                 +-------------------+  |                                   |   |   |
+  |                           |            |                                   |   |   |
+  |     +-------------+       | ping       |                                   |   |   |
+  |     |             |       |            |                                   |   |   |
+  |     | ping        v       v            |                                   |   |   |
+  |     |           +-------------------+  |                                   |   |   |
+  |     +-----------|     safe_nwo      |  |                                   |   |   |
+  |                 +-------------------+  |                                   |   |   |
+  |                   |                    |                                   |   |   |
+  |                   | close              | close                             |   |   |
+  |                   v                    v                                   |   |   |
+  |                 +----------------------------------+   nowayout            |   |   |
+  |                 |                                  |   other_threads       |   |   |
+  |                 |        closed_running_nwo        | ----------------+     |   |   |
+  |                 |                                  |                 |     |   |   |
+  +---------------- |                                  | <---------------+     |   |   |
+                    +----------------------------------+                       |   |   |
+                        |        nowayout             ^                        |   |   |
+                        +-----------------------------+                        |   |   |
+                                                                               |   |   |
+                                                                               |   |   |
+                               +-------------------+           +--------+      |   |   |
+                               |                   |           |        |------+---+   |
+                               |      started      |  start    | opened |      |       |
+             +---------------- |                   | <-------- |        |>-----+-------+
+             |                 +-------------------+           +--------+      |       ^
+             |                   |                                             |       |
+             |                   | set_safe_timeout              +-------------+-------+
+             |                   v                               |             |
+             |                 +-------------------+             |             |
+             |                 |                   |             |             |
+             |                 |        set        |             |             |
+  +----------+---------------> |                   |             |             |
+  |          |                 +-------------------+             |             |
+  |          |                   |                               |             |
+  |          |                   | ping                          |             |
+  |          |                   v                               |             |
+  |          |                 +-------------------+   ping      |             |
+  |          |                 |                   | -------+    |             |
+  |          |           +---- |       safe        |        |    |             |
+  |          |           |     |                   | <------+    |             |
+  |          |           |     +-------------------+             |             |
+  |          |           |       |                               |             |
+  |          | stop      |       | stop                          |             |
+  |          |           |       v                               |             |
+  |          |           |     +-------------------+   close     |             |
+  |          +-----------+---> |      stopped      |-------------+             |
+  |                      |     +-------------------+                           |
+  |                      +---+                                                 |
+  |                          | close                                           |
+  |                          v                                                 |
+  |     other_threads  +----------------------------------------+              |

Documentation/trace/rv/watchdog-monitor.rst line 232

Quoted 10 lines
Documentation/trace/rv/watchdog-monitor.rst new file +10 -0
+  |   +--------------> |                                        |              |
+  |   |                |             closed_running             |              |
+  |   +--------------- |                                        |--------------+
+  |                    +----------------------------------------+
+  |                               |          ^
+  |                         open  |          | close
+  |                               v          |
+  |    set_safe_timeout       +-------------------+
+  +-------------------------> |     reopened      |
+                                +-------------------+

So I must check the model matrix in patch18 and the above graph to check if the matrix have the not right value or something in this graph go wrong.

open
o init state ------> opened state: there lack a direction arrow.

nwo
o closed_running state -------> closed_running_nwo state: this is the state trasfer indicated from matrix, but the above graph is not consistent with it. In graph:
nwo
closed_running state -------> nwo state. I prefer the state closed_running_nwo.

nwo
other_thread
o closed_running_nwo state ---------------> closed_running_nwo state. The graph above have a duplicate nwo event leading to state transfer. open
closed_running_nwo state ---------------> started_nwo state. I think it should transfer to opened_nwo state.

set_safe_timeout
o reopened state ------------------> set state: the direction is from reopened state to set state. The graph above have two direction arrow. stop
reopenped state ------------------> opened state. why stop event trigger it to opened state not stoped state. I think it should transfer to stoped state like the started state do.

o started state do not have link to closed_running state by close event in graph. Like started_nwo state do, it should have a direct close event to link to closed_running state.

Modified the above graph like this:

+
+ #==================================# other_threads

+              H                                  H ----------------+
+ -----------> H               init               H                 |
+              H                                  H <---------------+
+              #==================================#
+                      |     |     ^
+                      |     |     |               close
+                      |     |     +----------------------------------------------------+
+                      |     |                                                          |
+                      |     |                     open                                 |
+                      |     +------------------------------------------------------+   |
+                      |                                                            |   |
+                      |  nowayout                                                  |   |
+                      v                                                            |   |
+    nowayout        +-------------------+                                          |   |
+    other_threads   |                   |                                          |   |
+  +---------------- |        nwo        |                                          |   |
+  |                 |                   |                                          |   |
+  +---------------> |                   | <+                                       |   |
+                    +-------------------+  |                                       |   |
+                      |                    |                                       |   |
+                      | open               | close                                 |   |
+                      v                    |                                       |   |
+                    +-------------------+  |                                       |   |
+  +---------------> |    opened_nwo     | -+                                       |   |
+  |                 +-------------------+                                          |   |
+  |                   |                                                            |   |
+  |                   | start                                                      |   |
+  |                   v                                                            |   |
+  |                 +-------------------+                                          |   |
+  |                 |    started_nwo    | -+                                       |   |
+  |                 +-------------------+  |                                       |   |
+  |                   |                    |                                       |   |
+  | open              | set_safe_timeout   |                                       |   |
+  |                   v                    |                                       |   |
+  |                 +-------------------+  |                                       |   |
+  |                 |      set_nwo      |  |                                       |   |
+  |                 +-------------------+  |                                       |   |
+  |                           |            |                                       |   |
+  |     +-------------+       | ping       |                                       |   |
+  |     |             |       |            |                                       |   |
+  |     | ping        v       v            |                                       |   |
+  |     |           +-------------------+  |                                       |   |
+  |     +-----------|     safe_nwo      |  |                                       |   |
+  |                 +-------------------+  |                                       |   |
+  |                   |                    |                                       |   |
+  |                   | close              | close                                 |   |
+  |                   v                    v                                       |   |
+  |                 +----------------------------------+   nowayout                |   |
+  |                 |                                  |   other_threads           |   |
+  |                 |        closed_running_nwo        | ----------------+         |   |
+  |                 |                                  |                 |         |   |
+  +---------------- |                                  | <---------------+         |   |
+                    +----------------------------------+                           |   |
+                                        ^                                          |   |
+                                        | nowayout                                 |   |
+                                        +--------------------------------------+   |   |
+                                                                               |   |   |
+                               +-------------------+         	 +--------+     |   |   |
+                               |                   |   start    |        |<----+---+   |
+                               |      started      | <--------- | opened |     |       |
+             +---------------- |                   | ---------+ |        |>----+-------+
+             |                 +-------------------+   close  | +--------+     |       ^
+             |                   |                            |                |       |
+             |                   | set_safe_timeout           |    +-----------+-------+
+             |                   v                            |    |           |
+             |                 +-------------------+          |    |           |
+             |                 |                   |          |    |           |
+             |                 |        set        |          |    |           |
+  +----------+---------------> |                   |          |    |           |
+  |          |                 +-------------------+          |    |           |
+  |          |                   |                            |    |           |
+  |          |                   | ping                       |    |           |
+  |          |                   v                            |    |           |
+  |          |                 +-------------------+   ping   |    |           |
+  |          |                 |                   | -------+ |    |           |
+  |          |           +---- |       safe        |        | |    |           |
+  |          |           |     |                   | <------+ |    |           |
+  |          |           |     +-------------------+          |    |           |
+  |          |           |       |                            |    |           |
+  |          | stop      |       | stop                       |    |           |
+  |          |           |       v                            |    |           |
+  |          |           |     +-------------------+   close  |    |           |
+  |          +-----------+---> |      stopped      |----------+----+           |
+  |                      |     +-------------------+          |                |
+  |                      +---+             ^                  |                |
+  |                          | close       |                  |                |
+  |                          |             +------------------+------+         |
   |                          v                                v      |         |
+  |     other_threads  +----------------------------------------+    |         |
+  |   +--------------> |                                        |    |         |
+  |   |                |             closed_running             |    |         |
+  |   +--------------- |                                        |----+---------+
+  |                    +----------------------------------------+    |
+  |                               |          ^                       |
+  |                         open  |          | close                 |stop
+  |                               v          |                       |
+  |    set_safe_timeout       +-------------------+                  |
+  +-------------------------- |     reopened      |------------------+

+ +-------------------+

Thanks

Quoted 20 lines
Documentation/trace/rv/watchdog-monitor.rst new file +17 -0
+It is important to note that the events sched_keep_alive and keep_alive
+are not allowed in the monitor (they are said to be blocked events).
+The execution of any blocked events leads the RV monitor to react.
+
+Additional options
+------------------
+
+The RV monitor also has a set of options enabled via kernel command
+line/module options. They are:
+
+ - watchdog_id: the device id to monitor (default 0);
+ - dont_stop: once enabled, do not allow the RV monitor to be stopped (default off);
+ - safe_timeout: define a maximum safe value that a user-space application can
+   set as the watchdog timeout (default unlimited);
+ - check_timeout: After every ping, check if the time left in the watchdog is less
+   than or equal to the last timeout set for the watchdog. It only works for watchdog
+   devices that provide the get_timeleft() function (default off);
Signature · 1 line
2.35.1
#76 lore raw

Re: [PATCH V4 20/20] Documentation/rv: Add watchdog-monitor documentation

Quoted 3 lines · Tao Zhou

So I must check the model matrix in patch18 and the above graph to check if the matrix have the not right value or something in this graph go wrong.

It was my fault, the ASCII art generated by graph-easy was too wide, so I had to manually change it... and so I broke it.

At least this validates that it is better to have it automatically generated :-)

Thanks for reviewing it Tao, I do appreciate these kind of reviews!!!!

-- Daniel

#77 lore raw
In reply to Daniel Bristot de Oliveira

Hi Daniel,

Quoted 44 lines · Daniel Bristot de Oliveira

Over the last years, I've been exploring the possibility of verifying the Linux kernel behavior using Runtime Verification.

Runtime Verification (RV) is a lightweight (yet rigorous) method that complements classical exhaustive verification techniques (such as model checking and theorem proving) with a more practical approach for complex systems.

Instead of relying on a fine-grained model of a system (e.g., a re-implementation a instruction level), RV works by analyzing the trace of the system's actual execution, comparing it against a formal specification of the system behavior.

The usage of deterministic automaton for RV is a well-established approach. In the specific case of the Linux kernel, you can check how to model complex behavior of the Linux kernel with this paper:

DE OLIVEIRA, Daniel Bristot; CUCINOTTA, Tommaso; DE OLIVEIRA, Romulo Silva. *Efficient formal verification for the Linux kernel.* In: International Conference on Software Engineering and Formal Methods. Springer, Cham, 2019. p. 315-332.

And how efficient is this approach here:

DE OLIVEIRA, Daniel B.; DE OLIVEIRA, Romulo S.; CUCINOTTA, Tommaso. *A thread synchronization model for the PREEMPT_RT Linux kernel.* Journal of Systems Architecture, 2020, 107: 101729.

tlrd: it is possible to model complex behaviors in a modular way, with an acceptable overhead (even for production systems). See this presentation at 2019's ELCE: https://www.youtube.com/watch?v=BfTuEHafNgg

Here I am proposing a more practical approach for the usage of deterministic automata for runtime verification, and it includes:

- An interface for controlling the verification;
- A tool and set of headers that enables the automatic code generation of the RV monitor (Monitor Synthesis);
- Sample monitors to evaluate the interface;
- A sample monitor developed in the context of the Elisa Project demonstrating how to use RV in the context of safety-critical systems.

Quoted 2 lines

Given that RV is a tracing consumer, the code is being placed inside the tracing subsystem (Steven and I have been talking about it for a while).

This is interesting work!

I applied the series on top of commit 78ca55889a549a9a194c6ec666836329b774ab6d in upstream. Then, I got some compile/link error for CONFIG_RV_MON_WIP and CONFIG_RV_MON_SAFE_WTD. I was able to compile the kernel with these two configs disabled. However, I hit the some issue with monitors/wwnr/enabled :

    [root@eth50-1 ~]# cd /sys/kernel/debug/tracing/rv/
    [root@eth50-1 rv]# cat available_monitors
    wwnr
    [root@eth50-1 rv]# echo wwnr > enabled_monitors
    [root@eth50-1 rv]# cd monitors/
    [root@eth50-1 monitors]# cd wwnr/
    [root@eth50-1 wwnr]# ls
    desc  enable  reactors
    [root@eth50-1 wwnr]# cat enable
    1
    [root@eth50-1 wwnr]# echo 0 > enable   <<< hangs

The last echo command hangs forever on a qemu vm. I haven't figured out why this happens though.

I also have a more general question: can we do RV with BPF and simplify the work? AFAICT, the idea of RV is to maintain a state machine based on events. If something unexpected happens, call the reactor.

IIUC, BPF has most of these building blocks ready for use. With BPF, we can ship many RV monitors without much kernel changes.

Here is my toy wwnr in bpftrace. The reactor is "print to console". It runs on most systems with BPF and tracepoint enabled. I probably missed some events, as a result, the script triggers the "reactor" a lot.

=============== 8< ======================
[root@ ~]# cat wwnr.bt
/*
* task_state[pid]
* not_running = 1
* running = 2
*/
tracepoint:sched:sched_switch

{
        if (args->prev_state == 0x0001 /* TASK_INTERRUPTIBLE */) {
           /* after first suspension */
           @task_state[args->prev_pid] = 1;
        } else {
           if (@task_state[args->prev_pid] == 1) {
              printf("Something wrong, call reactor\n");
           }
           @task_state[args->prev_pid] = 1;
        }
        @task_state[args->next_pid] = 2;
}

tracepoint:sched:sched_wakeup

{
        if (@task_state[args->pid] == 2) {
           printf("Something wrong, call reactor\n");
           }
         @task_state[args->pid] = 2;
}

[root@ ~]# bpftrace wwnr.bt
<<<< some print >>>>
=============== 8< ======================

Does this (BPF for RV) make any sense?

Thanks,
Song

#78 lore raw
Quoted 6 lines · Song Liu

This is interesting work!

I applied the series on top of commit 78ca55889a549a9a194c6ec666836329b774ab6d in upstream. Then, I got some compile/link error for CONFIG_RV_MON_WIP and CONFIG_RV_MON_SAFE_WTD. I was able to compile the kernel with these two configs disabled.

I rebased the code and... it compiled. Maybe it was missing some config options that I forgot to set as "depends on" in the Kconfig.

Can you check if it was the same problem automatically reported?

Any further information here would help. I will revisit this.

However, I hit the some issue with monitors/wwnr/enabled :

Quoted 13 lines
    [root@eth50-1 ~]# cd /sys/kernel/debug/tracing/rv/
    [root@eth50-1 rv]# cat available_monitors
    wwnr
    [root@eth50-1 rv]# echo wwnr > enabled_monitors
    [root@eth50-1 rv]# cd monitors/
    [root@eth50-1 monitors]# cd wwnr/
    [root@eth50-1 wwnr]# ls
    desc  enable  reactors
    [root@eth50-1 wwnr]# cat enable
    1
    [root@eth50-1 wwnr]# echo 0 > enable   <<< hangs
Quoted 2 lines

The last echo command hangs forever on a qemu vm. I haven't figured out why this happens though.

I could reproduce it. It is an error in the return code of monitor_enable_write_data(), I fixed it locally (return retval ? retval : count; // needs more test), and will add it to the next version. Thanks!

Quoted 6 lines

I also have a more general question: can we do RV with BPF and simplify the work? AFAICT, the idea of RV is to maintain a state machine based on events. If something unexpected happens, call the reactor.

IIUC, BPF has most of these building blocks ready for use. With BPF, we can ship many RV monitors without much kernel changes.

I am aware of bpftrace and bpf + libbpf, and I have a PoC tool doing most of the work I do in C/kernel in C/bpf.

From the cover letter:

"Things kept for a second moment (after this patchset):
[...]
- dot2bpf"

The point is that there are use-cases in which the users need the code in C. One of those is the work being done in the Linux Foundation Elisa group. There will be more formalism, like timed automata... which will require infra-structure that is easily accessible in C... including synchronization, and reactors that are available only in C on "per use-cases" basis - for example on embedded devices.

On the other hand, there is ongoing research on asynchronous RV in which I am only using BPF on the instrumentation side, for more complex formalism running the processing in user-space (but for a different use-case, with different timing and logical properties).

I see C and BPF (and rust?...) as complementary tools that I can use, and we will have them all :-).

In this first series, I am adding the DA monitor in the kernel (motivated by Elisa), and the basic kernel interface. After that, I will add the dot2bpf for the cases in which BPF is a viable option... other formalism... other extensions from Elisa... and so on... collecting these methods in a single place.

-- Daniel

Quoted 2 lines

Thanks,
Song

#79 lore raw
Quoted 9 lines · Daniel Bristot de Oliveira
Quoted 6 lines · Song Liu

This is interesting work!

I applied the series on top of commit 78ca55889a549a9a194c6ec666836329b774ab6d in upstream. Then, I got some compile/link error for CONFIG_RV_MON_WIP and CONFIG_RV_MON_SAFE_WTD. I was able to compile the kernel with these two configs disabled.

Quoted 6 lines

I rebased the code and... it compiled. Maybe it was missing some config options that I forgot to set as "depends on" in the Kconfig.

Can you check if it was the same problem automatically reported?

Any further information here would help. I will revisit this.

Here are the error messages I got: https://pastebin.com/zJxMA6RK , and attached is the config file I used.

Quoted 38 lines

However, I hit the some issue with monitors/wwnr/enabled :

Quoted 13 lines
    [root@eth50-1 ~]# cd /sys/kernel/debug/tracing/rv/
    [root@eth50-1 rv]# cat available_monitors
    wwnr
    [root@eth50-1 rv]# echo wwnr > enabled_monitors
    [root@eth50-1 rv]# cd monitors/
    [root@eth50-1 monitors]# cd wwnr/
    [root@eth50-1 wwnr]# ls
    desc  enable  reactors
    [root@eth50-1 wwnr]# cat enable
    1
    [root@eth50-1 wwnr]# echo 0 > enable   <<< hangs
Quoted 2 lines

The last echo command hangs forever on a qemu vm. I haven't figured out why this happens though.

I could reproduce it. It is an error in the return code of monitor_enable_write_data(), I fixed it locally (return retval ? retval : count; // needs more test), and will add it to the next version. Thanks!

Quoted 6 lines

I also have a more general question: can we do RV with BPF and simplify the work? AFAICT, the idea of RV is to maintain a state machine based on events. If something unexpected happens, call the reactor.

IIUC, BPF has most of these building blocks ready for use. With BPF, we can ship many RV monitors without much kernel changes.

I am aware of bpftrace and bpf + libbpf, and I have a PoC tool doing most of the work I do in C/kernel in C/bpf.

From the cover letter:

"Things kept for a second moment (after this patchset):
[...]
- dot2bpf"

Quoted 6 lines

The point is that there are use-cases in which the users need the code in C. One of those is the work being done in the Linux Foundation Elisa group. There will be more formalism, like timed automata... which will require infra-structure that is easily accessible in C... including synchronization, and reactors that are available only in C on "per use-cases" basis - for example on embedded devices.

Where can I find more information about the constraints of these use cases? I am asking because there are multiple ways to load a BPF program to the system. If the constraint is that we cannot have bpftrace or bcc in the system, maybe it is ok to run a standalone binary (written in C, compiled on a different system). Or maybe we can load BPF programs in a kernel module, or compile the BPF programs into the kernel? (Yes, we can do it now, check kernel/bpf/preload). If any of these works, we can benefit from the good properties of BPF. For example, we can update the RV models without rebooting the system; and we can reuse various BPF maps, so we don't need to add union rv_task_monitor to task_struct.

Of course, we are out of luck if these systems cannot enable CONFIG_BPF at all. But I guess this is not common for modern embedded systems?

Thanks,
Song

#80 lore raw
Quoted 59 lines · Song Liu
Quoted 9 lines · Daniel Bristot de Oliveira
Quoted 6 lines · Song Liu

This is interesting work!

I applied the series on top of commit 78ca55889a549a9a194c6ec666836329b774ab6d in upstream. Then, I got some compile/link error for CONFIG_RV_MON_WIP and CONFIG_RV_MON_SAFE_WTD. I was able to compile the kernel with these two configs disabled.

Quoted 6 lines

I rebased the code and... it compiled. Maybe it was missing some config options that I forgot to set as "depends on" in the Kconfig.

Can you check if it was the same problem automatically reported?

Any further information here would help. I will revisit this.

Here are the error messages I got: https://pastebin.com/zJxMA6RK , and attached is the config file I used.

Quoted 38 lines

However, I hit the some issue with monitors/wwnr/enabled :

Quoted 13 lines
    [root@eth50-1 ~]# cd /sys/kernel/debug/tracing/rv/
    [root@eth50-1 rv]# cat available_monitors
    wwnr
    [root@eth50-1 rv]# echo wwnr > enabled_monitors
    [root@eth50-1 rv]# cd monitors/
    [root@eth50-1 monitors]# cd wwnr/
    [root@eth50-1 wwnr]# ls
    desc  enable  reactors
    [root@eth50-1 wwnr]# cat enable
    1
    [root@eth50-1 wwnr]# echo 0 > enable   <<< hangs
Quoted 2 lines

The last echo command hangs forever on a qemu vm. I haven't figured out why this happens though.

I could reproduce it. It is an error in the return code of monitor_enable_write_data(), I fixed it locally (return retval ? retval : count; // needs more test), and will add it to the next version. Thanks!

Quoted 6 lines

I also have a more general question: can we do RV with BPF and simplify the work? AFAICT, the idea of RV is to maintain a state machine based on events. If something unexpected happens, call the reactor.

IIUC, BPF has most of these building blocks ready for use. With BPF, we can ship many RV monitors without much kernel changes.

I am aware of bpftrace and bpf + libbpf, and I have a PoC tool doing most of the work I do in C/kernel in C/bpf.

From the cover letter:

"Things kept for a second moment (after this patchset):
[...]
- dot2bpf"

Quoted 8 lines
Quoted 6 lines

The point is that there are use-cases in which the users need the code in C. One of those is the work being done in the Linux Foundation Elisa group. There will be more formalism, like timed automata... which will require infra-structure that is easily accessible in C... including synchronization, and reactors that are available only in C on "per use-cases" basis - for example on embedded devices.

Where can I find more information about the constraints of these use cases?

Check the LF elisa workgroup.

Quoted 4 lines

I am asking because there are multiple ways to load a BPF program to the system. If the constraint is that we cannot have bpftrace or bcc in the system, maybe it is ok to run a standalone binary (written in C, compiled on a different system).

as I said... *I am aware of that*. I do like BPF! I was already convinced I will having things in BPF :-)

dot2bpf does stand alone application, C + libbpf (and I did it this way to have the most of flexibility), it works (for the things that are possible in BPF). It shares most of the work in C/kernel, I will add it in the second patch series.

Or maybe we can load BPF programs in a kernel module, or compile

Quoted 3 lines

the BPF programs into the kernel? (Yes, we can do it now, check kernel/bpf/preload). If any of these works, we can benefit from the good properties of BPF.

RV will take all these benefits, it is in the todo list as I said in this thread. But the in kernel version also has its facilities.

For example, we can update the RV models without

Quoted 5 lines

rebooting the system; and we can reuse various BPF maps, so we don't need to add union rv_task_monitor to task_struct.

Of course, we are out of luck if these systems cannot enable CONFIG_BPF at all. But I guess this is not common for modern embedded systems?

I understand your motivations, and I agree with the benefits of BPF, but I also see benefits of having it in kernel as well.

So, RV will go with both, they are not mutually exclusive.

Thanks!
-- Daniel

Quoted 2 lines

Thanks,
Song

#81 lore raw
Quoted 1 line · Daniel Bristot de Oliveira

[...]

Quoted 11 lines
Quoted 9 lines
Quoted 7 lines

The point is that there are use-cases in which the users need the code in C. One of those is the work being done in the Linux Foundation Elisa group. There will be more formalism, like timed automata... which will require infra-structure that is easily accessible in C... including synchronization, and reactors that are available only in C on "per use-cases" basis - for example on embedded devices.

Where can I find more information about the constraints of these use cases?

Check the LF elisa workgroup.

Thanks for the information. It looks interesting.

Quoted 6 lines
Quoted 5 lines

I am asking because there are multiple ways to load a BPF program to the system. If the constraint is that we cannot have bpftrace or bcc in the system, maybe it is ok to run a standalone binary (written in C, compiled on a different system).

Quoted 6 lines

as I said... *I am aware of that*. I do like BPF! I was already convinced I will having things in BPF :-)

dot2bpf does stand alone application, C + libbpf (and I did it this way to have the most of flexibility), it works (for the things that are possible in BPF). It shares most of the work in C/kernel, I will add it in the second patch series.

This is great! Looking forward to trying it out. :)

Thanks,
Song

#82 lore raw
In reply to Daniel Bristot de Oliveira

Hi Daniel,

After reading things in paper and the previous versions these days slowly from me, I choose to join the thread this time not because I understand them clearly. Sorry for not saving your email bandwidth..

Quoted 46 lines · Daniel Bristot de Oliveira

Over the last years, I've been exploring the possibility of verifying the Linux kernel behavior using Runtime Verification.

Runtime Verification (RV) is a lightweight (yet rigorous) method that complements classical exhaustive verification techniques (such as model checking and theorem proving) with a more practical approach for complex systems.

Instead of relying on a fine-grained model of a system (e.g., a re-implementation a instruction level), RV works by analyzing the trace of the system's actual execution, comparing it against a formal specification of the system behavior.

The usage of deterministic automaton for RV is a well-established approach. In the specific case of the Linux kernel, you can check how to model complex behavior of the Linux kernel with this paper:

DE OLIVEIRA, Daniel Bristot; CUCINOTTA, Tommaso; DE OLIVEIRA, Romulo Silva. *Efficient formal verification for the Linux kernel.* In: International Conference on Software Engineering and Formal Methods. Springer, Cham, 2019. p. 315-332.

And how efficient is this approach here:

DE OLIVEIRA, Daniel B.; DE OLIVEIRA, Romulo S.; CUCINOTTA, Tommaso. *A thread synchronization model for the PREEMPT_RT Linux kernel.* Journal of Systems Architecture, 2020, 107: 101729.

tlrd: it is possible to model complex behaviors in a modular way, with an acceptable overhead (even for production systems). See this presentation at 2019's ELCE: https://www.youtube.com/watch?v=BfTuEHafNgg

Here I am proposing a more practical approach for the usage of deterministic automata for runtime verification, and it includes:

- An interface for controlling the verification;
- A tool and set of headers that enables the automatic code generation of the RV monitor (Monitor Synthesis);
- Sample monitors to evaluate the interface;
- A sample monitor developed in the context of the Elisa Project demonstrating how to use RV in the context of safety-critical systems.

Given that RV is a tracing consumer, the code is being placed inside the tracing subsystem (Steven and I have been talking about it for a while).

Quoted 6 lines

Changes from v3:
- Rebased on 5.19
(rostedt's request were made on 1x1 meetings)
- Moved monitors to monitors/$name/ (Rostedt)
- Consolidate the tracepoints into a single include file in the default directory (trave/events/rv.h) (Rostedt)

s/trave\(\/events\/rv.h\)/trace\1/

Quoted 150 lines

- The tracepoints now record the entire string to the buffer.
- Change the enable_monitors to disable monitors with ! (instead of -). (Rostedt)
- Add a suffix to the state/events enums, to avoid conflict in the vmlinux.h used by eBPF.
- The models are now placed in the $name.h (it used to store the tracepoints, but they are now consolidated in a single file)
- dot2c and dot2k updated to the changes
- models re-generated with these new standards.
- user-space tools moved to an directory outside of tools/tracing as other methods of verification/log sources are planned.
Changes from v2:
- Tons of checkpatch and kernel test robot
- Moved files to better places
- Adjusted watchdog tracepoints patch (Guenter Roeck)
- Added pretimeout watchdog events (Peter Enderborg)
- Used task struct to store per-task monitors (Peter Zijlstra)
- Changed the instrumentation to use internal definition of tracepoint and check the callback signature (Steven Rostedt)
- Used printk_deferred() and removed the comment about deadlocks (Shuah Khan/John Ogness)
- Some simplifications:
- Removed the safe watchdog nowayout for now (myself)
- Removed export symbols for now (myself)
Changes from V1:
- rebased to the latest kernel;
- code cleanup;
- the watchdog dev monitor;
- safety app;

Things kept for a second moment (after this patchset):
- Add a reactor tha enables the visualization of the visited states via KCOV (Marco Elver & Dmitry Vyukov)
- Add a CRC method to check from user-space if the values exported by the monitor were not corrupted by any other kernel task (Gabriele Paoloni)
- Export symbols for external modules
- dot2bpf

Daniel Bristot de Oliveira (20):
rv: Add Runtime Verification (RV) interface
rv: Add runtime reactors interface
rv/include: Add helper functions for deterministic automata rv/include: Add deterministic automata monitor definition via C macros rv/include: Add instrumentation helper functions
tools/rv: Add dot2c
tools/rv: Add dot2k
rv/monitor: Add the wip monitor skeleton created by dot2k rv/monitor: wip instrumentation and Makefile/Kconfig entries rv/monitor: Add the wwnr monitor skeleton created by dot2k rv/monitor: wwnr instrumentation and Makefile/Kconfig entries rv/reactor: Add the printk reactor
rv/reactor: Add the panic reactor
Documentation/rv: Add a basic documentation Documentation/rv: Add deterministic automata monitor synthesis
documentation
Documentation/rv: Add deterministic automata instrumentation
documentation
watchdog/dev: Add tracepoints
rv/monitor: Add safe watchdog monitor
rv/safety_app: Add a safety_app sample
Documentation/rv: Add watchdog-monitor documentation

 Documentation/trace/index.rst                 |   1 +
 .../trace/rv/da_monitor_instrumentation.rst   | 223 ++++++
 .../trace/rv/da_monitor_synthesis.rst         | 284 +++++++
 Documentation/trace/rv/index.rst              |   9 +
 .../trace/rv/runtime-verification.rst         | 233 ++++++
 Documentation/trace/rv/watchdog-monitor.rst   | 250 ++++++
 drivers/watchdog/watchdog_dev.c               |  43 +-
 drivers/watchdog/watchdog_pretimeout.c        |   2 +
 include/linux/rv.h                            |  38 +
 include/linux/sched.h                         |  11 +
 include/linux/watchdog.h                      |   7 +-
 include/rv/automata.h                         |  49 ++
 include/rv/da_monitor.h                       | 419 ++++++++++
 include/rv/instrumentation.h                  |  23 +
 include/rv/rv.h                               |  32 +
 include/trace/events/rv.h                     | 153 ++++
 include/trace/events/watchdog.h               | 101 +++
 kernel/fork.c                                 |  14 +
 kernel/trace/Kconfig                          |   2 +
 kernel/trace/Makefile                         |   2 +
 kernel/trace/rv/Kconfig                       |  84 ++
 kernel/trace/rv/Makefile                      |   9 +
 kernel/trace/rv/monitors/safe_wtd/safe_wtd.c  | 300 +++++++
 kernel/trace/rv/monitors/safe_wtd/safe_wtd.h  |  84 ++
 kernel/trace/rv/monitors/wip/wip.c            | 110 +++
 kernel/trace/rv/monitors/wip/wip.h            |  38 +
 kernel/trace/rv/monitors/wwnr/wwnr.c          | 109 +++
 kernel/trace/rv/monitors/wwnr/wwnr.h          |  38 +
 kernel/trace/rv/reactor_panic.c               |  44 +
 kernel/trace/rv/reactor_printk.c              |  43 +
 kernel/trace/rv/rv.c                          | 757 ++++++++++++++++++
 kernel/trace/rv/rv.h                          |  54 ++
 kernel/trace/rv/rv_reactors.c                 | 476 +++++++++++
 kernel/trace/trace.c                          |   4 +
 kernel/trace/trace.h                          |   2 +
 tools/verification/dot2/Makefile              |  26 +
 tools/verification/dot2/automata.py           | 179 +++++
 tools/verification/dot2/dot2c                 |  30 +
 tools/verification/dot2/dot2c.py              | 244 ++++++
 tools/verification/dot2/dot2k                 |  50 ++
 tools/verification/dot2/dot2k.py              | 177 ++++
 .../dot2/dot2k_templates/main_global.c        |  94 +++
 .../dot2/dot2k_templates/main_per_cpu.c       |  94 +++
 .../dot2/dot2k_templates/main_per_task.c      |  94 +++
 tools/verification/safety_app/Makefile        |  51 ++
 tools/verification/safety_app/safety_app.c    | 614 ++++++++++++++
 46 files changed, 5691 insertions(+), 10 deletions(-)
 create mode 100644 Documentation/trace/rv/da_monitor_instrumentation.rst
 create mode 100644 Documentation/trace/rv/da_monitor_synthesis.rst
 create mode 100644 Documentation/trace/rv/index.rst
 create mode 100644 Documentation/trace/rv/runtime-verification.rst
 create mode 100644 Documentation/trace/rv/watchdog-monitor.rst
 create mode 100644 include/linux/rv.h
 create mode 100644 include/rv/automata.h
 create mode 100644 include/rv/da_monitor.h
 create mode 100644 include/rv/instrumentation.h
 create mode 100644 include/rv/rv.h
 create mode 100644 include/trace/events/rv.h
 create mode 100644 include/trace/events/watchdog.h
 create mode 100644 kernel/trace/rv/Kconfig
 create mode 100644 kernel/trace/rv/Makefile
 create mode 100644 kernel/trace/rv/monitors/safe_wtd/safe_wtd.c
 create mode 100644 kernel/trace/rv/monitors/safe_wtd/safe_wtd.h
 create mode 100644 kernel/trace/rv/monitors/wip/wip.c
 create mode 100644 kernel/trace/rv/monitors/wip/wip.h
 create mode 100644 kernel/trace/rv/monitors/wwnr/wwnr.c
 create mode 100644 kernel/trace/rv/monitors/wwnr/wwnr.h
 create mode 100644 kernel/trace/rv/reactor_panic.c
 create mode 100644 kernel/trace/rv/reactor_printk.c
 create mode 100644 kernel/trace/rv/rv.c
 create mode 100644 kernel/trace/rv/rv.h
 create mode 100644 kernel/trace/rv/rv_reactors.c
 create mode 100644 tools/verification/dot2/Makefile
 create mode 100644 tools/verification/dot2/automata.py
 create mode 100644 tools/verification/dot2/dot2c
 create mode 100644 tools/verification/dot2/dot2c.py
 create mode 100644 tools/verification/dot2/dot2k
 create mode 100644 tools/verification/dot2/dot2k.py
 create mode 100644 tools/verification/dot2/dot2k_templates/main_global.c
 create mode 100644 tools/verification/dot2/dot2k_templates/main_per_cpu.c
 create mode 100644 tools/verification/dot2/dot2k_templates/main_per_task.c
 create mode 100644 tools/verification/safety_app/Makefile
 create mode 100644 tools/verification/safety_app/safety_app.c
Signature · 1 line
2.35.1