Automate manual steps in monitor creation
Subsystems (2)
Gabriele Monaco
maintainercover Automate manual steps in monitor creation
While creating a new monitor in RV, besides generating code from dot2k, there are a few manual steps which can be tedious and error prone, like adding the tracepoints, makefile lines and kconfig, or selecting events that start the monitor in the initial state.
This patchset tries to automate as much as possible among those steps to make creating a new RV monitor much quicker. It is still required to select proper tracepoints, this step is harder to automate in a general way and, in several cases, would still need user intervention.
The first 3 patches are preparatory work around the templates.
Patch 1 fixes the template directory used if dot2k is not installed but is run from the kernel tree (tools/verification/).
Patch 2 unifies the templates for the 3 monitor types to change them easier.
Patch 3 makes the format used for variables replaced by the dot2k script a bit more robust.
Patch 4 implements the already present -n and -D flags to set the monitor name and description, respectively.
Patch 5 moves most of the monitor-specific code to the monitor directory (tracepoints, Kconfig). The user still needs to manually fill some content but the copy-paste-change is reduced.
Patch 6 adapts dot2k to produce monitors as defined in patch 5, adding the necessary templates. Additionally, while printing the checklist, dot2k also suggests what lines to add and where.
Patch 7 adds the possibility to further automate what is done in patch 5 and 6. If dot2k finds a valid kernel source and the -a flag is passed, it automatically changes the files accordingly (RV Makefile, tracepoints and Kconfig) and moves the new monitor folder in the final position.
Patch 8 allows to automatically select events that start the monitor, to achieve this, the automata parser can now understand if there are events that always lead to the initial state or are only valid in the initial state. dot2k uses da_handle_start_event with the former and, as a fallback, da_handle_start_run_event for the latter.
Gabriele Monaco (8):
verification/dot2k: Fix template directory detection verification/dot2k: Unify main.c templates verification/dot2k: More robust template variables verification/dot2k: Add support for name and description options
rv: Simplify manual steps in monitor creation verification/dot2k: Simplify manual steps in monitor creation verification/dot2k: Auto patch current kernel source verification/dot2k: Implement event type detection
kernel/trace/rv/Kconfig | 27 +-- kernel/trace/rv/Makefile | 3 + kernel/trace/rv/monitors/wip/Kconfig | 12 + kernel/trace/rv/monitors/wip/wip.c | 2 +- kernel/trace/rv/monitors/wip/wip_trace.h | 15 ++ kernel/trace/rv/monitors/wwnr/Kconfig | 11 + kernel/trace/rv/monitors/wwnr/wwnr.c | 2 +- kernel/trace/rv/monitors/wwnr/wwnr_trace.h | 16 ++ kernel/trace/rv/rv.c | 2 +- .../events/rv.h => kernel/trace/rv/rv_trace.h | 26 +- tools/verification/dot2/automata.py | 36 ++- tools/verification/dot2/dot2c.py | 4 +- tools/verification/dot2/dot2k | 17 +- tools/verification/dot2/dot2k.py | 226 +++++++++++++++--- .../verification/dot2/dot2k_templates/Kconfig | 6 + .../verification/dot2/dot2k_templates/main.c | 91 +++++++ .../dot2/dot2k_templates/main_global.c | 91 ------- .../dot2/dot2k_templates/main_per_cpu.c | 91 ------- .../dot2/dot2k_templates/main_per_task.c | 91 ------- .../verification/dot2/dot2k_templates/trace.h | 13 + 20 files changed, 419 insertions(+), 363 deletions(-) create mode 100644 kernel/trace/rv/monitors/wip/Kconfig create mode 100644 kernel/trace/rv/monitors/wip/wip_trace.h create mode 100644 kernel/trace/rv/monitors/wwnr/Kconfig create mode 100644 kernel/trace/rv/monitors/wwnr/wwnr_trace.h rename include/trace/events/rv.h => kernel/trace/rv/rv_trace.h (79%) create mode 100644 tools/verification/dot2/dot2k_templates/Kconfig create mode 100644 tools/verification/dot2/dot2k_templates/main.c delete mode 100644 tools/verification/dot2/dot2k_templates/main_global.c delete mode 100644 tools/verification/dot2/dot2k_templates/main_per_cpu.c delete mode 100644 tools/verification/dot2/dot2k_templates/main_per_task.c create mode 100644 tools/verification/dot2/dot2k_templates/trace.h
base-commit: aef25be35d23ec768eed08bfcf7ca3cf9685bc28
Signature · 1 line
2.47.1
Gabriele Monaco
maintainerPATCH 1/8 verification/dot2k: Fix template directory detection
dot2k can be run as installed (e.g. make install) or from the kernel tree. In the former case it looks for templates in a known location; in the latter, the PWD has to be <linux>/tools/verification to properly import python modules. The current version looks for the template in a wrong directory in this latter case.
This patch adjusts the directory where dot2k looks for templates if run from the kernel tree (i.e. not installed).
Additionally we fix a few simple pylint warnings in boolean expressions.
--- tools/verification/dot2/dot2k.py | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
tools/verification/dot2/dot2k.py +8 -8
@@ -14,14 +14,14 @@ import os class dot2k(Dot2c): monitor_types = { "global" : 1, "per_cpu" : 2, "per_task" : 3 } - monitor_templates_dir = "dot2k/rv_templates/" + monitor_templates_dir = "dot2/dot2k_templates/" monitor_type = "per_cpu" def __init__(self, file_path, MonitorType): super().__init__(file_path) self.monitor_type = self.monitor_types.get(MonitorType) - if self.monitor_type == None: + if self.monitor_type is None: raise Exception("Unknown monitor type: %s" % MonitorType) self.monitor_type = MonitorType
@@ -31,7 +31,7 @@ class dot2k(Dot2c): def __fill_rv_templates_dir(self): - if os.path.exists(self.monitor_templates_dir) == True: + if os.path.exists(self.monitor_templates_dir): return if platform.system() != "Linux":
@@ -39,11 +39,11 @@ class dot2k(Dot2c): kernel_path = "/lib/modules/%s/build/tools/verification/dot2/dot2k_templates/" % (platform.release()) - if os.path.exists(kernel_path) == True: + if os.path.exists(kernel_path): self.monitor_templates_dir = kernel_path return - if os.path.exists("/usr/share/dot2/dot2k_templates/") == True: + if os.path.exists("/usr/share/dot2/dot2k_templates/"): self.monitor_templates_dir = "/usr/share/dot2/dot2k_templates/" return
@@ -98,7 +98,7 @@ class dot2k(Dot2c): def fill_main_c(self): main_c = self.main_c min_type = self.get_minimun_type() - nr_events = self.events.__len__() + nr_events = len(self.events) tracepoint_handlers = self.fill_tracepoint_handlers_skel() tracepoint_attach = self.fill_tracepoint_attach_probe() tracepoint_detach = self.fill_tracepoint_detach_helper()
@@ -160,8 +160,8 @@ class dot2k(Dot2c): def __get_main_name(self): path = "%s/%s" % (self.name, "main.c") - if os.path.exists(path) == False: - return "main.c" + if not os.path.exists(path): + return "main.c" return "__main.c" def print_files(self):
Signature · 1 line
2.47.1
Gabriele Monaco
maintainerPATCH 2/8 verification/dot2k: Unify main.c templates
In reply to Gabriele Monacodot2k has 3 templates, one per monitor type, but the only difference among them is the DECLARE_DA_MON_* call, keeping 3 almost identical templates requires more work whenever we introduce a change.
This patch removes the 3 dot2k templates and replaces them with a generic one, we then adjust the model type from the script.
---
tools/verification/dot2/dot2k.py | 7 +-
.../dot2k_templates/{main_global.c => main.c} | 2 +-
.../dot2/dot2k_templates/main_per_cpu.c | 91 -------------------
.../dot2/dot2k_templates/main_per_task.c | 91 -------------------
4 files changed, 7 insertions(+), 184 deletions(-)
rename tools/verification/dot2/dot2k_templates/{main_global.c => main.c} (97%)
delete mode 100644 tools/verification/dot2/dot2k_templates/main_per_cpu.c
delete mode 100644 tools/verification/dot2/dot2k_templates/main_per_task.ctools/verification/dot2/dot2k.py +6 -1
@@ -26,7 +26,7 @@ class dot2k(Dot2c): self.monitor_type = MonitorType self.__fill_rv_templates_dir() - self.main_c = self.__open_file(self.monitor_templates_dir + "main_" + MonitorType + ".c") + self.main_c = self.__open_file(self.monitor_templates_dir + "main.c") self.enum_suffix = "_%s" % self.name def __fill_rv_templates_dir(self):
@@ -69,6 +69,9 @@ class dot2k(Dot2c): # cut off the last \n return string[:-1] + def fill_monitor_type(self): + return self.monitor_type.upper() + def fill_tracepoint_handlers_skel(self): buff = [] for event in self.events:
@@ -97,12 +100,14 @@ class dot2k(Dot2c): def fill_main_c(self): main_c = self.main_c + monitor_type = self.fill_monitor_type() min_type = self.get_minimun_type() nr_events = len(self.events) tracepoint_handlers = self.fill_tracepoint_handlers_skel() tracepoint_attach = self.fill_tracepoint_attach_probe() tracepoint_detach = self.fill_tracepoint_detach_helper() + main_c = main_c.replace("MONITOR_TYPE", monitor_type) main_c = main_c.replace("MIN_TYPE", min_type) main_c = main_c.replace("MODEL_NAME", self.name) main_c = main_c.replace("NR_EVENTS", str(nr_events))
tools/verification/dot2/dot2k_templates/main.c renamed from tools/verification/dot2/dot2k_templates/main_global.c +1 -1
@@ -28,7 +28,7 @@* The rv monitor reference is needed for the monitor declaration. */ static struct rv_monitor rv_MODEL_NAME; -DECLARE_DA_MON_GLOBAL (MODEL_NAME ,MIN_TYPE ); +DECLARE_DA_MON_MONITOR_TYPE (MODEL_NAME ,MIN_TYPE );/* * This is the instrumentation part of the monitor.
tools/verification/dot2/dot2k_templates/main_per_cpu.c deleted +0 -91
91 lines not on this page · raw message
tools/verification/dot2/dot2k_templates/main_per_task.c deleted +0 -91
91 lines not on this page · raw message
Signature · 1 line
2.47.1
Gabriele Monaco
maintainerPATCH 3/8 verification/dot2k: More robust template variables
In reply to Gabriele MonacoThe dot2k templates currently have variables that are automatically filled by the script marked as an uppercase VARIABLE. This requires some care while adding new variables to avoid using valid keywords and get them unexpectedly substituted.
This patch switches the variables to the %%VARIABLE%% notation to make the pattern substitution more robust.
--- tools/verification/dot2/dot2k.py | 14 +++--- .../verification/dot2/dot2k_templates/main.c | 50 +++++++++---------- 2 files changed, 32 insertions(+), 32 deletions(-)
tools/verification/dot2/dot2k.py +7 -7
@@ -107,13 +107,13 @@ class dot2k(Dot2c): tracepoint_attach = self.fill_tracepoint_attach_probe() tracepoint_detach = self.fill_tracepoint_detach_helper() - main_c = main_c.replace("MONITOR_TYPE", monitor_type) - main_c = main_c.replace("MIN_TYPE", min_type) - main_c = main_c.replace("MODEL_NAME", self.name) - main_c = main_c.replace("NR_EVENTS", str(nr_events)) - main_c = main_c.replace("TRACEPOINT_HANDLERS_SKEL", tracepoint_handlers) - main_c = main_c.replace("TRACEPOINT_ATTACH", tracepoint_attach) - main_c = main_c.replace("TRACEPOINT_DETACH", tracepoint_detach) + main_c = main_c.replace("%%MONITOR_TYPE%%", monitor_type) + main_c = main_c.replace("%%MIN_TYPE%%", min_type) + main_c = main_c.replace("%%MODEL_NAME%%", self.name) + main_c = main_c.replace("%%NR_EVENTS%%", str(nr_events)) + main_c = main_c.replace("%%TRACEPOINT_HANDLERS_SKEL%%", tracepoint_handlers) + main_c = main_c.replace("%%TRACEPOINT_ATTACH%%", tracepoint_attach) + main_c = main_c.replace("%%TRACEPOINT_DETACH%%", tracepoint_detach) return main_c
tools/verification/dot2/dot2k_templates/main.c +25 -25
102 lines not on this page · raw message
Signature · 1 line
2.47.1
Gabriele Monaco
maintainerPATCH 4/8 verification/dot2k: Add support for name and description options
In reply to Gabriele MonacoThe dot2k command includes options to set a model name with -n and a description with -D, however those are not used in practice.
This patch allows to specify a custom model name (by default the name of the dot file without extension) and a description which overrides the one in the C file.
--- tools/verification/dot2/automata.py | 4 ++-- tools/verification/dot2/dot2c.py | 4 ++-- tools/verification/dot2/dot2k | 6 +----- tools/verification/dot2/dot2k.py | 8 +++++--- tools/verification/dot2/dot2k_templates/main.c | 4 ++-- 5 files changed, 12 insertions(+), 14 deletions(-)
tools/verification/dot2/automata.py +2 -2
@@ -19,9 +19,9 @@ class Automata: invalid_state_str = "INVALID_STATE" - def __init__(self, file_path): + def __init__(self, file_path, model_name=None): self.__dot_path = file_path - self.name = self.__get_model_name() + self.name = model_name or self.__get_model_name() self.__dot_lines = self.__open_dot() self.states, self.initial_state, self.final_states = self.__get_state_variables() self.events = self.__get_event_variables()
tools/verification/dot2/dot2c.py +2 -2
@@ -22,8 +22,8 @@ class Dot2c(Automata): struct_automaton_def = "automaton" var_automaton_def = "aut" - def __init__(self, file_path): - super().__init__(file_path) + def __init__(self, file_path, model_name=None): + super().__init__(file_path, model_name) self.line_length = 100 def __buff_to_string(self, buff):
tools/verification/dot2/dot2k +1 -5
@@ -25,16 +25,12 @@ if __name__ == '__main__': print("Opening and parsing the dot file %s" % params.dot_file) try: - monitor=dot2k(params.dot_file, params.monitor_type) + monitor=dot2k(params.dot_file, params.monitor_type, vars(params)) except Exception as e: print('Error: '+ str(e)) print("Sorry : :-(") sys.exit(1) - # easier than using argparse action. - if params.model_name != None: - print(params.model_name) - print("Writing the monitor into the directory %s" % monitor.name) monitor.print_files() print("Almost done, checklist")
tools/verification/dot2/dot2k.py +5 -3
@@ -17,17 +17,18 @@ class dot2k(Dot2c): monitor_templates_dir = "dot2/dot2k_templates/" monitor_type = "per_cpu" - def __init__(self, file_path, MonitorType): - super().__init__(file_path) + def __init__(self, file_path, MonitorType, extra_params={}): + super().__init__(file_path, extra_params.get("model_name")) self.monitor_type = self.monitor_types.get(MonitorType) if self.monitor_type is None: - raise Exception("Unknown monitor type: %s" % MonitorType) + raise ValueError("Unknown monitor type: %s" % MonitorType) self.monitor_type = MonitorType self.__fill_rv_templates_dir() self.main_c = self.__open_file(self.monitor_templates_dir + "main.c") self.enum_suffix = "_%s" % self.name + self.description = extra_params.get("description", self.name) or "auto-generated" def __fill_rv_templates_dir(self):
@@ -114,6 +115,7 @@ class dot2k(Dot2c): main_c = main_c.replace("%%TRACEPOINT_HANDLERS_SKEL%%", tracepoint_handlers) main_c = main_c.replace("%%TRACEPOINT_ATTACH%%", tracepoint_attach) main_c = main_c.replace("%%TRACEPOINT_DETACH%%", tracepoint_detach) + main_c = main_c.replace("%%DESCRIPTION%%", self.description) return main_c
tools/verification/dot2/dot2k_templates/main.c +2 -2
@@ -65,7 +65,7 @@ static void disable_%%MODEL_NAME%%(void)*/ static struct rv_monitor rv_%%MODEL_NAME%% = { .name ="%%MODEL_NAME%%" , - .description ="auto-generated %%MODEL_NAME%%" , + .description ="%%DESCRIPTION%%" , .enable = enable_%%MODEL_NAME%%, .disable = disable_%%MODEL_NAME%%, .reset = da_monitor_reset_all_%%MODEL_NAME%%,
@@ -88,4 +88,4 @@ module_exit(unregister_%%MODEL_NAME%%);MODULE_LICENSE ("GPL" );MODULE_AUTHOR ("dot2k: auto-generated" ); -MODULE_DESCRIPTION ("%%MODEL_NAME%%" ); +MODULE_DESCRIPTION ("%%MODEL_NAME%%: %%DESCRIPTION%%" );
Signature · 1 line
2.47.1
Gabriele Monaco
maintainerPATCH 5/8 Simplify manual steps in monitor creation
In reply to Gabriele MonacoWhile creating a new monitor in RV, besides generating code from dot2k, there are a few manual steps which can be tedious and error prone, like adding the tracepoints, makefile lines and kconfig.
This patch restructures the existing monitors to keep some files in the monitor's folder itself, which can be automatically generated by future versions of dot2k.
Monitors have now their own Kconfig and tracepoint snippets. For simplicity, the main tracepoint definition, is moved to the RV directory, it defines only the tracepoint classes and includes the monitor-specific tracepoints, which reside in the monitor directory.
Tracepoints and Kconfig no longer need to be copied and adapted from existing ones but only need to be included in the main files. The Makefile remains untouched since there's little advantage in having a separated Makefile for each monitor with a single line and including it in the main RV Makefile.
--- kernel/trace/rv/Kconfig | 25 ++----------------- kernel/trace/rv/Makefile | 2 ++ kernel/trace/rv/monitors/wip/Kconfig | 12 +++++++++ kernel/trace/rv/monitors/wip/wip.c | 2 +- kernel/trace/rv/monitors/wip/wip_trace.h | 15 +++++++++++ kernel/trace/rv/monitors/wwnr/Kconfig | 11 ++++++++ kernel/trace/rv/monitors/wwnr/wwnr.c | 2 +- kernel/trace/rv/monitors/wwnr/wwnr_trace.h | 16 ++++++++++++ kernel/trace/rv/rv.c | 2 +- .../events/rv.h => kernel/trace/rv/rv_trace.h | 24 ++++-------------- 10 files changed, 66 insertions(+), 45 deletions(-) create mode 100644 kernel/trace/rv/monitors/wip/Kconfig create mode 100644 kernel/trace/rv/monitors/wip/wip_trace.h create mode 100644 kernel/trace/rv/monitors/wwnr/Kconfig create mode 100644 kernel/trace/rv/monitors/wwnr/wwnr_trace.h rename include/trace/events/rv.h => kernel/trace/rv/rv_trace.h (79%)
kernel/trace/rv/Kconfig +2 -23
@@ -25,30 +25,9 @@ menuconfig RV For further information, see: Documentation/trace/rv/runtime-verification.rst -config RV_MON_WIP - depends on RV - depends on PREEMPT_TRACER - select DA_MON_EVENTS_IMPLICIT - bool "wip monitor" - help - Enable wip (wakeup in preemptive) sample monitor that illustrates - the usage of per-cpu monitors, and one limitation of the - preempt_disable/enable events. - - For further information, see: - Documentation/trace/rv/monitor_wip.rst +source "kernel/trace/rv/monitors/wip/Kconfig" -config RV_MON_WWNR - depends on RV - select DA_MON_EVENTS_ID - bool "wwnr monitor" - help - Enable wwnr (wakeup while not running) sample monitor, this is a - sample monitor that illustrates the usage of per-task monitor. - The model is borken on purpose: it serves to test reactors. - - For further information, see: - Documentation/trace/rv/monitor_wwnr.rst +source "kernel/trace/rv/monitors/wwnr/Kconfig" config RV_REACTORS bool "Runtime verification reactors"
kernel/trace/rv/Makefile +2 -0
@@ -1,5 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 +ccflags-y += -I $(src) # needed for trace events + obj-$(CONFIG_RV) += rv.o obj-$(CONFIG_RV_MON_WIP) += monitors/wip/wip.o obj-$(CONFIG_RV_MON_WWNR) += monitors/wwnr/wwnr.o
kernel/trace/rv/monitors/wip/Kconfig new file +12 -0
@@ -0,0 +1,12 @@ +config RV_MON_WIP + depends on RV + depends on PREEMPT_TRACER + select DA_MON_EVENTS_IMPLICIT + bool "wip monitor" + help + Enable wip (wakeup in preemptive) sample monitor that illustrates + the usage of per-cpu monitors, and one limitation of the + preempt_disable/enable events. + + For further information, see: + Documentation/trace/rv/monitor_wip.rst
kernel/trace/rv/monitors/wip/wip.c +1 -1
@@ -10,7 +10,7 @@#define MODULE_NAME "wip" -#include <trace/events/rv.h> +#include <rv_trace.h> #include <trace/events/sched.h> #include <trace/events/preemptirq.h>
kernel/trace/rv/monitors/wip/wip_trace.h new file +15 -0
@@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* +* Snippet to be included in rv_trace.h +*/ + +#ifdef CONFIG_RV_MON_WIP +DEFINE_EVENT (event_da_monitor, event_wip, + TP_PROTO(char *state, char *event, char *next_state, bool final_state), +TP_ARGS (state, event, next_state, final_state)); + +DEFINE_EVENT (error_da_monitor, error_wip, +TP_PROTO (char *state, char *event), +TP_ARGS (state, event)); +#endif /* CONFIG_RV_MON_WIP */
kernel/trace/rv/monitors/wwnr/Kconfig new file +11 -0
@@ -0,0 +1,11 @@ +config RV_MON_WWNR + depends on RV + select DA_MON_EVENTS_ID + bool "wwnr monitor" + help + Enable wwnr (wakeup while not running) sample monitor, this is a + sample monitor that illustrates the usage of per-task monitor. + The model is borken on purpose: it serves to test reactors. + + For further information, see: + Documentation/trace/rv/monitor_wwnr.rst
kernel/trace/rv/monitors/wwnr/wwnr.c +1 -1
@@ -10,7 +10,7 @@#define MODULE_NAME "wwnr" -#include <trace/events/rv.h> +#include <rv_trace.h> #include <trace/events/sched.h> #include "wwnr.h"
kernel/trace/rv/monitors/wwnr/wwnr_trace.h new file +16 -0
@@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* +* Snippet to be included in rv_trace.h +*/ + +#ifdef CONFIG_RV_MON_WWNR +/* id is the pid of the task */ +DEFINE_EVENT (event_da_monitor_id, event_wwnr, + TP_PROTO(int id, char *state, char *event, char *next_state, bool final_state), +TP_ARGS (id, state, event, next_state, final_state)); + +DEFINE_EVENT (error_da_monitor_id, error_wwnr, + TP_PROTO(int id, char *state, char *event), +TP_ARGS (id, state, event)); +#endif /* CONFIG_RV_MON_WWNR */
kernel/trace/rv/rv.c +1 -1
@@ -145,7 +145,7 @@#ifdef CONFIG_DA_MON_EVENTS #define CREATE_TRACE_POINTS -#include <trace/events/rv.h> +#include <rv_trace.h> #endif #include "rv.h"
kernel/trace/rv/rv_trace.h renamed from include/trace/events/rv.h +5 -19
@@ -57,15 +57,8 @@ DECLARE_EVENT_CLASS(error_da_monitor, __entry->state) ); -#ifdef CONFIG_RV_MON_WIP -DEFINE_EVENT (event_da_monitor, event_wip, - TP_PROTO(char *state, char *event, char *next_state, bool final_state), -TP_ARGS (state, event, next_state, final_state)); - -DEFINE_EVENT (error_da_monitor, error_wip, -TP_PROTO (char *state, char *event), -TP_ARGS (state, event)); -#endif /* CONFIG_RV_MON_WIP */ +#include <monitors/wip/wip_trace.h> +#endif /* CONFIG_DA_MON_EVENTS_IMPLICIT */ #ifdef CONFIG_DA_MON_EVENTS_ID
@@ -123,20 +116,13 @@ DECLARE_EVENT_CLASS(error_da_monitor_id, __entry->state) ); -#ifdef CONFIG_RV_MON_WWNR -/* id is the pid of the task */ -DEFINE_EVENT (event_da_monitor_id, event_wwnr, - TP_PROTO(int id, char *state, char *event, char *next_state, bool final_state), -TP_ARGS (id, state, event, next_state, final_state)); - -DEFINE_EVENT (error_da_monitor_id, error_wwnr, - TP_PROTO(int id, char *state, char *event), -TP_ARGS (id, state, event)); -#endif /* CONFIG_RV_MON_WWNR */ +#include <monitors/wwnr/wwnr_trace.h> #endif /* CONFIG_DA_MON_EVENTS_ID */ #endif /* _TRACE_RV_H */ /* This part ust be outside protection */ #undef TRACE_INCLUDE_PATH +#define TRACE_INCLUDE_PATH . +#define TRACE_INCLUDE_FILE rv_trace#include <trace/define_trace.h>
Signature · 1 line
2.47.1
Gabriele Monaco
maintainerPATCH 6/8 verification/dot2k: Simplify manual steps in monitor creation
In reply to Gabriele MonacoThis patch reduces and simplifies the manual steps still needed in creating a new RV monitor.
It extends the dot2k script to create a tracepoint snippet and a Kconfig file for the newly generated monitor. Those files can be kept in the monitor's directory but shall be included in the main tracepoint header and Kconfig.
Together with the checklist, dot2k now suggests the lines to add to those files for inclusion and the Makefile line to compile the new monitor:
Writing the monitor into the directory monitor_name
Almost done, checklist
- Edit the monitor_name/monitor_name.c to add the instrumentation
- Edit kernel/trace/rv/rv_trace.h:
Add this line where other tracepoints are included and DA_MON_EVENTS_ID is defined:
#include <monitors/monitor_name/monitor_name_trace.h>
- Edit kernel/trace/rv/Makefile:
Add this line where other monitors are included:
obj-$(CONFIG_RV_MON_MONITOR_NAME) += monitors/monitor_name/monitor_name.o
- Edit kernel/trace/rv/Kconfig:
Add this line where other monitors are included:
source "kernel/trace/rv/monitors/monitor_name/Kconfig"
- Move monitor_name/ to the kernel's monitor directory (kernel/trace/rv/monitors)
--- tools/verification/dot2/dot2k | 8 +- tools/verification/dot2/dot2k.py | 86 +++++++++++++++++++ .../verification/dot2/dot2k_templates/Kconfig | 6 ++ .../verification/dot2/dot2k_templates/main.c | 2 +- .../verification/dot2/dot2k_templates/trace.h | 13 +++ 5 files changed, 110 insertions(+), 5 deletions(-) create mode 100644 tools/verification/dot2/dot2k_templates/Kconfig create mode 100644 tools/verification/dot2/dot2k_templates/trace.h
tools/verification/dot2/dot2k +4 -4
@@ -35,7 +35,7 @@ if __name__ == '__main__': monitor.print_files() print("Almost done, checklist") print(" - Edit the %s/%s.c to add the instrumentation" % (monitor.name, monitor.name)) - print(" - Edit include/trace/events/rv.h to add the tracepoint entry") - print(" - Move it to the kernel's monitor directory") - print(" - Edit kernel/trace/rv/Makefile") - print(" - Edit kernel/trace/rv/Kconfig") + print(monitor.fill_tracepoint_tooltip()) + print(monitor.fill_makefile_tooltip()) + print(monitor.fill_kconfig_tooltip()) + print(" - Move %s/ to the kernel's monitor directory (%s/monitors)" % (monitor.name, monitor.rv_dir))
tools/verification/dot2/dot2k.py +86 -0
107 lines not on this page · raw message
tools/verification/dot2/dot2k_templates/Kconfig new file +6 -0
@@ -0,0 +1,6 @@ +config RV_MON_%%MODEL_NAME_UP%% + depends on RV + select %%MONITOR_CLASS_TYPE%% + bool "%%MODEL_NAME%% monitor" + help + %%DESCRIPTION%%
tools/verification/dot2/dot2k_templates/main.c +1 -1
@@ -14,7 +14,7 @@* XXX: include required tracepoint headers, e.g., * #include <trace/events/sched.h> */ -#include <trace/events/rv.h> +#include <rv_trace.h> /* * This is the self-generated part of the monitor. Generally, there is no need
tools/verification/dot2/dot2k_templates/trace.h new file +13 -0
@@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* +* Snippet to be included in rv_trace.h +*/ + +#ifdef CONFIG_RV_MON_%%MODEL_NAME_UP%% +DEFINE_EVENT(event_%%MONITOR_CLASS%%, event_%%MODEL_NAME%%, +%%TRACEPOINT_ARGS_SKEL_EVENT%%); + +DEFINE_EVENT(error_%%MONITOR_CLASS%%, error_%%MODEL_NAME%%, +%%TRACEPOINT_ARGS_SKEL_ERROR%%); +#endif /* CONFIG_RV_MON_%%MODEL_NAME_UP%% */
Signature · 1 line
2.47.1
Gabriele Monaco
maintainerPATCH 7/8 verification/dot2k: Auto patch current kernel source
In reply to Gabriele Monacodot2k suggests a list of changes to the kernel tree while adding a
monitor: edit tracepoints header, Makefile, Kconfig and moving the monitor folder. Those changes can be easily run automatically.
This patch adds a flag to dot2k to alter the kernel source.
The kernel source directory can be either assumed from the PWD, or from the running kernel, if installed.
This feature works best if the kernel tree is a git repository, so that its easier to make sure there are no unintended changes.
The main RV files (e.g. Makefile) have now a comment placeholder that can be useful for manual editing (e.g. to know where to add new monitors) and it is used by the script to append the required lines.
We also slightly adapt the file handling functions in dot2k: __open_file is now called __read_file and also closes the file before returning the content; __create_file is now a more general __write_file, we no longer return on FileExistsError (not thrown while opening), a new __create_file simply calls __write_file specifying the monitor folder in the path.
--- kernel/trace/rv/Kconfig | 2 +- kernel/trace/rv/Makefile | 1 + kernel/trace/rv/rv_trace.h | 2 + tools/verification/dot2/dot2k | 5 +- tools/verification/dot2/dot2k.py | 92 +++++++++++++++++++++++++++----- 5 files changed, 86 insertions(+), 16 deletions(-)
kernel/trace/rv/Kconfig +1 -1
@@ -26,8 +26,8 @@ menuconfig RV Documentation/trace/rv/runtime-verification.rst source "kernel/trace/rv/monitors/wip/Kconfig" - source "kernel/trace/rv/monitors/wwnr/Kconfig" +# Add new monitors here config RV_REACTORS bool "Runtime verification reactors"
kernel/trace/rv/Makefile +1 -0
@@ -5,6 +5,7 @@ ccflags-y += -I $(src) # needed for trace events obj-$(CONFIG_RV) += rv.o obj-$(CONFIG_RV_MON_WIP) += monitors/wip/wip.o obj-$(CONFIG_RV_MON_WWNR) += monitors/wwnr/wwnr.o +# Add new monitors here obj-$(CONFIG_RV_REACTORS) += rv_reactors.o obj-$(CONFIG_RV_REACT_PRINTK) += reactor_printk.o obj-$(CONFIG_RV_REACT_PANIC) += reactor_panic.o
kernel/trace/rv/rv_trace.h +2 -0
@@ -58,6 +58,7 @@ DECLARE_EVENT_CLASS(error_da_monitor, );#include <monitors/wip/wip_trace.h> +// Add new monitors based on CONFIG_DA_MON_EVENTS_IMPLICIT here #endif /* CONFIG_DA_MON_EVENTS_IMPLICIT */
@@ -117,6 +118,7 @@ DECLARE_EVENT_CLASS(error_da_monitor_id, );#include <monitors/wwnr/wwnr_trace.h> +// Add new monitors based on CONFIG_DA_MON_EVENTS_ID here #endif /* CONFIG_DA_MON_EVENTS_ID */ #endif /* _TRACE_RV_H */
tools/verification/dot2/dot2k +4 -1
@@ -21,6 +21,9 @@ if __name__ == '__main__': parser.add_argument('-t', "--monitor_type", dest="monitor_type", required=True) parser.add_argument('-n', "--model_name", dest="model_name", required=False) parser.add_argument("-D", "--description", dest="description", required=False) + parser.add_argument("-a", "--auto_patch", dest="auto_patch", + action="store_true", required=False, + help="Patch the kernel in place") params = parser.parse_args() print("Opening and parsing the dot file %s" % params.dot_file)
@@ -38,4 +41,4 @@ if __name__ == '__main__': print(monitor.fill_tracepoint_tooltip()) print(monitor.fill_makefile_tooltip()) print(monitor.fill_kconfig_tooltip()) - print(" - Move %s/ to the kernel's monitor directory (%s/monitors)" % (monitor.name, monitor.rv_dir)) + print(monitor.fill_monitor_tooltip())
tools/verification/dot2/dot2k.py +78 -14
159 lines not on this page · raw message
Signature · 1 line
2.47.1
Steven Rostedt
maintainerRe: [PATCH 7/8] verification/dot2k: Auto patch current kernel source
Quoted 5 lines · On Fri, 27 Dec 2024 15:47:51 +0100 Gabriele Monaco <[email protected]> wrote:
dot2k suggests a list of changes to the kernel tree while adding a
monitor: edit tracepoints header,Makefile,Kconfigand moving the monitor folder. Those changes can be easily run automatically.This patch adds a flag to dot2k to alter the kernel source.
I'll pull in these patches for linux-next, but the above should be written as:
Add a flag to dot2k to alter the kernel source.
I made the update. This is for future reference. Never use "This patch" or "This commit" in a change log.
Thanks,
-- Steve
Quoted 18 lines
The kernel source directory can be either assumed from the PWD, or from the running kernel, if installed.
This feature works best if the kernel tree is a git repository, so that its easier to make sure there are no unintended changes.The main RV files (e.g.
Makefile) have now a comment placeholder that can be useful for manual editing (e.g. to know where to add new monitors) and it is used by the script to append the required lines.We also slightly adapt the file handling functions in dot2k:
__open_fileis now called__read_fileand also closes the file before returning the content;__create_fileis now a more general__write_file, we no longer return on FileExistsError (not thrown while opening), a new__create_filesimply calls__write_filespecifying the monitor folder in the path.Signed-off-by Gabriele Monaco <[email protected]>
Gabriele Monaco
maintainerRe: [PATCH 7/8] verification/dot2k: Auto patch current kernel source
Il 27 dicembre 2024 19:44:40 UTC, Steven Rostedt <[email protected]> ha scritto:
Quoted 12 lines
Quoted 5 lines · On Fri, 27 Dec 2024 15:47:51 +0100 Gabriele Monaco <[email protected]> wrote:
dot2k suggests a list of changes to the kernel tree while adding a
monitor: edit tracepoints header,Makefile,Kconfigand moving the monitor folder. Those changes can be easily run automatically.This patch adds a flag to dot2k to alter the kernel source.
I'll pull in these patches for linux-next, but the above should be written as:
Quoted 9 lines
Add a flag to dot2k to alter the kernel source.
I made the update. This is for future reference. Never use "This patch" or "This commit" in a change log.
Thanks,
-- Steve
Alright, thanks for the note and the fast reply!
Cheers,
Gabriele
Quoted 11 lines
Quoted 11 lines
The kernel source directory can be either assumed from the PWD, or from the running kernel, if installed.
This feature works best if the kernel tree is a git repository, so that its easier to make sure there are no unintended changes.The main RV files (e.g.
Makefile) have now a comment placeholder that can be useful for manual editing (e.g. to know where to add new monitors) and it is used by the script to append the required lines.
Quoted 9 lines
Quoted 8 lines
We also slightly adapt the file handling functions in dot2k:
__open_fileis now called__read_fileand also closes the file before returning the content;__create_fileis now a more general__write_file, we no longer return on FileExistsError (not thrown while opening), a new__create_filesimply calls__write_filespecifying the monitor folder in the path.Signed-off-by Gabriele Monaco <[email protected]>
Gabriele Monaco
Senior Software Engineer - Kernel Real Time
Red Hat
Gabriele Monaco
maintainerPATCH 8/8 verification/dot2k: Implement event type detection
In reply to Gabriele MonacoCurrently dot2k treats all events equally and registers them with a general da_handle_event. This is however just part of the work because some events are necessary to understand when the monitor is entering the initial state.
Specifically, the da_handle_start_event takes care of setting the monitor in the initial state and da_handle_start_run_event also registers the current event in the newly enabled monitor. da_handle_start_event can be used on events that only lead to the initial state (as it is currently done in the example monitors), while da_handle_start_run_event could be used on events that are only valid from the initial one.
Failing to set at least one of those functions to handle events makes the monitor useless, since it will never be activated.
This patch adapts dot2k to parse the events that surely lead to the initial state and set da_handle_start_event for those, if no such event is found but some events are only valid in the initial event, we instead set da_handle_start_run_event (it isn't necessary to set both). We still add a comment to warn the user to make sure this change is matching the model definition.
--- tools/verification/dot2/automata.py | 32 +++++++++++++++++++++++++++++ tools/verification/dot2/dot2k.py | 11 ++++++++-- 2 files changed, 41 insertions(+), 2 deletions(-)
tools/verification/dot2/automata.py +32 -0
@@ -26,6 +26,7 @@ class Automata: self.states, self.initial_state, self.final_states = self.__get_state_variables() self.events = self.__get_event_variables() self.function = self.__create_matrix() + self.events_start, self.events_start_run = self.__store_init_events() def __get_model_name(self): basename = ntpath.basename(self.__dot_path)
@@ -172,3 +173,34 @@ class Automata: cursor += 1 return matrix + + def __store_init_events(self): + events_start = [False] * len(self.events) + events_start_run = [False] * len(self.events) + for i, _ in enumerate(self.events): + curr_event_will_init = 0 + curr_event_from_init = False + curr_event_used = 0 + for j, _ in enumerate(self.states): + if self.function[j][i] != self.invalid_state_str: + curr_event_used += 1 + if self.function[j][i] == self.initial_state: + curr_event_will_init += 1 + if self.function[0][i] != self.invalid_state_str: + curr_event_from_init = True + # this event always leads to init + if curr_event_will_init and curr_event_used == curr_event_will_init: + events_start[i] = True + # this event is only called from init + if curr_event_from_init and curr_event_used == 1: + events_start_run[i] = True + return events_start, events_start_run + + def is_start_event(self, event): + return self.events_start[self.events.index(event)] + + def is_start_run_event(self, event): + # prefer handle_start_event if there + if any(self.events_start): + return False + return self.events_start_run[self.events.index(event)]
tools/verification/dot2/dot2k.py +9 -2
@@ -110,11 +110,18 @@ class dot2k(Dot2c): for event in self.events: buff.append("static void handle_%s(void *data, /* XXX: fill header */)" % event) buff.append("{") + handle = "handle_event" + if self.is_start_event(event): + buff.append("\t/* XXX: validate that this event always leads to the initial state */") + handle = "handle_start_event" + elif self.is_start_run_event(event): + buff.append("\t/* XXX: validate that this event is only valid in the initial state */") + handle = "handle_start_run_event" if self.monitor_type == "per_task": buff.append("\tstruct task_struct *p = /* XXX: how do I get p? */;"); - buff.append("\tda_handle_event_%s(p, %s%s);" % (self.name, event, self.enum_suffix)); + buff.append("\tda_%s_%s(p, %s%s);" % (handle, self.name, event, self.enum_suffix)); else: - buff.append("\tda_handle_event_%s(%s%s);" % (self.name, event, self.enum_suffix)); + buff.append("\tda_%s_%s(%s%s);" % (handle, self.name, event, self.enum_suffix)); buff.append("}") buff.append("") return self.__buff_to_string(buff)
Signature · 1 line
2.47.1