Skip to contentSkip to conversation

Fixes on Deterministic and Hybrid Automata

PATCH In review Gabriele Monaco 3 participants 18 messages Jun 1 – Jul 2

Subsystems (3)
lore
Share & tools
lore

Gabriele Monaco

author maintainer
#0 lore raw

v4 cover Fixes on Deterministic and Hybrid Automata

Fix issues that were reported by bots or visible only after integration:
* Make sure timers are always terminated and waited for when disabling the monitor or when the target terminates
* Run per-cpu monitors with migration disabled since preemption is now enabled from tracepoints
* Fix a wrong __user specifier in a helper function
* Other cleanup and concurrency issues

Differences since V3 [1]:
* Use separate reset functions for HA init/reset to avoid globals
* Reset monitor before sync stopping the timer on HA per-task exit hook

Differences since V2 [2]:
* Applied from reviews changes to commit messages
* Rearranged order to put non-fixes and not-reviewed patches in the end
* Synchronise monitors before resetting them to avoid rearming
* Protect against racing timer callbacks during destruction

Differences since V1 [3]:
* Fix memory consistency with timer callbacks racing with resets
* Add per-obj deallocation hook in rvgen generated code
* Do not rely on clean monitor when initialising HA
* Add tracepoint synchronisation before returning per-task slots
* Fix suffix strip in dot2k
* Generate stub deallocation hooks instead of failing build when per-obj miss those

[1] - https://lore.kernel.org/lkml/[email protected]
[2] - https://lore.kernel.org/lkml/[email protected]
[3] - https://lore.kernel.org/lkml/[email protected]

Cc Steven Rostedt <[email protected]>
Cc Nam Cao <[email protected]>
Cc Wen Yang <[email protected]>

Gabriele Monaco (12):
rv: Fix __user specifier usage in extract_params()
rv: Reset per-task DA monitors before releasing the slot
rv: Prevent in-flight per-task handlers from using invalid slots
rv: Ensure all pending probes terminate on per-obj monitor destroy
rv: Do not rely on clean monitor when initialising HA
rv: Add automatic cleanup handlers for per-task HA monitors
rv: Ensure synchronous cleanup for HA monitors
rv: Prevent task migration while handling per-CPU events
rv: Use 0 to check preemption enabled in opid verification/rvgen: Fix suffix strip in dot2k
rv: Fix read_lock scope in per-task DA cleanup verification/rvgen: Generate cleanup hook for per-obj monitor

Wen Yang (1):
rv: Fix monitor start ordering and memory ordering for monitoring flag

 include/rv/da_monitor.h                       | 145 ++++++++++++++----
 include/rv/ha_monitor.h                       |  91 ++++++++++-
 include/rv/ltl_monitor.h                      |   1 +
 kernel/trace/rv/monitors/deadline/deadline.h  |   3 +-
 kernel/trace/rv/monitors/nomiss/nomiss.c      |   4 +-
 kernel/trace/rv/monitors/opid/opid.c          |  12 +-
 kernel/trace/rv/monitors/stall/stall.c        |   4 +-
 tools/verification/rvgen/rvgen/dot2k.py       |  19 ++-
 .../rvgen/rvgen/templates/dot2k/main.c        |   4 +-
 9 files changed, 234 insertions(+), 49 deletions(-)

base-commit: e43ffb69e0438cddd72aaa30898b4dc446f664f8

Signature · 1 line
2.54.0
author maintainer
#1 lore raw

PATCH v4 1/13 Fix __user specifier usage in extract_params()

The attributes variables extracted from syscalls in the helper are both defined with the __user specifier although only the actual pointer to user data should be marked.

Remove the __user specifier from attr.

---
 kernel/trace/rv/monitors/deadline/deadline.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
kernel/trace/rv/monitors/deadline/deadline.h +2 -1
index 0bbfd2543..78fca873d 100644
@@ -95,7 +95,8 @@ static inline u8 get_server_type(struct task_struct *tsk)
 static inline int extract_params(struct pt_regs *regs, long id, pid_t *pid_out)
 {
 	size_t size = offsetofend(struct sched_attr, sched_flags);
-	struct sched_attr __user *uattr, attr;
+	struct sched_attr __user *uattr;
+	struct sched_attr attr;
 	int new_policy = -1, ret;
 	unsigned long args[6];
 
Signature · 1 line
2.54.0
author maintainer
#2 lore raw

PATCH v4 2/13 Reset per-task DA monitors before releasing the slot

In reply to Gabriele Monaco

Per-task monitors use task_mon_slot to determine which slot in the array to use for the monitor. During destruction, this slot is returned but this is done before resetting the monitor. As a result, the monitor's reset is in fact resetting a slot that is outside of the array (RV_PER_TASK_MONITOR_INIT).

Release the slot only after the reset to avoid out-of-bound memory access.

Fixes f5587d1b6ec93 ("rv: Add Hybrid Automata monitor type")
---
 include/rv/da_monitor.h | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
include/rv/da_monitor.h +3 -2
index 39765ff6f..1459fb3df 100644
@@ -309,10 +309,11 @@ static inline void da_monitor_destroy(void)
 		WARN_ONCE(1, "Disabling a disabled monitor: " __stringify(MONITOR_NAME));
 		return;
 	}
-	rv_put_task_monitor_slot(task_mon_slot);
-	task_mon_slot = RV_PER_TASK_MONITOR_INIT;
 
 	da_monitor_reset_all();
+
+	rv_put_task_monitor_slot(task_mon_slot);
+	task_mon_slot = RV_PER_TASK_MONITOR_INIT;
 }
 
 #elif RV_MON_TYPE == RV_MON_PER_OBJ
Signature · 1 line
2.54.0
author maintainer
#3 lore raw

PATCH v4 3/13 Prevent in-flight per-task handlers from using invalid slots

In reply to Gabriele Monaco

Per-task monitors use a slot in the task_struct->rv[] array and store that locally (e.g. task_mon_slot), this slot is returned during the destruction process but currently hanlers can be running while that slot is returning and this race may lead to accessing an invalid slot.

Synchronise with all in-flight tracepoint handlers using tracepoint_synchronize_unregister() before returning the slot.

Fixes f5587d1b6ec9 ("rv: Add Hybrid Automata monitor type")
Fixes a9769a5b9878 ("rv: Add support for LTL monitors")
---
 include/rv/da_monitor.h  | 4 ++++
 include/rv/ltl_monitor.h | 1 +
 2 files changed, 5 insertions(+)
include/rv/da_monitor.h +4 -0
index 1459fb3df..cc97cc5df 100644
@@ -302,6 +302,9 @@ static int da_monitor_init(void)
 
 /*
  * da_monitor_destroy - return the allocated slot
+ *
+ * Wait for all in-flight handlers before returning the slot to avoid
+ * out-of-bound accesses.
  */
 static inline void da_monitor_destroy(void)
 {
@@ -310,6 +313,7 @@ static inline void da_monitor_destroy(void)
 		return;
 	}
 
+	tracepoint_synchronize_unregister();
 	da_monitor_reset_all();
 
 	rv_put_task_monitor_slot(task_mon_slot);
include/rv/ltl_monitor.h +1 -0
index eff60cd61..38e792401 100644
@@ -77,6 +77,7 @@ static void ltl_monitor_destroy(void)
 {
 	rv_detach_trace_probe(name, task_newtask, handle_task_newtask);
 
+	tracepoint_synchronize_unregister();
 	rv_put_task_monitor_slot(ltl_monitor_slot);
 	ltl_monitor_slot = RV_PER_TASK_MONITOR_INIT;
 }
Signature · 1 line
2.54.0
author maintainer
#4 lore raw

PATCH v4 4/13 Ensure all pending probes terminate on per-obj monitor destroy

In reply to Gabriele Monaco

The monitor disable/destroy sequence detaches all probes and resets the monitor's data, however it doesn't wait for pending probes. This is an issue with per-object monitors, which free the monitor storage.

Call tracepoint_synchronize_unregister() to make sure to wait for all pending probes before destroying the monitor storage.

Fixes 4a24127bd6cb ("rv: Add support for per-object monitors in DA/HA")
---
 include/rv/da_monitor.h | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
include/rv/da_monitor.h +3 -2
index cc97cc5df..a7e103654 100644
@@ -511,9 +511,10 @@ static inline void da_monitor_destroy(void)
 	struct hlist_node *tmp;
 	int bkt;
 
+	tracepoint_synchronize_unregister();
 	/*
-	 * This function is called after all probes are disabled, we need only
-	 * worry about concurrency against old events.
+	 * This function is called after all probes are disabled and no longer
+	 * pending, we can safely assume no concurrent user.
 	 */
 	synchronize_rcu();
 	hash_for_each_safe(da_monitor_ht, bkt, tmp, mon_storage, node) {
Signature · 1 line
2.54.0
author maintainer
#5 lore raw

PATCH v4 5/13 Fix monitor start ordering and memory ordering for monitoring flag

In reply to Gabriele Monaco

From: Wen Yang <[email protected]>

da_monitor_start() set monitoring=1 before calling da_monitor_init_hook(), may racing with the sched_switch handler:

  da_monitor_start()               sched_switch handler
  -------------------------        ---------------------------------
  da_mon->monitoring = 1;
                                   if (da_monitoring(da_mon))  /* true  */
                                       ha_start_timer_ns(...);
                                       /* hrtimer->base == NULL, crash */
  da_monitor_init_hook(da_mon);
  /* hrtimer_setup() sets base */

Fix the ordering and pair with release/acquire semantics:

  da_monitor_init_hook(da_mon);
  smp_store_release(&da_mon->monitoring, 1);    /* da_monitor_start()  */
  return smp_load_acquire(&da_mon->monitoring); /* da_monitoring()     */

On ARM64 a plain STR + LDR does not form a release-acquire pair, so the load can observe monitoring=1 while hrtimer->base is still NULL. The plain accesses are also data races under KCSAN.

Use WRITE_ONCE for the monitoring=0 store in da_monitor_reset() to cover the reset path.

Fixes 792575348ff7 ("rv/include: Add deterministic automata monitor definition via C macros")
---
 include/rv/da_monitor.h | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
include/rv/da_monitor.h +5 -3
index a7e103654..60dc39f26 100644
@@ -82,7 +82,7 @@ static void react(enum states curr_state, enum events event)
 static inline void da_monitor_reset(struct da_monitor *da_mon)
 {
 	da_monitor_reset_hook(da_mon);
-	da_mon->monitoring = 0;
+	WRITE_ONCE(da_mon->monitoring, 0);
 	da_mon->curr_state = model_get_initial_state();
 }
 
@@ -95,8 +95,9 @@ static inline void da_monitor_reset(struct da_monitor *da_mon)
 static inline void da_monitor_start(struct da_monitor *da_mon)
 {
 	da_mon->curr_state = model_get_initial_state();
-	da_mon->monitoring = 1;
 	da_monitor_init_hook(da_mon);
+	/* Pairs with smp_load_acquire in da_monitoring(). */
+	smp_store_release(&da_mon->monitoring, 1);
 }
 
 /*
@@ -104,7 +105,8 @@ static inline void da_monitor_start(struct da_monitor *da_mon)
  */
 static inline bool da_monitoring(struct da_monitor *da_mon)
 {
-	return da_mon->monitoring;
+	/* Pairs with smp_store_release in da_monitor_start(). */
+	return smp_load_acquire(&da_mon->monitoring);
 }
 
 /*
Signature · 1 line
2.54.0
author maintainer
#6 lore raw

PATCH v4 6/13 Do not rely on clean monitor when initialising HA

In reply to Gabriele Monaco

Hybrid Automata monitors hook into the DA implementation when doing da_monitor_reset(). This function is called both on initialisation and teardown, HA monitors try to cancel a timer only when it's initialised relying on the da_mon->monitoring flag. This flag could however be corrupted during initialisation. This happens for instance on per-task monitors that share the same storage with different type of monitors like LTL or in case of races during a previous teardown.

Stop relying on the monitoring flag during initialisation, assume that can have any value, so use a separate da_reset_state() skiping timer cancellation.
New monitors (e.g. new tasks) are always zero-initialised so it is safe to rely on the monitoring flag for those.

---
 include/rv/da_monitor.h | 91 +++++++++++++++++++++++++++++++++--------
 include/rv/ha_monitor.h |  2 +-
 2 files changed, 76 insertions(+), 17 deletions(-)
include/rv/da_monitor.h +75 -16
index 60dc39f26..ec9bc88bd 100644
@@ -76,14 +76,22 @@ static void react(enum states curr_state, enum events event)
 		 model_get_state_name(curr_state));
 }
 
+/*
+ * da_monitor_reset_state - reset a monitor and setting it to init state
+ */
+static inline void da_monitor_reset_state(struct da_monitor *da_mon)
+{
+	WRITE_ONCE(da_mon->monitoring, 0);
+	da_mon->curr_state = model_get_initial_state();
+}
+
 /*
  * da_monitor_reset - reset a monitor and setting it to init state
  */
 static inline void da_monitor_reset(struct da_monitor *da_mon)
 {
 	da_monitor_reset_hook(da_mon);
-	WRITE_ONCE(da_mon->monitoring, 0);
-	da_mon->curr_state = model_get_initial_state();
+	da_monitor_reset_state(da_mon);
 }
 
 /*
@@ -158,12 +166,28 @@ static struct da_monitor *da_get_monitor(void)
 	return &DA_MON_NAME;
 }
 
+/*
+ * __da_monitor_reset_all - reset the single monitor
+ */
+static void __da_monitor_reset_all(void (*reset)(struct da_monitor *))
+{
+	reset(da_get_monitor());
+}
+
 /*
  * da_monitor_reset_all - reset the single monitor
  */
 static void da_monitor_reset_all(void)
 {
-	da_monitor_reset(da_get_monitor());
+	__da_monitor_reset_all(da_monitor_reset);
+}
+
+/*
+ * da_monitor_reset_state_all - reset the single monitor
+ */
+static inline void da_monitor_reset_state_all(void)
+{
+	__da_monitor_reset_all(da_monitor_reset_state);
 }
 
 /*
@@ -171,7 +195,7 @@ static void da_monitor_reset_all(void)
  */
 static inline int da_monitor_init(void)
 {
-	da_monitor_reset_all();
+	da_monitor_reset_state_all();
 	return 0;
 }
 
@@ -202,25 +226,41 @@ static struct da_monitor *da_get_monitor(void)
 }
 
 /*
- * da_monitor_reset_all - reset all CPUs' monitor
+ * __da_monitor_reset_all - reset all CPUs' monitor
  */
-static void da_monitor_reset_all(void)
+static void __da_monitor_reset_all(void (*reset)(struct da_monitor *))
 {
 	struct da_monitor *da_mon;
 	int cpu;
 
 	for_each_cpu(cpu, cpu_online_mask) {
 		da_mon = per_cpu_ptr(&DA_MON_NAME, cpu);
-		da_monitor_reset(da_mon);
+		reset(da_mon);
 	}
 }
 
+/*
+ * da_monitor_reset_all - reset all CPUs' monitor
+ */
+static void da_monitor_reset_all(void)
+{
+	__da_monitor_reset_all(da_monitor_reset);
+}
+
+/*
+ * da_monitor_reset_state_all - reset all CPUs' monitor
+ */
+static inline void da_monitor_reset_state_all(void)
+{
+	__da_monitor_reset_all(da_monitor_reset_state);
+}
+
 /*
  * da_monitor_init - initialize all CPUs' monitor
  */
 static inline int da_monitor_init(void)
 {
-	da_monitor_reset_all();
+	da_monitor_reset_state_all();
 	return 0;
 }
 
@@ -269,19 +309,29 @@ static inline da_id_type da_get_id(struct da_monitor *da_mon)
 	return da_get_target(da_mon)->pid;
 }
 
-static void da_monitor_reset_all(void)
+static void __da_monitor_reset_all(void (*reset)(struct da_monitor *))
 {
 	struct task_struct *g, *p;
 	int cpu;
 
 	read_lock(&tasklist_lock);
 	for_each_process_thread(g, p)
-		da_monitor_reset(da_get_monitor(p));
+		reset(da_get_monitor(p));
 	for_each_present_cpu(cpu)
-		da_monitor_reset(da_get_monitor(idle_task(cpu)));
+		reset(da_get_monitor(idle_task(cpu)));
 	read_unlock(&tasklist_lock);
 }
 
+static void da_monitor_reset_all(void)
+{
+	__da_monitor_reset_all(da_monitor_reset);
+}
+
+static inline void da_monitor_reset_state_all(void)
+{
+	__da_monitor_reset_all(da_monitor_reset_state);
+}
+
 /*
  * da_monitor_init - initialize the per-task monitor
  *
@@ -298,7 +348,7 @@ static int da_monitor_init(void)
 
 	task_mon_slot = slot;
 
-	da_monitor_reset_all();
+	da_monitor_reset_state_all();
 	return 0;
 }
 
@@ -490,15 +540,24 @@ static inline void da_destroy_storage(da_id_type id)
 	kfree_rcu(mon_storage, rcu);
 }
 
-static void da_monitor_reset_all(void)
+static void __da_monitor_reset_all(void (*reset)(struct da_monitor *))
 {
 	struct da_monitor_storage *mon_storage;
 	int bkt;
 
-	rcu_read_lock();
+	guard(rcu)();
 	hash_for_each_rcu(da_monitor_ht, bkt, mon_storage, node)
-		da_monitor_reset(&mon_storage->rv.da_mon);
-	rcu_read_unlock();
+		reset(&mon_storage->rv.da_mon);
+}
+
+static void da_monitor_reset_all(void)
+{
+	__da_monitor_reset_all(da_monitor_reset);
+}
+
+static inline void da_monitor_reset_state_all(void)
+{
+	__da_monitor_reset_all(da_monitor_reset_state);
 }
 
 static inline int da_monitor_init(void)
include/rv/ha_monitor.h +1 -1
index d59507e8c..bd8705556 100644
@@ -153,12 +153,12 @@ static inline void ha_monitor_init_env(struct da_monitor *da_mon)
  * Called from a hook in the DA reset functions, it supplies the da_mon
  * corresponding to the current ha_mon.
  * Not all hybrid automata require the timer, still clear it for simplicity.
+ * Monitors that never started have their timer uninitialized, do not stop those.
  */
 static inline void ha_monitor_reset_env(struct da_monitor *da_mon)
 {
 	struct ha_monitor *ha_mon = to_ha_monitor(da_mon);
 
-	/* Initialisation resets the monitor before initialising the timer */
 	if (likely(da_monitoring(da_mon)))
 		ha_cancel_timer(ha_mon);
 }
Signature · 1 line
2.54.0
Reviewed-by on v4 6/13
#7 lore raw

Re: [PATCH v4 06/13] rv: Do not rely on clean monitor when initialising HA

Quoted 14 lines · Gabriele Monaco

Hybrid Automata monitors hook into the DA implementation when doing da_monitor_reset(). This function is called both on initialisation and teardown, HA monitors try to cancel a timer only when it's initialised relying on the da_mon->monitoring flag. This flag could however be corrupted during initialisation. This happens for instance on per-task monitors that share the same storage with different type of monitors like LTL or in case of races during a previous teardown.

Stop relying on the monitoring flag during initialisation, assume that can have any value, so use a separate da_reset_state() skiping timer cancellation.
New monitors (e.g. new tasks) are always zero-initialised so it is safe to rely on the monitoring flag for those.

Quoted 6 lines
author maintainer
#8 lore raw

PATCH v4 7/13 Add automatic cleanup handlers for per-task HA monitors

In reply to Gabriele Monaco

Hybrid automata monitors may start timers, depending on the model, these may remain active on an exiting task and cause false positives or even access freed memory.

Add an enable/disable hook in the HA code, currently only populated by the per-task handler for registration and deregistration. This hooks to the sched_process_exit event and ensures the timer is stopped for every exiting task. The handler is enabled automatically but may be disabled, for instance if the monitor uses the event for another purpose (but should still manually ensure timers are stopped).

Fixes f5587d1b6ec9 ("rv: Add Hybrid Automata monitor type")
---
 include/rv/ha_monitor.h                       | 60 +++++++++++++++++++
 kernel/trace/rv/monitors/nomiss/nomiss.c      |  4 +-
 kernel/trace/rv/monitors/opid/opid.c          |  4 +-
 kernel/trace/rv/monitors/stall/stall.c        |  4 +-
 .../rvgen/rvgen/templates/dot2k/main.c        |  4 +-
 5 files changed, 68 insertions(+), 8 deletions(-)
include/rv/ha_monitor.h +60 -0
index bd8705556..4002b5247 100644
@@ -28,6 +28,7 @@ static inline void ha_monitor_init_env(struct da_monitor *da_mon);
 static inline void ha_monitor_reset_env(struct da_monitor *da_mon);
 static inline void ha_setup_timer(struct ha_monitor *ha_mon);
 static inline bool ha_cancel_timer(struct ha_monitor *ha_mon);
+static inline void ha_cancel_timer_sync(struct ha_monitor *ha_mon);
 static bool ha_monitor_handle_constraint(struct da_monitor *da_mon,
 					 enum states curr_state,
 					 enum events event,
@@ -37,6 +38,26 @@ static bool ha_monitor_handle_constraint(struct da_monitor *da_mon,
 #define da_monitor_init_hook ha_monitor_init_env
 #define da_monitor_reset_hook ha_monitor_reset_env
 
+#if !defined(HA_SKIP_AUTO_CLEANUP) && RV_MON_TYPE == RV_MON_PER_TASK
+/*
+ * Automatic cleanup handlers for per-task HA monitors, only skip if you know
+ * what you are doing (e.g. you want to implement cleanup manually in another
+ * handler doing more things).
+ */
+static void ha_handle_sched_process_exit(void *data, struct task_struct *p,
+					 bool group_dead);
+
+#define ha_monitor_enable_hook()                                             \
+	rv_attach_trace_probe(__stringify(MONITOR_NAME), sched_process_exit, \
+			      ha_handle_sched_process_exit)
+#define ha_monitor_disable_hook()                                            \
+	rv_detach_trace_probe(__stringify(MONITOR_NAME), sched_process_exit, \
+			      ha_handle_sched_process_exit)
+#else
+#define ha_monitor_enable_hook() ((void)0)
+#define ha_monitor_disable_hook() ((void)0)
+#endif
+
 #include <rv/da_monitor.h>
 #include <linux/seq_buf.h>
 
@@ -115,6 +136,22 @@ static enum hrtimer_restart ha_monitor_timer_callback(struct hrtimer *hrtimer);
 #define ha_get_ns() 0
 #endif /* HA_CLK_NS */
 
+static int ha_monitor_init(void)
+{
+	int ret;
+
+	ret = da_monitor_init();
+	if (ret == 0)
+		ha_monitor_enable_hook();
+	return ret;
+}
+
+static void ha_monitor_destroy(void)
+{
+	ha_monitor_disable_hook();
+	da_monitor_destroy();
+}
+
 /* Should be supplied by the monitor */
 static u64 ha_get_env(struct ha_monitor *ha_mon, enum envs env, u64 time_ns);
 static bool ha_verify_constraint(struct ha_monitor *ha_mon,
@@ -200,6 +237,20 @@ static inline void ha_trace_error_env(struct ha_monitor *ha_mon,
 {
 	CONCATENATE(trace_error_env_, MONITOR_NAME)(id, curr_state, event, env);
 }
+
+#if !defined(HA_SKIP_AUTO_CLEANUP) && RV_MON_TYPE == RV_MON_PER_TASK
+static void ha_handle_sched_process_exit(void *data, struct task_struct *p,
+					 bool group_dead)
+{
+	struct da_monitor *da_mon = da_get_monitor(p);
+
+	if (likely(da_monitoring(da_mon))) {
+		da_monitor_reset(da_mon);
+		ha_cancel_timer_sync(to_ha_monitor(da_mon));
+	}
+}
+#endif
+
 #endif /* RV_MON_TYPE */
 
 /*
@@ -412,6 +463,10 @@ static inline bool ha_cancel_timer(struct ha_monitor *ha_mon)
 {
 	return timer_delete(&ha_mon->timer);
 }
+static inline void ha_cancel_timer_sync(struct ha_monitor *ha_mon)
+{
+	timer_delete_sync(&ha_mon->timer);
+}
 #elif HA_TIMER_TYPE == HA_TIMER_HRTIMER
 /*
  * Helper functions to handle the monitor timer.
@@ -463,6 +518,10 @@ static inline bool ha_cancel_timer(struct ha_monitor *ha_mon)
 {
 	return hrtimer_try_to_cancel(&ha_mon->hrtimer) == 1;
 }
+static inline void ha_cancel_timer_sync(struct ha_monitor *ha_mon)
+{
+	hrtimer_cancel(&ha_mon->hrtimer);
+}
 #else /* HA_TIMER_NONE */
 /*
  * Start function is intentionally not defined, monitors using timers must
@@ -473,6 +532,7 @@ static inline bool ha_cancel_timer(struct ha_monitor *ha_mon)
 {
 	return false;
 }
+static inline void ha_cancel_timer_sync(struct ha_monitor *ha_mon) { }
 #endif
 
 #endif
kernel/trace/rv/monitors/nomiss/nomiss.c +2 -2
index 31f90f363..8ead8783c 100644
@@ -227,7 +227,7 @@ static int enable_nomiss(void)
 {
 	int retval;
 
-	retval = da_monitor_init();
+	retval = ha_monitor_init();
 	if (retval)
 		return retval;
 
@@ -263,7 +263,7 @@ static void disable_nomiss(void)
 	rv_detach_trace_probe("nomiss", sched_switch, handle_sched_switch);
 	rv_detach_trace_probe("nomiss", sched_wakeup, handle_sched_wakeup);
 
-	da_monitor_destroy();
+	ha_monitor_destroy();
 }
 
 static struct rv_monitor rv_this = {
kernel/trace/rv/monitors/opid/opid.c +2 -2
index 4594c7c46..2922318c6 100644
@@ -73,7 +73,7 @@ static int enable_opid(void)
 {
 	int retval;
 
-	retval = da_monitor_init();
+	retval = ha_monitor_init();
 	if (retval)
 		return retval;
 
@@ -90,7 +90,7 @@ static void disable_opid(void)
 	rv_detach_trace_probe("opid", sched_set_need_resched_tp, handle_sched_need_resched);
 	rv_detach_trace_probe("opid", sched_waking, handle_sched_waking);
 
-	da_monitor_destroy();
+	ha_monitor_destroy();
 }
 
 /*
kernel/trace/rv/monitors/stall/stall.c +2 -2
index 9ccfda6b0..3c38fb1a0 100644
@@ -103,7 +103,7 @@ static int enable_stall(void)
 {
 	int retval;
 
-	retval = da_monitor_init();
+	retval = ha_monitor_init();
 	if (retval)
 		return retval;
 
@@ -120,7 +120,7 @@ static void disable_stall(void)
 	rv_detach_trace_probe("stall", sched_switch, handle_sched_switch);
 	rv_detach_trace_probe("stall", sched_wakeup, handle_sched_wakeup);
 
-	da_monitor_destroy();
+	ha_monitor_destroy();
 }
 
 static struct rv_monitor rv_this = {
tools/verification/rvgen/rvgen/templates/dot2k/main.c +2 -2
index bf0999f66..889446760 100644
@@ -35,7 +35,7 @@ static int enable_%%MODEL_NAME%%(void)
 {
 	int retval;
 
-	retval = da_monitor_init();
+	retval = %%MONITOR_CLASS%%_monitor_init();
 	if (retval)
 		return retval;
 
@@ -50,7 +50,7 @@ static void disable_%%MODEL_NAME%%(void)
 
 %%TRACEPOINT_DETACH%%
 
-	da_monitor_destroy();
+	%%MONITOR_CLASS%%_monitor_destroy();
 }
 
 /*
Signature · 1 line
2.54.0
author maintainer
#9 lore raw

PATCH v4 8/13 Ensure synchronous cleanup for HA monitors

In reply to Gabriele Monaco

HA monitors may start timers, all cleanup functions currently stop the timers asynchronously to avoid sleeping in the wrong context. Nothing makes sure running callbacks terminate on cleanup.

Run the entire HA timer callback in an RCU read-side critical section, this way we can simply synchronize_rcu() with any pending timer and are sure any cleanup using kfree_rcu() runs after callbacks terminated. Additionally make sure any unlikely callback running late won't run any code if the monitor is marked as disabled or if destruction started. Use memory barriers to serialise with racing resets.

Fixes f5587d1b6ec9 ("rv: Add Hybrid Automata monitor type")
Fixes 4a24127bd6cb ("rv: Add support for per-object monitors in DA/HA")
---
 include/rv/da_monitor.h | 19 ++++++++++++++++---
 include/rv/ha_monitor.h | 29 ++++++++++++++++++++++++++---
 2 files changed, 42 insertions(+), 6 deletions(-)
include/rv/da_monitor.h +16 -3
index ec9bc88bd..1f440c781 100644
@@ -57,6 +57,15 @@ static struct rv_monitor rv_this;
 #define da_monitor_reset_hook(da_mon)
 #endif
 
+/*
+ * Hook to allow the implementation of hybrid automata: define it with a
+ * function that waits for the termination of all monitors background
+ * activities (e.g. all timers). This hook can sleep.
+ */
+#ifndef da_monitor_sync_hook
+#define da_monitor_sync_hook()
+#endif
+
 /*
  * Type for the target id, default to int but can be overridden.
  * A long type can work as hash table key (PER_OBJ) but will be downgraded to
@@ -82,7 +91,8 @@ static void react(enum states curr_state, enum events event)
 static inline void da_monitor_reset_state(struct da_monitor *da_mon)
 {
 	WRITE_ONCE(da_mon->monitoring, 0);
-	da_mon->curr_state = model_get_initial_state();
+	/* Pair with load in __ha_monitor_timer_callback */
+	smp_store_release(&da_mon->curr_state, model_get_initial_state());
 }
 
 /*
@@ -205,6 +215,7 @@ static inline int da_monitor_init(void)
 static inline void da_monitor_destroy(void)
 {
 	da_monitor_reset_all();
+	da_monitor_sync_hook();
 }
 
 #elif RV_MON_TYPE == RV_MON_PER_CPU
@@ -270,6 +281,7 @@ static inline int da_monitor_init(void)
 static inline void da_monitor_destroy(void)
 {
 	da_monitor_reset_all();
+	da_monitor_sync_hook();
 }
 
 #elif RV_MON_TYPE == RV_MON_PER_TASK
@@ -367,6 +379,7 @@ static inline void da_monitor_destroy(void)
 
 	tracepoint_synchronize_unregister();
 	da_monitor_reset_all();
+	da_monitor_sync_hook();
 
 	rv_put_task_monitor_slot(task_mon_slot);
 	task_mon_slot = RV_PER_TASK_MONITOR_INIT;
@@ -573,13 +586,13 @@ static inline void da_monitor_destroy(void)
 	int bkt;
 
 	tracepoint_synchronize_unregister();
+	da_monitor_reset_all();
+	da_monitor_sync_hook();
 	/*
 	 * This function is called after all probes are disabled and no longer
 	 * pending, we can safely assume no concurrent user.
 	 */
-	synchronize_rcu();
 	hash_for_each_safe(da_monitor_ht, bkt, tmp, mon_storage, node) {
-		da_monitor_reset_hook(&mon_storage->rv.da_mon);
 		hash_del_rcu(&mon_storage->node);
 		kfree(mon_storage);
 	}
include/rv/ha_monitor.h +26 -3
index 4002b5247..28d3c74ca 100644
@@ -37,6 +37,7 @@ static bool ha_monitor_handle_constraint(struct da_monitor *da_mon,
 #define da_monitor_event_hook ha_monitor_handle_constraint
 #define da_monitor_init_hook ha_monitor_init_env
 #define da_monitor_reset_hook ha_monitor_reset_env
+#define da_monitor_sync_hook() synchronize_rcu()
 
 #if !defined(HA_SKIP_AUTO_CLEANUP) && RV_MON_TYPE == RV_MON_PER_TASK
 /*
@@ -136,10 +137,13 @@ static enum hrtimer_restart ha_monitor_timer_callback(struct hrtimer *hrtimer);
 #define ha_get_ns() 0
 #endif /* HA_CLK_NS */
 
+static bool ha_mon_destroying;
+
 static int ha_monitor_init(void)
 {
 	int ret;
 
+	WRITE_ONCE(ha_mon_destroying, false);
 	ret = da_monitor_init();
 	if (ret == 0)
 		ha_monitor_enable_hook();
@@ -148,6 +152,7 @@ static int ha_monitor_init(void)
 
 static void ha_monitor_destroy(void)
 {
+	WRITE_ONCE(ha_mon_destroying, true);
 	ha_monitor_disable_hook();
 	da_monitor_destroy();
 }
@@ -288,12 +293,30 @@ static bool ha_monitor_handle_constraint(struct da_monitor *da_mon,
 	return false;
 }
 
+/*
+ * __ha_monitor_timer_callback - generic callback representation
+ *
+ * This callback runs in an RCU read-side critical section to allow the
+ * destruction sequence to easily synchronize_rcu() with all pending timers
+ * after asynchronously disabling them. The ha_mon_destroying check ensures
+ * any callback entering the RCU section after synchronize_rcu() completes
+ * will see the flag and bail out immediately.
+ */
 static inline void __ha_monitor_timer_callback(struct ha_monitor *ha_mon)
 {
-	enum states curr_state = READ_ONCE(ha_mon->da_mon.curr_state);
 	DECLARE_SEQ_BUF(env_string, ENV_BUFFER_SIZE);
-	u64 time_ns = ha_get_ns();
-
+	enum states curr_state;
+	u64 time_ns;
+
+	guard(rcu)();
+	if (unlikely(READ_ONCE(ha_mon_destroying)))
+		return;
+	/* Ensure consistent curr_state if we race with da_monitor_reset */
+	curr_state = smp_load_acquire(&ha_mon->da_mon.curr_state);
+	if (unlikely(!da_monitor_handling_event(&ha_mon->da_mon)))
+		return;
+
+	time_ns = ha_get_ns();
 	ha_get_env_string(&env_string, ha_mon, time_ns);
 	ha_react(curr_state, EVENT_NONE, env_string.buffer);
 	ha_trace_error_env(ha_mon, model_get_state_name(curr_state),
Signature · 1 line
2.54.0
Reviewed-by on v4 8/13
#10 lore raw

Re: [PATCH v4 08/13] rv: Ensure synchronous cleanup for HA monitors

Quoted 11 lines · Gabriele Monaco

HA monitors may start timers, all cleanup functions currently stop the timers asynchronously to avoid sleeping in the wrong context. Nothing makes sure running callbacks terminate on cleanup.

Run the entire HA timer callback in an RCU read-side critical section, this way we can simply synchronize_rcu() with any pending timer and are sure any cleanup using kfree_rcu() runs after callbacks terminated. Additionally make sure any unlikely callback running late won't run any code if the monitor is marked as disabled or if destruction started. Use memory barriers to serialise with racing resets.

Quoted 3 lines
Fixes f5587d1b6ec9 ("rv: Add Hybrid Automata monitor type")
Fixes 4a24127bd6cb ("rv: Add support for per-object monitors in DA/HA")
author maintainer
#11 lore raw

PATCH v4 9/13 Prevent task migration while handling per-CPU events

In reply to Gabriele Monaco

Tracepoint handlers are fully preemptible after a46023d5616 ("tracing:
Guard __DECLARE_TRACE() use of __DO_TRACE_CALL() with SRCU-fast"). When a per-CPU monitor handles an event, it retrieves the monitor state using a per-CPU pointer. If the event itself doesn't disable preemption, the task can migrate to a different CPU and we risk updating the wrong monitor.

Mitigate this by explicitly disabling task migration before acquiring the monitor pointer. This cannot guarantee the monitor runs on the correct CPU but reduces the race condition window and prevents warnings.

---
 include/rv/da_monitor.h | 11 +++++++++++
 1 file changed, 11 insertions(+)
include/rv/da_monitor.h +11 -0
index 1f440c781..34b8fba9e 100644
@@ -218,6 +218,10 @@ static inline void da_monitor_destroy(void)
 	da_monitor_sync_hook();
 }
 
+#ifndef da_implicit_guard
+#define da_implicit_guard()
+#endif
+
 #elif RV_MON_TYPE == RV_MON_PER_CPU
 /*
  * Functions to define, init and get a per-cpu monitor.
@@ -284,6 +288,10 @@ static inline void da_monitor_destroy(void)
 	da_monitor_sync_hook();
 }
 
+#ifndef da_implicit_guard
+#define da_implicit_guard() guard(migrate)()
+#endif
+
 #elif RV_MON_TYPE == RV_MON_PER_TASK
 /*
  * Functions to define, init and get a per-task monitor.
@@ -756,6 +764,7 @@ static inline bool __da_handle_start_run_event(struct da_monitor *da_mon,
  */
 static inline void da_handle_event(enum events event)
 {
+	da_implicit_guard();
 	__da_handle_event(da_get_monitor(), event, 0);
 }
 
@@ -771,6 +780,7 @@ static inline void da_handle_event(enum events event)
  */
 static inline bool da_handle_start_event(enum events event)
 {
+	da_implicit_guard();
 	return __da_handle_start_event(da_get_monitor(), event, 0);
 }
 
@@ -782,6 +792,7 @@ static inline bool da_handle_start_event(enum events event)
  */
 static inline bool da_handle_start_run_event(enum events event)
 {
+	da_implicit_guard();
 	return __da_handle_start_run_event(da_get_monitor(), event, 0);
 }
 
Signature · 1 line
2.54.0
author maintainer
#12 lore raw

PATCH v4 10/13 Use 0 to check preemption enabled in opid

In reply to Gabriele Monaco

Tracepoint handlers no longer run with preemption disabled by default since a46023d5616 ("tracing: Guard __DECLARE_TRACE() use of __DO_TRACE_CALL() with SRCU-fast"), the opid monitor should now count 1 in the preemption count as preemption disabled.

Change the rule for preempt_off to preempt > 0.

---
 kernel/trace/rv/monitors/opid/opid.c | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)
kernel/trace/rv/monitors/opid/opid.c +1 -7
index 2922318c6..3b6a85e81 100644
@@ -22,14 +22,8 @@ static u64 ha_get_env(struct ha_monitor *ha_mon, enum envs_opid env, u64 time_ns
 	if (env == irq_off_opid)
 		return irqs_disabled();
 	else if (env == preempt_off_opid) {
-		/*
-		 * If CONFIG_PREEMPTION is enabled, then the tracepoint itself disables
-		 * preemption (adding one to the preempt_count). Since we are
-		 * interested in the preempt_count at the time the tracepoint was
-		 * hit, we consider 1 as still enabled.
-		 */
 		if (IS_ENABLED(CONFIG_PREEMPTION))
-			return (preempt_count() & PREEMPT_MASK) > 1;
+			return (preempt_count() & PREEMPT_MASK) > 0;
 		return true;
 	}
 	return ENV_INVALID_VALUE;
Signature · 1 line
2.54.0
author maintainer
#13 lore raw

PATCH v4 11/13 verification/rvgen: Fix suffix strip in dot2k

In reply to Gabriele Monaco

__start_to_invariant_check() and __get_constraint_env() parse the environment variable's name from sources that have it padded with the monitor name. This is removed using rstrip(), which is not meant to strip a substring but rather a set of characters.

Use removesuffix() to actually get rid of the trailing _<monitor name>.

Fixes a82adadb16894 ("verification/rvgen: Add support for Hybrid Automata")
---
 tools/verification/rvgen/rvgen/dot2k.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tools/verification/rvgen/rvgen/dot2k.py +2 -2
index e6f476b90..110cfd69e 100644
@@ -215,14 +215,14 @@ class ha2k(dot2k):
     def __get_constraint_env(self, constr: str) -> str:
         """Extract the second argument from an ha_ function"""
         env = constr.split("(")[1].split()[1].rstrip(")").rstrip(",")
-        assert env.rstrip(f"_{self.name}") in self.envs
+        assert env.removesuffix(f"_{self.name}") in self.envs
         return env
 
     def __start_to_invariant_check(self, constr: str) -> str:
         # by default assume the timer has ns expiration
         env = self.__get_constraint_env(constr)
         clock_type = "ns"
-        if self.env_types.get(env.rstrip(f"_{self.name}")) == "j":
+        if self.env_types.get(env.removesuffix(f"_{self.name}")) == "j":
             clock_type = "jiffy"
 
         return f"return ha_check_invariant_{clock_type}(ha_mon, {env}, time_ns)"
Signature · 1 line
2.54.0
author maintainer
#14 lore raw

PATCH v4 12/13 Fix read_lock scope in per-task DA cleanup

In reply to Gabriele Monaco

The da_monitor_reset_all() function for per-task monitors takes tasklist_lock while iterating over tasks, then keeps it also while iterating over idle tasks (one per CPU). The latter is not necessary since the lock needs to guard only for_each_process_thread().

Use a scoped_guard for more compact syntax and adjust the scope only where the lock is necessary.

---
 include/rv/da_monitor.h | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
include/rv/da_monitor.h +4 -4
index 34b8fba9e..08e5d0c59 100644
@@ -334,12 +334,12 @@ static void __da_monitor_reset_all(void (*reset)(struct da_monitor *))
 	struct task_struct *g, *p;
 	int cpu;
 
-	read_lock(&tasklist_lock);
-	for_each_process_thread(g, p)
-		reset(da_get_monitor(p));
+	scoped_guard(read_lock, &tasklist_lock) {
+		for_each_process_thread(g, p)
+			reset(da_get_monitor(p));
+	}
 	for_each_present_cpu(cpu)
 		reset(da_get_monitor(idle_task(cpu)));
-	read_unlock(&tasklist_lock);
 }
 
 static void da_monitor_reset_all(void)
Signature · 1 line
2.54.0
maintainer
#15 lore raw

Re: [PATCH v4 12/13] rv: Fix read_lock scope in per-task DA cleanup

Quoted 11 lines · On Mon, 1 Jun 2026 17:38:39 +0200 Gabriele Monaco <[email protected]> wrote:

The da_monitor_reset_all() function for per-task monitors takes tasklist_lock while iterating over tasks, then keeps it also while iterating over idle tasks (one per CPU). The latter is not necessary since the lock needs to guard only for_each_process_thread().

Use a scoped_guard for more compact syntax and adjust the scope only where the lock is necessary.

Hmm, this and patch 13 didn't get applied. Do you plan on sending them later?

-- Steve

author maintainer
#16 lore raw

Re: [PATCH v4 12/13] rv: Fix read_lock scope in per-task DA cleanup

Quoted 8 lines · Steven Rostedt
Quoted 5 lines · On Mon,  1 Jun 2026 17:38:39 +0200 Gabriele Monaco <[email protected]> wrote:

The da_monitor_reset_all() function for per-task monitors takes tasklist_lock while iterating over tasks, then keeps it also while iterating over idle tasks (one per CPU). The latter is not necessary since the lock needs to guard only for_each_process_thread().

Quoted 8 lines
Quoted 6 lines

Use a scoped_guard for more compact syntax and adjust the scope only where the lock is necessary.

Hmm, this and patch 13 didn't get applied. Do you plan on sending them later?

Yes, I didn't add them to the pull request since they weren't quite fixes, then ended up not sending a "normal" pull request for 7.2 just for these. I'm going to put them together for the next round.

Thanks,
Gabriele

author maintainer
#17 lore raw

PATCH v4 13/13 verification/rvgen: Generate cleanup hook for per-obj monitor

In reply to Gabriele Monaco

Per-object monitors can allocate memory dynamically and such memory is required for the lifetime of the object, then it should be freed with the appropriate call.

Force the generation scripts to add a cleanup function the user will need to wire to the appropriate event (e.g. sched_process_exit for tasks). This can be safely removed if the object will never cease to exist before disabling the monitor (e.g. if following only static variables).

---
 tools/verification/rvgen/rvgen/dot2k.py | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
tools/verification/rvgen/rvgen/dot2k.py +15 -0
index 110cfd69e..3060aa4b9 100644
@@ -17,6 +17,9 @@ from .automata import _EventConstraintKey, _StateConstraintKey, AutomataError
 class dot2k(Monitor, Dot2c):
     template_dir = "dot2k"
 
+    # only needed for the per-obj cleanup hook
+    cleanup_marker = "obj_cleanup"
+
     def __init__(self, file_path, MonitorType, extra_params={}):
         self.monitor_type = MonitorType
         Monitor.__init__(self, extra_params)
@@ -56,18 +59,30 @@ class dot2k(Monitor, Dot2c):
                 buff.append(f"\tda_{handle}({event}{self.enum_suffix});")
             buff.append("}")
             buff.append("")
+        if self.monitor_type == "per_obj":
+            buff.append("/* XXX: obj is being destroyed, remove if not required (e.g. obj is static) */")
+            buff.append(f"static void handle_{self.cleanup_marker}(void *data, /* XXX: fill header */)")
+            buff.append("{")
+            buff.append("\tint id = /* XXX: how do I get the id? */;")
+            buff.append("\tda_destroy_storage(id);")
+            buff.append("}")
+            buff.append("")
         return '\n'.join(buff)
 
     def fill_tracepoint_attach_probe(self) -> str:
         buff = []
         for event in self.events:
             buff.append(f"\trv_attach_trace_probe(\"{self.name}\", /* XXX: tracepoint */, handle_{event});")
+        if self.monitor_type == "per_obj":
+            buff.append(f"\trv_attach_trace_probe(\"{self.name}\", /* XXX: cleanup tracepoint */, handle_{self.cleanup_marker});")
         return '\n'.join(buff)
 
     def fill_tracepoint_detach_helper(self) -> str:
         buff = []
         for event in self.events:
             buff.append(f"\trv_detach_trace_probe(\"{self.name}\", /* XXX: tracepoint */, handle_{event});")
+        if self.monitor_type == "per_obj":
+            buff.append(f"\trv_detach_trace_probe(\"{self.name}\", /* XXX: cleanup tracepoint */, handle_{self.cleanup_marker});")
         return '\n'.join(buff)
 
     def fill_model_h_header(self) -> list[str]:
Signature · 1 line
2.54.0