Skip to contentSkip to conversation

non-RT-safe rtdm_event_*wait

RFC In review Jan Kiszka 2 participants 4 messages May 18, 2006 – May 22, 2006

lore
Share & tools
lore

Jan Kiszka

author
#0 lore raw

RFC 1/1 non-RT-safe rtdm_event_*wait

Hi,

attached is an experimental patch to allow blocking from non-RT context on RTDM events. Petr Cervenka made me think about this extension which could partially open RTDM drivers also for non-shadowed, i.e. standard Linux tasks.

Why not simply shadowing non-RT waiters? Because this kind of sync support should be cheaper than adding yet another (though low-prio) task to the real-time domain ("should" == I didn't benchmark my believe). And it's more convenient because no shadow setup is required prior to blocking on "dual-use" resources.

The patch seems to work (see attached simple test, also try ^C for interruption testing), but requires more refactoring before being applied. This will not happen before release 2.2 anyway. I also want to add non-RT support for rtdm_sem in the final patch (rtdm_mutex makes no sense due to risk of prio inversion). The whole non-RT support will furthermore become optional to avoid code bloating.

Given that we then would have more powerful context-agnostic synchronisation services for RTDM drivers, we may even consider registering certain RTDM devices also with standard Linux, thus making them available to Linux users without the need to use librtdm services. Well, I haven't thought about all implications and limitation of such a step, as well as its actual usefulness. It's just an idea so far.

Feedback is welcome, specifically about real tests or bugs you may spot in this code.

Jan

rtdm/rtdm_driver.h +7 -2
Index: include/rtdm/rtdm_driver.h
===================================================================
@@ -921,6 +921,7 @@ static inline void rtdm_toseq_init(rtdm_
 
 typedef struct {
     xnsynch_t                       synch_base;
+    struct list_head                nrt_wait_queue;
 } rtdm_event_t;
 
 #define RTDM_EVENT_PENDING          XNSYNCH_SPARE1
@@ -928,15 +929,18 @@ typedef struct {
 static inline void rtdm_event_init(rtdm_event_t *event, unsigned long pending)
 {
     xnsynch_init(&event->synch_base, XNSYNCH_PRIO);
+    INIT_LIST_HEAD(&event->nrt_wait_queue);
     if (pending)
         xnsynch_set_flags(&event->synch_base, RTDM_EVENT_PENDING);
 }
 
 void _rtdm_synch_flush(xnsynch_t *synch, unsigned long reason);
+void _rtdm_nrtev_destroy(struct list_head *wait_queue);
 
 static inline void rtdm_event_destroy(rtdm_event_t *event)
 {
     _rtdm_synch_flush(&event->synch_base, XNRMID);
+    _rtdm_nrtev_destroy(&event->nrt_wait_queue);
 }
 
 int rtdm_event_wait(rtdm_event_t *event);
@@ -944,10 +948,11 @@ int rtdm_event_timedwait(rtdm_event_t *e
                          rtdm_toseq_t *timeout_seq);
 void rtdm_event_signal(rtdm_event_t *event);
 
-static inline void rtdm_event_pulse(rtdm_event_t *event)
+/*static inline void rtdm_event_pulse(rtdm_event_t *event)
 {
     _rtdm_synch_flush(&event->synch_base, 0);
-}
+}*/
+void rtdm_event_pulse(rtdm_event_t *event);
 
 void rtdm_event_clear(rtdm_event_t *event);
 
skins/rtdm/drvlib.c +180 -25

259 lines not on this page · raw message

skins/rtdm/module.c +10 -2
Index: ksrc/skins/rtdm/module.c
===================================================================
@@ -49,6 +49,7 @@
 #endif /* __KERNEL__ */
 
 #include "rtdm/device.h"
+#include "rtdm/drvlib.h"
 #include "rtdm/proc.h"
 
 
@@ -97,10 +98,14 @@ int SKIN_INIT(rtdm)
     if (err)
         goto cleanup_pod;
 
+    err = rtdm_nrtev_init();
+    if (err)
+        goto cleanup_dev;
+
 #ifdef CONFIG_PROC_FS
     err = rtdm_proc_init();
     if (err)
-        goto cleanup_dev;
+        goto cleanup_nrtev;
 #endif /* CONFIG_PROC_FS */
 
 #if defined(__KERNEL__) && defined(CONFIG_XENO_OPT_PERVASIVE)
@@ -124,9 +129,12 @@ int SKIN_INIT(rtdm)
 #ifdef CONFIG_PROC_FS
     rtdm_proc_cleanup();
 
-  cleanup_dev:
+  cleanup_nrtev:
 #endif /* CONFIG_PROC_FS */
 
+    rtdm_nrtev_cleanup();
+
+  cleanup_dev:
     rtdm_dev_cleanup();
 
   cleanup_pod:
skins/rtdm/drvlib.h +39 -0
Index: ksrc/skins/rtdm/drvlib.h
===================================================================
@@ -0,0 +1,39 @@
+/*
+ * Copyright (C) 2006 Jan Kiszka <[email protected]>.
+ *
+ * Xenomai is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * Xenomai is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with Xenomai; if not, write to the Free Software Foundation,
+ * Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ */
+
+#ifndef _RTDM_DRVLIB_H
+#define _RTDM_DRVLIB_H
+
+#include <rtdm/rtdm_driver.h>
+
+
+extern rtdm_nrtsig_t nrtevent;
+
+void rtdm_nrtev_handler(rtdm_nrtsig_t nrt_sig);
+
+static inline int rtdm_nrtev_init(void)
+{
+    return rtdm_nrtsig_init(&nrtevent, rtdm_nrtev_handler);
+}
+
+static inline void rtdm_nrtev_cleanup(void)
+{
+    rtdm_nrtsig_destroy(&nrtevent);
+}
+
+#endif /* _RTDM_DRVLIB_H */

#include <linux/module.h>
#include <rtdm/rtdm_driver.h>

rtdm_task_t task;
rtdm_event_t ev;

void task_fnc(void *arg)

{
    rtdm_task_sleep(20*1000000000LL);
    rtdm_event_signal(&ev);
    printk("signalled\n");
}

int nrtev_test(void)

{
    rtdm_event_init(&ev, 0);

    rtdm_task_init(&task, "nrtev_test", task_fnc, NULL, 20, 0);

    printk("waiting...\n");
    printk("res1 = %d\n", rtdm_event_timedwait(&ev, 5000000000LL, NULL));
    printk("res2 = %d\n", rtdm_event_timedwait(&ev, -1, NULL));
    printk("res3 = %d\n", rtdm_event_timedwait(&ev, 0, NULL));

    rtdm_task_join_nrt(&task, 100);
    printk("done\n");

    return -ENOANO;
}

module_init(nrtev_test);

#1 lore raw
Quoted 10 lines · Jan Kiszka

Hi,

attached is an experimental patch to allow blocking from non-RT context on RTDM events. Petr Cervenka made me think about this extension which could partially open RTDM drivers also for non-shadowed, i.e. standard Linux tasks.

Why not simply shadowing non-RT waiters? Because this kind of sync support should be cheaper than adding yet another (though low-prio) task to the real-time domain ("should" == I didn't benchmark my believe). And

I would rather move the logic to the sync object at nucleus level since it's a useful feature for all skins, but doing so, we would also drag a fair amount of complexity just for the sake of not having to shadow a Linux task. IMHO, leaving this at the skin level forks the implementation of the synchronization management, which raises the issue of duplicating functionalities only depending on the calling context.

Looking at the issue differently, we already know that we could shadow SCHED_NORMAL/SCHED_OTHER threads without promoting them to the SCHED_FIFO class - you actually submitted this patch, and it's waiting for -rc2 to be released before I merge it into mainline.

Given that, a shadow thread which remains in secondary mode is seen as suspended by the Xenomai scheduler, which means that the additional overhead is close to nil (aside of an additional shadow tcb), until it tries to grab the resource and thus moves to the primary domain, just for going to sleep.

IOW, I would rather use the allow-non-SCHED_FIFO-shadow patch to allow regular Linux tasks to pend seamlessly on Xenomai sync objects as true shadows, and add to the rtdm blocking syscall handler some code that would transparently promote any calling regular task as a Xenomai shadow on the fly, before undergoing the request. The other advantage of such approach is that the non-RT pending task is still known from the Xenomai sub-system (/proc/xenomai/ interface and so on), which may be useful given that it interacts with it. You could still apply a FIFO pending order with mixed RT and non-RT tasks the same way.

Moving RTDM to native preemption would also be simpler, since RTDM would expect the issue to be handled at core level directly, and in the preempt-rt case, the issue of the caller's domain is actually a non-issue.

Signature · 1 line
Philippe.
author
#2 lore raw
Quoted 24 lines · Philippe Gerum
Quoted 10 lines · Jan Kiszka

Hi,

attached is an experimental patch to allow blocking from non-RT context on RTDM events. Petr Cervenka made me think about this extension which could partially open RTDM drivers also for non-shadowed, i.e. standard Linux tasks.

Why not simply shadowing non-RT waiters? Because this kind of sync support should be cheaper than adding yet another (though low-prio) task to the real-time domain ("should" == I didn't benchmark my believe). And

I would rather move the logic to the sync object at nucleus level since it's a useful feature for all skins, but doing so, we would also drag a fair amount of complexity just for the sake of not having to shadow a Linux task. IMHO, leaving this at the skin level forks the implementation of the synchronization management, which raises the issue of duplicating functionalities only depending on the calling context.

Looking at the issue differently, we already know that we could shadow SCHED_NORMAL/SCHED_OTHER threads without promoting them to the SCHED_FIFO class - you actually submitted this patch, and it's waiting for -rc2 to be released before I merge it into mainline.

Quoted 5 lines

Given that, a shadow thread which remains in secondary mode is seen as suspended by the Xenomai scheduler, which means that the additional overhead is close to nil (aside of an additional shadow tcb), until it tries to grab the resource and thus moves to the primary domain, just for going to sleep.

I'm not concerned about a passive shadow, my aim is to reduce the impact of active shadow threads without strict timing requirements. When there is a larger number of them, they should not influence RT scheduling significantly for just being woken up.

The other point might be the code path length of both non-RT wakeup suspend/wakeup mechanisms. It's only an assessment, but I think this way would be shorter. Anyway, numbers would speak clearer, and I guess this should be done first before any further design decision.

Quoted 10 lines

IOW, I would rather use the allow-non-SCHED_FIFO-shadow patch to allow regular Linux tasks to pend seamlessly on Xenomai sync objects as true shadows, and add to the rtdm blocking syscall handler some code that would transparently promote any calling regular task as a Xenomai shadow on the fly, before undergoing the request. The other advantage of such approach is that the non-RT pending task is still known from the Xenomai sub-system (/proc/xenomai/ interface and so on), which may be useful given that it interacts with it. You could still apply a FIFO pending order with mixed RT and non-RT tasks the same way.

Hmm, on-demand shadowing might be an option. Ok, once non-SCHED_FIFO is merged, we should have a look at both approaches from the performance POV and compare potential differences with the required additional code.

Quoted 5 lines

Moving RTDM to native preemption would also be simpler, since RTDM would expect the issue to be handled at core level directly, and in the preempt-rt case, the issue of the caller's domain is actually a non-issue.

Well, this is already the case for my proposal: no special handling of rtdm_event_* depending on the caller's context anymore.

Jan

#3 lore raw
Quoted 9 lines · Jan Kiszka
Quoted 5 lines · Philippe Gerum

Given that, a shadow thread which remains in secondary mode is seen as suspended by the Xenomai scheduler, which means that the additional overhead is close to nil (aside of an additional shadow tcb), until it tries to grab the resource and thus moves to the primary domain, just for going to sleep.

Quoted 5 lines

I'm not concerned about a passive shadow, my aim is to reduce the impact of active shadow threads without strict timing requirements. When there is a larger number of them, they should not influence RT scheduling significantly for just being woken up.

This issue already exists now, since interrupts are off when Linux performs a context switch, so that we can't wreck the whole thing by allowing a shadow to preempt it in the middle of the mm switch. Since non-RT shadow would run at priority level 0, my feeling is that it would not change the current situation, except perhaps on SMP where the nklock would be held a little more.

Quoted 5 lines

The other point might be the code path length of both non-RT wakeup suspend/wakeup mechanisms. It's only an assessment, but I think this way would be shorter. Anyway, numbers would speak clearer, and I guess this should be done first before any further design decision.

Agreed.

Quoted 17 lines
Quoted 10 lines

IOW, I would rather use the allow-non-SCHED_FIFO-shadow patch to allow regular Linux tasks to pend seamlessly on Xenomai sync objects as true shadows, and add to the rtdm blocking syscall handler some code that would transparently promote any calling regular task as a Xenomai shadow on the fly, before undergoing the request. The other advantage of such approach is that the non-RT pending task is still known from the Xenomai sub-system (/proc/xenomai/ interface and so on), which may be useful given that it interacts with it. You could still apply a FIFO pending order with mixed RT and non-RT tasks the same way.

Hmm, on-demand shadowing might be an option. Ok, once non-SCHED_FIFO is merged, we should have a look at both approaches from the performance POV and compare potential differences with the required additional code.

Quoted 9 lines
Quoted 4 lines

Moving RTDM to native preemption would also be simpler, since RTDM would expect the issue to be handled at core level directly, and in the preempt-rt case, the issue of the caller's domain is actually a non-issue.

Well, this is already the case for my proposal: no special handling of rtdm_event_* depending on the caller's context anymore.

But still, you have to change RTDM's code do implement that, more than you would have to do in promoting a regular task as a non-RT shadow.

Quoted 2 lines

Jan

Signature · 1 line
Philippe.