Host realtime clock support

Augment the shared vdso page with all data required to implement a clock CLOCK_HOST_REALTIME that provides time information synchronised with the NTP-corrected time in the Linux kernel.

The definition of the hostrt data is placed in a separate head file so that we can use it irregardless of pervasive rt support is compiled in or not.

Quoted 5 lines · Wolfgang Mauerer

Augment the shared vdso page with all data required to implement a clock CLOCK_HOST_REALTIME that provides time information synchronised with the NTP-corrected time in the Linux kernel.

Quoted 3 lines

The definition of the hostrt data is placed in a separate head file so that we can use it irregardless of pervasive rt support is compiled in or not.

Ok. Would not it be more simple to define the nkvdso even without pervasive support?

ksrc/nucleus/module.c int __init __xeno_sys_init(void) · line 134

Quoted 3 lines

ksrc/nucleus/module.c +2 -0

+#if defined(CONFIG_XENO_OPT_HOSTRT) && !defined(CONFIG_XENO_OPT_PERVASIVE)
+	nkhostrt_data = xnarch_alloc_host_mem(sizeof(struct xnarch_hostrt_data));

should be sizeof(*nkhostrt_data)

But why a dynamic allocation at all?

Quoted 6 lines · Gilles Chanteperdrix

Quoted 5 lines · Wolfgang Mauerer

Augment the shared vdso page with all data required to implement a clock CLOCK_HOST_REALTIME that provides time information synchronised with the NTP-corrected time in the Linux kernel.

Quoted 5 lines

Quoted 3 lines

The definition of the hostrt data is placed in a separate head file so that we can use it irregardless of pervasive rt support is compiled in or not.

Ok. Would not it be more simple to define the nkvdso even without pervasive support?

Thought about that, but having a shared kernel/userland object without pervasive RT support seemed a bit unnatural to me.

Quoted 12 lines · Wolfgang Mauerer

Quoted 11 lines · Gilles Chanteperdrix

Quoted 8 lines · Wolfgang Mauerer

Augment the shared vdso page with all data required to implement a clock CLOCK_HOST_REALTIME that provides time information synchronised with the NTP-corrected time in the Linux kernel.

The definition of the hostrt data is placed in a separate head file so that we can use it irregardless of pervasive rt support is compiled in or not.

Ok. Would not it be more simple to define the nkvdso even without pervasive support?

Quoted 2 lines

Thought about that, but having a shared kernel/userland object without pervasive RT support seemed a bit unnatural to me.

The idea would be to have a static nkvdso in the non-pervasive case, we do not need the mapping stuff. But at least, if we do this once and for all for nkvdso, we will not have to repeat the code which handles the non-pervasive case for all the data in nkvdso which need to exist in that case.

Wall time management is typically assisted by the NTP protocol in the Linux context, but this information is not propagated to Xenomai. This patch adds support for a CLOCK_HOST_REALTIME clock id that is coupled to the host operating system's realtime clock. The required information from the Kernel into Xenomai. The data exchange is designed to allow for lockless reading from userland.

Signed-off-by Wolfgang Mauerer <[email protected]>

Signed-off-by Jan Kiszka <[email protected]>

---
 include/posix/time.h     |    2 +
 ksrc/skins/posix/clock.c |   90 +++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 91 insertions(+), 1 deletions(-)

include/posix/time.h +2 -0

index 4f2d760..938feb6 100644
@@ -50,6 +50,8 @@
 #define CLOCK_MONOTONIC 1
 #endif /* CLOCK_MONOTONIC */
 
+#define CLOCK_HOST_REALTIME 16
+
 #if defined(__KERNEL__) || defined(__XENO_SIM__)
 
 struct sigevent;

ksrc/skins/posix/clock.c +0 -0

index 553e123..f5a789b 100644

Quoted 35 lines · Wolfgang Mauerer

Wall time management is typically assisted by the NTP protocol in the Linux context, but this information is not propagated to Xenomai. This patch adds support for a CLOCK_HOST_REALTIME clock id that is coupled to the host operating system's realtime clock. The required information from the Kernel into Xenomai. The data exchange is designed to allow for lockless reading from userland.

Signed-off-by Wolfgang Mauerer <[email protected]>

Signed-off-by Jan Kiszka <[email protected]>

---
 include/posix/time.h     |    2 +
 ksrc/skins/posix/clock.c |   90 +++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 91 insertions(+), 1 deletions(-)

include/posix/time.h +2 -0

index 4f2d760..938feb6 100644
@@ -50,6 +50,8 @@
 #define CLOCK_MONOTONIC 1
 #endif /* CLOCK_MONOTONIC */
 
+#define CLOCK_HOST_REALTIME 16
+
 #if defined(__KERNEL__) || defined(__XENO_SIM__)
 
 struct sigevent;

ksrc/skins/posix/clock.c +0 -0

index 553e123..f5a789b 100644
@@ -50,6 +50,13 @@
  *@{*/
 

ksrc/skins/posix/clock.c line 54

Quoted 3 lines

ksrc/skins/posix/clock.c +2 -0

 #include <posix/thread.h>
+#include <linux/ipipe_tickdev.h>
+#include <linux/math64.h>

Do we really need this? Because we probably do not have it in 2.4 kernels. Besides, we do not include linux/ headers in Xenomai code.

ksrc/skins/posix/clock.c int clock_getres(clockid_t clock_id, struct timespec *res) · line 163

Quoted 7 lines

ksrc/skins/posix/clock.c +6 -0

+static int do_clock_host_realtime(struct timespec *tp)
+{
+#ifdef CONFIG_XENO_OPT_HOSTRT
(...)
+	return 0;
+#else /* CONFIG_XENO_OPT_HOSTRT */
+	return -1;

return -EINVAL;

ksrc/skins/posix/clock.c int clock_gettime(clockid_t clock_id, struct timespec *tp) · line 211

Quoted 2 lines

ksrc/skins/posix/clock.c +2 -0

+	case CLOCK_HOST_REALTIME:
+		if (do_clock_host_realtime(tp) != 0) {

		rc = do_clock_host_realtime(tp);
		if (rc < 0) {
			thread_set_errno(-rc);
			return -1;
		}

... and for reading the contents of the hostrt data.

Quoted 2 lines · Wolfgang Mauerer

... and for reading the contents of the hostrt data.

Just realized: Please also update clocktest. Should already work with the new clock ID, but requires a cosmetic output patch.

Quoted 5 lines · Jan Kiszka

Quoted 2 lines · Wolfgang Mauerer

... and for reading the contents of the hostrt data.

Just realized: Please also update clocktest. Should already work with the new clock ID, but requires a cosmetic output patch.

Yes, I thought about that. Maybe even if clocktest uses the new clock_id, we could drop the example?

I do not think it is a good idea to provide as an example code which uses the internals of the implementation (though it makes sense to use the implementation internals in the clocktest code). And if we remove that, only clock_gettime(CLOCK_HOST_REALTIME) remains, which does not seem to need an example.

Quoted 6 lines · Gilles Chanteperdrix

Quoted 5 lines · Jan Kiszka

Quoted 2 lines · Wolfgang Mauerer

... and for reading the contents of the hostrt data.

Just realized: Please also update clocktest. Should already work with the new clock ID, but requires a cosmetic output patch.

Quoted 9 lines

Yes, I thought about that. Maybe even if clocktest uses the new clock_id, we could drop the example?

I do not think it is a good idea to provide as an example code which uses the internals of the implementation (though it makes sense to use the implementation internals in the clocktest code). And if we remove that, only clock_gettime(CLOCK_HOST_REALTIME) remains, which does not seem to need an example.

I would not keep it as an example, but as a test case for checking that nothing is wrong with the data on the shared page. Which does not fit quite well into clocktest. Although I have no problem with just omitting the stuff if reading the (internal) content of the data page is deemed inappropriate.

Quoted 16 lines · Wolfgang Mauerer

Quoted 6 lines · Gilles Chanteperdrix

Quoted 5 lines · Jan Kiszka

Quoted 2 lines · Wolfgang Mauerer

... and for reading the contents of the hostrt data.

Just realized: Please also update clocktest. Should already work with the new clock ID, but requires a cosmetic output patch.

Quoted 9 lines

Yes, I thought about that. Maybe even if clocktest uses the new clock_id, we could drop the example?

I do not think it is a good idea to provide as an example code which uses the internals of the implementation (though it makes sense to use the implementation internals in the clocktest code). And if we remove that, only clock_gettime(CLOCK_HOST_REALTIME) remains, which does not seem to need an example.

Quoted 5 lines

I would not keep it as an example, but as a test case for checking that nothing is wrong with the data on the shared page. Which does not fit quite well into clocktest. Although I have no problem with just omitting the stuff if reading the (internal) content of the data page is deemed inappropriate.

Well, I see nothing wrong with checking the internals at the beginning of clocktest is the clockid passed to clocktest is CLOCK_HOST_REALTIME. But Jan may have another opinion, since he is the one who wrote this code.

Quoted 2 lines · Wolfgang Mauerer

... and for reading the contents of the hostrt data.

Just realized: Please also update clocktest. Should already work with the new clock ID, but requires a cosmetic output patch.

(This is the Xenomai part of the mechanism, please see the ipipe
mailing list for the patches that provide the required basis infrastructure)

This patch series extends Xenomai with a new clock, CLOCK_HOST_REALTIME. It allows for sharing NTP-corrected real time timestamps between Linux/ipipe and Xenomai. The data are also available in userspace and can be read without switching to kernel mode. Notice, however, that the new clock only enables to read to time, but cannot serve as a full time basis. Some changes to the ipipe layer are required as basis.

In contrast to the initial approach, we don't use a transactional mechanism to copy the information over from Linux, but use classical synchronisation. The code can be compiled in conditionally for both, ipipe and Xenomai. When disabled by architectures that don't support apt clock sources, there is no runtime-overhead associated with the feature.

Some points that may require further discussion:

- POSIX only specifies a few clock_ids, and these have already been extended by the Linux kernel. We use the maximum id (16) for the new clock, but it might also make sense to use 7 (CLOCK_MONOTONIC_COARSE+1) or 4 (CLOCK_THREAD_CPUTIME_ID+1).

- The current implementation deals with x86_64's TSC. Support for other architectures can be added. Additionally, the user has to make sure that the TSC clock source remains active once selected. To implement deactivation (e.g., when the Linux clock source is changed), more ipipe hooks would be required, though.

There are two alternatives including other architectures:

* We can create a new clocksource that abstracts the per-architecture differences, and use this clocksource as basis for Xenomai. Essentially, this means mapping all desired non-x86-Clocksources to the interface offered in this patch. This requires more changes in the ipipe layer than variant B, namely,
* We can create a union in struct xnvdso of all arch-specific clock datasets and introduce feature flags like XNVDSO_FEAT_HOSTRT_X86, XNVDSO_FEAT_HOSTRT_WHATEVER. The reader-side code then needs to match the data provided, which requires more changes on the Xenomai side.

Quoted 32 lines · Wolfgang Mauerer

(This is the Xenomai part of the mechanism, please see the ipipe
mailing list for the patches that provide the required basis infrastructure)

This patch series extends Xenomai with a new clock, CLOCK_HOST_REALTIME. It allows for sharing NTP-corrected real time timestamps between Linux/ipipe and Xenomai. The data are also available in userspace and can be read without switching to kernel mode. Notice, however, that the new clock only enables to read to time, but cannot serve as a full time basis. Some changes to the ipipe layer are required as basis.

In contrast to the initial approach, we don't use a transactional mechanism to copy the information over from Linux, but use classical synchronisation. The code can be compiled in conditionally for both, ipipe and Xenomai. When disabled by architectures that don't support apt clock sources, there is no runtime-overhead associated with the feature.

Some points that may require further discussion:

- POSIX only specifies a few clock_ids, and these have already been extended by the Linux kernel. We use the maximum id (16) for the new clock, but it might also make sense to use 7 (CLOCK_MONOTONIC_COARSE+1) or 4 (CLOCK_THREAD_CPUTIME_ID+1).

- The current implementation deals with x86_64's TSC. Support for other architectures can be added. Additionally, the user has to make sure that the TSC clock source remains active once selected. To implement deactivation (e.g., when the Linux clock source is changed), more ipipe hooks would be required, though.

There are two alternatives including other architectures:

Quoted 10 lines

* We can create a new clocksource that abstracts the per-architecture differences, and use this clocksource as basis for Xenomai. Essentially, this means mapping all desired non-x86-Clocksources to the interface offered in this patch. This requires more changes in the ipipe layer than variant B, namely,
* We can create a union in struct xnvdso of all arch-specific clock datasets and introduce feature flags like XNVDSO_FEAT_HOSTRT_X86, XNVDSO_FEAT_HOSTRT_WHATEVER. The reader-side code then needs to match the data provided, which requires more changes on the Xenomai side.

The dataset is the same on all architectures, since we provide the same "clocksource abstraction" on the 5 architectures we support: a TSC emulation. So, a simple approach is simply cut-and-paste the x86 update_vsyscall code for other architectures, another approach is to put this code in a wrapper which we call on all architectures.

This update_vsyscall code would call ipipe_dispatch_event to pass the data to Xenomai.

Quoted 43 lines · Gilles Chanteperdrix

Quoted 32 lines · Wolfgang Mauerer

(This is the Xenomai part of the mechanism, please see the ipipe
mailing list for the patches that provide the required basis infrastructure)

This patch series extends Xenomai with a new clock, CLOCK_HOST_REALTIME. It allows for sharing NTP-corrected real time timestamps between Linux/ipipe and Xenomai. The data are also available in userspace and can be read without switching to kernel mode. Notice, however, that the new clock only enables to read to time, but cannot serve as a full time basis. Some changes to the ipipe layer are required as basis.

In contrast to the initial approach, we don't use a transactional mechanism to copy the information over from Linux, but use classical synchronisation. The code can be compiled in conditionally for both, ipipe and Xenomai. When disabled by architectures that don't support apt clock sources, there is no runtime-overhead associated with the feature.

Some points that may require further discussion:

- POSIX only specifies a few clock_ids, and these have already been extended by the Linux kernel. We use the maximum id (16) for the new clock, but it might also make sense to use 7 (CLOCK_MONOTONIC_COARSE+1) or 4 (CLOCK_THREAD_CPUTIME_ID+1).

- The current implementation deals with x86_64's TSC. Support for other architectures can be added. Additionally, the user has to make sure that the TSC clock source remains active once selected. To implement deactivation (e.g., when the Linux clock source is changed), more ipipe hooks would be required, though.

There are two alternatives including other architectures:

Quoted 10 lines

* We can create a new clocksource that abstracts the per-architecture differences, and use this clocksource as basis for Xenomai. Essentially, this means mapping all desired non-x86-Clocksources to the interface offered in this patch. This requires more changes in the ipipe layer than variant B, namely,
* We can create a union in struct xnvdso of all arch-specific clock datasets and introduce feature flags like XNVDSO_FEAT_HOSTRT_X86, XNVDSO_FEAT_HOSTRT_WHATEVER. The reader-side code then needs to match the data provided, which requires more changes on the Xenomai side.

Quoted 5 lines

The dataset is the same on all architectures, since we provide the same "clocksource abstraction" on the 5 architectures we support: a TSC emulation. So, a simple approach is simply cut-and-paste the x86 update_vsyscall code for other architectures, another approach is to put this code in a wrapper which we call on all architectures.

this disables the possibility of using non-TSC time sources that can also be accessed from userland. That's certainly not a requirement for us, I'd just like to mention it.

We also need to ensure that the host has not switched away from TSC because in this case, the NTP correction values delivered from Linux are for a different clocksource than the one used by Xenomai. So we need to detect a switch from TSC to non-TSC, which can only be done in arch-specific code -- I don't see a generic way to know which clocksources are based on the TSC and which one are not.

Quoted 3 lines

This update_vsyscall code would call ipipe_dispatch_event to pass the data to Xenomai.

okay. That seems to have dropped from my radar, as I don't recall having any issues with this requirement.

Quoted 49 lines · Wolfgang Mauerer

Quoted 48 lines · Gilles Chanteperdrix

Quoted 32 lines · Wolfgang Mauerer

(This is the Xenomai part of the mechanism, please see the ipipe
mailing list for the patches that provide the required basis infrastructure)

This patch series extends Xenomai with a new clock, CLOCK_HOST_REALTIME. It allows for sharing NTP-corrected real time timestamps between Linux/ipipe and Xenomai. The data are also available in userspace and can be read without switching to kernel mode. Notice, however, that the new clock only enables to read to time, but cannot serve as a full time basis. Some changes to the ipipe layer are required as basis.

In contrast to the initial approach, we don't use a transactional mechanism to copy the information over from Linux, but use classical synchronisation. The code can be compiled in conditionally for both, ipipe and Xenomai. When disabled by architectures that don't support apt clock sources, there is no runtime-overhead associated with the feature.

Some points that may require further discussion:

- POSIX only specifies a few clock_ids, and these have already been extended by the Linux kernel. We use the maximum id (16) for the new clock, but it might also make sense to use 7 (CLOCK_MONOTONIC_COARSE+1) or 4 (CLOCK_THREAD_CPUTIME_ID+1).

- The current implementation deals with x86_64's TSC. Support for other architectures can be added. Additionally, the user has to make sure that the TSC clock source remains active once selected. To implement deactivation (e.g., when the Linux clock source is changed), more ipipe hooks would be required, though.

There are two alternatives including other architectures:

Quoted 10 lines

* We can create a new clocksource that abstracts the per-architecture differences, and use this clocksource as basis for Xenomai. Essentially, this means mapping all desired non-x86-Clocksources to the interface offered in this patch. This requires more changes in the ipipe layer than variant B, namely,
* We can create a union in struct xnvdso of all arch-specific clock datasets and introduce feature flags like XNVDSO_FEAT_HOSTRT_X86, XNVDSO_FEAT_HOSTRT_WHATEVER. The reader-side code then needs to match the data provided, which requires more changes on the Xenomai side.

The dataset is the same on all architectures, since we provide the same "clocksource abstraction" on the 5 architectures we support: a TSC emulation. So, a simple approach is simply cut-and-paste the x86 update_vsyscall code for other architectures, another approach is to put this code in a wrapper which we call on all architectures.

Quoted 3 lines

this disables the possibility of using non-TSC time sources that can also be accessed from userland. That's certainly not a requirement for us, I'd just like to mention it.

This would only make sense on x86, as other architectures seem to have only one clock-source. And since other clocksources than TSC have a higher overhead, I do not think it is likely that we would like to use them.

Anyway, we can start with TSC and if the need arises, fix that later.

Quoted 7 lines

We also need to ensure that the host has not switched away from TSC because in this case, the NTP correction values delivered from Linux are for a different clocksource than the one used by Xenomai. So we need to detect a switch from TSC to non-TSC, which can only be done in arch-specific code -- I don't see a generic way to know which clocksources are based on the TSC and which one are not.

Well, all architectures call ipipe_request_tickdev, so, we can check if the clock pointer passed to update_vsyscall is the same as the one returned by ipipe_request_tickdev. Do not know if this is really possible, I would need to check more closely how ipipe_request_tickdev works.

Quoted 2 lines · Wolfgang Mauerer

Quoted 1 line · Gilles Chanteperdrix

Wolfgang Mauerer wrote:

Quoted 5 lines

Quoted 3 lines

This update_vsyscall code would call ipipe_dispatch_event to pass the data to Xenomai.

okay. That seems to have dropped from my radar, as I don't recall having any issues with this requirement.

I am sure that I always thought that it should be done this way. But maybe I did not say it... Sorry if that is the case.

Some points that may require further discussion:

- POSIX only specifies a few clock_ids, and these have already been extended by the Linux kernel. We use the maximum id (16) for the new clock, but it might also make sense to use 7 (CLOCK_MONOTONIC_COARSE+1) or 4 (CLOCK_THREAD_CPUTIME_ID+1).

Quoted 6 lines · Wolfgang Mauerer

Some points that may require further discussion:

- POSIX only specifies a few clock_ids, and these have already been extended by the Linux kernel. We use the maximum id (16) for the new clock, but it might also make sense to use 7 (CLOCK_MONOTONIC_COARSE+1) or 4 (CLOCK_THREAD_CPUTIME_ID+1).

Why are we limited to 16 clocks? I mean we do not go trough any kernel/glibc path, so it looks like we can use any number. Or did I miss something?