Common Trace Format Updates (upcoming 1.8.3)
cover Common Trace Format Updates (upcoming 1.8.3)
Hi,
We noticed a few unspecified areas in the CTF 1.8.2 specification which require some clarifications. We therefore plan to do a 1.8.3 release of the specification including those changes.
Review is welcome,
Thanks,
Mathieu
Geneviève Bastien (1):
Clarify that unlisted enum values are implementation-defined
Mathieu Desnoyers (2):
Clarify monotonicity requirement on timestamp begin Clarify that timestamp begin/end need to be complete clock values
common-trace-format-specification.md | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-)
Signature · 6 lines
2.11.0 _______________________________________________ lttng-dev mailing list [email protected] https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
RFC PATCH 1/3 Clarify that unlisted enum values are implementation-defined
From: Geneviève Bastien <[email protected]>
--- common-trace-format-specification.md | 3 +++ 1 file changed, 3 insertions(+)
common-trace-format-specification.md +3 -0
@@ -464,6 +464,9 @@ enum { } ~~~ +The mappings in the enumeration type do not have to be exhaustive. +Unlisted values are implementation defined. + ### 4.2 Compound types Compound are aggregation of type declarations. Compound types include
Signature · 6 lines
2.11.0 _______________________________________________ lttng-dev mailing list [email protected] https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
Re: [RFC PATCH CTF 1/3] Clarify that unlisted enum values are implementation-defined
Quoted 14 lines · Mathieu Desnoyers
From: Geneviève Bastien <[email protected]>
Signed-off-by Geneviève Bastien <[email protected]>Signed-off-by Mathieu Desnoyers <[email protected]>--- common-trace-format-specification.md | 3 +++ 1 file changed, 3 insertions(+)common-trace-format-specification.md +0
-0
common-trace-format-specification.md enum { · line 469
Quoted 6 lines
common-trace-format-specification.md +3
-0@@ -464,6 +464,9 @@ enum { } ~~~ +The mappings in the enumeration type do not have to be exhaustive. +Unlisted values are implementation defined. +
This is too vague to be useful knowing that the main rationale for this change is to allow enums to express some type of bitfield of flags in the tracer and readers [1].
What is the meaning of an unmapped value? This section should at least describe the correct interpretation of unmapped values as flags and when it is appropriate to do so.
Thanks,
Jérémie
Quoted 6 lines
common-trace-format-specification.md +0
-1### 4.2 Compound types Compound are aggregation of type declarations. Compound types include --2.11.0
Signature · 7 lines
Jérémie Galarneau EfficiOS Inc. http://www.efficios.com _______________________________________________ lttng-dev mailing list [email protected] https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
Re: [RFC PATCH CTF 1/3] Clarify that unlisted enum values are implementation-defined
Quoted 28 lines · ----- On Apr 23, 2020, at 6:51 PM, Jeremie Galarneau…
Quoted 15 lines · Mathieu Desnoyers
From: Geneviève Bastien <[email protected]>
Signed-off-by Geneviève Bastien <[email protected]>Signed-off-by Mathieu Desnoyers <[email protected]>--- common-trace-format-specification.md | 3 +++ 1 file changed, 3 insertions(+)common-trace-format-specification.md +0
-0common-trace-format-specification.md enum { · line 469
Quoted 6 lines
common-trace-format-specification.md +3
-0@@ -464,6 +464,9 @@ enum { } ~~~ +The mappings in the enumeration type do not have to be exhaustive. +Unlisted values are implementation defined. +This is too vague to be useful knowing that the main rationale for this change is to allow enums to express some type of bitfield of flags in the tracer and readers [1].
Quoted 3 lines
What is the meaning of an unmapped value? This section should at least describe the correct interpretation of unmapped values as flags and when it is appropriate to do so.
Considering that this is a patchlevel update to CTF, I would not expect that we introduce new features in the specification. Only clarifications to parts of the specification that were unclear.
Specifying a new behavior related to unmapped values would fall IMO into the realm of "new feature", and would belong to a CTF 1.9 or CTF 2.0. As we all know, there is a CTF 2.0 in the making, but it does not solve the immediate problem of LTTng 2.12 which produces those unmapped enum values within traces identified as CTF 1.8.
Moreover, to add to the problem, Babeltrace 2 has a strict match on CTF 1.8 in the CTF source plugin, and won't accept a CTF 1.9 trace (unlike Babeltrace 1 which would emit a warning about possibly unsupported features, but would accept a 1.9 CTF trace nevertheless).
So this is why I am proposing this minimal clarification to the CTF 1.8
specification: that unmapped enum values are implementation defined (rather than saying nothing about them). Considering that tracers can generate this kind of trace data anyway, it's really a consideration that should have been explicitly expressed in the specification from the start and was an involuntary omission.
So perhaps we need to state something more than just "implementation defined", but it's unclear what without ending up adding features into a patchlevel update.
Thanks,
Mathieu
Quoted 17 lines
Thanks,
Jérémie[1] https://review.lttng.org/c/babeltrace/+/3045
Quoted 6 lines
common-trace-format-specification.md +0
-1### 4.2 Compound types Compound are aggregation of type declarations. Compound types include --2.11.0
--
Jérémie Galarneau
EfficiOS Inc.
http://www.efficios.com
Signature · 7 lines
Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com _______________________________________________ lttng-dev mailing list [email protected] https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
Re: [RFC PATCH CTF 1/3] Clarify that unlisted enum values are implementation-defined
In reply to Mathieu Desnoyers via lttng-dev----- Original Message -----
Quoted 24 lines
From: "Mathieu Desnoyers" <[email protected]>
To: "gbastien+lttng" <[email protected]>, "Matthew Khouzam" <[email protected]>, [email protected], [email protected], "Jeremie Galarneau" <[email protected]>
Cc: "lttng-dev" <[email protected]>, "Mathieu Desnoyers" <[email protected]>
Sent: Thursday, 23 April, 2020 16:52:24
Subject: [RFC PATCH CTF 1/3] Clarify that unlisted enum values are implementation-definedFrom: Geneviève Bastien <[email protected]>
Signed-off-by Geneviève Bastien <[email protected]>Signed-off-by Mathieu Desnoyers <[email protected]>---
common-trace-format-specification.md | 3 +++1 file changed, 3 insertions(+)common-trace-format-specification.md +0
-0@@ -464,6 +464,9 @@ enum { }~~~
common-trace-format-specification.md enum { · line 469
Quoted 3 lines
common-trace-format-specification.md +3
-0+The mappings in the enumeration type do not have to be exhaustive. +Unlisted values are implementation defined. +
Why not just:
An enumeration field can have an integral value which its type does not map to a string.
?
Phil
Quoted 5 lines
### 4.2 Compound types
Compound are aggregation of type declarations. Compound types include --
2.11.0
_______________________________________________
lttng-dev mailing list
[email protected]
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
Re: [diamon-discuss] [RFC PATCH CTF 1/3] Clarify that unlisted enum values are implementation-defined
Quoted 35 lines · ----- On Apr 28, 2020, at 2:40 PM, Philippe Proulx [email protected] wrote:
----- Original Message -----
Quoted 28 lines
From: "Mathieu Desnoyers" <[email protected]>
To: "gbastien+lttng" <[email protected]>, "Matthew Khouzam" <[email protected]>, [email protected], [email protected], "Jeremie Galarneau" <[email protected]>
Cc: "lttng-dev" <[email protected]>, "Mathieu Desnoyers" <[email protected]>
Sent: Thursday, 23 April, 2020 16:52:24
Subject: [RFC PATCH CTF 1/3] Clarify that unlisted enum values are implementation-definedFrom: Geneviève Bastien <[email protected]>
Signed-off-by Geneviève Bastien <[email protected]>Signed-off-by Mathieu Desnoyers <[email protected]>---
common-trace-format-specification.md | 3 +++1 file changed, 3 insertions(+)common-trace-format-specification.md +0
-0@@ -464,6 +464,9 @@ enum { }~~~
common-trace-format-specification.md enum { · line 469
Quoted 3 lines
common-trace-format-specification.md +3
-0+The mappings in the enumeration type do not have to be exhaustive. +Unlisted values are implementation defined. +Why not just:
Quoted 4 lines
An enumeration field can have an integral value which its type does not map to a string.
?
Good point, I will use that wording.
Thanks,
Mathieu
Quoted 8 lines
Phil
Quoted 5 lines
### 4.2 Compound types
Compound are aggregation of type declarations. Compound types include --
2.11.0
Signature · 3 lines
Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com
Re: [diamon-discuss] [RFC PATCH CTF 1/3] Clarify that unlisted enum values are implementation-defined
Quoted 37 lines · ----- On Apr 28, 2020, at 2:51 PM, Mathieu Desnoyers…
Quoted 35 lines · ----- On Apr 28, 2020, at 2:40 PM, Philippe Proulx [email protected] wrote:
----- Original Message -----
Quoted 28 lines
From: "Mathieu Desnoyers" <[email protected]>
To: "gbastien+lttng" <[email protected]>, "Matthew Khouzam" <[email protected]>, [email protected], [email protected], "Jeremie Galarneau" <[email protected]>
Cc: "lttng-dev" <[email protected]>, "Mathieu Desnoyers" <[email protected]>
Sent: Thursday, 23 April, 2020 16:52:24
Subject: [RFC PATCH CTF 1/3] Clarify that unlisted enum values are implementation-definedFrom: Geneviève Bastien <[email protected]>
Signed-off-by Geneviève Bastien <[email protected]>Signed-off-by Mathieu Desnoyers <[email protected]>---
common-trace-format-specification.md | 3 +++1 file changed, 3 insertions(+)common-trace-format-specification.md +0
-0@@ -464,6 +464,9 @@ enum { }~~~
common-trace-format-specification.md enum { · line 469
Quoted 3 lines
common-trace-format-specification.md +3
-0+The mappings in the enumeration type do not have to be exhaustive. +Unlisted values are implementation defined. +Why not just:
Quoted 6 lines
Quoted 4 lines
An enumeration field can have an integral value which its type does not map to a string.
?
Good point, I will use that wording.
Geneviève pointed out on IRC that the sentence above is weird. Would the following convey the right meaning ?
"An enumeration field can have an integral value for which the associated enumeration type does not map to a string."
Thanks,
Mathieu
Quoted 18 lines
Thanks,
Mathieu
Quoted 8 lines
Phil
Quoted 5 lines
### 4.2 Compound types
Compound are aggregation of type declarations. Compound types include --
2.11.0--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
Signature · 3 lines
Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com
Re: [diamon-discuss] [RFC PATCH CTF 1/3] Clarify that unlisted enum values are implementation-defined
----- Original Message -----
Quoted 53 lines
From: "Mathieu Desnoyers" <[email protected]>
To: "Philippe Proulx" <[email protected]>
Cc: "gbastien+lttng" <[email protected]>, "Matthew Khouzam" <[email protected]>, "Jeremie Galarneau" <[email protected]>, "lttng-dev" <[email protected]>, "diamon-discuss" <[email protected]>
Sent: Wednesday, 29 April, 2020 08:08:04
Subject: Re: [RFC PATCH CTF 1/3] Clarify that unlisted enum values are implementation-defined----- On Apr 28, 2020, at 2:51 PM, Mathieu Desnoyers
Quoted 42 lines · [email protected] wrote:
Quoted 39 lines · ----- On Apr 28, 2020, at 2:40 PM, Philippe Proulx [email protected] wrote:
----- Original Message -----
Quoted 28 lines
From: "Mathieu Desnoyers" <[email protected]>
To: "gbastien+lttng" <[email protected]>, "Matthew Khouzam" <[email protected]>, [email protected], [email protected], "Jeremie Galarneau" <[email protected]>
Cc: "lttng-dev" <[email protected]>, "Mathieu Desnoyers" <[email protected]>
Sent: Thursday, 23 April, 2020 16:52:24
Subject: [RFC PATCH CTF 1/3] Clarify that unlisted enum values are implementation-definedFrom: Geneviève Bastien <[email protected]>
Signed-off-by Geneviève Bastien <[email protected]>Signed-off-by Mathieu Desnoyers <[email protected]>---
common-trace-format-specification.md | 3 +++1 file changed, 3 insertions(+)common-trace-format-specification.md +0
-0@@ -464,6 +464,9 @@ enum { }~~~
common-trace-format-specification.md enum { · line 469
Quoted 3 lines
common-trace-format-specification.md +3
-0+The mappings in the enumeration type do not have to be exhaustive. +Unlisted values are implementation defined. +Why not just:
An enumeration field can have an integral value which its type does not map to a string.
?
Quoted 8 lines
Quoted 1 line
Good point, I will use that wording.
Geneviève pointed out on IRC that the sentence above is weird. Would the following convey the right meaning ?
"An enumeration field can have an integral value for which the associated enumeration
type does not map to a string."
Sure.
Phil
Quoted 28 lines
Thanks,
Mathieu
Quoted 10 lines
Thanks,
Mathieu
Quoted 5 lines
Phil
Quoted 2 lines
### 4.2 Compound types
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
Compound are aggregation of type declarations. Compound types include --
2.11.0--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
RFC PATCH 2/3 Clarify monotonicity requirement on timestamp begin
In reply to Mathieu Desnoyers via lttng-dev--- common-trace-format-specification.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
common-trace-format-specification.md +6 -4
@@ -836,10 +836,12 @@ TSDL metadata): range between these timestamps should include all event timestamps assigned to events contained within the packet. The timestamp at the beginning of an event packet is guaranteed to be below or equal the - timestamp at the end of that event packet. The timestamp at the end - of an event packet is guaranteed to be below or equal the - timestamps at the end of any following packet within the same stream. - See [Clocks](#spec8) for more detail. + timestamp at the end of that event packet. The timestamp at the + beginning of an event packet is guaranteed to be above or equal the + timestamps at the beginning of any prior packet within the same + stream. The timestamp at the end of an event packet is guaranteed to + be below or equal the timestamps at the end of any following packet + within the same stream. See [Clocks](#spec8) for more detail. * **Events discarded count**. Snapshot of a per-stream free-running counter, counting the number of events discarded that were supposed to be written in the stream after the last event in
Signature · 1 line
2.11.0
Re: [RFC PATCH CTF 2/3] Clarify monotonicity requirement on timestamp begin
----- Original Message -----
Quoted 18 lines
From: "Mathieu Desnoyers" <[email protected]>
To: "gbastien+lttng" <[email protected]>, "Matthew Khouzam" <[email protected]>, [email protected], [email protected], "Jeremie Galarneau" <[email protected]>
Cc: "lttng-dev" <[email protected]>, "Mathieu Desnoyers" <[email protected]>
Sent: Thursday, 23 April, 2020 16:52:25
Subject: [RFC PATCH CTF 2/3] Clarify monotonicity requirement on timestamp beginSigned-off-by Mathieu Desnoyers <[email protected]>---
common-trace-format-specification.md | 10 ++++++----1 file changed, 6 insertions(+), 4 deletions(-)common-trace-format-specification.md +0
-0
common-trace-format-specification.md TSDL metadata): · line 840
Quoted 9 lines
common-trace-format-specification.md +2
-4@@ -836,10 +836,12 @@ TSDL metadata): range between these timestamps should include all event timestamps assigned to events contained within the packet. The timestamp at the beginning of an event packet is guaranteed to be below or equal the - timestamp at the end of that event packet. The timestamp at the end - of an event packet is guaranteed to be below or equal the - timestamps at the end of any following packet within the same stream. - See [Clocks](#spec8) for more detail. + timestamp at the end of that event packet. The timestamp at the + beginning of an event packet is guaranteed to be above or equal the
"greater than or equal to"
common-trace-format-specification.md TSDL metadata): · line 843
Quoted 3 lines
common-trace-format-specification.md +3
-0+ timestamps at the beginning of any prior packet within the same + stream. The timestamp at the end of an event packet is guaranteed to + be below or equal the timestamps at the end of any following packet
"less than or equal to"
Phil
Quoted 6 lines
common-trace-format-specification.md +1
-1+ within the same stream. See [Clocks](#spec8) for more detail. * **Events discarded count**. Snapshot of a per-stream free-running counter, counting the number of events discarded that were supposed to be written in the stream after the last event in --2.11.0
Re: [diamon-discuss] [RFC PATCH CTF 2/3] Clarify monotonicity requirement on timestamp begin
Quoted 25 lines · ----- On Apr 28, 2020, at 2:42 PM, Philippe Proulx [email protected] wrote:
----- Original Message -----
Quoted 24 lines
From: "Mathieu Desnoyers" <[email protected]>
To: "gbastien+lttng" <[email protected]>, "Matthew Khouzam" <[email protected]>,
[email protected], [email protected], "Jeremie Galarneau" <[email protected]>
Cc: "lttng-dev" <[email protected]>, "Mathieu Desnoyers" <[email protected]>
Sent: Thursday, 23 April, 2020 16:52:25
Subject: [RFC PATCH CTF 2/3] Clarify monotonicity requirement on timestamp beginSigned-off-by Mathieu Desnoyers <[email protected]>---
common-trace-format-specification.md | 10 ++++++----1 file changed, 6 insertions(+), 4 deletions(-)common-trace-format-specification.md +0
-0@@ -836,10 +836,12 @@ TSDL metadata): range between these timestamps should include all event timestamps assigned to events contained within the packet. The timestamp at the beginning of an event packet is guaranteed to be below or equal the
Quoted 8 lines
common-trace-format-specification.md TSDL metadata): · line 840
Quoted 6 lines
common-trace-format-specification.md +2
-4- timestamp at the end of that event packet. The timestamp at the end - of an event packet is guaranteed to be below or equal the - timestamps at the end of any following packet within the same stream. - See [Clocks](#spec8) for more detail. + timestamp at the end of that event packet. The timestamp at the + beginning of an event packet is guaranteed to be above or equal the"greater than or equal to"
done
Quoted 6 lines
common-trace-format-specification.md TSDL metadata): · line 843
Quoted 4 lines
common-trace-format-specification.md +3
-0+ timestamps at the beginning of any prior packet within the same + stream. The timestamp at the end of an event packet is guaranteed to + be below or equal the timestamps at the end of any following packet"less than or equal to"
done,
Thanks!
Mathieu
Quoted 9 lines
Phil
Quoted 6 lines
common-trace-format-specification.md +1
-1+ within the same stream. See [Clocks](#spec8) for more detail. * **Events discarded count**. Snapshot of a per-stream free-running counter, counting the number of events discarded that were supposed to be written in the stream after the last event in --2.11.0
Signature · 3 lines
Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com
RFC PATCH 3/3 Clarify that timestamp begin/end need to be complete clock values
In reply to Mathieu Desnoyers via lttng-dev--- common-trace-format-specification.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
common-trace-format-specification.md +4 -3
@@ -1580,7 +1580,8 @@ stream { }; ~~~ -For a N-bit integer type referring to a clock, if the integer overflows +Within the stream event context, event context, and event payload, +fields of N-bit integer type referring to a clock, if the integer overflows compared to the N low order bits of the clock prior value found in the same stream, then it is assumed that one, and only one, overflow occurred. It is therefore important that events encoding time on a small
@@ -1589,8 +1590,8 @@ N-bit overflow occurs. In a packet context, clock field names ending with `_begin` and `_end` have a special meaning: this refers to the timestamps at, respectively, -the beginning and the end of each packet. - +the beginning and the end of each packet. Those are required to be +complete representations of the clock value. ## A. Helper macros
Signature · 1 line
2.11.0