Skip to contentSkip to conversation

Teach babeltrace understand "native" and "network" for byte_order

PATCH Mathieu Desnoyers 2 participants 6 messages Mar 15, 2013 – Mar 19, 2013 5 messages not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw
Quoted 11 lines · * Yao Qi ([email protected]) wrote:

Hi,
We wrote byte_order in metadata either "le" or "be", according to the host endianness. Today, when I check the ctf specification, I find "native" fits our needs well. However, after I replace "le" with "native" in metadata, I got the following error message:

[error] get_trace_byte_order: unexpected string "native". Should be "native", "network", "be" or "le".

This patch teaches babeltrace understand "native" and "network". With this patch applied, the output looks right now. Let me know if I need run testsuite, and how.

your interpretation of what the byte order should accept is based on section 4.1.3 of the CTF spec, which discusses what byte order can be accepted by a type.

However, here you are proposing to change the byte order recognised at the trace declaration level. Unfortunately, we cannot assume anything about native endianness of the traced machine until we have the endianness information at the trace declaration level, as the trace can come from any source.

Possibly that the CTF spec would need to be clarified. Currently, we need to look at:

7.4 TSDL Examples

trace {
...
byte_order = be OR le; /* Endianness (required) */
...
};

to understand that only be and le should be expected in the trace description.

Thoughts ?

Thanks,

Mathieu

Quoted 29 lines
---
 .../ctf/metadata/ctf-visitor-generate-io-struct.c  |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)
formats/ctf/metadata/ctf-visitor-generate-io-struct.c +4 -0
index 9bcc170..cea2877 100644
@@ -1227,6 +1227,10 @@ int get_trace_byte_order(FILE *fd, int depth, struct ctf_node *unary_expression)
 		byte_order = BIG_ENDIAN;
 	else if (!strcmp(unary_expression->u.unary_expression.u.string, "le"))
 		byte_order = LITTLE_ENDIAN;
+	else if (!strcmp(unary_expression->u.unary_expression.u.string, "native"))
+		byte_order = BYTE_ORDER;
+	else if (!strcmp(unary_expression->u.unary_expression.u.string, "network"))
+		byte_order = BIG_ENDIAN;
 	else {
 		fprintf(fd, "[error] %s: unexpected string \"%s\". Should be \"be\" or \"le\".\n",
 			__func__, unary_expression->u.unary_expression.u.string);
Signature · 7 lines
1.7.7.6


_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

A message that is not in the archive (sent off-list or not yet received): 20130315120753.GA30973@Krystal

#3 lore raw
Quoted 34 lines · * Mathieu Desnoyers ([email protected]) wrote:
Quoted 11 lines · * Yao Qi ([email protected]) wrote:

Hi,
We wrote byte_order in metadata either "le" or "be", according to the host endianness. Today, when I check the ctf specification, I find "native" fits our needs well. However, after I replace "le" with "native" in metadata, I got the following error message:

[error] get_trace_byte_order: unexpected string "native". Should be "native", "network", "be" or "le".

This patch teaches babeltrace understand "native" and "network". With this patch applied, the output looks right now. Let me know if I need run testsuite, and how.

your interpretation of what the byte order should accept is based on section 4.1.3 of the CTF spec, which discusses what byte order can be accepted by a type.

However, here you are proposing to change the byte order recognised at the trace declaration level. Unfortunately, we cannot assume anything about native endianness of the traced machine until we have the endianness information at the trace declaration level, as the trace can come from any source.

Possibly that the CTF spec would need to be clarified. Currently, we need to look at:

7.4 TSDL Examples

trace {
...
byte_order = be OR le; /* Endianness (required) */
...
};

Quoted 4 lines

to understand that only be and le should be expected in the trace description.

Thoughts ?

Clarified with CTF commit:

commit 7e58b8bfd86eea7b2929f2ee37dad7e167e11500

Author: Mathieu Desnoyers <[email protected]>
Date:   Fri Mar 15 10:31:37 2013 -0400

Clarify trace byte_order

common-trace-format-specification.txt +5 -0
index 1cd3c3d..4ab3ed9 100644
@@ -181,6 +181,11 @@ TSDL meta-data representation:
 
   byte_order = native OR network OR be OR le;	/* network and be are aliases */
 
+Even though the trace description section is not per se a type, for sake
+of clarity, it should be noted that native and network byte orders are
+only allowed within type declaration. The byte_order specified in the
+trace description section only accepts be OR le values.
+
 4.1.4 Size
 
 Type size, in bits, for integers and floats is that returned by "sizeof()" in C
Quoted 39 lines

Thanks,

Mathieu

Quoted 29 lines
---
 .../ctf/metadata/ctf-visitor-generate-io-struct.c  |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)
formats/ctf/metadata/ctf-visitor-generate-io-struct.c +4 -0
index 9bcc170..cea2877 100644
@@ -1227,6 +1227,10 @@ int get_trace_byte_order(FILE *fd, int depth, struct ctf_node *unary_expression)
 		byte_order = BIG_ENDIAN;
 	else if (!strcmp(unary_expression->u.unary_expression.u.string, "le"))
 		byte_order = LITTLE_ENDIAN;
+	else if (!strcmp(unary_expression->u.unary_expression.u.string, "native"))
+		byte_order = BYTE_ORDER;
+	else if (!strcmp(unary_expression->u.unary_expression.u.string, "network"))
+		byte_order = BIG_ENDIAN;
 	else {
 		fprintf(fd, "[error] %s: unexpected string \"%s\". Should be \"be\" or \"le\".\n",
 			__func__, unary_expression->u.unary_expression.u.string);
Signature · 7 lines
1.7.7.6


_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#4 lore raw
In reply to a message not in the archive
Quoted 9 lines · Mathieu Desnoyers

your interpretation of what the byte order should accept is based on section 4.1.3 of the CTF spec, which discusses what byte order can be accepted by a type.

However, here you are proposing to change the byte order recognised at the trace declaration level. Unfortunately, we cannot assume anything about native endianness of the traced machine until we have the endianness information at the trace declaration level, as the trace can come from any source.

It is clear to me now. Thanks for the clarification.

Quoted 3 lines

Possibly that the CTF spec would need to be clarified. Currently, we need to look at:

The doc part is fine, but this error message is confusing,

[error] get_trace_byte_order: unexpected string "native". Should be "native", "network", "be" or "le".

Looks it was fixed. When I update my tree, the error message is updated.

I'll give a comment in your comment to CTF spec.

Signature · 6 lines
Yao (齐尧)

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

A message that is not in the archive (sent off-list or not yet received): 20130315143337.GA959@Krystal

#6 lore raw

Review

Quoted 7 lines · Mathieu Desnoyers

byte_order = native OR network OR be OR le; /* network and be are aliases */

(quoted diff) +5 -0
+Even though the trace description section is not per se a type, for sake
+of clarity, it should be noted that native and network byte orders are
+only allowed within type declaration. The byte_order specified in the
+trace description section only accepts be OR le values.
+

The first and last sentence below is a little bit confusing to me.

4.1.3 Byte order

By default, the native endianness of the source architecture is used.
....
If not specified, the byte order is native.

1. "native endianness" appear in the spec for the first time, without a precise explanation. After your explanation in the last mail, I got to know it is the endianess described in the trace description.

2. "source architecture" is not clear to me.

Maybe, we can say something like:

"By default, byte order of a basic type is the byte order described in the trace description. It can be overridden by specifying a "byte_order" attribute for a basic type. Typical use-case is to specify the network byte order (big endian: "be") to save data captured from the network into the trace without conversion."

I also suggest that we get rid of the term "native" here. IIUC, we have four options for byte order, default, network, le and be. The latter three are easy to understand, and if attribute "byte_order" is not specified, it is the default. On the other hand, "native endianness" means the endianness of underlying machine, which is a little different.

I am not good at documentation, nor a native English speaker. My suggestion can be wrong.

Signature · 6 lines
Yao (齐尧)

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

A message that is not in the archive (sent off-list or not yet received): [email protected]

#8 lore raw
Quoted 32 lines · * Yao Qi ([email protected]) wrote:

Review

Quoted 7 lines · Mathieu Desnoyers

byte_order = native OR network OR be OR le; /* network and be are aliases */

(quoted diff) +5 -0
+Even though the trace description section is not per se a type, for sake
+of clarity, it should be noted that native and network byte orders are
+only allowed within type declaration. The byte_order specified in the
+trace description section only accepts be OR le values.
+

The first and last sentence below is a little bit confusing to me.

4.1.3 Byte order

By default, the native endianness of the source architecture is used.
....
If not specified, the byte order is native.

1. "native endianness" appear in the spec for the first time, without a precise explanation. After your explanation in the last mail, I got to know it is the endianess described in the trace description.

2. "source architecture" is not clear to me.

Maybe, we can say something like:

"By default, byte order of a basic type is the byte order described in the trace description. It can be overridden by specifying a "byte_order" attribute for a basic type. Typical use-case is to specify the network byte order (big endian: "be") to save data captured from the network into the trace without conversion."

Quoted 5 lines

I also suggest that we get rid of the term "native" here. IIUC, we have four options for byte order, default, network, le and be. The latter three are easy to understand, and if attribute "byte_order" is not specified, it is the default. On the other hand, "native endianness" means the endianness of underlying machine, which is a little different.

I added an explanatory paragraph for "native" in addition to your recommended changes. We cannot remove an keyword from the specification lightly, because this would break compatibility.

Quoted 3 lines

I am not good at documentation, nor a native English speaker. My suggestion can be wrong.

Here is the update. Comments are welcome!

Thanks,

Mathieu

commit 237324a20398c5c8eaf92aff0eace177ef8a747f

Author: Mathieu Desnoyers <[email protected]>
Date:   Fri Mar 15 11:32:03 2013 -0400

Clarify byte order section

common-trace-format-specification.txt +11 -8
index 4ab3ed9..797dec1 100644
@@ -171,20 +171,23 @@ TSDL meta-data attribute representation of a specific alignment:
 
 4.1.3 Byte order
 
-By default, the native endianness of the source architecture is used.
-Byte order can be overridden for a basic type by specifying a "byte_order"
-attribute. Typical use-case is to specify the network byte order (big endian:
-"be") to save data captured from the network into the trace without conversion.
-If not specified, the byte order is native.
+By default, byte order of a basic type is the byte order described in
+the trace description.  It can be overridden by specifying a
+"byte_order" attribute for a basic type.  Typical use-case is to specify
+the network byte order (big endian: "be") to save data captured from the
+network into the trace without conversion.
 
 TSDL meta-data representation:
 
   byte_order = native OR network OR be OR le;	/* network and be are aliases */
 
+The "native" keyword selects the byte order described in the trace
+description. The "network" byte order is an alias for big endian.
+
 Even though the trace description section is not per se a type, for sake
-of clarity, it should be noted that native and network byte orders are
-only allowed within type declaration. The byte_order specified in the
-trace description section only accepts be OR le values.
+of clarity, it should be noted that "native" and "network" byte orders
+are only allowed within type declaration. The byte_order specified in
+the trace description section only accepts "be" or "le" values.
 
 4.1.4 Size
 
Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

A message that is not in the archive (sent off-list or not yet received): 20130315153320.GA1659@Krystal

#10 lore raw
Quoted 3 lines · Mathieu Desnoyers

I added an explanatory paragraph for "native" in addition to your recommended changes. We cannot remove an keyword from the specification lightly, because this would break compatibility.

It looks pretty good. Thanks for doing this.

Signature · 6 lines
Yao (齐尧)

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev