Streamlining code base?
cover Streamlining code base?
Hi Clark and John,
I'd like to streamline all tools, so that the usage, command arguments and the output is more consistent. Every tool seems to be a bit different which makes all automatization around the tools more complex than needed.
Eventually, I'd like to add an easy machine parseable output from all tools. So what do you think? Stupid effort, go away?
Here are a bunch of patches to illustrate what I mean. Before invest too much time I'd like to discuss it, e.g. format.
Cheers,
Daniel
Daniel Wagner (4):
cyclicdeadline: Streamline usage output and man page
deadline_test: Streamline usage output and man page
oslat: Streamline usage output and man page
oslat: Use string parser utilies
src/include/rt-utils.h | 1 + src/lib/rt-utils.c | 30 ++++++ src/oslat/oslat.8 | 2 +- src/oslat/oslat.c | 143 ++++++++-------------------- src/sched_deadline/cyclicdeadline.8 | 24 ++--- src/sched_deadline/cyclicdeadline.c | 41 ++++---- src/sched_deadline/deadline_test.8 | 37 ++++--- src/sched_deadline/deadline_test.c | 47 +++++---- 8 files changed, 145 insertions(+), 180 deletions(-)
Signature · 1 line
2.28.0
PATCH 1/4 cyclicdeadline: Streamline usage output and man page
--- src/sched_deadline/cyclicdeadline.8 | 24 ++++++++--------- src/sched_deadline/cyclicdeadline.c | 41 +++++++++++++++-------------- 2 files changed, 33 insertions(+), 32 deletions(-)
src/sched_deadline/cyclicdeadline.8 +12 -12
@@ -17,7 +17,7 @@ cyclicdeadline \- This program is used to test the deadline scheduler (SCHED_DEA .PP .SH SYNOPSIS .B cyclicdeadline -.RI "[-ha] [-c cpulist] [-i interval] [-s step] [-t nr_threads] [-D duration] +.RI "[-ha] [-c CPUSET] [-D TIME] [-i INTV] [-s STEP] [-t NUM]" .PP .SH DESCRIPTION .B cyclicdeadline
@@ -25,28 +25,28 @@ is a cyclictest style program for testing the deadline scheduler .PP .SH OPTIONS .TP -.B \-h -Show this help menu -.TP .B \-a Use all CPUs .TP -.B \-c cpulist +.B \-c CPUSET Comma / hypen separated list of CPUs to run deadline tasks on .TP -.B \-i interval +.B \-D TIME +Specify a length for the test to run +Append 'm', 'h', or 'd' to specify minutes, hours, or days +.TP +.B \-h +Show this help menu +.TP +.B \-i INTV The shortest deadline for the tasks in us. (default 1000us) .TP -.B \-s step +.B \-s STEP The amount to increase the deadline for each task in us. (default 500us) .TP -.B \-t nr_threads +.B \-t NUM The number of threads to run as deadline (default 1) -.TP -.B \-D time -Specify a length for the test to run .br -Append 'm', 'h', or 'd' to specify minutes, hours, or days .SH AUTHOR cyclicdeadline was written by Steven Rostedt <[email protected]> .PP
src/sched_deadline/cyclicdeadline.c +21 -20
55 lines not on this page · raw message
Signature · 1 line
2.28.0
PATCH 2/4 deadline_test: Streamline usage output and man page
In reply to Daniel Wagner--- src/sched_deadline/deadline_test.8 | 37 ++++++++++++----------- src/sched_deadline/deadline_test.c | 47 +++++++++++++++--------------- 2 files changed, 41 insertions(+), 43 deletions(-)
src/sched_deadline/deadline_test.8 +18 -19
@@ -19,45 +19,44 @@ deadline_test \- High resolution test program This program is used to test the deadline scheduler (SCHED_DEADLINE tasks) .SH SYNOPSIS .B deadline_test -.RI "[ \-hb ] [ \-r prio ] [ \-c cpulist ] [ \-i interval ] [ \-p percent ] [ \-P percent ] [ \-t threads ] [ \-s step[us) ]" - +.RI "[ \-hb ] [ \-c CPUSET ] [ \-i INTV ] [ \-p PERCENT ] [ \-P PERCENT ] \ +[ \-r PRIO ] [ \-s STEP ] [ \-t NUM ]" .SH OPTIONS .TP -.B \-h -Show this help menu -.br -.TP .B \-b Bind on the last cpu. (shortcut for -c <lastcpu>) .br .TP -.B \-r prio -Add an RT task with given prio to stress system +.B \-c CPUSET +Comma/hyphen separated list of CPUs to run deadline tasks on .br .TP -.B \-c cpulist -Comma/hyphen separated list of CPUs to run deadline tasks on +.B \-h +Show this help menu .br .TP -.B \-i interval +.B \-i INTV The shortest deadline for the tasks .br .TP -.B \-p percent +.B \-p PERCENT The percent of bandwidth to use (1-90%) .br .TP -.B \-P percent -The percent of runtime for execution completion - (Default 100%) +.B \-P PERCENT +The percent of runtime for execution completion (default 100%) .br .TP -.B \-t threads -The number of threads to run as deadline (default 1) +.B \-r PRIO +Add an RT task with given prio to stress system .br .TP -.B \-s step(us) -The amount to increase the deadline for each task (default 500us) +.B \-s STEP +The amount to increase the deadline for each task in us (default 500us) +.br +.TP +.B \-t NUM +The number of threads to run as deadline (default 1) .br .SH AUTHOR Deadline test was written by Steven Rostedt <[email protected]>
src/sched_deadline/deadline_test.c +23 -24
64 lines not on this page · raw message
Signature · 1 line
2.28.0
PATCH 3/4 oslat: Streamline usage output and man page
In reply to Daniel Wagner--- src/oslat/oslat.8 | 2 +- src/oslat/oslat.c | 76 +++++++++++++++++++++++------------------------ 2 files changed, 38 insertions(+), 40 deletions(-)
src/oslat/oslat.8 +1 -1
@@ -36,7 +36,7 @@ Total memory usage will be this value multiplies 2*N, because there will be src/dst buffers for each thread, and N is the number of processors for testing. .TP -.B \-t, \-\-runtime=SEC +.B \-D, \-\-duration=TIME Specify test duration, e.g., 60, 20m, 2H (m/M: minutes, h/H: hours, d/D: days). By default the unit is s/second. .TP
src/oslat/oslat.c +37 -39
108 lines not on this page · raw message
Signature · 1 line
2.28.0
PATCH 4/4 oslat: Use string parser utilies
In reply to Daniel WagnerUse available parse_time_string() instead locally implemented version. While at it move the mem parser helper to the global utility header.
--- src/include/rt-utils.h | 1 + src/lib/rt-utils.c | 30 +++++++++++++++++++ src/oslat/oslat.c | 67 ++---------------------------------------- 3 files changed, 33 insertions(+), 65 deletions(-)
src/include/rt-utils.h +1 -0
@@ -28,6 +28,7 @@ uint32_t string_to_policy(const char *str);pid_t gettid (void );int parse_time_string (char *val); +int parse_mem_string (char *str,uint64_t *val);void enable_trace_mark (void );void tracemark (char *fmt, ...) __attribute__((format (printf,1 ,2 )));
src/lib/rt-utils.c +30 -0
@@ -362,6 +362,36 @@ int parse_time_string(char *val)return t; } +int parse_mem_string (char *str,uint64_t *val) +{ +char *endptr; +int v =strtol (str, &endptr,10 ); + +if (!*endptr) +return v; + +switch (*endptr) { +case 'g' : +case 'G' : + v *=1024 ; +case 'm' : +case 'M' : + v *=1024 ; +case 'k' : +case 'K' : + v *=1024 ; +case 'b' : +case 'B' : +break ; +default : +return -1 ; + } + + *val = v; + +return 0 ; +} +void open_tracemark_fd (void ) {char path[MAX_PATH ];
src/oslat/oslat.c +2 -65
85 lines not on this page · raw message
Signature · 1 line
2.28.0
Quoted 9 lines · On Wed, 2 Sep 2020 10:27:21 +0200 Daniel Wagner <[email protected]> wrote:
Hi Clark and John,
I'd like to streamline all tools, so that the usage, command arguments and the output is more consistent. Every tool seems to be a bit different which makes all automatization around the tools more complex than needed.
Eventually, I'd like to add an easy machine parseable output from all tools. So what do you think? Stupid effort, go away?
I think it's a noble goal and I'd be up for it, especially trying to come up with a parsable output format. Got any thoughts on it? Personally I'd go for (in order of preference):
XML
JSON
Random Gibberish
Any Damned thing
YAML
Clark
Did I mention that I hate YAML?
Quoted 26 lines
Here are a bunch of patches to illustrate what I mean. Before invest too much time I'd like to discuss it, e.g. format.
Cheers,
DanielDaniel Wagner (4):
cyclicdeadline: Streamline usage output and man page
deadline_test: Streamline usage output and man page
oslat: Streamline usage output and man page
oslat: Use string parser utiliessrc/include/rt-utils.h | 1 + src/lib/rt-utils.c | 30 ++++++ src/oslat/oslat.8 | 2 +- src/oslat/oslat.c | 143 ++++++++-------------------- src/sched_deadline/cyclicdeadline.8 | 24 ++--- src/sched_deadline/cyclicdeadline.c | 41 ++++---- src/sched_deadline/deadline_test.8 | 37 ++++--- src/sched_deadline/deadline_test.c | 47 +++++---- 8 files changed, 145 insertions(+), 180 deletions(-)Signature · 1 line
2.28.0
Signature · 2 lines
The United States Coast Guard Ruining Natural Selection since 1790
Quoted 9 lines · Clark Williams
I think it's a noble goal and I'd be up for it, especially trying to come up with a parsable output format. Got any thoughts on it? Personally I'd go for (in order of preference):
XML
JSON
Random Gibberish
Any Damned thing
YAML
For jitterdebugger I ended up adding a bunch of plugins for the output format. So I don't have to argue which format it best, though if I had to choose I'd properly pick JSON.
Quoted 1 line
Did I mention that I hate YAML?
Named must be your fear before banish it you can.
Quoted 11 lines · Daniel Wagner
Quoted 9 lines · Clark Williams
I think it's a noble goal and I'd be up for it, especially trying to come up with a parsable output format. Got any thoughts on it? Personally I'd go for (in order of preference):
XML
JSON
Random Gibberish
Any Damned thing
YAML
Quoted 8 lines
For jitterdebugger I ended up adding a bunch of plugins for the output format. So I don't have to argue which format it best, though if I had to choose I'd properly pick JSON.
Quoted 1 line
Did I mention that I hate YAML?
Named must be your fear before banish it you can.
We have some software called rteval that parses the output of cyclictest.
https://git.kernel.org/pub/scm/utils/rteval/rteval.git/
(I'll take patches for that too)
It runs cyclictest under-the-covers like this
cyclictest -qmu -h 2000 -p95 -t -a
and parses the output.
The output is then transformed into XML.
What I like about this, is that it keeps cyclictest lightweight since the data it parses is just text. Then rteval could generate whatever kind of output you are interested in. xml for clark, yaml for the yammering crowd and so on.
I'm not saying this is the only legitimate approach, but I would like to keep the individual tools lightweight in rt-tests, so that you could run them individually in the embedded space as well as on large machines.
So one way to start would be to make sure that the other tools in the rt-tests suite are capable of generating a similar kind of output, that could then be processed by other tools.
John
Quoted 3 lines · John Kacur
I'm not saying this is the only legitimate approach, but I would like to keep the individual tools lightweight in rt-tests, so that you could run them individually in the embedded space as well as on large machines.
Sure, though I still think it would make sense to keep it flexible and have a compile switch if needed. I'd like to avoid additional steps between the measurement and interpretation. Any time we want to add/change a thing, we need to touch more places.
Furthermore, if you look at my implementation for the JSON output of jitterdebugger is really small:
https://github.com/igaw/jitterdebugger/blob/master/jitterdebugger.c#L158
Adding a XML version for this should be in the same size.
If you insist on a single format (XML) that's also okay for me. Python ships all the necessary libraries these days. Though the JSON format is simpler to transform into native data types such as dictionaries and lists, IMO.
Anyway, I don't have any strong feeling and have no real problem with one format and a transform script.
Quoted 3 lines
So one way to start would be to make sure that the other tools in the rt-tests suite are capable of generating a similar kind of output, that could then be processed by other tools.
Yes, I'd like to get consistent output from all tools.
Though I think we should start from the beginning and getting all command line arguments parser, help texts and man pages consistent.
Quoted 13 lines · Daniel Wagner
Quoted 3 lines · John Kacur
I'm not saying this is the only legitimate approach, but I would like to keep the individual tools lightweight in rt-tests, so that you could run them individually in the embedded space as well as on large machines.
Sure, though I still think it would make sense to keep it flexible and have a compile switch if needed. I'd like to avoid additional steps between the measurement and interpretation. Any time we want to add/change a thing, we need to touch more places.
Furthermore, if you look at my implementation for the JSON output of jitterdebugger is really small:
Quoted 8 lines
https://github.com/igaw/jitterdebugger/blob/master/jitterdebugger.c#L158
Adding a XML version for this should be in the same size.
If you insist on a single format (XML) that's also okay for me. Python ships all the necessary libraries these days. Though the JSON format is simpler to transform into native data types such as dictionaries and lists, IMO.
If you just read a text stream like rteval does, and store in a native data type, then it is trivial to output any data format.
However, your implementation for your jitterdebugger is truly small, I don't want to stomp on your creativity, so spin off a git branch and go to it. I can be persuaded.
John