tester: Add framework for serialized test execution
RFC PATCH 1/1 tester: Add framework for serialized test execution
This adds a framework for running series of noninteractive tests that are capable of executing more complex scenarios than a usual unit test.
The framework enables staging of a test case processing:
- test pre-setup: can be used to prepare an initial testing environment potentially common to between several test cases, e.g., put an emulator in a certain well known state.
- test setup: used to setup conditions specific for the test case
- test run: execute actual test case
- test teardown: Cleans up after test setup
- test post-teardown: Cleans up after test pre-setup
Additional command line options allow to run a selected group of tests based either on a group prefix of the test name or a substring in test description.
--- Makefile.am | 6 +- ell/tester.c | 786 +++++++++++++++++++++++++++++++++++++++++++++++++++ ell/tester.h | 94 ++++++ 3 files changed, 884 insertions(+), 2 deletions(-) create mode 100644 ell/tester.c create mode 100644 ell/tester.h
Makefile.am +4 -2
@@ -58,7 +58,8 @@ pkginclude_HEADERS = ell/ell.h \ ell/gpio.h \ ell/path.h \ ell/icmp6.h \ - ell/acd.h + ell/acd.h \ + ell/tester.h lib_LTLIBRARIES = ell/libell.la
@@ -141,7 +142,8 @@ ell_libell_la_SOURCES = $(linux_headers) \ ell/path.c \ ell/icmp6.c \ ell/icmp6-private.h \ - ell/acd.c + ell/acd.c \ + ell/tester.c ell_libell_la_LDFLAGS = -Wl,--no-undefined \ -Wl,--version-script=$(top_srcdir)/ell/ell.sym \
ell/tester.c new file +786 -0
786 lines not on this page · raw message
ell/tester.h new file +94 -0
94 lines not on this page · raw message
Signature · 1 line
2.26.2
Hi Inga,
Quoted 13 lines · Inga Stotland
This adds a framework for running series of noninteractive tests that are capable of executing more complex scenarios than a usual unit test.
The framework enables staging of a test case processing:
- test pre-setup: can be used to prepare an initial testing environment potentially common to between several test cases, e.g., put an emulator in a certain well known state.
- test setup: used to setup conditions specific for the test case
- test run: execute actual test case
- test teardown: Cleans up after test setup
- test post-teardown: Cleans up after test pre-setup
Quoted 7 lines
Additional command line options allow to run a selected group of tests based either on a group prefix of the test name or a substring in test description.
--- Makefile.am | 6 +- ell/tester.c | 786 +++++++++++++++++++++++++++++++++++++++++++++++++++ ell/tester.h | 94 ++++++
Hmm, ell.sym entries?
Quoted 4 lines
3 files changed, 884 insertions(+), 2 deletions(-)create mode 100644
ell/tester.c
create mode 100644ell/tester.h
<snip>
ell/tester.c line 104
Quoted 13 lines
ell/tester.c new file +13
-0+ +static char *tester_name; + +static struct l_queue *test_list; +static const struct l_queue_entry *test_entry; +static struct timeval tester_start; + +static bool option_list; +static const char *option_prefix; +static const char *option_string; + +static bool terminated; +
Curious. This tester framework is part of the main ell library, so I would have assumed it is a 'class' so to speak. But you use static vars here. So is l_tester only used from a dedicated binary?
If not, do the l_tester_* signatures need to be updated to take an l_tester instance and appropriate constructors / destructors defined?
<snip>
Review
Quoted 2 lines
(quoted diff) +2
-0+ +void *l_tester_get_data(void)
LIB_EXPORT?
Quoted 8 lines
ell/tester.c new file +8
-0+{ +struct test_case *test; + +if (!test_entry) +return NULL ; + + test = test_entry->data; +
ell/tester.c line 156
Quoted 10 lines
ell/tester.c new file +10
-0+return test->user_data; +} + +static double get_elapsed_time (struct timeval *base) +{ +static struct timeval now, elapsed; + +gettimeofday (&now,NULL ); +timersub (&now, base, &elapsed); +
Have you considered using l_time* from ell/time.h?
ell/tester.c line 159
Quoted 3 lines
ell/tester.c new file +3
-0+return elapsed.tv_sec + ((double ) elapsed.tv_usec) /1000000 ; +} +
<snip>
The rest of the code seems straightforward and nothing else really jumped out at me.
Regards,
-Denis
Stotland
Hi Denis,
On Mon, 2021-03-08 at 16:19 -0600, Denis Kenzior wrote:
Hi Inga,
On 3/5/21 3:37 PM, Inga Stotland wrote:
This adds a framework for running series of noninteractive tests
that are capable of executing more complex scenarios than a usual unit
test.
The framework enables staging of a test case processing:
- test pre-setup: can be used to prepare an initial testing environment
potentially common to between several test cases, e.g.,
put an emulator in a certain well known state.
- test setup: used to setup conditions specific for the test case
- test run: execute actual test case
- test teardown: Cleans up after test setup
- test post-teardown: Cleans up after test pre-setup
Additional command line options allow to run a selected group of tests
based either on a group prefix of the test name or a substring in test
description.
---
Makefile.am | 6 +-
ell/tester.c | 786 +++++++++++++++++++++++++++++++++++++++++++++++++++
ell/tester.h | 94 ++++++
Hmm, ell.sym entries?
3 files changed, 884 insertions(+), 2 deletions(-)
create mode 100644 ell/tester.c
create mode 100644 ell/tester.h
<snip>
+
+static char *tester_name;
+
+static struct l_queue *test_list;
+static const struct l_queue_entry *test_entry;
+static struct timeval tester_start;
+
+static bool option_list;
+static const char *option_prefix;
+static const char *option_string;
+
+static bool terminated;
+
Curious. This tester framework is part of the main ell library, so I would have
assumed it is a 'class' so to speak. But you use static vars here. So is
l_tester only used from a dedicated binary?
If not, do the l_tester_* signatures need to be updated to take an l_tester
instance and appropriate constructors / destructors defined?
<snip>
+
+void *l_tester_get_data(void)
LIB_EXPORT?
+{
+ struct test_case *test;
+
+ if (!test_entry)
+ return NULL;
+
+ test = test_entry->data;
+
+ return test->user_data;
+}
+
+static double get_elapsed_time(struct timeval *base)
+{
+ static struct timeval now, elapsed;
+
+ gettimeofday(&now, NULL);
+ timersub(&now, base, &elapsed);
+
Have you considered using l_time* from ell/time.h?
+ return elapsed.tv_sec + ((double) elapsed.tv_usec) / 1000000;
+}
+
<snip>
The rest of the code seems straightforward and nothing else really jumped out at me.
Regards,
-Denis
Thanks for reviewing. Will work on v2.
Regards,
Inga