Skip to contentSkip to conversation

test: Complete the suite migration

PATCH Superseded Simon Glass 2 participants 32 messages Jan 20, 2025 – Jan 23, 2025 Kbuild

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover test: Complete the suite migration

This series completes the removal of test commands for suites. With this it is possible to declare a suite (including init and uninit functions) without needing to write a command.

It also adds timing for test suites, so we can keep track of how long things take.

Simon Glass (28):
test: Fix a stray asterisk in ut_run_list()
test: Add up the number of tests manually
test: Keep track of suite duration
test: Show the average time per test
test: Leave out the prefix when printing test names
test: Support an init/uninit functions for test suites
test: Tweak FDT-overlay tests
test: Move env-test rule into test/
test: Move optee-test rule into test/
test: Move fdt-overlay-test rule into test/
test: Make all tests depend on UNIT_TEST
test: Move fdt_overlay init into a function
test: Update fdt_overlay test to use fdtdec functions
test: Update fdt_overlay to do init from tests
test: Drop the function for running fdt_overlay tests
test: Update bootstd to do init from tests
test: Drop the function for running bootstd tests
test: Update optee to do init and uninit from tests
test: Split optee tests into three functions
test: Drop the function for running optee tests
test: Rename optee test-file
test: Drop support for test commands
test: Make cmd_ut_category() static
test: Drop suites.h
test: Allow running a selection of suites
test: Move code out of cmd_ut_category()
test: Do flag-processing in the correct place
test: Update documentation

 Makefile                                      |   3 -
 doc/develop/tests_writing.rst                 |  52 ++--
 doc/usage/cmd/ut.rst                          | 145 ++++++++---
 drivers/timer/sandbox_timer.c                 |   5 +
 include/test/fdt_overlay.h                    |  19 ++
 include/test/optee.h                          |   4 +-
 include/test/overlay.h                        |  15 --
 include/test/suites.h                         |  43 ----
 include/test/test.h                           |  28 ++
 include/time.h                                |  15 +-
 test/Kconfig                                  |  23 +-
 test/Makefile                                 |   3 +
 test/boot/bootdev.c                           |   1 -
 test/boot/bootflow.c                          |   1 -
 test/boot/bootm.c                             |   1 -
 test/boot/bootmeth.c                          |   1 -
 test/boot/bootstd_common.c                    |  29 +--
 test/boot/bootstd_common.h                    |  10 +-
 test/boot/expo.c                              |   1 -
 test/boot/image.c                             |   1 -
 test/boot/measurement.c                       |   1 -
 test/boot/upl.c                               |   1 -
 test/boot/vbe_simple.c                        |   4 -
 test/cmd/addrmap.c                            |   1 -
 test/cmd/bdinfo.c                             |   1 -
 test/cmd/exit.c                               |   1 -
 test/cmd/fdt.c                                |   1 -
 test/cmd/font.c                               |   1 -
 test/cmd/loadm.c                              |   1 -
 test/cmd/mbr.c                                |   1 -
 test/cmd/pci_mps.c                            |   1 -
 test/cmd/seama.c                              |   1 -
 test/cmd/setexpr.c                            |   1 -
 test/cmd_ut.c                                 | 243 ++++++++++--------
 test/common/bloblist.c                        |   1 -
 test/dm/Kconfig                               |   2 +-
 test/env/Kconfig                              |   1 -
 test/env/cmd_ut_env.c                         |   1 -
 test/{overlay => fdt_overlay}/Kconfig         |   4 +-
 test/{overlay => fdt_overlay}/Makefile        |   2 +-
 .../cmd_ut_fdt_overlay.c}                     | 221 +++++++---------
 .../test-fdt-base.dts                         |   0
 .../test-fdt-overlay-stacked.dtso             |   0
 .../test-fdt-overlay.dtso                     |   0
 test/log/cont_test.c                          |   1 -
 test/log/nolog_test.c                         |   1 -
 test/log/pr_cont_test.c                       |   1 -
 test/log/syslog_test.c                        |   1 -
 test/log/syslog_test_ndebug.c                 |   1 -
 test/optee/Kconfig                            |   2 +-
 test/optee/Makefile                           |   2 +-
 test/optee/{cmd_ut_optee.c => optee.c}        | 105 +++++---
 test/py/tests/test_suite.py                   |  34 ++-
 test/test-main.c                              |  41 ++-
 54 files changed, 587 insertions(+), 493 deletions(-)
 create mode 100644 include/test/fdt_overlay.h
 delete mode 100644 include/test/overlay.h
 delete mode 100644 include/test/suites.h
 rename test/{overlay => fdt_overlay}/Kconfig (81%)
 rename test/{overlay => fdt_overlay}/Makefile (89%)
 rename test/{overlay/cmd_ut_overlay.c => fdt_overlay/cmd_ut_fdt_overlay.c} (56%)
 rename test/{overlay => fdt_overlay}/test-fdt-base.dts (100%)
 rename test/{overlay => fdt_overlay}/test-fdt-overlay-stacked.dtso (100%)
 rename test/{overlay => fdt_overlay}/test-fdt-overlay.dtso (100%)
 rename test/optee/{cmd_ut_optee.c => optee.c} (70%)
Signature · 1 line
2.43.0
#1 lore raw

PATCH 1/28 test: Fix a stray asterisk in ut_run_list()

Drop the unwanted asterisk in the comment.

---

 test/test-main.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
test/test-main.c +1 -1
index 22b9b46d9cd..815f54bebd5 100644
@@ -701,7 +701,7 @@ int ut_run_list(struct unit_test_state *uts, const char *category,
 		has_dm_tests = true;
 		/*
 		 * If we have no device tree, or it only has a root node, then
-		 * these * tests clearly aren't going to work...
+		 * these tests clearly aren't going to work...
 		 */
 		if (!gd->fdt_blob || fdt_next_node(gd->fdt_blob, 0, NULL) < 0) {
 			puts("Please run with test device tree:\n"
Signature · 1 line
2.43.0
#2 lore raw

PATCH 2/28 test: Add up the number of tests manually

In reply to Simon Glass

All tests should belong to a suite, but if there is a suite we don't know about (e.g. not added to cmd_ut.c) then the totals will not add up. Add a check for this.

---

 test/cmd_ut.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
test/cmd_ut.c +8 -2
index fbfdaaae0b5..cc30c517c51 100644
@@ -233,11 +233,11 @@ static int do_ut_info(struct cmd_tbl *cmdtp, int flag, int argc,
 
 	flags = cmd_arg1(argc, argv);
 	if (flags && !strcmp("-s", flags)) {
-		int i;
+		int i, total;
 
 		puts("\nTests  Suite         Purpose");
 		puts("\n-----  ------------  -------------------------\n");
-		for (i = 0; i < ARRAY_SIZE(suites); i++) {
+		for (i = 0, total = 0; i < ARRAY_SIZE(suites); i++) {
 			struct suite *ste = &suites[i];
 			long n_ent = ste->end - ste->start;
 
@@ -248,7 +248,13 @@ static int do_ut_info(struct cmd_tbl *cmdtp, int flag, int argc,
 			else  /* suite is not present */
 				continue;
 			printf("  %-13.13s %s\n", ste->name, ste->help);
+			total += n_ent;
 		}
+		puts("-----  ------------  -------------------------\n");
+		printf("%5d  %-13.13s\n", total, "Total");
+
+		if (UNIT_TEST_ALL_COUNT() != total)
+			puts("Error: Suite test-count does not match total\n");
 	}
 
 	return 0;
Signature · 1 line
2.43.0
#3 lore raw

PATCH 3/28 test: Keep track of suite duration

In reply to Simon Glass

Show the time taken by each test suite with 'ut all' and the total time for all suites.

Take care to remove any sandbox time-offset from the values.

Fix the comment-format on timer_test_add_offset() while we are here.

---

 drivers/timer/sandbox_timer.c |  5 +++++
 include/test/test.h           |  4 ++++
 include/time.h                | 15 ++++++++++++++-
 test/Kconfig                  |  9 +++++++++
 test/test-main.c              | 17 ++++++++++++++++-
 5 files changed, 48 insertions(+), 2 deletions(-)
drivers/timer/sandbox_timer.c +5 -0
index e8b54a02965..c1baf3c69ec 100644
@@ -18,6 +18,11 @@ void timer_test_add_offset(unsigned long offset)
 	sandbox_timer_offset += offset;
 }
 
+ulong timer_test_get_offset(void)
+{
+	return sandbox_timer_offset;
+};
+
 u64 notrace timer_early_get_count(void)
 {
 	return os_get_nsec() / 1000 + sandbox_timer_offset * 1000;
include/test/test.h +4 -0
index bac43c81d63..25c7712d160 100644
@@ -16,11 +16,15 @@
  * @skip_count: Number of tests that were skipped
  * @test_count: Number of tests run. If a test is run muiltiple times, only one
  *	is counted
+ * @start: Timer value when test started
+ * @duration_ms: Suite duration in milliseconds
  */
 struct ut_stats {
 	int fail_count;
 	int skip_count;
 	int test_count;
+	ulong start;
+	ulong duration_ms;
 };
 
 /*
include/time.h +14 -1
index 3b2ba091247..f5b86bf70fe 100644
@@ -28,7 +28,7 @@ uint64_t get_timer_us(uint64_t base);
  */
 unsigned long get_timer_us_long(unsigned long base);
 
-/*
+/**
  * timer_test_add_offset()
  *
  * Allow tests to add to the time reported through lib/time.c functions
@@ -36,6 +36,19 @@ unsigned long get_timer_us_long(unsigned long base);
  */
 void timer_test_add_offset(unsigned long offset);
 
+#ifdef CONFIG_SANDBOX
+/**
+ * timer_test_get_offset()
+ *
+ * Get the total offset currently being added the time
+ *
+ * Return:: number of milliseconds the system time has been advanced
+ */
+ulong timer_test_get_offset(void);
+#else
+static inline ulong timer_test_get_offset(void) { return 0; }
+#endif
+
 /**
  * usec_to_tick() - convert microseconds to clock ticks
  *
test/Kconfig +9 -0
index 558a9cd49b4..1c8e3d16300 100644
@@ -20,6 +20,15 @@ config SPL_UNIT_TEST
 	  of-platdata and SPL handover. To run these tests with the sandbox_spl
 	  board, use the -u (unit test) option.
 
+config UNIT_TEST_DURATION
+	bool "Report unit-test duration"
+	depends on UNIT_TEST
+	default y
+	help
+	  Enable this short the time taken by each test suite. This is reported
+	  after the suite runs, alongside the pass/fail results. In addition,
+	  an overall total is reported if multiple suites are run.
+
 config UT_LIB
 	bool "Unit tests for library functions"
 	depends on UNIT_TEST
test/test-main.c +16 -1
index 815f54bebd5..1d821a3fe72 100644
@@ -14,6 +14,7 @@
 #include <net.h>
 #include <of_live.h>
 #include <os.h>
+#include <spl.h>
 #include <usb.h>
 #include <dm/ofnode.h>
 #include <dm/root.h>
@@ -680,6 +681,8 @@ void ut_report(struct ut_stats *stats, int run_count)
 	else
 		printf("Tests");
 	printf(" run: %d, ", stats->test_count);
+	if (stats)
+		printf("%ld ms, ", stats->duration_ms);
 	if (stats->skip_count)
 		printf("skipped: %d, ", stats->skip_count);
 	printf("failures: %d\n", stats->fail_count);
@@ -692,9 +695,15 @@ int ut_run_list(struct unit_test_state *uts, const char *category,
 {
 	;
 	bool has_dm_tests = false;
+	ulong start_offset = 0;
+	ulong test_offset = 0;
 	int ret;
 
 	memset(&uts->cur, '\0', sizeof(struct ut_stats));
+	if (CONFIG_IS_ENABLED(UNIT_TEST_DURATION)) {
+		uts->cur.start = get_timer(0);
+		start_offset = timer_test_get_offset();
+	}
 
 	if (!CONFIG_IS_ENABLED(OF_PLATDATA) &&
 	    ut_list_has_dm_tests(tests, count, prefix, select_name)) {
@@ -732,13 +741,19 @@ int ut_run_list(struct unit_test_state *uts, const char *category,
 	if (has_dm_tests)
 		dm_test_restore(uts->of_root);
 
-	ut_report(&uts->cur, 1);
 	if (ret == -ENOENT)
 		printf("Test '%s' not found\n", select_name);
+	if (CONFIG_IS_ENABLED(UNIT_TEST_DURATION)) {
+		test_offset = timer_test_get_offset() - start_offset;
+
+		uts->cur.duration_ms = get_timer(uts->cur.start) - test_offset;
+	}
+	ut_report(&uts->cur, 1);
 
 	uts->total.skip_count += uts->cur.skip_count;
 	uts->total.fail_count += uts->cur.fail_count;
 	uts->total.test_count += uts->cur.test_count;
+	uts->total.duration_ms += uts->cur.duration_ms;
 	uts->run_count++;
 
 	return ret;
Signature · 1 line
2.43.0
#4 lore raw

PATCH 4/28 test: Show the average time per test

In reply to Simon Glass

Show the average duration of a test, so we can keep track of how it is trending. Report the suite with the longest average test to encourage people to improve it.

Add a function to update the stats based on the results from a single suite and another to show the summary information.

Make this optional, since sandbox's SPL tests do not have a timer driver and people may want to print results without times.

---

 include/test/test.h |  4 ++++
 test/cmd_ut.c       | 32 ++++++++++++++++++++++++++++++++
 test/test-main.c    | 10 +++++++---
 3 files changed, 43 insertions(+), 3 deletions(-)
include/test/test.h +4 -0
index 25c7712d160..877fda8d5aa 100644
@@ -33,6 +33,8 @@ struct ut_stats {
  * @cur: Statistics for the current run
  * @total: Statistics for all test runs
  * @run_count: Number of times ut_run_list() has been called
+ * @worst: Sute which had the first per-text run time
+ * @worst_ms: Time taken by that test
  * @start: Store the starting mallinfo when doing leak test
  * @of_live: true to use livetree if available, false to use flattree
  * @of_root: Record of the livetree root node (used for setting up tests)
@@ -56,6 +58,8 @@ struct unit_test_state {
 	struct ut_stats cur;
 	struct ut_stats total;
 	int run_count;
+	const struct suite *worst;
+	int worst_ms;
 	struct mallinfo start;
 	struct device_node *of_root;
 	bool of_live;
test/cmd_ut.c +32 -0
index cc30c517c51..c96277d89a1 100644
@@ -192,6 +192,36 @@ static int run_suite(struct unit_test_state *uts, struct suite *ste,
 	return ret;
 }
 
+static void show_stats(struct unit_test_state *uts)
+{
+	if (uts->run_count < 2)
+		return;
+
+	ut_report(&uts->total, uts->run_count);
+	if (CONFIG_IS_ENABLED(UNIT_TEST_DURATION) &&
+	    uts->total.test_count && uts->worst) {
+		ulong avg = uts->total.duration_ms / uts->total.test_count;
+
+		printf("Average test time: %ld ms, worst case '%s' took %d ms\n",
+		       avg, uts->worst->name, uts->worst_ms);
+	}
+}
+
+static void update_stats(struct unit_test_state *uts, const struct suite *ste)
+{
+	if (CONFIG_IS_ENABLED(UNIT_TEST_DURATION) && uts->cur.test_count) {
+		ulong avg;
+
+		avg = uts->cur.duration_ms ?
+			uts->cur.duration_ms /
+			uts->cur.test_count : 0;
+		if (avg > uts->worst_ms) {
+			uts->worst_ms = avg;
+			uts->worst = ste;
+		}
+	}
+}
+
 static int do_ut_all(struct unit_test_state *uts, struct cmd_tbl *cmdtp,
 		     int flag, int argc, char *const argv[])
 {
@@ -208,6 +238,7 @@ static int do_ut_all(struct unit_test_state *uts, struct cmd_tbl *cmdtp,
 			retval = run_suite(uts, ste, cmdtp, flag, 1, argv);
 			if (!any_fail)
 				any_fail = retval;
+			update_stats(uts, ste);
 		}
 	}
 	ut_report(&uts->total, uts->run_count);
@@ -306,6 +337,7 @@ static int do_ut(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[])
 
 		ret = run_suite(&uts, ste, cmdtp, flag, argc, argv);
 	}
+	show_stats(&uts);
 	if (ret)
 		return ret;
 	ut_uninit_state(&uts);
test/test-main.c +7 -3
index 1d821a3fe72..597afa25f77 100644
@@ -677,12 +677,16 @@ static int ut_run_tests(struct unit_test_state *uts, const char *prefix,
 void ut_report(struct ut_stats *stats, int run_count)
 {
 	if (run_count > 1)
-		printf("Suites run: %d, total tests", run_count);
+		printf("Total tests");
 	else
 		printf("Tests");
 	printf(" run: %d, ", stats->test_count);
-	if (stats)
-		printf("%ld ms, ", stats->duration_ms);
+	if (stats && stats->test_count) {
+		ulong dur = stats->duration_ms;
+
+		printf("%ld ms, average: %ld ms, ", dur,
+		       dur ? dur / stats->test_count : 0);
+	}
 	if (stats->skip_count)
 		printf("skipped: %d, ", stats->skip_count);
 	printf("failures: %d\n", stats->fail_count);
Signature · 1 line
2.43.0
#5 lore raw

PATCH 5/28 test: Leave out the prefix when printing test names

In reply to Simon Glass

When tests are all in the same suite it is annoying to have to read all the common text after each name. Skip this to help the user.

---

 test/cmd_ut.c               |  2 +-
 test/py/tests/test_suite.py |  7 ++++---
 test/test-main.c            | 15 ++++++++++-----
 3 files changed, 15 insertions(+), 9 deletions(-)
test/cmd_ut.c +1 -1
index c96277d89a1..0b923ee7e2e 100644
@@ -184,7 +184,7 @@ static int run_suite(struct unit_test_state *uts, struct suite *ste,
 		char prefix[30];
 
 		/* use a standard prefix */
-		snprintf(prefix, sizeof(prefix), "%s_test", ste->name);
+		snprintf(prefix, sizeof(prefix), "%s_test_", ste->name);
 		ret = cmd_ut_category(uts, ste->name, prefix, ste->start,
 				      n_ents, argc, argv);
 	}
test/py/tests/test_suite.py +4 -3
index 73c185349b4..ae127301fd7 100644
@@ -66,11 +66,12 @@ def collect_info(cons, output):
             msg = m.group(3)
             if DEBUG_ME:
                 cons.log.info(f"test_name {test_name} msg '{msg}'")
-            if msg == ' (flat tree)' and test_name not in tests:
-                tests.add(test_name)
+            full_name = f'{cur_suite}.{test_name}'
+            if msg == ' (flat tree)' and full_name not in tests:
+                tests.add(full_name)
                 test_count += 1
             if not msg or 'skipped as it is manual' in msg:
-                tests.add(test_name)
+                tests.add(full_name)
                 test_count += 1
         if DEBUG_ME:
             cons.log.info(f'test_count {test_count}')
test/test-main.c +10 -5
index 597afa25f77..8c0d820032c 100644
@@ -514,11 +514,12 @@ static int ut_run_test(struct unit_test_state *uts, struct unit_test *test,
  *	the first call to this function. On exit, @uts->cur.fail_count is
  *	incremented by the number of failures (0, one hopes)
  * @test: Test to run
+ * @leaf: Part of the name to show, or NULL to use test->name
  * Return: 0 if all tests passed, -EAGAIN if the test should be skipped, -1 if
  *	any failed
  */
 static int ut_run_test_live_flat(struct unit_test_state *uts,
-				 struct unit_test *test)
+				 struct unit_test *test, const char *leaf)
 {
 	int runs, ret;
 
@@ -530,7 +531,7 @@ static int ut_run_test_live_flat(struct unit_test_state *uts,
 	if (CONFIG_IS_ENABLED(OF_LIVE)) {
 		if (!(test->flags & UTF_FLAT_TREE)) {
 			uts->of_live = true;
-			ret = ut_run_test(uts, test, test->name);
+			ret = ut_run_test(uts, test, leaf ?: test->name);
 			if (ret != -EAGAIN) {
 				ut_assertok(ret);
 				runs++;
@@ -558,7 +559,7 @@ static int ut_run_test_live_flat(struct unit_test_state *uts,
 	    (!runs || ut_test_run_on_flattree(test)) &&
 	    !(gd->flags & GD_FLG_FDT_CHANGED)) {
 		uts->of_live = false;
-		ret = ut_run_test(uts, test, test->name);
+		ret = ut_run_test(uts, test, leaf ?: test->name);
 		if (ret != -EAGAIN) {
 			ut_assertok(ret);
 			runs++;
@@ -594,6 +595,7 @@ static int ut_run_tests(struct unit_test_state *uts, const char *prefix,
 			struct unit_test *tests, int count,
 			const char *select_name, const char *test_insert)
 {
+	int prefix_len = prefix ? strlen(prefix) : 0;
 	struct unit_test *test, *one;
 	int found = 0;
 	int pos = 0;
@@ -646,7 +648,7 @@ static int ut_run_tests(struct unit_test_state *uts, const char *prefix,
 
 		uts->cur.test_count++;
 		if (one && upto == pos) {
-			ret = ut_run_test_live_flat(uts, one);
+			ret = ut_run_test_live_flat(uts, one, NULL);
 			if (uts->cur.fail_count != old_fail_count) {
 				printf("Test '%s' failed %d times (position %d)\n",
 				       one->name,
@@ -656,8 +658,11 @@ static int ut_run_tests(struct unit_test_state *uts, const char *prefix,
 			return -EBADF;
 		}
 
+		if (prefix_len && !strncmp(test_name, prefix, prefix_len))
+			test_name = test_name + prefix_len;
+
 		for (i = 0; i < uts->runs_per_test; i++)
-			ret = ut_run_test_live_flat(uts, test);
+			ret = ut_run_test_live_flat(uts, test, test_name);
 		if (uts->cur.fail_count != old_fail_count) {
 			printf("Test '%s' failed %d times\n", test_name,
 			       uts->cur.fail_count - old_fail_count);
Signature · 1 line
2.43.0
#6 lore raw

PATCH 6/28 test: Support an init/uninit functions for test suites

In reply to Simon Glass

Some suites need things to be set up before they can run. Add a way to declare an init function using the UNIT_TEST_INIT() macro. The init function is just like any other test, but is always placed first so that it runs before all the other test functions in the suite.

Add an uninit function as well, to clean up after the test.

---

 include/test/test.h | 20 ++++++++++++++++++++
 test/test-main.c    |  3 ++-
 2 files changed, 22 insertions(+), 1 deletion(-)
include/test/test.h +20 -0
index 877fda8d5aa..0f2b68a5dee 100644
@@ -100,6 +100,8 @@ enum ut_flags {
 	UTF_ETH_BOOTDEV	= BIT(9),	/* enable Ethernet bootdevs */
 	UTF_SF_BOOTDEV	= BIT(10),	/* enable SPI flash bootdevs */
 	UFT_BLOBLIST	= BIT(11),	/* test changes gd->bloblist */
+	UTF_INIT	= BIT(12),	/* test inits a suite */
+	UTF_UNINIT	= BIT(13),	/* test uninits a suite */
 };
 
 /**
@@ -147,6 +149,24 @@ struct unit_test {
 		.func = _name,						\
 	}
 
+/* init function for unit-test suite (the 'A' makes it first) */
+#define UNIT_TEST_INIT(_name, _flags, _suite)				\
+	ll_entry_declare(struct unit_test, A ## _name, ut_ ## _suite) = {	\
+		.file = __FILE__,					\
+		.name = #_name,						\
+		.flags = (_flags) | UTF_INIT,				\
+		.func = _name,						\
+	}
+
+/* uninit function for unit-test suite (the 'aaa' makes it last) */
+#define UNIT_TEST_UNINIT(_name, _flags, _suite)				\
+	ll_entry_declare(struct unit_test, zzz ## _name, ut_ ## _suite) = { \
+		.file = __FILE__,					\
+		.name = #_name,						\
+		.flags = (_flags) | UTF_UNINIT,				\
+		.func = _name,						\
+	}
+
 /* Get the start of a list of unit tests for a particular suite */
 #define UNIT_TEST_SUITE_START(_suite) \
 	ll_entry_start(struct unit_test, ut_ ## _suite)
test/test-main.c +2 -1
index 8c0d820032c..ee855bfe2ed 100644
@@ -620,7 +620,8 @@ static int ut_run_tests(struct unit_test_state *uts, const char *prefix,
 		const char *test_name = test->name;
 		int ret, i, old_fail_count;
 
-		if (!test_matches(prefix, test_name, select_name))
+		if (!(test->flags & (UTF_INIT | UTF_UNINIT)) &&
+		    !test_matches(prefix, test_name, select_name))
 			continue;
 
 		if (test->flags & UTF_MANUAL) {
Signature · 1 line
2.43.0
#7 lore raw

PATCH 7/28 test: Tweak FDT-overlay tests

In reply to Simon Glass

Use fdt_overlay consistently in the identifiers and file/dir names.

---

 Makefile                                      |  2 +-
 include/test/{overlay.h => fdt_overlay.h}     |  4 +-
 include/test/suites.h                         |  4 +-
 test/Kconfig                                  |  2 +-
 test/cmd_ut.c                                 |  8 +--
 test/{overlay => fdt_overlay}/Kconfig         |  2 +-
 test/{overlay => fdt_overlay}/Makefile        |  2 +-
 .../cmd_ut_fdt_overlay.c}                     | 49 ++++++++++---------
 .../test-fdt-base.dts                         |  0
 .../test-fdt-overlay-stacked.dtso             |  0
 .../test-fdt-overlay.dtso                     |  0
 test/py/tests/test_suite.py                   |  4 +-
 12 files changed, 39 insertions(+), 38 deletions(-)
 rename include/test/{overlay.h => fdt_overlay.h} (66%)
 rename test/{overlay => fdt_overlay}/Kconfig (93%)
 rename test/{overlay => fdt_overlay}/Makefile (89%)
 rename test/{overlay/cmd_ut_overlay.c => fdt_overlay/cmd_ut_fdt_overlay.c} (77%)
 rename test/{overlay => fdt_overlay}/test-fdt-base.dts (100%)
 rename test/{overlay => fdt_overlay}/test-fdt-overlay-stacked.dtso (100%)
 rename test/{overlay => fdt_overlay}/test-fdt-overlay.dtso (100%)
Makefile +1 -1
index 43966004d5f..1e29796602b 100644
@@ -895,7 +895,7 @@ endif
 libs-$(CONFIG_$(PHASE_)UNIT_TEST) += test/
 libs-$(CONFIG_UT_ENV) += test/env/
 libs-$(CONFIG_UT_OPTEE) += test/optee/
-libs-$(CONFIG_UT_OVERLAY) += test/overlay/
+libs-$(CONFIG_UT_FDT_OVERLAY) += test/fdt_overlay/
 
 libs-y += $(if $(wildcard $(srctree)/board/$(BOARDDIR)/Makefile),board/$(BOARDDIR)/)
 
include/test/fdt_overlay.h renamed from include/test/overlay.h +2 -2
similarity index 66%
rename from include/test/overlay.h
rename to include/test/fdt_overlay.h
index 5dc98399ce7..34e8020a496 100644
@@ -9,7 +9,7 @@
 
 #include <test/test.h>
 
-/* Declare a new environment test */
-#define OVERLAY_TEST(_name, _flags)	UNIT_TEST(_name, _flags, overlay)
+/* Declare a new FDT-overlay test */
+#define FDT_OVERLAY_TEST(_name, _flags)	UNIT_TEST(_name, _flags, fdt_overlay)
 
 #endif /* __TEST_OVERLAY_H__ */
include/test/suites.h +2 -2
index 774dd893378..dce720b4e78 100644
@@ -36,8 +36,8 @@ int cmd_ut_category(struct unit_test_state *uts, const char *name,
 
 int do_ut_bootstd(struct unit_test_state *uts, struct cmd_tbl *cmdtp, int flag,
 		  int argc, char *const argv[]);
+int do_ut_fdt_overlay(struct unit_test_state *uts, struct cmd_tbl *cmdtp,
+		      int flag, int argc, char *const argv[]);
 int do_ut_optee(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[]);
-int do_ut_overlay(struct unit_test_state *uts, struct cmd_tbl *cmdtp, int flag,
-		  int argc, char *const argv[]);
 
 #endif /* __TEST_SUITES_H__ */
test/Kconfig +1 -1
index 1c8e3d16300..01d64642fdb 100644
@@ -120,7 +120,7 @@ source "test/env/Kconfig"
 source "test/image/Kconfig"
 source "test/lib/Kconfig"
 source "test/optee/Kconfig"
-source "test/overlay/Kconfig"
+source "test/fdt_overlay/Kconfig"
 
 config POST
 	bool "Power On Self Test support"
test/cmd_ut.c +4 -4
index 0b923ee7e2e..958b591c605 100644
@@ -105,6 +105,7 @@ SUITE_DECL(dm);
 SUITE_DECL(env);
 SUITE_DECL(exit);
 SUITE_DECL(fdt);
+SUITE_DECL(fdt_overlay);
 SUITE_DECL(font);
 SUITE_DECL(hush);
 SUITE_DECL(lib);
@@ -114,7 +115,6 @@ SUITE_DECL(mbr);
 SUITE_DECL(measurement);
 SUITE_DECL(mem);
 SUITE_DECL(optee);
-SUITE_DECL(overlay);
 SUITE_DECL(pci_mps);
 SUITE_DECL(seama);
 SUITE_DECL(setexpr);
@@ -134,6 +134,9 @@ static struct suite suites[] = {
 	SUITE(env, "environment"),
 	SUITE(exit, "shell exit and variables"),
 	SUITE(fdt, "fdt command"),
+#ifdef CONFIG_UT_FDT_OVERLAY
+	SUITE_CMD(fdt_overlay, do_ut_fdt_overlay, "device tree overlays"),
+#endif
 	SUITE(font, "font command"),
 	SUITE(hush, "hush behaviour"),
 	SUITE(lib, "library functions"),
@@ -144,9 +147,6 @@ static struct suite suites[] = {
 	SUITE(mem, "memory-related commands"),
 #ifdef CONFIG_UT_OPTEE
 	SUITE_CMD(optee, do_ut_optee, "OP-TEE"),
-#endif
-#ifdef CONFIG_UT_OVERLAY
-	SUITE_CMD(overlay, do_ut_overlay, "device tree overlays"),
 #endif
 	SUITE(pci_mps, "PCI Express Maximum Payload Size"),
 	SUITE(seama, "seama command parameters loading and decoding"),
test/fdt_overlay/Kconfig renamed from test/overlay/Kconfig +1 -1
similarity index 93%
rename from test/overlay/Kconfig
rename to test/fdt_overlay/Kconfig
index 881848968bb..c2bb70fc970 100644
@@ -1,4 +1,4 @@
-config UT_OVERLAY
+config UT_FDT_OVERLAY
 	bool "Enable Device Tree Overlays Unit Tests"
 	depends on UNIT_TEST && OF_CONTROL && SANDBOX
 	default y
test/fdt_overlay/Makefile renamed from test/overlay/Makefile +1 -1
similarity index 89%
rename from test/overlay/Makefile
rename to test/fdt_overlay/Makefile
index 47937e3c108..5625c0d8885 100644
@@ -4,7 +4,7 @@
 # Copyright (c) 2016 Free Electrons
 
 # Test files
-obj-y += cmd_ut_overlay.o
+obj-y += cmd_ut_fdt_overlay.o
 
 DTC_FLAGS += -@
 
test/fdt_overlay/cmd_ut_fdt_overlay.c renamed from test/overlay/cmd_ut_overlay.c +25 -24

132 lines not on this page · raw message

test/fdt_overlay/test-fdt-base.dts renamed from test/overlay/test-fdt-base.dts +0 -0
similarity index 100%
rename from test/overlay/test-fdt-base.dts
rename to test/fdt_overlay/test-fdt-base.dts
test/fdt_overlay/test-fdt-overlay-stacked.dtso renamed from test/overlay/test-fdt-overlay-stacked.dtso +0 -0
similarity index 100%
rename from test/overlay/test-fdt-overlay-stacked.dtso
rename to test/fdt_overlay/test-fdt-overlay-stacked.dtso
test/fdt_overlay/test-fdt-overlay.dtso renamed from test/overlay/test-fdt-overlay.dtso +0 -0
similarity index 100%
rename from test/overlay/test-fdt-overlay.dtso
rename to test/fdt_overlay/test-fdt-overlay.dtso
test/py/tests/test_suite.py +2 -2
index ae127301fd7..1e02d67efe2 100644
@@ -7,10 +7,10 @@ import re
 # List of test suites we expect to find with 'ut info' and 'ut all'
 EXPECTED_SUITES = [
     'addrmap', 'bdinfo', 'bloblist', 'bootm', 'bootstd',
-    'cmd', 'common', 'dm', 'env', 'exit',
+    'cmd', 'common', 'dm', 'env', 'exit', 'fdt_overlay',
     'fdt', 'font', 'hush', 'lib',
     'loadm', 'log', 'mbr', 'measurement', 'mem',
-    'overlay', 'pci_mps', 'setexpr', 'upl',
+    'pci_mps', 'setexpr', 'upl',
     ]
 
 
Signature · 1 line
2.43.0
#8 lore raw

PATCH 8/28 test: Move env-test rule into test/

In reply to Simon Glass

The Makefile rules for tests should be within test/Makefile so move the 'env' rule over.

---

 Makefile      | 1 -
 test/Makefile | 1 +
 2 files changed, 1 insertion(+), 1 deletion(-)
Makefile +0 -1
index 1e29796602b..c86e289718b 100644
@@ -893,7 +893,6 @@ ifdef CONFIG_POST
 libs-y += post/
 endif
 libs-$(CONFIG_$(PHASE_)UNIT_TEST) += test/
-libs-$(CONFIG_UT_ENV) += test/env/
 libs-$(CONFIG_UT_OPTEE) += test/optee/
 libs-$(CONFIG_UT_FDT_OVERLAY) += test/fdt_overlay/
 
test/Makefile +1 -0
index 47a07d653a9..17716ef384a 100644
@@ -19,6 +19,7 @@ obj-y += ut.o
 ifeq ($(CONFIG_XPL_BUILD),)
 obj-y += boot/
 obj-$(CONFIG_UNIT_TEST) += common/
+obj-$(CONFIG_UT_ENV) += env/
 obj-y += log/
 else
 obj-$(CONFIG_SPL_UT_LOAD) += image/
Signature · 1 line
2.43.0
#9 lore raw

PATCH 9/28 test: Move optee-test rule into test/

In reply to Simon Glass

The Makefile rules for tests should be within test/Makefile so move the 'optee' rule over.

---

 Makefile      | 1 -
 test/Makefile | 1 +
 2 files changed, 1 insertion(+), 1 deletion(-)
Makefile +0 -1
index c86e289718b..5d1b3d0f217 100644
@@ -893,7 +893,6 @@ ifdef CONFIG_POST
 libs-y += post/
 endif
 libs-$(CONFIG_$(PHASE_)UNIT_TEST) += test/
-libs-$(CONFIG_UT_OPTEE) += test/optee/
 libs-$(CONFIG_UT_FDT_OVERLAY) += test/fdt_overlay/
 
 libs-y += $(if $(wildcard $(srctree)/board/$(BOARDDIR)/Makefile),board/$(BOARDDIR)/)
test/Makefile +1 -0
index 17716ef384a..5ef8c379cb2 100644
@@ -14,6 +14,7 @@ endif
 ifneq ($(CONFIG_HUSH_PARSER),)
 obj-$(CONFIG_$(XPL_)CMDLINE) += hush/
 endif
+obj-$(CONFIG_UT_OPTEE) += optee/
 obj-y += ut.o
 
 ifeq ($(CONFIG_XPL_BUILD),)
Signature · 1 line
2.43.0
#10 lore raw

PATCH 10/28 test: Move fdt-overlay-test rule into test/

In reply to Simon Glass

The Makefile rules for tests should be within test/Makefile so move the 'fdt-overlay' rule over.

---

 Makefile      | 1 -
 test/Makefile | 1 +
 2 files changed, 1 insertion(+), 1 deletion(-)
Makefile +0 -1
index 5d1b3d0f217..655359d254a 100644
@@ -893,7 +893,6 @@ ifdef CONFIG_POST
 libs-y += post/
 endif
 libs-$(CONFIG_$(PHASE_)UNIT_TEST) += test/
-libs-$(CONFIG_UT_FDT_OVERLAY) += test/fdt_overlay/
 
 libs-y += $(if $(wildcard $(srctree)/board/$(BOARDDIR)/Makefile),board/$(BOARDDIR)/)
 
test/Makefile +1 -0
index 5ef8c379cb2..99d4797d968 100644
@@ -21,6 +21,7 @@ ifeq ($(CONFIG_XPL_BUILD),)
 obj-y += boot/
 obj-$(CONFIG_UNIT_TEST) += common/
 obj-$(CONFIG_UT_ENV) += env/
+obj-$(CONFIG_UT_FDT_OVERLAY) += fdt_overlay/
 obj-y += log/
 else
 obj-$(CONFIG_SPL_UT_LOAD) += image/
Signature · 1 line
2.43.0
#11 lore raw

PATCH 11/28 test: Make all tests depend on UNIT_TEST

In reply to Simon Glass

Rather than having this condition defined separately for each suite, bracket all options with 'if UNIT_TEST'.

---

 test/Kconfig             | 14 ++++++--------
 test/dm/Kconfig          |  2 +-
 test/env/Kconfig         |  1 -
 test/fdt_overlay/Kconfig |  2 +-
 test/optee/Kconfig       |  2 +-
 5 files changed, 9 insertions(+), 12 deletions(-)
test/Kconfig +6 -8
index 01d64642fdb..31016eedbf8 100644
@@ -20,9 +20,10 @@ config SPL_UNIT_TEST
 	  of-platdata and SPL handover. To run these tests with the sandbox_spl
 	  board, use the -u (unit test) option.
 
+if UNIT_TEST
+
 config UNIT_TEST_DURATION
 	bool "Report unit-test duration"
-	depends on UNIT_TEST
 	default y
 	help
 	  Enable this short the time taken by each test suite. This is reported
@@ -31,7 +32,6 @@ config UNIT_TEST_DURATION
 
 config UT_LIB
 	bool "Unit tests for library functions"
-	depends on UNIT_TEST
 	default y if !SANDBOX_VPL
 	help
 	  Enables the 'ut lib' command which tests library functions like
@@ -72,16 +72,15 @@ config UT_LIB_RSA
 	  Enables rsa_verify() test, currently rsa_verify_with_pkey only()
 	  only, at the 'ut lib' command.
 
-endif
+endif  # UT_LIB
 
 config UT_BOOTSTD
 	bool "Unit tests for standard boot"
-	depends on UNIT_TEST && BOOTSTD && SANDBOX
+	depends on BOOTSTD && SANDBOX
 	default y
 
 config UT_COMPRESSION
 	bool "Unit test for compression"
-	depends on UNIT_TEST
 	depends on CMDLINE && GZIP_COMPRESSED && BZIP2 && LZMA && LZO && LZ4 && ZSTD
 	default y
 	help
@@ -90,7 +89,6 @@ config UT_COMPRESSION
 
 config UT_LOG
 	bool "Unit tests for logging functions"
-	depends on UNIT_TEST
 	default y
 	help
 	  Enables the 'ut log' command which tests logging functions like
@@ -99,7 +97,6 @@ config UT_LOG
 
 config UT_TIME
 	bool "Unit tests for time functions"
-	depends on UNIT_TEST
 	help
 	  Enables the 'ut time' command which tests that the time functions
 	  work correctly. The test is fairly simple and will not catch all
@@ -108,7 +105,6 @@ config UT_TIME
 
 config UT_UNICODE
 	bool "Unit tests for Unicode functions"
-	depends on UNIT_TEST
 	default y
 	select CHARSET
 	help
@@ -122,6 +118,8 @@ source "test/lib/Kconfig"
 source "test/optee/Kconfig"
 source "test/fdt_overlay/Kconfig"
 
+endif  # UNIT_TEST
+
 config POST
 	bool "Power On Self Test support"
 	help
test/dm/Kconfig +1 -1
index e5b341e523a..640421c2a79 100644
@@ -1,6 +1,6 @@
 config UT_DM
 	bool "Enable driver model unit test command"
-	depends on SANDBOX && UNIT_TEST
+	depends on SANDBOX
 	help
 	  This enables the 'ut dm' command which runs a series of unit
 	  tests on the driver model code. Each subsystem (uclass) is tested.
test/env/Kconfig +0 -1
index 6cb82337b36..21d88f47a6d 100644
@@ -1,6 +1,5 @@
 config UT_ENV
 	bool "Enable env unit tests"
-	depends on UNIT_TEST
 	default y
 	help
 	  This enables the 'ut env' command which runs a series of unit
test/fdt_overlay/Kconfig +1 -1
index c2bb70fc970..c50b8822544 100644
@@ -1,6 +1,6 @@
 config UT_FDT_OVERLAY
 	bool "Enable Device Tree Overlays Unit Tests"
-	depends on UNIT_TEST && OF_CONTROL && SANDBOX
+	depends on OF_CONTROL && SANDBOX
 	default y
 	select OF_LIBFDT_OVERLAY
 	help
test/optee/Kconfig +1 -1
index 2f6834aa3b4..63e2cbf79c7 100644
@@ -1,6 +1,6 @@
 config UT_OPTEE
 	bool "Enable OP-TEE Unit Tests"
-	depends on UNIT_TEST && OF_CONTROL && OPTEE
+	depends on OF_CONTROL && OPTEE
 	default y
 	help
 	  This enables the 'ut optee' command which runs a series of unit
Signature · 1 line
2.43.0
#12 lore raw

PATCH 12/28 test: Move fdt_overlay init into a function

In reply to Simon Glass

Move the init code into a separate function since it is quite large. Adjust it to use unit-test functions which have become available since the test was written.

---

 test/fdt_overlay/cmd_ut_fdt_overlay.c | 113 +++++++++++++-------------
 1 file changed, 57 insertions(+), 56 deletions(-)
test/fdt_overlay/cmd_ut_fdt_overlay.c +57 -56
index 9fe18b60aca..e4ed7de850c 100644
@@ -26,6 +26,61 @@ extern u32 __dtbo_test_fdt_overlay_stacked_begin;
 
 static void *fdt;
 
+static int fdt_overlay_init(struct unit_test_state *uts)
+{
+	void *fdt_base = &__dtb_test_fdt_base_begin;
+	void *fdt_overlay = &__dtbo_test_fdt_overlay_begin;
+	void *fdt_overlay_stacked = &__dtbo_test_fdt_overlay_stacked_begin;
+	void *fdt_overlay_copy, *fdt_overlay_stacked_copy;
+
+	ut_assertok(fdt_check_header(fdt_base));
+	ut_assertok(fdt_check_header(fdt_overlay));
+
+	fdt = malloc(FDT_COPY_SIZE);
+	fdt_overlay_copy = malloc(FDT_COPY_SIZE);
+	fdt_overlay_stacked_copy = malloc(FDT_COPY_SIZE);
+	ut_assertnonnull(fdt);
+	ut_assertnonnull(fdt_overlay_copy);
+	ut_assertnonnull(fdt_overlay_stacked_copy);
+
+	/*
+	 * Resize the FDT to 4k so that we have room to operate on
+	 *
+	 * (and relocate it since the memory might be mapped
+	 * read-only)
+	 */
+	ut_assertok(fdt_open_into(fdt_base, fdt, FDT_COPY_SIZE));
+
+	/*
+	 * Resize the overlay to 4k so that we have room to operate on
+	 *
+	 * (and relocate it since the memory might be mapped
+	 * read-only)
+	 */
+	ut_assertok(fdt_open_into(fdt_overlay, fdt_overlay_copy,
+				  FDT_COPY_SIZE));
+
+	/*
+	 * Resize the stacked overlay to 4k so that we have room to operate on
+	 *
+	 * (and relocate it since the memory might be mapped
+	 * read-only)
+	 */
+	ut_assertok(fdt_open_into(fdt_overlay_stacked, fdt_overlay_stacked_copy,
+				  FDT_COPY_SIZE));
+
+	/* Apply the overlay */
+	ut_assertok(fdt_overlay_apply(fdt, fdt_overlay_copy));
+
+	/* Apply the stacked overlay */
+	ut_assertok(fdt_overlay_apply(fdt, fdt_overlay_stacked_copy));
+
+	free(fdt_overlay_stacked_copy);
+	free(fdt_overlay_copy);
+
+	return 0;
+}
+
 static int ut_fdt_getprop_u32_by_index(void *fdt, const char *path,
 				    const char *name, int index,
 				    u32 *out)
@@ -215,67 +270,13 @@ int do_ut_fdt_overlay(struct unit_test_state *uts, struct cmd_tbl *cmdtp,
 {
 	struct unit_test *tests = UNIT_TEST_SUITE_START(fdt_overlay);
 	const int n_ents = UNIT_TEST_SUITE_COUNT(fdt_overlay);
-	void *fdt_base = &__dtb_test_fdt_base_begin;
-	void *fdt_overlay = &__dtbo_test_fdt_overlay_begin;
-	void *fdt_overlay_stacked = &__dtbo_test_fdt_overlay_stacked_begin;
-	void *fdt_overlay_copy, *fdt_overlay_stacked_copy;
 	int ret = -ENOMEM;
 
-	ut_assertok(fdt_check_header(fdt_base));
-	ut_assertok(fdt_check_header(fdt_overlay));
-
-	fdt = malloc(FDT_COPY_SIZE);
-	if (!fdt)
-		goto err1;
-
-	fdt_overlay_copy = malloc(FDT_COPY_SIZE);
-	if (!fdt_overlay_copy)
-		goto err2;
-
-	fdt_overlay_stacked_copy = malloc(FDT_COPY_SIZE);
-	if (!fdt_overlay_stacked_copy)
-		goto err3;
-
-	/*
-	 * Resize the FDT to 4k so that we have room to operate on
-	 *
-	 * (and relocate it since the memory might be mapped
-	 * read-only)
-	 */
-	ut_assertok(fdt_open_into(fdt_base, fdt, FDT_COPY_SIZE));
-
-	/*
-	 * Resize the overlay to 4k so that we have room to operate on
-	 *
-	 * (and relocate it since the memory might be mapped
-	 * read-only)
-	 */
-	ut_assertok(fdt_open_into(fdt_overlay, fdt_overlay_copy,
-				  FDT_COPY_SIZE));
-
-	/*
-	 * Resize the stacked overlay to 4k so that we have room to operate on
-	 *
-	 * (and relocate it since the memory might be mapped
-	 * read-only)
-	 */
-	ut_assertok(fdt_open_into(fdt_overlay_stacked, fdt_overlay_stacked_copy,
-				  FDT_COPY_SIZE));
-
-	/* Apply the overlay */
-	ut_assertok(fdt_overlay_apply(fdt, fdt_overlay_copy));
-
-	/* Apply the stacked overlay */
-	ut_assertok(fdt_overlay_apply(fdt, fdt_overlay_stacked_copy));
-
+	ut_assertok(fdt_overlay_init(uts));
 	ret = cmd_ut_category(uts, "fdt_overlay", "fdt_overlay_test_", tests,
 			      n_ents, argc, argv);
 
-	free(fdt_overlay_stacked_copy);
-err3:
-	free(fdt_overlay_copy);
-err2:
 	free(fdt);
-err1:
+
 	return ret;
 }
Signature · 1 line
2.43.0
#13 lore raw

PATCH 13/28 test: Update fdt_overlay test to use fdtdec functions

In reply to Simon Glass

Use the helpers provided for this purpose, rather than different ones in this particular test.

Leave fdt_getprop_str() alone as it seems to have more value.

---

 test/fdt_overlay/cmd_ut_fdt_overlay.c | 73 ++++++++-------------------
 1 file changed, 22 insertions(+), 51 deletions(-)
test/fdt_overlay/cmd_ut_fdt_overlay.c +22 -51
index e4ed7de850c..ac8a3b2be23 100644
@@ -6,6 +6,7 @@
 
 #include <command.h>
 #include <errno.h>
+#include <fdtdec.h>
 #include <fdt_support.h>
 #include <image.h>
 #include <log.h>
@@ -81,33 +82,6 @@ static int fdt_overlay_init(struct unit_test_state *uts)
 	return 0;
 }
 
-static int ut_fdt_getprop_u32_by_index(void *fdt, const char *path,
-				    const char *name, int index,
-				    u32 *out)
-{
-	const fdt32_t *val;
-	int node_off;
-	int len;
-
-	node_off = fdt_path_offset(fdt, path);
-	if (node_off < 0)
-		return node_off;
-
-	val = fdt_getprop(fdt, node_off, name, &len);
-	if (!val || (len < (sizeof(uint32_t) * (index + 1))))
-		return -FDT_ERR_NOTFOUND;
-
-	*out = fdt32_to_cpu(*(val + index));
-
-	return 0;
-}
-
-static int ut_fdt_getprop_u32(void *fdt, const char *path, const char *name,
-			   u32 *out)
-{
-	return ut_fdt_getprop_u32_by_index(fdt, path, name, 0, out);
-}
-
 static int fdt_getprop_str(void *fdt, const char *path, const char *name,
 			   const char **out)
 {
@@ -125,11 +99,12 @@ static int fdt_getprop_str(void *fdt, const char *path, const char *name,
 
 static int fdt_overlay_test_change_int_property(struct unit_test_state *uts)
 {
-	u32 val = 0;
+	int off;
 
-	ut_assertok(ut_fdt_getprop_u32(fdt, "/test-node", "test-int-property",
-				    &val));
-	ut_asserteq(43, val);
+	off = fdt_path_offset(fdt, "/test-node");
+	ut_assert(off >= 0);
+
+	ut_asserteq(43, fdtdec_get_uint(fdt, off, "test-int-property", 0));
 
 	return CMD_RET_SUCCESS;
 }
@@ -202,7 +177,7 @@ FDT_OVERLAY_TEST(fdt_overlay_test_add_subnode_property, 0);
 static int fdt_overlay_test_local_phandle(struct unit_test_state *uts)
 {
 	uint32_t local_phandle;
-	u32 val = 0;
+	u32 val[2];
 	int off;
 
 	off = fdt_path_offset(fdt, "/new-local-node");
@@ -211,13 +186,10 @@ static int fdt_overlay_test_local_phandle(struct unit_test_state *uts)
 	local_phandle = fdt_get_phandle(fdt, off);
 	ut_assert(local_phandle);
 
-	ut_assertok(ut_fdt_getprop_u32_by_index(fdt, "/", "test-several-phandle",
-					     0, &val));
-	ut_asserteq(local_phandle, val);
-
-	ut_assertok(ut_fdt_getprop_u32_by_index(fdt, "/", "test-several-phandle",
-					     1, &val));
-	ut_asserteq(local_phandle, val);
+	ut_assertok(fdtdec_get_int_array(fdt, 0, "test-several-phandle", val,
+					 ARRAY_SIZE(val)));
+	ut_asserteq(local_phandle, val[0]);
+	ut_asserteq(local_phandle, val[1]);
 
 	return CMD_RET_SUCCESS;
 }
@@ -226,7 +198,7 @@ FDT_OVERLAY_TEST(fdt_overlay_test_local_phandle, 0);
 static int fdt_overlay_test_local_phandles(struct unit_test_state *uts)
 {
 	uint32_t local_phandle, test_phandle;
-	u32 val = 0;
+	u32 val[2];
 	int off;
 
 	off = fdt_path_offset(fdt, "/new-local-node");
@@ -241,13 +213,10 @@ static int fdt_overlay_test_local_phandles(struct unit_test_state *uts)
 	test_phandle = fdt_get_phandle(fdt, off);
 	ut_assert(test_phandle);
 
-	ut_assertok(ut_fdt_getprop_u32_by_index(fdt, "/", "test-phandle", 0,
-					     &val));
-	ut_asserteq(test_phandle, val);
-
-	ut_assertok(ut_fdt_getprop_u32_by_index(fdt, "/", "test-phandle", 1,
-					     &val));
-	ut_asserteq(local_phandle, val);
+	ut_assertok(fdtdec_get_int_array(fdt, 0, "test-phandle", val,
+					 ARRAY_SIZE(val)));
+	ut_asserteq(test_phandle, val[0]);
+	ut_asserteq(local_phandle, val[1]);
 
 	return CMD_RET_SUCCESS;
 }
@@ -255,11 +224,13 @@ FDT_OVERLAY_TEST(fdt_overlay_test_local_phandles, 0);
 
 static int fdt_overlay_test_stacked(struct unit_test_state *uts)
 {
-	u32 val = 0;
+	int off;
+
+	off = fdt_path_offset(fdt, "/new-local-node");
+	ut_assert(off > 0);
 
-	ut_assertok(ut_fdt_getprop_u32(fdt, "/new-local-node",
-				       "stacked-test-int-property", &val));
-	ut_asserteq(43, val);
+	ut_asserteq(43,
+		    fdtdec_get_uint(fdt, off, "stacked-test-int-property", 0));
 
 	return CMD_RET_SUCCESS;
 }
Signature · 1 line
2.43.0
#14 lore raw

PATCH 14/28 test: Update fdt_overlay to do init from tests

In reply to Simon Glass

Rather than having an init function and then running the tests, create a test-init function to do it. This will allow us to get rid of the command function.

---

 include/test/fdt_overlay.h            | 4 ++++
 test/fdt_overlay/cmd_ut_fdt_overlay.c | 2 +-
 2 files changed, 5 insertions(+), 1 deletion(-)
include/test/fdt_overlay.h +4 -0
index 34e8020a496..251ad0ec97a 100644
@@ -12,4 +12,8 @@
 /* Declare a new FDT-overlay test */
 #define FDT_OVERLAY_TEST(_name, _flags)	UNIT_TEST(_name, _flags, fdt_overlay)
 
+/* Declare init for FDT-overlay test */
+#define FDT_OVERLAY_TEST_INIT(_name, _flags)		\
+	UNIT_TEST_INIT(_name, _flags, fdt_overlay)
+
 #endif /* __TEST_OVERLAY_H__ */
test/fdt_overlay/cmd_ut_fdt_overlay.c +1 -1
index ac8a3b2be23..c716c6bd6b0 100644
@@ -81,6 +81,7 @@ static int fdt_overlay_init(struct unit_test_state *uts)
 
 	return 0;
 }
+FDT_OVERLAY_TEST_INIT(fdt_overlay_init, 0);
 
 static int fdt_getprop_str(void *fdt, const char *path, const char *name,
 			   const char **out)
@@ -243,7 +244,6 @@ int do_ut_fdt_overlay(struct unit_test_state *uts, struct cmd_tbl *cmdtp,
 	const int n_ents = UNIT_TEST_SUITE_COUNT(fdt_overlay);
 	int ret = -ENOMEM;
 
-	ut_assertok(fdt_overlay_init(uts));
 	ret = cmd_ut_category(uts, "fdt_overlay", "fdt_overlay_test_", tests,
 			      n_ents, argc, argv);
 
Signature · 1 line
2.43.0
#15 lore raw

PATCH 15/28 test: Drop the function for running fdt_overlay tests

In reply to Simon Glass

Use the new suite-runner to run these tests instead.

---

 include/test/suites.h                 |  2 --
 test/cmd_ut.c                         |  4 +---
 test/fdt_overlay/cmd_ut_fdt_overlay.c | 15 ---------------
 3 files changed, 1 insertion(+), 20 deletions(-)
include/test/suites.h +0 -2
index dce720b4e78..78f5cdb4bdf 100644
@@ -36,8 +36,6 @@ int cmd_ut_category(struct unit_test_state *uts, const char *name,
 
 int do_ut_bootstd(struct unit_test_state *uts, struct cmd_tbl *cmdtp, int flag,
 		  int argc, char *const argv[]);
-int do_ut_fdt_overlay(struct unit_test_state *uts, struct cmd_tbl *cmdtp,
-		      int flag, int argc, char *const argv[]);
 int do_ut_optee(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[]);
 
 #endif /* __TEST_SUITES_H__ */
test/cmd_ut.c +1 -3
index 958b591c605..c4a42ee2e20 100644
@@ -134,9 +134,7 @@ static struct suite suites[] = {
 	SUITE(env, "environment"),
 	SUITE(exit, "shell exit and variables"),
 	SUITE(fdt, "fdt command"),
-#ifdef CONFIG_UT_FDT_OVERLAY
-	SUITE_CMD(fdt_overlay, do_ut_fdt_overlay, "device tree overlays"),
-#endif
+	SUITE(fdt_overlay, "device tree overlays"),
 	SUITE(font, "font command"),
 	SUITE(hush, "hush behaviour"),
 	SUITE(lib, "library functions"),
test/fdt_overlay/cmd_ut_fdt_overlay.c +0 -15
index c716c6bd6b0..3244f2d3582 100644
@@ -236,18 +236,3 @@ static int fdt_overlay_test_stacked(struct unit_test_state *uts)
 	return CMD_RET_SUCCESS;
 }
 FDT_OVERLAY_TEST(fdt_overlay_test_stacked, 0);
-
-int do_ut_fdt_overlay(struct unit_test_state *uts, struct cmd_tbl *cmdtp,
-		      int flag, int argc, char *const argv[])
-{
-	struct unit_test *tests = UNIT_TEST_SUITE_START(fdt_overlay);
-	const int n_ents = UNIT_TEST_SUITE_COUNT(fdt_overlay);
-	int ret = -ENOMEM;
-
-	ret = cmd_ut_category(uts, "fdt_overlay", "fdt_overlay_test_", tests,
-			      n_ents, argc, argv);
-
-	free(fdt);
-
-	return ret;
-}
Signature · 1 line
2.43.0
#16 lore raw

PATCH 16/28 test: Update bootstd to do init from tests

In reply to Simon Glass

Rather than having an init function and then running the tests, create a test-init function to do it. This will allow us to get rid of the command function.

---

 test/boot/bootstd_common.c | 18 +++++++++---------
 test/boot/bootstd_common.h | 10 +---------
 test/boot/vbe_simple.c     |  3 ---
 3 files changed, 10 insertions(+), 21 deletions(-)
test/boot/bootstd_common.c +9 -9
index 724e3d9bdd2..e28cae7f374 100644
@@ -21,8 +21,14 @@
 /* tracks whether bootstd_setup_for_tests() has been run yet */
 bool vbe_setup_done;
 
-/* set up MMC for VBE tests */
-int bootstd_setup_for_tests(void)
+/**
+ * bootstd_setup_for_tests() - Set up MMC data for VBE tests
+ *
+ * Some data is needed for VBE tests to work. This function sets that up.
+ *
+ * @return 0 if OK, -ve on error
+ */
+static int bootstd_setup_for_tests(struct unit_test_state *uts)
 {
 	ALLOC_CACHE_ALIGN_BUFFER(u8, buf, MMC_MAX_BLOCK_LEN);
 	struct udevice *mmc;
@@ -55,6 +61,7 @@ int bootstd_setup_for_tests(void)
 
 	return 0;
 }
+BOOTSTD_TEST_INIT(bootstd_setup_for_tests, 0);
 
 int bootstd_test_drop_bootdev_order(struct unit_test_state *uts)
 {
@@ -99,13 +106,6 @@ int do_ut_bootstd(struct unit_test_state *uts, struct cmd_tbl *cmdtp, int flag,
 {
 	struct unit_test *tests = UNIT_TEST_SUITE_START(bootstd);
 	const int n_ents = UNIT_TEST_SUITE_COUNT(bootstd);
-	int ret;
-
-	ret = bootstd_setup_for_tests();
-	if (ret) {
-		printf("Failed to set up for bootstd tests (err=%d)\n", ret);
-		return CMD_RET_FAILURE;
-	}
 
 	return cmd_ut_category(uts, "bootstd", "bootstd_",
 			       tests, n_ents, argc, argv);
test/boot/bootstd_common.h +1 -9
index ea3ecd1166c..c61698adc02 100644
@@ -13,6 +13,7 @@
 
 /* Declare a new bootdev test */
 #define BOOTSTD_TEST(_name, _flags)	UNIT_TEST(_name, _flags, bootstd)
+#define BOOTSTD_TEST_INIT(_name, _flags) UNIT_TEST_INIT(_name, _flags, bootstd)
 
 #define NVDATA_START_BLK	((0x400 + 0x400) / MMC_MAX_BLOCK_LEN)
 #define VERSION_START_BLK	((0x400 + 0x800) / MMC_MAX_BLOCK_LEN)
@@ -35,15 +36,6 @@ struct unit_test_state;
  */
 int bootstd_test_drop_bootdev_order(struct unit_test_state *uts);
 
-/**
- * bootstd_setup_for_tests() - Set up MMC data for VBE tests
- *
- * Some data is needed for VBE tests to work. This function sets that up.
- *
- * @return 0 if OK, -ve on error
- */
-int bootstd_setup_for_tests(void);
-
 /**
  * bootstd_test_check_mmc_hunter() - Check that the mmc bootdev hunter was used
  *
test/boot/vbe_simple.c +0 -3
index 4fe4323b401..2fe74e1f6c6 100644
@@ -33,9 +33,6 @@ static int vbe_simple_test_base(struct unit_test_state *uts)
 	ofnode node;
 	u32 vernum;
 
-	/* Set up the VBE info */
-	ut_assertok(bootstd_setup_for_tests());
-
 	/* Read the version back */
 	ut_assertok(vbe_find_by_any("firmware0", &dev));
 	ut_assertok(bootmeth_get_state_desc(dev, info, sizeof(info)));
Signature · 1 line
2.43.0
#17 lore raw

PATCH 17/28 test: Drop the function for running bootstd tests

In reply to Simon Glass

Use the new suite-runner to run these tests instead.

---

 include/test/suites.h      |  2 --
 test/boot/bootstd_common.c | 10 ----------
 test/cmd_ut.c              |  4 +---
 3 files changed, 1 insertion(+), 15 deletions(-)
include/test/suites.h +0 -2
index 78f5cdb4bdf..692633dcaa4 100644
@@ -34,8 +34,6 @@ int cmd_ut_category(struct unit_test_state *uts, const char *name,
 		    const char *prefix, struct unit_test *tests, int n_ents,
 		    int argc, char *const argv[]);
 
-int do_ut_bootstd(struct unit_test_state *uts, struct cmd_tbl *cmdtp, int flag,
-		  int argc, char *const argv[]);
 int do_ut_optee(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[]);
 
 #endif /* __TEST_SUITES_H__ */
test/boot/bootstd_common.c +0 -10
index e28cae7f374..6a84810983c 100644
@@ -100,13 +100,3 @@ void bootstd_reset_usb(void)
 {
 	usb_started = false;
 }
-
-int do_ut_bootstd(struct unit_test_state *uts, struct cmd_tbl *cmdtp, int flag,
-		  int argc, char *const argv[])
-{
-	struct unit_test *tests = UNIT_TEST_SUITE_START(bootstd);
-	const int n_ents = UNIT_TEST_SUITE_COUNT(bootstd);
-
-	return cmd_ut_category(uts, "bootstd", "bootstd_",
-			       tests, n_ents, argc, argv);
-}
test/cmd_ut.c +1 -3
index c4a42ee2e20..1885c78be41 100644
@@ -125,9 +125,7 @@ static struct suite suites[] = {
 	SUITE(bdinfo, "bdinfo (board info) command"),
 	SUITE(bloblist, "bloblist implementation"),
 	SUITE(bootm, "bootm command"),
-#ifdef CONFIG_UT_BOOTSTD
-	SUITE_CMD(bootstd, do_ut_bootstd, "standard boot implementation"),
-#endif
+	SUITE(bootstd, "standard boot implementation"),
 	SUITE(cmd, "various commands"),
 	SUITE(common, "tests for common/ directory"),
 	SUITE(dm, "driver model"),
Signature · 1 line
2.43.0
#18 lore raw

PATCH 18/28 test: Update optee to do init and uninit from tests

In reply to Simon Glass

Rather than having an init function and then running the tests, create a test-init function to do it. This will allow us to get rid of the command function.

Fix the comment abotu 'environment' while we are here.

---

 include/test/optee.h      |  4 ++-
 test/optee/cmd_ut_optee.c | 55 +++++++++++++++++++++++++--------------
 2 files changed, 38 insertions(+), 21 deletions(-)
include/test/optee.h +3 -1
index f4255b39ee3..0a548a59e83 100644
@@ -8,7 +8,9 @@
 
 #include <test/test.h>
 
-/* Declare a new environment test */
+/* Declare a new optee test */
 #define OPTEE_TEST(_name, _flags)	UNIT_TEST(_name, _flags, optee)
+#define OPTEE_TEST_INIT(_name, _flags)	UNIT_TEST_INIT(_name, _flags, optee)
+#define OPTEE_TEST_UNINIT(_name, _flags) UNIT_TEST_UNINIT(_name, _flags, optee)
 
 #endif /* __TEST_OPTEE_H__ */
test/optee/cmd_ut_optee.c +35 -20

67 lines not on this page · raw message

Signature · 1 line
2.43.0
#19 lore raw

PATCH 19/28 test: Split optee tests into three functions

In reply to Simon Glass

These tests run three different checks on the nodes, but the logic is currently all in one tests.

Split the code out into three different tests, which do different setup and then run the same checks.

---

 test/optee/cmd_ut_optee.c | 57 +++++++++++++++++++++++++++------------
 1 file changed, 40 insertions(+), 17 deletions(-)
test/optee/cmd_ut_optee.c +40 -17
index 792de304c15..71833e5f6ab 100644
@@ -81,7 +81,6 @@ static int optee_fdt_firmware(struct unit_test_state *uts)
 
 	return CMD_RET_SUCCESS;
 }
-OPTEE_TEST(optee_fdt_firmware, 0);
 
 static int optee_fdt_protected_memory(struct unit_test_state *uts)
 {
@@ -124,38 +123,62 @@ static int optee_fdt_protected_memory(struct unit_test_state *uts)
 
 	return CMD_RET_SUCCESS;
 }
-OPTEE_TEST(optee_fdt_protected_memory, 0);
 
-int do_ut_optee(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[])
+/* (1) Try to copy optee nodes from empty dt */
+static int optee_fdt_copy_empty(struct unit_test_state *uts)
 {
-	struct unit_test *tests = UNIT_TEST_SUITE_START(optee);
-	const int n_ents = UNIT_TEST_SUITE_COUNT(optee);
-	struct unit_test_state *uts;
-	void *fdt_optee = &__dtb_test_optee_optee_begin;
 	void *fdt_no_optee = &__dtb_test_optee_no_optee_begin;
-	void *fdt_base = &__dtb_test_optee_base_begin;
-	int ret = -ENOMEM;
 
-	/*
-	 * (1) Try to copy optee nodes from empty dt.
-	 * This should still run successfully.
-	 */
+	/* This should still run successfully */
 	ut_assertok(optee_copy_fdt_nodes(fdt_no_optee, fdt));
 
 	expect_success = false;
-	ret = cmd_ut_category("optee", "", tests, n_ents, argc, argv);
+	ut_assertok(optee_fdt_firmware(uts));
+	ut_assertok(optee_fdt_protected_memory(uts));
+
+	return 0;
+}
+OPTEE_TEST(optee_fdt_copy_empty, 0);
+
+/* (2) Try to copy optee nodes from prefilled dt */
+static int optee_fdt_copy_prefilled(struct unit_test_state *uts)
+{
+	void *fdt_optee = &__dtb_test_optee_optee_begin;
 
-	/* (2) Try to copy optee nodes from prefilled dt */
 	ut_assertok(optee_copy_fdt_nodes(fdt_optee, fdt));
 
 	expect_success = true;
-	ret = cmd_ut_category("optee", "", tests, n_ents, argc, argv);
+	ut_assertok(optee_fdt_firmware(uts));
+	ut_assertok(optee_fdt_protected_memory(uts));
+
+	return 0;
+}
+OPTEE_TEST(optee_fdt_copy_prefilled, 0);
+
+/* (3) Try to copy OP-TEE nodes into a already filled DT */
+static int optee_fdt_copy_already_filled(struct unit_test_state *uts)
+{
+	void *fdt_optee = &__dtb_test_optee_optee_begin;
 
-	/* (3) Try to copy OP-TEE nodes into a already filled DT */
 	ut_assertok(fdt_open_into(fdt_optee, fdt, FDT_COPY_SIZE));
 	ut_assertok(optee_copy_fdt_nodes(fdt_optee, fdt));
 
 	expect_success = true;
+	ut_assertok(optee_fdt_firmware(uts));
+	ut_assertok(optee_fdt_protected_memory(uts));
+
+	return 0;
+}
+OPTEE_TEST(optee_fdt_copy_already_filled, 0);
+
+int do_ut_optee(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[])
+{
+	struct unit_test *tests = UNIT_TEST_SUITE_START(optee);
+	const int n_ents = UNIT_TEST_SUITE_COUNT(optee);
+	struct unit_test_state *uts;
+	void *fdt_base = &__dtb_test_optee_base_begin;
+	int ret = -ENOMEM;
+
 	ret = cmd_ut_category("optee", "", tests, n_ents, argc, argv);
 
 	free(fdt);
Signature · 1 line
2.43.0
#20 lore raw

PATCH 20/28 test: Drop the function for running optee tests

In reply to Simon Glass

Use the new suite-runner to run these tests instead.

---

 include/test/suites.h     |  2 --
 test/cmd_ut.c             |  4 +---
 test/optee/cmd_ut_optee.c | 14 --------------
 3 files changed, 1 insertion(+), 19 deletions(-)
include/test/suites.h +0 -2
index 692633dcaa4..a2f982bf6fa 100644
@@ -34,6 +34,4 @@ int cmd_ut_category(struct unit_test_state *uts, const char *name,
 		    const char *prefix, struct unit_test *tests, int n_ents,
 		    int argc, char *const argv[]);
 
-int do_ut_optee(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[]);
-
 #endif /* __TEST_SUITES_H__ */
test/cmd_ut.c +1 -3
index 1885c78be41..6d1e20ef02e 100644
@@ -141,9 +141,7 @@ static struct suite suites[] = {
 	SUITE(mbr, "mbr command"),
 	SUITE(measurement, "TPM-based measured boot"),
 	SUITE(mem, "memory-related commands"),
-#ifdef CONFIG_UT_OPTEE
-	SUITE_CMD(optee, do_ut_optee, "OP-TEE"),
-#endif
+	SUITE(optee, "OP-TEE"),
 	SUITE(pci_mps, "PCI Express Maximum Payload Size"),
 	SUITE(seama, "seama command parameters loading and decoding"),
 	SUITE(setexpr, "setexpr command"),
test/optee/cmd_ut_optee.c +0 -14
index 71833e5f6ab..df7b877025f 100644
@@ -170,17 +170,3 @@ static int optee_fdt_copy_already_filled(struct unit_test_state *uts)
 	return 0;
 }
 OPTEE_TEST(optee_fdt_copy_already_filled, 0);
-
-int do_ut_optee(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[])
-{
-	struct unit_test *tests = UNIT_TEST_SUITE_START(optee);
-	const int n_ents = UNIT_TEST_SUITE_COUNT(optee);
-	struct unit_test_state *uts;
-	void *fdt_base = &__dtb_test_optee_base_begin;
-	int ret = -ENOMEM;
-
-	ret = cmd_ut_category("optee", "", tests, n_ents, argc, argv);
-
-	free(fdt);
-	return ret;
-}
Signature · 1 line
2.43.0
#21 lore raw

PATCH 21/28 test: Rename optee test-file

In reply to Simon Glass

This has nothing to do with commands anymore, so rename the file.

---

 test/optee/Makefile                    | 2 +-
 test/optee/{cmd_ut_optee.c => optee.c} | 0
 2 files changed, 1 insertion(+), 1 deletion(-)
 rename test/optee/{cmd_ut_optee.c => optee.c} (100%)
test/optee/Makefile +1 -1
index 8793fd7ad61..ec56750fa80 100644
@@ -3,7 +3,7 @@
 # Copyright (C) 2019, Theobroma Systems Design und Consulting GmbH
 
 # Test files
-obj-y += cmd_ut_optee.o
+obj-y += optee.o
 
 DTC_FLAGS += -@
 
test/optee/optee.c renamed from test/optee/cmd_ut_optee.c +0 -0
similarity index 100%
rename from test/optee/cmd_ut_optee.c
rename to test/optee/optee.c
Signature · 1 line
2.43.0
#22 lore raw

PATCH 22/28 test: Drop support for test commands

In reply to Simon Glass

Now that everything is using the new test-suite features, drop support for running commands.

Fix a missing closing-bracket while we are here.

---

 include/test/suites.h |  5 -----
 test/cmd_ut.c         | 43 ++++++++++++-------------------------------
 2 files changed, 12 insertions(+), 36 deletions(-)
include/test/suites.h +0 -5
index a2f982bf6fa..687a8a13e7f 100644
@@ -7,14 +7,9 @@
 #ifndef __TEST_SUITES_H__
 #define __TEST_SUITES_H__
 
-struct cmd_tbl;
 struct unit_test;
 struct unit_test_state;
 
-/* 'command' functions normally called do_xxx where xxx is the command name */
-typedef int (*ut_cmd_func)(struct unit_test_state *uts, struct cmd_tbl *cmd,
-			   int flags, int argc, char *const argv[]);
-
 /**
  * cmd_ut_category() - Run a category of unit tests
  *
test/cmd_ut.c +12 -31

87 lines not on this page · raw message

Signature · 1 line
2.43.0
#23 lore raw

PATCH 23/28 test: Make cmd_ut_category() static

In reply to Simon Glass

This function is not used outside the cmd_ut file anymore, so make it static.

---

 include/test/suites.h | 22 ----------------------
 test/cmd_ut.c         |  6 +++---
 2 files changed, 3 insertions(+), 25 deletions(-)
include/test/suites.h +0 -22
index 687a8a13e7f..f817da747c2 100644
@@ -7,26 +7,4 @@
 #ifndef __TEST_SUITES_H__
 #define __TEST_SUITES_H__
 
-struct unit_test;
-struct unit_test_state;
-
-/**
- * cmd_ut_category() - Run a category of unit tests
- *
- * @uts: Unit-test state, which must be ready for use, i.e. ut_init_state()
- *	has been called. The caller is responsible for calling
- *	ut_uninit_state() after this function returns
- * @name:	Category name
- * @prefix:	Prefix of test name
- * @tests:	List of tests to run
- * @n_ents:	Number of tests in @tests
- * @argc:	Argument count provided. Must be >= 1. If this is 1 then all
- *		tests are run, otherwise only the one named @argv[1] is run.
- * @argv:	Arguments: argv[1] is the test to run (if @argc >= 2)
- * Return: 0 if OK, CMD_RET_FAILURE on failure
- */
-int cmd_ut_category(struct unit_test_state *uts, const char *name,
-		    const char *prefix, struct unit_test *tests, int n_ents,
-		    int argc, char *const argv[]);
-
 #endif /* __TEST_SUITES_H__ */
test/cmd_ut.c +3 -3
index b3821a0fe68..31ded16325f 100644
@@ -35,9 +35,9 @@ static int do_ut_all(struct unit_test_state *uts, struct cmd_tbl *cmdtp,
 static int do_ut_info(struct cmd_tbl *cmdtp, int flag, int argc,
 		      char *const argv[]);
 
-int cmd_ut_category(struct unit_test_state *uts, const char *name,
-		    const char *prefix, struct unit_test *tests, int n_ents,
-		    int argc, char *const argv[])
+static int cmd_ut_category(struct unit_test_state *uts, const char *name,
+			   const char *prefix, struct unit_test *tests,
+			   int n_ents, int argc, char *const argv[])
 {
 	const char *test_insert = NULL;
 	int runs_per_text = 1;
Signature · 1 line
2.43.0
#24 lore raw

PATCH 24/28 test: Drop suites.h

In reply to Simon Glass

This file is empty now. Remove it and its uses.

---

 include/test/suites.h                 | 10 ----------
 test/boot/bootdev.c                   |  1 -
 test/boot/bootflow.c                  |  1 -
 test/boot/bootm.c                     |  1 -
 test/boot/bootmeth.c                  |  1 -
 test/boot/bootstd_common.c            |  1 -
 test/boot/expo.c                      |  1 -
 test/boot/image.c                     |  1 -
 test/boot/measurement.c               |  1 -
 test/boot/upl.c                       |  1 -
 test/boot/vbe_simple.c                |  1 -
 test/cmd/addrmap.c                    |  1 -
 test/cmd/bdinfo.c                     |  1 -
 test/cmd/exit.c                       |  1 -
 test/cmd/fdt.c                        |  1 -
 test/cmd/font.c                       |  1 -
 test/cmd/loadm.c                      |  1 -
 test/cmd/mbr.c                        |  1 -
 test/cmd/pci_mps.c                    |  1 -
 test/cmd/seama.c                      |  1 -
 test/cmd/setexpr.c                    |  1 -
 test/cmd_ut.c                         |  1 -
 test/common/bloblist.c                |  1 -
 test/env/cmd_ut_env.c                 |  1 -
 test/fdt_overlay/cmd_ut_fdt_overlay.c |  1 -
 test/log/cont_test.c                  |  1 -
 test/log/nolog_test.c                 |  1 -
 test/log/pr_cont_test.c               |  1 -
 test/log/syslog_test.c                |  1 -
 test/log/syslog_test_ndebug.c         |  1 -
 test/optee/optee.c                    |  1 -
 31 files changed, 40 deletions(-)
 delete mode 100644 include/test/suites.h
include/test/suites.h deleted +0 -10
deleted file mode 100644
index f817da747c2..00000000000
@@ -1,10 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-/*
- * (C) Copyright 2015
- * Joe Hershberger, National Instruments, [email protected]
- */
-
-#ifndef __TEST_SUITES_H__
-#define __TEST_SUITES_H__
-
-#endif /* __TEST_SUITES_H__ */
test/boot/bootdev.c +0 -1
index 8c44afd9297..5f07430714e 100644
@@ -12,7 +12,6 @@
 #include <bootflow.h>
 #include <mapmem.h>
 #include <os.h>
-#include <test/suites.h>
 #include <test/ut.h>
 #include "bootstd_common.h"
 
test/boot/bootflow.c +0 -1
index 35580cee90c..312d36bea16 100644
@@ -22,7 +22,6 @@
 #include <dm/device-internal.h>
 #include <dm/lists.h>
 #include <linux/libfdt.h>
-#include <test/suites.h>
 #include <test/ut.h>
 #include "bootstd_common.h"
 #include "../../boot/bootflow_internal.h"
test/boot/bootm.c +0 -1
index 1ce5c3cd6bf..2437fa6f3f6 100644
@@ -7,7 +7,6 @@
 
 #include <bootm.h>
 #include <asm/global_data.h>
-#include <test/suites.h>
 #include <test/test.h>
 #include <test/ut.h>
 
test/boot/bootmeth.c +0 -1
index 18ae6d7fe13..577f259fb37 100644
@@ -9,7 +9,6 @@
 #include <bootmeth.h>
 #include <bootstd.h>
 #include <dm.h>
-#include <test/suites.h>
 #include <test/ut.h>
 #include "bootstd_common.h"
 
test/boot/bootstd_common.c +0 -1
index 6a84810983c..052c0fe5cc6 100644
@@ -13,7 +13,6 @@
 #include <mmc.h>
 #include <usb.h>
 #include <linux/log2.h>
-#include <test/suites.h>
 #include <test/ut.h>
 #include <u-boot/crc.h>
 #include "bootstd_common.h"
test/boot/expo.c +0 -1
index db14ff86f8b..1d283a2ac95 100644
@@ -10,7 +10,6 @@
 #include <menu.h>
 #include <video.h>
 #include <linux/input.h>
-#include <test/suites.h>
 #include <test/ut.h>
 #include "bootstd_common.h"
 #include <test/cedit-test.h>
test/boot/image.c +0 -1
index 0894e30587f..4df7b17ce88 100644
@@ -7,7 +7,6 @@
  */
 
 #include <image.h>
-#include <test/suites.h>
 #include <test/ut.h>
 #include "bootstd_common.h"
 
test/boot/measurement.c +0 -1
index 5a49c7a6b23..1d38663fc0f 100644
@@ -8,7 +8,6 @@
 
 #include <bootm.h>
 #include <malloc.h>
-#include <test/suites.h>
 #include <test/test.h>
 #include <test/ut.h>
 #include <asm/io.h>
test/boot/upl.c +0 -1
index aa58cdf083b..eec89026fc3 100644
@@ -10,7 +10,6 @@
 #include <mapmem.h>
 #include <upl.h>
 #include <dm/ofnode.h>
-#include <test/suites.h>
 #include <test/test.h>
 #include <test/ut.h>
 #include "bootstd_common.h"
test/boot/vbe_simple.c +0 -1
index 2fe74e1f6c6..c37de627c52 100644
@@ -11,7 +11,6 @@
 #include <image.h>
 #include <of_live.h>
 #include <vbe.h>
-#include <test/suites.h>
 #include <test/ut.h>
 #include "bootstd_common.h"
 
test/cmd/addrmap.c +0 -1
index 1f2deb15052..72798b96edd 100644
@@ -6,7 +6,6 @@
  */
 
 #include <console.h>
-#include <test/suites.h>
 #include <test/ut.h>
 
 /* Declare a new addrmap test */
test/cmd/bdinfo.c +0 -1
index 11f717bf43e..78055eaceef 100644
@@ -10,7 +10,6 @@
 #include <mapmem.h>
 #include <asm/global_data.h>
 #include <dm/uclass.h>
-#include <test/suites.h>
 #include <test/ut.h>
 #include <dm.h>
 #include <env.h>
test/cmd/exit.c +0 -1
index 71c37edcdf6..fdde054b928 100644
@@ -8,7 +8,6 @@
 #include <console.h>
 #include <mapmem.h>
 #include <asm/global_data.h>
-#include <test/suites.h>
 #include <test/ut.h>
 
 DECLARE_GLOBAL_DATA_PTR;
test/cmd/fdt.c +0 -1
index ab6dbd45e54..c11c181c807 100644
@@ -10,7 +10,6 @@
 #include <mapmem.h>
 #include <asm/global_data.h>
 #include <linux/libfdt.h>
-#include <test/suites.h>
 #include <test/ut.h>
 
 DECLARE_GLOBAL_DATA_PTR;
test/cmd/font.c +0 -1
index af88d1b5459..7ae648d7395 100644
@@ -8,7 +8,6 @@
 #include <console.h>
 #include <dm.h>
 #include <video_console.h>
-#include <test/suites.h>
 #include <test/ut.h>
 
 /* Declare a new fdt test */
test/cmd/loadm.c +0 -1
index 3c623aa655f..043cd25dfb6 100644
@@ -13,7 +13,6 @@
 #include <mapmem.h>
 #include <asm/global_data.h>
 #include <dm/test.h>
-#include <test/suites.h>
 #include <test/test.h>
 #include <test/ut.h>
 
test/cmd/mbr.c +0 -1
index 45bab04923a..e651256a4cb 100644
@@ -15,7 +15,6 @@
 #include <dm/device-internal.h>
 #include <dm/lists.h>
 #include <linux/sizes.h>
-#include <test/suites.h>
 #include <test/ut.h>
 
 DECLARE_GLOBAL_DATA_PTR;
test/cmd/pci_mps.c +0 -1
index 8b3ea4a6134..6618c247d13 100644
@@ -8,7 +8,6 @@
  */
 
 #include <console.h>
-#include <test/suites.h>
 #include <test/ut.h>
 
 #define PCI_MPS_TEST(_name, _flags) UNIT_TEST(_name, _flags, pci_mps)
test/cmd/seama.c +0 -1
index 1edc3fcac5a..39f85f1c502 100644
@@ -7,7 +7,6 @@
 
 #include <command.h>
 #include <dm.h>
-#include <test/suites.h>
 #include <test/test.h>
 #include <test/ut.h>
 
test/cmd/setexpr.c +0 -1
index 7f318a42ead..3bfcc9aad72 100644
@@ -9,7 +9,6 @@
 #include <console.h>
 #include <mapmem.h>
 #include <dm/test.h>
-#include <test/suites.h>
 #include <test/ut.h>
 
 #define BUF_SIZE	0x100
test/cmd_ut.c +0 -1
index 31ded16325f..b44e60d5a87 100644
@@ -7,7 +7,6 @@
 #include <command.h>
 #include <console.h>
 #include <vsprintf.h>
-#include <test/suites.h>
 #include <test/test.h>
 #include <test/ut.h>
 
test/common/bloblist.c +0 -1
index 9467abfa8e1..44a2310b390 100644
@@ -6,7 +6,6 @@
 #include <bloblist.h>
 #include <log.h>
 #include <mapmem.h>
-#include <test/suites.h>
 #include <test/test.h>
 #include <test/ut.h>
 
test/env/cmd_ut_env.c +0 -1
index 81d1bb2f80d..43f1b7d1cef 100644
@@ -5,7 +5,6 @@
  */
 
 #include <test/env.h>
-#include <test/suites.h>
 #include <test/ut.h>
 
 static int env_test_env_cmd(struct unit_test_state *uts)
test/fdt_overlay/cmd_ut_fdt_overlay.c +0 -1
index 3244f2d3582..0084033b6ca 100644
@@ -16,7 +16,6 @@
 
 #include <test/fdt_overlay.h>
 #include <test/ut.h>
-#include <test/suites.h>
 
 /* 4k ought to be enough for anybody */
 #define FDT_COPY_SIZE	(4 * SZ_1K)
test/log/cont_test.c +0 -1
index 32b1c792367..3b3b791f025 100644
@@ -9,7 +9,6 @@
 #include <asm/global_data.h>
 #include <test/log.h>
 #include <test/test.h>
-#include <test/suites.h>
 #include <test/ut.h>
 
 DECLARE_GLOBAL_DATA_PTR;
test/log/nolog_test.c +0 -1
index 341dbfc9310..1913e6817a7 100644
@@ -13,7 +13,6 @@
 #include <asm/global_data.h>
 #include <test/log.h>
 #include <test/test.h>
-#include <test/suites.h>
 #include <test/ut.h>
 
 DECLARE_GLOBAL_DATA_PTR;
test/log/pr_cont_test.c +0 -1
index 7734e927f98..67d8ac59db9 100644
@@ -8,7 +8,6 @@
 #include <console.h>
 #include <test/log.h>
 #include <test/test.h>
-#include <test/suites.h>
 #include <test/ut.h>
 #include <asm/global_data.h>
 #include <linux/printk.h>
test/log/syslog_test.c +0 -1
index c4180f775b9..98b91436580 100644
@@ -15,7 +15,6 @@
 #include <hexdump.h>
 #include <test/log.h>
 #include <test/test.h>
-#include <test/suites.h>
 #include <test/ut.h>
 #include <asm/eth.h>
 #include "syslog_test.h"
test/log/syslog_test_ndebug.c +0 -1
index b10e636812b..dfd0217c1e4 100644
@@ -12,7 +12,6 @@
 #include <hexdump.h>
 #include <test/log.h>
 #include <test/test.h>
-#include <test/suites.h>
 #include <test/ut.h>
 #include <asm/eth.h>
 #include "syslog_test.h"
test/optee/optee.c +0 -1
index df7b877025f..658621fa2fa 100644
@@ -14,7 +14,6 @@
 
 #include <test/ut.h>
 #include <test/optee.h>
-#include <test/suites.h>
 
 /* 4k ought to be enough for anybody */
 #define FDT_COPY_SIZE	(4 * SZ_1K)
Signature · 1 line
2.43.0
#25 lore raw

PATCH 25/28 test: Allow running a selection of suites

In reply to Simon Glass

Enhance the ut command to accept a comma-separated list of test suites to run. Report the summary information for these at the end.

---

 test/cmd_ut.c               | 32 ++++++++++++++++++++------------
 test/py/tests/test_suite.py |  5 +++++
 test/test-main.c            |  2 +-
 3 files changed, 26 insertions(+), 13 deletions(-)
test/cmd_ut.c +20 -12
index b44e60d5a87..2ef64d1182b 100644
@@ -218,7 +218,6 @@ static int do_ut_all(struct unit_test_state *uts, struct cmd_tbl *cmdtp,
 			update_stats(uts, ste);
 		}
 	}
-	ut_report(&uts->total, uts->run_count);
 
 	return any_fail;
 }
@@ -282,7 +281,7 @@ static int do_ut(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[])
 {
 	struct unit_test_state uts;
 	struct suite *ste;
-	const char *name;
+	char *name;
 	int ret;
 
 	if (argc < 2)
@@ -299,17 +298,26 @@ static int do_ut(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[])
 	} else if (!strcmp(name, "info")) {
 		ret = do_ut_info(cmdtp, flag, argc, argv);
 	} else {
-		ste = find_suite(argv[0]);
-		if (!ste) {
-			printf("Suite '%s' not found\n", argv[0]);
-			return CMD_RET_FAILURE;
-		} else if (!has_tests(ste)) {
-			/* perhaps a Kconfig option needs to be set? */
-			printf("Suite '%s' is not enabled\n", argv[0]);
-			return CMD_RET_FAILURE;
-		}
+		int any_fail = 0;
+		const char *p;
+
+		for (; p = strtok(name, ","), p; name = NULL) {
+			ste = find_suite(p);
+			if (!ste) {
+				printf("Suite '%s' not found\n", p);
+				return CMD_RET_FAILURE;
+			} else if (!has_tests(ste)) {
+				/* perhaps a Kconfig option needs to be set? */
+				printf("Suite '%s' is not enabled\n", p);
+				return CMD_RET_FAILURE;
+			}
 
-		ret = run_suite(&uts, ste, cmdtp, flag, argc, argv);
+			ret = run_suite(&uts, ste, cmdtp, flag, argc, argv);
+			if (!any_fail)
+				any_fail = ret;
+			update_stats(&uts, ste);
+		}
+		ret = any_fail;
 	}
 	show_stats(&uts);
 	if (ret)
test/py/tests/test_suite.py +5 -0
index 1e02d67efe2..d0025a7ba30 100644
@@ -187,3 +187,8 @@ def xtest_suite(u_boot_console, u_boot_config):
 
         assert suite_count == len(EXPECTED_SUITES)
         assert total_test_count == len(all_tests)
+
+    # Run three suites
+    with cons.log.section('Check multiple suites'):
+        output = cons.run_command('ut bloblist,setexpr,mem')
+    assert 'Suites run: 3' in output
test/test-main.c +1 -1
index ee855bfe2ed..cabc736a524 100644
@@ -683,7 +683,7 @@ static int ut_run_tests(struct unit_test_state *uts, const char *prefix,
 void ut_report(struct ut_stats *stats, int run_count)
 {
 	if (run_count > 1)
-		printf("Total tests");
+		printf("Suites run: %d, total tests", run_count);
 	else
 		printf("Tests");
 	printf(" run: %d, ", stats->test_count);
Signature · 1 line
2.43.0
#26 lore raw

PATCH 26/28 test: Move code out of cmd_ut_category()

In reply to Simon Glass

Move the logic from this function into run_suite(), on the way to having flag parsing in the top-level 'ut' command instead of its children.

---

 test/cmd_ut.c | 62 +++++++++++++++++++++------------------------------
 1 file changed, 25 insertions(+), 37 deletions(-)
test/cmd_ut.c +25 -37
index 2ef64d1182b..2e62081a3dc 100644
@@ -34,40 +34,6 @@ static int do_ut_all(struct unit_test_state *uts, struct cmd_tbl *cmdtp,
 static int do_ut_info(struct cmd_tbl *cmdtp, int flag, int argc,
 		      char *const argv[]);
 
-static int cmd_ut_category(struct unit_test_state *uts, const char *name,
-			   const char *prefix, struct unit_test *tests,
-			   int n_ents, int argc, char *const argv[])
-{
-	const char *test_insert = NULL;
-	int runs_per_text = 1;
-	bool force_run = false;
-	int ret;
-
-	while (argc > 1 && *argv[1] == '-') {
-		const char *str = argv[1];
-
-		switch (str[1]) {
-		case 'r':
-			runs_per_text = dectoul(str + 2, NULL);
-			break;
-		case 'f':
-			force_run = true;
-			break;
-		case 'I':
-			test_insert = str + 2;
-			break;
-		}
-		argv++;
-		argc--;
-	}
-
-	ret = ut_run_list(uts, name, prefix, tests, n_ents,
-			  cmd_arg1(argc, argv), runs_per_text, force_run,
-			  test_insert);
-
-	return ret ? CMD_RET_FAILURE : 0;
-}
-
 /* declare linker-list symbols for the start and end of a suite */
 #define SUITE_DECL(_name) \
 	ll_start_decl(suite_start_ ## _name, struct unit_test, ut_ ## _name); \
@@ -157,14 +123,36 @@ static int run_suite(struct unit_test_state *uts, struct suite *ste,
 		     char *const argv[])
 {
 	int n_ents = ste->end - ste->start;
+	const char *test_insert = NULL;
+	int runs_per_text = 1;
+	bool force_run = false;
 	char prefix[30];
 	int ret;
 
-
 	/* use a standard prefix */
 	snprintf(prefix, sizeof(prefix), "%s_test_", ste->name);
-	ret = cmd_ut_category(uts, ste->name, prefix, ste->start, n_ents,
-			      argc, argv);
+
+	while (argc > 1 && *argv[1] == '-') {
+		const char *str = argv[1];
+
+		switch (str[1]) {
+		case 'r':
+			runs_per_text = dectoul(str + 2, NULL);
+			break;
+		case 'f':
+			force_run = true;
+			break;
+		case 'I':
+			test_insert = str + 2;
+			break;
+		}
+		argv++;
+		argc--;
+	}
+
+	ret = ut_run_list(uts, ste->name, prefix, ste->start, n_ents,
+			  cmd_arg1(argc, argv), runs_per_text, force_run,
+			  test_insert);
 
 	return ret;
 }
Signature · 1 line
2.43.0
#27 lore raw

PATCH 27/28 test: Do flag-processing in the correct place

In reply to Simon Glass

At present the 'ut' command handles its flags in a strange way, in that they must come after the subcommand.

So, we must use 'ut bloblist -r2' to run the bloblist tests twice. This is an artefact of the way tests were run, through subcommands.

It is now possible to correct this, by doing flag-processing before running the suite.

Update the code to handle this, so that 'ut -r2 bloblist' works. Update the 'test_suite' test to check the new arguments.

Add a sanity-check for -I while we are here.

---

 test/cmd_ut.c               | 93 +++++++++++++++++++------------------
 test/py/tests/test_suite.py | 20 ++++++--
 2 files changed, 66 insertions(+), 47 deletions(-)
test/cmd_ut.c +49 -44
index 2e62081a3dc..07d0a91f697 100644
@@ -28,11 +28,11 @@ struct suite {
 	const char *help;
 };
 
-static int do_ut_all(struct unit_test_state *uts, struct cmd_tbl *cmdtp,
-		     int flag, int argc, char *const argv[]);
+static int do_ut_all(struct unit_test_state *uts, const char *select_name,
+		     int runs_per_test, bool force_run,
+		     const char *test_insert);
 
-static int do_ut_info(struct cmd_tbl *cmdtp, int flag, int argc,
-		      char *const argv[]);
+static int do_ut_info(bool show_suites);
 
 /* declare linker-list symbols for the start and end of a suite */
 #define SUITE_DECL(_name) \
@@ -119,40 +119,18 @@ static bool has_tests(struct suite *ste)
 
 /** run_suite() - Run a suite of tests */
 static int run_suite(struct unit_test_state *uts, struct suite *ste,
-		     struct cmd_tbl *cmdtp, int flag, int argc,
-		     char *const argv[])
+		     const char *select_name, int runs_per_test, bool force_run,
+		     const char *test_insert)
 {
 	int n_ents = ste->end - ste->start;
-	const char *test_insert = NULL;
-	int runs_per_text = 1;
-	bool force_run = false;
 	char prefix[30];
 	int ret;
 
 	/* use a standard prefix */
 	snprintf(prefix, sizeof(prefix), "%s_test_", ste->name);
 
-	while (argc > 1 && *argv[1] == '-') {
-		const char *str = argv[1];
-
-		switch (str[1]) {
-		case 'r':
-			runs_per_text = dectoul(str + 2, NULL);
-			break;
-		case 'f':
-			force_run = true;
-			break;
-		case 'I':
-			test_insert = str + 2;
-			break;
-		}
-		argv++;
-		argc--;
-	}
-
 	ret = ut_run_list(uts, ste->name, prefix, ste->start, n_ents,
-			  cmd_arg1(argc, argv), runs_per_text, force_run,
-			  test_insert);
+			  select_name, runs_per_test, force_run, test_insert);
 
 	return ret;
 }
@@ -187,8 +165,8 @@ static void update_stats(struct unit_test_state *uts, const struct suite *ste)
 	}
 }
 
-static int do_ut_all(struct unit_test_state *uts, struct cmd_tbl *cmdtp,
-		     int flag, int argc, char *const argv[])
+static int do_ut_all(struct unit_test_state *uts, const char *select_name,
+		     int runs_per_test, bool force_run, const char *test_insert)
 {
 	int i;
 	int retval;
@@ -196,11 +174,11 @@ static int do_ut_all(struct unit_test_state *uts, struct cmd_tbl *cmdtp,
 
 	for (i = 0; i < ARRAY_SIZE(suites); i++) {
 		struct suite *ste = &suites[i];
-		char *const argv[] = {(char *)ste->name, NULL};
 
 		if (has_tests(ste)) {
 			printf("----Running %s tests----\n", ste->name);
-			retval = run_suite(uts, ste, cmdtp, flag, 1, argv);
+			retval = run_suite(uts, ste, select_name, runs_per_test,
+					   force_run, test_insert);
 			if (!any_fail)
 				any_fail = retval;
 			update_stats(uts, ste);
@@ -210,11 +188,9 @@ static int do_ut_all(struct unit_test_state *uts, struct cmd_tbl *cmdtp,
 	return any_fail;
 }
 
-static int do_ut_info(struct cmd_tbl *cmdtp, int flag, int argc,
-		      char *const argv[])
+static int do_ut_info(bool show_suites)
 {
 	int suite_count, i;
-	const char *flags;
 
 	for (suite_count = 0, i = 0; i < ARRAY_SIZE(suites); i++) {
 		struct suite *ste = &suites[i];
@@ -226,8 +202,7 @@ static int do_ut_info(struct cmd_tbl *cmdtp, int flag, int argc,
 	printf("Test suites: %d\n", suite_count);
 	printf("Total tests: %d\n", (int)UNIT_TEST_ALL_COUNT());
 
-	flags = cmd_arg1(argc, argv);
-	if (flags && !strcmp("-s", flags)) {
+	if (show_suites) {
 		int i, total;
 
 		puts("\nTests  Suite         Purpose");
@@ -267,24 +242,53 @@ static struct suite *find_suite(const char *name)
 
 static int do_ut(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[])
 {
+	const char *test_insert = NULL, *select_name;
 	struct unit_test_state uts;
+	bool show_suites = false;
+	bool force_run = false;
+	int runs_per_text = 1;
 	struct suite *ste;
 	char *name;
 	int ret;
 
-	if (argc < 2)
-		return CMD_RET_USAGE;
-
 	/* drop initial "ut" arg */
 	argc--;
 	argv++;
 
+	while (argc > 0 && *argv[0] == '-') {
+		const char *str = argv[0];
+
+		switch (str[1]) {
+		case 'r':
+			runs_per_text = dectoul(str + 2, NULL);
+			break;
+		case 'f':
+			force_run = true;
+			break;
+		case 'I':
+			test_insert = str + 2;
+			if (!strchr(test_insert, ':'))
+				return CMD_RET_USAGE;
+			break;
+		case 's':
+			show_suites = true;
+			break;
+		}
+		argv++;
+		argc--;
+	}
+
+	if (argc < 1)
+		return CMD_RET_USAGE;
+
 	ut_init_state(&uts);
 	name = argv[0];
+	select_name = cmd_arg1(argc, argv);
 	if (!strcmp(name, "all")) {
-		ret = do_ut_all(&uts, cmdtp, flag, argc, argv);
+		ret = do_ut_all(&uts, select_name, runs_per_text, force_run,
+				test_insert);
 	} else if (!strcmp(name, "info")) {
-		ret = do_ut_info(cmdtp, flag, argc, argv);
+		ret = do_ut_info(show_suites);
 	} else {
 		int any_fail = 0;
 		const char *p;
@@ -300,7 +304,8 @@ static int do_ut(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[])
 				return CMD_RET_FAILURE;
 			}
 
-			ret = run_suite(&uts, ste, cmdtp, flag, argc, argv);
+			ret = run_suite(&uts, ste, select_name, runs_per_text,
+					force_run, test_insert);
 			if (!any_fail)
 				any_fail = ret;
 			update_stats(&uts, ste);
test/py/tests/test_suite.py +17 -3
index d0025a7ba30..9ddc883394b 100644
@@ -135,7 +135,7 @@ def xtest_suite(u_boot_console, u_boot_config):
 
        - The number of suites matches that reported by the 'ut info'
        - Where available, the number of tests is each suite matches that
-         reported by 'ut info -s'
+         reported by 'ut -s info'
        - The total number of tests adds up to the total that are actually run
          with 'ut all'
        - All suites are run with 'ut all'
@@ -167,7 +167,7 @@ def xtest_suite(u_boot_console, u_boot_config):
 
     # Run 'ut info' and compare with the log results
     with cons.log.section('Check suite test-counts'):
-        output = cons.run_command('ut info -s')
+        output = cons.run_command('ut -s info')
 
         suite_count, total_test_count, test_count = process_ut_info(cons,
                                                                     output)
@@ -191,4 +191,18 @@ def xtest_suite(u_boot_console, u_boot_config):
     # Run three suites
     with cons.log.section('Check multiple suites'):
         output = cons.run_command('ut bloblist,setexpr,mem')
-    assert 'Suites run: 3' in output
+        assert 'Suites run: 3' in output
+
+    # Run a particular test
+    with cons.log.section('Check single test'):
+        output = cons.run_command('ut bloblist reloc')
+        assert 'Test: reloc: bloblist.c' in output
+
+    # Run tests multiple times
+    with cons.log.section('Check multiple runs'):
+        output = cons.run_command('ut -r2 bloblist')
+        lines = output.splitlines()
+        run = len([line for line in lines if 'Test:' in line])
+        count = re.search(r'Tests run: (\d*)', lines[-1]).group(1)
+
+        assert run == 2 * int(count)
Signature · 1 line
2.43.0
#28 lore raw

PATCH 28/28 test: Update documentation

In reply to Simon Glass

Update documentation for how to write tests and the 'ut' command.

---

 doc/develop/tests_writing.rst |  52 ++++++------
 doc/usage/cmd/ut.rst          | 145 ++++++++++++++++++++++++++--------
 test/cmd_ut.c                 |  12 +--
 3 files changed, 149 insertions(+), 60 deletions(-)
doc/develop/tests_writing.rst +29 -23
index 54efb7e1b04..5f3c43d5da2 100644
@@ -261,7 +261,7 @@ with the suite. For example, to add a new mem_search test::
    /* Test 'ms' command with 32-bit values */
    static int mem_test_ms_new_thing(struct unit_test_state *uts)
    {
-         /* test code here*/
+         /* test code here */
 
          return 0;
    }
@@ -291,32 +291,20 @@ suite. For example::
    /* Declare a new wibble test */
    #define WIBBLE_TEST(_name, _flags)   UNIT_TEST(_name, _flags, wibble_test)
 
-   /* Tetss go here */
-
-   /* At the bottom of the file: */
-
-   int do_ut_wibble(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[])
-   {
-     struct unit_test *tests = UNIT_TEST_SUITE_START(wibble_test);
-     const int n_ents = UNIT_TEST_SUITE_COUNT(wibble_test);
-
-     return cmd_ut_category("cmd_wibble", "wibble_test_", tests, n_ents, argc, argv);
-   }
+   /* Tests go here */
 
 Then add new tests to it as above.
 
 Register this new suite in test/cmd_ut.c by adding to cmd_ut_sub[]::
 
-  /* Within cmd_ut_sub[]... */
-
-  U_BOOT_CMD_MKENT(wibble, CONFIG_SYS_MAXARGS, 1, do_ut_wibble, "", ""),
+  /* with the other SUITE_DECL() declarations */
+  SUITE_DECL(wibble);
 
-and adding new help to ut_help_text[]::
+  /* Within suites[]... */
+  SUITE(wibble, "my test of wibbles");
 
-  "ut wibble - Test the wibble feature\n"
-
-If your feature is conditional on a particular Kconfig, then you can use #ifdef
-to control that.
+If your feature is conditional on a particular Kconfig, you do not need to add
+an #ifdef since the suite will automatically be compiled out in that case.
 
 Finally, add the test to the build by adding to the Makefile in the same
 directory::
@@ -326,17 +314,35 @@ directory::
 Note that CMDLINE is never enabled in SPL, so this test will only be present in
 U-Boot proper. See below for how to do SPL tests.
 
-As before, you can add an extra Kconfig check if needed::
+You can add an extra Kconfig check if needed::
 
   ifneq ($(CONFIG_$(XPL_)WIBBLE),)
   obj-$(CONFIG_$(XPL_)CMDLINE) += wibble.o
   endif
 
+Each suite can have an optional init and uninit function. These are run before
+and after any suite tests, respectively::
+
+   #define WIBBLE_TEST_INIT(_name, _flags)  UNIT_TEST_INIT(_name, _flags, wibble_test)
+   #define WIBBLE_TEST_UNINIT(_name, _flags)  UNIT_TEST_UNINIT(_name, _flags, wibble_test)
 
-Example commit: 919e7a8fb64 ("test: Add a simple test for bloblist") [1]
+   static int wibble_test_init(struct unit_test_state *uts)
+   {
+         /* init code here */
+
+         return 0;
+   }
+   WIBBLE_TEST_INIT(wibble_test_init, 0);
 
-[1] https://gitlab.denx.de/u-boot/u-boot/-/commit/919e7a8fb64
+   static int wibble_test_uninit(struct unit_test_state *uts)
+   {
+         /* uninit code here */
+
+         return 0;
+   }
+   WIBBLE_TEST_INIT(wibble_test_uninit, 0);
 
+Both functions are included in the totals for each suite.
 
 Making the test run from pytest
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
doc/usage/cmd/ut.rst +113 -32

182 lines not on this page · raw message

test/cmd_ut.c +7 -5
index 07d0a91f697..d00f5bff242 100644
@@ -321,14 +321,16 @@ static int do_ut(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[])
 }
 
 U_BOOT_LONGHELP(ut,
-	"[-r] [-f] [<suite>] - run unit tests\n"
+	"[-rs] [-f] [-I<n>:<one_test>][<suites>] - run unit tests\n"
 	"   -r<runs>   Number of times to run each test\n"
 	"   -f         Force 'manual' tests to run as well\n"
-	"   <suite>    Test suite to run, or all\n"
+	"   -I         Test to run after <n> other tests have run\n"
+	"   -s         Show all suites with ut info\n"
+	"   <suites>   Comma-separated list of suites to run\n"
 	"\n"
-	"\nOptions for <suite>:"
-	"\nall - execute all enabled tests"
-	"\ninfo [-s] - show info about tests [and suites]"
+	"Options for <suite>:\n"
+	"all       - execute all enabled tests\n"
+	"info      - show info about tests [and suites]"
 	);
 
 U_BOOT_CMD(
Signature · 1 line
2.43.0
#29 lore raw
In reply to Simon Glass
Quoted 6 lines · Simon Glass

This series completes the removal of test commands for suites. With this it is possible to declare a suite (including init and uninit functions) without needing to write a command.

It also adds timing for test suites, so we can keep track of how long things take.

This causes ut_dm_dm_test_cmd_mux_list to fail now:
https://source.denx.de/u-boot/u-boot/-/jobs/1001470

There was a trivial rebase needed for one patch, but the rest of the series looks otherwise fine.

Signature · 1 line
Tom
#30 lore raw

Hi Tom,

Quoted 7 lines · Tom Rini
Quoted 4 lines · Simon Glass

This series completes the removal of test commands for suites. With this it is possible to declare a suite (including init and uninit functions) without needing to write a command.

Quoted 5 lines
Quoted 2 lines

It also adds timing for test suites, so we can keep track of how long things take.

This causes ut_dm_dm_test_cmd_mux_list to fail now:
https://source.denx.de/u-boot/u-boot/-/jobs/1001470

Yes, I saw that as well, but i figured it was unrelated. There's definitely something odd going on in the tests :-( Have you seen any other such random failures?

Quoted 3 lines

There was a trivial rebase needed for one patch, but the rest of the series looks otherwise fine.

OK, ta.

Regards,
Simon

#31 lore raw
Quoted 13 lines · Simon Glass

Hi Tom,

Quoted 10 lines · Tom Rini
Quoted 6 lines · Simon Glass

This series completes the removal of test commands for suites. With this it is possible to declare a suite (including init and uninit functions) without needing to write a command.

It also adds timing for test suites, so we can keep track of how long things take.

Quoted 6 lines
Quoted 2 lines

This causes ut_dm_dm_test_cmd_mux_list to fail now:
https://source.denx.de/u-boot/u-boot/-/jobs/1001470

Yes, I saw that as well, but i figured it was unrelated. There's definitely something odd going on in the tests :-( Have you seen any other such random failures?

Only the hash md5 one I've mentioned before. I didn't bisect this one down to a specific commit since I was testing other things.

Signature · 1 line
Tom