test: allow taking test names from commandline
PATCH 1/1 test: allow taking test names from commandline
While having the ability to run a test based off the DPDK_TEST environment variable is useful, it's often easier to specify the test name as a commandline parameter to a test binary. This also allows the test runs to be saved as part of the shell cmdline history.
This patch adds support for checking all parameters after the EAL ones, and running all valid autotests requested - either from DPDK_TEST or on the commandline. This also allows multiple tests to be run in a single automated session, which is useful for working with components which have multiple test suites.
--- app/test/commands.c | 11 +++++++++++ app/test/test.c | 40 +++++++++++++++++++++++++++++++++------- app/test/test.h | 1 + 3 files changed, 45 insertions(+), 7 deletions(-)
app/test/commands.c +11 -0
@@ -381,3 +381,14 @@ int commands_init(void) cmd_autotest_autotest.string_data.str = commands;return 0 ; } + +int command_valid (const char *cmd) +{ +struct test_command *t; + + TAILQ_FOREACH(t, &commands_list, next) { +if (strcmp (t->command, cmd) ==0 ) +return 1 ; + } +return 0 ; +}
app/test/test.c +33 -7
@@ -93,6 +93,9 @@ main(int argc, char **argv) {#ifdef RTE_LIB_CMDLINE struct cmdline *cl; +char *tests[argc];/* store an array of tests to run */ +int test_count =0 ; +int i;#endif char *extra_args;int ret;
@@ -147,6 +150,7 @@ main(int argc, char **argv) } argv += ret; + argc -= ret; prgname = argv[0 ];
@@ -165,7 +169,25 @@ main(int argc, char **argv)#ifdef RTE_LIB_CMDLINE char *dpdk_test =getenv ("DPDK_TEST" ); -if (dpdk_test &&strlen (dpdk_test)) { + +if (dpdk_test &&strlen (dpdk_test) ==0 ) + dpdk_test =NULL ; + +if (dpdk_test && !command_valid (dpdk_test)) { +RTE_LOG (WARNING ,APP ,"Invalid DPDK_TEST value '%s'\n" , dpdk_test); + dpdk_test =NULL ; + } + +if (dpdk_test) + tests[test_count++] = dpdk_test; +for (i =1 ; i < argc; i++) { +if (!command_valid (argv[i])) +RTE_LOG (WARNING ,APP ,"Invalid test requested: '%s'\n" , argv[i]); +else + tests[test_count++] = argv[i]; + } + +if (test_count >0 ) {char buf[1024 ]; cl =cmdline_new (main_ctx,"RTE>>" ,0 ,1 );
@@ -174,13 +196,17 @@ main(int argc, char **argv) goto out; } -snprintf (buf,sizeof (buf),"%s\n" , dpdk_test); -if (cmdline_in (cl, buf,strlen (buf)) <0 ) { -printf ("error on cmdline input\n" ); +for (i =0 ; i < test_count; i++) { +snprintf (buf,sizeof (buf),"%s\n" , tests[i]); +if (cmdline_in (cl, buf,strlen (buf)) <0 ) { +printf ("error on cmdline input\n" ); - ret =-1 ; - }else { - ret = last_test_result; + ret =-1 ; + }else + ret = last_test_result; + +if (ret !=0 ) +break ; }cmdline_free (cl); goto out;
app/test/test.h +1 -0
@@ -152,6 +152,7 @@ extern int last_test_result;extern const char *prgname;int commands_init (void ); +int command_valid (const char *cmd);int test_mp_secondary (void );int test_timer_secondary (void );
--
2.27.0
Quoted 5 lines · Bruce Richardson
While having the ability to run a test based off the
DPDK_TESTenvironment variable is useful, it's often easier to specify the test name as a commandline parameter to a test binary. This also allows the test runs to be saved as part of the shell cmdline history.
Quoted 8 lines
This patch adds support for checking all parameters after the EAL ones, and running all valid autotests requested - either from
DPDK_TESTor on the commandline. This also allows multiple tests to be run in a single automated session, which is useful for working with components which have multiple test suites.Signed-off-by Bruce Richardson <[email protected]>---
LGTM, and I think it can be useful for running specific tests when developers make changes.
Quoted 8 lines · Aaron Conole
Quoted 5 lines · Bruce Richardson
While having the ability to run a test based off the
DPDK_TESTenvironment variable is useful, it's often easier to specify the test name as a commandline parameter to a test binary. This also allows the test runs to be saved as part of the shell cmdline history.
Quoted 8 lines
Quoted 7 lines
This patch adds support for checking all parameters after the EAL ones, and running all valid autotests requested - either from
DPDK_TESTor on the commandline. This also allows multiple tests to be run in a single automated session, which is useful for working with components which have multiple test suites.Signed-off-by Bruce Richardson <[email protected]>Acked-by Aaron Conole <[email protected]>
Applied with updated commitlog, thanks.
Signature · 1 line
David Marchand
Quoted 5 lines · Bruce Richardson
While having the ability to run a test based off the
DPDK_TESTenvironment variable is useful, it's often easier to specify the test name as a commandline parameter to a test binary. This also allows the test runs to be saved as part of the shell cmdline history.
I don't get the argument about history:
$ history |grep DPDK_TEST 10615 2021-03-24 10:42:11 ninja-build -C build -j4 && DPDK_TEST=logs_autotest ./build/app/test/dpdk-test --no-huge -m 512 --log-level=lib.eal:debug 10636 2021-03-24 10:51:09 ninja-build -C build -j4 && DPDK_TEST=logs_autotest ./build/app/test/dpdk-test --no-huge -m 512 --log-level=lib.eal:debug 10653 2021-03-24 11:17:01 ninja-build -C build -j4 && DPDK_TEST=kvargs_autotest ./build/app/test/dpdk-test --no-huge -m 512 --log-level=lib.eal:debug 10794 2021-03-25 18:37:48 history |grep DPDK_TEST
Quoted 6 lines
This patch adds support for checking all parameters after the EAL ones, and running all valid autotests requested - either from
DPDK_TESTor on the commandline. This also allows multiple tests to be run in a single automated session, which is useful for working with components which have multiple test suites.
The same could be achieved splitting DPDK_TEST content with spaces, since test names don't contain one.
Signature · 1 line
David Marchand
Quoted 10 lines · David Marchand
Quoted 5 lines · Bruce Richardson
While having the ability to run a test based off the
DPDK_TESTenvironment variable is useful, it's often easier to specify the test name as a commandline parameter to a test binary. This also allows the test runs to be saved as part of the shell cmdline history.I don't get the argument about history:
Quoted 12 lines
$ history |grep DPDK_TEST 10615 2021-03-24 10:42:11 ninja-build -C build -j4 && DPDK_TEST=logs_autotest ./build/app/test/dpdk-test --no-huge -m 512 --log-level=lib.eal:debug 10636 2021-03-24 10:51:09 ninja-build -C build -j4 && DPDK_TEST=logs_autotest ./build/app/test/dpdk-test --no-huge -m 512 --log-level=lib.eal:debug 10653 2021-03-24 11:17:01 ninja-build -C build -j4 && DPDK_TEST=kvargs_autotest ./build/app/test/dpdk-test --no-huge -m 512 --log-level=lib.eal:debug 10794 2021-03-25 18:37:48 history |grep DPDK_TEST
Sure, if you always specify the test name explicitly for each command, rather than running the one test multiple times having set it separately in the environment.
Overall, though I take the point that from a history saving point of view it's a minor saving.
Quoted 11 lines
Quoted 7 lines
This patch adds support for checking all parameters after the EAL ones, and running all valid autotests requested - either from
DPDK_TESTor on the commandline. This also allows multiple tests to be run in a single automated session, which is useful for working with components which have multiple test suites.The same could be achieved splitting
DPDK_TESTcontent with spaces, since test names don't contain one.
Yep, that's a useful enhancement too, but I still thing it's better to just have the list of tests appended to the test binary command rather than have to be worrying about properly quoting a specific environment variable at the start of each command.
/Bruce
Quoted 38 lines · Bruce Richardson
Quoted 10 lines · David Marchand
Quoted 5 lines · Bruce Richardson
While having the ability to run a test based off the
DPDK_TESTenvironment variable is useful, it's often easier to specify the test name as a commandline parameter to a test binary. This also allows the test runs to be saved as part of the shell cmdline history.I don't get the argument about history:
Quoted 12 lines
$ history |grep DPDK_TEST 10615 2021-03-24 10:42:11 ninja-build -C build -j4 && DPDK_TEST=logs_autotest ./build/app/test/dpdk-test --no-huge -m 512 --log-level=lib.eal:debug 10636 2021-03-24 10:51:09 ninja-build -C build -j4 && DPDK_TEST=logs_autotest ./build/app/test/dpdk-test --no-huge -m 512 --log-level=lib.eal:debug 10653 2021-03-24 11:17:01 ninja-build -C build -j4 && DPDK_TEST=kvargs_autotest ./build/app/test/dpdk-test --no-huge -m 512 --log-level=lib.eal:debug 10794 2021-03-25 18:37:48 history |grep DPDK_TESTSure, if you always specify the test name explicitly for each command, rather than running the one test multiple times having set it separately in the environment.
Overall, though I take the point that from a history saving point of view it's a minor saving.Quoted 8 lines
Quoted 7 lines
This patch adds support for checking all parameters after the EAL ones, and running all valid autotests requested - either from
DPDK_TESTor on the commandline. This also allows multiple tests to be run in a single automated session, which is useful for working with components which have multiple test suites.
Quoted 9 lines
Quoted 3 lines
The same could be achieved splitting
DPDK_TESTcontent with spaces, since test names don't contain one.Yep, that's a useful enhancement too, but I still thing it's better to just have the list of tests appended to the test binary command rather than have to be worrying about properly quoting a specific environment variable at the start of each command.
Ping on this.
Other instances where using cmdline is preferred over environment
* Calling tests using sudo.
* Calling tests as an execute action when doing a git rebase
Yes, again in both cases, other workarounds are generally available (e.g. sudo -E, and exporting to environemtn before rebase), but also generally less convenient.
/Bruce
Quoted 44 lines · Bruce Richardson
Quoted 42 lines · Bruce Richardson
Quoted 10 lines · David Marchand
Quoted 5 lines · Bruce Richardson
While having the ability to run a test based off the
DPDK_TESTenvironment variable is useful, it's often easier to specify the test name as a commandline parameter to a test binary. This also allows the test runs to be saved as part of the shell cmdline history.I don't get the argument about history:
Quoted 12 lines
$ history |grep DPDK_TEST 10615 2021-03-24 10:42:11 ninja-build -C build -j4 && DPDK_TEST=logs_autotest ./build/app/test/dpdk-test --no-huge -m 512 --log-level=lib.eal:debug 10636 2021-03-24 10:51:09 ninja-build -C build -j4 && DPDK_TEST=logs_autotest ./build/app/test/dpdk-test --no-huge -m 512 --log-level=lib.eal:debug 10653 2021-03-24 11:17:01 ninja-build -C build -j4 && DPDK_TEST=kvargs_autotest ./build/app/test/dpdk-test --no-huge -m 512 --log-level=lib.eal:debug 10794 2021-03-25 18:37:48 history |grep DPDK_TESTSure, if you always specify the test name explicitly for each command, rather than running the one test multiple times having set it separately in the environment.
Overall, though I take the point that from a history saving point of view it's a minor saving.Quoted 11 lines
Quoted 7 lines
This patch adds support for checking all parameters after the EAL ones, and running all valid autotests requested - either from
DPDK_TESTor on the commandline. This also allows multiple tests to be run in a single automated session, which is useful for working with components which have multiple test suites.The same could be achieved splitting
DPDK_TESTcontent with spaces, since test names don't contain one.
Quoted 6 lines
Quoted 5 lines
Yep, that's a useful enhancement too, but I still thing it's better to just have the list of tests appended to the test binary command rather than have to be worrying about properly quoting a specific environment variable at the start of each command.
Ping on this.
I have been using the DPDK_TEST= method for quite some time. But I don't think I ever had a need to run multiple tests since I usually track regressions or hard to reproduce failures. When developing a new test, idem, I used DPDK_TEST=.
Quoted 4 lines
Other instances where using cmdline is preferred over environment
* Calling tests using sudo.
I guess calling a shell with the same command would work, something like:
$ sudo sh -c 'DPDK_TEST=debug_autotest ..../build/app/test/dpdk-test --no-huge -m 512'
Quoted 1 line
* Calling tests as an execute action when doing a git rebase
I am pretty sure I used DPDK_TEST= in rebase execute actions in the past too. This probably works too:
$ git rebase -i HEAD^ -x 'DPDK_TEST=debug_autotest build/app/test/dpdk-test --no-huge -m 512'
Quoted 4 lines
Yes, again in both cases, other workarounds are generally available (e.g. sudo -E, and exporting to environemtn before rebase), but also generally less convenient.
You find it less convenient when you want to run multiple tests. I don't have that use case but I don't mind taking it. Can you simply adjust the commitlog, or propose a new wording I can take when applying?
Thanks.
Signature · 1 line
David Marchand
Quoted 60 lines · David Marchand
Quoted 44 lines · Bruce Richardson
Quoted 42 lines · Bruce Richardson
Quoted 10 lines · David Marchand
Quoted 5 lines · Bruce Richardson
While having the ability to run a test based off the
DPDK_TESTenvironment variable is useful, it's often easier to specify the test name as a commandline parameter to a test binary. This also allows the test runs to be saved as part of the shell cmdline history.I don't get the argument about history:
Quoted 12 lines
$ history |grep DPDK_TEST 10615 2021-03-24 10:42:11 ninja-build -C build -j4 && DPDK_TEST=logs_autotest ./build/app/test/dpdk-test --no-huge -m 512 --log-level=lib.eal:debug 10636 2021-03-24 10:51:09 ninja-build -C build -j4 && DPDK_TEST=logs_autotest ./build/app/test/dpdk-test --no-huge -m 512 --log-level=lib.eal:debug 10653 2021-03-24 11:17:01 ninja-build -C build -j4 && DPDK_TEST=kvargs_autotest ./build/app/test/dpdk-test --no-huge -m 512 --log-level=lib.eal:debug 10794 2021-03-25 18:37:48 history |grep DPDK_TESTSure, if you always specify the test name explicitly for each command, rather than running the one test multiple times having set it separately in the environment.
Overall, though I take the point that from a history saving point of view it's a minor saving.Quoted 11 lines
Quoted 7 lines
This patch adds support for checking all parameters after the EAL ones, and running all valid autotests requested - either from
DPDK_TESTor on the commandline. This also allows multiple tests to be run in a single automated session, which is useful for working with components which have multiple test suites.The same could be achieved splitting
DPDK_TESTcontent with spaces, since test names don't contain one.Quoted 6 lines
Quoted 5 lines
Yep, that's a useful enhancement too, but I still thing it's better to just have the list of tests appended to the test binary command rather than have to be worrying about properly quoting a specific environment variable at the start of each command.
Ping on this.
I have been using the DPDK_TEST= method for quite some time. But I don't think I ever had a need to run multiple tests since I usually track regressions or hard to reproduce failures. When developing a new test, idem, I used DPDK_TEST=.
Quoted 1 line
Quoted 7 lines
Quoted 3 lines
Other instances where using cmdline is preferred over environment
* Calling tests using sudo.
I guess calling a shell with the same command would work, something like:
$ sudo sh -c 'DPDK_TEST=debug_autotest ..../build/app/test/dpdk-test --no-huge -m 512'
Not exactly the simplest command, compared to just
"sudo /path/to/dpdk-test --no-huge -m512 -- debug_autotest"
Using sudo -E is what I tend to prefer in this case. :-)
Quoted 10 lines
Quoted 2 lines
* Calling tests as an execute action when doing a git rebase
I am pretty sure I used DPDK_TEST= in rebase execute actions in the past too. This probably works too:
$ git rebase -i HEAD^ -x 'DPDK_TEST=debug_autotest build/app/test/dpdk-test --no-huge -m 512'Quoted 1 line
Quoted 9 lines
Quoted 3 lines
Yes, again in both cases, other workarounds are generally available (e.g. sudo -E, and exporting to environemtn before rebase), but also generally less convenient.
You find it less convenient when you want to run multiple tests. I don't have that use case but I don't mind taking it. Can you simply adjust the commitlog, or propose a new wording I can take when applying?
Would just replacing the first paragraph as below be suitable? Since it's only a single sentence, the second paragraph can also be merged straight in with it to leave a 1-paragraph commit log message.
"While having the ability to run a test based off the DPDK_TEST environment variable is useful, it's sometimes more convenient to specify the test name as a commandline parameter to a test binary."
/Bruce