Skip to contentSkip to conversation

Add support for a test tags

PATCH Superseded Cyril Hrubis 5 participants 15 messages Oct 14, 2019 – Oct 17, 2019 Library code

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover Add support for a test tags

This is a newest version of the test tags, first patch adds a support for test tags to a test library, the second one annotates tests with tags. Not everything is annotated at the moment but fair amount of tests is included.

These patches are also a stepping stone for the test metadata project that will allow us to do much more than this. You can have a look at:

https://github.com/metan-ucw/ltp/tree/master/docparse

Also if you run a make in the docparse directory it will generate test metadata along with a single html page with nice test metadata description.

While the whole project is work-in-progress, the test tags are finished and may add a value even without the rest of the changes, see the first commit for details.

Cyril Hrubis (2):
lib: Add support for test tags
testcases/kernel: Add tags

 include/tst_test.h                            | 10 +++
 lib/tst_test.c                                | 77 +++++++++++++++++--
 testcases/cve/cve-2014-0196.c                 |  5 ++
 testcases/cve/cve-2015-3290.c                 |  5 ++
 testcases/cve/cve-2016-10044.c                |  5 ++
 testcases/cve/cve-2016-7042.c                 |  5 ++
 testcases/cve/cve-2016-7117.c                 |  5 ++
 testcases/cve/cve-2017-16939.c                |  5 ++
 testcases/cve/cve-2017-17052.c                |  5 ++
 testcases/cve/cve-2017-17053.c                |  5 ++
 testcases/cve/cve-2017-2618.c                 |  5 ++
 testcases/cve/cve-2017-2671.c                 |  5 ++
 testcases/cve/meltdown.c                      |  6 +-
 testcases/cve/stack_clash.c                   |  4 +
 testcases/kernel/crypto/af_alg01.c            |  5 ++
 testcases/kernel/crypto/af_alg02.c            |  5 ++
 testcases/kernel/crypto/af_alg03.c            |  4 +
 testcases/kernel/crypto/af_alg04.c            |  4 +
 testcases/kernel/crypto/af_alg05.c            |  8 +-
 testcases/kernel/crypto/af_alg06.c            |  4 +
 testcases/kernel/crypto/crypto_user01.c       |  6 ++
 testcases/kernel/crypto/crypto_user02.c       |  4 +
 testcases/kernel/crypto/pcrypt_aead01.c       |  5 ++
 testcases/kernel/sound/snd_timer01.c          |  6 ++
 testcases/kernel/syscalls/accept/accept02.c   |  5 ++
 testcases/kernel/syscalls/acct/acct02.c       |  4 +
 testcases/kernel/syscalls/add_key/add_key02.c |  5 ++
 testcases/kernel/syscalls/bpf/bpf_prog02.c    |  4 +
 .../syscalls/futex/futex_cmp_requeue02.c      |  5 ++
 testcases/kernel/syscalls/keyctl/keyctl02.c   |  4 +
 testcases/kernel/syscalls/keyctl/keyctl04.c   |  5 ++
 testcases/kernel/syscalls/keyctl/keyctl05.c   |  4 +
 .../kernel/syscalls/move_pages/move_pages12.c |  6 ++
 .../kernel/syscalls/realpath/realpath01.c     |  4 +
 .../syscalls/request_key/request_key05.c      |  4 +
 testcases/kernel/syscalls/sbrk/sbrk03.c       |  4 +
 testcases/kernel/syscalls/sendto/sendto02.c   |  4 +
 .../kernel/syscalls/setrlimit/setrlimit06.c   |  4 +
 38 files changed, 251 insertions(+), 9 deletions(-)
Signature · 1 line
2.21.0
#1 lore raw

PATCH 1/2 lib: Add support for test tags

The newly introduced test tags are generic name-value pairs that can hold test metadata, the intended use for now is to store kernel commit hashes for kernel reproducers as well as CVE ids. The mechanism is however choosen to be very generic so that it's easy to add basically any information later on.

As it is the main purpose is to print hints for a test failures. If a test that has been written as a kernel reproducer fails it prints nice URL pointing to a kernel commit that may be missing.

Example output:

--------------------------------------------------------------------------
tst_test.c:1145: INFO: Timeout per run is 0h 05m 00s add_key02.c:98: FAIL: unexpected error with key type 'asymmetric': EINVAL add_key02.c:98: FAIL: unexpected error with key type 'cifs.idmap': EINVAL add_key02.c:98: FAIL: unexpected error with key type 'cifs.spnego': EINVAL add_key02.c:98: FAIL: unexpected error with key type 'pkcs7_test': EINVAL add_key02.c:98: FAIL: unexpected error with key type 'rxrpc': EINVAL add_key02.c:98: FAIL: unexpected error with key type 'rxrpc_s': EINVAL add_key02.c:98: FAIL: unexpected error with key type 'user': EINVAL add_key02.c:98: FAIL: unexpected error with key type 'logon': EINVAL

HINT: This is a regression test for linux kernel, see commit:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5649645d725c

HINT: This test also tests for CVE-2017-15274, see:

https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-15274

Summary:

passed   0
failed   8
skipped  0

warnings 0
--------------------------------------------------------------------------

---
 include/tst_test.h | 10 ++++++
 lib/tst_test.c     | 77 +++++++++++++++++++++++++++++++++++++++++-----
 2 files changed, 80 insertions(+), 7 deletions(-)
include/tst_test.h +10 -0
index 84acf2c59..4a51b6d16 100644
@@ -108,6 +108,11 @@ int tst_parse_int(const char *str, int *val, int min, int max);
 int tst_parse_long(const char *str, long *val, long min, long max);
 int tst_parse_float(const char *str, float *val, float min, float max);
 
+struct tst_tag {
+	const char *name;
+	const char *value;
+};
+
 extern unsigned int tst_variant;
 
 struct tst_test {
@@ -212,6 +217,11 @@ struct tst_test {
 	 * NULL-terminated array of capability settings
 	 */
 	struct tst_cap *caps;
+
+	/*
+	 * {NULL, NULL} terminated array of tags.
+	 */
+	const struct tst_tag *tags;
 };
 
 /*
lib/tst_test.c +70 -7

120 lines not on this page · raw message

Signature · 1 line
2.21.0
#2 lore raw

PATCH 1/2 lib: Add support for test tags

Quoted 10 lines · Cyril Hrubis

The newly introduced test tags are generic name-value pairs that can hold test metadata, the intended use for now is to store kernel commit hashes for kernel reproducers as well as CVE ids. The mechanism is however choosen to be very generic so that it's easy to add basically any information later on.

As it is the main purpose is to print hints for a test failures. If a test that has been written as a kernel reproducer fails it prints nice URL pointing to a kernel commit that may be missing.

Nice to see this feature patchset achieved, it looks quite clear&good to me. Just have some tiny comments below.

Quoted 92 lines

Example output:

--------------------------------------------------------------------------
tst_test.c:1145: INFO: Timeout per run is 0h 05m 00s add_key02.c:98: FAIL: unexpected error with key type 'asymmetric': EINVAL add_key02.c:98: FAIL: unexpected error with key type 'cifs.idmap': EINVAL add_key02.c:98: FAIL: unexpected error with key type 'cifs.spnego': EINVAL add_key02.c:98: FAIL: unexpected error with key type 'pkcs7_test': EINVAL add_key02.c:98: FAIL: unexpected error with key type 'rxrpc': EINVAL add_key02.c:98: FAIL: unexpected error with key type 'rxrpc_s': EINVAL add_key02.c:98: FAIL: unexpected error with key type 'user': EINVAL add_key02.c:98: FAIL: unexpected error with key type 'logon': EINVAL

HINT: This is a regression test for linux kernel, see commit:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5649645d725c

HINT: This test also tests for CVE-2017-15274, see:

https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-15274

Summary:

passed   0
failed   8
skipped  0

warnings 0
--------------------------------------------------------------------------

---
 include/tst_test.h | 10 ++++++
 lib/tst_test.c     | 77 +++++++++++++++++++++++++++++++++++++++++-----
 2 files changed, 80 insertions(+), 7 deletions(-)
include/tst_test.h +5 -0
index 84acf2c59..4a51b6d16 100644
@@ -108,6 +108,11 @@ int tst_parse_int(const char *str, int *val, int min,
 int max);
 int tst_parse_long(const char *str, long *val, long min, long max);
 int tst_parse_float(const char *str, float *val, float min, float max);
 
+struct tst_tag {
+       const char *name;
+       const char *value;
+};
+
 extern unsigned int tst_variant;
 

struct tst_test {

include/tst_test.h +5 -0
@@ -212,6 +217,11 @@ struct tst_test {
         * NULL-terminated array of capability settings
         */
        struct tst_cap *caps;
+
+       /*
+        * {NULL, NULL} terminated array of tags.
+        */
+       const struct tst_tag *tags;
 };
 
 /*
lib/tst_test.c +1 -0
index 6239acf89..2129f38cb 100644
@@ -31,6 +31,9 @@
 #include "old_device.h"
 #include "old_tmpdir.h"
 
+#define LINUX_GIT_URL "

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=
"

lib/tst_test.c +5 -0
+#define CVE_DB_URL "https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-"
+
 struct tst_test *tst_test;
 
 static const char *tid;
@@ -414,6 +417,9 @@ static void print_help(void)
 {
        unsigned int i;
 
+       fprintf(stderr, "Options\n");
+       fprintf(stderr, "-------\n\n");
+
        for (i = 0; i < ARRAY_SIZE(options); i++)
                fprintf(stderr, "%s\n", options[i].help);
 

lib/tst_test.c static void print_help(void) · line 434

Quoted 5 lines
lib/tst_test.c +1 -0
@@ -424,6 +430,27 @@ static void print_help(void)
                fprintf(stderr, "%s\n", tst_test->options[i].help);
 }
 
+static void print_test_info(void)
 

print_test_info sounds like general information for the test, maybe print_tags() is a better/precise name?

Quoted 48 lines
lib/tst_test.c +20 -0
+{
+       unsigned int i;
+       const struct tst_tag *tags = tst_test->tags;
+
+       printf("\nTags\n");
+       printf("----\n\n");
+
+       if (tags) {
+               for (i = 0; tags[i].name; i++) {
+                       if (!strcmp(tags[i].name, "CVE"))
+                               printf(CVE_DB_URL "%s\n", tags[i].value);
+                       else if (!strcmp(tags[i].name, "linux-git"))
+                               printf(LINUX_GIT_URL "%s\n",
 tags[i].value);
+                       else
+                               printf("%s: %s\n", tags[i].name,
 tags[i].value);
+                       printf("\n");
+               }
+       }
+}
+

static void check_option_collision(void)

 {
        unsigned int i, j;
lib/tst_test.c +11 -0
@@ -499,6 +526,7 @@ static void parse_opts(int argc, char *argv[])
                break;
                case 'h':
                        print_help();
+                       print_test_info();
                        exit(0);
                case 'i':
                        iterations = atoi(optarg);
@@ -584,26 +612,61 @@ int tst_parse_float(const char *str, float *val,
 float min, float max)
        return 0;
 }
 
+static void print_colored(const char *str)
+{
+       if (tst_color_enabled(STDOUT_FILENO))
+               printf("%s%s%s", ANSI_COLOR_YELLOW, str, ANSI_COLOR_RESET);
+       else
+               printf("%s", str);
+}
+
+static void print_failure_hints(void)
+{

lib/tst_test.c int tst_parse_float(const char *str, float *val, · line 635

Quoted 10 lines
lib/tst_test.c +10 -0
+       unsigned int i;
+       const struct tst_tag *tags = tst_test->tags;
+
+       if (!tags)
+               return;
+
+       for (i = 0; tags[i].name; i++) {
+               if (!strcmp(tags[i].name, "linux-git")) {
+                       printf("\n");
+                       print_colored("HINT: ");

+ printf("This is a regression test for linux kernel,

Quoted 3 lines

see commit:\n\n"
+ LINUX_GIT_URL "%s\n", tags[i].value);

This sentence 'HINT: This is a ...' will be printed many times if there are many commits in tags, I prefer to see only once in front of these linux-kernel links.

Quoted 9 lines
lib/tst_test.c +7 -0
+               }
+
+               if (!strcmp(tags[i].name, "CVE")) {
+                       printf("\n");
+                       print_colored("HINT: ");
+                       printf("This test also tests for CVE-%s, see:\n\n"
+                              CVE_DB_URL "%s\n", tags[i].value,

tags[i].value);

Here as well.

Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20191015/df8ab998/attachment.htm>
#3 lore raw

PATCH 1/2 lib: Add support for test tags

Hello,

Quoted 16 lines · Li Wang
Quoted 16 lines

}

lib/tst_test.c +14 -0
+static void print_colored(const char *str)
+{
+       if (tst_color_enabled(STDOUT_FILENO))
+               printf("%s%s%s", ANSI_COLOR_YELLOW, str, ANSI_COLOR_RESET);
+       else
+               printf("%s", str);
+}
+
+static void print_failure_hints(void)
+{
+       unsigned int i;
+       const struct tst_tag *tags = tst_test->tags;
+
+       if (!tags)
Quoted 9 lines

lib/tst_test.c int tst_parse_float(const char *str, float *val, float min, float max) · line 634

Quoted 6 lines
lib/tst_test.c +6 -0
+               return;
+
+       for (i = 0; tags[i].name; i++) {
+               if (!strcmp(tags[i].name, "linux-git")) {
+                       printf("\n");
+                       print_colored("HINT: ");

+ printf("This is a regression test for linux kernel,

Quoted 1 line

see commit:\n\n"

It may be better just to print "Associated Linux kernel commit: ..." and "Associated bug ID: CVE-..." in the loop. Then we can avoid any discussions about what type of test it is.

Also, a test description tag could be added if necessary.

Maybe it would also be a good idea to link back to the test source code on github? Possibly this info could be injected at build time?

Quoted 23 lines
Quoted 2 lines

+ LINUX_GIT_URL "%s\n", tags[i].value);

This sentence 'HINT: This is a ...' will be printed many times if there are many commits in tags, I prefer to see only once in front of these linux-kernel links.

Quoted 9 lines
lib/tst_test.c +7 -0
+               }
+
+               if (!strcmp(tags[i].name, "CVE")) {
+                       printf("\n");
+                       print_colored("HINT: ");
+                       printf("This test also tests for CVE-%s, see:\n\n"
+                              CVE_DB_URL "%s\n", tags[i].value,

tags[i].value);

Here as well.

Signature · 2 lines
Regards,
Li Wang
Signature · 2 lines
Thank you,
Richard.
#4 lore raw

PATCH 1/2 lib: Add support for test tags

Hi!

Quoted 3 lines

It may be better just to print "Associated Linux kernel commit: ..." and "Associated bug ID: CVE-..." in the loop. Then we can avoid any discussions about what type of test it is.

I will try that.

Quoted 1 line

Also, a test description tag could be added if necessary.

My plan is actually to keep the description in the top level comment but parse that by the docparse tool, then we will have a browseable test documentation that could be put to the web and we can point to that here.

Quoted 2 lines

Maybe it would also be a good idea to link back to the test source code on github? Possibly this info could be injected at build time?

I tried to inject things at a build time and it wasn't pretty, so I would like to avoid that.

Signature · 2 lines
Cyril Hrubis
[email protected]
#5 lore raw

PATCH 1/2 lib: Add support for test tags

In reply to Li Wang

Hi!

Quoted 5 lines
Quoted 2 lines

+static void print_test_info(void)

print_test_info sounds like general information for the test, maybe print_tags() is a better/precise name?

I named it as such in a case that we will add anything else there, but I do not have a strong opinion, I can change that if you insist.

Quoted 14 lines

lib/tst_test.c int tst_parse_float(const char *str, float *val, float min, float max) · line 634

Quoted 12 lines
lib/tst_test.c +12 -0
+static void print_failure_hints(void)
+{
+       unsigned int i;
+       const struct tst_tag *tags = tst_test->tags;
+
+       if (!tags)
+               return;
+
+       for (i = 0; tags[i].name; i++) {
+               if (!strcmp(tags[i].name, "linux-git")) {
+                       printf("\n");
+                       print_colored("HINT: ");

+ printf("This is a regression test for linux kernel,

Quoted 7 lines
Quoted 3 lines

see commit:\n\n"
+ LINUX_GIT_URL "%s\n", tags[i].value);

This sentence 'HINT: This is a ...' will be printed many times if there are many commits in tags, I prefer to see only once in front of these linux-kernel links.

Good catch, I will do something about this.

Signature · 2 lines
Cyril Hrubis
[email protected]
#6 lore raw

PATCH 1/2 lib: Add support for test tags

In reply to Cyril Hrubis

Hi,

lib/tst_test.c static void print_help(void) · line 445

Quoted 5 lines
lib/tst_test.c +5 -0
+	if (tags) {
+		for (i = 0; tags[i].name; i++) {
+			if (!strcmp(tags[i].name, "CVE"))
+				printf(CVE_DB_URL "%s\n", tags[i].value);
+			else if (!strcmp(tags[i].name, "linux-git"))

Not sure if it's worth of defining some enums instead of "CVE", "linux-git" (would catch typos).

lib/tst_test.c static void print_help(void) · line 449

Quoted 5 lines
lib/tst_test.c +4 -0
+				printf(LINUX_GIT_URL "%s\n", tags[i].value);
 
+			else
+				printf("%s: %s\n", tags[i].name, tags[i].value);
+			printf("\n");

Kind regards,
Petr

#7 lore raw

PATCH 1/2 lib: Add support for test tags

Hi!

Quoted 7 lines

lib/tst_test.c static void print_help(void) · line 445

Quoted 5 lines
lib/tst_test.c +5 -0
+	if (tags) {
+		for (i = 0; tags[i].name; i++) {
+			if (!strcmp(tags[i].name, "CVE"))
+				printf(CVE_DB_URL "%s\n", tags[i].value);
+			else if (!strcmp(tags[i].name, "linux-git"))

Not sure if it's worth of defining some enums instead of "CVE", "linux-git" (would catch typos).

I wanted to avoid enums because they are not flexible enough. The plan here is that docparse would do sanity checks on test metadata and fail the compilation if it founds typos there. With the checks in the docparse tool we can easily check for tags that looks like typos, i.e. check the levenshtein distance from all known tags and print a nice looking error message...

Signature · 2 lines
Cyril Hrubis
[email protected]
#8 lore raw

PATCH 1/2 lib: Add support for test tags

Hi Cyril,

Quoted 8 lines
Quoted 7 lines

lib/tst_test.c static void print_help(void) · line 445

Quoted 5 lines
lib/tst_test.c +5 -0
+	if (tags) {
+		for (i = 0; tags[i].name; i++) {
+			if (!strcmp(tags[i].name, "CVE"))
+				printf(CVE_DB_URL "%s\n", tags[i].value);
+			else if (!strcmp(tags[i].name, "linux-git"))

Not sure if it's worth of defining some enums instead of "CVE", "linux-git" (would catch typos).

Quoted 6 lines

I wanted to avoid enums because they are not flexible enough. The plan here is that docparse would do sanity checks on test metadata and fail the compilation if it founds typos there. With the checks in the docparse tool we can easily check for tags that looks like typos, i.e. check the levenshtein distance from all known tags and print a nice looking error message...

OK, sounds good :).

Kind regards,
Petr

#9 lore raw

PATCH 2/2 testcases/kernel: Add linux git and CVE tags

In reply to Cyril Hrubis

Not all tests are annotated at the moment but the majority has been finished for now.

---
 testcases/cve/cve-2014-0196.c                         | 5 +++++
 testcases/cve/cve-2015-3290.c                         | 5 +++++
 testcases/cve/cve-2016-10044.c                        | 5 +++++
 testcases/cve/cve-2016-7042.c                         | 5 +++++
 testcases/cve/cve-2016-7117.c                         | 5 +++++
 testcases/cve/cve-2017-16939.c                        | 5 +++++
 testcases/cve/cve-2017-17052.c                        | 5 +++++
 testcases/cve/cve-2017-17053.c                        | 5 +++++
 testcases/cve/cve-2017-2618.c                         | 5 +++++
 testcases/cve/cve-2017-2671.c                         | 5 +++++
 testcases/cve/meltdown.c                              | 6 +++++-
 testcases/cve/stack_clash.c                           | 4 ++++
 testcases/kernel/crypto/af_alg01.c                    | 5 +++++
 testcases/kernel/crypto/af_alg02.c                    | 5 +++++
 testcases/kernel/crypto/af_alg03.c                    | 4 ++++
 testcases/kernel/crypto/af_alg04.c                    | 4 ++++
 testcases/kernel/crypto/af_alg05.c                    | 8 +++++++-
 testcases/kernel/crypto/af_alg06.c                    | 4 ++++
 testcases/kernel/crypto/crypto_user01.c               | 6 ++++++
 testcases/kernel/crypto/crypto_user02.c               | 4 ++++
 testcases/kernel/crypto/pcrypt_aead01.c               | 5 +++++
 testcases/kernel/sound/snd_timer01.c                  | 6 ++++++
 testcases/kernel/syscalls/accept/accept02.c           | 5 +++++
 testcases/kernel/syscalls/acct/acct02.c               | 4 ++++
 testcases/kernel/syscalls/add_key/add_key02.c         | 5 +++++
 testcases/kernel/syscalls/bpf/bpf_prog02.c            | 4 ++++
 testcases/kernel/syscalls/futex/futex_cmp_requeue02.c | 5 +++++
 testcases/kernel/syscalls/keyctl/keyctl02.c           | 4 ++++
 testcases/kernel/syscalls/keyctl/keyctl04.c           | 5 +++++
 testcases/kernel/syscalls/keyctl/keyctl05.c           | 4 ++++
 testcases/kernel/syscalls/move_pages/move_pages12.c   | 6 ++++++
 testcases/kernel/syscalls/realpath/realpath01.c       | 4 ++++
 testcases/kernel/syscalls/request_key/request_key05.c | 4 ++++
 testcases/kernel/syscalls/sbrk/sbrk03.c               | 4 ++++
 testcases/kernel/syscalls/sendto/sendto02.c           | 4 ++++
 testcases/kernel/syscalls/setrlimit/setrlimit06.c     | 4 ++++
 36 files changed, 171 insertions(+), 2 deletions(-)
testcases/cve/cve-2014-0196.c +5 -0
index 6997fffe4..012cbb7cd 100644
@@ -141,4 +141,9 @@ static struct tst_test test = {
 	.setup = setup,
 	.cleanup = cleanup,
 	.test_all = run,
+	.tags = (const struct tst_tag[]) {
+		{"linux-git", "4291086b1f08"},
+		{"CVE", "2014-0196"},
+		{}
+	}
 };
testcases/cve/cve-2015-3290.c +5 -0
index fe630e4af..6c4fd57ac 100644
@@ -469,6 +469,11 @@ static struct tst_test test = {
 	.setup = setup,
 	.timeout = TIMEOUT,
 	.test_all = run,
+	.tags = (const struct tst_tag[]) {
+		{"linux-git", "9b6e6a8334d5"},
+		{"CVE", "2015-3290"},
+		{}
+	}
 };
 
 #else /* HAVE_PERF_EVENT_ATTR && (defined(__x86_64__) || defined(__i386__)) */
testcases/cve/cve-2016-10044.c +5 -0
index c2e79df31..7519676a1 100644
@@ -62,4 +62,9 @@ static struct tst_test test = {
 	.test_all = run,
 	.cleanup = cleanup,
 	.min_kver = "2.6.8",
+	.tags = (const struct tst_tag[]) {
+		{"linux-git", "22f6b4d34fcf"},
+		{"CVE", "2016-10044"},
+		{}
+	}
 };
testcases/cve/cve-2016-7042.c +5 -0
index 35950adc7..4434265dd 100644
@@ -71,4 +71,9 @@ static struct tst_test test = {
 	.setup = setup,
 	.cleanup = cleanup,
 	.test_all = do_test,
+	.tags = (const struct tst_tag[]) {
+		{"linux-git", "03dab869b7b2"},
+		{"CVE", "2016-7042"},
+		{}
+	}
 };
testcases/cve/cve-2016-7117.c +5 -0
index 5d9f842aa..dca002924 100644
@@ -150,4 +150,9 @@ static struct tst_test test = {
 	.setup = setup,
 	.cleanup = cleanup,
 	.min_kver = "2.6.33",
+	.tags = (const struct tst_tag[]) {
+		{"linux-git", "a2e2725541fa"},
+		{"CVE", "2016-7117"},
+		{}
+	}
 };
testcases/cve/cve-2017-16939.c +5 -0
index 4ecc86337..9ab1eb2c3 100644
@@ -77,4 +77,9 @@ static void run(void)
 static struct tst_test test = {
 	.setup = setup,
 	.test_all = run,
+	.tags = (const struct tst_tag[]) {
+		{"linux-git", "1137b5e2529a"},
+		{"CVE", "2017-16939"},
+		{}
+	}
 };
testcases/cve/cve-2017-17052.c +5 -0
index 81123fce8..d89c62eb6 100644
@@ -111,4 +111,9 @@ static struct tst_test test = {
 	.cleanup = cleanup,
 	.setup = setup,
 	.test_all = run,
+	.tags = (const struct tst_tag[]) {
+		{"linux-git", "2b7e8665b4ff"},
+		{"CVE", "2017-17052"},
+		{}
+	}
 };
testcases/cve/cve-2017-17053.c +5 -0
index 646ec11cc..08a08211d 100644
@@ -154,6 +154,11 @@ static struct tst_test test = {
 	.setup = setup,
 	.cleanup = cleanup,
 	.test_all = run,
+	.tags = (const struct tst_tag[]) {
+		{"linux-git", "ccd5b3235180"},
+		{"CVE", "2017-17053"},
+		{}
+	}
 };
 
 #else
testcases/cve/cve-2017-2618.c +5 -0
index cf018d401..4ab3cfb8f 100644
@@ -51,4 +51,9 @@ static struct tst_test test = {
 	.forks_child = 1,
 	.setup = setup,
 	.test_all = do_test,
+	.tags = (const struct tst_tag[]) {
+		{"linux-git", "0c461cb727d1"},
+		{"CVE", "2017-2618"},
+		{}
+	}
 };
testcases/cve/cve-2017-2671.c +5 -0
index b92c1d94b..e72795d15 100644
@@ -109,4 +109,9 @@ static struct tst_test test = {
 	.test_all = run,
 	.cleanup = cleanup,
 	.needs_root = 1,
+	.tags = (const struct tst_tag[]) {
+		{"linux-git", "43a6684519ab"},
+		{"CVE", "2017-2671"},
+		{}
+	}
 };
testcases/cve/meltdown.c +5 -1
index 863f1a9e4..db00e1811 100644
@@ -377,7 +377,11 @@ static struct tst_test test = {
 	.setup = setup,
 	.test_all = run,
 	.cleanup = cleanup,
-	.min_kver = "2.6.32"
+	.min_kver = "2.6.32",
+	.tags = (const struct tst_tag[]) {
+		{"CVE", "2017-5754"},
+		{}
+	}
 };
 
 #else /* #if defined(__x86_64__) || defined(__i386__) */
testcases/cve/stack_clash.c +4 -0
index 3ea3ec491..ecbf5f6da 100644
@@ -250,4 +250,8 @@ static struct tst_test test = {
 	.needs_root = 1,
 	.setup = setup,
 	.test_all = stack_clash_test,
+	.tags = (const struct tst_tag[]) {
+		{"CVE", "2017-1000364"},
+		{}
+	}
 };
testcases/kernel/crypto/af_alg01.c +5 -0
index bb6e01738..d3e823e6c 100644
@@ -76,4 +76,9 @@ static void do_test(unsigned int i)
 static struct tst_test test = {
 	.test = do_test,
 	.tcnt = ARRAY_SIZE(hash_algs),
+	.tags = (const struct tst_tag[]) {
+		{"linux-git", "af3ff8045bbf"},
+		{"CVE", "2017-17806"},
+		{}
+	}
 };
testcases/kernel/crypto/af_alg02.c +5 -0
index 08087ecac..fab0010c9 100644
@@ -59,4 +59,9 @@ static struct tst_test test = {
 	.test_all = run,
 	.timeout = 20,
 	.needs_checkpoints = 1,
+	.tags = (const struct tst_tag[]) {
+		{"linux-git", "ecaaab564978"},
+		{"CVE", "2017-17805"},
+		{}
+	}
 };
testcases/kernel/crypto/af_alg03.c +4 -0
index 240c52835..5f214e48b 100644
@@ -28,4 +28,8 @@ static void run(void)
 
 static struct tst_test test = {
 	.test_all = run,
+	.tags = (const struct tst_tag[]) {
+		{"linux-git", "e57121d08c38"},
+		{}
+	}
 };
testcases/kernel/crypto/af_alg04.c +4 -0
index 48239e020..df049048a 100644
@@ -55,4 +55,8 @@ static void run(void)
 static struct tst_test test = {
 	.test_all = run,
 	.forks_child = 1,
+	.tags = (const struct tst_tag[]) {
+		{"linux-git", "bb2964810233"},
+		{}
+	}
 };
testcases/kernel/crypto/af_alg05.c +7 -1
index 964adffcc..df2b03546 100644
@@ -11,7 +11,7 @@
  * boundary.  Based on the reproducer from the commit message.  Note that this
  * issue only reproduces on certain architectures, such as arm and arm64.
  *
- * On some older kernel without commit 160544075f2 ("crypto: scatterwalk - Hide
+ * On some older kernel without commit 160544075f2a ("crypto: scatterwalk - Hide
  * PageSlab call to optimise away flush_dcache_page") , it doesn't use
  * ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE macro. It can crash on all architectures.
  * Without skcipher walk interface, it is also a regresstion test for commit
@@ -46,4 +46,10 @@ static void run(void)
 
 static struct tst_test test = {
 	.test_all = run,
+	.tags = (const struct tst_tag[]) {
+		{"linux-git", "8088d3dd4d7c"},
+		{"linux-git", "160544075f2a"},
+		{"linux-git", "0868def3e410"},
+		{}
+	}
 };
testcases/kernel/crypto/af_alg06.c +4 -0
index 586d91b1d..6acdf8532 100644
@@ -48,4 +48,8 @@ static void run(void)
 
 static struct tst_test test = {
 	.test_all = run,
+	.tags = (const struct tst_tag[]) {
+		{"linux-git", "8f9c46934848"},
+		{}
+	}
 };
testcases/kernel/crypto/crypto_user01.c +6 -0
index 163d7a2dd..47bf9f0d2 100644
@@ -192,4 +192,10 @@ static struct tst_test test = {
 	.setup = setup,
 	.test_all = run,
 	.cleanup = cleanup,
+	.tags = (const struct tst_tag[]) {
+		{"linux-git", "f43f39958beb"},
+		{"CVE", "2013-2547"},
+		{"CVE", "2018-19854"},
+		{}
+	}
 };
testcases/kernel/crypto/crypto_user02.c +4 -0
index 6090b5a4e..384e344f2 100644
@@ -105,4 +105,8 @@ static struct tst_test test = {
 	.cleanup = cleanup,
 	.needs_root = 1,
 	.forks_child = 1,
+	.tags = (const struct tst_tag[]) {
+		{"linux-git", "21d4120ec6f5"},
+		{}
+	}
 };
testcases/kernel/crypto/pcrypt_aead01.c +5 -0
index d9c27f69d..42ae259fb 100644
@@ -69,4 +69,9 @@ static struct tst_test test = {
 	.test_all = run,
 	.cleanup = cleanup,
 	.needs_root = 1,
+	.tags = (const struct tst_tag[]) {
+		{"linux-git", "d76c68109f37"},
+		{"CVE", "2017-5754"},
+		{}
+	}
 };
testcases/kernel/sound/snd_timer01.c +6 -0
index f62696659..ebac6e1c6 100644
@@ -140,4 +140,10 @@ static struct tst_test test = {
 	.test_all = run,
 	.setup = setup,
 	.cleanup = cleanup,
+	.tags = (const struct tst_tag[]) {
+		{"linux-git", "d11662f4f798"},
+		{"linux-git", "ba3021b2c79b"},
+		{"CVE", "2017-1000380"},
+		{NULL, NULL}
+	}
 };
testcases/kernel/syscalls/accept/accept02.c +5 -0
index 1a0f625c9..576b91ce8 100644
@@ -142,4 +142,9 @@ static struct tst_test test = {
 	.setup = setup,
 	.cleanup = cleanup,
 	.needs_checkpoints = 1,
+	.tags = (const struct tst_tag[]) {
+		{"CVE", "2017-8890"},
+		{"linux-git", "657831ff"},
+		{NULL, NULL},
+	}
 };
testcases/kernel/syscalls/acct/acct02.c +4 -0
index 4d95aafec..890568b08 100644
@@ -263,4 +263,8 @@ static struct tst_test test = {
 	.cleanup = cleanup,
 	.needs_tmpdir = 1,
 	.needs_root = 1,
+	.tags = (const struct tst_tag[]) {
+		{"linux-git", "4d9570158b626"},
+		{}
+	}
 };
testcases/kernel/syscalls/add_key/add_key02.c +5 -0
index 4a5cf054a..159234f67 100644
@@ -83,4 +83,9 @@ static void verify_add_key(unsigned int i)
 static struct tst_test test = {
 	.tcnt = ARRAY_SIZE(tcases),
 	.test = verify_add_key,
+	.tags = (const struct tst_tag[]) {
+		{"linux-git", "5649645d725c"},
+		{"CVE", "2017-15274"},
+		{}
+	}
 };
testcases/kernel/syscalls/bpf/bpf_prog02.c +4 -0
index fb8f3f781..47f313a6d 100644
@@ -190,5 +190,9 @@ static struct tst_test test = {
 		{&attr, .size = sizeof(*attr)},
 		{&msg, .size = sizeof(MSG)},
 		{},
+	},
+	.tags = (const struct tst_tag[]) {
+		{"linux-git", "3612af783cf5"},
+		{}
 	}
 };
testcases/kernel/syscalls/futex/futex_cmp_requeue02.c +5 -0
index 3ea19428c..228d8cff6 100644
@@ -73,4 +73,9 @@ static struct tst_test test = {
 	.cleanup = cleanup,
 	.test = verify_futex_cmp_requeue,
 	.tcnt = ARRAY_SIZE(tcases),
+	.tags = (const struct tst_tag[]) {
+		{"CVE", "2018-6927"},
+		{"linux-git", "fbe0e839d1e2"},
+		{}
+	}
 };
testcases/kernel/syscalls/keyctl/keyctl02.c +4 -0
index b034275d2..f4ee03f33 100644
@@ -136,4 +136,8 @@ static struct tst_test test = {
 	.setup = setup,
 	.cleanup = cleanup,
 	.test_all = do_test,
+	.tags = (const struct tst_tag[]) {
+		{"linux-git", "b4a1b4f5047e"},
+		{}
+	}
 };
testcases/kernel/syscalls/keyctl/keyctl04.c +5 -0
index d30d25b04..1fed23ca6 100644
@@ -43,4 +43,9 @@ static void do_test(void)
 
 static struct tst_test test = {
 	.test_all = do_test,
+	.tags = (const struct tst_tag[]) {
+		{"CVE", "2017-7472"},
+		{"linux-git", "c9f838d104fe"},
+		{}
+	}
 };
testcases/kernel/syscalls/keyctl/keyctl05.c +4 -0
index 653b838f5..c592eb49e 100644
@@ -207,4 +207,8 @@ static struct tst_test test = {
 	.tcnt = 3,
 	.test = do_test,
 	.forks_child = 1,
+	.tags = (const struct tst_tag[]) {
+		{"linux-git", "63a0b0509e70"},
+		{}
+	}
 };
testcases/kernel/syscalls/move_pages/move_pages12.c +6 -0
index c2927fb86..c906acbe3 100644
@@ -330,6 +330,12 @@ static struct tst_test test = {
 	.cleanup = cleanup,
 	.test = do_test,
 	.tcnt = ARRAY_SIZE(tcases),
+	.tags = (const struct tst_tag[]) {
+		{"linux-git", "e66f17ff7177"},
+		{"linux-git", "c9d398fa2378"},
+		{"linux-git", "4643d67e8cb0"},
+		{}
+	}
 };
 
 #else
testcases/kernel/syscalls/realpath/realpath01.c +4 -0
index fc2af2a1c..c0381e9cb 100644
@@ -41,4 +41,8 @@ static struct tst_test test = {
 	.setup = setup,
 	.needs_root = 1,
 	.needs_tmpdir = 1,
+	.tags = (const struct tst_tag[]) {
+		{"CVE", "2018-1000001"},
+		{}
+	}
 };
testcases/kernel/syscalls/request_key/request_key05.c +4 -0
index 129cc0507..a17d1d058 100644
@@ -31,4 +31,8 @@ static void run(void)
 
 static struct tst_test test = {
 	.test_all = run,
+	.tags = (const struct tst_tag[]) {
+		{"CVE", "2017-6951"},
+		{}
+	}
 };
testcases/kernel/syscalls/sbrk/sbrk03.c +4 -0
index 761a01c9d..80d2020ff 100644
@@ -66,4 +66,8 @@ static void sbrk_test(void)
 
 static struct tst_test test = {
 	.test_all = sbrk_test,
+	.tags = (const struct tst_tag[]) {
+		{"linux-git", "473a06572fcd"},
+		{}
+	}
 };
testcases/kernel/syscalls/sendto/sendto02.c +4 -0
index 5c9f00125..743101733 100644
@@ -77,4 +77,8 @@ static struct tst_test test = {
 	.setup = setup,
 	.cleanup = cleanup,
 	.test_all = verify_sendto,
+	.tags = (const struct tst_tag[]) {
+		{"linux-git", "6e51fe757259"},
+		{}
+	}
 };
testcases/kernel/syscalls/setrlimit/setrlimit06.c +4 -0
index 87e6b0e32..726b26841 100644
@@ -114,4 +114,8 @@ static struct tst_test test = {
 	.setup = setup,
 	.cleanup = cleanup,
 	.forks_child = 1,
+	.tags = (const struct tst_tag[]) {
+		{"linux-git", "c3bca5d450b62"},
+		{}
+	}
 };
Signature · 1 line
2.21.0
#10 lore raw

PATCH 2/2 testcases/kernel: Add linux git and CVE tags

Quoted 6 lines · Cyril Hrubis
testcases/cve/cve-2014-0196.c +0 -0
index 6997fffe4..012cbb7cd 100644

testcases/cve/cve-2014-0196.c static struct tst_test test = { · line 148

Quoted 8 lines
testcases/cve/cve-2014-0196.c +4 -0
@@ -141,4 +141,9 @@ static struct tst_test test = {
        .setup = setup,
        .cleanup = cleanup,
        .test_all = run,
+       .tags = (const struct tst_tag[]) {
+               {"linux-git", "4291086b1f08"},
+               {"CVE", "2014-0196"},
+               {}
 

Some of the tests use {NULL, NULL} to terminate tag array but some not, maybe we need to keep consistent in usage.

Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20191015/bc68b025/attachment.htm>
#11 lore raw

PATCH 2/2 testcases/kernel: Add linux git and CVE tags

Hi!

Quoted 8 lines
Quoted 8 lines
testcases/cve/cve-2014-0196.c +0 -0
index 6997fffe4..012cbb7cd 100644
@@ -141,4 +141,9 @@ static struct tst_test test = {
        .setup = setup,
        .cleanup = cleanup,
        .test_all = run,
Quoted 8 lines

testcases/cve/cve-2014-0196.c static struct tst_test test = {

Quoted 5 lines
testcases/cve/cve-2014-0196.c +4 -0
+       .tags = (const struct tst_tag[]) {
+               {"linux-git", "4291086b1f08"},
+               {"CVE", "2014-0196"},
+               {}
 

Some of the tests use {NULL, NULL} to terminate tag array but some not, maybe we need to keep consistent in usage.

I guess I missed some when I was converting {NULL, NULL} to just {}. I will fix that in v2.

Signature · 2 lines
Cyril Hrubis
[email protected]
#12 lore raw

PATCH 2/2 testcases/kernel: Add linux git and CVE tags

Hi,

Quoted 10 lines
Quoted 7 lines

testcases/cve/cve-2014-0196.c static struct tst_test test = { · line 147

Quoted 4 lines
testcases/cve/cve-2014-0196.c +4 -0
+       .tags = (const struct tst_tag[]) {
+               {"linux-git", "4291086b1f08"},
+               {"CVE", "2014-0196"},
+               {}

Some of the tests use {NULL, NULL} to terminate tag array but some not, maybe we need to keep consistent in usage.

I guess I missed some when I was converting {NULL, NULL} to just {}. I will fix that in v2.

+1

Nice work, thanks :).

nit: I don't that much blank lines, I'd prefer more compact output:

OPTIONS

-h       Prints this help
-i n     Execute test n times
-I x     Execute test for n seconds
-C ARG   Run child process with ARG arguments (used internally)

TAGS
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0c461cb727d1
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-2618

Kind regards,
Petr

#13 lore raw

PATCH 2/2 testcases/kernel: Add linux git and CVE tags

Petr Vorel <[email protected]> wrote:

...

Quoted 8 lines

nit: I don't that much blank lines, I'd prefer more compact output:
...
TAGS

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0c461cb727d1
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-2618

The explanation(HINT: ...) is needed, otherwise, it will misleading someone not familiar with LTP I think.

Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20191017/3f153d07/attachment.htm>