Support metadata groups
cover Support metadata groups
---
Andrea Cervesato (2):
metadata: add tests grouping support
doc: conf.py: Show groups in test catalog
doc/conf.py | 14 ++++++- metadata/metaparse.c | 88 ++++++++++++++++++++++++++++++++++++++++++++ metadata/tests/groups.c | 11 ++++++ metadata/tests/groups.c.json | 13 +++++++ 4 files changed, 125 insertions(+), 1 deletion(-)
---
base-commit: a3fda1dc1ce7da882a592a4877da7f214e468650
change-id: 20260612-metadata_groups-dd2430e21901
Best regards,
Signature · 5 lines
Andrea Cervesato <[email protected]> -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH 1/2 metadata: add tests grouping support
From: Andrea Cervesato <[email protected]>
Add groups field to metaparse JSON output, so we can filter out tests in kirk. Groups are derived from:
1. Source file path - the two nearest parent directories (immediate parent first), skipping 'kernel' as too generic. For example:
- testcases/kernel/syscalls/clone/clone01.c -> clone, syscalls
- testcases/kernel/kvm/kvm_pagefault01.c -> kvm
- testcases/cve/cve-2017-16939.c -> cve
2. @group tags in the doc comment block, e.g.:
/* * Test description.
*
* @group stress
*/Add test case for @group tag parsing.
--- metadata/metaparse.c | 88 ++++++++++++++++++++++++++++++++++++++++++++ metadata/tests/groups.c | 11 ++++++ metadata/tests/groups.c.json | 13 +++++++ 3 files changed, 112 insertions(+)
metadata/metaparse.c +88 -0
@@ -1168,6 +1168,92 @@ static void print_help(const char *prgname)exit (0 ); } +/* +* Add groups derived from the source file path. +* +* Groups are the two nearest parent directories (immediate parent +* first), skipping 'kernel' as it's too generic: +* +* testcases/kernel/syscalls/clone/clone01.c -> clone, syscalls +* testcases/kernel/kvm/kvm_pagefault01.c -> kvm +* testcases/cve/cve-2017-16939.c -> cve +*/ +static void add_path_groups (struct data_node *groups,const char *fname) +{ +char buf[256 ]; +int offsets[8 ]; +int ndirs =0 ; +int ngroups =0 ; +char *p; + +if (strncmp (fname,"testcases/" ,10 )) +return ; + +snprintf (buf,sizeof (buf),"%s" , fname +10 ); + + p =strtok (buf,"/" ); +while (p && ndirs <8 ) { + offsets[ndirs++] = p - buf; + p =strtok (NULL ,"/" ); + } + +/* Last element is the filename, skip it */ + ndirs--; + +for (int j = ndirs -1 ; j >=0 && ngroups <2 ; j--) { +if (!strcmp (buf + offsets[j],"kernel" )) +continue ; + +data_node_array_add (groups,data_node_string (buf + offsets[j])); + ngroups++; + } +} + +/* +* Add groups from @group tags in the doc comment block. +*/ +static void add_doc_groups (struct data_node *groups,struct data_node *doc) +{ +if (!doc || doc->type !=DATA_ARRAY ) +return ; + +for (unsigned int i =0 ; i <data_node_array_len (doc); i++) { +struct data_node *line = doc->array.array[i]; +const char *s; + +if (line->type !=DATA_STRING ) +continue ; + + s = line->string.val; + +while (*s && (*s ==' ' || *s =='\t' )) + s++; + +if (strncmp (s,"@group " ,7 )) +continue ; + + s +=7 ; +while (*s && (*s ==' ' || *s =='\t' )) + s++; + +if (*s) +data_node_array_add (groups,data_node_string (s)); + } +} + +static void build_groups (struct data_node *res,const char *fname) +{ +struct data_node *groups =data_node_array (); + +add_path_groups (groups, fname); +add_doc_groups (groups,data_node_hash_get (res,"doc" )); + +if (data_node_array_len (groups)) +data_node_hash_add (res,"groups" , groups); +else +data_node_free (groups); +} +int main (int argc,char *argv[]) {unsigned int i, j;
@@ -1238,6 +1324,8 @@ int main(int argc, char *argv[]) }data_node_hash_add (res,"fname" ,data_node_string (argv[optind])); +build_groups (res, argv[optind]); +printf (" \"%s\": " ,strip_name (argv[optind]));data_to_json (res, stdout,2 );data_node_free (res);
metadata/tests/groups.c new file +11 -0
@@ -0,0 +1,11 @@ +// SPDX-License-Identifier: GPL-2.0-or-later + +/*\ +* Test for @group tag parsing. +* +* @group stress +* @group regression +*/ + +static struct tst_test test = { +};
metadata/tests/groups.c.json new file +13 -0
@@ -0,0 +1,13 @@ + "groups": { + "doc": [ + "Test for @group tag parsing.", + "", + "@group stress", + "@group regression" + ], + "fname": "groups.c", + "groups": [ + "stress", + "regression" + ] + }
Signature · 5 lines
2.51.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] metadata: add tests grouping support
Hi Andrea,
Quoted 1 line · Andrea Cervesato
metadata: add tests grouping support
--- [PATCH 1/2] ---
Quoted 5 lines
2. @group tags in the doc comment block, e.g.:
/* * Test description. * * @group stress */
The example appears malformed. metaparse only extracts @group tags from doc comment blocks opened with /*\, not plain /* comments. The test file (groups.c) correctly uses /*\ as the opening.
Should this be:
/*\
* Test description.
*
* @group stress
*/
Verdict - Needs revision
---
Note:
The agent can sometimes produce false positives although often its findings are genuine. If you find issues with the review, please comment this email or ignore the suggestions.
Regards,
LTP AI Reviewer
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 1/2] metadata: add tests grouping support
In reply to Andrea CervesatoHi!
Quoted 10 lines
1. Source file path - the two nearest parent directories (immediate parent first), skipping 'kernel' as too generic. For example:
-testcases/kernel/syscalls/clone/clone01.c-> clone, syscalls
-testcases/kernel/kvm/kvm_pagefault01.c-> kvm
-testcases/cve/cve-2017-16939.c-> cve2. @group tags in the doc comment block, e.g.:
/* * Test description. * * @group stress
Maybe call it groups and allow white-space separated list?
I'm not really sure about the syntax, but I guess that @groups foo bar will do.
Quoted 31 lines
*/Add test case for @group tag parsing.
Signed-off-by Andrea Cervesato <[email protected]>--- metadata/metaparse.c | 88 ++++++++++++++++++++++++++++++++++++++++++++ metadata/tests/groups.c | 11 ++++++ metadata/tests/groups.c.json | 13 +++++++ 3 files changed, 112 insertions(+)metadata/metaparse.c +12
-0@@ -1168,6 +1168,92 @@ static void print_help(const char *prgname)exit (0 ); } +/* +* Add groups derived from the source file path. +* +* Groups are the two nearest parent directories (immediate parent +* first), skipping 'kernel' as it's too generic: +* +* testcases/kernel/syscalls/clone/clone01.c -> clone, syscalls +* testcases/kernel/kvm/kvm_pagefault01.c -> kvm +* testcases/cve/cve-2017-16939.c -> cve +*/ +static void add_path_groups (struct data_node *groups,const char *fname) +{
metadata/metaparse.c static void print_help(const char *prgname) · line 1192
Quoted 10 lines
metadata/metaparse.c +10
-0+char buf[256 ]; +int offsets[8 ]; +int ndirs =0 ; +int ngroups =0 ; +char *p; + +if (strncmp (fname,"testcases/" ,10 )) +return ; + +snprintf (buf,sizeof (buf),"%s" , fname +10 );
Maybe avoid static buffers here with:
buf = strdup(fname + 10);
metadata/metaparse.c static void print_help(const char *prgname) · line 1195
Quoted 3 lines
metadata/metaparse.c +3
-0+ p =strtok (buf,"/" ); +while (p && ndirs <8 ) { + offsets[ndirs++] = p - buf;
Why do we store offset rather than the pointer to the string?
dirs[ndirs++] = p;
metadata/metaparse.c static void print_help(const char *prgname) · line 1208
Quoted 13 lines
metadata/metaparse.c +13
-0+ p =strtok (NULL ,"/" ); + } + +/* Last element is the filename, skip it */ + ndirs--; + +for (int j = ndirs -1 ; j >=0 && ngroups <2 ; j--) { +if (!strcmp (buf + offsets[j],"kernel" )) +continue ; + +data_node_array_add (groups,data_node_string (buf + offsets[j])); + ngroups++; + }
We can avoid the complex loop by adding a function to add the group with the "kernel" filter and just calling it twice here:
add_group(groups, dirs[ndirs-1]); add_group(groups, dirs[ndirs-2]);
Quoted 22 lines
metadata/metaparse.c +22
-0+} + +/* +* Add groups from @group tags in the doc comment block. +*/ +static void add_doc_groups (struct data_node *groups,struct data_node *doc) +{ +if (!doc || doc->type !=DATA_ARRAY ) +return ; + +for (unsigned int i =0 ; i <data_node_array_len (doc); i++) { +struct data_node *line = doc->array.array[i]; +const char *s; + +if (line->type !=DATA_STRING ) +continue ; + + s = line->string.val; + +while (*s && (*s ==' ' || *s =='\t' )) + s++; +
metadata/metaparse.c static void print_help(const char *prgname) · line 1240
Quoted 10 lines
metadata/metaparse.c +10
-0+if (strncmp (s,"@group " ,7 )) +continue ; + + s +=7 ; +while (*s && (*s ==' ' || *s =='\t' )) + s++; + +if (*s) +data_node_array_add (groups,data_node_string (s)); + }
If we hook up into the multiline_comment() function we can consume the line as well so that it does not appear in the parsed doc string. We would have to pass the groups node from the main all the way to multiline_comment() something as (uncomplete):
metadata/metaparse.c +17 -9
@@ -50,7 +50,12 @@ static const char *eat_asterisk_space(const char *c)return c; } -static void multiline_comment (FILE *f,struct data_node *doc) +static int parse_groups (struct data_node *groups,char *line) +{ + ... +} + +static void multiline_comment (FILE *f,struct data_node *doc,struct data_node *groups) {int c;int state =0 ;
@@ -65,8 +70,11 @@ static void multiline_comment(FILE *f, struct data_node *doc)struct data_node *line; buf[bufp] =0 ; line =data_node_string (eat_asterisk_space (buf)); -if (data_node_array_add (doc, line)) -WARN ("doc string comment truncated" ); + +if (!parse_groups (groups, line)) { +if (data_node_array_add (doc, line)) +WARN ("doc string comment truncated" ); + } bufp =0 ;continue ; }
@@ -100,7 +108,7 @@ static void multiline_comment(FILE *f, struct data_node *doc)static const char doc_prefix[] ="\\\n" ; -static void maybe_doc_comment (FILE *f,struct data_node *doc) +static void maybe_doc_comment (FILE *f,struct data_node *doc,struct data_groups *groups) {int c, i;
@@ -113,14 +121,14 @@ static void maybe_doc_comment(FILE *f, struct data_node *doc)if (c =='*' )ungetc (c, f); -multiline_comment (f,NULL ); +multiline_comment (f,NULL ,NULL );return ; } -multiline_comment (f, doc); +multiline_comment (f, doc, groups); } -static void maybe_comment (FILE *f,struct data_node *doc) +static void maybe_comment (FILE *f,struct data_node *doc,struct data_node *groups) {int c =getc (f);
@@ -129,7 +137,7 @@ static void maybe_comment(FILE *f, struct data_node *doc)remove_to_newline (f);break ;case '*' : -maybe_doc_comment (f, doc); +maybe_doc_comment (f, doc, groups);break ;default :ungetc (c, f);
@@ -194,7 +202,7 @@ static char *next_token2(FILE *f, char *buf, size_t buf_len, struct data_node *d buf[i++] = c;break ;case '/' : -maybe_comment (f, doc); +maybe_comment (f, doc, groups);break ;case '"' : in_str =1 ;
Quoted 71 lines
metadata/metaparse.c +17
-0+} + +static void build_groups (struct data_node *res,const char *fname) +{ +struct data_node *groups =data_node_array (); + +add_path_groups (groups, fname); +add_doc_groups (groups,data_node_hash_get (res,"doc" )); + +if (data_node_array_len (groups)) +data_node_hash_add (res,"groups" , groups); +else +data_node_free (groups); +} +int main (int argc,char *argv[]) {unsigned int i, j;@@ -1238,6 +1324,8 @@ int main(int argc, char *argv[]) }data_node_hash_add (res,"fname" ,data_node_string (argv[optind])); +build_groups (res, argv[optind]); +printf (" \"%s\": " ,strip_name (argv[optind]));data_to_json (res, stdout,2 );data_node_free (res);metadata/tests/groups.c new file +11
-0@@ -0,0 +1,11 @@ +// SPDX-License-Identifier: GPL-2.0-or-later + +/*\ +* Test for @group tag parsing. +* +* @group stress +* @group regression +*/ + +static struct tst_test test = { +};metadata/tests/groups.c.json new file +13
-0@@ -0,0 +1,13 @@ + "groups": { + "doc": [ + "Test for @group tag parsing.", + "", + "@group stress", + "@group regression" + ], + "fname": "groups.c", + "groups": [ + "stress", + "regression" + ] + }Signature · 6 lines
-- 2.51.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Signature · 5 lines
Cyril Hrubis [email protected] -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 1/2] metadata: add tests grouping support
Hi Cyril,
Quoted 12 lines
Hi!
Quoted 10 lines
1. Source file path - the two nearest parent directories (immediate parent first), skipping 'kernel' as too generic. For example:
-testcases/kernel/syscalls/clone/clone01.c-> clone, syscalls
-testcases/kernel/kvm/kvm_pagefault01.c-> kvm
-testcases/cve/cve-2017-16939.c-> cve2. @group tags in the doc comment block, e.g.:
/* * Test description. * * @group stress
Quoted 4 lines
Maybe call it groups and allow white-space separated list?
I'm not really sure about the syntax, but I guess that @groups foo bar will do.
I was thinking that having @group foo is cleaner. When we start to have multiple groups it can be really to handle multiple groups: for instance, we might end up having one line over 80 chars in the description.
@groups foo1 bar1 foo2 bar2 foo3 bar3 ..
Instead, we might have
@group foo1
@group bar1
@group foo2
@group bar2
@group foo3
@group bar3
These strings are short, but if we have long names it can be weird to read having all of them on a single line.
Quoted 45 lines
Quoted 32 lines
*/Add test case for @group tag parsing.
Signed-off-by Andrea Cervesato <[email protected]>--- metadata/metaparse.c | 88 ++++++++++++++++++++++++++++++++++++++++++++ metadata/tests/groups.c | 11 ++++++ metadata/tests/groups.c.json | 13 +++++++ 3 files changed, 112 insertions(+)metadata/metaparse.c +12
-0@@ -1168,6 +1168,92 @@ static void print_help(const char *prgname)exit (0 ); } +/* +* Add groups derived from the source file path. +* +* Groups are the two nearest parent directories (immediate parent +* first), skipping 'kernel' as it's too generic: +* +* testcases/kernel/syscalls/clone/clone01.c -> clone, syscalls +* testcases/kernel/kvm/kvm_pagefault01.c -> kvm +* testcases/cve/cve-2017-16939.c -> cve +*/ +static void add_path_groups (struct data_node *groups,const char *fname) +{metadata/metaparse.c static void print_help(const char *prgname) · line 1192
Quoted 10 lines
metadata/metaparse.c +10
-0+char buf[256 ]; +int offsets[8 ]; +int ndirs =0 ; +int ngroups =0 ; +char *p; + +if (strncmp (fname,"testcases/" ,10 )) +return ; + +snprintf (buf,sizeof (buf),"%s" , fname +10 );Maybe avoid static buffers here with:
Quoted 2 lines
buf = strdup(fname + 10);
we can eventually just loop on file name and using offsets, but a small static buffer makes things clear. I will try to handle it differently.
Quoted 7 lines
metadata/metaparse.c static void print_help(const char *prgname) · line 1195
Quoted 3 lines
metadata/metaparse.c +3
-0+ p =strtok (buf,"/" ); +while (p && ndirs <8 ) { + offsets[ndirs++] = p - buf;Why do we store offset rather than the pointer to the string?
dirs[ndirs++] = p;
Leftover of experiments I guess..
Quoted 18 lines
Quoted 4 lines
metadata/metaparse.c +3
-0+ p =strtok (NULL ,"/" ); + } +metadata/metaparse.c static void print_help(const char *prgname) · line 1208
Quoted 10 lines
metadata/metaparse.c +10
-0+/* Last element is the filename, skip it */ + ndirs--; + +for (int j = ndirs -1 ; j >=0 && ngroups <2 ; j--) { +if (!strcmp (buf + offsets[j],"kernel" )) +continue ; + +data_node_array_add (groups,data_node_string (buf + offsets[j])); + ngroups++; + }We can avoid the complex loop by adding a function to add the group with the "kernel" filter and just calling it twice here:
Quoted 2 lines
add_group(groups, dirs[ndirs-1]); add_group(groups, dirs[ndirs-2]);
This makes sense yes, we are not gonna add any folder after kernel anyway.
Quoted 30 lines
Quoted 30 lines
metadata/metaparse.c +29
-0+} + +/* +* Add groups from @group tags in the doc comment block. +*/ +static void add_doc_groups (struct data_node *groups,struct data_node *doc) +{ +if (!doc || doc->type !=DATA_ARRAY ) +return ; + +for (unsigned int i =0 ; i <data_node_array_len (doc); i++) { +struct data_node *line = doc->array.array[i]; +const char *s; + +if (line->type !=DATA_STRING ) +continue ; + + s = line->string.val; + +while (*s && (*s ==' ' || *s =='\t' )) + s++; + +if (strncmp (s,"@group " ,7 )) +continue ; + + s +=7 ; +while (*s && (*s ==' ' || *s =='\t' )) + s++; +
Quoted 8 lines
metadata/metaparse.c static void print_help(const char *prgname) · line 1240
Quoted 3 lines
metadata/metaparse.c +3
-0+if (*s) +data_node_array_add (groups,data_node_string (s)); + }If we hook up into the
multiline_comment()function we can consume the line as well so that it does not appear in the parsed doc string. We would have to pass the groups node from the main all the way tomultiline_comment()something as (uncomplete):
Will take a look into it.
--
Andrea Cervesato
SUSE QE Automation Engineer Linux
[email protected]
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 1/2] metadata: add tests grouping support
Hi!
Quoted 15 lines
I was thinking that having
@group foois cleaner. When we start to have multiple groups it can be really to handle multiple groups: for instance, we might end up having one line over 80 chars in the description.@groups foo1 bar1 foo2 bar2 foo3 bar3 ..
Instead, we might have
@group foo1
@group bar1
@group foo2
@group bar2
@group foo3
@group bar3
Quoted 2 lines
These strings are short, but if we have long names it can be weird to read having all of them on a single line.
I wasn't saying that we should support only one @groups command, rather than that we would do:
@groups foo1 foo2 foo3
@groups bar1 bar2 bar3
That would allow us to write reasonably short lines and limit the number of lines to a reasonable number as well.
Signature · 5 lines
Cyril Hrubis [email protected] -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 1/2] metadata: add tests grouping support
Quoted 5 lines
@groups foo1 foo2 foo3
@groups bar1 bar2 bar3That would allow us to write reasonably short lines and limit the number of lines to a reasonable number as well.
That makes sense yes
--
Andrea Cervesato
SUSE QE Automation Engineer Linux
[email protected]
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH 2/2 doc: conf.py: Show groups in test catalog
In reply to Andrea CervesatoFrom: Andrea Cervesato <[email protected]>
Display the groups field from metadata in the generated test catalog pages. Groups appear after the source link and before the test description.
Filter out @group tag lines from the test description to avoid duplication since they are already shown in the groups list.
--- doc/conf.py | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)
doc/conf.py +13 -1
@@ -394,6 +394,7 @@ def _generate_setup_table(keys): 'needs_checkpoints', 'forks_child', 'tags', + 'groups', ] my_keys = {k: v for k, v in keys.items() if k not in exclude} if len(my_keys) == 0:
@@ -482,11 +483,22 @@ def generate_test_catalog(_): '' ]) - # test description + # groups information + groups = conf.get('groups', None) + if groups: + text.extend([ + '', + f"**Groups**: {', '.join(groups)}", + '' + ]) + + # test description (filter out @group tags) desc = conf.get('doc', None) if desc: desc_text = [] for line in desc: + if line.lstrip().startswith('@group '): + continue line = regexp.sub(r'**\1**', line) desc_text.append(line)
Signature · 5 lines
2.51.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp