Skip to contentSkip to conversation

Test metadata extraction

PATCH Superseded Petr Vorel 4 participants 23 messages Nov 3, 2020 – Nov 16, 2020 Kbuild

lore
Share & tools
lore

Petr Vorel

author
#0 lore raw

v2 cover Test metadata extraction

Hi,

address some of the notes added by Li.

changes v1->v2:

3rd commit
https://patchwork.ozlabs.org/project/ltp/patch/[email protected]/
* add buf[i++] = c; to fix parsing ""
* check fname is valid before opening it
* fix some of checkpatch.pl problems

9th commit
* add perl-libwww-perl for fedora/centos

11 th commit
* add *.css *.js to CLEAN_TARGETS

TODO
* I didn't filter "options". But agree they should be handled better than now.
* some checkpatch.pl warnings left, do we want to bother?
docparse/docparse.c:53: WARNING: Missing a blank line after declarations docparse/docparse.c:206: ERROR: do not use assignment in if condition docparse/docparse.c:211: WARNING: Missing a blank line after declarations docparse/docparse.c:246: ERROR: do not use assignment in if condition docparse/docparse.c:288: WARNING: Missing a blank line after declarations docparse/docparse.c:297: WARNING: static const char * array should probably be static const char * const docparse/docparse.c:352: WARNING: static const char * array should probably be static const char * const
total: 2 errors, 5 warnings, 423 lines checked

* I didn't change docparse/README.md (4th commit, Jan had some notes, could you phrase what should be there?)

Anything else to change?

Tested: https://travis-ci.org/github/pevik/ltp/builds/741217630

Kind regards,
Petr

Cyril Hrubis (4):
docparse: Add test documentation parser
docparse: Add README
syscalls: Add a few documentation comments
syscalls: Move needs_drivers inside of the tst_test struct

Petr Vorel (7):
make: Support compiling native build tools
travis: Add git
make: Allow {INSTALL, MAKE}_TARGETS be a directory
make: Allow CLEAN_TARGETS to remove directories
travis: Install docparse dependencies
docparse: Add configure options
docparse: Generate html and pdf using asciidoc{, tor}

 Makefile                                      |   4 +
 configure.ac                                  |  32 +-
 docparse/.gitignore                           |   7 +
 docparse/Makefile                             |  77 ++++
 docparse/README.md                            | 248 ++++++++++
 docparse/data_storage.h                       | 299 ++++++++++++
 docparse/docparse.c                           | 423 +++++++++++++++++
 docparse/parse.sh                             |  41 ++
 docparse/testinfo.pl                          | 424 ++++++++++++++++++
 include/mk/config.mk.in                       |  21 +
 include/mk/env_post.mk                        |   3 +-
 include/mk/features.mk.in                     |   5 +
 include/mk/functions.mk                       |   3 +-
 include/mk/generic_leaf_target.inc            |  16 +-
 include/mk/rules.mk                           |   8 +
 m4/ax_compare_version.m4                      | 177 ++++++++
 m4/ax_prog_perl_modules.m4                    |  77 ++++
 m4/ltp-docparse.m4                            | 112 +++++
 testcases/kernel/syscalls/abort/abort01.c     |  16 +-
 testcases/kernel/syscalls/accept/accept01.c   |   8 +-
 testcases/kernel/syscalls/accept/accept02.c   |   7 +-
 testcases/kernel/syscalls/acct/acct01.c       |   5 +
 testcases/kernel/syscalls/acct/acct02.c       |   6 +-
 .../kernel/syscalls/fsetxattr/fsetxattr02.c   |  10 +-
 testcases/kernel/syscalls/ioctl/ioctl08.c     |   9 +-
 travis/alpine.sh                              |   4 +
 travis/debian.minimal.sh                      |   8 +-
 travis/debian.sh                              |  10 +-
 travis/fedora.sh                              |  13 +-
 travis/tumbleweed.sh                          |   9 +-
 30 files changed, 2043 insertions(+), 39 deletions(-)
 create mode 100644 docparse/.gitignore
 create mode 100644 docparse/Makefile
 create mode 100644 docparse/README.md
 create mode 100644 docparse/data_storage.h
 create mode 100644 docparse/docparse.c
 create mode 100755 docparse/parse.sh
 create mode 100755 docparse/testinfo.pl
 create mode 100644 m4/ax_compare_version.m4
 create mode 100644 m4/ax_prog_perl_modules.m4
 create mode 100644 m4/ltp-docparse.m4
Signature · 1 line
2.29.1
author
#1 lore raw

PATCH v2 1/11 make: Support compiling native build tools

Add HOST_MAKE_TARGETS make target and HOSTCC,
HOST_{CFLAGS,LDFLAGS} make variables.

Needed for cross-compilation.

NOTE: detect cross compilation with comparing $(build) and $(host) instead of using $cross_compiling configure variable, which would require move the detection into m4 macro.

---
 configure.ac                       |  2 ++
 include/mk/config.mk.in            | 21 +++++++++++++++++++++
 include/mk/env_post.mk             |  3 +--
 include/mk/generic_leaf_target.inc |  6 ++++++
 include/mk/rules.mk                |  8 ++++++++
 5 files changed, 38 insertions(+), 2 deletions(-)
configure.ac +2 -0
index 03e4e09c9..05672f8f6 100644
@@ -13,6 +13,8 @@ AC_CONFIG_FILES([ \
     execltp \
 ])
 
+AC_ARG_VAR(HOSTCC, [The C compiler on the host])
+
 AM_MAINTAINER_MODE([enable])
 
 AC_CANONICAL_HOST
include/mk/config.mk.in +21 -0
index 427608a17..8f73a6a34 100644
@@ -31,6 +31,19 @@ RANLIB			:= @RANLIB@
 STRIP			:= @STRIP@
 YACC			:= @YACC@
 
+HOSTCC  = @HOSTCC@
+build := @build@
+host := @host@
+ifeq ($(strip $(HOSTCC)),)
+# native build, respect CC
+ifeq ($(build),$(host))
+HOSTCC := $(CC)
+else
+# cross compilation
+HOSTCC := cc
+endif
+endif
+
 AIO_LIBS		:= @AIO_LIBS@
 CAP_LIBS		:= @CAP_LIBS@
 ACL_LIBS		:= @ACL_LIBS@
@@ -70,6 +83,14 @@ WCFLAGS			?= -Wall -W @GCC_WARN_OLDSTYLE@
 LDFLAGS			+= $(WLDFLAGS)
 CFLAGS			+= $(DEBUG_CFLAGS) $(OPT_CFLAGS) $(WCFLAGS)
 
+ifeq ($(strip $(HOST_CFLAGS)),)
+HOST_CFLAGS := $(CFLAGS)
+endif
+
+ifeq ($(strip $(HOST_LDFLAGS)),)
+HOST_LDFLAGS := $(LDFLAGS)
+endif
+
 LINUX_VERSION		:= @LINUX_VERSION@
 LINUX_DIR		:= @LINUX_DIR@
 LINUX_VERSION_MAJOR	:= @LINUX_VERSION_MAJOR@
include/mk/env_post.mk +1 -2
index 44a333198..d52ad9f0b 100644
@@ -48,11 +48,10 @@ LDFLAGS				+= -L$(top_builddir)/lib/android_librt
 endif
 
 MAKE_TARGETS			?= $(notdir $(patsubst %.c,%,$(wildcard $(abs_srcdir)/*.c)))
-
 MAKE_TARGETS			:= $(filter-out $(FILTER_OUT_MAKE_TARGETS),$(MAKE_TARGETS))
 
 # with only *.dwo, .[0-9]+.dwo can not be cleaned
-CLEAN_TARGETS			+= $(MAKE_TARGETS) *.o *.pyc .cache.mk *.dwo .*.dwo
+CLEAN_TARGETS			+= $(MAKE_TARGETS) $(HOST_MAKE_TARGETS) *.o *.pyc .cache.mk *.dwo .*.dwo
 
 # Majority of the files end up in testcases/bin...
 INSTALL_DIR			?= testcases/bin
include/mk/generic_leaf_target.inc +6 -0
index dd54d05e9..e6fa107d1 100644
@@ -57,6 +57,8 @@
 #				     rope to hang one's self in the event of
 #				     unwanted behavior.
 #
+# $(HOST_MAKE_TARGETS)	: Host tools which use $HOSTCC.
+#
 # $(CLEAN_TARGETS)		: What targets should be cleaned (must be
 #				  real files). This will automatically append
 #				  adds the .o suffix to all files referenced
@@ -92,6 +94,10 @@
 
 .PHONY: all clean install
 
+ifneq ($(strip $(MAKE_TARGETS)),)
+$(MAKE_TARGETS) += $(HOST_MAKE_TARGETS)
+endif
+
 $(MAKE_TARGETS): | $(MAKE_DEPS)
 
 all: $(MAKE_TARGETS)
include/mk/rules.mk +8 -0
index 6a22e43af..c8f4bbbbe 100644
@@ -22,6 +22,14 @@ else
 	@echo LD $(target_rel_dir)$@
 endif
 
+$(HOST_MAKE_TARGETS): %: %.c
+ifdef VERBOSE
+	$(HOSTCC) $(HOST_CFLAGS) $(HOST_LDFLAGS) $< $(HOST_LDLIBS) -o $@
+else
+	@$(HOSTCC) $(HOST_CFLAGS) $(HOST_LDFLAGS) $< $(HOST_LDLIBS) -o $@
+	@echo HOSTCC $(target_rel_dir)$@
+endif
+
 %: %.c
 ifdef VERBOSE
 	$(CC) $(CPPFLAGS) $(CFLAGS) $(LDFLAGS) $^ $(LTPLDLIBS) $(LDLIBS) -o $@
Signature · 1 line
2.29.1
author
#2 lore raw

PATCH v2 2/11 travis: Add git

In reply to Petr Vorel

Needed in parse.sh script in next commit.

---
 travis/alpine.sh     | 1 +
 travis/debian.sh     | 1 +
 travis/fedora.sh     | 1 +
 travis/tumbleweed.sh | 1 +
 4 files changed, 4 insertions(+)
travis/alpine.sh +1 -0
index 61ef144a8..f8960bed0 100755
@@ -10,6 +10,7 @@ apk add \
 	automake \
 	clang \
 	gcc \
+	git \
 	keyutils-dev \
 	libaio-dev \
 	libacl \
travis/debian.sh +1 -0
index b759a9576..28685f4d3 100755
@@ -17,6 +17,7 @@ apt install -y --no-install-recommends \
 	devscripts \
 	clang \
 	gcc \
+	git \
 	libacl1 \
 	libacl1-dev \
 	libaio-dev \
travis/fedora.sh +1 -0
index 990a84daf..3c224f71e 100755
@@ -8,6 +8,7 @@ yum -y install \
 	make \
 	clang \
 	gcc \
+	git \
 	findutils \
 	libtirpc \
 	libtirpc-devel \
travis/tumbleweed.sh +1 -0
index 4d5e9da79..6247daa98 100755
@@ -8,6 +8,7 @@ zypper --non-interactive install --force-resolution --no-recommends \
 	clang \
 	findutils \
 	gcc \
+	git \
 	gzip \
 	make \
 	kernel-default-devel \
Signature · 1 line
2.29.1
author
#3 lore raw

PATCH v2 3/11 docparse: Add test documentation parser

In reply to Petr Vorel

From: Cyril Hrubis <[email protected]>

This commit implements a simple parser to pick up metadata that are nicely structured in the tst_test structure. In order to do that it implements very simple and very basic C tokenizer.

The result of the metadata extraction is one big json file that contains data extracted from all new library testcases plus some information about the LTP version etc.

The tokenizer also extracts special documentation comments that are then used for a test description. The documentation format has been choosen to be asciidoc.

Special documentation comments starts with string "/*\\n" and is, by convention ended with the same, so it looks like:

/*\
* [DOCUMENTATION]
* - foo
* - bar
*
* ...
*
\*/

[ pvorel: fixes
- parse.sh: git related fixes, out-of-tree build
- docparse: Eat also space following *
- Makefile: remove header rule (broken for clang)
]

---
 Makefile                |   2 +-
 docparse/.gitignore     |   2 +
 docparse/Makefile       |  19 ++
 docparse/data_storage.h | 299 ++++++++++++++++++++++++++++
 docparse/docparse.c     | 423 ++++++++++++++++++++++++++++++++++++++++
 docparse/parse.sh       |  41 ++++
 docparse/testinfo.pl    |  40 ++++
 7 files changed, 825 insertions(+), 1 deletion(-)
 create mode 100644 docparse/.gitignore
 create mode 100644 docparse/Makefile
 create mode 100644 docparse/data_storage.h
 create mode 100644 docparse/docparse.c
 create mode 100755 docparse/parse.sh
 create mode 100755 docparse/testinfo.pl
Makefile +1 -1
index bf5077231..3830fb6d4 100644
@@ -62,7 +62,7 @@ $(1):: | $$(abs_top_builddir)/$$(basename $$(subst -,.,$(1)))
 endif
 endef
 
-COMMON_TARGETS		+= testcases tools
+COMMON_TARGETS		+= testcases tools docparse
 # Don't want to nuke the original files if we're installing in-build-tree.
 ifneq ($(BUILD_TREE_STATE),$(BUILD_TREE_SRCDIR_INSTALL))
 INSTALL_TARGETS		+= runtest scenario_groups testscripts
docparse/.gitignore new file +2 -0
new file mode 100644
index 000000000..f636ed847
@@ -0,0 +1,2 @@
+/docparse
+/metadata.json
docparse/Makefile new file +19 -0
new file mode 100644
index 000000000..94ba83ffe
@@ -0,0 +1,19 @@
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2019 Cyril Hrubis <[email protected]>
+
+top_srcdir		?= ..
+
+include $(top_srcdir)/include/mk/env_pre.mk
+include $(top_srcdir)/include/mk/functions.mk
+
+MAKE_TARGETS		:= metadata.json
+HOST_MAKE_TARGETS	:= docparse
+
+INSTALL_DIR = metadata
+
+.PHONY: metadata.json
+
+metadata.json: docparse
+	$(abs_srcdir)/parse.sh > metadata.json
+
+include $(top_srcdir)/include/mk/generic_leaf_target.mk
docparse/data_storage.h new file +299 -0

299 lines not on this page · raw message

docparse/docparse.c new file +423 -0

423 lines not on this page · raw message

docparse/parse.sh new file +41 -0

41 lines not on this page · raw message

docparse/testinfo.pl new file +40 -0
new file mode 100755
index 000000000..d93d7d701
@@ -0,0 +1,40 @@
+#!/usr/bin/perl
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2019 Cyril Hrubis <[email protected]>
+
+use strict;
+use warnings;
+
+use JSON;
+use Data::Dumper;
+
+sub load_json
+{
+	my ($fname) = @_;
+	local $/;
+
+	open(my $fh, '<', $fname) or die("Can't open $fname $!");
+
+	return <$fh>;
+}
+
+sub query_flag
+{
+	my ($json, $flag) = @_;
+
+	my $tests = $json->{'tests'};
+
+	foreach my $key (sort(keys %$tests)) {
+		if ($tests->{$key}->{$flag}) {
+			if ($tests->{$key}->{$flag} eq "1") {
+				print("$key\n");
+			} else {
+				print("$key:\n" . Dumper($tests->{$key}->{$flag}) . "\n");
+			}
+		}
+	}
+}
+
+my $json = decode_json(load_json($ARGV[0]));
+
+query_flag($json, $ARGV[1]);
Signature · 1 line
2.29.1
author
#4 lore raw

PATCH v2 3/11 docparse: Add test documentation parser

Hi,

Quoted 2 lines
docparse/docparse.c new file +0 -0

...

Quoted 9 lines
docparse/docparse.c new file +9 -0
+const char *next_token(FILE *f, struct data_node *doc)
+{
+	size_t i = 0;
+	static char buf[4096];
+	int c;
+	int in_str = 0;
+
+	for (;;) {
+		c = fgetc(f);

docparse/docparse.c line 144

Quoted 10 lines
docparse/docparse.c new file +10 -0
+
+		if (c == EOF)
+			goto exit;
+
+		if (in_str) {
+			if (c == '"') {
+				if (i == 0 || buf[i-1] != '\\') {
+					buf[i++] = c;
+					goto exit;
+				}

IMHO this change (adding buf[i++] = c;), which Li suggested, is causing build failures:
https://travis-ci.org/github/pevik/ltp/builds/741217630

/usr/src/ltp/docparse/testinfo.pl metadata.json
, or ] expected while parsing array, at character offset 340 (before "",\n "0c461cb72...") at /usr/src/ltp/docparse/testinfo.pl line 379.
make[1]: *** [/usr/src/ltp/docparse/Makefile:60: txt] Error 255 make[1]: Leaving directory '/usr/src/ltp-build/docparse'
make: *** [/usr/src/ltp/Makefile:108: docparse-all] Error 2
make: *** Waiting for unfinished jobs....

It's strange that it's not failing in all tests.

Anyway, test is working without it:
https://travis-ci.org/github/pevik/ltp/builds/741322879

@@ -138,10 +138,8 @@ const char *next_token(FILE *f, struct data_node *doc)

                if (in_str) {
                        if (c == '"') {
-                               if (i == 0 || buf[i-1] != '\\') {
-                                       buf[i++] = c;
+                               if (i == 0 || buf[i-1] != '\\')
                                        goto exit;
-                               }
                        }

                        buf[i++] = c;

----

Li, could you please comment?

Kind regards,
Petr

#5 lore raw

PATCH v2 3/11 docparse: Add test documentation parser

Quoted 29 lines · Petr Vorel

Hi,

Quoted 2 lines
docparse/docparse.c new file +0 -0

...

Quoted 9 lines
docparse/docparse.c new file +9 -0
+const char *next_token(FILE *f, struct data_node *doc)
+{
+     size_t i = 0;
+     static char buf[4096];
+     int c;
+     int in_str = 0;
+
+     for (;;) {
+             c = fgetc(f);

docparse/docparse.c

Quoted 10 lines
docparse/docparse.c new file +10 -0
+
+             if (c == EOF)
+                     goto exit;
+
+             if (in_str) {
+                     if (c == '"') {
+                             if (i == 0 || buf[i-1] != '\\') {
+                                     buf[i++] = c;
+                                     goto exit;
+                             }

IMHO this change (adding buf[i++] = c;), which Li suggested, is causing build failures:
https://travis-ci.org/github/pevik/ltp/builds/741217630

Quoted 8 lines

/usr/src/ltp/docparse/testinfo.pl metadata.json
, or ] expected while parsing array, at character offset 340 (before
"",\n "0c461cb72...") at /usr/src/ltp/docparse/testinfo.pl line 379.
make[1]: *** [/usr/src/ltp/docparse/Makefile:60: txt] Error 255 make[1]: Leaving directory '/usr/src/ltp-build/docparse'
make: *** [/usr/src/ltp/Makefile:108: docparse-all] Error 2
make: *** Waiting for unfinished jobs....

Sorry for the error, it because of wrongly parsed tst_test struct with including .min_kver = "2.6.32", then output into metada.json like:

# cat metadata.json
...
  "cpuset01": {
   "needs_root": "1",
   "forks_child": "1",
   "min_kver": "2.6.32"",  <====== here has two "" in the end
   "fname": "testcases/kernel/mem/cpuset/cpuset01.c"
  },

So the "$(abs_srcdir)/testinfo.pl metadata.json" in Makefile report failure as above.

==========

Another fix I can think of is just to avoid return NULL if detecting in the string("").
Hope this will work for us, I will help to check it again tomorrow.

docparse/docparse.c +2 -4
@@ -138,10 +138,8 @@ const char *next_token(FILE *f, struct data_node *doc)
 
                if (in_str) {
                        if (c == '"') {
-                               if (i == 0 || buf[i-1] != '\\') {
-                                       buf[i++] = c;
+                               if (i == 0 || buf[i-1] != '\\')
                                        goto exit;
-                               }
                        }
 
                        buf[i++] = c;
@@ -189,7 +187,7 @@ const char *next_token(FILE *f, struct data_node *doc)
        }
 
 exit:
-       if (i == 0)
+       if (i == 0 && !in_str)
                return NULL;
 
        buf[i] = 0;
Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20201104/56373c64/attachment-0001.htm>
author
#6 lore raw

PATCH v2 3/11 docparse: Add test documentation parser

Hi Li,

...

Quoted 49 lines
Quoted 11 lines

IMHO this change (adding buf[i++] = c;), which Li suggested, is causing build failures:
https://travis-ci.org/github/pevik/ltp/builds/741217630

/usr/src/ltp/docparse/testinfo.pl metadata.json
, or ] expected while parsing array, at character offset 340 (before
"",\n "0c461cb72...") at /usr/src/ltp/docparse/testinfo.pl line 379.
make[1]: *** [/usr/src/ltp/docparse/Makefile:60: txt] Error 255 make[1]: Leaving directory '/usr/src/ltp-build/docparse'
make: *** [/usr/src/ltp/Makefile:108: docparse-all] Error 2
make: *** Waiting for unfinished jobs....

Sorry for the error, it because of wrongly parsed tst_test struct with including .min_kver = "2.6.32", then output into metada.json like:

# cat metadata.json
...
  "cpuset01": {
   "needs_root": "1",
   "forks_child": "1",
   "min_kver": "2.6.32"",  <====== here has two "" in the end
   "fname": "testcases/kernel/mem/cpuset/cpuset01.c"
  },

So the "$(abs_srcdir)/testinfo.pl metadata.json" in Makefile report failure as above.

==========

Another fix I can think of is just to avoid return NULL if detecting in the string("").
Hope this will work for us, I will help to check it again tomorrow.

docparse/docparse.c +1 -3
@@ -138,10 +138,8 @@ const char *next_token(FILE *f, struct data_node *doc)
 
                if (in_str) {
                        if (c == '"') {
-                               if (i == 0 || buf[i-1] != '\\') {
-                                       buf[i++] = c;
+                               if (i == 0 || buf[i-1] != '\\')
                                        goto exit;
-                               }
                        }
 
                        buf[i++] = c;

docparse/docparse.c const char *next_token(FILE *f, struct data_node *doc) · line 193

Quoted 8 lines
docparse/docparse.c +1 -1
@@ -189,7 +187,7 @@ const char *next_token(FILE *f, struct data_node *doc)
        }
 
 exit:
-       if (i == 0)
+       if (i == 0 && !in_str)
                return NULL;
 
        buf[i] = 0;

Thanks for explanation. Yes, that's a working solution:
https://travis-ci.org/github/pevik/ltp/builds/741418867

Kind regards,
Petr

author
#7 lore raw

PATCH v2 4/11 docparse: Add README

In reply to Petr Vorel

From: Cyril Hrubis <[email protected]>

* example of C source and JSON
* note about exporting timeouts

[ Tim: fix typos and clean up grammar ]
---
 docparse/README.md | 248 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 248 insertions(+)
 create mode 100644 docparse/README.md
docparse/README.md new file +248 -0
new file mode 100644
index 000000000..7e4847ba2
@@ -0,0 +1,248 @@
+Motivation for metadata exctraction
+===================================
+
+Exporting documentation
+-----------------------
+
+This allow us to build browseable documentation for the testcases, e.g. a
+catalogue of test information that would be searchable etc. At this point there
+is a single page generated from the extracted data that tries to outline the
+intent.
+
+
+Propagating test requirements
+-----------------------------
+
+Some subtests require differnt hardware resources/software versions/etc. the
+test execution framework needs to consume these so that it can locate proper
+hardware, install proper software, etc.
+
+Some examples of requriments are:
+
+* Test needs at least 1GB of RAM.
+
+* Test needs a block device at least 512MB in size
+
+* Test needs a NUMA machine with two memory nodes and at least 300 free pages on each node
+
+* Test needs i2c eeprom connected on a i2c bus
+
+* Test needs two serial ports connected via null-modem cable
+
+
+With this information extracted from the tests the testrunner can then map the
+requiremnts on the available machines in a lab and select a proper machine for
+the particular (sub)set of testcases as well as supply a particular test with
+additional information needed for the test, such as address of the i2c device,
+paths to the serial devices, etc. In the case of virtual machines the test could
+also dynamically prepare the correct environment for the test on demand.
+
+
+Parallel test execution
+-----------------------
+
+An LTP testrun on a modern hardware wastes most of the machine resources
+because the testcases are running sequentially. However in order to execute
+tests in parallel we need to know which system resources are utilized by a
+given test, as obviously we cannot run two tests that monopolize the same
+resource. In some cases we would also need to partition the system resource
+accordingly, e.g. if we have two memory stress tests running at the same time
+we will need to cap each of these tests on half of the available memory, or
+make sure that sum of the memory used by these two tests is not greater an
+available memory.
+
+Examples of such tests are:
+
+* Tests that mess with global system state
+   - system time (e.g. settimeofday() test and leap second test)
+   - SysV SHM
+   - ...
+
+* Tests that use block device
+
+* Tests that work with a particular hardware resource
+  - i2c eeprom test
+  - serial port tests
+  - ...
+
+Exporting test runtime/timeout to the testrunner
+------------------------------------------------
+
+Currently most of the testrunners usually do not know for how long is the test
+supposed to run, this means that we have to guess some upper limit on how long
+a test is supposed to run. The value is usually twice of the maximal runtime
+for all testcases or whole suite or even larger. This means that we are wasting
+time in the case that the test ends up stuck and we could have failed it much
+sooner in most of the cases. This becomes quite important for a kernel
+regression tests that do crash the host, if the information that the test is
+supposed to crash a kernel under a minute is exported to the testrunner we can
+reboot the machine much faster in an event of a crash.
+
+Getting rid of runtest files
+----------------------------
+
+This would also allow us to get rid of the unflexible and hard to maintain
+runtest files. Once this system is in place we will have a list of all tests
+along with their respective metadata - which means that we will be able to
+generate subsets of the test easily on the fly.
+
+In order to achieve this we need two things:
+
+Each test will describe which syscall/functionality it tests in the metadata.
+Then we could define groups of tests based on that. I.e. instead of having
+syscall runtest file we would ask the testrunner to run all test that have a
+defined which syscall they test, or whose filename matches a particular syscall name.
+
+Secondly we will have to store the test variants in the test metadata instead
+of putting them in a file that is unrelated to the test.
+
+For example:
+
+* To run CVE related test we would select testcases with CVE tag
+
+* To run IPC test we will define a list of IPC syscalls and run all syscall
+  test that are in the list
+
+* And many more...
+
+
+Implementation
+==============
+
+The docparser is implemented as a minimal C tokenizer that can parse and
+extract code comments and C structures. The docparser then runs over all C
+sources in the testcases directory and if tst\_test structure is present in the
+source it's parsed and the result is included in the resulting metadata.
+
+During parsing the metadata is stored in a simple key/value storage that more
+or less follows C structure layout, i.e. can include hash, array, and string.
+Once the parsing is finished the result is filtered so that only interesting
+fields of the tst\_test structure are included and then converted into JSON
+output.
+
+This process produces one big JSON file with metadata for all tests, that
+is then installed along with the testcases. This would then be used by the
+testrunner.
+
+The test requirements are stored in the tst\_test structure either as
+bitflags, integers or arrays of strings:
+
+```c
+struct tst_test test = {
+	...
+	/* tests needs to run with UID=1 */
+	.needs_root = 1,
+
+	/*
+	 * Tests needs a block device at least 1024MB in size and also
+	 * mkfs.ext4 installed.
+	 */
+	.needs_device = 1,
+	.dev_min_size = 1024,
+	.dev_fs_type = ext4,
+
+	/* Indicates that the test is messing with system wall clock */
+	.restore_wallclock = 1,
+
+	/* Tests needs uinput either compiled in or loaded as a module */
+	.needs_drivers = (const char *[]) {
+		"uinput",
+		NULL
+	},
+
+	/* Tests needs enabled kernel config flags */
+	.needs_kconfigs = (const char *[]) {
+		"CONFIG_X86_INTEL_UMIP=y",
+		NULL
+	},
+
+	/* Additional array of key value pairs */
+	.tags = (const struct tst_tag[]) {
+                {"linux-git", "43a6684519ab"},
+                {"CVE", "2017-2671"},
+                {NULL, NULL}
+        }
+};
+```
+
+The test documentation is stored in a special comment such as:
+
+```
+/*\
+ * Test description
+ *
+ * This is a test description.
+ * Consisting of several lines.
+\*/
+```
+
+Which will yield following json output:
+
+```json
+ "testcaseXY": {
+  "needs_root": "1",
+  "needs_device": "1",
+  "dev_min_size": "1024",
+  "dev_fs_type": "ext4",
+  "restore_wallclock": "1",
+  "needs_drivers": [
+    "uinput",
+  ],
+  "needs_kconfigs": [
+    "CONFIG_X86_INTEL_UMIP=y",
+  ],
+  "tags": [
+    [
+     "linux-git",
+     "43a6684519ab"
+    ],
+    [
+     "CVE",
+     "2017-2671"
+    ],
+   ],
+  "doc": [
+    "Test description",
+    "",
+    "This is a test description.",
+    "Consisting of several lines."
+  ],
+  "fname": "testcases/kernel/syscalls/foo/testcaseXY.c"
+ },
+```
+
+The final JSON file is JSON object of test descriptions indexed by a test name
+with a header describing the testsuite:
+
+```json
+{
+ "testsuite": "Linux Test Project",
+ "testsuite_short": "LTP",
+ "url": "https://github.com/linux-test-project/ltp/",
+ "scm_url_base": "https://github.com/linux-test-project/ltp/tree/master/",
+ "timeout": 300,
+ "version": "20200930",
+ "tests": {
+  "testcaseXY": {
+   ...
+  },
+  ...
+ }
+}
+```
+
+Open Points
+===========
+
+There are stil some loose ends. Mostly it's not well defined where to put
+things and how to format them.
+
+* Some of the hardware requirements are already listed in the tst\_test. Should
+  we put all of them there?
+
+* What would be the format for test documentation and how to store things such
+  as test variants there?
+
+So far this proof of concept generates a metadata file. I guess that we need
+actual consumers which will help to settle things down, I will try to look into
+making use of this in the runltp-ng at least as a reference implementation.
Signature · 1 line
2.29.1
author
#8 lore raw

PATCH v2 5/11 syscalls: Add a few documentation comments

In reply to Petr Vorel

From: Cyril Hrubis <[email protected]>

So that it shows up in the resulting json file.

---
 testcases/kernel/syscalls/abort/abort01.c   | 16 ++++++++++------
 testcases/kernel/syscalls/accept/accept01.c |  8 +++++---
 testcases/kernel/syscalls/accept/accept02.c |  7 +++++--
 testcases/kernel/syscalls/acct/acct01.c     |  5 +++++
 testcases/kernel/syscalls/acct/acct02.c     |  6 ++++--
 5 files changed, 29 insertions(+), 13 deletions(-)
testcases/kernel/syscalls/abort/abort01.c +10 -6
index 9505a5eec..b93324b34 100644
@@ -5,14 +5,18 @@
  *   01/02/2003	Port to LTP	[email protected]
  *   11/11/2002: Ported to LTP Suite by Ananda
  *   06/30/2001	Port to Linux	[email protected]
- *
- * ALGORITHM
- *	Fork child.  Have child abort, check return status.
- *
- * RESTRICTIONS
- *      The ulimit for core file size must be greater than 0.
  */
 
+/*\
+ * [DESCRIPTION]
+ *  Checks that process which called abort() gets killed by SIGIOT and dumps core.
+ *
+ * [ALGORITHM]
+ *  - Fork child.
+ *  - Child calls abort.
+ *  - Parent checks return status.
+\*/
+
 #include <sys/types.h>
 #include <sys/wait.h>
 #include <errno.h>
testcases/kernel/syscalls/accept/accept01.c +5 -3
index 4e30906f2..01d6db84c 100644
@@ -3,11 +3,13 @@
 /*
  *   Copyright (c) International Business Machines  Corp., 2001
  *   07/2001 Ported by Wayne Boyer
- *
- *   Description:
- *     Verify that accept() returns the proper errno for various failure cases
  */
 
+/*\
+ * [DESCRIPTION]
+ * Verify that accept() returns the proper errno for various failure cases.
+\*/
+
 #include <stdio.h>
 #include <unistd.h>
 #include <errno.h>
testcases/kernel/syscalls/accept/accept02.c +5 -2
index 37ab8b64f..7fb6a494a 100644
@@ -3,7 +3,10 @@
  * Copyright (c) 2019 SUSE LLC
  * Author: Christian Amann <[email protected]>
  */
-/* Test for CVE-2017-8890
+/*\
+ * [DESCRIPTION]
+ *
+ * Test for CVE-2017-8890
  *
  * In Kernels up to 4.10.15 missing commit 657831ff the multicast
  * group information of a socket gets copied over to a newly created
@@ -16,7 +19,7 @@
  *
  * For more information about this CVE see:
  * https://www.suse.com/security/cve/CVE-2017-8890/
- */
+\*/
 
 #include <errno.h>
 #include <sys/socket.h>
testcases/kernel/syscalls/acct/acct01.c +5 -0
index c161d2a2c..60e81bfad 100644
@@ -7,6 +7,11 @@
 /* 12/03/2002	Port to LTP     [email protected] */
 /* 06/30/2001	Port to Linux	[email protected] */
 
+/*\
+ * [DOCUMENTATION]
+ *  Verify that acct() returns proper errno on failure.
+\*/
+
 #include <sys/types.h>
 #include <sys/stat.h>
 #include <errno.h>
testcases/kernel/syscalls/acct/acct02.c +4 -2
index 8ee1bfcf8..e718e7df4 100644
@@ -3,7 +3,9 @@
  *  Copyright (c) SUSE LLC, 2019
  *  Author: Christian Amann <[email protected]>
  */
-/*
+/*\
+ * [DOCUMENTATION]
+ *
  * This tests if the kernel writes correct data to the
  * process accounting file.
  *
@@ -19,7 +21,7 @@
  *
  * This is also accidental regression test for:
  * 4d9570158b626 kernel/acct.c: fix the acct->needcheck check in check_free_space()
- */
+\*/
 
 #include <sys/stat.h>
 #include <errno.h>
Signature · 1 line
2.29.1
#9 lore raw

PATCH v2 5/11 syscalls: Add a few documentation comments

Hello Petr,

Quoted 4 lines · Petr Vorel

+ * [DESCRIPTION]
+ * Checks that process which called abort() gets killed by SIGIOT and dumps core.

...

testcases/kernel/syscalls/acct/acct01.c line 14

Quoted 5 lines
testcases/kernel/syscalls/acct/acct01.c +5 -0
+/*\
+ * [DOCUMENTATION]
+ *  Verify that acct() returns proper errno on failure.
+\*/
+

I noticed that in some places the DESCRIPTION tag is used and in another DOCUMENTATION. What is the difference between the two and which should be used when?

Regards,
Rado

author
#10 lore raw

PATCH v2 5/11 syscalls: Add a few documentation comments

Hi Rado, all,

Quoted 10 lines

Hello Petr,

Quoted 3 lines · Petr Vorel

+ * [DESCRIPTION]
+ * Checks that process which called abort() gets killed by SIGIOT and dumps core.

...

Quoted 9 lines

testcases/kernel/syscalls/acct/acct01.c line 14

Quoted 5 lines
testcases/kernel/syscalls/acct/acct01.c +5 -0
+/*\
+ * [DOCUMENTATION]
+ *  Verify that acct() returns proper errno on failure.
+\*/
+

I noticed that in some places the DESCRIPTION tag is used and in another DOCUMENTATION. What is the difference between the two and which should be used when?

I'd suggest to use [DESCRIPTION] and drop [DOCUMENTATION] (whole metadata are documentation).

It'd be good to document that in README.md

Kind regards,
Petr

#11 lore raw

PATCH v2 5/11 syscalls: Add a few documentation comments

In reply to Radoslav Kolev

Hi!

Quoted 6 lines
Quoted 3 lines

+ * [DESCRIPTION]
+ * Checks that process which called abort() gets killed by SIGIOT and dumps core.

...

Quoted 9 lines

testcases/kernel/syscalls/acct/acct01.c line 14

Quoted 5 lines
testcases/kernel/syscalls/acct/acct01.c +5 -0
+/*\
+ * [DOCUMENTATION]
+ *  Verify that acct() returns proper errno on failure.
+\*/
+

I noticed that in some places the DESCRIPTION tag is used and in another DOCUMENTATION. What is the difference between the two and which should be used when?

This is one of the open points at the moment. The exact format of the test description needs to be decided on and written down.

However I wanted to get the code reviewed and included in upstream even though there are a few loose ends since it's complex enough as it is and the detail can be worked on later. Especially since I want to make use of the metadata for parallel test execution and the documentation part is not that important there.

Signature · 2 lines
Cyril Hrubis
[email protected]
author
#12 lore raw

PATCH v2 5/11 syscalls: Add a few documentation comments

Hi Cyril,

Quoted 16 lines
Quoted 6 lines
Quoted 3 lines

+ * [DESCRIPTION]
+ * Checks that process which called abort() gets killed by SIGIOT and dumps core.

...

Quoted 9 lines

testcases/kernel/syscalls/acct/acct01.c line 14

Quoted 5 lines
testcases/kernel/syscalls/acct/acct01.c +5 -0
+/*\
+ * [DOCUMENTATION]
+ *  Verify that acct() returns proper errno on failure.
+\*/
+

I noticed that in some places the DESCRIPTION tag is used and in another DOCUMENTATION. What is the difference between the two and which should be used when?

Quoted 2 lines

This is one of the open points at the moment. The exact format of the test description needs to be decided on and written down.

+1.

Quoted 5 lines

However I wanted to get the code reviewed and included in upstream even though there are a few loose ends since it's complex enough as it is and the detail can be worked on later. Especially since I want to make use of the metadata for parallel test execution and the documentation part is not that important there.

+1. I also wouldn't want to block whole patchset because documentation is not set.

Kind regards,
Petr

author
#13 lore raw

PATCH v2 6/11 syscalls: Move needs_drivers inside of the tst_test struct

In reply to Petr Vorel

From: Cyril Hrubis <[email protected]>

---
 testcases/kernel/syscalls/fsetxattr/fsetxattr02.c | 10 ++++------
 testcases/kernel/syscalls/ioctl/ioctl08.c         |  9 ++++-----
 2 files changed, 8 insertions(+), 11 deletions(-)
testcases/kernel/syscalls/fsetxattr/fsetxattr02.c +4 -6
index 205e80c95..3aea4b59e 100644
@@ -241,11 +241,6 @@ static void cleanup(void)
 	}
 }
 
-static const char *const needed_drivers[] = {
-	"brd",
-	NULL,
-};
-
 static struct tst_test test = {
 	.setup = setup,
 	.test = verify_fsetxattr,
@@ -254,7 +249,10 @@ static struct tst_test test = {
 	.needs_devfs = 1,
 	.mntpoint = MNTPOINT,
 	.needs_root = 1,
-	.needs_drivers = needed_drivers,
+	.needs_drivers = (const char *const[]) {
+		"brd",
+		NULL,
+	},
 };
 
 #else /* HAVE_SYS_XATTR_H */
testcases/kernel/syscalls/ioctl/ioctl08.c +4 -5
index dca898a65..f7d11815d 100644
@@ -112,10 +112,6 @@ static void setup(void)
 			sizeof(struct file_dedupe_range_info));
 }
 
-static const char *const needed_drivers[] = {
-	"btrfs",
-	NULL,
-};
 
 static struct tst_test test = {
 	.test = verify_ioctl,
@@ -127,7 +123,10 @@ static struct tst_test test = {
 	.mount_device = 1,
 	.mntpoint = MNTPOINT,
 	.dev_fs_type = "btrfs",
-	.needs_drivers = needed_drivers,
+	.needs_drivers = (const char *const[]) {
+		"btrfs",
+		NULL,
+	},
 };
 #else
 	TST_TEST_TCONF(
Signature · 1 line
2.29.1
author
#14 lore raw

PATCH v2 7/11 make: Allow {INSTALL, MAKE}_TARGETS be a directory

In reply to Petr Vorel

by detecting it and adding required -d parameter for install and installing whole directory with -t.

This will be needed for metadata metadata.chunked target.

---
 include/mk/functions.mk | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
include/mk/functions.mk +2 -1
index 79c6193ca..e86dbccdc 100644
@@ -35,7 +35,8 @@ INSTALL_FILES		+= $$(abspath $$(DESTDIR)/$(3)/$(1))
 
 $$(abspath $$(DESTDIR)/$(3)/$(1)): \
     $$(abspath $$(dir $$(DESTDIR)/$(3)/$(1)))
-	install -m $$(INSTALL_MODE) "$(2)/$(1)" "$$@"
+	install -m $$(INSTALL_MODE) $(shell test -d "$(2)/$(1)" && echo "-d") $(PARAM) "$(2)/$(1)" $$@
+	$(shell test -d "$(2)/$(1)" && echo "install -m "'$$(INSTALL_MODE) $(PARAM)' "$(2)/$(1)/*" -t '$$@')
 endef
 
 #
Signature · 1 line
2.29.1
author
#15 lore raw

PATCH v2 8/11 make: Allow CLEAN_TARGETS to remove directories

In reply to Petr Vorel

make clean on leaf target can contain also directories.

Changing delete to recursive delete (-r parameter for rm) should be safe thus detection whether actually requited is not implemented.

This will be needed for metadata metadata.chunked target.

---
 include/mk/generic_leaf_target.inc | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
include/mk/generic_leaf_target.inc +5 -5
index e6fa107d1..64953f89a 100644
@@ -60,10 +60,10 @@
 # $(HOST_MAKE_TARGETS)	: Host tools which use $HOSTCC.
 #
 # $(CLEAN_TARGETS)		: What targets should be cleaned (must be
-#				  real files). This will automatically append
-#				  adds the .o suffix to all files referenced
-#				  by $(MAKE_TARGETS)) to CLEAN_TARGETS, if
-#				  MAKE_TARGETS wasn't defined (see
+#				  real files or directories). This will automatically append
+#				  adds the .o suffix to all files referenced by
+#				  $(MAKE_TARGETS)) to CLEAN_TARGETS, if MAKE_TARGETS wasn't
+#				  defined (see
 #				  $(MAKE_TARGETS)).
 # $(INSTALL_MODE)		: What mode should we using when calling
 # 				  install(1)?
@@ -103,7 +103,7 @@ $(MAKE_TARGETS): | $(MAKE_DEPS)
 all: $(MAKE_TARGETS)
 
 clean:: $(CLEAN_DEPS)
-	-$(RM) -f $(CLEAN_TARGETS)
+	-$(RM) -f -r $(CLEAN_TARGETS)
 
 $(INSTALL_FILES): | $(INSTALL_DEPS)
 
Signature · 1 line
2.29.1
author
#16 lore raw

PATCH v2 9/11 travis: Install docparse dependencies

In reply to Petr Vorel
---
 travis/alpine.sh         |  3 +++
 travis/debian.minimal.sh |  8 +++++++-
 travis/debian.sh         |  9 ++++++++-
 travis/fedora.sh         | 12 +++++++++---
 travis/tumbleweed.sh     |  8 +++++++-
 5 files changed, 34 insertions(+), 6 deletions(-)
travis/alpine.sh +3 -0
index f8960bed0..b793a9fbe 100755
@@ -6,6 +6,8 @@ apk update
 
 apk add \
 	acl-dev \
+	asciidoc \
+	asciidoctor \
 	autoconf \
 	automake \
 	clang \
@@ -23,6 +25,7 @@ apk add \
 	musl-dev \
 	numactl-dev \
 	openssl-dev \
+	perl-json \
 	pkgconfig
 
 cat /etc/os-release
travis/debian.minimal.sh +7 -1
index 3f1941969..5e6ba8662 100755
@@ -2,7 +2,11 @@
 # Copyright (c) 2018-2020 Petr Vorel <[email protected]>
 set -ex
 
-apt remove -y \
+apt="apt remove -y"
+
+$apt \
+	asciidoc \
+	asciidoctor \
 	libacl1-dev \
 	libaio-dev \
 	libaio1 \
@@ -17,3 +21,5 @@ apt remove -y \
 	libsepol1-dev \
 	libssl-dev \
 	libtirpc-dev
+
+$apt asciidoc-base ruby-asciidoctor || true
travis/debian.sh +8 -1
index 28685f4d3..743b79001 100755
@@ -8,8 +8,12 @@ grep -v oldstable-updates /etc/apt/sources.list > /tmp/sources.list && mv /tmp/s
 
 apt update
 
-apt install -y --no-install-recommends \
+apt="apt install -y --no-install-recommends"
+
+$apt \
 	acl-dev \
+	asciidoc \
+	asciidoctor \
 	autoconf \
 	automake \
 	build-essential \
@@ -26,6 +30,7 @@ apt install -y --no-install-recommends \
 	libcap2 \
 	libc6 \
 	libc6-dev \
+	libjson-perl \
 	libkeyutils-dev \
 	libkeyutils1 \
 	libmm-dev \
@@ -40,4 +45,6 @@ apt install -y --no-install-recommends \
 	lsb-release \
 	pkg-config
 
+$apt ruby-asciidoctor-pdf || true
+
 df -hT
travis/fedora.sh +9 -3
index 3c224f71e..665915a47 100755
@@ -2,7 +2,10 @@
 # Copyright (c) 2018-2020 Petr Vorel <[email protected]>
 set -ex
 
-yum -y install \
+yum="yum -y install"
+
+$yum \
+	asciidoc \
 	autoconf \
 	automake \
 	make \
@@ -12,8 +15,11 @@ yum -y install \
 	findutils \
 	libtirpc \
 	libtirpc-devel \
+	perl-JSON \
+	perl-libwww-perl \
 	pkg-config \
 	redhat-lsb-core
 
-# CentOS 8 doesn't have libmnl-devel
-yum -y install libmnl-devel || yum -y install libmnl
+# CentOS 8 fixes
+$yum libmnl-devel || $yum libmnl
+$yum rubygem-asciidoctor || true
travis/tumbleweed.sh +7 -1
index 6247daa98..43ca3252a 100755
@@ -2,7 +2,10 @@
 # Copyright (c) 2018-2020 Petr Vorel <[email protected]>
 set -ex
 
-zypper --non-interactive install --force-resolution --no-recommends \
+zyp="zypper --non-interactive install --force-resolution --no-recommends"
+
+$zyp \
+	asciidoc \
 	autoconf \
 	automake \
 	clang \
@@ -23,4 +26,7 @@ zypper --non-interactive install --force-resolution --no-recommends \
 	libtirpc-devel \
 	linux-glibc-devel \
 	lsb-release \
+	perl-JSON \
 	pkg-config
+
+$zyp ruby2.7-rubygem-asciidoctor || $zyp ruby2.5-rubygem-asciidoctor
Signature · 1 line
2.29.1
author
#17 lore raw

PATCH v2 10/11 docparse: Add configure options

In reply to Petr Vorel

We build HTML and PDF docs with:
- asciidoctor (the default as it produces better output, any version)
- asciidoc (required >= 9)

For asciidoctor we support only native asciidoctor-pdf plugin (we don't support DocBook, Apache FO, or LaTeX intermediary formats).

The goal is to have smart default: try to generate HTML (or also PDF if enabled) docs if suitable generator available.

But allow user to chose:
1) disable whole metadata generation (--disable-metadata)
2) disable metadata HTML generation (--disable-metadata-html)
3) enable metadata PDF generation (--enable-metadata-pdf)
4) generator (--with-metadata-generator=asciidoc|asciidoctor)

Used 2 ax_* (third party) macros from autoconf-archive.

---
 Makefile                   |   6 +-
 configure.ac               |  30 ++++++-
 include/mk/features.mk.in  |   5 ++
 m4/ax_compare_version.m4   | 177 +++++++++++++++++++++++++++++++++++++
 m4/ax_prog_perl_modules.m4 |  77 ++++++++++++++++
 m4/ltp-docparse.m4         | 112 +++++++++++++++++++++++
 6 files changed, 405 insertions(+), 2 deletions(-)
 create mode 100644 m4/ax_compare_version.m4
 create mode 100644 m4/ax_prog_perl_modules.m4
 create mode 100644 m4/ltp-docparse.m4
Makefile +5 -1
index 3830fb6d4..56812d77b 100644
@@ -62,7 +62,11 @@ $(1):: | $$(abs_top_builddir)/$$(basename $$(subst -,.,$(1)))
 endif
 endef
 
-COMMON_TARGETS		+= testcases tools docparse
+COMMON_TARGETS		+= testcases tools
+ifeq ($(WITH_METADATA),yes)
+COMMON_TARGETS		+= docparse
+endif
+
 # Don't want to nuke the original files if we're installing in-build-tree.
 ifneq ($(BUILD_TREE_STATE),$(BUILD_TREE_SRCDIR_INSTALL))
 INSTALL_TARGETS		+= runtest scenario_groups testscripts
configure.ac +29 -1
index 05672f8f6..06be1c094 100644
@@ -189,7 +189,7 @@ AC_CHECK_TYPES([struct xt_entry_match, struct xt_entry_target],,,[
 
 # Tools knobs
 
-# Expect
+# Bash
 AC_ARG_WITH([bash],
   [AC_HELP_STRING([--with-bash],
     [have the Bourne Again Shell interpreter])],
@@ -202,6 +202,34 @@ else
     AC_SUBST([WITH_BASH],["no"])
 fi
 
+# metadata
+AC_ARG_ENABLE([metadata],
+  [AC_HELP_STRING([--disable-metadata],
+	[Disable metadata generation (both HTML and PDF, default no)])],
+  [], [enable_metadata=yes]
+)
+AC_ARG_ENABLE([metadata_html],
+  [AC_HELP_STRING([--disable-metadata-html],
+	[Disable metadata HTML generation (default no)])],
+  [], [enable_metadata_html=yes]
+)
+
+AC_ARG_ENABLE([metadata_pdf],
+  [AC_HELP_STRING([--enable-metadata-pdf],
+	[Enable metadata PDF generation (default no)])],
+  [], [enable_metadata_pdf=no]
+)
+
+AC_ARG_WITH([metadata_generator],
+  [AC_HELP_STRING([--with-metadata-generator=asciidoc|asciidoctor],
+	[Specify metadata generator to use (default autodetect)])],
+  [with_metadata_generator=$withval],
+  [with_metadata_generator=detect]
+)
+
+LTP_DOCPARSE
+
+# Expect
 AC_ARG_WITH([expect],
   [AC_HELP_STRING([--with-expect],
     [have the Tcl/expect library])],
include/mk/features.mk.in +5 -0
index 8e561b738..ecb15a0f7 100644
@@ -27,6 +27,11 @@ WITH_PERL			:= @WITH_PERL@
 
 WITH_PYTHON			:= @WITH_PYTHON@
 
+METADATA_GENERATOR		:= @METADATA_GENERATOR@
+WITH_METADATA			:= @WITH_METADATA@
+WITH_METADATA_HTML		:= @WITH_METADATA_HTML@
+WITH_METADATA_PDF		:= @WITH_METADATA_PDF@
+
 # Features knobs
 
 # Test suite knobs
m4/ax_compare_version.m4 new file +177 -0

177 lines not on this page · raw message

m4/ax_prog_perl_modules.m4 new file +77 -0

77 lines not on this page · raw message

m4/ltp-docparse.m4 new file +112 -0

112 lines not on this page · raw message

Signature · 1 line
2.29.1
author
#18 lore raw

PATCH v2 11/11 docparse: Generate html and pdf using asciidoc{, tor}

In reply to Petr Vorel

Rewrite testinfo.pl to generate *.txt pages in asciidoc format which is then regenerated to html (and pdf if enabled) using asciidoc,{tor}.

Replace getting Linux kernel git commit messages from local git repository (needed after having all tests in single page, because API has access limits; it's also better to generate everything once thus don't depend on network connection).

---
 docparse/.gitignore  |   5 +
 docparse/Makefile    |  58 +++++++
 docparse/testinfo.pl | 406 +++++++++++++++++++++++++++++++++++++++++--
 3 files changed, 458 insertions(+), 11 deletions(-)
docparse/.gitignore +5 -0
index f636ed847..7a87b4234 100644
@@ -1,2 +1,7 @@
+/*.txt
+/docbook-xsl.css
 /docparse
 /metadata.json
+/metadata.html
+/metadata.pdf
+/metadata.chunked/
docparse/Makefile +58 -0

77 lines not on this page · raw message

docparse/testinfo.pl +395 -11

434 lines not on this page · raw message

Signature · 1 line
2.29.1
author
#19 lore raw

v2 cover Test metadata extraction

In reply to Petr Vorel

Hi,

Quoted 28 lines

address some of the notes added by Li.

changes v1->v2:

3rd commit
https://patchwork.ozlabs.org/project/ltp/patch/[email protected]/
* add buf[i++] = c; to fix parsing ""
* check fname is valid before opening it
* fix some of checkpatch.pl problems

9th commit
* add perl-libwww-perl for fedora/centos

11 th commit
* add *.css *.js to CLEAN_TARGETS

TODO
* I didn't filter "options". But agree they should be handled better than now.
* some checkpatch.pl warnings left, do we want to bother?
docparse/docparse.c:53: WARNING: Missing a blank line after declarations docparse/docparse.c:206: ERROR: do not use assignment in if condition docparse/docparse.c:211: WARNING: Missing a blank line after declarations docparse/docparse.c:246: ERROR: do not use assignment in if condition docparse/docparse.c:288: WARNING: Missing a blank line after declarations docparse/docparse.c:297: WARNING: static const char * array should probably be static const char * const docparse/docparse.c:352: WARNING: static const char * array should probably be static const char * const
total: 2 errors, 5 warnings, 423 lines checked

Quoted 4 lines

* I didn't change docparse/README.md (4th commit, Jan had some notes, could you phrase what should be there?)

Anything else to change?

Anybody against pushing whole patchset [1] with fix below for third commit ("docparse: Add test documentation parser").

Kind regards,
Petr

[1] https://patchwork.ozlabs.org/project/ltp/list/?series=211956&state=*

Fix for third commit:

docparse.c +2 -4
index 63d131c87..be6125353 100644
@@ -138,10 +138,8 @@ const char *next_token(FILE *f, struct data_node *doc)
 
 		if (in_str) {
 			if (c == '"') {
-				if (i == 0 || buf[i-1] != '\\') {
-					buf[i++] = c;
+				if (i == 0 || buf[i-1] != '\\')
 					goto exit;
-				}
 			}
 
 			buf[i++] = c;
@@ -189,7 +187,7 @@ const char *next_token(FILE *f, struct data_node *doc)
 	}
 
 exit:
-	if (i == 0)
+	if (i == 0 && !in_str)
 		return NULL;
 
 	buf[i] = 0;
author
#20 lore raw

v2 cover Test metadata extraction

Hi Li,

...

Quoted 2 lines

Anybody against pushing whole patchset [1] with fix below for third commit ("docparse: Add test documentation parser").

Going to merge whole patchset with diff below
(https://github.com/pevik/ltp/commits/test-metadata-extraction.v2.fixes).

Can I add you Reviewed-by: tag to whole patchset or a subset?

Kind regards,
Petr

Quoted 32 lines

Kind regards,
Petr

[1] https://patchwork.ozlabs.org/project/ltp/list/?series=211956&state=*

Fix for third commit:

docparse.c +2 -4
index 63d131c87..be6125353 100644
@@ -138,10 +138,8 @@ const char *next_token(FILE *f, struct data_node *doc)
 
 		if (in_str) {
 			if (c == '"') {
-				if (i == 0 || buf[i-1] != '\\') {
-					buf[i++] = c;
+				if (i == 0 || buf[i-1] != '\\')
 					goto exit;
-				}
 			}
 
 			buf[i++] = c;
@@ -189,7 +187,7 @@ const char *next_token(FILE *f, struct data_node *doc)
 	}
 
 exit:
-	if (i == 0)
+	if (i == 0 && !in_str)
 		return NULL;
 
 	buf[i] = 0;
#21 lore raw

v2 cover Test metadata extraction

Hi Petr,

Quoted 12 lines · Petr Vorel

Hi Li,

...

Quoted 1 line

Anybody against pushing whole patchset [1] with fix below for third

commit

Quoted 1 line

("docparse: Add test documentation parser").

Going to merge whole patchset with diff below
(https://github.com/pevik/ltp/commits/test-metadata-extraction.v2.fixes).

Can I add you Reviewed-by: tag to whole patchset or a subset?

For the whole patchset:

Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20201116/fd6b3c28/attachment.htm>
author
#22 lore raw

v2 cover Test metadata extraction

Hi Li,

Quoted 11 lines

Hi Petr,

Quoted 6 lines · Petr Vorel

Hi Li,

...

Quoted 1 line

Anybody against pushing whole patchset [1] with fix below for third

commit

Quoted 1 line

("docparse: Add test documentation parser").

Quoted 8 lines
Quoted 4 lines

Going to merge whole patchset with diff below
(https://github.com/pevik/ltp/commits/test-metadata-extraction.v2.fixes).

Can I add you Reviewed-by: tag to whole patchset or a subset?

For the whole patchset:

Thanks a lot, merged!

Kind regards,
Petr