Skip to contentSkip to conversation

Test metadata extraction

PATCH New Cyril Hrubis 4 participants 21 messages Oct 5, 2020 – Oct 23, 2020 Kbuild

lore
Share & tools
lore
#0 lore raw

cover Test metadata extraction

This patchset adds a test metadata extraction into LTP and also documentation generator that produces browseable HTML documentation from the exported metadata. For detailed description of the idea and implementation see the patch that adds README.md.

While the idea is quite new the code is mature enough to be included in the upstream repository and I'm also worried that we will not get any feedback or users of the metadata unless it's included in the upstream git.

The next step is to use the extracted metadata in runltp-ng in the proof of concept parallel executor that has been written by Ritchie and posted to this mailing list as well.

Cyril Hrubis (4):
docparse: Add test documentation parser
docparse: Add README
syscalls: Add a few documentation comments
syscalls: Move needs_drivers inside of the tst_test struct

Petr Vorel (7):
make: Support compiling native build tools
travis: Add git
make: Allow {INSTALL,MAKE}_TARGETS be a directory
make: Allow CLEAN_TARGETS to remove directories
travis: Install docparse dependencies
docparse: Add configure options
docparse: Generate html and pdf using asciidoc{,tor}

 Makefile                                      |   4 +
 configure.ac                                  |  32 +-
 docparse/.gitignore                           |   7 +
 docparse/Makefile                             |  77 ++++
 docparse/README.md                            | 248 ++++++++++
 docparse/data_storage.h                       | 299 ++++++++++++
 docparse/docparse.c                           | 415 +++++++++++++++++
 docparse/parse.sh                             |  41 ++
 docparse/testinfo.pl                          | 424 ++++++++++++++++++
 include/mk/config.mk.in                       |  21 +
 include/mk/env_post.mk                        |   3 +-
 include/mk/features.mk.in                     |   5 +
 include/mk/functions.mk                       |   3 +-
 include/mk/generic_leaf_target.inc            |  16 +-
 include/mk/rules.mk                           |   8 +
 m4/ax_compare_version.m4                      | 177 ++++++++
 m4/ax_prog_perl_modules.m4                    |  77 ++++
 m4/ltp-docparse.m4                            | 112 +++++
 testcases/kernel/syscalls/abort/abort01.c     |  16 +-
 testcases/kernel/syscalls/accept/accept01.c   |   8 +-
 testcases/kernel/syscalls/accept/accept02.c   |   7 +-
 testcases/kernel/syscalls/acct/acct01.c       |   5 +
 testcases/kernel/syscalls/acct/acct02.c       |   6 +-
 .../kernel/syscalls/fsetxattr/fsetxattr02.c   |  10 +-
 testcases/kernel/syscalls/ioctl/ioctl08.c     |   9 +-
 travis/alpine.sh                              |   4 +
 travis/debian.minimal.sh                      |   8 +-
 travis/debian.sh                              |  10 +-
 travis/fedora.sh                              |  12 +-
 travis/tumbleweed.sh                          |   9 +-
 30 files changed, 2034 insertions(+), 39 deletions(-)
 create mode 100644 docparse/.gitignore
 create mode 100644 docparse/Makefile
 create mode 100644 docparse/README.md
 create mode 100644 docparse/data_storage.h
 create mode 100644 docparse/docparse.c
 create mode 100755 docparse/parse.sh
 create mode 100755 docparse/testinfo.pl
 create mode 100644 m4/ax_compare_version.m4
 create mode 100644 m4/ax_prog_perl_modules.m4
 create mode 100644 m4/ltp-docparse.m4
Signature · 1 line
2.26.2
#1 lore raw

PATCH 1/11 make: Support compiling native build tools

From: Petr Vorel <[email protected]>

Add HOST_MAKE_TARGETS make target and HOSTCC,
HOST_{CFLAGS,LDFLAGS} make variables.

Needed for cross-compilation.

NOTE: detect cross compilation with comparing $(build) and $(host) instead of using $cross_compiling configure variable, which would require move the detection into m4 macro.

---
 configure.ac                       |  2 ++
 include/mk/config.mk.in            | 21 +++++++++++++++++++++
 include/mk/env_post.mk             |  3 +--
 include/mk/generic_leaf_target.inc |  6 ++++++
 include/mk/rules.mk                |  8 ++++++++
 5 files changed, 38 insertions(+), 2 deletions(-)
configure.ac +2 -0
index 03e4e09c9..05672f8f6 100644
@@ -13,6 +13,8 @@ AC_CONFIG_FILES([ \
     execltp \
 ])
 
+AC_ARG_VAR(HOSTCC, [The C compiler on the host])
+
 AM_MAINTAINER_MODE([enable])
 
 AC_CANONICAL_HOST
include/mk/config.mk.in +21 -0
index 427608a17..8f73a6a34 100644
@@ -31,6 +31,19 @@ RANLIB			:= @RANLIB@
 STRIP			:= @STRIP@
 YACC			:= @YACC@
 
+HOSTCC  = @HOSTCC@
+build := @build@
+host := @host@
+ifeq ($(strip $(HOSTCC)),)
+# native build, respect CC
+ifeq ($(build),$(host))
+HOSTCC := $(CC)
+else
+# cross compilation
+HOSTCC := cc
+endif
+endif
+
 AIO_LIBS		:= @AIO_LIBS@
 CAP_LIBS		:= @CAP_LIBS@
 ACL_LIBS		:= @ACL_LIBS@
@@ -70,6 +83,14 @@ WCFLAGS			?= -Wall -W @GCC_WARN_OLDSTYLE@
 LDFLAGS			+= $(WLDFLAGS)
 CFLAGS			+= $(DEBUG_CFLAGS) $(OPT_CFLAGS) $(WCFLAGS)
 
+ifeq ($(strip $(HOST_CFLAGS)),)
+HOST_CFLAGS := $(CFLAGS)
+endif
+
+ifeq ($(strip $(HOST_LDFLAGS)),)
+HOST_LDFLAGS := $(LDFLAGS)
+endif
+
 LINUX_VERSION		:= @LINUX_VERSION@
 LINUX_DIR		:= @LINUX_DIR@
 LINUX_VERSION_MAJOR	:= @LINUX_VERSION_MAJOR@
include/mk/env_post.mk +1 -2
index 44a333198..d52ad9f0b 100644
@@ -48,11 +48,10 @@ LDFLAGS				+= -L$(top_builddir)/lib/android_librt
 endif
 
 MAKE_TARGETS			?= $(notdir $(patsubst %.c,%,$(wildcard $(abs_srcdir)/*.c)))
-
 MAKE_TARGETS			:= $(filter-out $(FILTER_OUT_MAKE_TARGETS),$(MAKE_TARGETS))
 
 # with only *.dwo, .[0-9]+.dwo can not be cleaned
-CLEAN_TARGETS			+= $(MAKE_TARGETS) *.o *.pyc .cache.mk *.dwo .*.dwo
+CLEAN_TARGETS			+= $(MAKE_TARGETS) $(HOST_MAKE_TARGETS) *.o *.pyc .cache.mk *.dwo .*.dwo
 
 # Majority of the files end up in testcases/bin...
 INSTALL_DIR			?= testcases/bin
include/mk/generic_leaf_target.inc +6 -0
index dd54d05e9..e6fa107d1 100644
@@ -57,6 +57,8 @@
 #				     rope to hang one's self in the event of
 #				     unwanted behavior.
 #
+# $(HOST_MAKE_TARGETS)	: Host tools which use $HOSTCC.
+#
 # $(CLEAN_TARGETS)		: What targets should be cleaned (must be
 #				  real files). This will automatically append
 #				  adds the .o suffix to all files referenced
@@ -92,6 +94,10 @@
 
 .PHONY: all clean install
 
+ifneq ($(strip $(MAKE_TARGETS)),)
+$(MAKE_TARGETS) += $(HOST_MAKE_TARGETS)
+endif
+
 $(MAKE_TARGETS): | $(MAKE_DEPS)
 
 all: $(MAKE_TARGETS)
include/mk/rules.mk +8 -0
index 6a22e43af..c8f4bbbbe 100644
@@ -22,6 +22,14 @@ else
 	@echo LD $(target_rel_dir)$@
 endif
 
+$(HOST_MAKE_TARGETS): %: %.c
+ifdef VERBOSE
+	$(HOSTCC) $(HOST_CFLAGS) $(HOST_LDFLAGS) $< $(HOST_LDLIBS) -o $@
+else
+	@$(HOSTCC) $(HOST_CFLAGS) $(HOST_LDFLAGS) $< $(HOST_LDLIBS) -o $@
+	@echo HOSTCC $(target_rel_dir)$@
+endif
+
 %: %.c
 ifdef VERBOSE
 	$(CC) $(CPPFLAGS) $(CFLAGS) $(LDFLAGS) $^ $(LTPLDLIBS) $(LDLIBS) -o $@
Signature · 1 line
2.26.2
#2 lore raw

PATCH 2/11 travis: Add git

In reply to Cyril Hrubis

From: Petr Vorel <[email protected]>

Needed in parse.sh script in next commit.

---
 travis/alpine.sh     | 1 +
 travis/debian.sh     | 1 +
 travis/fedora.sh     | 1 +
 travis/tumbleweed.sh | 1 +
 4 files changed, 4 insertions(+)
travis/alpine.sh +1 -0
index 61ef144a8..f8960bed0 100755
@@ -10,6 +10,7 @@ apk add \
 	automake \
 	clang \
 	gcc \
+	git \
 	keyutils-dev \
 	libaio-dev \
 	libacl \
travis/debian.sh +1 -0
index b759a9576..28685f4d3 100755
@@ -17,6 +17,7 @@ apt install -y --no-install-recommends \
 	devscripts \
 	clang \
 	gcc \
+	git \
 	libacl1 \
 	libacl1-dev \
 	libaio-dev \
travis/fedora.sh +1 -0
index 990a84daf..3c224f71e 100755
@@ -8,6 +8,7 @@ yum -y install \
 	make \
 	clang \
 	gcc \
+	git \
 	findutils \
 	libtirpc \
 	libtirpc-devel \
travis/tumbleweed.sh +1 -0
index 4d5e9da79..6247daa98 100755
@@ -8,6 +8,7 @@ zypper --non-interactive install --force-resolution --no-recommends \
 	clang \
 	findutils \
 	gcc \
+	git \
 	gzip \
 	make \
 	kernel-default-devel \
Signature · 1 line
2.26.2
#3 lore raw

PATCH 3/11 docparse: Add test documentation parser

In reply to Cyril Hrubis

From: Cyril Hrubis <[email protected]>

This commit implements a simple parser to pick up metadata that are nicely structured in the tst_test structure. In order to do that it implements very simple and very basic C tokenizer.

The result of the metadata extraction is one big json file that contains data extracted from all new library testcases plus some information about the LTP version etc.

The tokenizer also extracts special documentation comments that are then used for a test description. The documentation format has been choosen to be asciidoc.

Special documentation comments starts with string "/*\\n" and is, by convention ended with the same, so it looks like:

/*\
* [DOCUMENTATION]
* - foo
* - bar
*
* ...
*
\*/

[ pvorel: fixes
- parse.sh: git related fixes, out-of-tree build
- docparse: Eat also space following *
- Makefile: remove header rule (broken for clang)
]

---
 Makefile                |   2 +-
 docparse/.gitignore     |   2 +
 docparse/Makefile       |  19 ++
 docparse/data_storage.h | 299 +++++++++++++++++++++++++++++
 docparse/docparse.c     | 415 ++++++++++++++++++++++++++++++++++++++++
 docparse/parse.sh       |  41 ++++
 docparse/testinfo.pl    |  40 ++++
 7 files changed, 817 insertions(+), 1 deletion(-)
 create mode 100644 docparse/.gitignore
 create mode 100644 docparse/Makefile
 create mode 100644 docparse/data_storage.h
 create mode 100644 docparse/docparse.c
 create mode 100755 docparse/parse.sh
 create mode 100755 docparse/testinfo.pl
Makefile +1 -1
index bf5077231..3830fb6d4 100644
@@ -62,7 +62,7 @@ $(1):: | $$(abs_top_builddir)/$$(basename $$(subst -,.,$(1)))
 endif
 endef
 
-COMMON_TARGETS		+= testcases tools
+COMMON_TARGETS		+= testcases tools docparse
 # Don't want to nuke the original files if we're installing in-build-tree.
 ifneq ($(BUILD_TREE_STATE),$(BUILD_TREE_SRCDIR_INSTALL))
 INSTALL_TARGETS		+= runtest scenario_groups testscripts
docparse/.gitignore new file +2 -0
new file mode 100644
index 000000000..f636ed847
@@ -0,0 +1,2 @@
+/docparse
+/metadata.json
docparse/Makefile new file +19 -0
new file mode 100644
index 000000000..94ba83ffe
@@ -0,0 +1,19 @@
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2019 Cyril Hrubis <[email protected]>
+
+top_srcdir		?= ..
+
+include $(top_srcdir)/include/mk/env_pre.mk
+include $(top_srcdir)/include/mk/functions.mk
+
+MAKE_TARGETS		:= metadata.json
+HOST_MAKE_TARGETS	:= docparse
+
+INSTALL_DIR = metadata
+
+.PHONY: metadata.json
+
+metadata.json: docparse
+	$(abs_srcdir)/parse.sh > metadata.json
+
+include $(top_srcdir)/include/mk/generic_leaf_target.mk
docparse/data_storage.h new file +299 -0

299 lines not on this page · raw message

docparse/docparse.c new file +415 -0

415 lines not on this page · raw message

docparse/parse.sh new file +41 -0

41 lines not on this page · raw message

docparse/testinfo.pl new file +40 -0
new file mode 100755
index 000000000..d93d7d701
@@ -0,0 +1,40 @@
+#!/usr/bin/perl
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2019 Cyril Hrubis <[email protected]>
+
+use strict;
+use warnings;
+
+use JSON;
+use Data::Dumper;
+
+sub load_json
+{
+	my ($fname) = @_;
+	local $/;
+
+	open(my $fh, '<', $fname) or die("Can't open $fname $!");
+
+	return <$fh>;
+}
+
+sub query_flag
+{
+	my ($json, $flag) = @_;
+
+	my $tests = $json->{'tests'};
+
+	foreach my $key (sort(keys %$tests)) {
+		if ($tests->{$key}->{$flag}) {
+			if ($tests->{$key}->{$flag} eq "1") {
+				print("$key\n");
+			} else {
+				print("$key:\n" . Dumper($tests->{$key}->{$flag}) . "\n");
+			}
+		}
+	}
+}
+
+my $json = decode_json(load_json($ARGV[0]));
+
+query_flag($json, $ARGV[1]);
Signature · 1 line
2.26.2
#4 lore raw

PATCH 3/11 docparse: Add test documentation parser

Quoted 8 lines · Cyril Hrubis
docparse/docparse.c new file +8 -0
+const char *next_token(FILE *f, struct data_node *doc)
+{
+       size_t i = 0;
+       static char buf[4096];
+       int c;
+       int in_str = 0;
+
+       for (;;) {

docparse/docparse.c line 142

Quoted 10 lines
docparse/docparse.c new file +10 -0
+               c = fgetc(f);
+
+               if (c == EOF)
+                       goto exit;
+
+               if (in_str) {
+                       if (c == '"') {
+                               if (i == 0 || buf[i-1] != '\\')
+                                       goto exit;
+                       }

There is a problem in handle a special string token here, which can not parse the "" correctly in many test cases.

e.g.

# ./docparse ../testcases/kernel/syscalls/fsopen/fsopen01.c
# ./docparse ../testcases/kernel/fs/ftest/ftest02.c
....

We got nothing output from the above two tests parsing because they contains "" in their sentence, it makes next_token() exit too early.

TEST(move_mount(fsmfd, "", AT_FDCWD, MNTPOINT,
MOVE_MOUNT_F_EMPTY_PATH));

Quoted 96 lines
docparse/docparse.c new file +96 -0
+
+                       buf[i++] = c;
+                       continue;
+               }
+
+               switch (c) {
+               case '{':
+               case '}':
+               case ';':
+               case '(':
+               case ')':
+               case '=':
+               case ',':
+               case '[':
+               case ']':
+                       if (i) {
+                               ungetc(c, f);
+                               goto exit;
+                       }
+
+                       buf[i++]=c;
+                       goto exit;
+               case '0' ... '9':
+               case 'a' ... 'z':
+               case 'A' ... 'Z':
+               case '.':
+               case '_':
+               case '-':
+                       buf[i++]=c;
+               break;
+               case '/':
+                       maybe_comment(f, doc);
+               break;
+               case '"':
+                       in_str = 1;
+               break;
+               case ' ':
+               case '\n':
+               case '\t':
+                       if (i)
+                               goto exit;
+               break;
+               }
+       }
+
+exit:
+       if (i == 0)
+               return NULL;
+
+       buf[i] = 0;
+       return buf;
+}
+
+#define WARN(str) fprintf(stderr, str "\n")
+
+static int parse_array(FILE *f, struct data_node *node)
+{
+       const char *token;
+
+       for (;;) {
+               if (!(token = next_token(f, NULL)))
+                       return 1;
+
+               if (!strcmp(token, "{")) {
+                       struct data_node *ret = data_node_array();
+                       parse_array(f, ret);
+
+                       if (data_node_array_len(ret))
+                               data_node_array_add(node, ret);
+                       else
+                               data_node_free(ret);
+
+                       continue;
+               }
+
+               if (!strcmp(token, "}"))
+                       return 0;
+
+               if (!strcmp(token, ","))
+                       continue;
+
+               if (!strcmp(token, "NULL"))
+                       continue;
+
+               struct data_node *str = data_node_string(token);
+
+               data_node_array_add(node, str);
+       }
+
+       return 0;
+}
+
+static const char *tokens[] = {
+       "static",
+       "struct",
+       "tst_test",

docparse/docparse.c line 307

Quoted 10 lines
docparse/docparse.c new file +10 -0
+       "test",
+       "=",
+       "{",
+};
+
+static struct data_node *parse_file(const char *fname)
+{
+       int state = 0, found = 0;
+       const char *token;
+

Seems we'd better check the fname is valid before opening it.

       if (access(fname, F_OK)) {
               fprintf(stderr, "file %s is not exist\n", fname);
               return NULL;
       }

docparse/docparse.c line 351

Quoted 8 lines
docparse/docparse.c new file +8 -0
+static const char *filter_out[] = {
+       "test",
+       "test_all",
+       "setup",
+       "cleanup",
+       "tcnt",
+       "mntpoint",
+       "bufs",

I guess the "options" should be also filtered out here?

--
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20201023/483b507f/attachment-0001.htm>

#5 lore raw

PATCH 3/11 docparse: Add test documentation parser

Quoted 25 lines · Li Wang
Quoted 8 lines · Cyril Hrubis
docparse/docparse.c new file +8 -0
+const char *next_token(FILE *f, struct data_node *doc)
+{
+       size_t i = 0;
+       static char buf[4096];
+       int c;
+       int in_str = 0;
+
+       for (;;) {

docparse/docparse.c line 142

Quoted 10 lines
docparse/docparse.c new file +10 -0
+               c = fgetc(f);
+
+               if (c == EOF)
+                       goto exit;
+
+               if (in_str) {
+                       if (c == '"') {
+                               if (i == 0 || buf[i-1] != '\\')
+                                       goto exit;
+                       }

There is a problem in handle a special string token here, which can not parse the "" correctly in many test cases.

Quoted 5 lines

e.g.

# ./docparse ../testcases/kernel/syscalls/fsopen/fsopen01.c
# ./docparse ../testcases/kernel/fs/ftest/ftest02.c

The ftest02.c has not been converted to new API, please ignore.

To fix this problem I propose to add a simple line as:

@@ -137,8 +138,10 @@ const char *next_token(FILE *f, struct data_node *doc)

                if (in_str) {
                        if (c == '"') {
-                               if (i == 0 || buf[i-1] != '\\')
+                               if (i == 0 || buf[i-1] != '\\') {
+                                       buf[i++] = c;
                                        goto exit;
+                               }
                        }
Quoted 8 lines

....

We got nothing output from the above two tests parsing because they contains "" in their sentence, it makes next_token() exit too early.

TEST(move_mount(fsmfd, "", AT_FDCWD, MNTPOINT,
MOVE_MOUNT_F_EMPTY_PATH));

Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20201023/6111a36b/attachment.htm>
#6 lore raw

PATCH 4/11 docparse: Add README

In reply to Cyril Hrubis

From: Cyril Hrubis <[email protected]>

* example of C source and JSON
* note about exporting timeouts

[ Tim: fix typos and clean up grammar ]
---
 docparse/README.md | 248 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 248 insertions(+)
 create mode 100644 docparse/README.md
docparse/README.md new file +248 -0
new file mode 100644
index 000000000..7e4847ba2
@@ -0,0 +1,248 @@
+Motivation for metadata exctraction
+===================================
+
+Exporting documentation
+-----------------------
+
+This allow us to build browseable documentation for the testcases, e.g. a
+catalogue of test information that would be searchable etc. At this point there
+is a single page generated from the extracted data that tries to outline the
+intent.
+
+
+Propagating test requirements
+-----------------------------
+
+Some subtests require differnt hardware resources/software versions/etc. the
+test execution framework needs to consume these so that it can locate proper
+hardware, install proper software, etc.
+
+Some examples of requriments are:
+
+* Test needs at least 1GB of RAM.
+
+* Test needs a block device at least 512MB in size
+
+* Test needs a NUMA machine with two memory nodes and at least 300 free pages on each node
+
+* Test needs i2c eeprom connected on a i2c bus
+
+* Test needs two serial ports connected via null-modem cable
+
+
+With this information extracted from the tests the testrunner can then map the
+requiremnts on the available machines in a lab and select a proper machine for
+the particular (sub)set of testcases as well as supply a particular test with
+additional information needed for the test, such as address of the i2c device,
+paths to the serial devices, etc. In the case of virtual machines the test could
+also dynamically prepare the correct environment for the test on demand.
+
+
+Parallel test execution
+-----------------------
+
+An LTP testrun on a modern hardware wastes most of the machine resources
+because the testcases are running sequentially. However in order to execute
+tests in parallel we need to know which system resources are utilized by a
+given test, as obviously we cannot run two tests that monopolize the same
+resource. In some cases we would also need to partition the system resource
+accordingly, e.g. if we have two memory stress tests running at the same time
+we will need to cap each of these tests on half of the available memory, or
+make sure that sum of the memory used by these two tests is not greater an
+available memory.
+
+Examples of such tests are:
+
+* Tests that mess with global system state
+   - system time (e.g. settimeofday() test and leap second test)
+   - SysV SHM
+   - ...
+
+* Tests that use block device
+
+* Tests that work with a particular hardware resource
+  - i2c eeprom test
+  - serial port tests
+  - ...
+
+Exporting test runtime/timeout to the testrunner
+------------------------------------------------
+
+Currently most of the testrunners usually do not know for how long is the test
+supposed to run, this means that we have to guess some upper limit on how long
+a test is supposed to run. The value is usually twice of the maximal runtime
+for all testcases or whole suite or even larger. This means that we are wasting
+time in the case that the test ends up stuck and we could have failed it much
+sooner in most of the cases. This becomes quite important for a kernel
+regression tests that do crash the host, if the information that the test is
+supposed to crash a kernel under a minute is exported to the testrunner we can
+reboot the machine much faster in an event of a crash.
+
+Getting rid of runtest files
+----------------------------
+
+This would also allow us to get rid of the unflexible and hard to maintain
+runtest files. Once this system is in place we will have a list of all tests
+along with their respective metadata - which means that we will be able to
+generate subsets of the test easily on the fly.
+
+In order to achieve this we need two things:
+
+Each test will describe which syscall/functionality it tests in the metadata.
+Then we could define groups of tests based on that. I.e. instead of having
+syscall runtest file we would ask the testrunner to run all test that have a
+defined which syscall they test, or whose filename matches a particular syscall name.
+
+Secondly we will have to store the test variants in the test metadata instead
+of putting them in a file that is unrelated to the test.
+
+For example:
+
+* To run CVE related test we would select testcases with CVE tag
+
+* To run IPC test we will define a list of IPC syscalls and run all syscall
+  test that are in the list
+
+* And many more...
+
+
+Implementation
+==============
+
+The docparser is implemented as a minimal C tokenizer that can parse and
+extract code comments and C structures. The docparser then runs over all C
+sources in the testcases directory and if tst\_test structure is present in the
+source it's parsed and the result is included in the resulting metadata.
+
+During parsing the metadata is stored in a simple key/value storage that more
+or less follows C structure layout, i.e. can include hash, array, and string.
+Once the parsing is finished the result is filtered so that only interesting
+fields of the tst\_test structure are included and then converted into JSON
+output.
+
+This process produces one big JSON file with metadata for all tests, that
+is then installed along with the testcases. This would then be used by the
+testrunner.
+
+The test requirements are stored in the tst\_test structure either as
+bitflags, integers or arrays of strings:
+
+```c
+struct tst_test test = {
+	...
+	/* tests needs to run with UID=1 */
+	.needs_root = 1,
+
+	/*
+	 * Tests needs a block device at least 1024MB in size and also
+	 * mkfs.ext4 installed.
+	 */
+	.needs_device = 1,
+	.dev_min_size = 1024,
+	.dev_fs_type = ext4,
+
+	/* Indicates that the test is messing with system wall clock */
+	.restore_wallclock = 1,
+
+	/* Tests needs uinput either compiled in or loaded as a module */
+	.needs_drivers = (const char *[]) {
+		"uinput",
+		NULL
+	},
+
+	/* Tests needs enabled kernel config flags */
+	.needs_kconfigs = (const char *[]) {
+		"CONFIG_X86_INTEL_UMIP=y",
+		NULL
+	},
+
+	/* Additional array of key value pairs */
+	.tags = (const struct tst_tag[]) {
+                {"linux-git", "43a6684519ab"},
+                {"CVE", "2017-2671"},
+                {NULL, NULL}
+        }
+};
+```
+
+The test documentation is stored in a special comment such as:
+
+```
+/*\
+ * Test description
+ *
+ * This is a test description.
+ * Consisting of several lines.
+\*/
+```
+
+Which will yield following json output:
+
+```json
+ "testcaseXY": {
+  "needs_root": "1",
+  "needs_device": "1",
+  "dev_min_size": "1024",
+  "dev_fs_type": "ext4",
+  "restore_wallclock": "1",
+  "needs_drivers": [
+    "uinput",
+  ],
+  "needs_kconfigs": [
+    "CONFIG_X86_INTEL_UMIP=y",
+  ],
+  "tags": [
+    [
+     "linux-git",
+     "43a6684519ab"
+    ],
+    [
+     "CVE",
+     "2017-2671"
+    ],
+   ],
+  "doc": [
+    "Test description",
+    "",
+    "This is a test description.",
+    "Consisting of several lines."
+  ],
+  "fname": "testcases/kernel/syscalls/foo/testcaseXY.c"
+ },
+```
+
+The final JSON file is JSON object of test descriptions indexed by a test name
+with a header describing the testsuite:
+
+```json
+{
+ "testsuite": "Linux Test Project",
+ "testsuite_short": "LTP",
+ "url": "https://github.com/linux-test-project/ltp/",
+ "scm_url_base": "https://github.com/linux-test-project/ltp/tree/master/",
+ "timeout": 300,
+ "version": "20200930",
+ "tests": {
+  "testcaseXY": {
+   ...
+  },
+  ...
+ }
+}
+```
+
+Open Points
+===========
+
+There are stil some loose ends. Mostly it's not well defined where to put
+things and how to format them.
+
+* Some of the hardware requirements are already listed in the tst\_test. Should
+  we put all of them there?
+
+* What would be the format for test documentation and how to store things such
+  as test variants there?
+
+So far this proof of concept generates a metadata file. I guess that we need
+actual consumers which will help to settle things down, I will try to look into
+making use of this in the runltp-ng at least as a reference implementation.
Signature · 1 line
2.26.2
#7 lore raw

PATCH 4/11 docparse: Add README

----- Original Message -----

Quoted 11 lines
docparse/README.md new file +7 -0
+Open Points
+===========
+
+There are stil some loose ends. Mostly it's not well defined where to put
+things and how to format them.
+
+* Some of the hardware requirements are already listed in the tst\_test.

Should

docparse/README.md new file +3 -0
+  we put all of them there?
+
+* What would be the format for test documentation and how to store things
Quoted 2 lines

such
+ as test variants there?

I'm assuming you don't mean ".test_variants" here, but runtest entries using same binary with different parameters. Currently we have a "tag" that can refer to binary+parameters pair. Which is also useful, for skipping tests - e.g. they run long, they are broken, or older kernel is known to crash - we have a list of checks that modify runtest files before actual run to avoid running into known issues (or save time).

Quoted 13 lines
docparse/README.md new file +3 -0
+
+So far this proof of concept generates a metadata file. I guess that we need
+actual consumers which will help to settle things down, I will try to look

into
+making use of this in the runltp-ng at least as a reference implementation. --
2.26.2

--
Mailing list info: https://lists.linux.it/listinfo/ltp

#8 lore raw

PATCH 4/11 docparse: Add README

Hi!

Quoted 12 lines
Quoted 12 lines
docparse/README.md new file +7 -0
+Open Points
+===========
+
+There are stil some loose ends. Mostly it's not well defined where to put
+things and how to format them.
+
+* Some of the hardware requirements are already listed in the tst\_test.

Should

docparse/README.md new file +3 -0
+  we put all of them there?
+
+* What would be the format for test documentation and how to store things

such

Quoted 8 lines
Quoted 1 line

+ as test variants there?

I'm assuming you don't mean ".test_variants" here, but runtest entries using same binary with different parameters. Currently we have a "tag" that can refer to binary+parameters pair. Which is also useful, for skipping tests - e.g. they run long, they are broken, or older kernel is known to crash - we have a list of checks that modify runtest files before actual run to avoid running into known issues (or save time).

Yes I do mean binary parameters here. And yes we will need a way how to express skiplist for this as well.

Signature · 2 lines
Cyril Hrubis
[email protected]
#9 lore raw

PATCH 5/11 syscalls: Add a few documentation comments

In reply to Cyril Hrubis

From: Cyril Hrubis <[email protected]>

So that it shows up in the resulting json file.

---
 testcases/kernel/syscalls/abort/abort01.c   | 16 ++++++++++------
 testcases/kernel/syscalls/accept/accept01.c |  8 +++++---
 testcases/kernel/syscalls/accept/accept02.c |  7 +++++--
 testcases/kernel/syscalls/acct/acct01.c     |  5 +++++
 testcases/kernel/syscalls/acct/acct02.c     |  6 ++++--
 5 files changed, 29 insertions(+), 13 deletions(-)
testcases/kernel/syscalls/abort/abort01.c +10 -6
index 9505a5eec..b93324b34 100644
@@ -5,14 +5,18 @@
  *   01/02/2003	Port to LTP	[email protected]
  *   11/11/2002: Ported to LTP Suite by Ananda
  *   06/30/2001	Port to Linux	[email protected]
- *
- * ALGORITHM
- *	Fork child.  Have child abort, check return status.
- *
- * RESTRICTIONS
- *      The ulimit for core file size must be greater than 0.
  */
 
+/*\
+ * [DESCRIPTION]
+ *  Checks that process which called abort() gets killed by SIGIOT and dumps core.
+ *
+ * [ALGORITHM]
+ *  - Fork child.
+ *  - Child calls abort.
+ *  - Parent checks return status.
+\*/
+
 #include <sys/types.h>
 #include <sys/wait.h>
 #include <errno.h>
testcases/kernel/syscalls/accept/accept01.c +5 -3
index 4e30906f2..01d6db84c 100644
@@ -3,11 +3,13 @@
 /*
  *   Copyright (c) International Business Machines  Corp., 2001
  *   07/2001 Ported by Wayne Boyer
- *
- *   Description:
- *     Verify that accept() returns the proper errno for various failure cases
  */
 
+/*\
+ * [DESCRIPTION]
+ * Verify that accept() returns the proper errno for various failure cases.
+\*/
+
 #include <stdio.h>
 #include <unistd.h>
 #include <errno.h>
testcases/kernel/syscalls/accept/accept02.c +5 -2
index 37ab8b64f..7fb6a494a 100644
@@ -3,7 +3,10 @@
  * Copyright (c) 2019 SUSE LLC
  * Author: Christian Amann <[email protected]>
  */
-/* Test for CVE-2017-8890
+/*\
+ * [DESCRIPTION]
+ *
+ * Test for CVE-2017-8890
  *
  * In Kernels up to 4.10.15 missing commit 657831ff the multicast
  * group information of a socket gets copied over to a newly created
@@ -16,7 +19,7 @@
  *
  * For more information about this CVE see:
  * https://www.suse.com/security/cve/CVE-2017-8890/
- */
+\*/
 
 #include <errno.h>
 #include <sys/socket.h>
testcases/kernel/syscalls/acct/acct01.c +5 -0
index c161d2a2c..60e81bfad 100644
@@ -7,6 +7,11 @@
 /* 12/03/2002	Port to LTP     [email protected] */
 /* 06/30/2001	Port to Linux	[email protected] */
 
+/*\
+ * [DOCUMENTATION]
+ *  Verify that acct() returns proper errno on failure.
+\*/
+
 #include <sys/types.h>
 #include <sys/stat.h>
 #include <errno.h>
testcases/kernel/syscalls/acct/acct02.c +4 -2
index 8ee1bfcf8..e718e7df4 100644
@@ -3,7 +3,9 @@
  *  Copyright (c) SUSE LLC, 2019
  *  Author: Christian Amann <[email protected]>
  */
-/*
+/*\
+ * [DOCUMENTATION]
+ *
  * This tests if the kernel writes correct data to the
  * process accounting file.
  *
@@ -19,7 +21,7 @@
  *
  * This is also accidental regression test for:
  * 4d9570158b626 kernel/acct.c: fix the acct->needcheck check in check_free_space()
- */
+\*/
 
 #include <sys/stat.h>
 #include <errno.h>
Signature · 1 line
2.26.2
#10 lore raw

PATCH 6/11 syscalls: Move needs_drivers inside of the tst_test struct

In reply to Cyril Hrubis
---
 testcases/kernel/syscalls/fsetxattr/fsetxattr02.c | 10 ++++------
 testcases/kernel/syscalls/ioctl/ioctl08.c         |  9 ++++-----
 2 files changed, 8 insertions(+), 11 deletions(-)
testcases/kernel/syscalls/fsetxattr/fsetxattr02.c +4 -6
index 205e80c95..3aea4b59e 100644
@@ -241,11 +241,6 @@ static void cleanup(void)
 	}
 }
 
-static const char *const needed_drivers[] = {
-	"brd",
-	NULL,
-};
-
 static struct tst_test test = {
 	.setup = setup,
 	.test = verify_fsetxattr,
@@ -254,7 +249,10 @@ static struct tst_test test = {
 	.needs_devfs = 1,
 	.mntpoint = MNTPOINT,
 	.needs_root = 1,
-	.needs_drivers = needed_drivers,
+	.needs_drivers = (const char *const[]) {
+		"brd",
+		NULL,
+	},
 };
 
 #else /* HAVE_SYS_XATTR_H */
testcases/kernel/syscalls/ioctl/ioctl08.c +4 -5
index dca898a65..f7d11815d 100644
@@ -112,10 +112,6 @@ static void setup(void)
 			sizeof(struct file_dedupe_range_info));
 }
 
-static const char *const needed_drivers[] = {
-	"btrfs",
-	NULL,
-};
 
 static struct tst_test test = {
 	.test = verify_ioctl,
@@ -127,7 +123,10 @@ static struct tst_test test = {
 	.mount_device = 1,
 	.mntpoint = MNTPOINT,
 	.dev_fs_type = "btrfs",
-	.needs_drivers = needed_drivers,
+	.needs_drivers = (const char *const[]) {
+		"btrfs",
+		NULL,
+	},
 };
 #else
 	TST_TEST_TCONF(
Signature · 1 line
2.26.2
#11 lore raw

PATCH 7/11 make: Allow {INSTALL, MAKE}_TARGETS be a directory

In reply to Cyril Hrubis

From: Petr Vorel <[email protected]>

by detecting it and adding required -d parameter for install and installing whole directory with -t.

This will be needed for metadata metadata.chunked target.

---
 include/mk/functions.mk | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
include/mk/functions.mk +2 -1
index 79c6193ca..e86dbccdc 100644
@@ -35,7 +35,8 @@ INSTALL_FILES		+= $$(abspath $$(DESTDIR)/$(3)/$(1))
 
 $$(abspath $$(DESTDIR)/$(3)/$(1)): \
     $$(abspath $$(dir $$(DESTDIR)/$(3)/$(1)))
-	install -m $$(INSTALL_MODE) "$(2)/$(1)" "$$@"
+	install -m $$(INSTALL_MODE) $(shell test -d "$(2)/$(1)" && echo "-d") $(PARAM) "$(2)/$(1)" $$@
+	$(shell test -d "$(2)/$(1)" && echo "install -m "'$$(INSTALL_MODE) $(PARAM)' "$(2)/$(1)/*" -t '$$@')
 endef
 
 #
Signature · 1 line
2.26.2
#12 lore raw

PATCH 8/11 make: Allow CLEAN_TARGETS to remove directories

In reply to Cyril Hrubis

From: Petr Vorel <[email protected]>

make clean on leaf target can contain also directories.

Changing delete to recursive delete (-r parameter for rm) should be safe thus detection whether actually requited is not implemented.

This will be needed for metadata metadata.chunked target.

---
 include/mk/generic_leaf_target.inc | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
include/mk/generic_leaf_target.inc +5 -5
index e6fa107d1..64953f89a 100644
@@ -60,10 +60,10 @@
 # $(HOST_MAKE_TARGETS)	: Host tools which use $HOSTCC.
 #
 # $(CLEAN_TARGETS)		: What targets should be cleaned (must be
-#				  real files). This will automatically append
-#				  adds the .o suffix to all files referenced
-#				  by $(MAKE_TARGETS)) to CLEAN_TARGETS, if
-#				  MAKE_TARGETS wasn't defined (see
+#				  real files or directories). This will automatically append
+#				  adds the .o suffix to all files referenced by
+#				  $(MAKE_TARGETS)) to CLEAN_TARGETS, if MAKE_TARGETS wasn't
+#				  defined (see
 #				  $(MAKE_TARGETS)).
 # $(INSTALL_MODE)		: What mode should we using when calling
 # 				  install(1)?
@@ -103,7 +103,7 @@ $(MAKE_TARGETS): | $(MAKE_DEPS)
 all: $(MAKE_TARGETS)
 
 clean:: $(CLEAN_DEPS)
-	-$(RM) -f $(CLEAN_TARGETS)
+	-$(RM) -f -r $(CLEAN_TARGETS)
 
 $(INSTALL_FILES): | $(INSTALL_DEPS)
 
Signature · 1 line
2.26.2
#13 lore raw

PATCH 9/11 travis: Install docparse dependencies

In reply to Cyril Hrubis

From: Petr Vorel <[email protected]>

---
 travis/alpine.sh         |  3 +++
 travis/debian.minimal.sh |  8 +++++++-
 travis/debian.sh         |  9 ++++++++-
 travis/fedora.sh         | 11 ++++++++---
 travis/tumbleweed.sh     |  8 +++++++-
 5 files changed, 33 insertions(+), 6 deletions(-)
travis/alpine.sh +3 -0
index f8960bed0..b793a9fbe 100755
@@ -6,6 +6,8 @@ apk update
 
 apk add \
 	acl-dev \
+	asciidoc \
+	asciidoctor \
 	autoconf \
 	automake \
 	clang \
@@ -23,6 +25,7 @@ apk add \
 	musl-dev \
 	numactl-dev \
 	openssl-dev \
+	perl-json \
 	pkgconfig
 
 cat /etc/os-release
travis/debian.minimal.sh +7 -1
index 3f1941969..5e6ba8662 100755
@@ -2,7 +2,11 @@
 # Copyright (c) 2018-2020 Petr Vorel <[email protected]>
 set -ex
 
-apt remove -y \
+apt="apt remove -y"
+
+$apt \
+	asciidoc \
+	asciidoctor \
 	libacl1-dev \
 	libaio-dev \
 	libaio1 \
@@ -17,3 +21,5 @@ apt remove -y \
 	libsepol1-dev \
 	libssl-dev \
 	libtirpc-dev
+
+$apt asciidoc-base ruby-asciidoctor || true
travis/debian.sh +8 -1
index 28685f4d3..743b79001 100755
@@ -8,8 +8,12 @@ grep -v oldstable-updates /etc/apt/sources.list > /tmp/sources.list && mv /tmp/s
 
 apt update
 
-apt install -y --no-install-recommends \
+apt="apt install -y --no-install-recommends"
+
+$apt \
 	acl-dev \
+	asciidoc \
+	asciidoctor \
 	autoconf \
 	automake \
 	build-essential \
@@ -26,6 +30,7 @@ apt install -y --no-install-recommends \
 	libcap2 \
 	libc6 \
 	libc6-dev \
+	libjson-perl \
 	libkeyutils-dev \
 	libkeyutils1 \
 	libmm-dev \
@@ -40,4 +45,6 @@ apt install -y --no-install-recommends \
 	lsb-release \
 	pkg-config
 
+$apt ruby-asciidoctor-pdf || true
+
 df -hT
travis/fedora.sh +8 -3
index 3c224f71e..6127d367d 100755
@@ -2,7 +2,10 @@
 # Copyright (c) 2018-2020 Petr Vorel <[email protected]>
 set -ex
 
-yum -y install \
+yum="yum -y install"
+
+$yum \
+	asciidoc \
 	autoconf \
 	automake \
 	make \
@@ -12,8 +15,10 @@ yum -y install \
 	findutils \
 	libtirpc \
 	libtirpc-devel \
+	perl-JSON \
 	pkg-config \
 	redhat-lsb-core
 
-# CentOS 8 doesn't have libmnl-devel
-yum -y install libmnl-devel || yum -y install libmnl
+# CentOS 8 fixes
+$yum libmnl-devel || $yum libmnl
+$yum rubygem-asciidoctor || true
travis/tumbleweed.sh +7 -1
index 6247daa98..43ca3252a 100755
@@ -2,7 +2,10 @@
 # Copyright (c) 2018-2020 Petr Vorel <[email protected]>
 set -ex
 
-zypper --non-interactive install --force-resolution --no-recommends \
+zyp="zypper --non-interactive install --force-resolution --no-recommends"
+
+$zyp \
+	asciidoc \
 	autoconf \
 	automake \
 	clang \
@@ -23,4 +26,7 @@ zypper --non-interactive install --force-resolution --no-recommends \
 	libtirpc-devel \
 	linux-glibc-devel \
 	lsb-release \
+	perl-JSON \
 	pkg-config
+
+$zyp ruby2.7-rubygem-asciidoctor || $zyp ruby2.5-rubygem-asciidoctor
Signature · 1 line
2.26.2
#14 lore raw

PATCH 9/11 travis: Install docparse dependencies

Cyril Hrubis <[email protected]> wrote:
...

Quoted 15 lines
travis/fedora.sh +4 -1
@@ -2,7 +2,10 @@
 # Copyright (c) 2018-2020 Petr Vorel <[email protected]>
 set -ex
 
-yum -y install \
+yum="yum -y install"
+
+$yum \
+       asciidoc \
        autoconf \
        automake \
        make \

travis/fedora.sh yum -y install \ · line 18

Quoted 4 lines
travis/fedora.sh +1 -0
@@ -12,8 +15,10 @@ yum -y install \
        findutils \
        libtirpc \
        libtirpc-devel \
+       perl-JSON \

Package perl-libwww-perl is needed for fedora/centos as well.

Signature · 2 lines
Regards,
Li Wang
#15 lore raw

PATCH 10/11 docparse: Add configure options

In reply to Cyril Hrubis

From: Petr Vorel <[email protected]>

We build HTML and PDF docs with:
- asciidoctor (the default as it produces better output, any version)
- asciidoc (required >= 9)

For asciidoctor we support only native asciidoctor-pdf plugin (we don't support DocBook, Apache FO, or LaTeX intermediary formats).

The goal is to have smart default: try to generate HTML (or also PDF if enabled) docs if suitable generator available.

But allow user to chose:
1) disable whole metadata generation (--disable-metadata)
2) disable metadata HTML generation (--disable-metadata-html)
3) enable metadata PDF generation (--enable-metadata-pdf)
4) generator (--with-metadata-generator=asciidoc|asciidoctor)

Used 2 ax_* (third party) macros from autoconf-archive.

---
 Makefile                   |   6 +-
 configure.ac               |  30 ++++++-
 include/mk/features.mk.in  |   5 ++
 m4/ax_compare_version.m4   | 177 +++++++++++++++++++++++++++++++++++++
 m4/ax_prog_perl_modules.m4 |  77 ++++++++++++++++
 m4/ltp-docparse.m4         | 112 +++++++++++++++++++++++
 6 files changed, 405 insertions(+), 2 deletions(-)
 create mode 100644 m4/ax_compare_version.m4
 create mode 100644 m4/ax_prog_perl_modules.m4
 create mode 100644 m4/ltp-docparse.m4
Makefile +5 -1
index 3830fb6d4..56812d77b 100644
@@ -62,7 +62,11 @@ $(1):: | $$(abs_top_builddir)/$$(basename $$(subst -,.,$(1)))
 endif
 endef
 
-COMMON_TARGETS		+= testcases tools docparse
+COMMON_TARGETS		+= testcases tools
+ifeq ($(WITH_METADATA),yes)
+COMMON_TARGETS		+= docparse
+endif
+
 # Don't want to nuke the original files if we're installing in-build-tree.
 ifneq ($(BUILD_TREE_STATE),$(BUILD_TREE_SRCDIR_INSTALL))
 INSTALL_TARGETS		+= runtest scenario_groups testscripts
configure.ac +29 -1
index 05672f8f6..06be1c094 100644
@@ -189,7 +189,7 @@ AC_CHECK_TYPES([struct xt_entry_match, struct xt_entry_target],,,[
 
 # Tools knobs
 
-# Expect
+# Bash
 AC_ARG_WITH([bash],
   [AC_HELP_STRING([--with-bash],
     [have the Bourne Again Shell interpreter])],
@@ -202,6 +202,34 @@ else
     AC_SUBST([WITH_BASH],["no"])
 fi
 
+# metadata
+AC_ARG_ENABLE([metadata],
+  [AC_HELP_STRING([--disable-metadata],
+	[Disable metadata generation (both HTML and PDF, default no)])],
+  [], [enable_metadata=yes]
+)
+AC_ARG_ENABLE([metadata_html],
+  [AC_HELP_STRING([--disable-metadata-html],
+	[Disable metadata HTML generation (default no)])],
+  [], [enable_metadata_html=yes]
+)
+
+AC_ARG_ENABLE([metadata_pdf],
+  [AC_HELP_STRING([--enable-metadata-pdf],
+	[Enable metadata PDF generation (default no)])],
+  [], [enable_metadata_pdf=no]
+)
+
+AC_ARG_WITH([metadata_generator],
+  [AC_HELP_STRING([--with-metadata-generator=asciidoc|asciidoctor],
+	[Specify metadata generator to use (default autodetect)])],
+  [with_metadata_generator=$withval],
+  [with_metadata_generator=detect]
+)
+
+LTP_DOCPARSE
+
+# Expect
 AC_ARG_WITH([expect],
   [AC_HELP_STRING([--with-expect],
     [have the Tcl/expect library])],
include/mk/features.mk.in +5 -0
index 8e561b738..ecb15a0f7 100644
@@ -27,6 +27,11 @@ WITH_PERL			:= @WITH_PERL@
 
 WITH_PYTHON			:= @WITH_PYTHON@
 
+METADATA_GENERATOR		:= @METADATA_GENERATOR@
+WITH_METADATA			:= @WITH_METADATA@
+WITH_METADATA_HTML		:= @WITH_METADATA_HTML@
+WITH_METADATA_PDF		:= @WITH_METADATA_PDF@
+
 # Features knobs
 
 # Test suite knobs
m4/ax_compare_version.m4 new file +177 -0

177 lines not on this page · raw message

m4/ax_prog_perl_modules.m4 new file +77 -0

77 lines not on this page · raw message

m4/ltp-docparse.m4 new file +112 -0

112 lines not on this page · raw message

Signature · 1 line
2.26.2
#16 lore raw

PATCH 11/11 docparse: Generate html and pdf using asciidoc{, tor}

In reply to Cyril Hrubis

From: Petr Vorel <[email protected]>

Rewrite testinfo.pl to generate *.txt pages in asciidoc format which is then regenerated to html (and pdf if enabled) using asciidoc,{tor}.

Replace getting Linux kernel git commit messages from local git repository (needed after having all tests in single page, because API has access limits; it's also better to generate everything once thus don't depend on network connection).

---
 docparse/.gitignore  |   5 +
 docparse/Makefile    |  58 +++++++
 docparse/testinfo.pl | 406 +++++++++++++++++++++++++++++++++++++++++--
 3 files changed, 458 insertions(+), 11 deletions(-)
docparse/.gitignore +5 -0
index f636ed847..7a87b4234 100644
@@ -1,2 +1,7 @@
+/*.txt
+/docbook-xsl.css
 /docparse
 /metadata.json
+/metadata.html
+/metadata.pdf
+/metadata.chunked/
docparse/Makefile +58 -0

77 lines not on this page · raw message

docparse/testinfo.pl +395 -11

434 lines not on this page · raw message

Signature · 1 line
2.26.2
#17 lore raw

PATCH 11/11 docparse: Generate html and pdf using asciidoc{, tor}

Quoted 24 lines · Cyril Hrubis
docparse/Makefile +13 -0
@@ -1,19 +1,77 @@
 # SPDX-License-Identifier: GPL-2.0-or-later
 # Copyright (c) 2019 Cyril Hrubis <[email protected]>
+# Copyright (c) 2020 Petr Vorel <[email protected]>
 
 top_srcdir             ?= ..
 
 include $(top_srcdir)/include/mk/env_pre.mk
 include $(top_srcdir)/include/mk/functions.mk
 
+ifeq ($(METADATA_GENERATOR),asciidoctor)
+METADATA_GENERATOR_CMD := asciidoctor
+METADATA_GENERATOR_PARAMS := -d book metadata.txt
+METADATA_GENERATOR_PARAMS_HTML := -b xhtml
+METADATA_GENERATOR_PARAMS_PDF := -b pdf -r asciidoctor-pdf
+else ifeq ($(METADATA_GENERATOR),asciidoc)
+METADATA_GENERATOR_CMD := a2x
+METADATA_GENERATOR_PARAMS := --xsltproc-opts "--stringparam toc.section.depth 1" -d book -L  --resource="$(PWD)" metadata.txt
+METADATA_GENERATOR_PARAMS_HTML := -f xhtml
+METADATA_GENERATOR_PARAMS_PDF := -f pdf
+METADATA_GENERATOR_PARAMS_HTML_CHUNKED := -f chunked
+else ifeq ($(METADATA_GENERATOR),)

docparse/Makefile line 31

Quoted 10 lines
docparse/Makefile +10 -0
+$(error 'METADATA_GENERATOR' not not configured, run ./configure in the root directory)
+else
+$(error '$(METADATA_GENERATOR)' not supported, only asciidoctor and asciidoc are supported)
+endif
+
+ifdef VERBOSE
+METADATA_GENERATOR_PARAMS += -v
+endif
+
+CLEAN_TARGETS          := *.txt

I guess the generated *.css *.js files should be deleted as well.

--
Regards,
Li Wang

#18 lore raw

PATCH 11/11 docparse: Generate html and pdf using asciidoc{, tor}

Hi,

Quoted 6 lines

Review

Quoted 2 lines · Cyril Hrubis
(quoted diff) +2 -0
+
+CLEAN_TARGETS          := *.txt

I guess the generated *.css *.js files should be deleted as well.

+1

Kind regards,
Petr

#20 lore raw

cover Test metadata extraction

In reply to Cyril Hrubis

Hi,

Quoted 9 lines

This patchset adds a test metadata extraction into LTP and also documentation generator that produces browseable HTML documentation from the exported metadata. For detailed description of the idea and implementation see the patch that adds README.md.

While the idea is quite new the code is mature enough to be included in the upstream repository and I'm also worried that we will not get any feedback or users of the metadata unless it's included in the upstream git.

Example of the output:
(generally I prefer asciidoctor, but PDF support is not always available) HTML could have custom javascript (probably jquery) based search/filtering.

= asciidoctor
* PDF
https://pevik.github.io/asciidoctor/metadata.pdf
There is missing TOC (as :toc: does not work and adding '-a toc' to generate as:
asciidoctor -d book metadata.txt -b pdf -r asciidoctor-pdf -a toc

puts TOC at the top.

* HTML single page
https://pevik.github.io/asciidoctor/metadata.html

= asciidoc
* PDF
https://pevik.github.io/asciidoc/metadata.pdf
There is ugly revision history at the top.

* HTML single page
https://pevik.github.io/asciidoc/metadata.html

* HTML chunked
https://pevik.github.io/asciidoc/metadata.chunked/index.html
https://pevik.github.io/asciidoc/metadata.chunked/ch01.html
https://pevik.github.io/asciidoc/metadata.chunked/ch02.html
https://pevik.github.io/asciidoc/metadata.chunked/ch03.html

Quoted 3 lines

The next step is to use the extracted metadata in runltp-ng in the proof of concept parallel executor that has been written by Ritchie and posted to this mailing list as well.

Kind regards,
Petr