Skip to contentSkip to conversation

Sparse based checker and rule proposal

PATCH In review Richard Palethorpe 4 participants 14 messages Jul 28, 2021 – Aug 30, 2021

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v3 cover Sparse based checker and rule proposal

Hello,

So it turns out that it is quite easy to implement the TST_RET/ERR check in Sparse. It compiles the code into an IR which is just the right abstraction for the test. It is also possible to inspect the AST with Sparse. It appears more difficult to inspect the AST before macros are expanded.

Sparse is not packaged as a shared library, but has few/no dependencies and is easy to compile. It seems like it was meant to be vendored. Including it as a git module seems reasonable to me. Please see tools/sparse/README.md.

There are still a lot of errors/noise when running 'make check' on the entire tree. These are mainly caused by old function definitions and such. They need to be fixed before the tool can be used properly.

Also I have tried to document the rule and created a list of rules. So this can also be taken as a formal proposal for the rule itself.

Thanks,

V3:
* Rename main to sparse-ltp
* Unify TEST/TST_EXP docs
* Add/fix a couple of links in the docs
* minor formatting changes in sparse-ltp
* Add note on building with -m32

V2:
* Automatically download and build sparse.
* Only build sparse if "make check" is run. It is filtered from "make all".
* Move libtsc.h out of the realtime tests dir. Note that checking of metldown.c now fails because it uses a GCC builtin sparse does not recognize.

As mentioned above, there are various errors during checking that need fixing. For the most part these are legit errors (usually old style function definitions or redefining of symbols). With stuff like metldown I am tempted to filter it, but OTOH it looks relatively straight forward to add a builtin to Sparse upstream. I just need time to do it. First though I would like to get "make check" working on the library, so we can put that in CI.

Richard Palethorpe (8):
Add Sparse based checker and TST_RET/ERR check Add 'make check' to the build system
doc: Add rules and recommendations list
doc: Remind authors and maintainers to run make check
doc: Document TEST macro and state TST_RET/ERR rule LTP-002 Reference LTP-002 rule in Cocci scripts
API: Move libtsc.h from realtime tests include to tst_tsc.h API/tst_tsc: Add guards and remove some boilerplate

 .gitmodules                                   |   3 +
 Makefile                                      |   8 +
 doc/c-test-api.txt                            |  54 ++++++-
 doc/library-api-writing-guidelines.txt        |  14 ++
 doc/maintainer-patch-review-checklist.txt     |   4 +-
 doc/rules.tsv                                 |   3 +
 doc/test-writing-guidelines.txt               |   8 +
 include/mk/env_post.mk                        |   8 +
 include/mk/generic_leaf_target.inc            |   5 +-
 include/mk/generic_trunk_target.inc           |   7 +-
 include/mk/lib.mk                             |   3 +
 include/mk/module.mk                          |   2 +
 include/mk/rules.mk                           |   9 ++
 include/mk/sparse.mk                          |   9 ++
 include/mk/testcases.mk                       |   1 +
 .../include/libtsc.h => include/tst_tsc.h     |  35 +----
 .../coccinelle/libltp-test-macro-vars.cocci   |   6 +-
 scripts/coccinelle/libltp-test-macro.cocci    |   4 +-
 testcases/cve/Makefile                        |   2 -
 testcases/cve/meltdown.c                      |   2 +-
 testcases/open_posix_testsuite/Makefile       |   4 +
 .../func/async_handler/async_handler_tsc.c    |   3 +-
 .../func/measurement/preempt_timing.c         |   3 +-
 .../realtime/func/measurement/rdtsc-latency.c |   3 +-
 tools/Makefile                                |   2 +
 tools/sparse/.gitignore                       |   1 +
 tools/sparse/Makefile                         |  27 ++++
 tools/sparse/README.md                        |  51 ++++++
 tools/sparse/sparse-ltp.c                     | 147 ++++++++++++++++++
 tools/sparse/sparse-src                       |   1 +
 30 files changed, 382 insertions(+), 47 deletions(-)
 create mode 100644 doc/rules.tsv
 create mode 100644 include/mk/sparse.mk
 rename testcases/realtime/include/libtsc.h => include/tst_tsc.h (53%)
 create mode 100644 tools/sparse/.gitignore
 create mode 100644 tools/sparse/Makefile
 create mode 100644 tools/sparse/README.md
 create mode 100644 tools/sparse/sparse-ltp.c
 create mode 160000 tools/sparse/sparse-src
Signature · 1 line
2.31.1
#1 lore raw

PATCH v3 1/8 Add Sparse based checker and TST_RET/ERR check

Vendors in Sparse as a git module. Then uses it to check for stores to TST_RET/ERR within the library.

---
 .gitmodules               |   3 +
 tools/Makefile            |   2 +
 tools/sparse/.gitignore   |   1 +
 tools/sparse/Makefile     |  27 +++++++
 tools/sparse/README.md    |  51 +++++++++++++
 tools/sparse/sparse-ltp.c | 147 ++++++++++++++++++++++++++++++++++++++
 tools/sparse/sparse-src   |   1 +
 7 files changed, 232 insertions(+)
 create mode 100644 tools/sparse/.gitignore
 create mode 100644 tools/sparse/Makefile
 create mode 100644 tools/sparse/README.md
 create mode 100644 tools/sparse/sparse-ltp.c
 create mode 160000 tools/sparse/sparse-src
.gitmodules +3 -0
index 1c9e9c38a..a3c34af4b 100644
@@ -1,3 +1,6 @@
 [submodule "testcases/kernel/mce-test"]
 	path = testcases/kernel/mce-test
 	url = git://git.kernel.org/pub/scm/linux/kernel/git/gong.chen/mce-test.git
+[submodule "tools/sparse/sparse-src"]
+	path = tools/sparse/sparse-src
+	url = git://git.kernel.org/pub/scm/devel/sparse/sparse.git
tools/Makefile +2 -0
index def1c6fa9..adbf4fe70 100644
@@ -28,4 +28,6 @@ INSTALL_TARGETS		:= *.awk *.pl *.sh html_report_header.txt
 
 INSTALL_DIR		:= bin
 
+FILTER_OUT_DIRS		+= sparse
+
 include $(top_srcdir)/include/mk/generic_trunk_target.mk
tools/sparse/.gitignore new file +1 -0
new file mode 100644
index 000000000..b2d2c37fa
@@ -0,0 +1 @@
+sparse-ltp
tools/sparse/Makefile new file +27 -0
new file mode 100644
index 000000000..4247dd86e
@@ -0,0 +1,27 @@
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2021 SUSE LLC <[email protected]>
+
+top_srcdir		?= ../..
+
+include $(top_srcdir)/include/mk/env_pre.mk
+include $(top_srcdir)/include/mk/functions.mk
+
+SPARSE_SRC	?= sparse-src
+
+$(SPARSE_SRC)/Makefile:
+ifeq ($(SPARSE_SRC),sparse-src)
+	git submodule update --init
+else
+	$(error "Can't find $(SPARSE_SRC)/Makefile")
+endif
+
+$(SPARSE_SRC)/libsparse.a: $(SPARSE_SRC)/Makefile
+	$(MAKE) -C $(SPARSE_SRC) libsparse.a
+
+HOST_MAKE_TARGETS	:= sparse-ltp
+MAKE_DEPS		+= $(SPARSE_SRC)/libsparse.a
+HOST_CFLAGS		+= -I$(SPARSE_SRC) -Werror
+HOST_LDLIBS		+= $(SPARSE_SRC)/libsparse.a
+
+
+include $(top_srcdir)/include/mk/generic_leaf_target.mk
tools/sparse/README.md new file +51 -0

51 lines not on this page · raw message

tools/sparse/sparse-ltp.c new file +147 -0

147 lines not on this page · raw message

tools/sparse/sparse-src new file +1 -0
new file mode 160000
index 000000000..8af243292
@@ -0,0 +1 @@
+Subproject commit 8af2432923486c753ab52cae70b94ee684121080
Signature · 1 line
2.31.1
#2 lore raw

PATCH v3 2/8 Add 'make check' to the build system

In reply to Richard Palethorpe

Allows one to run 'make check' or 'make check-$TARGET'. Which will execute the CHECK tool with that target's CC arguments and CHECK_FLAGS. By default the check tool is tools/sparse/main.

---
 Makefile                                | 8 ++++++++
 include/mk/env_post.mk                  | 8 ++++++++
 include/mk/generic_leaf_target.inc      | 5 ++++-
 include/mk/generic_trunk_target.inc     | 7 ++++++-
 include/mk/lib.mk                       | 3 +++
 include/mk/module.mk                    | 2 ++
 include/mk/rules.mk                     | 9 +++++++++
 include/mk/sparse.mk                    | 9 +++++++++
 include/mk/testcases.mk                 | 1 +
 testcases/open_posix_testsuite/Makefile | 4 ++++
 10 files changed, 54 insertions(+), 2 deletions(-)
 create mode 100644 include/mk/sparse.mk
Makefile +8 -0
index 56812d77b..3b0ba330d 100644
@@ -79,6 +79,7 @@ BOOTSTRAP_TARGETS	:= $(sort $(COMMON_TARGETS) $(CLEAN_TARGETS) $(INSTALL_TARGETS
 CLEAN_TARGETS		:= $(addsuffix -clean,$(CLEAN_TARGETS))
 INSTALL_TARGETS		:= $(addsuffix -install,$(INSTALL_TARGETS))
 MAKE_TARGETS		:= $(addsuffix -all,$(filter-out lib,$(COMMON_TARGETS)))
+CHECK_TARGETS		:= $(addsuffix -check,testcases lib)
 
 # There's no reason why we should run `all' twice. Otherwise we're just wasting
 # 3+ mins of useful CPU cycles on a modern machine, and even more time on an
@@ -108,6 +109,10 @@ $(MAKE_TARGETS) include-all lib-all libs-all:
 	$(MAKE) -C "$(subst -all,,$@)" \
 		-f "$(abs_top_srcdir)/$(subst -all,,$@)/Makefile" all
 
+$(CHECK_TARGETS): tools-all
+	$(MAKE) -C "$(subst -check,,$@)" \
+		-f "$(abs_top_srcdir)/$(subst -check,,$@)/Makefile" check
+
 # Let's not conflict with ac-clean, maintainer-clean, etc, so.
 $(filter-out include-clean,$(CLEAN_TARGETS))::
 	-$(MAKE) -C "$(subst -clean,,$@)" \
@@ -189,6 +194,9 @@ INSTALL_TARGETS		+= $(addprefix $(DESTDIR)/$(bindir)/,$(BINDIR_INSTALL_SCRIPTS))
 
 $(INSTALL_TARGETS): $(INSTALL_DIR) $(DESTDIR)/$(bindir)
 
+.PHONY: check
+check: $(CHECK_TARGETS)
+
 ## Install
 install: $(INSTALL_TARGETS)
 
include/mk/env_post.mk +8 -0
index 1d22f9c53..4722da907 100644
@@ -89,6 +89,14 @@ $(error You must define $$(prefix) before executing install)
 endif # END $(filter-out install,$(MAKECMDGOALS)),$(MAKECMDGOALS)
 endif
 
+CHECK_TARGETS			?= $(addprefix check-,$(notdir $(patsubst %.c,%,$(sort $(wildcard $(abs_srcdir)/*.c)))))
+CHECK_TARGETS			:= $(filter-out $(addprefix check-, $(FILTER_OUT_MAKE_TARGETS)), $(CHECK_TARGETS))
+CHECK				?= $(abs_top_srcdir)/tools/sparse/sparse-ltp
+
+ifeq ($(CHECK),$(abs_top_srcdir)/tools/sparse/sparse-ltp)
+CHECK_DEPS			+= $(CHECK)
+endif
+
 include $(top_srcdir)/include/mk/rules.mk
 
 endif
include/mk/generic_leaf_target.inc +4 -1
index 64953f89a..aa092a5a3 100644
@@ -92,7 +92,7 @@
 # INSTALL_DIR			:= $(libdir)
 #
 
-.PHONY: all clean install
+.PHONY: all clean install check
 
 ifneq ($(strip $(MAKE_TARGETS)),)
 $(MAKE_TARGETS) += $(HOST_MAKE_TARGETS)
@@ -109,4 +109,7 @@ $(INSTALL_FILES): | $(INSTALL_DEPS)
 
 install: $(INSTALL_FILES)
 
+$(CHECK_TARGETS): | $(CHECK_DEPS)
+check: $(CHECK_TARGETS)
+
 # vim: syntax=make
include/mk/generic_trunk_target.inc +6 -1
index fc59f944f..32a108fbf 100644
@@ -48,7 +48,7 @@
 
 include $(top_srcdir)/include/mk/functions.mk
 
-RECURSIVE_TARGETS		?= all install
+RECURSIVE_TARGETS		?= all install check
 
 $(eval $(get_make_dirs))
 
@@ -68,6 +68,9 @@ $(INSTALL_FILES): | $(INSTALL_DEPS)
 
 trunk-install: $(INSTALL_FILES)
 
+$(CHECK_TARGETS): | $(CHECK_DEPS)
+trunk-check: $(CHECK_TARGETS)
+
 # Avoid creating duplicate .PHONY references to all, clean, and install. IIRC,
 # I've seen some indeterministic behavior when one does this in the past with
 # GNU Make...
@@ -108,4 +111,6 @@ else
 endif
 endif
 
+check: trunk-check
+
 # vim: syntax=make
include/mk/lib.mk +3 -0
index f9b6c0aff..3bf63bf9e 100644
@@ -26,6 +26,7 @@
 # Makefile to include for libraries.
 
 include $(top_srcdir)/include/mk/env_pre.mk
+include $(top_srcdir)/include/mk/sparse.mk
 
 INSTALL_DIR	:= $(libdir)
 
@@ -57,6 +58,8 @@ LIBSRCS		:= $(filter-out $(FILTER_OUT_LIBSRCS),$(LIBSRCS))
 
 LIBOBJS		:= $(LIBSRCS:.c=.o)
 
+CHECK_TARGETS	:= $(addprefix check-,$(notdir $(LIBSRCS:.c=)))
+
 $(LIB): $(notdir $(LIBOBJS))
 	@if [ -z "$(strip $^)" ] ; then \
 		echo "Cowardly refusing to create empty archive"; \
include/mk/module.mk +2 -0
index 6c8814b96..3bb7350f1 100644
@@ -47,6 +47,8 @@ endif
 
 CLEAN_TARGETS += .dep_modules *.mod built-in.a
 
+CHECK_TARGETS := $(filter-out %.ko, $(CHECK_TARGETS))
+
 MODULE_SOURCES := $(patsubst %.ko,%.c,$(filter %.ko, $(MAKE_TARGETS)))
 
 # Ignoring the exit status of commands is done to be forward compatible with
include/mk/rules.mk +9 -0
index c8f4bbbbe..2a04b2b67 100644
@@ -37,3 +37,12 @@ else
 	@$(CC) $(CPPFLAGS) $(CFLAGS) $(LDFLAGS) $^ $(LTPLDLIBS) $(LDLIBS) -o $@
 	@echo CC $(target_rel_dir)$@
 endif
+
+.PHONY: $(CHECK_TARGETS)
+$(CHECK_TARGETS): check-%: %.c
+ifdef VERBOSE
+	$(CHECK) $(CHECK_FLAGS) $(CPPFLAGS) $(CFLAGS) $<
+else
+	@$(CHECK) $(CHECK_FLAGS) $(CPPFLAGS) $(CFLAGS) $<
+	@echo CHECK $(target_rel_dir)$<
+endif
include/mk/sparse.mk new file +9 -0
new file mode 100644
index 000000000..a86928393
@@ -0,0 +1,9 @@
+# Rules to make sparse tool(s) for inclusion in lib and testcases Makefiles
+
+SPARSE_DIR:= $(abs_top_builddir)/tools/sparse
+
+$(SPARSE_DIR)/sparse-ltp: $(SPARSE_DIR)
+	$(MAKE) -C "$^" all
+
+$(SPARSE_DIR): %:
+	mkdir -p "$@"
include/mk/testcases.mk +1 -0
index 1c81773d0..444020f16 100644
@@ -22,6 +22,7 @@
 
 include $(top_srcdir)/include/mk/env_pre.mk
 include $(top_srcdir)/include/mk/functions.mk
+include $(top_srcdir)/include/mk/sparse.mk
 
 APICMDS_DIR	:= $(abs_top_builddir)/tools/apicmds
 
testcases/open_posix_testsuite/Makefile +4 -0
index 205ecdc00..fea6db14b 100644
@@ -104,3 +104,7 @@ $(CRITICAL_MAKEFILE): \
 	$(top_srcdir)/LDFLAGS			\
 	$(top_srcdir)/LDLIBS
 	@$(MAKE) generate-makefiles
+
+.PHONY: check
+check:
+	@echo "Checker not yet supported by Open POSIX testsuite"
Signature · 1 line
2.31.1
#3 lore raw

PATCH v3 2/8 Add 'make check' to the build system

Hi Richie,

I see 'make check' requires all to have headers/dependencies installed, is that wanted?

$ make check
...
route-change-netlink.c:13:11: error: unable to open 'libmnl/libmnl.h'

<libmnl/libmnl.h> is guarded by #ifdef HAVE_LIBMNL (config.h). Maybe there is a way to pass it to sparse. But that'd silently prevent to check code we want to check, thus it's probably better to just require to have everything installed.

Kind regards,
Petr

#4 lore raw

PATCH v3 2/8 Add 'make check' to the build system

Hi!

Quoted 7 lines
$ make check
...
route-change-netlink.c:13:11: error: unable to open 'libmnl/libmnl.h'

<libmnl/libmnl.h> is guarded by #ifdef HAVE_LIBMNL (config.h). Maybe there is a way to pass it to sparse. But that'd silently prevent to check code we want to check, thus it's probably better to just require to have everything installed.

Well if you are hacking on a test that requires libnl and you want to run 'make check' before submission it has to be installed anyways. So this is not a big problem for the intended usage...

Signature · 2 lines
Cyril Hrubis
[email protected]
#5 lore raw

PATCH v3 2/8 Add 'make check' to the build system

Quoted 5 lines

Hi!

Quoted 3 lines
$ make check
...
route-change-netlink.c:13:11: error: unable to open 'libmnl/libmnl.h'
Quoted 7 lines
Quoted 3 lines

<libmnl/libmnl.h> is guarded by #ifdef HAVE_LIBMNL (config.h). Maybe there is a way to pass it to sparse. But that'd silently prevent to check code we want to check, thus it's probably better to just require to have everything installed.

Well if you are hacking on a test that requires libnl and you want to run 'make check' before submission it has to be installed anyways. So this is not a big problem for the intended usage...

Sure, you're right.

Kind regards,
Petr

#6 lore raw

PATCH v3 3/8 doc: Add rules and recommendations list

In reply to Richard Palethorpe

Create a list of rules which are feasible to check automatically. The file is a table of tab separated values. It's intended to be machine readable.

For now there is just an ID column, with IDs that look similar to CWE IDs or CVEs. For now these can just be used with 'git grep'. Also there is a description column.

Clearly this is not an exhaustive list. It just contains a library rule already stated in the guide and the issue Sparse checks for.

---
 doc/rules.tsv                   | 3 +++
 doc/test-writing-guidelines.txt | 7 +++++++
 2 files changed, 10 insertions(+)
 create mode 100644 doc/rules.tsv
doc/rules.tsv new file +3 -0
new file mode 100644
index 000000000..d4081ce0f
@@ -0,0 +1,3 @@
+ID	DESCRIPTION
+LTP-001	Library source files have tst_ prefix
+LTP-002	TST_RET and TST_ERR are never modified by test library functions
doc/test-writing-guidelines.txt +7 -0
index d57e52655..1197c9a0d 100644
@@ -10,6 +10,13 @@ NOTE: See also
       https://github.com/linux-test-project/ltp/wiki/Shell-Test-API[Shell Test API],
       https://github.com/linux-test-project/ltp/wiki/LTP-Library-API-Writing-Guidelines[LTP Library API Writing Guidelines].
 
+Rules and recommendations which are "machine checkable" should be
+tagged with an ID like +LTP-XXX+. There will be a corresponding entry
+in
+https://github.com/linux-test-project/ltp/tree/master/doc/rules.tsv[doc/rules.tsv]. When
+you run 'make check' or 'make check-test' it will display these IDs as
+a reference.
+
 1. Guide to clean and understandable code
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
Signature · 1 line
2.31.1
#7 lore raw

PATCH v3 4/8 doc: Remind authors and maintainers to run make check

In reply to Richard Palethorpe
---
 doc/maintainer-patch-review-checklist.txt | 4 ++--
 doc/test-writing-guidelines.txt           | 1 +
 2 files changed, 3 insertions(+), 2 deletions(-)
doc/maintainer-patch-review-checklist.txt +2 -2
index f6682b574..58662bed1 100644
@@ -39,8 +39,8 @@ New test should
 * Use new https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#22-writing-a-test-in-c[C API]
 * Test binaries are added into corresponding '.gitignore' files
 * Check coding style with
-  https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plain/scripts/checkpatch.pl[checkpatch.pl]
-  (more in https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#131-c-coding-style[C coding style])
+  https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plain/scripts/checkpatch.pl[checkpatch.pl] and `make check`
+  (more in https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#21-c-coding-style[C coding style])
 * Docparse documentation
 * If a test is a regression test it should include tags
   (more in https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#2238-test-tags[Test tags])
doc/test-writing-guidelines.txt +1 -0
index 1197c9a0d..340c80cc6 100644
@@ -310,6 +310,7 @@ NOTE: See also
 3. The runtest entires are in place
 4. Test binaries are added into corresponding '.gitignore' files
 5. Patches apply over the latest git
+6. 'make check' does not emit any warnings
 
 6.1 About .gitignore files
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
Signature · 1 line
2.31.1
#8 lore raw

PATCH v3 5/8 doc: Document TEST macro and state TST_RET/ERR rule LTP-002

In reply to Richard Palethorpe

There are cases where these variables can be safely used by the library. However it is a difficult problem to identify these cases automatically.

Identifying any library code which uses them is relatively easy. In fact it is easier to automate it than by code review. Because it is such a boring thing to repeatedly look for and comment on.

If a test library function needs these variables it can recreate its own private copies.

---
 doc/c-test-api.txt                     | 54 +++++++++++++++++++++++++-
 doc/library-api-writing-guidelines.txt | 14 +++++++
 2 files changed, 66 insertions(+), 2 deletions(-)
doc/c-test-api.txt +52 -2
index dd6c608a4..25b9beb7c 100644
@@ -291,8 +291,58 @@ static void test(void)
 }
 -------------------------------------------------------------------------------
 
-The 'TST_EXP_FAIL2()' is the same as 'TST_EXP_FAIL()' except the return value
-is expected to be non-negative integer if call passes.
+The 'TST_EXP_FAIL2()' is the same as 'TST_EXP_FAIL()' except the
+return value is expected to be non-negative integer if call
+passes. These macros build upon the +TEST()+ macro and associated
+variables.
+
+[source,c]
+-------------------------------------------------------------------------------
+TEST(socket(AF_INET, SOCK_RAW, 1));
+if (TST_RET > -1) {
+	tst_res(TFAIL, "Created raw socket");
+	SAFE_CLOSE(TST_RET);
+} else if (TST_ERR != EPERM) {
+	tst_res(TFAIL | TTERRNO,
+		"Failed to create socket for wrong reason");
+} else {
+	tst_res(TPASS | TTERRNO, "Didn't create raw socket");
+}
+-------------------------------------------------------------------------------
+
+The +TEST+ macro sets +TST_RET+ to its argument's return value and
++TST_ERR+ to +errno+. The +TTERNO+ flag can be used to print the error
+number's symbolic value.
+
+No LTP library function or macro, except those in 'tst_test_macros.h',
+will write to these variables (rule 'LTP-002'). So their values will
+not be changed unexpectedly.
+
+[source,c]
+-------------------------------------------------------------------------------
+TST_EXP_POSITIVE(wait(&status));
+
+if (!TST_PASS)
+	return;
+-------------------------------------------------------------------------------
+
+If the return value of 'wait' is positive. This macro will print a
+pass result and set +TST_PASS+ appropriately. If the return value is
+zero or negative, then it will print fail.
+
+As seen above, this and similar macros take optional variadic
+arguments. These begin with a format string and then appropriate
+values to be formatted. So
+
+[source,c]
+-------------------------------------------------------------------------------
+TST_EXP_PASS(chown("a/file", uid, gid), "chown(%s,%d,%d)",
+	     "a/file", uid, gid);
+-------------------------------------------------------------------------------
+
+expects +chown+ to return 0 and emits a pass or a fail. The arguments
+to +chown+ will be printed in either case. There are many similar
+macros to those shown here, please see 'tst_test_macros.h'.
 
 [source,c]
 -------------------------------------------------------------------------------
doc/library-api-writing-guidelines.txt +14 -0
index a4393fc54..2819d4177 100644
@@ -21,10 +21,24 @@ Don't forget to update docs when you change the API.
 2. C API
 --------
 
+2.1 LTP-001: Sources have tst_ prefix
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
 API source code is in headers `include/*.h`, `include/lapi/*.h` (backward
 compatibility for old kernel and libc) and C sources in `lib/*.c`. Files have
 'tst_' prefix.
 
+2.2 LTP-002: TST_RET and TST_ERR are not modified
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The test author is guaranteed that the test API will not modify these
+variables. This prevents silent errors where the return value and
+errno are overwritten before the test has chance to check them.
+
+The macros which are clearly intended to update these variables. That
+is +TEST+ and those in 'tst_test_macros.h'. Are of course allowed to
+update these variables.
+
 3. Shell API
 ------------
 
Signature · 1 line
2.31.1
#9 lore raw

PATCH v3 5/8 doc: Document TEST macro and state TST_RET/ERR rule LTP-002

Hi!
I've pushed all but this patch, see below.

Quoted 46 lines

-------------------------------------------------------------------------------

doc/c-test-api.txt +42 -2
-The 'TST_EXP_FAIL2()' is the same as 'TST_EXP_FAIL()' except the return value
-is expected to be non-negative integer if call passes.
+The 'TST_EXP_FAIL2()' is the same as 'TST_EXP_FAIL()' except the
+return value is expected to be non-negative integer if call
+passes. These macros build upon the +TEST()+ macro and associated
+variables.
+
+[source,c]
+-------------------------------------------------------------------------------
+TEST(socket(AF_INET, SOCK_RAW, 1));
+if (TST_RET > -1) {
+	tst_res(TFAIL, "Created raw socket");
+	SAFE_CLOSE(TST_RET);
+} else if (TST_ERR != EPERM) {
+	tst_res(TFAIL | TTERRNO,
+		"Failed to create socket for wrong reason");
+} else {
+	tst_res(TPASS | TTERRNO, "Didn't create raw socket");
+}
+-------------------------------------------------------------------------------
+
+The +TEST+ macro sets +TST_RET+ to its argument's return value and
++TST_ERR+ to +errno+. The +TTERNO+ flag can be used to print the error
+number's symbolic value.
+
+No LTP library function or macro, except those in 'tst_test_macros.h',
+will write to these variables (rule 'LTP-002'). So their values will
+not be changed unexpectedly.
+
+[source,c]
+-------------------------------------------------------------------------------
+TST_EXP_POSITIVE(wait(&status));
+
+if (!TST_PASS)
+	return;
+-------------------------------------------------------------------------------
+
+If the return value of 'wait' is positive. This macro will print a
+pass result and set +TST_PASS+ appropriately. If the return value is
+zero or negative, then it will print fail.
+
+As seen above, this and similar macros take optional variadic
+arguments. These begin with a format string and then appropriate
+values to be formatted. So

doc/c-test-api.txt static void test(void) · line 345

Quoted 10 lines
doc/c-test-api.txt +10 -0
+
+[source,c]
+-------------------------------------------------------------------------------
+TST_EXP_PASS(chown("a/file", uid, gid), "chown(%s,%d,%d)",
+	     "a/file", uid, gid);
+-------------------------------------------------------------------------------
+
+expects +chown+ to return 0 and emits a pass or a fail. The arguments
+to +chown+ will be printed in either case. There are many similar
+macros to those shown here, please see 'tst_test_macros.h'.

There is already TST_EXP_PASS description before the TST_EXP_FAIL so this duplicates it. Other than that the rest looks good.

Signature · 2 lines
Cyril Hrubis
[email protected]
#10 lore raw

PATCH v3 6/8 Reference LTP-002 rule in Cocci scripts

In reply to Richard Palethorpe

Avoid duplication of rule description

---
 scripts/coccinelle/libltp-test-macro-vars.cocci | 6 +-----
 scripts/coccinelle/libltp-test-macro.cocci      | 4 +---
 2 files changed, 2 insertions(+), 8 deletions(-)
scripts/coccinelle/libltp-test-macro-vars.cocci +1 -5
index ed5459a48..e0fe4e2a7 100644
@@ -1,11 +1,7 @@
 // SPDX-License-Identifier: GPL-2.0-or-later
 // Copyright (c) 2021 SUSE LLC  <[email protected]>
 
-// The TEST macro should not be used in the library because it sets
-// TST_RET and TST_ERR which are global variables. The test author
-// only expects these to be changed if *they* call TEST directly.
-
-// Find all positions where TEST's variables are used
+// Find violations of LTP-002
 @ find_use exists @
 expression E;
 @@
scripts/coccinelle/libltp-test-macro.cocci +1 -3
index 7563d23aa..86c55c6c4 100644
@@ -1,9 +1,7 @@
 // SPDX-License-Identifier: GPL-2.0-or-later
 // Copyright (c) 2021 SUSE LLC  <[email protected]>
 
-// The TEST macro should not be used in the library because it sets
-// TST_RET and TST_ERR which are global variables. The test author
-// only expects these to be changed if *they* call TEST directly.
+// Find and fix violations of rule LTP-002
 
 // Set with -D fix
 virtual fix
Signature · 1 line
2.31.1
#11 lore raw

PATCH v3 7/8 API: Move libtsc.h from realtime tests include to tst_tsc.h

In reply to Richard Palethorpe

Allow the meltdown test to use this file without modifying the CFLAGS. This avoids having to add the include also to the CHECK_FLAGS.

---
 testcases/realtime/include/libtsc.h => include/tst_tsc.h  | 0
 testcases/cve/Makefile                                    | 2 --
 testcases/cve/meltdown.c                                  | 2 +-
 testcases/realtime/func/async_handler/async_handler_tsc.c | 3 ++-
 testcases/realtime/func/measurement/preempt_timing.c      | 3 ++-
 testcases/realtime/func/measurement/rdtsc-latency.c       | 3 ++-
 6 files changed, 7 insertions(+), 6 deletions(-)
 rename testcases/realtime/include/libtsc.h => include/tst_tsc.h (100%)
include/tst_tsc.h renamed from testcases/realtime/include/libtsc.h +0 -0
similarity index 100%
rename from testcases/realtime/include/libtsc.h
rename to include/tst_tsc.h
testcases/cve/Makefile +0 -2
index ddf8b6fe1..c5308794d 100644
@@ -18,8 +18,6 @@ ifneq ($(ANDROID),1)
 cve-2014-0196:  LDLIBS += -lutil
 endif
 
-meltdown: CFLAGS += -I$(abs_srcdir)/../realtime/include
-
 ifneq (,$(filter $(HOST_CPU),x86 x86_64))
 meltdown: CFLAGS += -msse2
 endif
testcases/cve/meltdown.c +1 -1
index a387b3205..5a984aba3 100644
@@ -19,7 +19,7 @@
 
 #include <emmintrin.h>
 
-#include "libtsc.h"
+#include "tst_tsc.h"
 
 #define TARGET_OFFSET	9
 #define TARGET_SIZE	(1 << TARGET_OFFSET)
testcases/realtime/func/async_handler/async_handler_tsc.c +2 -1
index 7da4324a5..73d4ee5c6 100644
@@ -46,7 +46,8 @@
 #include <pthread.h>
 #include <librttest.h>
 #include <libstats.h>
-#include <libtsc.h>
+
+#include "tst_tsc.h"
 
 #define HANDLER_PRIO 98
 #define SIGNAL_PRIO 99
testcases/realtime/func/measurement/preempt_timing.c +2 -1
index 8b5333480..b84d54692 100644
@@ -52,7 +52,8 @@
 #include <sys/mman.h>
 #include <stdint.h>
 #include <librttest.h>
-#include <libtsc.h>
+
+#include "tst_tsc.h"
 
 #define ITERATIONS 1000000ULL
 #define INTERVALS 10
testcases/realtime/func/measurement/rdtsc-latency.c +2 -1
index d6ab89ff0..3829947bc 100644
@@ -44,7 +44,8 @@
 #include <errno.h>
 #include <stdint.h>
 #include <librttest.h>
-#include <libtsc.h>
+
+#include "tst_tsc.h"
 
 #define ITERATIONS 1000000
 
Signature · 1 line
2.31.1
#12 lore raw

PATCH v3 8/8 API/tst_tsc: Add guards and remove some boilerplate

In reply to Richard Palethorpe
---
 include/tst_tsc.h | 35 ++++++++---------------------------
 1 file changed, 8 insertions(+), 27 deletions(-)
include/tst_tsc.h +8 -27
index 9ad5fd659..3f49a6ca7 100644
@@ -1,28 +1,6 @@
-/******************************************************************************
- *
- *   Copyright ? International Business Machines  Corp., 2006-2008
- *
- *   This program is free software;  you can redistribute it and/or modify
- *   it under the terms of the GNU General Public License as published by
- *   the Free Software Foundation; either version 2 of the License, or
- *   (at your option) any later version.
- *
- *   This program is distributed in the hope that it will be useful,
- *   but WITHOUT ANY WARRANTY;  without even the implied warranty of
- *   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
- *   the GNU General Public License for more details.
- *
- *   You should have received a copy of the GNU General Public License
- *   along with this program;  if not, write to the Free Software
- *   Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * NAME
- *       libtsc.h
- *
- * DESCRIPTION
- *
- * USAGE:
- *       To be included in some testcases.
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright ? International Business Machines  Corp., 2006-2008
  *
  * AUTHOR
  *        Darren Hart <[email protected]>
@@ -30,8 +8,10 @@
  *
  * HISTORY
  *      It directly comes from the librttest.h (see its HISTORY).
- *
- *****************************************************************************/
+ */
+
+#ifndef TST_TSC_H
+#define TST_TSC_H
 
 #undef TSC_UNSUPPORTED
 
@@ -70,3 +50,4 @@
 #define TSC_UNSUPPORTED
 #endif
 
+#endif
Signature · 1 line
2.31.1