Sparse based checker and rule proposal
Subsystems (2)
cover Sparse based checker and rule proposal
Hello,
So it turns out that it is quite easy to implement the TST_RET/ERR check in Sparse. It compiles the code into an IR which is just the right abstraction for the test. It is also possible to inspect the AST with Sparse. It appears more difficult to inspect the AST before macros are expanded.
Sparse is not packaged as a shared library, but has few/no dependencies and is easy to compile. It seems like it was meant to be vendored. Including it as a git module seems reasonable to me. Please see tools/sparse/README.md.
There are still a lot of errors/noise when running 'make check' on the entire tree. These are mainly caused by old function definitions and such. They need to be fixed before the tool can be used properly.
Also I have tried to document the rule and created a list of rules. So this can also be taken as a formal proposal for the rule itself.
Thanks,
Richard Palethorpe (7):
Add Sparse based checker and TST_RET/ERR check Add 'make check' to the build system
meltdown: Set CFLAGS for check targets
doc: Add rules and recommendations list
doc: Remind authors and maintainers to run make check
doc: Document TEST macro and state TST_RET/ERR rule LTP-002 Reference LTP-002 rule in Cocci scripts
.gitmodules | 3 + Makefile | 8 + doc/c-test-api.txt | 47 ++++++ doc/library-api-writing-guidelines.txt | 14 ++ doc/maintainer-patch-review-checklist.txt | 2 +- doc/rules.tsv | 3 + doc/test-writing-guidelines.txt | 6 + include/mk/env_post.mk | 8 + include/mk/generic_leaf_target.inc | 5 +- include/mk/generic_trunk_target.inc | 7 +- include/mk/lib.mk | 3 + include/mk/module.mk | 2 + include/mk/rules.mk | 9 ++ include/mk/sparse.mk | 9 ++ include/mk/testcases.mk | 1 + .../coccinelle/libltp-test-macro-vars.cocci | 6 +- scripts/coccinelle/libltp-test-macro.cocci | 4 +- testcases/cve/Makefile | 1 + testcases/open_posix_testsuite/Makefile | 4 + tools/sparse/.gitignore | 1 + tools/sparse/Makefile | 20 +++ tools/sparse/README.md | 34 ++++ tools/sparse/main.c | 148 ++++++++++++++++++ tools/sparse/sparse-src | 1 + 24 files changed, 335 insertions(+), 11 deletions(-) create mode 100644 doc/rules.tsv create mode 100644 include/mk/sparse.mk create mode 100644 tools/sparse/.gitignore create mode 100644 tools/sparse/Makefile create mode 100644 tools/sparse/README.md create mode 100644 tools/sparse/main.c create mode 160000 tools/sparse/sparse-src
Signature · 1 line
2.31.1
PATCH 1/7 Add Sparse based checker and TST_RET/ERR check
Vendors in Sparse as a git module. Then uses it to check for stores to TST_RET/ERR within the library.
--- .gitmodules | 3 + tools/sparse/.gitignore | 1 + tools/sparse/Makefile | 20 ++++++ tools/sparse/README.md | 34 +++++++++ tools/sparse/main.c | 148 ++++++++++++++++++++++++++++++++++++++++ tools/sparse/sparse-src | 1 + 6 files changed, 207 insertions(+) create mode 100644 tools/sparse/.gitignore create mode 100644 tools/sparse/Makefile create mode 100644 tools/sparse/README.md create mode 100644 tools/sparse/main.c create mode 160000 tools/sparse/sparse-src
.gitmodules +3 -0
@@ -1,3 +1,6 @@ [submodule "testcases/kernel/mce-test"] path = testcases/kernel/mce-test url = git://git.kernel.org/pub/scm/linux/kernel/git/gong.chen/mce-test.git +[submodule "tools/sparse/sparse-src"] + path = tools/sparse/sparse-src + url = git://git.kernel.org/pub/scm/devel/sparse/sparse.git
tools/sparse/.gitignore new file +1 -0
@@ -0,0 +1 @@ +main
tools/sparse/Makefile new file +20 -0
@@ -0,0 +1,20 @@ +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2021 SUSE LLC <[email protected]> + +top_srcdir ?= ../.. + +include $(top_srcdir)/include/mk/env_pre.mk +include $(top_srcdir)/include/mk/functions.mk + +SPARSE_SRC ?= sparse-src + +$(SPARSE_SRC)/libsparse.a: + $(error "You need to build Sparse. See tools/sparse/README.md") + +HOST_MAKE_TARGETS := main +MAKE_DEPS += $(SPARSE_SRC)/libsparse.a +HOST_CFLAGS += -I$(SPARSE_SRC) +HOST_LDLIBS += $(SPARSE_SRC)/libsparse.a + + +include $(top_srcdir)/include/mk/generic_leaf_target.mk
tools/sparse/README.md new file +34 -0
@@ -0,0 +1,34 @@ +# Sparse based linting + +This tool checks LTP test and library code for common problems. + +## Building + +The bad news is you must get and build Sparse[^1]. The good news this only +takes a minute. + +If you already have the Sparse source code, then do the +following. Where `$SRC_PATH` is the path to the Sparse directory. + +```sh +$ cd tools/sparse +$ make -C $SRC_PATH -j$(nproc) # Optional +$ make SPARSE_SRC=$SRC_PATH +``` +If not then you can fetch it via the git submodule + +```sh +$ cd tools/sparse +$ git submodule update --init +$ cd sparse-src +$ make -C sparse-src -j$(nproc) +$ make +``` + +## Usage + +It is integrated with the LTP build system. Just run `make check` or +`make check-a_test01`, where `a_test01` is an arbitrary test +executable or object file. + +[1]: Many distributions have a Sparse package. This only contains some executables. There is no shared library
tools/sparse/main.c new file +148 -0
148 lines not on this page · raw message
tools/sparse/sparse-src new file +1 -0
@@ -0,0 +1 @@ +Subproject commit 8af2432923486c753ab52cae70b94ee684121080
Signature · 1 line
2.31.1
PATCH 1/7 Add Sparse based checker and TST_RET/ERR check
Hi!
Quoted 2 lines
Vendors in Sparse as a git module. Then uses it to check for stores to TST_RET/ERR within the library.
This sounds reasonable enough.
It would probably be even easier to copy the sources to the LTP tree and synchronize them manually, depending on how often is sparse updated it may end up better than a submodule.
Also isn't the submodule frozen on a given commit anyways (unless we add git submodule --remote --merge into the build steps)?
Quoted 81 lines
Signed-off-by Richard Palethorpe <[email protected]>--- .gitmodules | 3 + tools/sparse/.gitignore | 1 + tools/sparse/Makefile | 20 ++++++ tools/sparse/README.md | 34 +++++++++ tools/sparse/main.c | 148 ++++++++++++++++++++++++++++++++++++++++ tools/sparse/sparse-src | 1 + 6 files changed, 207 insertions(+) create mode 100644 tools/sparse/.gitignore create mode 100644 tools/sparse/Makefile create mode 100644 tools/sparse/README.md create mode 100644 tools/sparse/main.c create mode 160000 tools/sparse/sparse-src.gitmodules +3
-0@@ -1,3 +1,6 @@ [submodule "testcases/kernel/mce-test"] path = testcases/kernel/mce-test url = git://git.kernel.org/pub/scm/linux/kernel/git/gong.chen/mce-test.git +[submodule "tools/sparse/sparse-src"] + path = tools/sparse/sparse-src + url = git://git.kernel.org/pub/scm/devel/sparse/sparse.gittools/sparse/.gitignore new file +1
-0@@ -0,0 +1 @@ +maintools/sparse/Makefile new file +20
-0@@ -0,0 +1,20 @@ +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2021 SUSE LLC <[email protected]> + +top_srcdir ?= ../.. + +include $(top_srcdir)/include/mk/env_pre.mk +include $(top_srcdir)/include/mk/functions.mk + +SPARSE_SRC ?= sparse-src + +$(SPARSE_SRC)/libsparse.a: + $(error "You need to build Sparse. See tools/sparse/README.md") + +HOST_MAKE_TARGETS := main +MAKE_DEPS += $(SPARSE_SRC)/libsparse.a +HOST_CFLAGS += -I$(SPARSE_SRC) +HOST_LDLIBS += $(SPARSE_SRC)/libsparse.a + + +include $(top_srcdir)/include/mk/generic_leaf_target.mktools/sparse/README.md new file +16
-0@@ -0,0 +1,34 @@ +# Sparse based linting + +This tool checks LTP test and library code for common problems. + +## Building + +The bad news is you must get and build Sparse[^1]. The good news this only +takes a minute. + +If you already have the Sparse source code, then do the +following. Where `$SRC_PATH` is the path to the Sparse directory. + +```sh +$ cd tools/sparse +$ make -C $SRC_PATH -j$(nproc) # Optional +$ make SPARSE_SRC=$SRC_PATH
tools/sparse/README.md line 26
Quoted 10 lines
tools/sparse/README.md new file +10
-0+``` +If not then you can fetch it via the git submodule + +```sh +$ cd tools/sparse +$ git submodule update --init +$ cd sparse-src +$ make -C sparse-src -j$(nproc) +$ make +```
Can we add a build.sh script into the tools/sparse/ directory that will do exactly these steps?
Quoted 174 lines
tools/sparse/README.md new file +7
-0+## Usage + +It is integrated with the LTP build system. Just run `make check` or +`make check-a_test01`, where `a_test01` is an arbitrary test +executable or object file. + +[1]: Many distributions have a Sparse package. This only contains some executables. There is no shared librarytools/sparse/main.c new file +148
-0@@ -0,0 +1,148 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* +* Copyright (C) 2021 SUSE LLC <[email protected]> +* +* Sparse allows us to perform checks on the AST (struct symbol) or on +* a linearized representation. In the latter case we are given a set +* of entry points (functions) containing basic blocks of +* instructions. +* +* The basic blocks contain byte code in SSA form. This is similar to +* the the intermediate representation most compilers use during +* optimisation. +*/ +#include <stdarg.h> +#include <stdlib.h> +#include <stdio.h> +#include <string.h> +#include <ctype.h> +#include <unistd.h> +#include <fcntl.h> + +#include "lib.h" +#include "allocate.h" +#include "opcode.h" +#include "token.h" +#include "parse.h" +#include "symbol.h" +#include "expression.h" +#include "linearize.h" + +/* The rules for test, library and tool code are different */ +enum ltp_tu_kind { +LTP_LIB , +LTP_OTHER , +}; + +static enum ltp_tu_kind tu_kind =LTP_OTHER ; + +/* Check for LTP-002 +* +* Inspects the destination symbol of each store instruction. If it is +* TST_RET or TST_ERR then emit a warning. +*/ +static void check_lib_sets_TEST_vars (const struct instruction *insn) +{ +static struct ident *TST_RES_id, *TST_ERR_id; + +if (!TST_RES_id) { + TST_RES_id =built_in_ident ("TST_RET" ); + TST_ERR_id =built_in_ident ("TST_ERR" ); + } + +if (insn->opcode !=OP_STORE ) +return ; +if (insn->src->ident != TST_RES_id && + insn->src->ident != TST_ERR_id) +return ; + +warning (insn->pos, +"LTP-002: Library should not write to TST_RET or TST_ERR" ); +} + +static void do_basicblock_checks (struct basic_block *bb) +{ +struct instruction *insn; + + FOR_EACH_PTR(bb->insns, insn) { +if (!bb_reachable (insn->bb)) +continue ; + +if (tu_kind ==LTP_LIB ) +check_lib_sets_TEST_vars (insn); + }END_FOR_EACH_PTR (insn); +} + +static void do_entrypoint_checks (struct entrypoint *ep) +{ +struct basic_block *bb; + + FOR_EACH_PTR(ep->bbs, bb) { +do_basicblock_checks (bb); + }END_FOR_EACH_PTR (bb); +} + +/* Compile the AST into a graph of basicblocks */ +static void process_symbols (struct symbol_list *list) +{ +struct symbol *sym; + + FOR_EACH_PTR(list, sym) { +struct entrypoint *ep; + +expand_symbol (sym); + ep =linearize_symbol (sym); +if (!ep || !ep->entry) +continue ; + +do_entrypoint_checks (ep); + +if (dbg_entry) +show_entry (ep); + }END_FOR_EACH_PTR (sym); +} + +static void collect_info_from_args (const int argc,char *const *const argv) +{ +int i; + +for (i =0 ; i < argc; i++) { +if (!strcmp ("-DLTPLIB" , argv[i])) { + tu_kind =LTP_LIB ; + } + } +} + +int main (int argc,char **argv) +{ +struct string_list *filelist =NULL ; +char *file; + + Waddress_space =0 ; + Wbitwise =0 ; + Wcast_truncate =0 ; + Wcontext =0 ; + Wdecl =0 ; + Wexternal_function_has_definition =0 ; + Wflexible_array_array =0 ; + Wimplicit_int =0 ; + Wint_to_pointer_cast =0 ; + Wmemcpy_max_count =0 ; + Wnon_pointer_null =0 ; + Wone_bit_signed_bitfield =0 ; + Woverride_init =0 ; + Wpointer_to_int_cast =0 ; + Wvla =0 ; + + do_output =0 ; + +collect_info_from_args (argc, argv); + +process_symbols (sparse_initialize (argc, argv, &filelist)); + FOR_EACH_PTR(filelist, file) { +process_symbols (sparse (file)); + }END_FOR_EACH_PTR (file); + +report_stats (); +return 0 ; +}tools/sparse/sparse-src new file +1
-0@@ -0,0 +1 @@ +Subproject commit 8af2432923486c753ab52cae70b94ee684121080Signature · 5 lines
2.31.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Signature · 2 lines
Cyril Hrubis [email protected]
PATCH 1/7 Add Sparse based checker and TST_RET/ERR check
Hello Cyril,
Quoted 9 lines · Cyril Hrubis
Hi!
Quoted 2 lines
Vendors in Sparse as a git module. Then uses it to check for stores to TST_RET/ERR within the library.
This sounds reasonable enough.
It would probably be even easier to copy the sources to the LTP tree and synchronize them manually, depending on how often is sparse updated it may end up better than a submodule.
It is not updated often. Indeed Smatch did this, but I don't think that has helped adoption of either Smatch or Sparse.
Quoted 4 lines
Also isn't the submodule frozen on a given commit anyways (unless we add git submodule --remote --merge into the build steps)?
Yes. Submodules can be a pain, however I see a few advantages:
* You can easily upstream any changes made to the sub project. (no copy and paste)
* You get the full Git history without a seperate clone of upstream
* If someone already has upstream cloned they can just point the submodule towards it (not that it really matters with Sparse)
It is easy to get in a mess with submodules, but that is mainly just a concern for whoever is updating Sparse.
Quoted 86 lines
Quoted 86 lines
Signed-off-by Richard Palethorpe <[email protected]>--- .gitmodules | 3 + tools/sparse/.gitignore | 1 + tools/sparse/Makefile | 20 ++++++ tools/sparse/README.md | 34 +++++++++ tools/sparse/main.c | 148 ++++++++++++++++++++++++++++++++++++++++ tools/sparse/sparse-src | 1 + 6 files changed, 207 insertions(+) create mode 100644 tools/sparse/.gitignore create mode 100644 tools/sparse/Makefile create mode 100644 tools/sparse/README.md create mode 100644 tools/sparse/main.c create mode 160000 tools/sparse/sparse-src.gitmodules +3
-0@@ -1,3 +1,6 @@ [submodule "testcases/kernel/mce-test"] path = testcases/kernel/mce-test url = git://git.kernel.org/pub/scm/linux/kernel/git/gong.chen/mce-test.git +[submodule "tools/sparse/sparse-src"] + path = tools/sparse/sparse-src + url = git://git.kernel.org/pub/scm/devel/sparse/sparse.gittools/sparse/.gitignore new file +1
-0@@ -0,0 +1 @@ +maintools/sparse/Makefile new file +20
-0@@ -0,0 +1,20 @@ +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2021 SUSE LLC <[email protected]> + +top_srcdir ?= ../.. + +include $(top_srcdir)/include/mk/env_pre.mk +include $(top_srcdir)/include/mk/functions.mk + +SPARSE_SRC ?= sparse-src + +$(SPARSE_SRC)/libsparse.a: + $(error "You need to build Sparse. See tools/sparse/README.md") + +HOST_MAKE_TARGETS := main +MAKE_DEPS += $(SPARSE_SRC)/libsparse.a +HOST_CFLAGS += -I$(SPARSE_SRC) +HOST_LDLIBS += $(SPARSE_SRC)/libsparse.a + + +include $(top_srcdir)/include/mk/generic_leaf_target.mktools/sparse/README.md new file +21
-0@@ -0,0 +1,34 @@ +# Sparse based linting + +This tool checks LTP test and library code for common problems. + +## Building + +The bad news is you must get and build Sparse[^1]. The good news this only +takes a minute. + +If you already have the Sparse source code, then do the +following. Where `$SRC_PATH` is the path to the Sparse directory. + +```sh +$ cd tools/sparse +$ make -C $SRC_PATH -j$(nproc) # Optional +$ make SPARSE_SRC=$SRC_PATH +``` +If not then you can fetch it via the git submodule + +```sh +$ cd tools/sparse
Quoted 8 lines
tools/sparse/README.md line 26
Quoted 5 lines
tools/sparse/README.md new file +5
-0+$ git submodule update --init +$ cd sparse-src +$ make -C sparse-src -j$(nproc) +$ make +```Can we add a
build.shscript into thetools/sparse/directory that will do exactly these steps?
+1
Quoted 175 lines
Quoted 175 lines
tools/sparse/README.md new file +7
-0+## Usage + +It is integrated with the LTP build system. Just run `make check` or +`make check-a_test01`, where `a_test01` is an arbitrary test +executable or object file. + +[1]: Many distributions have a Sparse package. This only contains some executables. There is no shared librarytools/sparse/main.c new file +148
-0@@ -0,0 +1,148 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* +* Copyright (C) 2021 SUSE LLC <[email protected]> +* +* Sparse allows us to perform checks on the AST (struct symbol) or on +* a linearized representation. In the latter case we are given a set +* of entry points (functions) containing basic blocks of +* instructions. +* +* The basic blocks contain byte code in SSA form. This is similar to +* the the intermediate representation most compilers use during +* optimisation. +*/ +#include <stdarg.h> +#include <stdlib.h> +#include <stdio.h> +#include <string.h> +#include <ctype.h> +#include <unistd.h> +#include <fcntl.h> + +#include "lib.h" +#include "allocate.h" +#include "opcode.h" +#include "token.h" +#include "parse.h" +#include "symbol.h" +#include "expression.h" +#include "linearize.h" + +/* The rules for test, library and tool code are different */ +enum ltp_tu_kind { +LTP_LIB , +LTP_OTHER , +}; + +static enum ltp_tu_kind tu_kind =LTP_OTHER ; + +/* Check for LTP-002 +* +* Inspects the destination symbol of each store instruction. If it is +* TST_RET or TST_ERR then emit a warning. +*/ +static void check_lib_sets_TEST_vars (const struct instruction *insn) +{ +static struct ident *TST_RES_id, *TST_ERR_id; + +if (!TST_RES_id) { + TST_RES_id =built_in_ident ("TST_RET" ); + TST_ERR_id =built_in_ident ("TST_ERR" ); + } + +if (insn->opcode !=OP_STORE ) +return ; +if (insn->src->ident != TST_RES_id && + insn->src->ident != TST_ERR_id) +return ; + +warning (insn->pos, +"LTP-002: Library should not write to TST_RET or TST_ERR" ); +} + +static void do_basicblock_checks (struct basic_block *bb) +{ +struct instruction *insn; + + FOR_EACH_PTR(bb->insns, insn) { +if (!bb_reachable (insn->bb)) +continue ; + +if (tu_kind ==LTP_LIB ) +check_lib_sets_TEST_vars (insn); + }END_FOR_EACH_PTR (insn); +} + +static void do_entrypoint_checks (struct entrypoint *ep) +{ +struct basic_block *bb; + + FOR_EACH_PTR(ep->bbs, bb) { +do_basicblock_checks (bb); + }END_FOR_EACH_PTR (bb); +} + +/* Compile the AST into a graph of basicblocks */ +static void process_symbols (struct symbol_list *list) +{ +struct symbol *sym; + + FOR_EACH_PTR(list, sym) { +struct entrypoint *ep; + +expand_symbol (sym); + ep =linearize_symbol (sym); +if (!ep || !ep->entry) +continue ; + +do_entrypoint_checks (ep); + +if (dbg_entry) +show_entry (ep); + }END_FOR_EACH_PTR (sym); +} + +static void collect_info_from_args (const int argc,char *const *const argv) +{ +int i; + +for (i =0 ; i < argc; i++) { +if (!strcmp ("-DLTPLIB" , argv[i])) { + tu_kind =LTP_LIB ; + } + } +} + +int main (int argc,char **argv) +{ +struct string_list *filelist =NULL ; +char *file; + + Waddress_space =0 ; + Wbitwise =0 ; + Wcast_truncate =0 ; + Wcontext =0 ; + Wdecl =0 ; + Wexternal_function_has_definition =0 ; + Wflexible_array_array =0 ; + Wimplicit_int =0 ; + Wint_to_pointer_cast =0 ; + Wmemcpy_max_count =0 ; + Wnon_pointer_null =0 ; + Wone_bit_signed_bitfield =0 ; + Woverride_init =0 ; + Wpointer_to_int_cast =0 ; + Wvla =0 ; + + do_output =0 ; + +collect_info_from_args (argc, argv); + +process_symbols (sparse_initialize (argc, argv, &filelist)); + FOR_EACH_PTR(filelist, file) { +process_symbols (sparse (file)); + }END_FOR_EACH_PTR (file); + +report_stats (); +return 0 ; +}tools/sparse/sparse-src new file +1
-0@@ -0,0 +1 @@ +Subproject commit 8af2432923486c753ab52cae70b94ee684121080Signature · 5 lines
2.31.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Signature · 2 lines
Thank you, Richard.
PATCH 2/7 Add 'make check' to the build system
In reply to Richard PalethorpeAllows one to run 'make check' or 'make check-$TARGET'. Which will execute the CHECK tool with that target's CC arguments and CHECK_FLAGS. By default the check tool is tools/sparse/main.
--- Makefile | 8 ++++++++ include/mk/env_post.mk | 8 ++++++++ include/mk/generic_leaf_target.inc | 5 ++++- include/mk/generic_trunk_target.inc | 7 ++++++- include/mk/lib.mk | 3 +++ include/mk/module.mk | 2 ++ include/mk/rules.mk | 9 +++++++++ include/mk/sparse.mk | 9 +++++++++ include/mk/testcases.mk | 1 + testcases/open_posix_testsuite/Makefile | 4 ++++ 10 files changed, 54 insertions(+), 2 deletions(-) create mode 100644 include/mk/sparse.mk
Makefile +8 -0
@@ -79,6 +79,7 @@ BOOTSTRAP_TARGETS := $(sort $(COMMON_TARGETS) $(CLEAN_TARGETS) $(INSTALL_TARGETS CLEAN_TARGETS := $(addsuffix -clean,$(CLEAN_TARGETS)) INSTALL_TARGETS := $(addsuffix -install,$(INSTALL_TARGETS)) MAKE_TARGETS := $(addsuffix -all,$(filter-out lib,$(COMMON_TARGETS))) +CHECK_TARGETS := $(addsuffix -check,testcases lib) # There's no reason why we should run `all' twice. Otherwise we're just wasting # 3+ mins of useful CPU cycles on a modern machine, and even more time on an
@@ -108,6 +109,10 @@ $(MAKE_TARGETS) include-all lib-all libs-all: $(MAKE) -C "$(subst -all,,$@)" \ -f "$(abs_top_srcdir)/$(subst -all,,$@)/Makefile" all +$(CHECK_TARGETS): tools-all + $(MAKE) -C "$(subst -check,,$@)" \ + -f "$(abs_top_srcdir)/$(subst -check,,$@)/Makefile" check + # Let's not conflict with ac-clean, maintainer-clean, etc, so. $(filter-out include-clean,$(CLEAN_TARGETS)):: -$(MAKE) -C "$(subst -clean,,$@)" \
@@ -189,6 +194,9 @@ INSTALL_TARGETS += $(addprefix $(DESTDIR)/$(bindir)/,$(BINDIR_INSTALL_SCRIPTS)) $(INSTALL_TARGETS): $(INSTALL_DIR) $(DESTDIR)/$(bindir) +.PHONY: check +check: $(CHECK_TARGETS) + ## Install install: $(INSTALL_TARGETS)
include/mk/env_post.mk +8 -0
@@ -89,6 +89,14 @@ $(error You must define $$(prefix) before executing install) endif # END $(filter-out install,$(MAKECMDGOALS)),$(MAKECMDGOALS) endif +CHECK_TARGETS ?= $(addprefix check-,$(notdir $(patsubst %.c,%,$(sort $(wildcard $(abs_srcdir)/*.c))))) +CHECK_TARGETS := $(filter-out $(addprefix check-, $(FILTER_OUT_MAKE_TARGETS)), $(CHECK_TARGETS)) +CHECK ?= $(abs_top_srcdir)/tools/sparse/main + +ifeq ($(dir $(CHECK)),$(abs_top_srcdir)/tools/sparse/main) +CHECK_DEPS += $(CHECK) +endif + include $(top_srcdir)/include/mk/rules.mk endif
include/mk/generic_leaf_target.inc +4 -1
@@ -92,7 +92,7 @@ # INSTALL_DIR := $(libdir) # -.PHONY: all clean install +.PHONY: all clean install check ifneq ($(strip $(MAKE_TARGETS)),) $(MAKE_TARGETS) += $(HOST_MAKE_TARGETS)
@@ -109,4 +109,7 @@ $(INSTALL_FILES): | $(INSTALL_DEPS) install: $(INSTALL_FILES) +$(CHECK_TARGETS): | $(CHECK_DEPS) +check: $(CHECK_TARGETS) + # vim: syntax=make
include/mk/generic_trunk_target.inc +6 -1
@@ -48,7 +48,7 @@ include $(top_srcdir)/include/mk/functions.mk -RECURSIVE_TARGETS ?= all install +RECURSIVE_TARGETS ?= all install check $(eval $(get_make_dirs))
@@ -68,6 +68,9 @@ $(INSTALL_FILES): | $(INSTALL_DEPS) trunk-install: $(INSTALL_FILES) +$(CHECK_TARGETS): | $(CHECK_DEPS) +trunk-check: $(CHECK_TARGETS) + # Avoid creating duplicate .PHONY references to all, clean, and install. IIRC, # I've seen some indeterministic behavior when one does this in the past with # GNU Make...
@@ -108,4 +111,6 @@ else endif endif +check: trunk-check + # vim: syntax=make
include/mk/lib.mk +3 -0
@@ -26,6 +26,7 @@ # Makefile to include for libraries. include $(top_srcdir)/include/mk/env_pre.mk +include $(top_srcdir)/include/mk/sparse.mk INSTALL_DIR := $(libdir)
@@ -57,6 +58,8 @@ LIBSRCS := $(filter-out $(FILTER_OUT_LIBSRCS),$(LIBSRCS)) LIBOBJS := $(LIBSRCS:.c=.o) +CHECK_TARGETS := $(addprefix check-,$(notdir $(LIBSRCS:.c=))) + $(LIB): $(notdir $(LIBOBJS)) @if [ -z "$(strip $^)" ] ; then \ echo "Cowardly refusing to create empty archive"; \
include/mk/module.mk +2 -0
@@ -47,6 +47,8 @@ endif CLEAN_TARGETS += .dep_modules *.mod built-in.a +CHECK_TARGETS := $(filter-out %.ko, $(CHECK_TARGETS)) + MODULE_SOURCES := $(patsubst %.ko,%.c,$(filter %.ko, $(MAKE_TARGETS))) # Ignoring the exit status of commands is done to be forward compatible with
include/mk/rules.mk +9 -0
@@ -37,3 +37,12 @@ else @$(CC) $(CPPFLAGS) $(CFLAGS) $(LDFLAGS) $^ $(LTPLDLIBS) $(LDLIBS) -o $@ @echo CC $(target_rel_dir)$@ endif + +.PHONY: $(CHECK_TARGETS) +$(CHECK_TARGETS): check-%: %.c +ifdef VERBOSE + $(CHECK) $(CHECK_FLAGS) $(CPPFLAGS) $(CFLAGS) $< +else + @$(CHECK) $(CHECK_FLAGS) $(CPPFLAGS) $(CFLAGS) $< + @echo CHECK $(target_rel_dir)$< +endif
include/mk/sparse.mk new file +9 -0
@@ -0,0 +1,9 @@ +# Rules to make sparse tool(s) for inclusion in lib and testcases Makefiles + +SPARSE_DIR:= $(abs_top_builddir)/tools/sparse + +$(SPARSE_DIR)/main: $(SPARSE_DIR) + $(MAKE) -C "$^" all + +$(SPARSE_DIR): %: + mkdir -p "$@"
include/mk/testcases.mk +1 -0
@@ -22,6 +22,7 @@ include $(top_srcdir)/include/mk/env_pre.mk include $(top_srcdir)/include/mk/functions.mk +include $(top_srcdir)/include/mk/sparse.mk APICMDS_DIR := $(abs_top_builddir)/tools/apicmds
testcases/open_posix_testsuite/Makefile +4 -0
@@ -104,3 +104,7 @@ $(CRITICAL_MAKEFILE): \ $(top_srcdir)/LDFLAGS \ $(top_srcdir)/LDLIBS @$(MAKE) generate-makefiles + +.PHONY: check +check: + @echo "Checker not yet supported by Open POSIX testsuite"
Signature · 1 line
2.31.1
PATCH 3/7 meltdown: Set CFLAGS for check targets
In reply to Richard PalethorpeThe include set in the CFLAGS is necessary for Sparse to parse the test source. If a specific target has variables set on it. Then these won't be available to the corresponding check-target. Usually this is not a problem because most variables set on a target are just for linking and the checker doesn't need them.
--- testcases/cve/Makefile | 1 + 1 file changed, 1 insertion(+)
testcases/cve/Makefile +1 -0
@@ -19,6 +19,7 @@ cve-2014-0196: LDLIBS += -lutil endif meltdown: CFLAGS += -I$(abs_srcdir)/../realtime/include +check-meltdown: CFLAGS += -I$(abs_srcdir)/../realtime/include ifneq (,$(filter $(HOST_CPU),x86 x86_64)) meltdown: CFLAGS += -msse2
Signature · 1 line
2.31.1
PATCH 3/7 meltdown: Set CFLAGS for check targets
Hi!
Quoted 16 lines
The include set in the CFLAGS is necessary for Sparse to parse the test source. If a specific target has variables set on it. Then these won't be available to the corresponding check-target. Usually this is not a problem because most variables set on a target are just for linking and the checker doesn't need them.
Signed-off-by Richard Palethorpe <[email protected]>--- testcases/cve/Makefile | 1 + 1 file changed, 1 insertion(+)testcases/cve/Makefile +0
-0
testcases/cve/Makefile cve-2014-0196: LDLIBS += -lutil · line 22
Quoted 4 lines
testcases/cve/Makefile +1
-0@@ -19,6 +19,7 @@ cve-2014-0196: LDLIBS += -lutil endif meltdown: CFLAGS += -I$(abs_srcdir)/../realtime/include +check-meltdown: CFLAGS += -I$(abs_srcdir)/../realtime/include
Uff, this will likely bite us in a different ways later on.
I guess that it would be cleaner to move the libtsc.h into top level include so that it's clear that the header is used in several tests.
Signature · 2 lines
Cyril Hrubis [email protected]
PATCH 3/7 meltdown: Set CFLAGS for check targets
Hello,
Quoted 19 lines · Cyril Hrubis
Hi!
Quoted 18 lines
The include set in the CFLAGS is necessary for Sparse to parse the test source. If a specific target has variables set on it. Then these won't be available to the corresponding check-target. Usually this is not a problem because most variables set on a target are just for linking and the checker doesn't need them.
Signed-off-by Richard Palethorpe <[email protected]>--- testcases/cve/Makefile | 1 + 1 file changed, 1 insertion(+)testcases/cve/Makefile +0
-0@@ -19,6 +19,7 @@ cve-2014-0196: LDLIBS += -lutil endif
Quoted 7 lines
testcases/cve/Makefile cve-2014-0196: LDLIBS += -lutil · line 22
Quoted 2 lines
testcases/cve/Makefile +1
-0meltdown: CFLAGS += -I$(abs_srcdir)/../realtime/include +check-meltdown: CFLAGS += -I$(abs_srcdir)/../realtime/includeUff, this will likely bite us in a different ways later on.
I guess that it would be cleaner to move the
libtsc.hinto top level include so that it's clear that the header is used in several tests.
+1
Signature · 2 lines
Thank you, Richard.
PATCH 4/7 doc: Add rules and recommendations list
In reply to Richard PalethorpeCreate a list of rules which are feasible to check automatically. The file is a table of tab separated values. It's intended to be machine readable.
For now there is just an ID column, with IDs that look similar to CWE IDs or CVEs. For now these can just be used with 'git grep'. Also there is a description column.
Clearly this is not an exhaustive list. It just contains a library rule already stated in the guide and the issue Sparse checks for.
--- doc/rules.tsv | 3 +++ doc/test-writing-guidelines.txt | 5 +++++ 2 files changed, 8 insertions(+) create mode 100644 doc/rules.tsv
doc/rules.tsv new file +3 -0
@@ -0,0 +1,3 @@ +ID DESCRIPTION +LTP-001 Library source files have tst_ prefix +LTP-002 TST_RET and TST_ERR are never modified by test library functions
doc/test-writing-guidelines.txt +5 -0
@@ -10,6 +10,11 @@ NOTE: See also https://github.com/linux-test-project/ltp/wiki/Shell-Test-API[Shell Test API], https://github.com/linux-test-project/ltp/wiki/LTP-Library-API-Writing-Guidelines[LTP Library API Writing Guidelines]. +Rules and recommendations which are "machine checkable" should be +tagged with an ID like +LTP-XXX+. There will be a corresponding entry +in 'doc/rules.tsv'. When you run 'make check' or 'make check-test' it +will display these IDs as a reference. + 1. Guide to clean and understandable code ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Signature · 1 line
2.31.1
PATCH 4/7 doc: Add rules and recommendations list
Hi!
The idea looks reasonable to me.
Signature · 2 lines
Cyril Hrubis [email protected]
PATCH 5/7 doc: Remind authors and maintainers to run make check
In reply to Richard Palethorpe--- doc/maintainer-patch-review-checklist.txt | 2 +- doc/test-writing-guidelines.txt | 1 + 2 files changed, 2 insertions(+), 1 deletion(-)
doc/maintainer-patch-review-checklist.txt +1 -1
@@ -39,7 +39,7 @@ New test should * Use new https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#22-writing-a-test-in-c[C API] * Test binaries are added into corresponding '.gitignore' files * Check coding style with - https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plain/scripts/checkpatch.pl[checkpatch.pl] + https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plain/scripts/checkpatch.pl[checkpatch.pl] and `make check` (more in https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#131-c-coding-style[C coding style]) * Docparse documentation * If a test is a regression test it should include tags
doc/test-writing-guidelines.txt +1 -0
@@ -308,6 +308,7 @@ NOTE: See also 3. The runtest entires are in place 4. Test binaries are added into corresponding '.gitignore' files 5. Patches apply over the latest git +6. 'make check' does not emit any warnings 6.1 About .gitignore files ~~~~~~~~~~~~~~~~~~~~~~~~~~
Signature · 1 line
2.31.1
PATCH 5/7 doc: Remind authors and maintainers to run make check
Hi!
Looks obviously OK if we merge the rest.
Signature · 2 lines
Cyril Hrubis [email protected]
PATCH 6/7 doc: Document TEST macro and state TST_RET/ERR rule LTP-002
In reply to Richard PalethorpeThere are cases where these variables can be safely used by the library. However it is a difficult problem to identify these cases automatically.
Identifying any library code which uses them is relatively easy. In fact it is easier to automate it than by code review. Because it is such a boring thing to repeatedly look for and comment on.
If a test library function needs these variables it can recreate its own private copies.
--- doc/c-test-api.txt | 47 ++++++++++++++++++++++++++ doc/library-api-writing-guidelines.txt | 14 ++++++++ 2 files changed, 61 insertions(+)
doc/c-test-api.txt +47 -0
@@ -754,6 +754,53 @@ LTP_ALIGN(x, a) Aligns the x to be next multiple of a. The a must be power of 2. +[source,c] +------------------------------------------------------------------------------- +TEST(socket(AF_INET, SOCK_RAW, 1)); +if (TST_RET > -1) { + tst_res(TFAIL, "Created raw socket"); + SAFE_CLOSE(TST_RET); +} else if (TST_ERR != EPERM) { + tst_res(TFAIL | TTERRNO, + "Failed to create socket for wrong reason"); +} else { + tst_res(TPASS | TTERRNO, "Didn't create raw socket"); +} +------------------------------------------------------------------------------- + +The +TEST+ macro sets +TST_RET+ to its argument's return value and ++TST_ERR+ to +errno+. The +TTERNO+ flag can be used to print the error +number's symbolic value. + +No LTP library function or macro, except those in 'tst_test_macros.h', +will write to these variables (rule 'LTP-002'). So their values will +not be changed unexpectedly. + +[source,c] +------------------------------------------------------------------------------- +TST_EXP_POSITIVE(wait(&status)); + +if (!TST_PASS) + return; +------------------------------------------------------------------------------- + +If the return value of 'wait' is positive. This macro will print a +pass result and set +TST_PASS+ appropriately. If the return value is +zero or negative, then it will print fail. + +This and similar macros take optional variadic arguments. These begin +with a format string and then appropriate values to be formatted. + +[source,c] +------------------------------------------------------------------------------- +TST_EXP_PASS(chown("a/file", uid, gid), "chown(%s,%d,%d)", + "a/file", uid, gid); +------------------------------------------------------------------------------- + +Expects +chown+ to return 0 and emits a pass or a fail. The arguments +to +chown+ will be printed in either case. There are many similar +macros, please see 'tst_test_macros.h'. + 1.13 Filesystem type detection and skiplist ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
doc/library-api-writing-guidelines.txt +14 -0
@@ -21,10 +21,24 @@ Don't forget to update docs when you change the API. 2. C API -------- +2.1 LTP-001: Sources have tst_ prefix +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + API source code is in headers `include/*.h`, `include/lapi/*.h` (backward compatibility for old kernel and libc) and C sources in `lib/*.c`. Files have 'tst_' prefix. +2.2 LTP-002: TST_RET and TST_ERR are not modified +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The test author is guaranteed that the test API will not modify these +variables. This prevents silent errors where the return value and +errno are overwritten before the test has chance to check them. + +The macros which are clearly intended to update these variables. That +is +TEST+ and those in 'tst_test_macros.h'. Are of course allowed to +update these variables. + 3. Shell API ------------
Signature · 1 line
2.31.1
PATCH 6/7 doc: Document TEST macro and state TST_RET/ERR rule LTP-002
Quoted 37 lines · Richard Palethorpe via ltp
+[source,c]
+-------------------------------------------------------------------------------doc/c-test-api.txt +34
-0+TEST(socket(AF_INET, SOCK_RAW, 1)); +if (TST_RET > -1) { + tst_res(TFAIL, "Created raw socket"); + SAFE_CLOSE(TST_RET); +} else if (TST_ERR != EPERM) { + tst_res(TFAIL | TTERRNO, + "Failed to create socket for wrong reason"); +} else { + tst_res(TPASS | TTERRNO, "Didn't create raw socket"); +} +------------------------------------------------------------------------------- + +The +TEST+ macro sets +TST_RET+ to its argument's return value and ++TST_ERR+ to +errno+. The +TTERNO+ flag can be used to print the error +number's symbolic value. + +No LTP library function or macro, except those in 'tst_test_macros.h', +will write to these variables (rule 'LTP-002'). So their values will +not be changed unexpectedly. + +[source,c] +------------------------------------------------------------------------------- +TST_EXP_POSITIVE(wait(&status)); + +if (!TST_PASS) + return; +------------------------------------------------------------------------------- + +If the return value of 'wait' is positive. This macro will print a +pass result and set +TST_PASS+ appropriately. If the return value is +zero or negative, then it will print fail. + +This and similar macros take optional variadic arguments. These begin +with a format string and then appropriate values to be formatted.
doc/c-test-api.txt LTP_ALIGN(x, a) · line 802
Quoted 10 lines
doc/c-test-api.txt +10
-0+ +[source,c] +------------------------------------------------------------------------------- +TST_EXP_PASS(chown("a/file", uid, gid), "chown(%s,%d,%d)", + "a/file", uid, gid); +------------------------------------------------------------------------------- + +Expects +chown+ to return 0 and emits a pass or a fail. The arguments +to +chown+ will be printed in either case. There are many similar +macros, please see 'tst_test_macros.h'.
It appears that TST_EXP* macros (but not TEST) are documented elsewhere. So I should probably combine these comments with the existing docs.
Signature · 2 lines
Thank you, Richard.
PATCH 7/7 Reference LTP-002 rule in Cocci scripts
In reply to Richard PalethorpeAvoid duplication of rule description
--- scripts/coccinelle/libltp-test-macro-vars.cocci | 6 +----- scripts/coccinelle/libltp-test-macro.cocci | 4 +--- 2 files changed, 2 insertions(+), 8 deletions(-)
scripts/coccinelle/libltp-test-macro-vars.cocci +1 -5
@@ -1,11 +1,7 @@ // SPDX-License-Identifier: GPL-2.0-or-later // Copyright (c) 2021 SUSE LLC <[email protected]> -// The TEST macro should not be used in the library because it sets -// TST_RET and TST_ERR which are global variables. The test author -// only expects these to be changed if *they* call TEST directly. - -// Find all positions where TEST's variables are used +// Find violations of LTP-002 @ find_use exists @ expression E; @@
scripts/coccinelle/libltp-test-macro.cocci +1 -3
@@ -1,9 +1,7 @@ // SPDX-License-Identifier: GPL-2.0-or-later // Copyright (c) 2021 SUSE LLC <[email protected]> -// The TEST macro should not be used in the library because it sets -// TST_RET and TST_ERR which are global variables. The test author -// only expects these to be changed if *they* call TEST directly. +// Find and fix violations of rule LTP-002 // Set with -D fix virtual fix
Signature · 1 line
2.31.1
cover Sparse based checker and rule proposal
In reply to Richard PalethorpeHi!
Generally looks very good, I guess that it could be merged once we figure out how exactly we should vendor sparse checker.
Signature · 2 lines
Cyril Hrubis [email protected]