coccinelle: extend testing with check-pycocci
cover coccinelle: extend testing with check-pycocci
Coccinelle has its own test framework built-in to spatch, this is used to test against small tests cases written in tests/*.c with their respective tests/*.cocci and tests/*.res file. This is however a bit limited as it only handles single C file. In real world projects where more files are used we need to support testing against larger series of file structures.
pycocci already had SmPL <=> Patch equivalence support, we can leverage this to extend testing in a more generic fashion. This does that work.
For now, we disable pycocci-tests given we touch your git tree for these tests.
Luis R. Rodriguez (5):
pycocci: use os.path.abspath() for gitname
pycocci: add git branch support to git library
pycocci: skip proof if git dir is dirty
pycocci: add clean version of SmPL <=> Patch equivalence proof support
pycocci: enable SmPL <=> Patch equivalence tests
Makefile | 3 + docs/pycocci.1 | 24 +++++- scripts/pycocci-check.sh | 8 ++ tests/pycocci/code/0001-proto3/proto3.c | 7 ++ tests/pycocci/code/0001-proto3/proto3.h | 3 + tests/pycocci/patches/0001-proto3.cocci | 11 +++ .../0001-proto3/0001-change-prototypes.patch | 18 +++++ tools/pycocci | 89 ++++++++++++++++------ 8 files changed, 140 insertions(+), 23 deletions(-) create mode 100755 scripts/pycocci-check.sh create mode 100644 tests/pycocci/code/0001-proto3/proto3.c create mode 100644 tests/pycocci/code/0001-proto3/proto3.h create mode 100644 tests/pycocci/patches/0001-proto3.cocci create mode 100644 tests/pycocci/patches/0001-proto3/0001-change-prototypes.patch
Signature · 1 line
2.7.2
PATCH 1/5 pycocci: use os.path.abspath() for gitname
gitname() is currently used for SmPL <=> Patch equivalence proof support. It uses git to test for proofs and also show any differences found. If the target directory passed where the user wishes to do work on has is not part of a git tree, it creates a git tree for you there.
When using relative paths the check for if a target directory is part of a git tree currently always fails, the consequences of which are that if you try to show proof on an existing git tree you'll end up with a sub-git project, and as such a new .git under some directory under your target path.
Correct this by using absolute paths.
--- tools/pycocci | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
tools/pycocci +2 -1
@@ -246,9 +246,10 @@ def git_rev_parse(tree=None, extra_args=None): return stdout.split('\n', 1)[0] def gitname(path=None): + path = os.path.abspath(path) work_dir = path if not os.path.isdir(path): - work_dir = os.path.dirname(path) + work_dir = os.path.abspath(os.path.dirname(path)) process = subprocess.Popen(['git', 'rev-parse', '--show-toplevel', path], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, close_fds=True, universal_newlines=True, cwd=work_dir)
Signature · 1 line
2.7.2
PATCH 2/5 pycocci: add git branch support to git library
In reply to Luis R. RodriguezThis will be used later on an new command for pycocci.
--- tools/pycocci | 9 +++++++++ 1 file changed, 9 insertions(+)
tools/pycocci +9 -0
@@ -276,6 +276,15 @@ def git_checkout(tree=None, extra_args=None): process.wait() _check(process) +def git_branch(tree=None, extra_args=None): + cmd = ['git', 'branch' ] + extra_args + process = subprocess.Popen(cmd, + stdout=subprocess.PIPE, stderr=subprocess.STDOUT, + close_fds=True, universal_newlines=True, cwd=tree) + stdout = process.communicate()[0] + process.wait() + _check(process) + def git_commit_all(message, tree=None): git_add('.', tree=tree) process = subprocess.Popen(['git', 'commit', '--allow-empty', '-a', '-m', message],
Signature · 1 line
2.7.2
PATCH 3/5 pycocci: skip proof if git dir is dirty
In reply to Luis R. RodriguezSmPL <=> Patch equivalence proof support requires your git tree to be clean, if it is dirty it will fail. To avoid confusing users complain if their tree is dirty.
You will either need commit any local pending changes or discard them before proceeding with SmPL <=> Patch equivalence proof support.
Extend documentation for this.
--- docs/pycocci.1 | 3 +++ tools/pycocci | 9 +++++++++ 2 files changed, 12 insertions(+)
docs/pycocci.1 +3 -0
@@ -201,6 +201,9 @@ will set the tree back to the master branch. Each run of \fBpycocci --show-proof will create two new git branches. 8 random characters are postixed to each new git branch created to avoid conflicts with previous runs. +You must run this option with a clean git tree, if you have any pending changes +you must commit them or discard them. + .SH AUTHOR \fBpycocci\fP and this man page was written by Luis R. Rodriguez <[email protected]>
tools/pycocci +9 -0
@@ -575,6 +575,15 @@ def _main(): logwrite("Path (%s) not part of a git tree, creating one for you...\n" % (args.target_dir)) git_init(tree=args.target_dir) git_commit_all(tree=args.target_dir, message="Initial commit") + else: + diff_stat = git_diff(tree=args.target_dir, extra_args = [ '--stat' ] ) + if len(diff_stat) != 0: + logwrite("SmPL <--> Patch proof equivalence requires a " + "clean tree.\nYou have a dirty tree with " + "uncommitted changes:\n\n") + logwrite(diff_stat) + logwrite("\nEither commit these changes, or stash them away somewhere\n") + return -2 cmd = [ '--abbrev-ref', 'HEAD' ] current_branch = git_rev_parse(tree=args.target_dir, extra_args = cmd) logwrite("\n")
Signature · 1 line
2.7.2
PATCH 4/5 pycocci: add clean version of SmPL <=> Patch equivalence proof support
In reply to Luis R. RodriguezWhen working with automated tests against proofs you expect mosts tests to pass. Currently SmPL <=> Patch equivalence proof support is too chatty for automated tests, and it also leaves in place the two git branches it uses to test for proof. Add a new option for pycocci which will be silent unless errors occur, and will also remove the temporary git branches used for the proof unless an error is found.
Extend documentation for this. Note that these tests use git branches on your tree, pycocc-* branches are only left in your tree if an error has occured. Even if successful, even if branches are deleted there will be a period of time during which you will still see these branches on your git reflog, by default this is about 30 days. If you want to expedite clearing your reflog you can use this after testing:
git reflog expire --all --expire=now
--- docs/pycocci.1 | 21 +++++++++++++++++- tools/pycocci | 68 ++++++++++++++++++++++++++++++++++++++++------------------ 2 files changed, 67 insertions(+), 22 deletions(-)
docs/pycocci.1 +20 -1
@@ -204,8 +204,27 @@ git branch created to avoid conflicts with previous runs. You must run this option with a clean git tree, if you have any pending changes you must commit them or discard them. +.B\-c | \-\-clean\-proof +This does what -\-show\-proof does but this is completely silent unless an +error occurs. It will also remove the git branches used to test for the +equivalence proof, unless an error is found. If an error is found you can +inspect the two branches used to test for proof, refer to the documentation +on \-\-show\-proof for details about these branches. This option is useful +if you want to automate tests with proofs. + +Note that using this method will have created and subsequently if successful +deleted two git branches on your git tree. As a consequence of using git +branches your git reflog will show these branches, if you push your tree +out these branches will not be pushed as they were deleted, your reflog +however will keep these references locally until git expires them, by default +this is 30 days. If this is too chatty for you, you can run: + + git reflog expire \-\-all \-\-expire=now + +This will immediately clear old entries from your reflog. + .SH AUTHOR -\fBpycocci\fP and this man page was written by Luis R. Rodriguez <[email protected]> +\fBpycocci\fP and this man page was written by Luis R. Rodriguez <[email protected]> .SH REPORTING BUGS Send an mail to <[email protected]>
tools/pycocci +47 -21
146 lines not on this page · raw message
Signature · 1 line
2.7.2
PATCH 5/5 pycocci: enable SmPL <=> Patch equivalence tests
In reply to Luis R. RodriguezCoccinelle has integrated tests support into it, these tests are performed really fast, the current test framework enables testing with respective single C file and a respective cocci file, ie: foo.c, foo.cocci, foo.res. If one wants to add extensive tests, to reflect more real-life sized projects the current test scheme is a bit limited.
We can extend Coccinelle's existing test framework to support multiple files, header files, but a different can also consist of making use of SmPL <=> Patch equivalence support. Instead of relying on .res files, this relies on patches that provide a outline of differences using standard diff. This enables extending test coverage to more complex cases. This is not meant to replace Coccinelle's internal test functionality but rather complement it.
For now, since I can't figure out how to only remove from the reflog the temporary branch creation stuff without expiring your entire relog immediately, just keep the pycocci-check separate from the standard tests. If we're happy to have a chatty reflog when testing we can later enable this by default, but this just doesn't seem appropriate as a default setting.
--- Makefile | 3 +++ scripts/pycocci-check.sh | 8 ++++++++ tests/pycocci/code/0001-proto3/proto3.c | 7 +++++++ tests/pycocci/code/0001-proto3/proto3.h | 3 +++ tests/pycocci/patches/0001-proto3.cocci | 11 +++++++++++ .../patches/0001-proto3/0001-change-prototypes.patch | 18 ++++++++++++++++++ 6 files changed, 50 insertions(+) create mode 100755 scripts/pycocci-check.sh create mode 100644 tests/pycocci/code/0001-proto3/proto3.c create mode 100644 tests/pycocci/code/0001-proto3/proto3.h create mode 100644 tests/pycocci/patches/0001-proto3.cocci create mode 100644 tests/pycocci/patches/0001-proto3/0001-change-prototypes.patch
Makefile +3 -0
@@ -489,6 +489,9 @@ check: scripts/spatch @$(ECHO) running the test suite COCCINELLE_HOME="$$(pwd)" ./scripts/spatch --testall --no-update-score-file +pycocci-check: + COCCINELLE_HOME="$$(pwd)" ./scripts/pycocci-check.sh + # -inline 0 to see all the functions in the profile. # Can also use the profile framework in commons/ and run your program # with -profile.
scripts/pycocci-check.sh new file +8 -0
@@ -0,0 +1,8 @@ +#!/bin/bash +set -e + +for i in tests/pycocci/patches/*.cocci; do + TEST=${i%%.cocci} + CODE_TEST=tests/pycocci/code/$(basename $TEST) + ./tools/pycocci --clean-proof $i $CODE_TEST +done
tests/pycocci/code/0001-proto3/proto3.c new file +7 -0
@@ -0,0 +1,7 @@ +#include "proto3.h" + +static void bch_sched_event (struct BCState *bcs,int event) +{ + bcs->event |=1 << event; +schedule_work (&bcs->work); +}
tests/pycocci/code/0001-proto3/proto3.h new file +3 -0
@@ -0,0 +1,3 @@ +void bch_l2l1 (struct PStack *st,int pr,void *arg); +void bch_sched_event (struct BCState *bcs,int event); +void bch_empty_fifo (struct BCState *bcs,int count);
tests/pycocci/patches/0001-proto3.cocci new file +11 -0
@@ -0,0 +1,11 @@ +@@ +identifier fn2; +identifier bcs, ev; +@@ + +- fn2(struct BCState *bcs, int ev) { ++ fn2(int ev) { + ... + bcs->event |= 1 << ev; + schedule_work(&bcs->work); + }
tests/pycocci/patches/0001-proto3/0001-change-prototypes.patch new file +18 -0
@@ -0,0 +1,18 @@ +--- a/proto3.c ++++ b/proto3.c +@@ -1,7 +1,6 @@ + #include "proto3.h" + +-static void bch_sched_event(struct BCState *bcs, int event) +-{ ++static void bch_sched_event(int event) { + bcs->event |= 1 << event; + schedule_work(&bcs->work); + } +--- a/proto3.h ++++ b/proto3.h +@@ -1,3 +1,3 @@ + void bch_l2l1(struct PStack *st, int pr, void *arg); +-void bch_sched_event(struct BCState *bcs, int event); ++void bch_sched_event(int event); + void bch_empty_fifo(struct BCState *bcs, int count);
Signature · 1 line
2.7.2
PATCH 5/5 pycocci: enable SmPL <=> Patch equivalence tests
Quoted 5 lines
We can extend Coccinelle's existing test framework to support multiple files, header files, but a different can also consist of making use of SmPL <=> Patch equivalence support. Instead of relying on .res files, this relies on patches that provide a outline of differences using standard diff.
Does this wording need a bit more fine-tuning to make the understanding easier for the suggested software development?
Regards,
Markus
cover coccinelle: extend testing with check-pycocci
In reply to Luis R. Rodriguezfor all of the patches in this series.
julia
Quoted 44 lines · Luis R. Rodriguez
Coccinelle has its own test framework built-in to spatch, this is used to test against small tests cases written in tests/*.c with their respective tests/*.cocci and tests/*.res file. This is however a bit limited as it only handles single C file. In real world projects where more files are used we need to support testing against larger series of file structures.
pycocci already had SmPL <=> Patch equivalence support, we can leverage this to extend testing in a more generic fashion. This does that work.
For now, we disable pycocci-tests given we touch your git tree for these tests.
Luis R. Rodriguez (5):
pycocci: useos.path.abspath()for gitname
pycocci: add git branch support to git library
pycocci: skip proof if git dir is dirty
pycocci: add clean version of SmPL <=> Patch equivalence proof support
pycocci: enable SmPL <=> Patch equivalence testsMakefile | 3 + docs/pycocci.1 | 24 +++++- scripts/pycocci-check.sh | 8 ++ tests/pycocci/code/0001-proto3/proto3.c | 7 ++ tests/pycocci/code/0001-proto3/proto3.h | 3 + tests/pycocci/patches/0001-proto3.cocci | 11 +++ .../0001-proto3/0001-change-prototypes.patch | 18 +++++ tools/pycocci | 89 ++++++++++++++++------ 8 files changed, 140 insertions(+), 23 deletions(-) create mode 100755 scripts/pycocci-check.sh create mode 100644 tests/pycocci/code/0001-proto3/proto3.c create mode 100644 tests/pycocci/code/0001-proto3/proto3.h create mode 100644 tests/pycocci/patches/0001-proto3.cocci create mode 100644 tests/pycocci/patches/0001-proto3/0001-change-prototypes.patchSignature · 6 lines
2.7.2 _______________________________________________ Cocci mailing list Cocci at systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Luis R. Rodriguez
Appliedcover coccinelle: extend testing with check-pycocci
Quoted 3 lines · Julia Lawall
Acked-by Julia Lawall <[email protected]>for all of the patches in this series.
Thanks, applied and pushed.
Luis