coccinelle: extend test coverage
cover coccinelle: extend test coverage
I wanted to extend tests coverage for coccinelle but noticed that tests are limited to single c file, these are attached to respective .res files, but this also means that you have a direct relationship and requirement between tests from a single C file and a respective cocci file.
To go wild we want to be able to support really complex types of examples. Fortunatley the backports project has been using Coccinelle extensively daily, when a backports developers needed to replace a series of patches with a Coccinelle patch pycocci was extended to provide SmPL <--> Patch equivalence support. We can easily take advantage of this to extend Coccinelle test cases way beyond a single C file, but to more complex real life style projects.
I don't think we should convert all test cases to this new mechanism but, I'd like to see what are folks' thoughts over using pycocci SmPL <--> Patch equivalence proof support to help expand the tests cases for Coccinelle.
Luis R. Rodriguez (2):
pycocci: add missing new line
tests: convert sp.cocci test to pycocci proof
Makefile | 1 +
scripts/test.sh | 8 ++++++++
tests/{sp.cocci => 0001-sp.cocci} | 0
tests/0001-sp/replace-typedef.patch | 9 +++++++++
tests/{ => code/0001-sp}/sp.c | 0
tests/sp.res | 4 ----
tools/pycocci | 2 +-
7 files changed, 19 insertions(+), 5 deletions(-)
create mode 100755 scripts/test.sh
rename tests/{sp.cocci => 0001-sp.cocci} (100%)
create mode 100644 tests/0001-sp/replace-typedef.patch
rename tests/{ => code/0001-sp}/sp.c (100%)
delete mode 100644 tests/sp.resSignature · 1 line
2.7.2
RFC 1/2 pycocci: add missing new line
--- tools/pycocci | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/pycocci +1 -1
@@ -608,7 +608,7 @@ def _main(): cmd = [ '--stat', patch_branch_name + ".." + smpl_branch_name ] diff_stat = git_diff(tree=args.target_dir, extra_args = cmd) if len(diff_stat) == 0: - logwrite('\nSmPL patch fully replaces patch series!') + logwrite('\nSmPL patch fully replaces patch series!\n') else: logwrite('\nDifferences found:\n\n') logwrite('Change directory to %s and run:\n\n\tgit diff %s..%s\n\n' % (args.target_dir, patch_branch_name, smpl_branch_name))
Signature · 1 line
2.7.2
RFC 1/2 pycocci: add missing new line
Quoted 1 line · Luis R. Rodriguez
Signed-off-by Luis R. Rodriguez <[email protected]>
Quoted 25 lines
--- tools/pycocci | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)tools/pycocci +1
-1@@ -608,7 +608,7 @@ def _main(): cmd = [ '--stat', patch_branch_name + ".." + smpl_branch_name ] diff_stat = git_diff(tree=args.target_dir, extra_args = cmd) if len(diff_stat) == 0: - logwrite('\nSmPL patch fully replaces patch series!') + logwrite('\nSmPL patch fully replaces patch series!\n') else: logwrite('\nDifferences found:\n\n') logwrite('Change directory to %s and run:\n\n\tgit diff %s..%s\n\n' % (args.target_dir, patch_branch_name, smpl_branch_name))Signature · 6 lines
2.7.2 _______________________________________________ Cocci mailing list Cocci at systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
RFC 2/2 tests: convert sp.cocci test to pycocci proof
In reply to Luis R. Rodriguez---
Makefile | 1 +
scripts/test.sh | 8 ++++++++
tests/{sp.cocci => 0001-sp.cocci} | 0
tests/0001-sp/replace-typedef.patch | 9 +++++++++
tests/{ => code/0001-sp}/sp.c | 0
tests/sp.res | 4 ----
6 files changed, 18 insertions(+), 4 deletions(-)
create mode 100755 scripts/test.sh
rename tests/{sp.cocci => 0001-sp.cocci} (100%)
create mode 100644 tests/0001-sp/replace-typedef.patch
rename tests/{ => code/0001-sp}/sp.c (100%)
delete mode 100644 tests/sp.resMakefile +1 -0
@@ -488,6 +488,7 @@ check: scripts/spatch COCCINELLE_HOME="$$(pwd)" ./scripts/spatch --sp-file demos/hello/hello-python.cocci demos/hello/helloworld.c --very-quiet | grep -q 'Hello at: 2'; fi @$(ECHO) running the test suite COCCINELLE_HOME="$$(pwd)" ./scripts/spatch --testall --no-update-score-file + COCCINELLE_HOME="$$(pwd)" ./scripts/test.sh # -inline 0 to see all the functions in the profile. # Can also use the profile framework in commons/ and run your program
scripts/test.sh new file +8 -0
@@ -0,0 +1,8 @@ +#!/bin/bash +set -e + +for i in tests/0001-sp.cocci; do + TEST=${i%%.cocci} + CODE_TEST=tests/code/$(basename $TEST) + ./tools/pycocci -v -s $i $CODE_TEST +done
tests/0001-sp.cocci renamed from tests/sp.cocci +0 -0
tests/0001-sp/replace-typedef.patch new file +9 -0
@@ -0,0 +1,9 @@ +--- a/sp.c ++++ b/sp.c +@@ -1,4 +1,4 @@ +-typedef struct { ++struct name { + unsigned long gcr; + struct pci_dev *pci; +-} snd_card_als4000_t; ++};
tests/code/0001-sp/sp.c renamed from tests/sp.c +0 -0
tests/sp.res deleted +0 -4
@@ -1,4 +0,0 @@ -struct name { - unsigned long gcr; - struct pci_dev *pci; -};
Signature · 1 line
2.7.2
cover coccinelle: extend test coverage
In reply to Luis R. RodriguezQuoted 5 lines
I wanted to extend tests coverage for coccinelle but noticed that tests are limited to single c file, these are attached to respective .res files, but this also means that you have a direct relationship and requirement between tests from a single C file and a respective cocci file.
Do you imagine to measure test coverage with any advanced techniques?
Quoted 2 lines
To go wild we want to be able to support really complex types of examples.
Is there any kind of target conflict between nice software demonstrations and the desire for better test procedures?
Quoted 3 lines
We can easily take advantage of this to extend Coccinelle test cases way beyond a single C file, but to more complex real life style projects.
How much would you like to improve the automatic checking of generated patches?
Regards,
Markus