Skip to contentSkip to conversation

RFC - simple scanners and matching macros

Cocci Nicholas Mc Guire 2 participants 16 messages Dec 23, 2014 – Dec 24, 2014

lore
Share & tools
lore
#0 lore raw

Hi !

writing some cocci file to detect some completion related issues - for the function cases it works fine. If its correct I'm not sure. What the first one should be doing is find any siutation where a completion is reinitialized with init_completion rather than reinit_completion. so find the first init_completion() and take the position (rule c) then check if the completion object was used or passed to a function before the next init_completion
Q: do I need to handle more than those 4 cases to catch all ?

The second one should find sequential init_completion() of the same struct completion without that they are used in between so basically the inverse case of the first - I'm not sure if its worth the trouble though - in 3.18.0 there are 2 cases found and both were correct findings

The third scanner was to search for DECLARE_COMPLETION used in functions for declearing struct completion on automatic variables and transform them to DECLARE_COMPLETION_ONSTACK. Simple problem it is not working... - obviously Im overlooking something - it will just run through and report nothin.

Any hint would be appreciated.

One more procedural question - the patch-set generated should be posted here+lkml for a first review or should it go out to all the subsystem lists whose code is affected in CC as well ?

thx!
hofrat

first working case:
===================

@c@
expression cmp;
position p;
@@

init_completion at p(cmp)

@d@
expression E,c.cmp;
identifier f;
position c.p,p1;
@@

init_completion at p(cmp)
...
(
E = cmp
|
E = &cmp
|
f(..., cmp,...)
|
f(..., &cmp,...)
)
...
- init_completion at p1(cmp)
+ reinit_completion1(cmp)

2nd working case:
=================

@c@
expression cmp;
position p;
@@

init_completion at p(cmp)

@d@
expression E,c.cmp;
identifier f;
position c.p,p1;
@@

init_completion at p(cmp)
... when != E = cmp

      when != E = &cmp
      when != f(..., cmp,...)
      when != f(..., &cmp,...)

- init_completion at p1(cmp);

the not-working case:
=====================
@e@
expression cmp;
identifier f;
position p;
@@

f(...) {
...
- DECLARE_COMPLETION at p(cmp);
+ DECLARE_COMPLETION_ONSTACK(cmp);
...
}

#1 lore raw
Quoted 25 lines · Nicholas Mc Guire

Hi !

writing some cocci file to detect some completion related issues - for the function cases it works fine. If its correct I'm not sure. What the first one should be doing is find any siutation where a completion is reinitialized with init_completion rather than reinit_completion. so find the first init_completion() and take the position (rule c) then check if the completion object was used or passed to a function before the next init_completion
Q: do I need to handle more than those 4 cases to catch all ?

The second one should find sequential init_completion() of the same struct completion without that they are used in between so basically the inverse case of the first - I'm not sure if its worth the trouble though - in 3.18.0 there are 2 cases found and both were correct findings

The third scanner was to search for DECLARE_COMPLETION used in functions for declearing struct completion on automatic variables and transform them to DECLARE_COMPLETION_ONSTACK. Simple problem it is not working... - obviously Im overlooking something - it will just run through and report nothin.

Quoted 5 lines

Any hint would be appreciated.

One more procedural question - the patch-set generated should be posted here+lkml for a first review or should it go out to all the subsystem lists whose code is affected in CC as well ?

Patches should always go to the people indicated by the Linux script get_maintainers.pl, ie maintainers and mailing lists for the specific subsystems.

Once you are confident of the semantic patch, and think it may be useful for others, then it can go to the lkml and the maintainers of scripts/coccinelle, ie basically the list, a few people, and Michel Marek, who does the actual integration. For making the final version of the semantic patch, you can use the tool tools/sgen in the Coccinelle source tree, which makes the variants of the semantic patch that are supported by the Linux kernel.

Quoted 19 lines

thx!
hofrat

first working case:
===================

@c@
expression cmp;
position p;
@@

init_completion at p(cmp)

@d@
expression E,c.cmp;
identifier f;
position c.p,p1;
@@

Quoted 12 lines

init_completion at p(cmp)
...
(
E = cmp
|
E = &cmp
|
f(..., cmp,...)
|
f(..., &cmp,...)
)
...

Perhaps you want <+... ...+> here, if these assignments or function calls could occur more than once?

Quoted 20 lines

- init_completion at p1(cmp)
+ reinit_completion1(cmp)

2nd working case:
=================

@c@
expression cmp;
position p;
@@

init_completion at p(cmp)

@d@
expression E,c.cmp;
identifier f;
position c.p,p1;
@@

Quoted 6 lines

init_completion at p(cmp)
... when != E = cmp

      when != E = &cmp
      when != f(..., cmp,...)
      when != f(..., &cmp,...)

- init_completion at p1(cmp);

OK. If you allow there to be none of these calls in between, then you could merge these two rules, and change <+... ...+> to <... ...>. But I am not really sure what is the goal of these constraints at all. Why not just:

init_completion at p(cmp)
...
init_completion at p1(cmp)

Quoted 9 lines

the not-working case:
=====================
@e@
expression cmp;
identifier f;
position p;
@@

Quoted 6 lines

f(...) {
...
- DECLARE_COMPLETION at p(cmp);
+ DECLARE_COMPLETION_ONSTACK(cmp);
...
}

You need to add

declarer name DECLARE_COMPLETION;

among the metavariables.

julia

#2 lore raw
Quoted 37 lines · Julia Lawall
Quoted 25 lines · Nicholas Mc Guire

Hi !

writing some cocci file to detect some completion related issues - for the function cases it works fine. If its correct I'm not sure. What the first one should be doing is find any siutation where a completion is reinitialized with init_completion rather than reinit_completion. so find the first init_completion() and take the position (rule c) then check if the completion object was used or passed to a function before the next init_completion
Q: do I need to handle more than those 4 cases to catch all ?

The second one should find sequential init_completion() of the same struct completion without that they are used in between so basically the inverse case of the first - I'm not sure if its worth the trouble though - in 3.18.0 there are 2 cases found and both were correct findings

The third scanner was to search for DECLARE_COMPLETION used in functions for declearing struct completion on automatic variables and transform them to DECLARE_COMPLETION_ONSTACK. Simple problem it is not working... - obviously Im overlooking something - it will just run through and report nothin.

Quoted 5 lines

Any hint would be appreciated.

One more procedural question - the patch-set generated should be posted here+lkml for a first review or should it go out to all the subsystem lists whose code is affected in CC as well ?

Patches should always go to the people indicated by the Linux script get_maintainers.pl, ie maintainers and mailing lists for the specific subsystems.

Quoted 7 lines

Once you are confident of the semantic patch, and think it may be useful for others, then it can go to the lkml and the maintainers of scripts/coccinelle, ie basically the list, a few people, and Michel Marek, who does the actual integration. For making the final version of the semantic patch, you can use the tool tools/sgen in the Coccinelle source tree, which makes the variants of the semantic patch that are supported by the Linux kernel.

ok - then I'll send it out as individual patches and post the coccifiles if the patches are confirmed.

Quoted 33 lines
Quoted 20 lines

thx!
hofrat

first working case:
===================

@c@
expression cmp;
position p;
@@

init_completion at p(cmp)

@d@
expression E,c.cmp;
identifier f;
position c.p,p1;
@@

Quoted 12 lines

init_completion at p(cmp)
...
(
E = cmp
|
E = &cmp
|
f(..., cmp,...)
|
f(..., &cmp,...)
)
...

Quoted 2 lines

Perhaps you want <+... ...+> here, if these assignments or function calls could occur more than once?

fixed.

Quoted 28 lines
Quoted 21 lines

- init_completion at p1(cmp)
+ reinit_completion1(cmp)

2nd working case:
=================

@c@
expression cmp;
position p;
@@

init_completion at p(cmp)

@d@
expression E,c.cmp;
identifier f;
position c.p,p1;
@@

Quoted 6 lines

init_completion at p(cmp)
... when != E = cmp

      when != E = &cmp
      when != f(..., cmp,...)
      when != f(..., &cmp,...)

- init_completion at p1(cmp);

Quoted 9 lines

OK. If you allow there to be none of these calls in between, then you could merge these two rules, and change <+... ...+> to <... ...>. But I am not really sure what is the goal of these constraints at all. Why not just:

init_completion at p(cmp)
...
init_completion at p1(cmp)

because that would match cases like

init_completion(cmp)
wait_for_completion(cmp)
init_completion(cmp)

and that is not a double init but a reinit
So the intent of the construct is to catch
cases like:

+++ b/drivers/scsi/qla4xxx/ql4_os.c
@@ -8672,7 +8672,6 @@ static int qla4xxx_probe_adapter(struct

        init_completion(&ha->disable_acb_comp);
        init_completion(&ha->idc_comp);
        init_completion(&ha->link_up_comp);

- init_completion(&ha->disable_acb_comp);

where nothing was done with the completion object between the calls - so this should not be changed to a reinit_completion but dropped.

Quoted 16 lines
Quoted 9 lines

the not-working case:
=====================
@e@
expression cmp;
identifier f;
position p;
@@

Quoted 6 lines

f(...) {
...
- DECLARE_COMPLETION at p(cmp);
+ DECLARE_COMPLETION_ONSTACK(cmp);
...
}

Quoted 6 lines

You need to add

declarer name DECLARE_COMPLETION;

among the metavariables.

tried it as you suggested - but that fusses:

hofrat at debian:/tmp/linux-next$ make coccicheck COCCI=false_declare_completion.cocci MODE=patch

Fatal error: exception Failure("meta: parse error:
= File "false_declare_completion.cocci", line 15, column 0, charpos = 295

    around = 'declare', whole content = declare name DECLARE_COMPLETION;

")

spatch version is
hofrat at debian:/tmp/linux-next$ spatch --version
spatch version 1.0.0-rc21 with Python support and with Str regexp support

The complete cocci script is below

/* check for incorrect DECLARE_COMPLETION use within a function
*
* Options: --no-includes --include-headers
*/
virtual context
virtual patch
virtual org
virtual report

/* flag incorrect initializer*/
@e depends on patch && !(context || org || report)@ expression cmp;
identifier f;
declare name DECLARE_COMPLETION;
position p;
@@

f(...) {
...
- DECLARE_COMPLETION at p(cmp);
+ DECLARE_COMPLETION_ONSTACK(cmp);
...
}

@ep depends on !patch && (context || org || report)@ expression cmp;
identifier f;
declare name DECLARE_COMPLETION;
position p;
@@

f(...) {
...
* DECLARE_COMPLETION at p(cmp);
...
}

@script:python depends on org@
p << ep.p;
@@

msg="WARNING: possible incorrect use of DECLARE_COMPLETION" coccilib.org.print_todo(p[0], msg)

@script:python depends on report@
p << ep.p;
@@

msg="WARNING: possible incorrect use of DECLARE_COMPLETION" coccilib.report.print_report(p[0], msg)

#3 lore raw
Quoted 72 lines
Quoted 29 lines
Quoted 16 lines

first working case:
===================

@c@
expression cmp;
position p;
@@

init_completion at p(cmp)

@d@
expression E,c.cmp;
identifier f;
position c.p,p1;
@@

Quoted 12 lines

init_completion at p(cmp)
...
(
E = cmp
|
E = &cmp
|
f(..., cmp,...)
|
f(..., &cmp,...)
)
...

Quoted 2 lines

Perhaps you want <+... ...+> here, if these assignments or function calls could occur more than once?

fixed.

Quoted 28 lines
Quoted 21 lines

- init_completion at p1(cmp)
+ reinit_completion1(cmp)

2nd working case:
=================

@c@
expression cmp;
position p;
@@

init_completion at p(cmp)

@d@
expression E,c.cmp;
identifier f;
position c.p,p1;
@@

Quoted 6 lines

init_completion at p(cmp)
... when != E = cmp

      when != E = &cmp
      when != f(..., cmp,...)
      when != f(..., &cmp,...)

- init_completion at p1(cmp);

Quoted 9 lines

OK. If you allow there to be none of these calls in between, then you could merge these two rules, and change <+... ...+> to <... ...>. But I am not really sure what is the goal of these constraints at all. Why not just:

init_completion at p(cmp)
...
init_completion at p1(cmp)

Quoted 5 lines

because that would match cases like

init_completion(cmp)
wait_for_completion(cmp)
init_completion(cmp)

Doesn't this match the "first working case" rule?

Quoted 36 lines

and that is not a double init but a reinit
So the intent of the construct is to catch
cases like:

+++ b/drivers/scsi/qla4xxx/ql4_os.c

(quoted diff) +0 -1
@@ -8672,7 +8672,6 @@ static int qla4xxx_probe_adapter(struct
        init_completion(&ha->disable_acb_comp);
        init_completion(&ha->idc_comp);
        init_completion(&ha->link_up_comp);
-       init_completion(&ha->disable_acb_comp);
 

where nothing was done with the completion object between the calls - so this should not be changed to a reinit_completion but dropped.

Quoted 16 lines
Quoted 9 lines

the not-working case:
=====================
@e@
expression cmp;
identifier f;
position p;
@@

Quoted 6 lines

f(...) {
...
- DECLARE_COMPLETION at p(cmp);
+ DECLARE_COMPLETION_ONSTACK(cmp);
...
}

Quoted 6 lines

You need to add

declarer name DECLARE_COMPLETION;

among the metavariables.

Quoted 8 lines

tried it as you suggested - but that fusses:

hofrat at debian:/tmp/linux-next$ make coccicheck COCCI=false_declare_completion.cocci MODE=patch

Fatal error: exception Failure("meta: parse error:
= File "false_declare_completion.cocci", line 15, column 0, charpos = 295

    around = 'declare', whole content = declare name DECLARE_COMPLETION;

")

declarer name, with an "r" a the end, not declare name.

Quoted 24 lines

spatch version is
hofrat at debian:/tmp/linux-next$ spatch --version
spatch version 1.0.0-rc21 with Python support and with Str regexp support

The complete cocci script is below

/* check for incorrect DECLARE_COMPLETION use within a function
*
* Options: --no-includes --include-headers
*/
virtual context
virtual patch
virtual org
virtual report

/* flag incorrect initializer*/
@e depends on patch && !(context || org || report)@ expression cmp;
identifier f;
declare name DECLARE_COMPLETION;
position p;
@@

Quoted 5 lines

f(...) {
...
- DECLARE_COMPLETION at p(cmp);
+ DECLARE_COMPLETION_ONSTACK(cmp);
...

Actually, there should be <... ...> here too. Otherwise, you will allow only one DECLARE_COMPLETION in the function.

Quoted 9 lines

}

@ep depends on !patch && (context || org || report)@ expression cmp;
identifier f;
declare name DECLARE_COMPLETION;
position p;
@@

Quoted 5 lines

f(...) {
...
* DECLARE_COMPLETION at p(cmp);
...
}

Same here.

julia

Quoted 15 lines

@script:python depends on org@
p << ep.p;
@@

msg="WARNING: possible incorrect use of DECLARE_COMPLETION" coccilib.org.print_todo(p[0], msg)

@script:python depends on report@
p << ep.p;
@@

msg="WARNING: possible incorrect use of DECLARE_COMPLETION" coccilib.report.print_report(p[0], msg)

#4 lore raw
Quoted 72 lines · Julia Lawall
Quoted 72 lines
Quoted 29 lines
Quoted 16 lines

first working case:
===================

@c@
expression cmp;
position p;
@@

init_completion at p(cmp)

@d@
expression E,c.cmp;
identifier f;
position c.p,p1;
@@

Quoted 12 lines

init_completion at p(cmp)
...
(
E = cmp
|
E = &cmp
|
f(..., cmp,...)
|
f(..., &cmp,...)
)
...

Quoted 2 lines

Perhaps you want <+... ...+> here, if these assignments or function calls could occur more than once?

fixed.

Quoted 37 lines
Quoted 21 lines

- init_completion at p1(cmp)
+ reinit_completion1(cmp)

2nd working case:
=================

@c@
expression cmp;
position p;
@@

init_completion at p(cmp)

@d@
expression E,c.cmp;
identifier f;
position c.p,p1;
@@

Quoted 6 lines

init_completion at p(cmp)
... when != E = cmp

      when != E = &cmp
      when != f(..., cmp,...)
      when != f(..., &cmp,...)

- init_completion at p1(cmp);

OK. If you allow there to be none of these calls in between, then you could merge these two rules, and change <+... ...+> to <... ...>. But I am not really sure what is the goal of these constraints at all. Why not just:

init_completion at p(cmp)
...
init_completion at p1(cmp)

Quoted 8 lines
Quoted 5 lines

because that would match cases like

init_completion(cmp)
wait_for_completion(cmp)
init_completion(cmp)

Doesn't this match the "first working case" rule?

yes - just not sure if the first case is exhaustive - only if it is would the second rule in the simplified version be correct I believe.

Quoted 38 lines
Quoted 38 lines

and that is not a double init but a reinit
So the intent of the construct is to catch
cases like:

+++ b/drivers/scsi/qla4xxx/ql4_os.c

(quoted diff) +0 -1
@@ -8672,7 +8672,6 @@ static int qla4xxx_probe_adapter(struct
        init_completion(&ha->disable_acb_comp);
        init_completion(&ha->idc_comp);
        init_completion(&ha->link_up_comp);
-       init_completion(&ha->disable_acb_comp);
 

where nothing was done with the completion object between the calls - so this should not be changed to a reinit_completion but dropped.

Quoted 16 lines
Quoted 9 lines

the not-working case:
=====================
@e@
expression cmp;
identifier f;
position p;
@@

Quoted 6 lines

f(...) {
...
- DECLARE_COMPLETION at p(cmp);
+ DECLARE_COMPLETION_ONSTACK(cmp);
...
}

Quoted 6 lines

You need to add

declarer name DECLARE_COMPLETION;

among the metavariables.

tried it as you suggested - but that fusses:

Quoted 9 lines
Quoted 6 lines

hofrat at debian:/tmp/linux-next$ make coccicheck COCCI=false_declare_completion.cocci MODE=patch

Fatal error: exception Failure("meta: parse error:
= File "false_declare_completion.cocci", line 15, column 0, charpos = 295

    around = 'declare', whole content = declare name DECLARE_COMPLETION;

")

declarer name, with an "r" a the end, not declare name.

tried that first then removed the r assuming it was a typo

hofrat at debian:~/git/linux-next$ make coccicheck COCCI=false_declare_completion.cocci MODE=report

Please check for false positives in the output before submitting a patch. When using "patch" mode, carefully review the patch before submitting it.

297 298
Fatal error: exception Failure("meta: parse error:
= File "false_declare_completion.cocci", line 14, column 32, charpos = 297

    around = '@', whole content = declarer name DECLARE_COMPLETION@;

")

same for MODE=patch

Quoted 42 lines
Quoted 24 lines

spatch version is
hofrat at debian:/tmp/linux-next$ spatch --version
spatch version 1.0.0-rc21 with Python support and with Str regexp support

The complete cocci script is below

/* check for incorrect DECLARE_COMPLETION use within a function
*
* Options: --no-includes --include-headers
*/
virtual context
virtual patch
virtual org
virtual report

/* flag incorrect initializer*/
@e depends on patch && !(context || org || report)@ expression cmp;
identifier f;
declare name DECLARE_COMPLETION;
position p;
@@

Quoted 5 lines

f(...) {
...
- DECLARE_COMPLETION at p(cmp);
+ DECLARE_COMPLETION_ONSTACK(cmp);
...

Actually, there should be <... ...> here too. Otherwise, you will allow only one DECLARE_COMPLETION in the function.

Quoted 9 lines

}

@ep depends on !patch && (context || org || report)@ expression cmp;
identifier f;
declare name DECLARE_COMPLETION;
position p;
@@

Quoted 8 lines
Quoted 5 lines

f(...) {
...
* DECLARE_COMPLETION at p(cmp);
...
}

Same here.

thanks fixed this as well.

thx!
hofrat

#5 lore raw
Quoted 80 lines · Nicholas Mc Guire
Quoted 78 lines · Julia Lawall
Quoted 77 lines
Quoted 29 lines
Quoted 16 lines

first working case:
===================

@c@
expression cmp;
position p;
@@

init_completion at p(cmp)

@d@
expression E,c.cmp;
identifier f;
position c.p,p1;
@@

Quoted 12 lines

init_completion at p(cmp)
...
(
E = cmp
|
E = &cmp
|
f(..., cmp,...)
|
f(..., &cmp,...)
)
...

Quoted 2 lines

Perhaps you want <+... ...+> here, if these assignments or function calls could occur more than once?

fixed.

Quoted 28 lines
Quoted 21 lines

- init_completion at p1(cmp)
+ reinit_completion1(cmp)

2nd working case:
=================

@c@
expression cmp;
position p;
@@

init_completion at p(cmp)

@d@
expression E,c.cmp;
identifier f;
position c.p,p1;
@@

Quoted 6 lines

init_completion at p(cmp)
... when != E = cmp

      when != E = &cmp
      when != f(..., cmp,...)
      when != f(..., &cmp,...)

- init_completion at p1(cmp);

Quoted 9 lines

OK. If you allow there to be none of these calls in between, then you could merge these two rules, and change <+... ...+> to <... ...>. But I am not really sure what is the goal of these constraints at all. Why not just:

init_completion at p(cmp)
...
init_completion at p1(cmp)

because that would match cases like

init_completion(cmp)
wait_for_completion(cmp)
init_completion(cmp)

Quoted 6 lines
Quoted 2 lines

Doesn't this match the "first working case" rule?

yes - just not sure if the first case is exhaustive - only if it is would the second rule in the simplified version be correct I believe.

I mean, wouldn't the first rule match code like:

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

init_completion(cmp)
wait_for_completion(cmp)
init_completion(cmp)

and thus make a transformation that is not desired?

Quoted 4 lines

tried that first then removed the r assuming it was a typo

hofrat at debian:~/git/linux-next$ make coccicheck COCCI=false_declare_completion.cocci MODE=report

Quoted 8 lines

Please check for false positives in the output before submitting a patch. When using "patch" mode, carefully review the patch before submitting it.

297 298
Fatal error: exception Failure("meta: parse error:
= File "false_declare_completion.cocci", line 14, column 32, charpos = 297

    around = '@', whole content = declarer name DECLARE_COMPLETION@;

")

There seems to be a @ at the end of the declared declarer name. You can't use @ in a metavariable name.

julia

Quoted 57 lines

same for MODE=patch

Quoted 42 lines
Quoted 24 lines

spatch version is
hofrat at debian:/tmp/linux-next$ spatch --version
spatch version 1.0.0-rc21 with Python support and with Str regexp support

The complete cocci script is below

/* check for incorrect DECLARE_COMPLETION use within a function
*
* Options: --no-includes --include-headers
*/
virtual context
virtual patch
virtual org
virtual report

/* flag incorrect initializer*/
@e depends on patch && !(context || org || report)@ expression cmp;
identifier f;
declare name DECLARE_COMPLETION;
position p;
@@

Quoted 5 lines

f(...) {
...
- DECLARE_COMPLETION at p(cmp);
+ DECLARE_COMPLETION_ONSTACK(cmp);
...

Actually, there should be <... ...> here too. Otherwise, you will allow only one DECLARE_COMPLETION in the function.

Quoted 9 lines

}

@ep depends on !patch && (context || org || report)@ expression cmp;
identifier f;
declare name DECLARE_COMPLETION;
position p;
@@

Quoted 8 lines
Quoted 5 lines

f(...) {
...
* DECLARE_COMPLETION at p(cmp);
...
}

Same here.

thanks fixed this as well.

thx!
hofrat

#6 lore raw
Quoted 14 lines
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines

because that would match cases like

init_completion(cmp)
wait_for_completion(cmp)
init_completion(cmp)

Quoted 6 lines
Quoted 2 lines

Doesn't this match the "first working case" rule?

yes - just not sure if the first case is exhaustive - only if it is would the second rule in the simplified version be correct I believe.

Quoted 8 lines

I mean, wouldn't the first rule match code like:

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

init_completion(cmp)
wait_for_completion(cmp)
init_completion(cmp)

and thus make a transformation that is not desired?

yes it does and should. That second init_completion is incorrect it should be
init_completion(cmp)
...
wait_for_completion(cmp)
...
reinit_completion(cmp)

Quoted 7 lines
Quoted 7 lines

tried that first then removed the r assuming it was a typo

hofrat at debian:~/git/linux-next$ make coccicheck COCCI=false_declare_completion.cocci MODE=report

Please check for false positives in the output before submitting a patch. When using "patch" mode, carefully review the patch before submitting it.

Quoted 9 lines
Quoted 5 lines

297 298
Fatal error: exception Failure("meta: parse error:
= File "false_declare_completion.cocci", line 14, column 32, charpos = 297

    around = '@', whole content = declarer name DECLARE_COMPLETION@;

")

There seems to be a @ at the end of the declared declarer name. You can't use @ in a metavariable name.

sorry not sure how I managed that - gives me the same error though

hofrat at debian:/tmp/linux-next$ make coccicheck COCCI=false_declare_completion.cocci MODE=patch

Please check for false positives in the output before submitting a patch. When using "patch" mode, carefully review the patch before submitting it.

Fatal error: exception Failure("meta: parse error:
= File "false_declare_completion.cocci", line 27, column 14, charpos = 511

    around = 'DECLARE_COMPLETION', whole content = declarer name DECLARE_COMPLETION;

")
...

complete script:
/* check for incorrect DECLARE_COMPLETION use within a function
*
* Options: --no-includes --include-headers
*/
virtual context
virtual patch
virtual org
virtual report

/* flag incorrect initializer*/
@e depends on patch && !(context || org || report)@ expression cmp;
identifier f;
declarer name DECLARE_COMPLETION;
position p;
@@

f(...) {
...
- DECLARE_COMPLETION at p(cmp);
+ DECLARE_COMPLETION_ONSTACK(cmp);
...
}

@ep depends on !patch && (context || org || report)@ expression cmp;
identifier f;
declarer name DECLARE_COMPLETION;
position p;
@@

f(...) {
...
* DECLARE_COMPLETION at p(cmp);
...
}

@script:python depends on org@
p << ep.p;
@@

msg="WARNING: possible incorrect use of DECLARE_COMPLETION" coccilib.org.print_todo(p[0], msg)

thx!
hofrat

#7 lore raw
Quoted 5 lines

Fatal error: exception Failure("meta: parse error:
= File "false_declare_completion.cocci", line 27, column 14, charpos = 511

    around = 'DECLARE_COMPLETION', whole content = declarer name DECLARE_COMPLETION;

")
...

OK, it's not super intuitive, but the problem is that it wants declarer names, like typedefs, to be declared once and only once. And in deciding about this, it doesn't take into account the dependencies on virtual rules. So if you use -D patch, then it complains about a declaration in a case that says depends on !patch. And if you remove the declarer name declaration in the latter rule, when you use eg -D report, it takes into account the declaration in the previous, but disabled, -D patch rule.

So if you drop the second declarer name declaration, everything will be fine.

The fact that declaring a typedef, declarer name, or iterator name once was good enough was supposed to make things easier, but it seems to just make things confusing. Maybe I should get rid of this feature, or at least not complain about redeclarations that are consistent with the previous one.

julia

#8 lore raw
Quoted 14 lines · Julia Lawall
Quoted 5 lines

Fatal error: exception Failure("meta: parse error:
= File "false_declare_completion.cocci", line 27, column 14, charpos = 511

    around = 'DECLARE_COMPLETION', whole content = declarer name DECLARE_COMPLETION;

")
...

OK, it's not super intuitive, but the problem is that it wants declarer names, like typedefs, to be declared once and only once. And in deciding about this, it doesn't take into account the dependencies on virtual rules. So if you use -D patch, then it complains about a declaration in a case that says depends on !patch. And if you remove the declarer name declaration in the latter rule, when you use eg -D report, it takes into account the declaration in the previous, but disabled, -D patch rule.

Quoted 9 lines

So if you drop the second declarer name declaration, everything will be fine.

The fact that declaring a typedef, declarer name, or iterator name once was good enough was supposed to make things easier, but it seems to just make things confusing. Maybe I should get rid of this feature, or at least not complain about redeclarations that are consistent with the previous one.

it did not complain - but something seems to be going wrong now it consumes 100% CPU on all avaialble cores for about 8 minutes and finds no match

hofrat at debian:/tmp/linux-next$ time make coccicheck COCCI=false_declare_completion.cocci MODE=report

Please check for false positives in the output before submitting a patch. When using "patch" mode, carefully review the patch before submitting it.

real    8m9.438s
user    33m6.005s
sys     0m10.277s

about the same in MODE=patch, the other cocci file over the same tree ran

real    0m48.492s
user    2m26.627s
sys     0m1.933s

an example file where I had expected it to trigger is drivers/scsi/aha152x.c - but it did not.

cut down testcse would be:

test.c:
static int aha152x_device_reset(Scsi_Cmnd * SCpnt)

{
        struct Scsi_Host *shpnt = SCpnt->device->host;
        DECLARE_COMPLETION(done);
        int ret, issued, disconnected;
        unsigned char old_cmd_len = SCpnt->cmd_len;
        unsigned long flags;
        unsigned long timeleft;
        /* snippet from drivers/scsi/aha152x.c */
}

test.cocci:
@e@
expression cmp;
identifier f;
declarer name DECLARE_COMPLETION;
position p;
@@

f(...) {
<...
- DECLARE_COMPLETION@p(cmp);
+ DECLARE_COMPLETION_ONSTACK(cmp);
...>
}

which should trigger if I did not do something foolish again but it did not:

hofrat at debian:/tmp/linux-next$ spatch --sp-file test.cocci test.c
init_defs_builtins: /usr/local/share/coccinelle/standard.h
HANDLING: test.c

thx!
hofrat

#9 lore raw
Quoted 77 lines · Nicholas Mc Guire
Quoted 14 lines · Julia Lawall
Quoted 5 lines

Fatal error: exception Failure("meta: parse error:
= File "false_declare_completion.cocci", line 27, column 14, charpos = 511

    around = 'DECLARE_COMPLETION', whole content = declarer name DECLARE_COMPLETION;

")
...

OK, it's not super intuitive, but the problem is that it wants declarer names, like typedefs, to be declared once and only once. And in deciding about this, it doesn't take into account the dependencies on virtual rules. So if you use -D patch, then it complains about a declaration in a case that says depends on !patch. And if you remove the declarer name declaration in the latter rule, when you use eg -D report, it takes into account the declaration in the previous, but disabled, -D patch rule.

Quoted 9 lines

So if you drop the second declarer name declaration, everything will be fine.

The fact that declaring a typedef, declarer name, or iterator name once was good enough was supposed to make things easier, but it seems to just make things confusing. Maybe I should get rid of this feature, or at least not complain about redeclarations that are consistent with the previous one.

it did not complain - but something seems to be going wrong now it consumes 100% CPU on all avaialble cores for about 8 minutes and finds no match

hofrat at debian:/tmp/linux-next$ time make coccicheck COCCI=false_declare_completion.cocci MODE=report

Please check for false positives in the output before submitting a patch. When using "patch" mode, carefully review the patch before submitting it.

real    8m9.438s
user    33m6.005s
sys     0m10.277s

about the same in MODE=patch, the other cocci file over the same tree ran

real    0m48.492s
user    2m26.627s
sys     0m1.933s

an example file where I had expected it to trigger is drivers/scsi/aha152x.c - but it did not.

cut down testcse would be:

test.c:
static int aha152x_device_reset(Scsi_Cmnd * SCpnt)

{
        struct Scsi_Host *shpnt = SCpnt->device->host;
        DECLARE_COMPLETION(done);
        int ret, issued, disconnected;
        unsigned char old_cmd_len = SCpnt->cmd_len;
        unsigned long flags;
        unsigned long timeleft;
        /* snippet from drivers/scsi/aha152x.c */
}

test.cocci:
@e@
expression cmp;
identifier f;
declarer name DECLARE_COMPLETION;
position p;
@@

f(...) {
<...
- DECLARE_COMPLETION at p(cmp);
+ DECLARE_COMPLETION_ONSTACK(cmp);
...>
}

Quoted 6 lines

which should trigger if I did not do something foolish again but it did not:

hofrat at debian:/tmp/linux-next$ spatch --sp-file test.cocci test.c
init_defs_builtins: /usr/local/share/coccinelle/standard.h
HANDLING: test.c

It works fine for me. What version of Coccinelle do you have?

For the performance problem, could you send the current semantic patch again, so I could be sure to be testing the right thing?

thanks,
julia

#10 lore raw
Quoted 81 lines · Julia Lawall
Quoted 77 lines · Nicholas Mc Guire
Quoted 14 lines · Julia Lawall
Quoted 5 lines

Fatal error: exception Failure("meta: parse error:
= File "false_declare_completion.cocci", line 27, column 14, charpos = 511

    around = 'DECLARE_COMPLETION', whole content = declarer name DECLARE_COMPLETION;

")
...

OK, it's not super intuitive, but the problem is that it wants declarer names, like typedefs, to be declared once and only once. And in deciding about this, it doesn't take into account the dependencies on virtual rules. So if you use -D patch, then it complains about a declaration in a case that says depends on !patch. And if you remove the declarer name declaration in the latter rule, when you use eg -D report, it takes into account the declaration in the previous, but disabled, -D patch rule.

Quoted 9 lines

So if you drop the second declarer name declaration, everything will be fine.

The fact that declaring a typedef, declarer name, or iterator name once was good enough was supposed to make things easier, but it seems to just make things confusing. Maybe I should get rid of this feature, or at least not complain about redeclarations that are consistent with the previous one.

it did not complain - but something seems to be going wrong now it consumes 100% CPU on all avaialble cores for about 8 minutes and finds no match

hofrat at debian:/tmp/linux-next$ time make coccicheck COCCI=false_declare_completion.cocci MODE=report

Please check for false positives in the output before submitting a patch. When using "patch" mode, carefully review the patch before submitting it.

real    8m9.438s
user    33m6.005s
sys     0m10.277s

about the same in MODE=patch, the other cocci file over the same tree ran

real    0m48.492s
user    2m26.627s
sys     0m1.933s

an example file where I had expected it to trigger is drivers/scsi/aha152x.c - but it did not.

cut down testcse would be:

test.c:
static int aha152x_device_reset(Scsi_Cmnd * SCpnt)

{
        struct Scsi_Host *shpnt = SCpnt->device->host;
        DECLARE_COMPLETION(done);
        int ret, issued, disconnected;
        unsigned char old_cmd_len = SCpnt->cmd_len;
        unsigned long flags;
        unsigned long timeleft;
        /* snippet from drivers/scsi/aha152x.c */
}

test.cocci:
@e@
expression cmp;
identifier f;
declarer name DECLARE_COMPLETION;
position p;
@@

f(...) {
<...
- DECLARE_COMPLETION at p(cmp);
+ DECLARE_COMPLETION_ONSTACK(cmp);
...>
}

Quoted 9 lines
Quoted 6 lines

which should trigger if I did not do something foolish again but it did not:

hofrat at debian:/tmp/linux-next$ spatch --sp-file test.cocci test.c
init_defs_builtins: /usr/local/share/coccinelle/standard.h
HANDLING: test.c

It works fine for me. What version of Coccinelle do you have?

hofrat at debian:/tmp/linux-next$ spatch --version
spatch version 1.0.0-rc21 with Python support and with Str regexp support

will update to rc23 and retest - can this be a config/python version issue ? python version is 2.7.3 (Debian 7.6)

Quoted 3 lines

For the performance problem, could you send the current semantic patch again, so I could be sure to be testing the right thing?

files where this situation exists in linux-next (3.18.0) and which should trigger are:

 drivers/macintosh/ams/ams-pmu.c        line 52
 drivers/misc/sgi-gru/grukservices.c    line 1044
 drivers/scsi/aha152x.c                 line 1055
 drivers/usb/gadget/udc/fsl_qe_udc.c    line 2630
 drivers/usb/gadget/udc/fsl_udc_core.c  line 2529

the non-working semantic patch below:

/* check for incorrect DECLARE_COMPLETION use within a function
*
* Options: --no-includes --include-headers
*/
virtual context
virtual patch
virtual org
virtual report

/* flag incorrect initializer*/
@e depends on patch && !(context || org || report)@ expression cmp;
identifier f;
declarer name DECLARE_COMPLETION;
position p;
@@

f(...) {
<...
- DECLARE_COMPLETION@p(cmp);
+ DECLARE_COMPLETION_ONSTACK(cmp);
...>
}

@ep depends on !patch && (context || org || report)@ expression cmp;
identifier f;
position p;
@@

f(...) {
<...
* DECLARE_COMPLETION@p(cmp);
...>
}

@script:python depends on org@
p << ep.p;
@@

msg="WARNING: possible incorrect use of DECLARE_COMPLETION" coccilib.org.print_todo(p[0], msg)

@script:python depends on report@
p << ep.p;
@@

msg="WARNING: possible incorrect use of DECLARE_COMPLETION" coccilib.report.print_report(p[0], msg)

thx!
hofrat

#11 lore raw
Quoted 2 lines

will update to rc23 and retest - can this be a config/python version issue ? python version is 2.7.3 (Debian 7.6)

I have Python 2.7.5+

Quoted 11 lines
Quoted 3 lines

For the performance problem, could you send the current semantic patch again, so I could be sure to be testing the right thing?

files where this situation exists in linux-next (3.18.0) and which should trigger are:

 drivers/macintosh/ams/ams-pmu.c        line 52
 drivers/misc/sgi-gru/grukservices.c    line 1044
 drivers/scsi/aha152x.c                 line 1055
 drivers/usb/gadget/udc/fsl_qe_udc.c    line 2630
 drivers/usb/gadget/udc/fsl_udc_core.c  line 2529

I tried these files in their linux-next versions with the -D report option and got an answer more or less immediately (2 seconds). An example command line is:

spatch --sp-file dc.cocci -D report /var/linuxes/linux-next/drivers/scsi/aha152x.c

Does this work for you?

julia

Quoted 56 lines

the non-working semantic patch below:

/* check for incorrect DECLARE_COMPLETION use within a function
*
* Options: --no-includes --include-headers
*/
virtual context
virtual patch
virtual org
virtual report

/* flag incorrect initializer*/
@e depends on patch && !(context || org || report)@ expression cmp;
identifier f;
declarer name DECLARE_COMPLETION;
position p;
@@

f(...) {
<...
- DECLARE_COMPLETION at p(cmp);
+ DECLARE_COMPLETION_ONSTACK(cmp);
...>
}

@ep depends on !patch && (context || org || report)@ expression cmp;
identifier f;
position p;
@@

f(...) {
<...
* DECLARE_COMPLETION at p(cmp);
...>
}

@script:python depends on org@
p << ep.p;
@@

msg="WARNING: possible incorrect use of DECLARE_COMPLETION" coccilib.org.print_todo(p[0], msg)

@script:python depends on report@
p << ep.p;
@@

msg="WARNING: possible incorrect use of DECLARE_COMPLETION" coccilib.report.print_report(p[0], msg)

thx!
hofrat

#12 lore raw
Quoted 17 lines · Julia Lawall
Quoted 2 lines

will update to rc23 and retest - can this be a config/python version issue ? python version is 2.7.3 (Debian 7.6)

I have Python 2.7.5+

Quoted 11 lines
Quoted 3 lines

For the performance problem, could you send the current semantic patch again, so I could be sure to be testing the right thing?

files where this situation exists in linux-next (3.18.0) and which should trigger are:

 drivers/macintosh/ams/ams-pmu.c        line 52
 drivers/misc/sgi-gru/grukservices.c    line 1044
 drivers/scsi/aha152x.c                 line 1055
 drivers/usb/gadget/udc/fsl_qe_udc.c    line 2630
 drivers/usb/gadget/udc/fsl_udc_core.c  line 2529
Quoted 8 lines

I tried these files in their linux-next versions with the -D report option and got an answer more or less immediately (2 seconds). An example command line is:

spatch --sp-file dc.cocci -D report /var/linuxes/linux-next/drivers/scsi/aha152x.c

Does this work for you?

for the single file it returns more or less imediately but it does not report any findings

hofrat at debian:/tmp/linux-next$ spatch --sp-file false_declare_completion.cocci -D report drivers/scsi/aha152x.c
init_defs_builtins: /usr/local/share/coccinelle/standard.h
HANDLING: drivers/scsi/aha152x.c

if this is as fast as expected is hard to say for a single file resolution of "time" is not good enough - but it still looks like its slower than the other two files

hofrat at debian:/tmp/linux-next$ time spatch --sp-file false_declare_completion.cocci -D report drivers/scsi/aha152x.c
init_defs_builtins: /usr/local/share/coccinelle/standard.h
HANDLING: drivers/scsi/aha152x.c

real    0m0.362s
user    0m0.352s
sys     0m0.009s

hofrat at debian:/tmp/linux-next$ time spatch --sp-file false_init_compltion.cocci -D report drivers/scsi/aha152x.c
init_defs_builtins: /usr/local/share/coccinelle/standard.h
(ONCE) Expected tokens init_completion
Skipping:drivers/scsi/aha152x.c

real    0m0.067s
user    0m0.062s
sys     0m0.005s

(false_init_compltion.cocci does not/should not trigger on aha152x.c

hofrat at debian:/tmp/linux-next$ time spatch --sp-file false_init_compltion.cocci -D report drivers/block/aoe/aoecmd.c
init_defs_builtins: /usr/local/share/coccinelle/standard.h
HANDLING: drivers/block/aoe/aoecmd.c
drivers/block/aoe/aoecmd.c:1334:1-16: WARNING: possible reinit by init_completion

real    0m0.174s
user    0m0.151s
sys     0m0.012s

thx!
hofrat

#13 lore raw
Quoted 23 lines · Nicholas Mc Guire
Quoted 21 lines · Julia Lawall
Quoted 2 lines

will update to rc23 and retest - can this be a config/python version issue ? python version is 2.7.3 (Debian 7.6)

I have Python 2.7.5+

Quoted 11 lines
Quoted 3 lines

For the performance problem, could you send the current semantic patch again, so I could be sure to be testing the right thing?

files where this situation exists in linux-next (3.18.0) and which should trigger are:

 drivers/macintosh/ams/ams-pmu.c        line 52
 drivers/misc/sgi-gru/grukservices.c    line 1044
 drivers/scsi/aha152x.c                 line 1055
 drivers/usb/gadget/udc/fsl_qe_udc.c    line 2630
 drivers/usb/gadget/udc/fsl_udc_core.c  line 2529

I tried these files in their linux-next versions with the -D report option and got an answer more or less immediately (2 seconds). An example command line is:

Quoted 7 lines
Quoted 4 lines

spatch --sp-file dc.cocci -D report /var/linuxes/linux-next/drivers/scsi/aha152x.c

Does this work for you?

for the single file it returns more or less imediately but it does not report any findings

I get a result:

/var/linuxes/linux-next/drivers/scsi/aha152x.c:1058:1-19: WARNING:
possible incorrect use of DECLARE_COMPLETION

julia

Quoted 40 lines

hofrat at debian:/tmp/linux-next$ spatch --sp-file false_declare_completion.cocci -D report drivers/scsi/aha152x.c
init_defs_builtins: /usr/local/share/coccinelle/standard.h
HANDLING: drivers/scsi/aha152x.c

if this is as fast as expected is hard to say for a single file resolution of "time" is not good enough - but it still looks like its slower than the other two files

hofrat at debian:/tmp/linux-next$ time spatch --sp-file false_declare_completion.cocci -D report drivers/scsi/aha152x.c
init_defs_builtins: /usr/local/share/coccinelle/standard.h
HANDLING: drivers/scsi/aha152x.c

real    0m0.362s
user    0m0.352s
sys     0m0.009s

hofrat at debian:/tmp/linux-next$ time spatch --sp-file false_init_compltion.cocci -D report drivers/scsi/aha152x.c
init_defs_builtins: /usr/local/share/coccinelle/standard.h
(ONCE) Expected tokens init_completion
Skipping:drivers/scsi/aha152x.c

real    0m0.067s
user    0m0.062s
sys     0m0.005s

(false_init_compltion.cocci does not/should not trigger on aha152x.c

hofrat at debian:/tmp/linux-next$ time spatch --sp-file false_init_compltion.cocci -D report drivers/block/aoe/aoecmd.c
init_defs_builtins: /usr/local/share/coccinelle/standard.h
HANDLING: drivers/block/aoe/aoecmd.c
drivers/block/aoe/aoecmd.c:1334:1-16: WARNING: possible reinit by init_completion

real    0m0.174s
user    0m0.151s
sys     0m0.012s

thx!
hofrat

#14 lore raw
Quoted 32 lines · Julia Lawall
Quoted 28 lines · Nicholas Mc Guire
Quoted 25 lines · Julia Lawall
Quoted 2 lines

will update to rc23 and retest - can this be a config/python version issue ? python version is 2.7.3 (Debian 7.6)

I have Python 2.7.5+

Quoted 11 lines
Quoted 3 lines

For the performance problem, could you send the current semantic patch again, so I could be sure to be testing the right thing?

files where this situation exists in linux-next (3.18.0) and which should trigger are:

 drivers/macintosh/ams/ams-pmu.c        line 52
 drivers/misc/sgi-gru/grukservices.c    line 1044
 drivers/scsi/aha152x.c                 line 1055
 drivers/usb/gadget/udc/fsl_qe_udc.c    line 2630
 drivers/usb/gadget/udc/fsl_udc_core.c  line 2529

I tried these files in their linux-next versions with the -D report option and got an answer more or less immediately (2 seconds). An example command line is:

spatch --sp-file dc.cocci -D report /var/linuxes/linux-next/drivers/scsi/aha152x.c

Does this work for you?

Quoted 8 lines
Quoted 2 lines

for the single file it returns more or less imediately but it does not report any findings

I get a result:

/var/linuxes/linux-next/drivers/scsi/aha152x.c:1058:1-19: WARNING:
possible incorrect use of DECLARE_COMPLETION

with 1.0.0-rc23 I get the expected results ! both report and patch mode 1.0.0-rc21 no output.

is it worth rebuilding -rc21 to confirm that its in -rc21 and not some local screwup or is this not relevant ?

thanks a lot for your support on this !

thx!
hofrat

#15 lore raw
Quoted 43 lines · Nicholas Mc Guire
Quoted 32 lines · Julia Lawall
Quoted 28 lines · Nicholas Mc Guire
Quoted 25 lines · Julia Lawall
Quoted 2 lines

will update to rc23 and retest - can this be a config/python version issue ? python version is 2.7.3 (Debian 7.6)

I have Python 2.7.5+

Quoted 11 lines
Quoted 3 lines

For the performance problem, could you send the current semantic patch again, so I could be sure to be testing the right thing?

files where this situation exists in linux-next (3.18.0) and which should trigger are:

 drivers/macintosh/ams/ams-pmu.c        line 52
 drivers/misc/sgi-gru/grukservices.c    line 1044
 drivers/scsi/aha152x.c                 line 1055
 drivers/usb/gadget/udc/fsl_qe_udc.c    line 2630
 drivers/usb/gadget/udc/fsl_udc_core.c  line 2529

I tried these files in their linux-next versions with the -D report option and got an answer more or less immediately (2 seconds). An example command line is:

spatch --sp-file dc.cocci -D report /var/linuxes/linux-next/drivers/scsi/aha152x.c

Does this work for you?

Quoted 8 lines
Quoted 2 lines

for the single file it returns more or less imediately but it does not report any findings

I get a result:

/var/linuxes/linux-next/drivers/scsi/aha152x.c:1058:1-19: WARNING:
possible incorrect use of DECLARE_COMPLETION

Quoted 5 lines

with 1.0.0-rc23 I get the expected results ! both report and patch mode 1.0.0-rc21 no output.

is it worth rebuilding -rc21 to confirm that its in -rc21 and not some local screwup or is this not relevant ?

rc21 had some severe parsing problems, on strings made from multiple components, like "one" XXX "two". It could be that the relevant functions were simply not being parsed.

julia