Dependencies between spatch hunks
Hi all,
I am finding that a rule which matches a part of code, sometimes causes hunks which depend on it to act, and sometimes not to act
My spatch is successfully removing and converting hunks that I desire changed, however the requirement became apparent *to not process the file at all* if the variable is used in the probe function, so I added in a dependency on probe_id_unused (established in my previous mail thread).
Therefore, I would expect to be able to set the 'depends on' to be on 'probe_id_unused' for each of the actions, and have actions only taken if the full dependency chain (C4->C1 below) is met.
However, I get a non-consistent application of this, where some hunks operate when (I believe) they shouldn't:
My full spatch for reference, is at:
https://gist.github.com/kbingham/96477177dd20a72b1c2f
In essence, it does the following {C}hecks:
C1 - of_dev_id_present : Check for a struct of_device_id
C2 - dev_id : Check for a struct i2c_device_id
C3 - driver : Check and identify the probefunc in the driver structure C4 - probe_id_unused : Establish if the id is used in the probe function
Where C4 depends on C3 depends on C2 depends on C1
The aim is that if all of the above checks/identifiers are met, it will take the following actions:
A rewrite the probe function declaration
B re-point the function pointer in the driver structure
C remove the i2c_device_id reference
D remove the i2c_device_id array
E remove the MODULE_DEVICE_TABLE macro
For this example, I'll take three files from the kernel source, all of which meet conditions C1 - > C3 (but only F3 meets C4):
F1: drivers/gpio/gpio-pca953x.c : probe id used ( ! C4 )
F2: drivers/staging/iio/light/isl29018.c : probe id used ( ! C4 )
F3: sound/soc/codecs/wm8737.c : probe id UNUSED ( C4 )
If all the actions (A->E) start with @ depends on driver @ (to depend on C3) all the actions complete on these files. (Shown in sequence in the c file)
F1 : D E A B C
F2 : A D E B C
F3 : A D E B C
However, if all the actions depend on @ depends on probe_id_unused @ (depends on C4), Some actions complete, and some do not!
F1 : D E C (Unexpected behaviour - I expect F1 : <no change>)
F2 : D E C (Unexpected behaviour - I expect F2 : <no change>)
F3 : A D E B C (Expected behaviour)
So of course, I want actions D E and C to *not* complete on F1 and F2, but I can't understand why they do not comply with their 'depends' chain. Am I looking at a bug in Coccinelle here or a bug with my interpretation of the depends keyword?
Sorry for the long mail, and look forward to any ideas!
--
Regards
Kieran
Quoted 9 lines · Kieran Bingham
Hi all,
I am finding that a rule which matches a part of code, sometimes causes hunks which depend on it to act, and sometimes not to act
My spatch is successfully removing and converting hunks that I desire changed, however the requirement became apparent *to not process the file at all* if the variable is used in the probe function, so I added in a dependency on
probe_id_unused(established in my previous mail thread).
I'll check on the rest shortly, but if you really want to not process the file at all, just look for the condition in which that is the case, and then write a python rule that depends on it, and run Cocci.exit() (not sure about the exact syntax - writing an ocaml rule and putting Coccilib.exit() will also work).
julia
Quoted 65 lines
Therefore, I would expect to be able to set the 'depends on' to be on '
probe_id_unused' for each of the actions, and have actions only taken if the full dependency chain (C4->C1below) is met.However, I get a non-consistent application of this, where some hunks operate when (I believe) they shouldn't:
My full spatch for reference, is at:
https://gist.github.com/kbingham/96477177dd20a72b1c2f
In essence, it does the following {C}hecks:
C1 -of_dev_id_present: Check for a structof_device_id
C2 -dev_id: Check for a structi2c_device_id
C3 - driver : Check and identify the probefunc in the driver structure C4 -probe_id_unused: Establish if the id is used in the probe functionWhere C4 depends on C3 depends on C2 depends on C1
The aim is that if all of the above checks/identifiers are met, it will take the following actions:
A rewrite the probe function declaration
B re-point the function pointer in the driver structure
C remove thei2c_device_idreference
D remove thei2c_device_idarray
E remove theMODULE_DEVICE_TABLEmacroFor this example, I'll take three files from the kernel source, all of which meet conditions C1 - > C3 (but only F3 meets C4):
F1:
drivers/gpio/gpio-pca953x.c: probe id used ( ! C4 )
F2:drivers/staging/iio/light/isl29018.c: probe id used ( ! C4 )
F3:sound/soc/codecs/wm8737.c: probe id UNUSED ( C4 )If all the actions (
A->E) start with @ depends on driver @ (to depend on C3) all the actions complete on these files. (Shown in sequence in the c file)F1 : D E A B C
F2 : A D E B C
F3 : A D E B CHowever, if all the actions depend on @ depends on
probe_id_unused@ (depends on C4), Some actions complete, and some do not!F1 : D E C (Unexpected behaviour - I expect F1 : <no change>)
F2 : D E C (Unexpected behaviour - I expect F2 : <no change>)
F3 : A D E B C (Expected behaviour)So of course, I want actions D E and C to *not* complete on F1 and F2, but I can't understand why they do not comply with their 'depends' chain. Am I looking at a bug in Coccinelle here or a bug with my interpretation of the depends keyword?
Sorry for the long mail, and look forward to any ideas!
--
RegardsKieran
_______________________________________________
Cocci mailing list
Cocci at systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Quoted 13 lines · Julia Lawall
Quoted 9 lines · Kieran Bingham
Hi all,
I am finding that a rule which matches a part of code, sometimes causes hunks which depend on it to act, and sometimes not to act
My spatch is successfully removing and converting hunks that I desire changed, however the requirement became apparent *to not process the file at all* if the variable is used in the probe function, so I added in a dependency on
probe_id_unused(established in my previous mail thread).
Quoted 5 lines
I'll check on the rest shortly, but if you really want to not process the file at all, just look for the condition in which that is the case, and then write a python rule that depends on it, and run
Cocci.exit()(not sure about the exact syntax - writing an ocaml rule and puttingCoccilib.exit()will also work).
Thanks Julia,
The following worked for my needs:
@ script:python depends on driver && !probe_id_unused @ @@print("Probe function uses the ID parameter") cocci.exit()
However, interestingly - it's not useful if I specify a minimal set of files on the command line:
Using the command:
spatch --linux-spacing --sp-file patches/i2c-dt.cocci $A $B $C $D
HANDLING: sound/soc/codecs/wm8737.c drivers/staging/iio/light/isl29018.c drivers/gpio/gpio-pca953x.c ./drivers/rtc/rtc-ds1307.c
Probe function uses the ID parameter
kbingham at CookieMonster:/opt/projects/linux/kbuild-bbb/sources/linux$
appears to stop at the first file (which is expected to stop) without processing the next files.
However, simply using the '.' target - does appear to iterate through the entire sources successfully, parsing files it can, and stopping on files that it should stop on.
Thanks again,
--
Kieran
Quoted 67 lines
julia
Quoted 65 lines
Therefore, I would expect to be able to set the 'depends on' to be on '
probe_id_unused' for each of the actions, and have actions only taken if the full dependency chain (C4->C1below) is met.However, I get a non-consistent application of this, where some hunks operate when (I believe) they shouldn't:
My full spatch for reference, is at:
https://gist.github.com/kbingham/96477177dd20a72b1c2f
In essence, it does the following {C}hecks:
C1 -of_dev_id_present: Check for a structof_device_id
C2 -dev_id: Check for a structi2c_device_id
C3 - driver : Check and identify the probefunc in the driver structure C4 -probe_id_unused: Establish if the id is used in the probe functionWhere C4 depends on C3 depends on C2 depends on C1
The aim is that if all of the above checks/identifiers are met, it will take the following actions:
A rewrite the probe function declaration
B re-point the function pointer in the driver structure
C remove thei2c_device_idreference
D remove thei2c_device_idarray
E remove theMODULE_DEVICE_TABLEmacroFor this example, I'll take three files from the kernel source, all of which meet conditions C1 - > C3 (but only F3 meets C4):
F1:
drivers/gpio/gpio-pca953x.c: probe id used ( ! C4 )
F2:drivers/staging/iio/light/isl29018.c: probe id used ( ! C4 )
F3:sound/soc/codecs/wm8737.c: probe id UNUSED ( C4 )If all the actions (
A->E) start with @ depends on driver @ (to depend on C3) all the actions complete on these files. (Shown in sequence in the c file)F1 : D E A B C
F2 : A D E B C
F3 : A D E B CHowever, if all the actions depend on @ depends on
probe_id_unused@ (depends on C4), Some actions complete, and some do not!F1 : D E C (Unexpected behaviour - I expect F1 : <no change>)
F2 : D E C (Unexpected behaviour - I expect F2 : <no change>)
F3 : A D E B C (Expected behaviour)So of course, I want actions D E and C to *not* complete on F1 and F2, but I can't understand why they do not comply with their 'depends' chain. Am I looking at a bug in Coccinelle here or a bug with my interpretation of the depends keyword?
Sorry for the long mail, and look forward to any ideas!
--
RegardsKieran
_______________________________________________
Cocci mailing list
Cocci at systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Quoted 40 lines · Kieran Bingham
Quoted 13 lines · Julia Lawall
Quoted 9 lines · Kieran Bingham
Hi all,
I am finding that a rule which matches a part of code, sometimes causes hunks which depend on it to act, and sometimes not to act
My spatch is successfully removing and converting hunks that I desire changed, however the requirement became apparent *to not process the file at all* if the variable is used in the probe function, so I added in a dependency on
probe_id_unused(established in my previous mail thread).Quoted 5 lines
I'll check on the rest shortly, but if you really want to not process the file at all, just look for the condition in which that is the case, and then write a python rule that depends on it, and run
Cocci.exit()(not sure about the exact syntax - writing an ocaml rule and puttingCoccilib.exit()will also work).Thanks Julia,
The following worked for my needs:
@ script:python depends on driver && !probe_id_unused @ @@
print("Probe function uses the ID parameter")cocci.exit()However, interestingly - it's not useful if I specify a minimal set of files on the command line:
Using the command:
spatch --linux-spacing --sp-filepatches/i2c-dt.cocci$A $B $C $D
HANDLING:sound/soc/codecs/wm8737.cdrivers/staging/iio/light/isl29018.cdrivers/gpio/gpio-pca953x.c./drivers/rtc/rtc-ds1307.c
Probe function uses the ID parameter
kbingham at CookieMonster:/opt/projects/linux/kbuild-bbb/sources/linux$
Quoted 6 lines
appears to stop at the first file (which is expected to stop) without processing the next files.
However, simply using the '.' target - does appear to iterate through the entire sources successfully, parsing files it can, and stopping on files that it should stop on.
Putting multiple files on the command line is not the right way to process multiple files. What that means is to process those files all at once. Normally Coccinelle processes only one file at a time, which means that when there is a call from a function in one file to a function in another, it won't find the definition of the called function. If you put multiple files on the command line, it will see them all at once. The downside though is that Coccinelle will ultimately run more slowly, because it will be using a lot more memory at a given time.
If you have a directory where you want to process only some of the files, you can use the --file-groups <file> option. <file> could be eg
path/foo.cpath/bar.c
path/xyz.c
Then foo.c and bar.c will be processed at once, and xyz.c will be processed separately.
But it could be easier to just work on the whole directory and let Coccinelle to fail to do anything with most of the files.
julia
Quoted 5 lines · Julia Lawall
On Wed, 20 Apr 2016, Kieran Bingham wrote:
Quoted 7 lines
Quoted 7 lines · Julia Lawall
Quoted 6 lines · Kieran Bingham
Quoted 4 lines
Hi all,
I am finding that a rule which matches a part of code, sometimes
causes
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
hunks which depend on it to act, and sometimes not to act
My spatch is successfully removing and converting hunks that I desire changed, however the requirement became apparent *to not process the file at all* if the variable is used in the probe function, so I
added
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
in a dependency on
probe_id_unused(established in my previous mail
thread).
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
I'll check on the rest shortly, but if you really want to not process
the
Quoted 1 line
Quoted 1 line
Quoted 1 line
file at all, just look for the condition in which that is the case,
and
Quoted 46 lines
Quoted 24 lines
Quoted 3 lines
then write a python rule that depends on it, and run
Cocci.exit()(not sure about the exact syntax - writing an ocaml rule and puttingCoccilib.exit()will also work).Thanks Julia,
The following worked for my needs:
@ script:python depends on driver && !probe_id_unused @ @@
print("Probe function uses the ID parameter")cocci.exit()However, interestingly - it's not useful if I specify a minimal set of files on the command line:
Using the command:
spatch --linux-spacing --sp-filepatches/i2c-dt.cocci$A $B $C $D
HANDLING:sound/soc/codecs/wm8737.cdrivers/staging/iio/light/isl29018.cdrivers/gpio/gpio-pca953x.c./drivers/rtc/rtc-ds1307.c
Probe function uses the ID parameter
kbingham at CookieMonster:/opt/projects/linux/kbuild-bbb/sources/linux$Quoted 6 lines
appears to stop at the first file (which is expected to stop) without processing the next files.
However, simply using the '.' target - does appear to iterate through the entire sources successfully, parsing files it can, and stopping on files that it should stop on.
Putting multiple files on the command line is not the right way to process multiple files. What that means is to process those files all at once. Normally Coccinelle processes only one file at a time, which means that when there is a call from a function in one file to a function in another, it won't find the definition of the called function. If you put multiple files on the command line, it will see them all at once. The downside though is that Coccinelle will ultimately run more slowly, because it will be using a lot more memory at a given time.
If you have a directory where you want to process only some of the files, you can use the --file-groups <file> option. <file> could be eg
path/foo.cpath/bar.c
Quoted 7 lines
path/xyz.cThen
foo.candbar.cwill be processed at once, andxyz.cwill be processed separately.But it could be easier to just work on the whole directory and let Coccinelle to fail to do anything with most of the files.
Ah I see. I bet that explains my other dependency issue as well. It was probably matching state from other processed files in the same processing group.
I had been specifying a group of files to make sure the right thing was being done in the processing. In the whole tree about 250 files match so I was trying to reduce the set to verify.
It looks like the act of me trying to reduce the set changed the processing though ;-)
I guess I could create a single folder with the reduced set in, but it looks like its doing the right thing now with the exit ()
Thanks
Kieran
Quoted 2 lines
julia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://systeme.lip6.fr/pipermail/cocci/attachments/20160420/81814080/attachment.html>
Quoted 91 lines · Kieran Bingham
Quoted 5 lines · Julia Lawall
On Wed, 20 Apr 2016, Kieran Bingham wrote:
Quoted 7 lines
Quoted 7 lines · Julia Lawall
Quoted 6 lines · Kieran Bingham
Quoted 4 lines
Hi all,
I am finding that a rule which matches a part of code, sometimes
causes
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
hunks which depend on it to act, and sometimes not to act
My spatch is successfully removing and converting hunks that I desire changed, however the requirement became apparent *to not process the file at all* if the variable is used in the probe function, so I
added
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
in a dependency on
probe_id_unused(established in my previous mailthread).
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
I'll check on the rest shortly, but if you really want to not process
the
Quoted 1 line
Quoted 1 line
Quoted 1 line
file at all, just look for the condition in which that is the case,
and
Quoted 46 lines
Quoted 24 lines
Quoted 3 lines
then write a python rule that depends on it, and run
Cocci.exit()(not sure about the exact syntax - writing an ocaml rule and puttingCoccilib.exit()will also work).Thanks Julia,
The following worked for my needs:
@ script:python depends on driver && !probe_id_unused @ @@
print("Probe function uses the ID parameter")cocci.exit()However, interestingly - it's not useful if I specify a minimal set of files on the command line:
Using the command:
spatch --linux-spacing --sp-filepatches/i2c-dt.cocci$A $B $C $D
HANDLING:sound/soc/codecs/wm8737.cdrivers/staging/iio/light/isl29018.cdrivers/gpio/gpio-pca953x.c./drivers/rtc/rtc-ds1307.c
Probe function uses the ID parameter
kbingham at CookieMonster:/opt/projects/linux/kbuild-bbb/sources/linux$Quoted 6 lines
appears to stop at the first file (which is expected to stop) without processing the next files.
However, simply using the '.' target - does appear to iterate through the entire sources successfully, parsing files it can, and stopping on files that it should stop on.
Putting multiple files on the command line is not the right way to process multiple files.? What that means is to process those files all at once. Normally Coccinelle processes only one file at a time, which means that when there is a call from a function in one file to a function in another, it won't find the definition of the called function.? If you put multiple files on the command line, it will see them all at once.? The downside though is that Coccinelle will ultimately run more slowly, because it will be using a lot more memory at a given time.
If you have a directory where you want to process only some of the files, you can use the --file-groups <file> option.? <file> could be eg
path/foo.cpath/bar.cQuoted 7 lines
path/xyz.cThen
foo.candbar.cwill be processed at once, andxyz.cwill be processed separately.But it could be easier to just work on the whole directory and let Coccinelle to fail to do anything with most of the files.
Ah I see.? I bet that explains my other dependency issue as well.? It was probably matching state from other processed files in the same processing group.
I had been specifying a group of files to make sure the right thing was being done in the processing. In the whole tree about 250 files match so I was trying to reduce the set to verify.?
Quoted 5 lines
It looks like the act of me trying to reduce the set changed the processing though ;-)
I guess I could create a single folder with the reduced set in, but it looks like its doing the right thing now with the exit ()
You may find it useful to use glimpse or id-utils to index your files. In coccinelle/scripts there are idutils_index.sh and glimpseindex_cocci.sh, respectively. The you can use --use-glimpse or --use-idutils, and it will go directly to the files that contain the important keywords of your semantic patch. Not parsing the other files will be a big savings. The downside of course is that you have to redo the index every time your code base changes.
julia
Quoted 59 lines · Kieran Bingham
Hi all,
I am finding that a rule which matches a part of code, sometimes causes hunks which depend on it to act, and sometimes not to act
My spatch is successfully removing and converting hunks that I desire changed, however the requirement became apparent *to not process the file at all* if the variable is used in the probe function, so I added in a dependency on
probe_id_unused(established in my previous mail thread).Therefore, I would expect to be able to set the 'depends on' to be on '
probe_id_unused' for each of the actions, and have actions only taken if the full dependency chain (C4->C1below) is met.However, I get a non-consistent application of this, where some hunks operate when (I believe) they shouldn't:
My full spatch for reference, is at:
https://gist.github.com/kbingham/96477177dd20a72b1c2f
In essence, it does the following {C}hecks:
C1 -of_dev_id_present: Check for a structof_device_id
C2 -dev_id: Check for a structi2c_device_id
C3 - driver : Check and identify the probefunc in the driver structure C4 -probe_id_unused: Establish if the id is used in the probe functionWhere C4 depends on C3 depends on C2 depends on C1
The aim is that if all of the above checks/identifiers are met, it will take the following actions:
A rewrite the probe function declaration
B re-point the function pointer in the driver structure
C remove thei2c_device_idreference
D remove thei2c_device_idarray
E remove theMODULE_DEVICE_TABLEmacroFor this example, I'll take three files from the kernel source, all of which meet conditions C1 - > C3 (but only F3 meets C4):
F1:
drivers/gpio/gpio-pca953x.c: probe id used ( ! C4 )
F2:drivers/staging/iio/light/isl29018.c: probe id used ( ! C4 )
F3:sound/soc/codecs/wm8737.c: probe id UNUSED ( C4 )If all the actions (
A->E) start with @ depends on driver @ (to depend on C3) all the actions complete on these files. (Shown in sequence in the c file)F1 : D E A B C
F2 : A D E B C
F3 : A D E B CHowever, if all the actions depend on @ depends on
probe_id_unused@ (depends on C4), Some actions complete, and some do not!F1 : D E C (Unexpected behaviour - I expect F1 : <no change>)
F2 : D E C (Unexpected behaviour - I expect F2 : <no change>)
F3 : A D E B C (Expected behaviour)
Quoted 6 lines
So of course, I want actions D E and C to *not* complete on F1 and F2, but I can't understand why they do not comply with their 'depends' chain. Am I looking at a bug in Coccinelle here or a bug with my interpretation of the depends keyword?
Sorry for the long mail, and look forward to any ideas!
What version of Coccinelle do you have? I get the expected behavior. No change on F1 and F2, and a change on F3.
julia
Quoted 7 lines · Julia Lawall
Quoted 2 lines · Kieran Bingham
Hi all,
Quoted 7 lines
Quoted 7 lines
I am finding that a rule which matches a part of code, sometimes causes hunks which depend on it to act, and sometimes not to act
My spatch is successfully removing and converting hunks that I desire changed, however the requirement became apparent *to not process the file at all* if the variable is used in the probe function, so I added in a dependency on
probe_id_unused(established in my previous mail
thread).
Quoted 55 lines
Quoted 55 lines
Therefore, I would expect to be able to set the 'depends on' to be on '
probe_id_unused' for each of the actions, and have actions only taken if the full dependency chain (C4->C1below) is met.However, I get a non-consistent application of this, where some hunks operate when (I believe) they shouldn't:
My full spatch for reference, is at:
https://gist.github.com/kbingham/96477177dd20a72b1c2f
In essence, it does the following {C}hecks:
C1 -of_dev_id_present: Check for a structof_device_id
C2 -dev_id: Check for a structi2c_device_id
C3 - driver : Check and identify the probefunc in the driver structure C4 -probe_id_unused: Establish if the id is used in the probe functionWhere C4 depends on C3 depends on C2 depends on C1
The aim is that if all of the above checks/identifiers are met, it will take the following actions:
A rewrite the probe function declaration
B re-point the function pointer in the driver structure
C remove thei2c_device_idreference
D remove thei2c_device_idarray
E remove theMODULE_DEVICE_TABLEmacroFor this example, I'll take three files from the kernel source, all of which meet conditions C1 - > C3 (but only F3 meets C4):
F1:
drivers/gpio/gpio-pca953x.c: probe id used ( ! C4 )
F2:drivers/staging/iio/light/isl29018.c: probe id used ( ! C4 )
F3:sound/soc/codecs/wm8737.c: probe id UNUSED ( C4 )If all the actions (
A->E) start with @ depends on driver @ (to depend on C3) all the actions complete on these files. (Shown in sequence in the c file)F1 : D E A B C
F2 : A D E B C
F3 : A D E B CHowever, if all the actions depend on @ depends on
probe_id_unused@ (depends on C4), Some actions complete, and some do not!F1 : D E C (Unexpected behaviour - I expect F1 : <no change>)
F2 : D E C (Unexpected behaviour - I expect F2 : <no change>)
F3 : A D E B C (Expected behaviour)So of course, I want actions D E and C to *not* complete on F1 and F2, but I can't understand why they do not comply with their 'depends' chain. Am I looking at a bug in Coccinelle here or a bug with my interpretation of the depends keyword?
Quoted 4 lines
Quoted 1 line
Sorry for the long mail, and look forward to any ideas!
What version of Coccinelle do you have? I get the expected behavior. No change on F1 and F2, and a change on F3.
I think the fault was somewhere between the keyboard and chair :-)
I'm away from my keyboard now, so I can't check but I'll assume it was because I was specifying the files in one command line.
Regards
Kieran
Quoted 2 lines
julia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://systeme.lip6.fr/pipermail/cocci/attachments/20160420/dfcb6499/attachment-0001.html>