coccinelle: expand docs
v2 cover coccinelle: expand docs
This v2 modifies the ++ demo and documentation to something a bit more safe for its use, and while at it decided to extend the iteration documentation and document support for .cocciconfig.
Luis R. Rodriguez (3):
docs/demos: add a few ++ documentation and demos docs/manual/cocci_syntax.tex: extend with python iteration
docs: document .cocciconfig
demos/plusplus1.c | 4 + demos/plusplus1.cocci | 17 +++++ demos/plusplus1.res | 5 ++ demos/plusplus2.c | 7 ++ demos/plusplus2.cocci | 26 +++++++ demos/plusplus2.res | 10 +++ docs/manual/cocci_syntax.tex | 169 ++++++++++++++++++++++++++++++++++++++++--- docs/spatch.1.in | 32 +++++++- 8 files changed, 258 insertions(+), 12 deletions(-) create mode 100644 demos/plusplus1.c create mode 100644 demos/plusplus1.cocci create mode 100644 demos/plusplus1.res create mode 100644 demos/plusplus2.c create mode 100644 demos/plusplus2.cocci create mode 100644 demos/plusplus2.res
Signature · 1 line
2.8.2
PATCH v2 1/3 docs/demos: add a few ++ documentation and demos
This perhaps is not the best demo for use of ++ but it should suffice. This adds some basic documentation for it and a demo.
--- demos/plusplus1.c | 4 ++ demos/plusplus1.cocci | 17 ++++++++ demos/plusplus1.res | 5 +++ demos/plusplus2.c | 7 ++++ demos/plusplus2.cocci | 26 ++++++++++++ demos/plusplus2.res | 10 +++++ docs/manual/cocci_syntax.tex | 95 ++++++++++++++++++++++++++++++++++++++++++++ 7 files changed, 164 insertions(+) create mode 100644 demos/plusplus1.c create mode 100644 demos/plusplus1.cocci create mode 100644 demos/plusplus1.res create mode 100644 demos/plusplus2.c create mode 100644 demos/plusplus2.cocci create mode 100644 demos/plusplus2.res
demos/plusplus1.c new file +4 -0
@@ -0,0 +1,4 @@ +struct x { +int z; +char b; +};
demos/plusplus1.cocci new file +17 -0
@@ -0,0 +1,17 @@ +// This adds a float for one int variable found. +// Doing this however is limited to one int in the data structure, if +// more one int variable in the structure we would not know where to +// place the new one safely order-wise. See plusplus2.cocci for an +// example of dealing with this issue. + + at simpleplus@ +identifier x,v; +fresh identifier xx = v ## "_float"; +@@ + +struct x { ++ float xx; + ... + int v; + ... +}
demos/plusplus1.res new file +5 -0
@@ -0,0 +1,5 @@ +struct x { + float z_float; + int z; + char b; +};
demos/plusplus2.c new file +7 -0
@@ -0,0 +1,7 @@ +struct x { +int z; +int a; +char b; +int c; +int *d; +};
demos/plusplus2.cocci new file +26 -0
@@ -0,0 +1,26 @@ +// This can support having a structure with more than two int variables, +// and adding a respective float for each. It can do this as order does +// not matter. +// +// This uses ++ to support the fact that the rule may be working +// with multiple variables that we need to modify and that order +// does not matter. +// +// If you don't use "++" you'll get "already tagged token" error since +// Coccinelle is concerned that the user has no way of specifying the order +// in which they should appear. By using "++" you are telling Coccinelle +// +// "I know that a lot of things can collect here, and I'm OK +// with that. I'm also OK with things getting added out of order. + + at plusplus@ +identifier x,v; +fresh identifier xx = v ## "_float"; +@@ + +struct x { +++ float xx; + ... + int v; + ... +}
demos/plusplus2.res new file +10 -0
@@ -0,0 +1,10 @@ +struct x { + float a_float; + float c_float; + float z_float; + int z; + int a; + char b; + int c; + int *d; +};
docs/manual/cocci_syntax.tex +95 -0
107 lines not on this page · raw message
Signature · 1 line
2.8.2
PATCH v2 2/3 docs/manual/cocci_syntax.tex: extend with python iteration
In reply to Luis R. Rodriguez--- docs/manual/cocci_syntax.tex | 51 ++++++++++++++++++++++++++++++++++---------- 1 file changed, 40 insertions(+), 11 deletions(-)
docs/manual/cocci_syntax.tex +40 -11
@@ -1698,9 +1698,12 @@ of every token. \section{Iteration} It is possible to iterate Coccinelle, giving the subsequent iterations a -different set of virtual rules or virtual identifier bindings. And example -is found in {\tt demos/iteration.cocci}. The example shown there is as -follows: +different set of virtual rules or virtual identifier bindings. Coccinelle +currently supports iteration with both ocaml and python scripting. An +example with ocaml is fond in {\tt demos/iteration.cocci}, a python +exmaple is found in {\tt demos/python\_iteration.cocci}. + +The ocaml scripting iteration example starts as follows. \begin{quote} \begin{verbatim}
@@ -1724,6 +1727,29 @@ with Not_found -> \end{verbatim} \end{quote} +The respective python scripting iteration example starts as follows: + +\begin{quote} +\begin{verbatim} +virtual after_start + + at initialize:python@ +@@ + +seen = set() + +def add_if_not_present (source, f, file): + if (f, file) not in seen: + seen.add((f, file)) + it = Iteration() + if file != None: + it.set_files([file]) + it.add_virtual_rule(after_start) + it.add_virtual_identifier(err_ptr_function, f) + it.register() +\end{verbatim} +\end{quote} + The virtual rule {\tt after\_start} is used to distinguish between the first iteration (in which it is not considered to have matched) and all others. This is done by not mentioning {\tt after\_start} in the command
@@ -1734,11 +1760,12 @@ The main code for performing the iteration is found in the function {\tt {\tt register}. {\tt New iteration} creates a structure representing the new iteration. {\tt set\_files} sets the list of files to be considered on the new iteration. If this function is not called, the new iteration -treats the same files as the current iteration. {\tt add\_virtual\_rule A} -has the same effect as putting {\tt -D a} on the command line. Note that -the first letter of the rule name is capitalized, although this is not done -elsewhere. {\tt add\_virtual\_identifier X v} has the same effect as -putting {\tt -D x=v} on the command line. Note again the case change. +treats the same files as the current iteration. {\tt add\_virtual\_rule a} +has the same effect as putting {\tt -D a} on the command line. If +using ocaml scripting instead of python scripting the first letter of the rule +name is capitalized, although this is not done elsewhere. +{\tt add\_virtual\_identifier x v} has the same effect as putting {\tt -D x=v} +on the command line. Again, when using ocaml scripting there is a case change. {\tt extend\_virtual\_identifiers()} (not shown) preserves all virtual identifiers of the current iteration that are not overridden by calls to {\tt add\_virtual\_identifier}. Finally, the call to {\tt register} queues
@@ -1749,9 +1776,11 @@ Modification is not allowed when using iteration. Thus, it is required to use the {\tt --no-show-diff}, unless the semantic patch contains {\tt *}s (a semantic match rather than a semantic patch). -The remainder of the code above uses a hash table to ensure that the same -information is not enqueued more than once. Coccinelle itself provides no -support for this. +When using python scripting a tuple is used throughout the rest of the +code to ensure that the same information is not enqueued more than once. +When using ocaml scripting a hash table is used for the same purposes. +Coccinelle itself provides no support for this and as such addressing +this with scripting is necessary. %%% Local Variables: %%% mode: LaTeX
Signature · 1 line
2.8.2
PATCH v2 3/3 docs: document .cocciconfig
In reply to Luis R. RodriguezAdd documentation and example to manual and man page.
--- docs/manual/cocci_syntax.tex | 23 +++++++++++++++++++++++ docs/spatch.1.in | 32 +++++++++++++++++++++++++++++++- 2 files changed, 54 insertions(+), 1 deletion(-)
docs/manual/cocci_syntax.tex +23 -0
@@ -1782,6 +1782,29 @@ When using ocaml scripting a hash table is used for the same purposes. Coccinelle itself provides no support for this and as such addressing this with scripting is necessary. +\section{.cocciconfig support} + +Coccinelle supports enabling custom options to be preferred when running +spatch. This is supported through the search of .cocciconfig files in each of +the following directories, later lines extend and may override earlier ones: + +\begin{itemize} + \item Your current user's home directory is processed first + \item Your directory from which spatch is called is processed next + \item The directory provided with the --dir option is processed last, if used +\end{itemize} + +Newlines, even with \, are not tolerated in attribute values. An example +follows: + +\begin{quote} +\begin{verbatim} +[spatch] + options = --jobs 4 + options = --show-trying +\end{verbatim} +\end{quote} + %%% Local Variables: %%% mode: LaTeX %%% TeX-master: "main_grammar"
docs/spatch.1.in +31 -1
@@ -55,7 +55,8 @@ and at the \fBscripts/coccinelle\fP directory of the Linux Kernel source code. .SH OPTIONS -Here is a summary of the most commonly used options: +Here is a summary of the most commonly used options (also +see the "Configuration Mechanism" section below): .TP .B \-\-sp\-file \fI<file>\fP
@@ -566,6 +567,31 @@ It's for the other (internal) uses of the spatch program. \fB\-\-compare\-c\fR <file1> <file2> +.SH CONFIGURATION MECHANISM + +Coccinelle uses a simple text format to store customizations into +\.cocciconfig files, that can be per project, user, or target. The order +in which the configuration file is processed is as follows, the +later lines always extend and may override earlier ones: + + o Your current user's home directory is processed first + o Your directory from which spatch is called is processed next + o The directory provided with the --dir option is processed last, if used + +Such a configuration file may look like this: + +.if n \{\ +.RS 4 +.\} +.nf +[spatch] + options = --jobs 4 + options = --show-trying +.fi +.if n \{\ +.RE +.\} + .SH FILES .I @SHAREDIR@/standard.iso .RS
@@ -574,6 +600,10 @@ This file contains the default set of isomorphisms. .I @SHAREDIR@/standard.h .RS This file contains the default set of macro hints. +.RE +.I .cocciconfig +.RS +This file contains the custom set of spatch options. .SH ENVIRONMENT .IP COCCINELLE_HOME
Signature · 1 line
2.8.2
PATCH v2 1/3 docs/demos: add a few ++ documentation and demos
In reply to Luis R. RodriguezQuoted 1 line
+Coccinelle allows for transformations to enable modifying C code using
Can the word "for" be omitted here?
Quoted 1 line
+You may run into the situation where grammar you specificy for
Did you overlook to fix a typo here once more?
? specify ?
Quoted 1 line
+in code, in such cases Coccinelle cannot gaurantee order when making
? guarantee ?
Quoted 1 line
+additions. If you are sure that order does not matter you can use the
Have you got other preferences (than me) for the usage of commas?
Quoted 1 line
+may add things out of order. This may be for intance rather safe in
? instance ?
Quoted 1 line
+precise order of the where each float is placed is however not gauranteed
? guaranteed
How do you think about to improve spell checking for such update suggestions?
Regards,
Markus
PATCH v2 2/3 docs/manual/cocci_syntax.tex: extend with python iteration
In reply to Luis R. RodriguezQuoted 1 line
Signed-off-by Luis R. Rodriguez <[email protected]>
Would a short commit message be nice here?
Quoted 1 line
+example with ocaml is fond in {\tt
demos/iteration.cocci}, a python
* How do you think about the programming language name "OCaml"?
* Another typo correction:
? found ?
Regards,
Markus
Julia Lawall
AppliedPATCH v2 1/3 docs/demos: add a few ++ documentation and demos
In reply to Luis R. RodriguezQuoted 5 lines · Luis R. Rodriguez
This perhaps is not the best demo for use of ++ but it should suffice. This adds some basic documentation for it and a demo.
Signed-off-by Luis R. Rodriguez <[email protected]>
Applied. I have subsequently somewhat rewritten the text for docs/manual/cocci_syntax.tex.
julia
Quoted 242 lines
--- demos/plusplus1.c | 4 ++ demos/plusplus1.cocci | 17 ++++++++ demos/plusplus1.res | 5 +++ demos/plusplus2.c | 7 ++++ demos/plusplus2.cocci | 26 ++++++++++++ demos/plusplus2.res | 10 +++++ docs/manual/cocci_syntax.tex | 95 ++++++++++++++++++++++++++++++++++++++++++++ 7 files changed, 164 insertions(+) create mode 100644 demos/plusplus1.c create mode 100644 demos/plusplus1.cocci create mode 100644 demos/plusplus1.res create mode 100644 demos/plusplus2.c create mode 100644 demos/plusplus2.cocci create mode 100644 demos/plusplus2.resdemos/plusplus1.c new file +4
-0@@ -0,0 +1,4 @@ +struct x { +int z; +char b; +};demos/plusplus1.cocci new file +17
-0@@ -0,0 +1,17 @@ +// This adds a float for one int variable found. +// Doing this however is limited to one int in the data structure, if +// more one int variable in the structure we would not know where to +// place the new one safely order-wise. See plusplus2.cocci for an +// example of dealing with this issue. + + at simpleplus@ +identifier x,v; +fresh identifier xx = v ## "_float"; +@@ + +struct x { ++ float xx; + ... + int v; + ... +}demos/plusplus1.res new file +5
-0@@ -0,0 +1,5 @@ +struct x { + float z_float; + int z; + char b; +};demos/plusplus2.c new file +7
-0@@ -0,0 +1,7 @@ +struct x { +int z; +int a; +char b; +int c; +int *d; +};demos/plusplus2.cocci new file +26
-0@@ -0,0 +1,26 @@ +// This can support having a structure with more than two int variables, +// and adding a respective float for each. It can do this as order does +// not matter. +// +// This uses ++ to support the fact that the rule may be working +// with multiple variables that we need to modify and that order +// does not matter. +// +// If you don't use "++" you'll get "already tagged token" error since +// Coccinelle is concerned that the user has no way of specifying the order +// in which they should appear. By using "++" you are telling Coccinelle +// +// "I know that a lot of things can collect here, and I'm OK +// with that. I'm also OK with things getting added out of order. + + at plusplus@ +identifier x,v; +fresh identifier xx = v ## "_float"; +@@ + +struct x { +++ float xx; + ... + int v; + ... +}demos/plusplus2.res new file +10
-0@@ -0,0 +1,10 @@ +struct x { + float a_float; + float c_float; + float z_float; + int z; + int a; + char b; + int c; + int *d; +};docs/manual/cocci_syntax.tex +95
-1@@ -851,6 +851,11 @@ rule should apply if rule XXX was never matched at all. \section{Transformation} +Coccinelle allows for transformations to enable modifying C code using +very precise grammar. + +\subsection{Basic transformations} + The transformation specification essentially has the form of C code, except that lines to remove are annotated with \verb+-+ in the first column, and lines to add are annotated with \verb-+-. A transformation specification@@ -1100,6 +1105,96 @@ write Some kinds of terms can only appear in + code. These include comments, ifdefs, and attributes (\texttt{\_\_attribute\_\_((...))}). +\subsection{Advanced transformations} + +You may run into the situation where grammar you specificy for +transformations might associate itself with a few consecutive tokens +in code, in such cases Coccinelle cannot gaurantee order when making +additions. If you are sure that order does not matter you can use the +optional double addition token \texttt{++} to annotate that Coccinelle +may add things out of order. This may be for intance rather safe in +certain situations when extending a data structure with more members, +based on existing members of the data structure. The following rule helps +to extend a data structure with a respective float for one present int. +This works well with the simple \texttt{+}. + +\begin{lstlisting}[language=Cocci] + at simpleplus@ +identifier x,v; +fresh identifier xx = v ## "_float"; +@@ + +struct x { ++ float xx; + ... + int v; + ... +} +\end{lstlisting} + +This simpleplus rule transformation works well when only one token is being +modified in code. For instance if we had plusplus1.c: + +\begin{lstlisting}[language=C] +struct x { + int z; + char b; +}; +\end{lstlisting} + +If however you have two consecutive tokens that Coccinelle +can transform, order cannot be guaranteed for how Coccinelle +makes additions. If you are sure order does not matter for +the transformation you may use \texttt{++} instead, as follows: + +\begin{lstlisting}[language=Cocci] + at plusplus@ +identifier x,v; +fresh identifier xx = v ## "_float"; +@@ + +struct x { +++ float xx; + ... + int v; + ... +} +\end{lstlisting} + +This rule would work against say a plusplus2.c which had two +more tokens added in comparison to plusplus1.c: + +\begin{lstlisting}[language=C] +struct x { + int z; + int a; + char b; + int c; + int *d; +}; +\end{lstlisting} + +You would end up with a C file which would look similar to this, the +precise order of the where each float is placed is however not gauranteed +with respect to each other: + +\begin{lstlisting}[language=C] +struct x { + float a_float; + float c_float; + float z_float; + int z; + int a; + char b; + int c; + int *d; +}; +\end{lstlisting} + +If you used simpleplus rule on plusplus2.c you would end up with +an "already tagged token" error due to the ordering considerations +explained in this section. + \section{Types} \label{types} --2.8.2
_______________________________________________
Cocci mailing list
Cocci at systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Julia Lawall
AppliedPATCH v2 2/3 docs/manual/cocci_syntax.tex: extend with python iteration
In reply to Luis R. RodriguezQuoted 1 line · Luis R. Rodriguez
Signed-off-by Luis R. Rodriguez <[email protected]>
Applied, with a few subsequent adjustments.
julia
Quoted 95 lines
--- docs/manual/cocci_syntax.tex | 51 ++++++++++++++++++++++++++++++++++---------- 1 file changed, 40 insertions(+), 11 deletions(-)docs/manual/cocci_syntax.tex +40
-12@@ -1698,9 +1698,12 @@ of every token. \section{Iteration} It is possible to iterate Coccinelle, giving the subsequent iterations a -different set of virtual rules or virtual identifier bindings. And example -is found in {\tt demos/iteration.cocci}. The example shown there is as -follows: +different set of virtual rules or virtual identifier bindings. Coccinelle +currently supports iteration with both ocaml and python scripting. An +example with ocaml is fond in {\tt demos/iteration.cocci}, a python +exmaple is found in {\tt demos/python\_iteration.cocci}. + +The ocaml scripting iteration example starts as follows. \begin{quote} \begin{verbatim}@@ -1724,6 +1727,29 @@ with Not_found -> \end{verbatim} \end{quote} +The respective python scripting iteration example starts as follows: + +\begin{quote} +\begin{verbatim} +virtual after_start + + at initialize:python@ +@@ + +seen = set() + +def add_if_not_present (source, f, file): + if (f, file) not in seen: + seen.add((f, file)) + it = Iteration() + if file != None: + it.set_files([file]) + it.add_virtual_rule(after_start) + it.add_virtual_identifier(err_ptr_function, f) + it.register() +\end{verbatim} +\end{quote} + The virtual rule {\tt after\_start} is used to distinguish between the first iteration (in which it is not considered to have matched) and all others. This is done by not mentioning {\tt after\_start} in the command@@ -1734,11 +1760,12 @@ The main code for performing the iteration is found in the function {\tt {\tt register}. {\tt New iteration} creates a structure representing the new iteration. {\tt set\_files} sets the list of files to be considered on the new iteration. If this function is not called, the new iteration -treats the same files as the current iteration. {\tt add\_virtual\_rule A} -has the same effect as putting {\tt -D a} on the command line. Note that -the first letter of the rule name is capitalized, although this is not done -elsewhere. {\tt add\_virtual\_identifier X v} has the same effect as -putting {\tt -D x=v} on the command line. Note again the case change. +treats the same files as the current iteration. {\tt add\_virtual\_rule a} +has the same effect as putting {\tt -D a} on the command line. If +using ocaml scripting instead of python scripting the first letter of the rule +name is capitalized, although this is not done elsewhere. +{\tt add\_virtual\_identifier x v} has the same effect as putting {\tt -D x=v} +on the command line. Again, when using ocaml scripting there is a case change. {\tt extend\_virtual\_identifiers()} (not shown) preserves all virtual identifiers of the current iteration that are not overridden by calls to {\tt add\_virtual\_identifier}. Finally, the call to {\tt register} queues@@ -1749,9 +1776,11 @@ Modification is not allowed when using iteration. Thus, it is required to use the {\tt --no-show-diff}, unless the semantic patch contains {\tt *}s (a semantic match rather than a semantic patch). -The remainder of the code above uses a hash table to ensure that the same -information is not enqueued more than once. Coccinelle itself provides no -support for this. +When using python scripting a tuple is used throughout the rest of the +code to ensure that the same information is not enqueued more than once. +When using ocaml scripting a hash table is used for the same purposes. +Coccinelle itself provides no support for this and as such addressing +this with scripting is necessary. %%% Local Variables: %%% mode: LaTeX --2.8.2
_______________________________________________
Cocci mailing list
Cocci at systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Julia Lawall
AppliedPATCH v2 3/3 docs: document .cocciconfig
In reply to Luis R. RodriguezQuoted 3 lines · Luis R. Rodriguez
Add documentation and example to manual and man page.
Signed-off-by Luis R. Rodriguez <[email protected]>
Applied.
Quoted 104 lines
--- docs/manual/cocci_syntax.tex | 23 +++++++++++++++++++++++ docs/spatch.1.in | 32 +++++++++++++++++++++++++++++++- 2 files changed, 54 insertions(+), 1 deletion(-)docs/manual/cocci_syntax.tex +23
-0@@ -1782,6 +1782,29 @@ When using ocaml scripting a hash table is used for the same purposes. Coccinelle itself provides no support for this and as such addressing this with scripting is necessary. +\section{.cocciconfig support} + +Coccinelle supports enabling custom options to be preferred when running +spatch. This is supported through the search of .cocciconfig files in each of +the following directories, later lines extend and may override earlier ones: + +\begin{itemize} + \item Your current user's home directory is processed first + \item Your directory from which spatch is called is processed next + \item The directory provided with the --dir option is processed last, if used +\end{itemize} + +Newlines, even with \, are not tolerated in attribute values. An example +follows: + +\begin{quote} +\begin{verbatim} +[spatch] + options = --jobs 4 + options = --show-trying +\end{verbatim} +\end{quote} + %%% Local Variables: %%% mode: LaTeX %%% TeX-master: "main_grammar"docs/spatch.1.in +31
-2@@ -55,7 +55,8 @@ and at the \fBscripts/coccinelle\fP directory of the Linux Kernel source code. .SH OPTIONS -Here is a summary of the most commonly used options: +Here is a summary of the most commonly used options (also +see the "Configuration Mechanism" section below): .TP .B \-\-sp\-file \fI<file>\fP@@ -566,6 +567,31 @@ It's for the other (internal) uses of the spatch program. \fB\-\-compare\-c\fR <file1> <file2> +.SH CONFIGURATION MECHANISM + +Coccinelle uses a simple text format to store customizations into +\.cocciconfig files, that can be per project, user, or target. The order +in which the configuration file is processed is as follows, the +later lines always extend and may override earlier ones: + + o Your current user's home directory is processed first + o Your directory from which spatch is called is processed next + o The directory provided with the --dir option is processed last, if used + +Such a configuration file may look like this: + +.if n \{\ +.RS 4 +.\} +.nf +[spatch] + options = --jobs 4 + options = --show-trying +.fi +.if n \{\ +.RE +.\} + .SH FILES .I @SHAREDIR@/standard.iso .RS@@ -574,6 +600,10 @@ This file contains the default set of isomorphisms. .I @SHAREDIR@/standard.h .RS This file contains the default set of macro hints. +.RE +.I .cocciconfig +.RS +This file contains the custom set of spatch options. .SH ENVIRONMENT .IP COCCINELLE_HOME --2.8.2
_______________________________________________
Cocci mailing list
Cocci at systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci