Skip to contentSkip to conversation

scripts/coccinelle/misc/memcpy-assign.cocci: Replace memcpy with struct assignment

PATCH In review Peter Senna Tschudin 4 participants 10 messages Jan 23, 2013 – Feb 22, 2013

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 scripts/coccinelle/misc/memcpy-assign.cocci: Replace memcpy with struct assignment

There are error-prone memcpy() that can be replaced by struct assignment that are type-safe and much easier to read. This semantic patch looks for memcpy() that can be replaced by struct assignment.

Inspired by patches sent by Ezequiel Garcia <[email protected]>

---
Changes from V1:
Updated commit message
Changed Confidence comment to High on the semantic patch

 scripts/coccinelle/misc/memcpy-assign.cocci | 103 ++++++++++++++++++++++++++++
 1 file changed, 103 insertions(+)
 create mode 100644 scripts/coccinelle/misc/memcpy-assign.cocci
scripts/coccinelle/misc/memcpy-assign.cocci new file +103 -0
new file mode 100644
index 0000000..afd058b
@@ -0,0 +1,103 @@
+//
+// Replace memcpy with struct assignment.
+//
+// Confidence: High
+// Copyright: (C) 2012 Peter Senna Tschudin, INRIA/LIP6.  GPLv2.
+// URL: http://coccinelle.lip6.fr/
+// Comments:
+// Options: --no-includes --include-headers
+
+virtual patch
+virtual report
+virtual context
+virtual org
+
+ at r1 depends on !patch@
+identifier struct_name;
+struct struct_name to;
+struct struct_name from;
+struct struct_name *top;
+struct struct_name *fromp;
+position p;
+@@
+memcpy at p(\(&(to)\|top\), \(&(from)\|fromp\), \(sizeof(to)\|sizeof(from)\|sizeof(struct struct_name)\|sizeof(*top)\|sizeof(*fromp)\))
+
+ at script:python depends on report@
+p << r1.p;
+@@
+coccilib.report.print_report(p[0],"Replace memcpy with struct assignment")
+
+ at depends on context@
+position r1.p;
+@@
+*memcpy at p(...);
+
+ at script:python depends on org@
+p << r1.p;
+@@
+cocci.print_main("Replace memcpy with struct assignment",p)
+
+ at depends on patch@
+identifier struct_name;
+struct struct_name to;
+struct struct_name from;
+@@
+(
+-memcpy(&(to), &(from), sizeof(to));
++to = from;
+|
+-memcpy(&(to), &(from), sizeof(from));
++to = from;
+|
+-memcpy(&(to), &(from), sizeof(struct struct_name));
++to = from;
+)
+
+ at depends on patch@
+identifier struct_name;
+struct struct_name to;
+struct struct_name *from;
+@@
+(
+-memcpy(&(to), from, sizeof(to));
++to = *from;
+|
+-memcpy(&(to), from, sizeof(*from));
++to = *from;
+|
+-memcpy(&(to), from, sizeof(struct struct_name));
++to = *from;
+)
+
+ at depends on patch@
+identifier struct_name;
+struct struct_name *to;
+struct struct_name from;
+@@
+(
+-memcpy(to, &(from), sizeof(*to));
++ *to = from;
+|
+-memcpy(to, &(from), sizeof(from));
++ *to = from;
+|
+-memcpy(to, &(from), sizeof(struct struct_name));
++ *to = from;
+)
+
+@depends on patch@
+identifier struct_name;
+struct struct_name *to;
+struct struct_name *from;
+@@
+(
+-memcpy(to, from, sizeof(*to));
++ *to = *from;
+|
+-memcpy(to, from, sizeof(*from));
++ *to = *from;
+|
+-memcpy(to, from, sizeof(struct struct_name));
++ *to = *from;
+)
+
Signature · 1 line
1.7.11.7
#1 lore raw

[Cocci] Prepending '*' to an expression (Was: [PATCH V2] scripts/coccinelle/misc/memcpy-assign.cocci: Replace memcpy with struct assignment)

Hello,

Quoted 3 lines · Peter Senna Tschudin

There are error-prone memcpy() that can be replaced by struct assignment that are type-safe and much easier to read. This semantic patch looks for memcpy() that can be replaced by struct assignment.

thanks for the script by Peter this script as it proved useful for Wine too. Though in one case it generated the wrong patch.

@@
struct struct_name to;
struct struct_name from;
@@

'to' and 'from' above are expressions and not identifiers; that's a feature. The bug happened in the replacement
- memcpy(foos + count, &foo, sizeof(struct foo_t));
+ *foos + count = foo;

Shouldn't cocinelle automatically add the parenthesis when prepending a '*' or '&' to an expression which isn't an identifier?

The script itself is "safe" as the compiler will error out on that patch. And the people that just take generated patches and submit them without even compiling them deserve the public tar and feathering ;)

bye
michael

Quoted 122 lines

Inspired by patches sent by Ezequiel Garcia <[email protected]>

---
Changes from V1:
Updated commit message
Changed Confidence comment to High on the semantic patch

 scripts/coccinelle/misc/memcpy-assign.cocci | 103 ++++++++++++++++++++++++++++
 1 file changed, 103 insertions(+)
 create mode 100644 scripts/coccinelle/misc/memcpy-assign.cocci
scripts/coccinelle/misc/memcpy-assign.cocci new file +103 -0
new file mode 100644
index 0000000..afd058b
@@ -0,0 +1,103 @@
+//
+// Replace memcpy with struct assignment.
+//
+// Confidence: High
+// Copyright: (C) 2012 Peter Senna Tschudin, INRIA/LIP6.  GPLv2.
+// URL: http://coccinelle.lip6.fr/
+// Comments:
+// Options: --no-includes --include-headers
+
+virtual patch
+virtual report
+virtual context
+virtual org
+
+ at r1 depends on !patch@
+identifier struct_name;
+struct struct_name to;
+struct struct_name from;
+struct struct_name *top;
+struct struct_name *fromp;
+position p;
+@@
+memcpy at p(\(&(to)\|top\), \(&(from)\|fromp\), \(sizeof(to)\|sizeof(from)\|sizeof(struct struct_name)\|sizeof(*top)\|sizeof(*fromp)\))
+
+ at script:python depends on report@
+p << r1.p;
+@@
+coccilib.report.print_report(p[0],"Replace memcpy with struct assignment")
+
+ at depends on context@
+position r1.p;
+@@
+*memcpy at p(...);
+
+ at script:python depends on org@
+p << r1.p;
+@@
+cocci.print_main("Replace memcpy with struct assignment",p)
+
+ at depends on patch@
+identifier struct_name;
+struct struct_name to;
+struct struct_name from;
+@@
+(
+-memcpy(&(to), &(from), sizeof(to));
++to = from;
+|
+-memcpy(&(to), &(from), sizeof(from));
++to = from;
+|
+-memcpy(&(to), &(from), sizeof(struct struct_name));
++to = from;
+)
+
+ at depends on patch@
+identifier struct_name;
+struct struct_name to;
+struct struct_name *from;
+@@
+(
+-memcpy(&(to), from, sizeof(to));
++to = *from;
+|
+-memcpy(&(to), from, sizeof(*from));
++to = *from;
+|
+-memcpy(&(to), from, sizeof(struct struct_name));
++to = *from;
+)
+
+ at depends on patch@
+identifier struct_name;
+struct struct_name *to;
+struct struct_name from;
+@@
+(
+-memcpy(to, &(from), sizeof(*to));
++ *to = from;
+|
+-memcpy(to, &(from), sizeof(from));
++ *to = from;
+|
+-memcpy(to, &(from), sizeof(struct struct_name));
++ *to = from;
+)
+
+ at depends on patch@
+identifier struct_name;
+struct struct_name *to;
+struct struct_name *from;
+@@
+(
+-memcpy(to, from, sizeof(*to));
++ *to = *from;
+|
+-memcpy(to, from, sizeof(*from));
++ *to = *from;
+|
+-memcpy(to, from, sizeof(struct struct_name));
++ *to = *from;
+)
+
#2 lore raw

[Cocci] Prepending '*' to an expression (Was: [PATCH V2] scripts/coccinelle/misc/memcpy-assign.cocci: Replace memcpy with struct assignment)

Quoted 14 lines · Michael Stefaniuc

Hello,

Quoted 3 lines · Peter Senna Tschudin

There are error-prone memcpy() that can be replaced by struct assignment that are type-safe and much easier to read. This semantic patch looks for memcpy() that can be replaced by struct assignment.

thanks for the script by Peter this script as it proved useful for Wine too. Though in one case it generated the wrong patch.

@@
struct struct_name to;
struct struct_name from;
@@

Quoted 7 lines

'to' and 'from' above are expressions and not identifiers; that's a feature. The bug happened in the replacement
- memcpy(foos + count, &foo, sizeof(struct foo_t));
+ *foos + count = foo;

Shouldn't cocinelle automatically add the parenthesis when prepending a '*' or '&' to an expression which isn't an identifier?

Perhaps it would be possible, but there is nothing in place to do that.

Part of a solution would be to declare the metavariable in the unparenthesized case as an idexpression, ie idexpression struct struct_name *to. Then one could have a rule for an arbitrary expression afterwards, ie with the metavariable declaration above, that would explicitly add parentheses.

With that though you would end up with parentheses around eg x->y, because that is not considered to be an identifier. So one would also need a separate rule in the middle for that case. To ensure that the types are OK, you should be able to do:

@@
struct struct_name *to;
expression e;
identifier f;
...
@@

e->f at to

This should match the same term against both the pattern e->f and the pattern to.

Thanks for testing.

julia

#3 lore raw

[Cocci] Prepending '*' to an expression (Was: [PATCH V2] scripts/coccinelle/misc/memcpy-assign.cocci: Replace memcpy with struct assignment)

In reply to Michael Stefaniuc

Hello Michael,

Thank you for testing.

I made changes on the semantic patch. I believe it fixed the problem. Can you test it?

[]'s

Peter

Quoted 152 lines · Michael Stefaniuc

Hello,

Quoted 3 lines · Peter Senna Tschudin

There are error-prone memcpy() that can be replaced by struct assignment that are type-safe and much easier to read. This semantic patch looks for memcpy() that can be replaced by struct assignment.

thanks for the script by Peter this script as it proved useful for Wine too. Though in one case it generated the wrong patch.

@@
struct struct_name to;
struct struct_name from;
@@

'to' and 'from' above are expressions and not identifiers; that's a feature. The bug happened in the replacement
- memcpy(foos + count, &foo, sizeof(struct foo_t));
+ *foos + count = foo;

Shouldn't cocinelle automatically add the parenthesis when prepending a '*' or '&' to an expression which isn't an identifier?

The script itself is "safe" as the compiler will error out on that patch. And the people that just take generated patches and submit them without even compiling them deserve the public tar and feathering ;)

bye
michael

Quoted 122 lines

Inspired by patches sent by Ezequiel Garcia <[email protected]>

---
Changes from V1:
Updated commit message
Changed Confidence comment to High on the semantic patch

 scripts/coccinelle/misc/memcpy-assign.cocci | 103 ++++++++++++++++++++++++++++
 1 file changed, 103 insertions(+)
 create mode 100644 scripts/coccinelle/misc/memcpy-assign.cocci
scripts/coccinelle/misc/memcpy-assign.cocci new file +103 -0
new file mode 100644
index 0000000..afd058b
@@ -0,0 +1,103 @@
+//
+// Replace memcpy with struct assignment.
+//
+// Confidence: High
+// Copyright: (C) 2012 Peter Senna Tschudin, INRIA/LIP6.  GPLv2.
+// URL: http://coccinelle.lip6.fr/
+// Comments:
+// Options: --no-includes --include-headers
+
+virtual patch
+virtual report
+virtual context
+virtual org
+
+ at r1 depends on !patch@
+identifier struct_name;
+struct struct_name to;
+struct struct_name from;
+struct struct_name *top;
+struct struct_name *fromp;
+position p;
+@@
+memcpy at p(\(&(to)\|top\), \(&(from)\|fromp\), \(sizeof(to)\|sizeof(from)\|sizeof(struct struct_name)\|sizeof(*top)\|sizeof(*fromp)\))
+
+ at script:python depends on report@
+p << r1.p;
+@@
+coccilib.report.print_report(p[0],"Replace memcpy with struct assignment")
+
+ at depends on context@
+position r1.p;
+@@
+*memcpy at p(...);
+
+ at script:python depends on org@
+p << r1.p;
+@@
+cocci.print_main("Replace memcpy with struct assignment",p)
+
+ at depends on patch@
+identifier struct_name;
+struct struct_name to;
+struct struct_name from;
+@@
+(
+-memcpy(&(to), &(from), sizeof(to));
++to = from;
+|
+-memcpy(&(to), &(from), sizeof(from));
++to = from;
+|
+-memcpy(&(to), &(from), sizeof(struct struct_name));
++to = from;
+)
+
+ at depends on patch@
+identifier struct_name;
+struct struct_name to;
+struct struct_name *from;
+@@
+(
+-memcpy(&(to), from, sizeof(to));
++to = *from;
+|
+-memcpy(&(to), from, sizeof(*from));
++to = *from;
+|
+-memcpy(&(to), from, sizeof(struct struct_name));
++to = *from;
+)
+
+ at depends on patch@
+identifier struct_name;
+struct struct_name *to;
+struct struct_name from;
+@@
+(
+-memcpy(to, &(from), sizeof(*to));
++ *to = from;
+|
+-memcpy(to, &(from), sizeof(from));
++ *to = from;
+|
+-memcpy(to, &(from), sizeof(struct struct_name));
++ *to = from;
+)
+
+ at depends on patch@
+identifier struct_name;
+struct struct_name *to;
+struct struct_name *from;
+@@
+(
+-memcpy(to, from, sizeof(*to));
++ *to = *from;
+|
+-memcpy(to, from, sizeof(*from));
++ *to = *from;
+|
+-memcpy(to, from, sizeof(struct struct_name));
++ *to = *from;
+)
+
Signature · 8 lines
Peter
-------------- next part --------------
A non-text attachment was scrubbed...
Name: memcpy-assign.cocci
Type: application/octet-stream
Size: 2740 bytes
Desc: not available
URL: <https://systeme.lip6.fr/pipermail/cocci/attachments/20130125/82b7bb92/attachment.obj>
#4 lore raw

[Cocci] Prepending '*' to an expression (Was: [PATCH V2] scripts/coccinelle/misc/memcpy-assign.cocci: Replace memcpy with struct assignment)

Hello Peter,

Quoted 4 lines · Peter Senna Tschudin

Thank you for testing.

I made changes on the semantic patch. I believe it fixed the problem. Can you test it?

while it does fix the issue it makes the cocci script less readable and works only in that particular case. Not sure it is worth the effort as the bad generated patch can be trivially detected and easily fixed manually. My email was more of a bug report / feature request for coccinelle than an improvement request to the script.

Though I've noticed an other issue with the script: Any reason you use "Options: --no-includes"? While it does speed up things it misses quite a few cases as a lot of types are defined in the headers.

bye
michael

Quoted 27 lines · Michael Stefaniuc
Quoted 26 lines

Hello,

Quoted 3 lines · Peter Senna Tschudin

There are error-prone memcpy() that can be replaced by struct assignment that are type-safe and much easier to read. This semantic patch looks for memcpy() that can be replaced by struct assignment.

thanks for the script by Peter this script as it proved useful for Wine too. Though in one case it generated the wrong patch.

@@
struct struct_name to;
struct struct_name from;
@@

'to' and 'from' above are expressions and not identifiers; that's a feature. The bug happened in the replacement
- memcpy(foos + count, &foo, sizeof(struct foo_t));
+ *foos + count = foo;

Shouldn't cocinelle automatically add the parenthesis when prepending a '*' or '&' to an expression which isn't an identifier?

The script itself is "safe" as the compiler will error out on that patch. And the people that just take generated patches and submit them without even compiling them deserve the public tar and feathering ;)

#5 lore raw

[Cocci] Prepending '*' to an expression (Was: [PATCH V2] scripts/coccinelle/misc/memcpy-assign.cocci: Replace memcpy with struct assignment)

Quoted 13 lines · Michael Stefaniuc

Hello Peter,

Quoted 4 lines · Peter Senna Tschudin

Thank you for testing.

I made changes on the semantic patch. I believe it fixed the problem. Can you test it?

while it does fix the issue it makes the cocci script less readable and works only in that particular case. Not sure it is worth the effort as the bad generated patch can be trivially detected and easily fixed manually. My email was more of a bug report / feature request for coccinelle than an improvement request to the script.

Quoted 3 lines

Though I've noticed an other issue with the script: Any reason you use "Options: --no-includes"? While it does speed up things it misses quite a few cases as a lot of types are defined in the headers.

I think it should be --all-includes or even --recursive-includes

julia.

Quoted 36 lines

bye
michael

Quoted 20 lines · Michael Stefaniuc
Quoted 19 lines

Hello,

Quoted 3 lines · Peter Senna Tschudin

There are error-prone memcpy() that can be replaced by struct assignment that are type-safe and much easier to read. This semantic patch looks for memcpy() that can be replaced by struct assignment.

thanks for the script by Peter this script as it proved useful for Wine too. Though in one case it generated the wrong patch.

@@
struct struct_name to;
struct struct_name from;
@@

'to' and 'from' above are expressions and not identifiers; that's a feature. The bug happened in the replacement
- memcpy(foos + count, &foo, sizeof(struct foo_t));
+ *foos + count = foo;

Quoted 7 lines
Quoted 7 lines

Shouldn't cocinelle automatically add the parenthesis when prepending a '*' or '&' to an expression which isn't an identifier?

The script itself is "safe" as the compiler will error out on that patch. And the people that just take generated patches and submit them without even compiling them deserve the public tar and feathering ;)

_______________________________________________
Cocci mailing list
Cocci at systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

#6 lore raw

[Cocci] Prepending '*' to an expression (Was: [PATCH V2] scripts/coccinelle/misc/memcpy-assign.cocci: Replace memcpy with struct assignment)

Hello Julia,

Quoted 15 lines · Julia Lawall
Quoted 13 lines · Michael Stefaniuc

Hello Peter,

Quoted 4 lines · Peter Senna Tschudin

Thank you for testing.

I made changes on the semantic patch. I believe it fixed the problem. Can you test it?

while it does fix the issue it makes the cocci script less readable and works only in that particular case. Not sure it is worth the effort as the bad generated patch can be trivially detected and easily fixed manually. My email was more of a bug report / feature request for coccinelle than an improvement request to the script.

Quoted 5 lines
Quoted 3 lines

Though I've noticed an other issue with the script: Any reason you use "Options: --no-includes"? While it does speed up things it misses quite a few cases as a lot of types are defined in the headers.

I think it should be --all-includes or even --recursive-includes

no clue what the policy in the Kernel is but --recursive-includes is quite heavy handed as a default. I have one --recursive-includes still running for Wine and it accumulated until now over 1126 CPU minutes (Intel i5 mobile CPU with 2.4 GHz). It found 5 more cases while --no-includes found 11 and --local-includes additional 32. The --local-includes finished in less than 30 minutes (didn't time it exactly).

bye
michael

#7 lore raw

[Cocci] Prepending '*' to an expression (Was: [PATCH V2] scripts/coccinelle/misc/memcpy-assign.cocci: Replace memcpy with struct assignment)

Quoted 22 lines · Michael Stefaniuc

Hello Julia,

Quoted 19 lines · Julia Lawall
Quoted 16 lines · Michael Stefaniuc

Hello Peter,

Quoted 4 lines · Peter Senna Tschudin

Thank you for testing.

I made changes on the semantic patch. I believe it fixed the problem. Can you test it?

while it does fix the issue it makes the cocci script less readable and works only in that particular case. Not sure it is worth the effort as the bad generated patch can be trivially detected and easily fixed manually. My email was more of a bug report / feature request for coccinelle than an improvement request to the script.

Though I've noticed an other issue with the script: Any reason you use "Options: --no-includes"? While it does speed up things it misses quite a few cases as a lot of types are defined in the headers.

Quoted 7 lines
Quoted 1 line

I think it should be --all-includes or even --recursive-includes

no clue what the policy in the Kernel is but --recursive-includes is quite heavy handed as a default. I have one --recursive-includes still running for Wine and it accumulated until now over 1126 CPU minutes (Intel i5 mobile CPU with 2.4 GHz). It found 5 more cases while --no-includes found 11 and --local-includes additional 32. The --local-includes finished in less than 30 minutes (didn't time it exactly).

OK. It depends on how much type information is needed. A compromise could be to make a rule without type information that makes clear that one has less confidence in the result. Then the user could just go look up the type information if needed.

-all_includes could be good then. It only includes what is specifically referenced in the current file.

julia

#8 lore raw

[Cocci] Prepending '*' to an expression (Was: [PATCH V2] scripts/coccinelle/misc/memcpy-assign.cocci: Replace memcpy with struct assignment)

In reply to Michael Stefaniuc
Quoted 13 lines · Michael Stefaniuc
Quoted 12 lines · Julia Lawall
Quoted 11 lines · Michael Stefaniuc
Quoted 4 lines · Peter Senna Tschudin

Thank you for testing.

I made changes on the semantic patch. I believe it fixed the problem. Can you test it?

while it does fix the issue it makes the cocci script less readable and works only in that particular case. Not sure it is worth the effort as the bad generated patch can be trivially detected and easily fixed manually. My email was more of a bug report / feature request for coccinelle than an improvement request to the script.

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

Though I've noticed an other issue with the script: Any reason you use "Options: --no-includes"? While it does speed up things it misses quite a few cases as a lot of types are defined in the headers.

I think it should be --all-includes or even --recursive-includes

no clue what the policy in the Kernel is but --recursive-includes is quite heavy handed as a default. I have one --recursive-includes still running for Wine and it accumulated until now over 1126 CPU minutes

The --recursive-includes run finally finished after accumulating 4240 minutes of CPU time. There was no visible slowdown nor memory leak during the run which is pretty cool.

Quoted 3 lines

(Intel i5 mobile CPU with 2.4 GHz). It found 5 more cases while --no-includes found 11 and --local-includes additional 32. The --local-includes finished in less than 30 minutes (didn't time it exactly).

bye
michael

#9 lore raw
In reply to Peter Senna Tschudin
Quoted 11 lines · Peter Senna Tschudin

There are error-prone memcpy() that can be replaced by struct assignment that are type-safe and much easier to read. This semantic patch looks for memcpy() that can be replaced by struct assignment.

Inspired by patches sent by Ezequiel Garcia <[email protected]>

---
Changes from V1:
Updated commit message
Changed Confidence comment to High on the semantic patch

Applied to kbuild.git#misc, thanks.

Michal