Skip to contentSkip to conversation

Parsing errors in several headers

Cocci Jean-Sébastien Pédron 3 participants 20 messages Oct 28, 2014 – Oct 29, 2014

lore
Share & tools
lore
#0 lore raw

Hi!

I'm very new to Coccinelle and I don't understand why spatch(1) returns parsing errors on some headers and what I can do about that.

== First example ==

Header to parse:
https://github.com/freebsd/freebsd/blob/master/sys/ofed/include/linux/bitops.h

Full spatch command line and output:
http://pastebin.com/DbNhBAF7

spatch(1) fails to parse the #define at the end of the header. This #define starts a for() loop and uses functions declared ealier in the same file.

== Second example ==

Header to parse:
https://github.com/freebsd/freebsd/blob/master/sys/ofed/include/linux/spinlock.h

Full spatch command line and output:
http://pastebin.com/C0h5ZN7r

Here, it doesn't like the last #define again. This #define calls other macros defined in other headers.

FYI, I'm using Coccinelle 1.0.0rc19 on FreeBSD.

Can somebody help me understand what's wrong here?

Thanks!

Signature · 9 lines
Jean-S?bastien P?dron

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 949 bytes
Desc: OpenPGP digital signature
URL: <https://systeme.lip6.fr/pipermail/cocci/attachments/20141028/a7c088ee/attachment.asc>
#1 lore raw
Quoted 10 lines · Jean-S?bastien P?dron

Hi!

I'm very new to Coccinelle and I don't understand why spatch(1) returns parsing errors on some headers and what I can do about that.

== First example ==

Header to parse:
https://github.com/freebsd/freebsd/blob/master/sys/ofed/include/linux/bitops.h

Quoted 6 lines

Full spatch command line and output:
http://pastebin.com/DbNhBAF7

spatch(1) fails to parse the #define at the end of the header. This #define starts a for() loop and uses functions declared ealier in the same file.

Do you need to match or transform some code that is related to this? If not, it doesn't matter.

julia

Quoted 21 lines

== Second example ==

Header to parse:
https://github.com/freebsd/freebsd/blob/master/sys/ofed/include/linux/spinlock.h

Full spatch command line and output:
http://pastebin.com/C0h5ZN7r

Here, it doesn't like the last #define again. This #define calls other macros defined in other headers.

FYI, I'm using Coccinelle 1.0.0rc19 on FreeBSD.

Can somebody help me understand what's wrong here?

Thanks!

--
Jean-S?bastien P?dron

#2 lore raw
Quoted 2 lines · Julia Lawall

Do you need to match or transform some code that is related to this? If not, it doesn't matter.

Ok, then I must have a problem with my semantic patch, because it doesn't change anything :) I thought it was because of the parsing error.

Thank you for the fast reply!

Signature · 9 lines
Jean-S?bastien P?dron

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 949 bytes
Desc: OpenPGP digital signature
URL: <https://systeme.lip6.fr/pipermail/cocci/attachments/20141028/f0d061c8/attachment.asc>
#3 lore raw
Quoted 2 lines

Ok, then I must have a problem with my semantic patch, because it doesn't change anything :) I thought it was because of the parsing error.

Would you like to show your SmPL approach here so that we could help you more eventually?

Regards,
Markus

#4 lore raw
Quoted 5 lines · SF Markus Elfring
Quoted 2 lines

Ok, then I must have a problem with my semantic patch, because it doesn't change anything :) I thought it was because of the parsing error.

Would you like to show your SmPL approach here so that we could help you more eventually?

Sure! I wanted to add a #define just above a function prototype or the function itself.

Example #1:
int foo();

should become:
#define foo BAR(foo)
int foo();

Example #2:
int foo() { /* ... */ }

should become:
#define foo BAR(foo)
int foo() { /* ... */ }

Here's the script I used, which worked with example #1:

---8<---
@rule1@
identifier func;
@@

-#define func BAR(func)

@rule2@
identifier func;
type ret;
@@

+#define func BAR(func)
ret func(...);
---8<---

I can handle example #2 with this one:

---8<---
@rule1@
identifier func;
@@

-#define func BAR(func)

@rule2@
identifier func;
type ret;
@@

+#define func BAR(func)
ret func(...) { ... }
---8<---

However, I fail to assemble those two patches in a single concise one.

Thank you for offering your help :)

Signature · 9 lines
Jean-S?bastien P?dron

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 949 bytes
Desc: OpenPGP digital signature
URL: <https://systeme.lip6.fr/pipermail/cocci/attachments/20141028/09605fdc/attachment.asc>
#5 lore raw
Quoted 56 lines · Jean-S?bastien P?dron
Quoted 5 lines · SF Markus Elfring
Quoted 2 lines

Ok, then I must have a problem with my semantic patch, because it doesn't change anything :) I thought it was because of the parsing error.

Would you like to show your SmPL approach here so that we could help you more eventually?

Sure! I wanted to add a #define just above a function prototype or the function itself.

Example #1:
int foo();

should become:
#define foo BAR(foo)
int foo();

Example #2:
int foo() { /* ... */ }

should become:
#define foo BAR(foo)
int foo() { /* ... */ }

Here's the script I used, which worked with example #1:

---8<---
@rule1@
identifier func;
@@

-#define func BAR(func)

@rule2@
identifier func;
type ret;
@@

+#define func BAR(func)
ret func(...);
---8<---

I can handle example #2 with this one:

---8<---
@rule1@
identifier func;
@@

-#define func BAR(func)

@rule2@
identifier func;
type ret;
@@

Quoted 5 lines

+#define func BAR(func)
ret func(...) { ... }
---8<---

However, I fail to assemble those two patches in a single concise one.

It is not possible. You need two rules.

julia

Quoted 7 lines

Thank you for offering your help :)

Signature · 1 line
Jean-S?bastien P?dron
#6 lore raw
In reply to Jean-Sébastien Pédron
Quoted 2 lines

I wanted to add a #define just above a function prototype or the function itself.

[...]

Quoted 5 lines

@rule1@
identifier func;
@@

-#define func BAR(func)

I find this SmPL approach strange according to your description.
- Would you really like to delete the definition for a preprocessor symbol here?
- Should this semantic patch express the change to move the position of such a statement to other source code places?

Regards,
Markus

#7 lore raw
Quoted 7 lines · SF Markus Elfring
Quoted 2 lines

I wanted to add a #define just above a function prototype or the function itself.

[...]

Quoted 4 lines

@rule1@
identifier func;
@@

Quoted 6 lines
Quoted 1 line

-#define func BAR(func)

I find this SmPL approach strange according to your description.
- Would you really like to delete the definition for a preprocessor symbol here?
- Should this semantic patch express the change to move the position of such a statement to other source code places?

I wanted to be able to re-run spatch on an already patched content. The use case is to add new function prototypes manually and run spatch to handle the associated #define automatically.

Signature · 9 lines
Jean-S?bastien P?dron

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 949 bytes
Desc: OpenPGP digital signature
URL: <https://systeme.lip6.fr/pipermail/cocci/attachments/20141029/6bcfa88d/attachment.asc>
#8 lore raw

[Cocci] Addition of preprocessor statements with SmPL

Quoted 3 lines

I wanted to be able to re-run spatch on an already patched content. The use case is to add new function prototypes manually and run spatch to handle the associated #define automatically.

Thanks for a bit more explanation.

I imagine that you might need to specify more source code as adjustment context for a statement deletion to achieve a safer semantic patch.

Regards,
Markus

#16 lore raw

[Cocci] Addition of preprocessor statements with SmPL

In reply to Jean-Sébastien Pédron
Quoted 1 line

However, I fail to assemble those two patches in a single concise one.

How do you think about the following approach?

@Add_before_declaration@
identifier func;
type t;
@@
(
#define func BAR(func)
t func(...);
|
+#define func BAR(func)
t func(...);
)

@Add_before_implementation@
identifier func;
type t;
@@
(
#define func BAR(func)
t func(...)
{ ... }
|
+#define func BAR(func)
t func(...)
{ ... }
)

Do you need to express a detailed adjustment precondition for a safer semantic patch?

Regards,
Markus

#17 lore raw

[Cocci] Addition of preprocessor statements with SmPL

Quoted 16 lines · SF Markus Elfring
Quoted 1 line

However, I fail to assemble those two patches in a single concise one.

How do you think about the following approach?

@Add_before_declaration@
identifier func;
type t;
@@
(
#define func BAR(func)
t func(...);
|
+#define func BAR(func)
t func(...);
)

Quoted 13 lines

@Add_before_implementation@
identifier func;
type t;
@@
(
#define func BAR(func)
t func(...)
{ ... }
|
+#define func BAR(func)
t func(...)
{ ... }
)

This doesn't work. There is no kind of disjunction that includes a function definition in a branch.

julia

Quoted 12 lines

Do you need to express a detailed adjustment precondition for a safer semantic patch?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci at systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

#18 lore raw

[Cocci] Addition of preprocessor statements with SmPL

Quoted 14 lines
Quoted 5 lines

@Add_before_implementation@
identifier func;
type t;
@@
(

Quoted 8 lines

#define func BAR(func)
t func(...)
{ ... }
|
+#define func BAR(func)
t func(...)
{ ... }
)

Quoted 1 line

This doesn't work.

Interesting ...

Quoted 1 line

There is no kind of disjunction that includes a function definition in a branch.

Was I too optimistic that my suggestion with the specification of SmPL alternatives should also work?

Regards,
Markus

#19 lore raw

[Cocci] Addition of preprocessor statements with SmPL

Quoted 16 lines · SF Markus Elfring
Quoted 14 lines
Quoted 5 lines

@Add_before_implementation@
identifier func;
type t;
@@
(

Quoted 8 lines

#define func BAR(func)
t func(...)
{ ... }
|
+#define func BAR(func)
t func(...)
{ ... }
)

Quoted 1 line

This doesn't work.

Quoted 7 lines

Interesting ...

Quoted 1 line

There is no kind of disjunction that includes a function definition in a branch.

Was I too optimistic that my suggestion with the specification of SmPL alternatives should also work?

You can see with spatch --parse-cocci what works and what does not.

julia

#9 lore raw
In reply to Jean-Sébastien Pédron
Quoted 3 lines

I wanted to be able to re-run spatch on an already patched content. The use case is to add new function prototypes manually and run spatch to handle the associated #define automatically.

I think that you could do the following:

@r@
identifier func;
@@
#define func BAR(func)

@s@
identifier r.func;
position p;
@@

func at p(...) {...}

@@
identifier func;
position p != s.p;
@@

+#define func BAR(func)
func(...) { ... }

julia

#10 lore raw

[Cocci] Addition of preprocessor statements with SmPL

Quoted 18 lines · Julia Lawall
Quoted 3 lines

I wanted to be able to re-run spatch on an already patched content. The use case is to add new function prototypes manually and run spatch to handle the associated #define automatically.

I think that you could do the following:

@r@
identifier func;
@@
#define func BAR(func)

@s@
identifier r.func;
position p;
@@

func at p(...) {...}

Quoted 7 lines

@@
identifier func;
position p != s.p;
@@

+#define func BAR(func)
func(...) { ... }

I tried this semantic patch and it doesn't work: more #define are added with each run.

FWIW, the "BAR" identifier is unique in my use case: I'm sure the original headers won't have it. That's why I took a simple approach in removing all previous #define. However, I'm willing to learn the correct way to handle this for a future usage :)

I have an additional question: If there's a comment just above the function declaration/implementation, the #define is added above the comment. Is there a way to put it between the comment and the function?

Signature · 9 lines
Jean-S?bastien P?dron

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 949 bytes
Desc: OpenPGP digital signature
URL: <https://systeme.lip6.fr/pipermail/cocci/attachments/20141029/ccdb8366/attachment-0001.asc>
#11 lore raw

[Cocci] Addition of preprocessor statements with SmPL

Quoted 24 lines · Jean-S?bastien P?dron
Quoted 23 lines · Julia Lawall
Quoted 3 lines

I wanted to be able to re-run spatch on an already patched content. The use case is to add new function prototypes manually and run spatch to handle the associated #define automatically.

I think that you could do the following:

@r@
identifier func;
@@
#define func BAR(func)

@s@
identifier r.func;
position p;
@@

func at p(...) {...}

@@
identifier func;
position p != s.p;
@@

Quoted 5 lines
Quoted 2 lines

+#define func BAR(func)
func(...) { ... }

I tried this semantic patch and it doesn't work: more #define are added with each run.

OK, I will see what is wrong.

Quoted 8 lines

FWIW, the "BAR" identifier is unique in my use case: I'm sure the original headers won't have it. That's why I took a simple approach in removing all previous #define. However, I'm willing to learn the correct way to handle this for a future usage :)

I have an additional question: If there's a comment just above the function declaration/implementation, the #define is added above the comment. Is there a way to put it between the comment and the function?

No. It assumes that the comment is very attached to the function definition.

julia

#12 lore raw

[Cocci] Addition of preprocessor statements with SmPL

In reply to Jean-Sébastien Pédron
Quoted 19 lines · Jean-S?bastien P?dron
Quoted 18 lines · Julia Lawall
Quoted 3 lines

I wanted to be able to re-run spatch on an already patched content. The use case is to add new function prototypes manually and run spatch to handle the associated #define automatically.

I think that you could do the following:

@r@
identifier func;
@@
#define func BAR(func)

@s@
identifier r.func;
position p;
@@

func at p(...) {...}

Quoted 7 lines
Quoted 7 lines

@@
identifier func;
position p != s.p;
@@

+#define func BAR(func)
func(...) { ... }

Sorry, typo here. It should be func at p(...) { ... }

What is more unpleasant is that at least in my version of Coccinelle, it indents the function definition.

julia

#13 lore raw

[Cocci] Addition of preprocessor statements with SmPL

Quoted 26 lines · Julia Lawall
Quoted 24 lines · Jean-S?bastien P?dron
Quoted 23 lines · Julia Lawall
Quoted 3 lines

I wanted to be able to re-run spatch on an already patched content. The use case is to add new function prototypes manually and run spatch to handle the associated #define automatically.

I think that you could do the following:

@r@
identifier func;
@@
#define func BAR(func)

@s@
identifier r.func;
position p;
@@

func at p(...) {...}

@@
identifier func;
position p != s.p;
@@

Quoted 4 lines
Quoted 2 lines
Quoted 2 lines

+#define func BAR(func)
func(...) { ... }

Sorry, typo here. It should be func at p(...) { ... }

You're right, it's working now! Thank you :)

Quoted 2 lines

What is more unpleasant is that at least in my version of Coccinelle, it indents the function definition.

Here, with 1.0.0rc19, I don't have this problem.

Signature · 9 lines
Jean-S?bastien P?dron

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 949 bytes
Desc: OpenPGP digital signature
URL: <https://systeme.lip6.fr/pipermail/cocci/attachments/20141029/e4311a64/attachment.asc>
#15 lore raw
In reply to Julia Lawall
Quoted 7 lines

@@
identifier func;
position p != s.p;
@@

+#define func BAR(func)
func(...) { ... }

Will the shown position check really work when the variable p is not referenced in the patch specification block?

Regards,
Markus

#14 lore raw

[Cocci] Addition of preprocessor statements with SmPL

In reply to Jean-Sébastien Pédron
Quoted 3 lines · Jean-S?bastien P?dron
Quoted 1 line

Sorry, typo here. It should be func at p(...) { ... }

You're right, it's working now! Thank you :)

Here's the complete working semantic patch:

---8<---
@r@
identifier func;
@@

#define func BAR(func)

@s1@
identifier r.func;
position p;
@@

func at p(...) {...}

@@
identifier func;
position p != s1.p;
@@

+#define func BAR(func)
func at p(...) { ... }

@s2@
type ret;
identifier r.func;
position p;
@@

ret func at p(...);

@@
type ret;
identifier func;
position p != s2.p;
@@

+#define func BAR(func)
ret func@p(...);
---8<---

Thank you very much for your help!

Signature · 9 lines
Jean-S?bastien P?dron

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 949 bytes
Desc: OpenPGP digital signature
URL: <https://systeme.lip6.fr/pipermail/cocci/attachments/20141029/05a643e5/attachment.asc>