Skip to contentSkip to conversation

Matching outside of blocks

Cocci Thierry Reding 3 participants 7 messages Jan 9, 2013

lore
Share & tools
lore
#0 lore raw

Hi,

I'm trying to come up with a semantic patch to modify the meaning of the devm_request_and_ioremap() functions return value. The goal is to return an ERR_PTR() encoded errno in order to allow the proper error code to be propagated to the caller.

Since devm_request_and_ioremap() is called quite often, I thought this would be an interesting task to solve with coccinelle. I've come up with this semantic patch:

	virtual patch
	virtual report

	@depends on patch@
	expression e;
	@@

	 e = devm_request_and_ioremap(...);
	 ...
	(
	-if (\(e == NULL\|!e\)) {
	+if (IS_ERR(e)) {
	 	...
	-	return ...;
	+	return PTR_ERR(e);
	 }
	|
	-if (\(e == NULL\|!e\))
	-	return ...;
	+if (IS_ERR(e))
	+	return PTR_ERR(e);
	)

	@r depends on report@
	expression e;
	position p1;
	@@

	*e = devm_request_and_ioremap at p1(...);
	 ...
	(
	 if (\(e == NULL\|!e\)) {
	 	...
	 	return ...;
	 }
	|
	 if (\(e == NULL\|!e\))
	 	return ...;
	)

	@script:python depends on r@
	p1 << r.p1;
	@@

	msg = "ERROR: old devm_request_and_ioremap() API used on line %s" % (p1[0].line)
	coccilib.report.print_report(p1[0], msg)

This seems to work fine for most cases, but I've come across a tricky one in drivers/gpio/gpio-mxs.c. The code looks roughly like this:

	if (np) {
		...
		base = of_iomap(parent, 0);
		...
	} else {
		iores = platform_get_resource(...);
		base = devm_request_and_ioremap(...);
	}
	if (!base)
		return -EADDRNOTAVAIL;

The odd thing is that in report mode this case is actually detected, but the patch mode doesn't generate a hunk for this occurrence.

I wonder what I need to change to make this case work. From my very limited understanding of coccinelle it would seem that the issue is that the devm_request_and_ioremap() call and the return value check are in different blocks, but I have no idea how to take that into account. Then again, maybe the problem is something completely different.

Does anybody have any hints?

Thanks,
Thierry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <https://systeme.lip6.fr/pipermail/cocci/attachments/20130109/645627ea/attachment.asc>

#1 lore raw

Hi,

What version of Coccinelle are you using? I suggest you try the latest one available at:
http://coccinelle.lip6.fr/download.php

Quoted 6 lines · Thierry Reding

Hi,

I'm trying to come up with a semantic patch to modify the meaning of the devm_request_and_ioremap() functions return value. The goal is to return an ERR_PTR() encoded errno in order to allow the proper error code to be propagated to the caller.

Will you change the function itself so it will return ERR_PRT() encoded errno?

Quoted 43 lines

Since devm_request_and_ioremap() is called quite often, I thought this would be an interesting task to solve with coccinelle. I've come up with this semantic patch:

        virtual patch
        virtual report

        @depends on patch@
        expression e;
        @@

         e = devm_request_and_ioremap(...);
         ...
        (
        -if (\(e == NULL\|!e\)) {
        +if (IS_ERR(e)) {
                ...
        -       return ...;
        +       return PTR_ERR(e);
         }
        |
        -if (\(e == NULL\|!e\))
        -       return ...;
        +if (IS_ERR(e))
        +       return PTR_ERR(e);
        )

        @r depends on report@
        expression e;
        position p1;
        @@

        *e = devm_request_and_ioremap at p1(...);
         ...
        (
         if (\(e == NULL\|!e\)) {
                ...
                return ...;
         }
        |
         if (\(e == NULL\|!e\))
                return ...;
Quoted 8 lines
        )

        @script:python depends on r@
        p1 << r.p1;
        @@

        msg = "ERROR: old devm_request_and_ioremap() API used on line %s" % (p1[0].line)
        coccilib.report.print_report(p1[0], msg)

I made some changes for patch mode:
@@
expression e;
@@

e = devm_request_and_ioremap(...);
...
if
-(e == NULL)
+(IS_ERR(e))
{
<...
- return ...;
+ return PTR_ERR(e);
...>
}

You can save this text in a .cocci file then use it with:

$ cd /linux-source
$ spatch -dir . /tmp/file.cocci

Note that some isomorphisms like e = NULL and !e are automatically taken care for you. Check Coccinelle iso file for more details.

Quoted 15 lines

This seems to work fine for most cases, but I've come across a tricky one in drivers/gpio/gpio-mxs.c. The code looks roughly like this:

        if (np) {
                ...
                base = of_iomap(parent, 0);
                ...
        } else {
                iores = platform_get_resource(...);
                base = devm_request_and_ioremap(...);
        }
        if (!base)
                return -EADDRNOTAVAIL;
Quoted 2 lines

The odd thing is that in report mode this case is actually detected, but the patch mode doesn't generate a hunk for this occurrence.

The position variable is storing the position of devm_request_and_ioremap, this is why the report mode detect it. The problem is that base could receive its content from devm_request_and_ioremap(...); or from of_iomap(parent, 0);. For this case I would manually add different return for each case. Something like:

if (np) {
        ...
        base = of_iomap(parent, 0);
        ...
        if (!base)
                return -EADDRNOTAVAIL;
} else {
        iores = platform_get_resource(...);
        base = devm_request_and_ioremap(...);
        if (!base)
                return PTR_ERR(base);
}
Quoted 8 lines

I wonder what I need to change to make this case work. From my very limited understanding of coccinelle it would seem that the issue is that the devm_request_and_ioremap() call and the return value check are in different blocks, but I have no idea how to take that into account. Then again, maybe the problem is something completely different.

Does anybody have any hints?

1 - I would work on a Coccinelle script that works for the common cases and do the exceptions manually.
2 - Even for changes made by Coccinelle I would double check if the proposed transformation is what I was expecting.
3 - There are a lot of examples at: coccinellery.org

Quoted 9 lines

Thanks,
Thierry

_______________________________________________
Cocci mailing list
Cocci at systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

--
Peter

#6 lore raw
In reply to Thierry Reding
Quoted 9 lines · Thierry Reding

Hi,

I'm trying to come up with a semantic patch to modify the meaning of the devm_request_and_ioremap() functions return value. The goal is to return an ERR_PTR() encoded errno in order to allow the proper error code to be propagated to the caller.

Since devm_request_and_ioremap() is called quite often, I thought this would be an interesting task to solve with coccinelle.

Seems like a good idea.

Quoted 5 lines

I've come up with
this semantic patch:

virtual patch
virtual report

This is done in the Linux semantic patches, to allow people to use them in different ways. If you just want to run the semantic patch in one way, you don't have to bother with this.

Quoted 4 lines
	@depends on patch@
	expression e;
	@@
Quoted 8 lines
	 e = devm_request_and_ioremap(...);
	 ...
	(
	-if (\(e == NULL\|!e\)) {
	+if (IS_ERR(e)) {
	 	...
	-	return ...;
	+	return PTR_ERR(e);

Here you are assuming that the function returns an integer. Maybe that is always the case, given the overall constraints on the devm functions.

(Actually, I was just thinking yesterday that it was too bad that devm_request_and_ioremap returns such an undescriptive value. Because it combines two operations, and the return value doesn't indicate which one failed.)

Quoted 6 lines
	 }
	|
	-if (\(e == NULL\|!e\))
	-	return ...;
	+if (IS_ERR(e))
	+	return PTR_ERR(e);

I don't think you should need this. Isomorphisms (ie, rules that cause x == NULL to also match !x) should make { ... X } match just X also.

Quoted 40 lines
	)

	@r depends on report@
	expression e;
	position p1;
	@@

	*e = devm_request_and_ioremap at p1(...);
	 ...
	(
	 if (\(e == NULL\|!e\)) {
	 	...
	 	return ...;
	 }
	|
	 if (\(e == NULL\|!e\))
	 	return ...;
	)

	@script:python depends on r@
	p1 << r.p1;
	@@

	msg = "ERROR: old devm_request_and_ioremap() API used on line %s" % (p1[0].line)
	coccilib.report.print_report(p1[0], msg)

This seems to work fine for most cases, but I've come across a tricky one in drivers/gpio/gpio-mxs.c. The code looks roughly like this:

	if (np) {
		...
		base = of_iomap(parent, 0);
		...
	} else {
		iores = platform_get_resource(...);
		base = devm_request_and_ioremap(...);
	}
	if (!base)
		return -EADDRNOTAVAIL;
Quoted 4 lines

The odd thing is that in report mode this case is actually detected, but the patch mode doesn't generate a hunk for this occurrence.

I wonder what I need to change to make this case work.

Actually, this seems like a feature, not a bug. Coccinelle has no way of knowing that the other value of base can also be an argument to PTR_ERR. I think you will have to fix this case by hand. Or make a completely specific rule for it, which is probably not worthwhile.

The reason why you get different answers is that when you use * it searches for whether there exists a path with the given property. And when you don't use *, it checks that all paths have the property. Actually, in your case I would have expected Coccinelle to crash, because your path matches a node that can also be reached from a node that is not part of your path.

You can explicitly specify whether all paths should be considered or only a single path should be considered by putting forall or exists between the initial @@. Doing this is often useful when you are doing bug finding but use python to print line number information, rather than using * to get a diff. You can also put when forall or when exists on a ... if you need very fine-grained control.

julia

Quoted 11 lines

From my very
limited understanding of coccinelle it would seem that the issue is that the devm_request_and_ioremap() call and the return value check are in different blocks, but I have no idea how to take that into account. Then again, maybe the problem is something completely different.

Does anybody have any hints?

Thanks,
Thierry

#2 lore raw
In reply to Peter Senna Tschudin

Review

Quoted 10 lines

I made some changes for patch mode:
@@
expression e;
@@

e = devm_request_and_ioremap(...);
...
if

(quoted diff) +1 -1
-(e == NULL)
+(IS_ERR(e))

Actually, a slightly more defensive thing to do is

if (
- e == NULL
+ IS_ERR(e)
|| ...)

This will allow the null test to be anywhere in a toplevel ||, ie

   !e || x    x || !e   a || b || !e   a || !e || b
Quoted 6 lines

{
<...

(quoted diff) +1 -1
-       return ...;
+       return PTR_ERR(e);

...>
}

I started typing a long explanation as to why this was not a good idea, but now I realize that actually it is a good idea. If you put

if (e == NULL) {
...
return ...;
}

then the ... above the return will jump over gotos. For pattern matching (ie report mode) this is probably what you want. But for actual patching, this is not what you want at all. Functions that use devm functions often use gotos for their error handling, so there is just one return at the end of the function that is shared by everyone. You certainly don't want to turn this into a call to PTR_ERR. By using <... ...> you stay within the simple case where the return is right up between the braces of the if header.

On the other hand there are probably a lot of cases that you could transform automatically that you are missing. The common thing to do is say ret = -EXXX for some XXX, where ret is later returned. Probably what you would like to do is:

if (e == NULL) { // fill in the modification on the test
... when any
- ret = E;
+ ret = PTR_ERR(e);
...
return ret;
}

This should be put in a separate rule after the one with the <... ...>.

When any allows there to be any assignment before ret = E. What you are interested in is the last assignment, in particular the last assignment of the value that is returned. Not putting anything on the ... at the end will give you this shortest path property. If you are really worried about gotos, you could also put when != goto l; where l is an identifier metavariable on the first ... But I think that the chance of a problem there is minimal.

julia

Quoted 79 lines

You can save this text in a .cocci file then use it with:

$ cd /linux-source
$ spatch -dir . /tmp/file.cocci

Note that some isomorphisms like e = NULL and !e are automatically taken care for you. Check Coccinelle iso file for more details.

Quoted 15 lines

This seems to work fine for most cases, but I've come across a tricky one in drivers/gpio/gpio-mxs.c. The code looks roughly like this:

        if (np) {
                ...
                base = of_iomap(parent, 0);
                ...
        } else {
                iores = platform_get_resource(...);
                base = devm_request_and_ioremap(...);
        }
        if (!base)
                return -EADDRNOTAVAIL;
Quoted 2 lines

The odd thing is that in report mode this case is actually detected, but the patch mode doesn't generate a hunk for this occurrence.

The position variable is storing the position of devm_request_and_ioremap, this is why the report mode detect it. The problem is that base could receive its content from devm_request_and_ioremap(...); or from of_iomap(parent, 0);. For this case I would manually add different return for each case. Something like:

if (np) {
        ...
        base = of_iomap(parent, 0);
        ...
        if (!base)
                return -EADDRNOTAVAIL;
} else {
        iores = platform_get_resource(...);
        base = devm_request_and_ioremap(...);
        if (!base)
                return PTR_ERR(base);
}
Quoted 8 lines

I wonder what I need to change to make this case work. From my very limited understanding of coccinelle it would seem that the issue is that the devm_request_and_ioremap() call and the return value check are in different blocks, but I have no idea how to take that into account. Then again, maybe the problem is something completely different.

Does anybody have any hints?

1 - I would work on a Coccinelle script that works for the common cases and do the exceptions manually.
2 - Even for changes made by Coccinelle I would double check if the proposed transformation is what I was expecting.
3 - There are a lot of examples at: coccinellery.org

Quoted 9 lines

Thanks,
Thierry

_______________________________________________
Cocci mailing list
Cocci at systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

--
Peter
_______________________________________________
Cocci mailing list
Cocci at systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

#5 lore raw
In reply to Peter Senna Tschudin
Quoted 5 lines · Peter Senna Tschudin

Hi,

What version of Coccinelle are you using? I suggest you try the latest one available at:
http://coccinelle.lip6.fr/download.php

I use 1.0.0-rc15, which is the latest available in Arch Linux AUR. Last time I tried to install manually there were some problems and I was lazy. =)

Quoted 9 lines
Quoted 6 lines · Thierry Reding

Hi,

I'm trying to come up with a semantic patch to modify the meaning of the devm_request_and_ioremap() functions return value. The goal is to return an ERR_PTR() encoded errno in order to allow the proper error code to be propagated to the caller.

Will you change the function itself so it will return ERR_PRT() encoded errno?

Yes, that's the plan.

Quoted 17 lines

I made some changes for patch mode:
@@
expression e;
@@

e = devm_request_and_ioremap(...);
...
if

(quoted diff) +1 -1
-(e == NULL)
+(IS_ERR(e))

{
<...

(quoted diff) +1 -1
-       return ...;
+       return PTR_ERR(e);

...>
}

Quoted 6 lines

You can save this text in a .cocci file then use it with:

$ cd /linux-source
$ spatch -dir . /tmp/file.cocci

Note that some isomorphisms like e = NULL and !e are automatically taken care for you. Check Coccinelle iso file for more details.

Okay, that's certainly a lot simpler. Thanks.

Quoted 24 lines
Quoted 14 lines

This seems to work fine for most cases, but I've come across a tricky one in drivers/gpio/gpio-mxs.c. The code looks roughly like this:

        if (np) {
                ...
                base = of_iomap(parent, 0);
                ...
        } else {
                iores = platform_get_resource(...);
                base = devm_request_and_ioremap(...);
        }
        if (!base)
                return -EADDRNOTAVAIL;
Quoted 2 lines

The odd thing is that in report mode this case is actually detected, but the patch mode doesn't generate a hunk for this occurrence.

The position variable is storing the position of devm_request_and_ioremap, this is why the report mode detect it. The problem is that base could receive its content from devm_request_and_ioremap(...); or from of_iomap(parent, 0);. For this case I would manually add different return for each case. Something like:

Quoted 12 lines
if (np) {
        ...
        base = of_iomap(parent, 0);
        ...
        if (!base)
                return -EADDRNOTAVAIL;
} else {
        iores = platform_get_resource(...);
        base = devm_request_and_ioremap(...);
        if (!base)
                return PTR_ERR(base);
}

Right, now that you mention it, of_iomap() doesn't return ERR_PTR() at all, so the common if (!base) needs to be split up anyway.

Quoted 11 lines
Quoted 7 lines

I wonder what I need to change to make this case work. From my very limited understanding of coccinelle it would seem that the issue is that the devm_request_and_ioremap() call and the return value check are in different blocks, but I have no idea how to take that into account. Then again, maybe the problem is something completely different.

Does anybody have any hints?

1 - I would work on a Coccinelle script that works for the common cases and do the exceptions manually.
2 - Even for changes made by Coccinelle I would double check if the proposed transformation is what I was expecting.

Yes, that sounds like a good plan. I was going to review the generated patches anyway, but this one non-trivial case (and probably others) may need more thought and manual conversion might be safest anyway.

Quoted 1 line

3 - There are a lot of examples at: coccinellery.org

Nice, I hadn't tumbled upon that one yet. Thanks for the reference.

Thierry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <https://systeme.lip6.fr/pipermail/cocci/attachments/20130109/8e564fdb/attachment-0001.asc>

#3 lore raw
In reply to Julia Lawall
Quoted 18 lines · Julia Lawall

Review

Quoted 10 lines

I made some changes for patch mode:
@@
expression e;
@@

e = devm_request_and_ioremap(...);
...
if

(quoted diff) +1 -1
-(e == NULL)
+(IS_ERR(e))

Actually, a slightly more defensive thing to do is

if (

(quoted diff) +1 -1
-   e == NULL
+   IS_ERR(e)
 || ...)
Quoted 3 lines

This will allow the null test to be anywhere in a toplevel ||, ie

   !e || x    x || !e   a || b || !e   a || !e || b

Okay, I'll roll that into the semantic patch. I haven't seen any occurrence where the check is used in a composite condition, but you never know.

Quoted 37 lines
Quoted 6 lines

{
<...

(quoted diff) +1 -1
-       return ...;
+       return PTR_ERR(e);

...>
}

I started typing a long explanation as to why this was not a good idea, but now I realize that actually it is a good idea. If you put

if (e == NULL) {
...
return ...;
}

then the ... above the return will jump over gotos. For pattern matching (ie report mode) this is probably what you want. But for actual patching, this is not what you want at all. Functions that use devm functions often use gotos for their error handling, so there is just one return at the end of the function that is shared by everyone. You certainly don't want to turn this into a call to PTR_ERR. By using <... ...> you stay within the simple case where the return is right up between the braces of the if header.

On the other hand there are probably a lot of cases that you could transform automatically that you are missing. The common thing to do is say ret = -EXXX for some XXX, where ret is later returned. Probably what you would like to do is:

if (e == NULL) { // fill in the modification on the test
... when any
- ret = E;
+ ret = PTR_ERR(e);
...
return ret;
}

Quoted 1 line

This should be put in a separate rule after the one with the <... ...>.

I've currently only tested this on a few driver subdirectories to check that it generally works, but yes, something like the above could be useful to match more cases.

While we're at it, I was going to follow up with another semantic patch to remove error messages in the devm_request_and_ioremap() errors paths because the function outputs an error message itself already. In many cases this results in an if block becoming a single line, so CodingStyle requires the { } to be removed. Is there any way to do that automatically?

Quoted 7 lines

When any allows there to be any assignment before ret = E. What you are interested in is the last assignment, in particular the last assignment of the value that is returned. Not putting anything on the ... at the end will give you this shortest path property. If you are really worried about gotos, you could also put when != goto l; where l is an identifier metavariable on the first ... But I think that the chance of a problem there is minimal.

Thanks for explaining this. I had been wondering about "when any". I'm very new to coccinelle so there's still a lot to learn.

Thierry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <https://systeme.lip6.fr/pipermail/cocci/attachments/20130109/38be7c1d/attachment.asc>

#4 lore raw
Quoted 6 lines

While we're at it, I was going to follow up with another semantic patch to remove error messages in the devm_request_and_ioremap() errors paths because the function outputs an error message itself already. In many cases this results in an if block becoming a single line, so CodingStyle requires the { } to be removed. Is there any way to do that automatically?

One solution would be to put eg

if (...)
- {
(
goto l;
|
return ...;
)
- }

at the end of your semantic patch. That will affect everything, but if you don't mind doing a little extra work on the side, it could be helpful to submit patches to fix up those as well.

The other solution is to do it as part of removing the printing command. That is, to first have a rule that does something like (you can fill in all the transformation parts that I have left out):

x = devm...;
... when != x = e
if (x == NULL || ...)
- {
- f(...,c,...);
return ...;
- }

And then have a second rule that does the more general case where the { } don't need to be removed.

c should be declared as char [] c;

I'm just assuming that anything with a string argument is undesirable. On the other hand, there is no way to match something like KERN_EMERG "string". Another alternative would be just to put all of the names of the error reporting functions that you know of. You can also put them in a ( | ), ie

(
- printk(...);
|
- f(...,c,...);
)

It take the first match that works out, so you don't have to worry about the cases being disjoint.

julia

Quoted 13 lines
Quoted 7 lines

When any allows there to be any assignment before ret = E. What you are interested in is the last assignment, in particular the last assignment of the value that is returned. Not putting anything on the ... at the end will give you this shortest path property. If you are really worried about gotos, you could also put when != goto l; where l is an identifier metavariable on the first ... But I think that the chance of a problem there is minimal.

Thanks for explaining this. I had been wondering about "when any". I'm very new to coccinelle so there's still a lot to learn.

Thierry