Skip to contentSkip to conversation

coccinelle: semantic patch to check for inappropriate do_div() calls

PATCH In review Wen Yang 3 participants 10 messages Jan 10, 2020 – Jan 12, 2020

Subsystems (2)
lore
Share & tools
lore

Wen Yang

author
#0 lore raw

PATCH v3 1/1 coccinelle: semantic patch to check for inappropriate do_div() calls

do_div() does a 64-by-32 division.
When the divisor is unsigned long, u64, or s64,
do_div() truncates it to 32 bits, this means it
can test non-zero and be truncated to zero for division.
This semantic patch is inspired by Mateusz Guzik's patch:
commit b0ab99e7736a ("sched: Fix possible divide by zero in avg_atom() calculation")

Cc Julia Lawall <[email protected]>
Cc Gilles Muller <[email protected]>
Cc Nicolas Palix <[email protected]>
Cc Michal Marek <[email protected]>
Cc Matthias Maennich <[email protected]>
Cc Greg Kroah-Hartman <[email protected]>
Cc Masahiro Yamada <[email protected]>
Cc Thomas Gleixner <[email protected]>

---
v3:
- also filter out safe consts for context mode.
- cleanup code.

v2:
- add a special case for constants and checking whether the value is obviously safe and no warning is needed.
- fix 'WARNING:' twice in each case.
- extend the warning to say "consider using div64_xxx instead".

 scripts/coccinelle/misc/do_div.cocci | 155 +++++++++++++++++++++++++++
 1 file changed, 155 insertions(+)
 create mode 100644 scripts/coccinelle/misc/do_div.cocci
scripts/coccinelle/misc/do_div.cocci new file +155 -0
new file mode 100644
index 000000000000..79db083c5208
@@ -0,0 +1,155 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/// do_div() does a 64-by-32 division.
+/// When the divisor is long, unsigned long, u64, or s64,
+/// do_div() truncates it to 32 bits, this means it can test
+/// non-zero and be truncated to 0 for division on 64bit platforms.
+///
+//# This makes an effort to find those inappropriate do_div() calls.
+//
+// Confidence: Moderate
+// Copyright: (C) 2020 Wen Yang, Alibaba.
+// Comments:
+// Options: --no-includes --include-headers
+
+virtual context
+virtual org
+virtual report
+
+@initialize:python@
+@@
+
+def get_digit_type_and_value(str):
+    is_digit = False
+    value = 0
+
+    try:
+        if (str.isdigit()):
+           is_digit = True
+           value =  int(str, 0)
+        elif (str.upper().endswith('ULL')):
+           is_digit = True
+           value = int(str[:-3], 0)
+        elif (str.upper().endswith('LL')):
+           is_digit = True
+           value = int(str[:-2], 0)
+        elif (str.upper().endswith('UL')):
+           is_digit = True
+           value = int(str[:-2], 0)
+        elif (str.upper().endswith('L')):
+           is_digit = True
+           value = int(str[:-1], 0)
+        elif (str.upper().endswith('U')):
+           is_digit = True
+           value = int(str[:-1], 0)
+    except Exception as e:
+          print('Error:',e)
+          is_digit = False
+          value = 0
+    finally:
+        return is_digit, value
+
+def filter_out_safe_constants(str):
+    is_digit, value = get_digit_type_and_value(str)
+    if (is_digit):
+        if (value >= 0x100000000):
+            return True
+        else:
+            return False
+    else:
+        return True
+
+def construct_warnings(suggested_fun):
+    msg="WARNING: do_div() does a 64-by-32 division, please consider using %s instead."
+    return  msg % suggested_fun
+
+@depends on context@
+expression f;
+long l: script:python() { filter_out_safe_constants(l) };
+unsigned long ul : script:python() { filter_out_safe_constants(ul) };
+u64 ul64 : script:python() { filter_out_safe_constants(ul64) };
+s64 sl64 : script:python() { filter_out_safe_constants(sl64) };
+
+@@
+(
+* do_div(f, l);
+|
+* do_div(f, ul);
+|
+* do_div(f, ul64);
+|
+* do_div(f, sl64);
+)
+
+@r depends on (org || report)@
+expression f;
+position p;
+long l: script:python() { filter_out_safe_constants(l) };
+unsigned long ul : script:python() { filter_out_safe_constants(ul) };
+u64 ul64 : script:python() { filter_out_safe_constants(ul64) };
+s64 sl64 : script:python() { filter_out_safe_constants(sl64) };
+@@
+(
+do_div@p(f, l);
+|
+do_div@p(f, ul);
+|
+do_div@p(f, ul64);
+|
+do_div@p(f, sl64);
+)
+
+@script:python depends on org@
+p << r.p;
+ul << r.ul;
+@@
+
+coccilib.org.print_todo(p[0], construct_warnings("div64_ul"))
+
+@script:python depends on org@
+p << r.p;
+l << r.l;
+@@
+
+coccilib.org.print_todo(p[0], construct_warnings("div64_long"))
+
+@script:python depends on org@
+p << r.p;
+ul64 << r.ul64;
+@@
+
+coccilib.org.print_todo(p[0], construct_warnings("div64_u64"))
+
+@script:python depends on org@
+p << r.p;
+sl64 << r.sl64;
+@@
+
+coccilib.org.print_todo(p[0], construct_warnings("div64_s64"))
+
+@script:python depends on report@
+p << r.p;
+ul << r.ul;
+@@
+
+coccilib.report.print_report(p[0], construct_warnings("div64_ul"))
+
+@script:python depends on report@
+p << r.p;
+l << r.l;
+@@
+
+coccilib.report.print_report(p[0], construct_warnings("div64_long"))
+
+@script:python depends on report@
+p << r.p;
+sl64 << r.sl64;
+@@
+
+coccilib.report.print_report(p[0], construct_warnings("div64_s64"))
+
+@script:python depends on report@
+p << r.p;
+ul64 << r.ul64;
+@@
+
+coccilib.report.print_report(p[0], construct_warnings("div64_u64"))
Signature · 1 line
2.23.0
#1 lore raw
Quoted 1 line

+@initialize:python@

…

Quoted 1 line

+def filter_out_safe_constants(str):

…

Quoted 1 line

+def construct_warnings(str, suggested_fun):

* I suggest once more to adjust the dependency specifications for the usage of these functions by SmPL rules.

* Can the local variable “msg” be omitted?

Quoted 1 line

+coccilib.org.print_todo(p[0], construct_warnings("div64_ul"))

I suggest again to move the prefix “div64_” into the string literal of the function implementation.

The SmPL code for two disjunctions could become shorter.

Regards,
Markus

author
#2 lore raw
Quoted 9 lines · Markus Elfring
Quoted 1 line

+@initialize:python@

…

Quoted 1 line

+def filter_out_safe_constants(str):

…

Quoted 1 line

+def construct_warnings(str, suggested_fun):

* I suggest once more to adjust the dependency specifications for the usage of these functions by SmPL rules.

Most of the functions here are for all operation modes.

Quoted 8 lines

* Can the local variable “msg” be omitted?

Quoted 1 line

+coccilib.org.print_todo(p[0], construct_warnings("div64_ul"))

I suggest again to move the prefix “div64_” into the string literal of the function implementation.

“div64_ul” indicates the function name we recommend. As shown in the patch:

+def construct_warnings(suggested_fun):
+ msg="WARNING: do_div() does a 64-by-32 division, please consider
using %s instead."
+ return msg % suggested_fun
...
+coccilib.org.print_todo(p[0], construct_warnings("div64_ul"))

If we delete the prefix "div64_", it may reduce readability.

Quoted 3 lines

The SmPL code for two disjunctions could become shorter.

You may suggest to modify it as follows:
+@@
+*do_div(f, \( l \| ul \| ul64 \| sl64 \) );

We agree with Julia:
I don't se any point to this. The code matched will be the same in both cases. The original code is quite readable, without the ugly \( etc.

--
Regards,
Wen

Quoted 3 lines

Regards,
Markus

#3 lore raw
Quoted 4 lines
Quoted 2 lines

* I suggest once more to adjust the dependency specifications for the usage of these functions by SmPL rules.

Most of the functions here are for all operation modes.

I got an other understanding for this software.

You added the information “also filter out safe consts for context mode” to the patch change log.

Quoted 1 line
Quoted 1 line

* Can the local variable “msg” be omitted?

I would appreciate another fine-tuning also at this place.

Quoted 6 lines
Quoted 4 lines
Quoted 1 line

+coccilib.org.print_todo(p[0], construct_warnings("div64_ul"))

I suggest again to move the prefix “div64_” into the string literal of the function implementation.

“div64_ul” indicates the function name we recommend.

The intention can be fine.

Quoted 1 line

If we delete the prefix "div64_",

I suggest to use the text at an other place.

Quoted 1 line

it may reduce readability.

I find an other code variant also readable good enough.

Quoted 4 lines

+*do_div(f, \( l \| ul \| ul64 \| sl64 \) );

We agree with Julia:
I don't se any point to this.

Can the avoidance of duplicate source code (according to SmPL disjunctions) trigger positive effects on run time characteristics and software maintenance?

Regards,
Markus

maintainer
#4 lore raw
Quoted 7 lines
Quoted 4 lines

+*do_div(f, \( l \| ul \| ul64 \| sl64 \) );

We agree with Julia:
I don't se any point to this.

Can the avoidance of duplicate source code (according to SmPL disjunctions) trigger positive effects on run time characteristics and software maintenance?

Markus. Please stop asking this question. You are bothering people with this advice, why don't _you_ figure out once and for all whether the change that you suggest has any "positive effects on the run time characteristics"? Hint: it will not. You don't even have to run Coccinelle to see that. Just use spatch --parse-cocci on your two suggestions and you will see that they expand to the same thing. Coccinelle has a pass that propagates disjunctions at the sub-statement level to the statement level.

julia

#5 lore raw
Quoted 4 lines
Quoted 2 lines

Can the avoidance of duplicate source code (according to SmPL disjunctions) trigger positive effects on run time characteristics and software maintenance?

Markus. Please stop asking this question.

This will not happen for a while.

Quoted 1 line

You are bothering people with this advice,

I present just another view.

Quoted 3 lines

why don't _you_ figure out once and for all whether the change that you suggest has any "positive effects on the run time characteristics"?
Hint: it will not.

* How much attention do you give to the software development principle "Don't repeat yourself"?

* Can the file size of a SmPL script matter a bit?

Quoted 2 lines

Coccinelle has a pass that propagates disjunctions at the sub-statement level to the statement level.

This data processing can probably trigger further development considerations.

Regards,
Markus

maintainer Acked-by on v3 1/1
#6 lore raw
In reply to Wen Yang
Quoted 8 lines · Wen Yang

do_div() does a 64-by-32 division.
When the divisor is unsigned long, u64, or s64,
do_div() truncates it to 32 bits, this means it
can test non-zero and be truncated to zero for division.
This semantic patch is inspired by Mateusz Guzik's patch:
commit b0ab99e7736a ("sched: Fix possible divide by zero in avg_atom() calculation")

This looks good to me.

A small detail is that you don't need the parentheses in:

@r depends on (org || report)@

julia

Quoted 189 lines
Cc Julia Lawall <[email protected]>
Cc Gilles Muller <[email protected]>
Cc Nicolas Palix <[email protected]>
Cc Michal Marek <[email protected]>
Cc Matthias Maennich <[email protected]>
Cc Greg Kroah-Hartman <[email protected]>
Cc Masahiro Yamada <[email protected]>
Cc Thomas Gleixner <[email protected]>

---
v3:
- also filter out safe consts for context mode.
- cleanup code.

v2:
- add a special case for constants and checking whether the value is obviously safe and no warning is needed.
- fix 'WARNING:' twice in each case.
- extend the warning to say "consider using div64_xxx instead".

 scripts/coccinelle/misc/do_div.cocci | 155 +++++++++++++++++++++++++++
 1 file changed, 155 insertions(+)
 create mode 100644 scripts/coccinelle/misc/do_div.cocci
scripts/coccinelle/misc/do_div.cocci new file +155 -1
new file mode 100644
index 000000000000..79db083c5208
@@ -0,0 +1,155 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/// do_div() does a 64-by-32 division.
+/// When the divisor is long, unsigned long, u64, or s64,
+/// do_div() truncates it to 32 bits, this means it can test
+/// non-zero and be truncated to 0 for division on 64bit platforms.
+///
+//# This makes an effort to find those inappropriate do_div() calls.
+//
+// Confidence: Moderate
+// Copyright: (C) 2020 Wen Yang, Alibaba.
+// Comments:
+// Options: --no-includes --include-headers
+
+virtual context
+virtual org
+virtual report
+
+@initialize:python@
+@@
+
+def get_digit_type_and_value(str):
+    is_digit = False
+    value = 0
+
+    try:
+        if (str.isdigit()):
+           is_digit = True
+           value =  int(str, 0)
+        elif (str.upper().endswith('ULL')):
+           is_digit = True
+           value = int(str[:-3], 0)
+        elif (str.upper().endswith('LL')):
+           is_digit = True
+           value = int(str[:-2], 0)
+        elif (str.upper().endswith('UL')):
+           is_digit = True
+           value = int(str[:-2], 0)
+        elif (str.upper().endswith('L')):
+           is_digit = True
+           value = int(str[:-1], 0)
+        elif (str.upper().endswith('U')):
+           is_digit = True
+           value = int(str[:-1], 0)
+    except Exception as e:
+          print('Error:',e)
+          is_digit = False
+          value = 0
+    finally:
+        return is_digit, value
+
+def filter_out_safe_constants(str):
+    is_digit, value = get_digit_type_and_value(str)
+    if (is_digit):
+        if (value >= 0x100000000):
+            return True
+        else:
+            return False
+    else:
+        return True
+
+def construct_warnings(suggested_fun):
+    msg="WARNING: do_div() does a 64-by-32 division, please consider using %s instead."
+    return  msg % suggested_fun
+
+@depends on context@
+expression f;
+long l: script:python() { filter_out_safe_constants(l) };
+unsigned long ul : script:python() { filter_out_safe_constants(ul) };
+u64 ul64 : script:python() { filter_out_safe_constants(ul64) };
+s64 sl64 : script:python() { filter_out_safe_constants(sl64) };
+
+@@
+(
+* do_div(f, l);
+|
+* do_div(f, ul);
+|
+* do_div(f, ul64);
+|
+* do_div(f, sl64);
+)
+
+@r depends on (org || report)@
+expression f;
+position p;
+long l: script:python() { filter_out_safe_constants(l) };
+unsigned long ul : script:python() { filter_out_safe_constants(ul) };
+u64 ul64 : script:python() { filter_out_safe_constants(ul64) };
+s64 sl64 : script:python() { filter_out_safe_constants(sl64) };
+@@
+(
+do_div@p(f, l);
+|
+do_div@p(f, ul);
+|
+do_div@p(f, ul64);
+|
+do_div@p(f, sl64);
+)
+
+@script:python depends on org@
+p << r.p;
+ul << r.ul;
+@@
+
+coccilib.org.print_todo(p[0], construct_warnings("div64_ul"))
+
+@script:python depends on org@
+p << r.p;
+l << r.l;
+@@
+
+coccilib.org.print_todo(p[0], construct_warnings("div64_long"))
+
+@script:python depends on org@
+p << r.p;
+ul64 << r.ul64;
+@@
+
+coccilib.org.print_todo(p[0], construct_warnings("div64_u64"))
+
+@script:python depends on org@
+p << r.p;
+sl64 << r.sl64;
+@@
+
+coccilib.org.print_todo(p[0], construct_warnings("div64_s64"))
+
+@script:python depends on report@
+p << r.p;
+ul << r.ul;
+@@
+
+coccilib.report.print_report(p[0], construct_warnings("div64_ul"))
+
+@script:python depends on report@
+p << r.p;
+l << r.l;
+@@
+
+coccilib.report.print_report(p[0], construct_warnings("div64_long"))
+
+@script:python depends on report@
+p << r.p;
+sl64 << r.sl64;
+@@
+
+coccilib.report.print_report(p[0], construct_warnings("div64_s64"))
+
+@script:python depends on report@
+p << r.p;
+ul64 << r.ul64;
+@@
+
+coccilib.report.print_report(p[0], construct_warnings("div64_u64"))
--

2.23.0

#7 lore raw
In reply to Wen Yang
Quoted 1 line

This semantic patch is inspired by Mateusz Guzik's patch:

Does such a wording mean also that you would like to support the operation mode “patch” by this SmPL script?

Regards,
Markus

maintainer
#8 lore raw
Quoted 4 lines · Markus Elfring
Quoted 1 line

This semantic patch is inspired by Mateusz Guzik's patch:

Does such a wording mean also that you would like to support the operation mode “patch” by this SmPL script?

I see no reason why such a wording would imply such a thing.

julia

#9 lore raw
Quoted 1 line

I see no reason why such a wording would imply such a thing.

Thus I suggest once more to improve the distinction between patching and the description of source code searches in the commit message.

Regards,
Markus