Skip to contentSkip to conversation

Analysis for Linux source files

Cocci SF Markus Elfring 2 participants 11 messages Apr 2, 2013

lore
Share & tools
lore
#0 lore raw

Hello,

I try out a bit of static source code analysis again by the following command.

elfring at Sonne:~/Projekte/Coccinelle/lokal/demos/pass-through> XY=/usr/src/linux-stable/ && spatch -sp_file list_pass-through_functions.cocci \
-dir $XY \
-I ${XY}include \
-I ${XY}usr/include \
-I ${XY}arch/ia64/include \
-I ${XY}arch/ia64/include/uapi \
-I ${XY}arch/x86/include \
-I ${XY}arch/x86/include/generated \
-I /usr/include \
-I /usr/include/c++/4.7/tr1 \
-recursive_includes \

Quoted 1 line

list_pass-through_functions-linux.txt \

2>list_pass-through_functions-linux-errors.txt

I notice results like the following in my log file.

...
HANDLING: /usr/src/linux-stable/init/noinitramfs.c EXN:Failure("empty list, max_min_ii_by_pos")
Note: processing took 85.2s: /usr/src/linux-stable/init/noinitramfs.c
HANDLING: /usr/src/linux-stable/init/do_mounts_md.c EXN:Failure("empty list, max_min_ii_by_pos")
Note: processing took 109.5s: /usr/src/linux-stable/init/do_mounts_md.c
...

Can the shown error message be avoided?
What does it really mean?

openSUSE package: coccinelle 1.0.0-27.6
spatch version 1.0.0-rc14 with Python support and with Str regexp support

Regards,
Markus

#1 lore raw
Quoted 22 lines · SF Markus Elfring

Hello,

I try out a bit of static source code analysis again by the following command.

elfring at Sonne:~/Projekte/Coccinelle/lokal/demos/pass-through> XY=/usr/src/linux-stable/ && spatch -sp_file list_pass-through_functions.cocci \

(quoted diff) +0 -10
-dir $XY \
-I ${XY}include \
-I ${XY}usr/include \
-I ${XY}arch/ia64/include \
-I ${XY}arch/ia64/include/uapi \
-I ${XY}arch/x86/include \
-I ${XY}arch/x86/include/generated \
-I /usr/include \
-I /usr/include/c++/4.7/tr1 \
-recursive_includes \
Quoted 1 line

list_pass-through_functions-linux.txt \

2>list_pass-through_functions-linux-errors.txt

I notice results like the following in my log file.

Quoted 7 lines

...
HANDLING: /usr/src/linux-stable/init/noinitramfs.c EXN:Failure("empty list, max_min_ii_by_pos")
Note: processing took 85.2s: /usr/src/linux-stable/init/noinitramfs.c
HANDLING: /usr/src/linux-stable/init/do_mounts_md.c EXN:Failure("empty list, max_min_ii_by_pos")
Note: processing took 109.5s: /usr/src/linux-stable/init/do_mounts_md.c

Without your semantic patch and some example code, I don't know.

julia

#2 lore raw
Quoted 1 line

Without your semantic patch and some example code, I don't know.

I let the following SmPL filter pattern run on source files from the current Linux 3.8.5.
https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git

elfring at Sonne:~/Projekte/Coccinelle/lokal/demos/pass-through> cat list_pass-through_functions.cocci
@initialize:python@
import sys
result = []
mark = ['"', '', '"']
delimiter = '|'

def store_positions(fun, typ, pt_param, places, data_structure = ""):

    """Add source code positions to an internal list."""


    for place in places:


        fields = []


        fields.append(fun.ident)





        mark[1] = typ


        fields.append(''.join(mark))





        fields.append(pt_param.ident)


        fields.append(data_structure)





        mark[1] = place.file.replace('"', '""')


        fields.append(''.join(mark))





        fields.append(place.line)


        fields.append(str(int(place.column) + 1))


        result.append(delimiter.join(fields))

@is_pass_through@

identifier pt_function, input;
type data_type;
position pos;
@@
data_type at pos pt_function(...,data_type input,...) {
<+...
return input;
...+>
}

@is_pass_through_member_candidate@
identifier pt_function, input, tag;
type data_type;
position pos;
@@
struct tag
{
...
data_type at pos (*pt_function)(...,data_type input,...);
...
}

@script:python collection1 depends on is_pass_through@ typ << is_pass_through.data_type;
fun << is_pass_through.pt_function;
pt_param << is_pass_through.input;
places << is_pass_through.pos;
@@
store_positions(fun, typ, pt_param, places)

@script:python collection2 depends on is_pass_through_member_candidate@ tag << is_pass_through_member_candidate.tag;
typ << is_pass_through_member_candidate.data_type;
fun << is_pass_through_member_candidate.pt_function; pt_param << is_pass_through_member_candidate.input;
places << is_pass_through_member_candidate.pos;
@@
store_positions(fun, typ, pt_param, places, tag)

@finalize:python@
if result:
result.insert(0, delimiter.join(("function", '"data type"', "parameter",
'"contained in"', '"source file"', "line", "column")))
print("\r\n".join(result))
else:
sys.stderr.write("No result for this analysis!\n")

Regards,
Markus

#3 lore raw

When I try, it complains because the parameter fun of store_positions does not have an ident field. Do you not see that problem?

julia

Quoted 126 lines · SF Markus Elfring
Quoted 1 line

Without your semantic patch and some example code, I don't know.

I let the following SmPL filter pattern run on source files from the current Linux 3.8.5.
https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git

elfring at Sonne:~/Projekte/Coccinelle/lokal/demos/pass-through> cat list_pass-through_functions.cocci
@initialize:python@
import sys
result = []
mark = ['"', '', '"']
delimiter = '|'

def store_positions(fun, typ, pt_param, places, data_structure = ""):

    """Add source code positions to an internal list."""


    for place in places:


        fields = []


        fields.append(fun.ident)





        mark[1] = typ


        fields.append(''.join(mark))





        fields.append(pt_param.ident)


        fields.append(data_structure)





        mark[1] = place.file.replace('"', '""')


        fields.append(''.join(mark))





        fields.append(place.line)


        fields.append(str(int(place.column) + 1))


        result.append(delimiter.join(fields))

@is_pass_through@

identifier pt_function, input;
type data_type;
position pos;
@@
data_type at pos pt_function(...,data_type input,...) {
<+...
return input;
...+>
}

@is_pass_through_member_candidate@
identifier pt_function, input, tag;
type data_type;
position pos;
@@
struct tag
{
...
data_type at pos (*pt_function)(...,data_type input,...);
...
}

@script:python collection1 depends on is_pass_through@ typ << is_pass_through.data_type;
fun << is_pass_through.pt_function;
pt_param << is_pass_through.input;
places << is_pass_through.pos;
@@
store_positions(fun, typ, pt_param, places)

@script:python collection2 depends on is_pass_through_member_candidate@ tag << is_pass_through_member_candidate.tag;
typ << is_pass_through_member_candidate.data_type;
fun << is_pass_through_member_candidate.pt_function; pt_param << is_pass_through_member_candidate.input;
places << is_pass_through_member_candidate.pos;
@@
store_positions(fun, typ, pt_param, places, tag)

@finalize:python@
if result:
result.insert(0, delimiter.join(("function", '"data type"', "parameter",
'"contained in"', '"source file"', "line", "column")))
print("\r\n".join(result))
else:
sys.stderr.write("No result for this analysis!\n")

Regards,
Markus

#4 lore raw
Quoted 2 lines

When I try, it complains because the parameter fun of store_positions does not have an ident field. Do you not see that problem?

I'm sorry that I did not interpret the error message 'EXN:Failure("empty list, max_min_ii_by_pos")' as an open issue from my Python function "store_positions". I try to reuse the implementation that was already mentioned in a previous discussion.
http://lists.diku.dk/pipermail/cocci/2011-December/002373.html

It works again if I delete the attribute access ".ident"

elfring at Sonne:~/Projekte/Coccinelle/lokal/demos/pass-through> spatch -sp_file list_pass-through_functions2.cocci pt1.c
init_defs_builtins: /usr/share/coccinelle/standard.h
HANDLING: pt1.c
function|"data type"|parameter|"contained in"|"source file"|line|column my_copy|"char *"|destination||"pt1.c"|1|1
my_append|"char *"|prefix||"pt1.c"|8|1

Was the Pycaml interface changed anyhow in the meantime?
Is the corresponding Coccinelle API still described by a Wiki document?

elfring at Sonne:~/Projekte/Coccinelle/lokal/demos/pass-through> XY=/usr/src/linux-stable/ && spatch -sp_file list_pass-through_functions2.cocci \
-I ${XY}include \
-I ${XY}usr/include \
-I ${XY}arch/ia64/include \
-I ${XY}arch/ia64/include/uapi \
-I ${XY}arch/x86/include \
-I ${XY}arch/x86/include/generated \
-I /usr/include \
-I /usr/include/c++/4.7/tr1 \
-recursive_includes \
${XY}init/calibrate.c
init_defs_builtins: /usr/share/coccinelle/standard.h
HANDLING: /usr/src/linux-stable/init/calibrate.c
(ONCE) TYPE: header ia64intrin.h not found
(ONCE) TYPE: header stl/_prolog.h not found
(ONCE) TYPE: header gnu/stubs-x32.h not found
Fatal error: exception Failure("empty list, max_min_ii_by_pos")

Regards,
Markus

#5 lore raw
Quoted 10 lines · SF Markus Elfring
Quoted 2 lines

When I try, it complains because the parameter fun of store_positions does not have an ident field. Do you not see that problem?

I'm sorry that I did not interpret the error message 'EXN:Failure("empty list, max_min_ii_by_pos")' as an open issue from my Python function "store_positions". I try to reuse the implementation that was already mentioned in a previous discussion.
http://lists.diku.dk/pipermail/cocci/2011-December/002373.html

It works again if I delete the attribute access ".ident"

The empty list problem is entirely unrelated to the python problem.

Quoted 11 lines

elfring at Sonne:~/Projekte/Coccinelle/lokal/demos/pass-through> spatch -sp_file list_pass-through_functions2.cocci pt1.c
init_defs_builtins: /usr/share/coccinelle/standard.h
HANDLING: pt1.c
function|"data type"|parameter|"contained in"|"source file"|line|column my_copy|"char *"|destination||"pt1.c"|1|1
my_append|"char *"|prefix||"pt1.c"|8|1

Was the Pycaml interface changed anyhow in the meantime?

No, I don't think so.

Quoted 1 line

Is the corresponding Coccinelle API still described by a Wiki document?

Unfortunately, the wiki machine is not alive any more...

julia

Quoted 24 lines

elfring at Sonne:~/Projekte/Coccinelle/lokal/demos/pass-through> XY=/usr/src/linux-stable/ && spatch -sp_file list_pass-through_functions2.cocci \

(quoted diff) +0 -9
-I ${XY}include \
-I ${XY}usr/include \
-I ${XY}arch/ia64/include \
-I ${XY}arch/ia64/include/uapi \
-I ${XY}arch/x86/include \
-I ${XY}arch/x86/include/generated \
-I /usr/include \
-I /usr/include/c++/4.7/tr1 \
-recursive_includes \

${XY}init/calibrate.c
init_defs_builtins: /usr/share/coccinelle/standard.h
HANDLING: /usr/src/linux-stable/init/calibrate.c (ONCE) TYPE: header ia64intrin.h not found
(ONCE) TYPE: header stl/_prolog.h not found
(ONCE) TYPE: header gnu/stubs-x32.h not found
Fatal error: exception Failure("empty list, max_min_ii_by_pos")

Regards,
Markus

#6 lore raw
Quoted 1 line

The empty list problem is entirely unrelated to the python problem.

Can I do anything about it from my side?

Quoted 1 line

Unfortunately, the wiki machine is not alive any more...

Under which address will the description of the Pycaml interface (Coccinelle API) become available again?

Regards,
Markus

#7 lore raw
Quoted 9 lines · SF Markus Elfring
Quoted 1 line

The empty list problem is entirely unrelated to the python problem.

Can I do anything about it from my side?

Quoted 1 line

Unfortunately, the wiki machine is not alive any more...

Under which address will the description of the Pycaml interface (Coccinelle API) become available again?

Ideally there would be a man page, like there is for the ocaml interface. I can try to look into putting that together.

julia

#8 lore raw
In reply to SF Markus Elfring

I get the problem on the following code:

irqreturn_t
nouveau_irq_handler(DRM_IRQ_ARGS)

{
        struct drm_device *dev = arg;
        struct nouveau_device *device = nouveau_dev(dev);
        struct nouveau_mc *pmc = nouveau_mc(device);
        u32 stat;

        stat = nv_rd32(device, 0x000100);
        if (stat == 0 || stat == ~0)
                return IRQ_NONE;

        nv_subdev(pmc)->intr(nv_subdev(pmc));
        return IRQ_HANDLED;
}

I would suspect that there is some inconsistency in how it is parsing this kind of argument list. It seems to parse it OK, but then not be able to match it against a pattern like ...,x,... I can look at it in more detail tomorrow.

julia

#9 lore raw
In reply to SF Markus Elfring

Again, to understand the problem, I need the .cocci file, and preferably the .c file as well. Preferably a small .c file that shows the problem.

julia

Quoted 48 lines · SF Markus Elfring
Quoted 2 lines

When I try, it complains because the parameter fun of store_positions does not have an ident field. Do you not see that problem?

I'm sorry that I did not interpret the error message 'EXN:Failure("empty list, max_min_ii_by_pos")' as an open issue from my Python function "store_positions". I try to reuse the implementation that was already mentioned in a previous discussion.
http://lists.diku.dk/pipermail/cocci/2011-December/002373.html

It works again if I delete the attribute access ".ident"

elfring at Sonne:~/Projekte/Coccinelle/lokal/demos/pass-through> spatch -sp_file list_pass-through_functions2.cocci pt1.c
init_defs_builtins: /usr/share/coccinelle/standard.h
HANDLING: pt1.c
function|"data type"|parameter|"contained in"|"source file"|line|column my_copy|"char *"|destination||"pt1.c"|1|1
my_append|"char *"|prefix||"pt1.c"|8|1

Was the Pycaml interface changed anyhow in the meantime?
Is the corresponding Coccinelle API still described by a Wiki document?

elfring at Sonne:~/Projekte/Coccinelle/lokal/demos/pass-through> XY=/usr/src/linux-stable/ && spatch -sp_file list_pass-through_functions2.cocci \

(quoted diff) +0 -9
-I ${XY}include \
-I ${XY}usr/include \
-I ${XY}arch/ia64/include \
-I ${XY}arch/ia64/include/uapi \
-I ${XY}arch/x86/include \
-I ${XY}arch/x86/include/generated \
-I /usr/include \
-I /usr/include/c++/4.7/tr1 \
-recursive_includes \

${XY}init/calibrate.c
init_defs_builtins: /usr/share/coccinelle/standard.h
HANDLING: /usr/src/linux-stable/init/calibrate.c (ONCE) TYPE: header ia64intrin.h not found
(ONCE) TYPE: header stl/_prolog.h not found
(ONCE) TYPE: header gnu/stubs-x32.h not found
Fatal error: exception Failure("empty list, max_min_ii_by_pos")

Regards,
Markus

#10 lore raw
Quoted 2 lines

Again, to understand the problem, I need the .cocci file, and preferably the .c file as well. Preferably a small .c file that shows the problem.

Would you like to reproduce my issue with the following SmPL filter pattern on
the source file "https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/tree/init/calibrate.c?id=8595c539f0360477189eef91f6337ba44962f72d"?

elfring at Sonne:~/Projekte/Coccinelle/lokal/demos/pass-through> cat list_pass-through_functions2.cocci
@initialize:python@
import sys
result = []
mark = ['"', '', '"']
delimiter = '|'

def store_positions(fun, typ, pt_param, places, data_structure = ""):

    """Add source code positions to an internal list."""
    for place in places:
        fields = []
        fields.append(fun)

        mark[1] = typ
        fields.append(''.join(mark))

        fields.append(pt_param)
        fields.append(data_structure)

        mark[1] = place.file.replace('"', '""')
        fields.append(''.join(mark))

        fields.append(place.line)
        fields.append(str(int(place.column) + 1))
        result.append(delimiter.join(fields))

@is_pass_through@
identifier pt_function, input;
type data_type;
position pos;
@@
data_type at pos pt_function(...,data_type input,...) {
<+...
return input;
...+>
}

@is_pass_through_member_candidate@
identifier pt_function, input, tag;
type data_type;
position pos;
@@
struct tag
{
...
data_type at pos (*pt_function)(...,data_type input,...);
...
}

@script:python collection1 depends on is_pass_through@ typ << is_pass_through.data_type;
fun << is_pass_through.pt_function;
pt_param << is_pass_through.input;
places << is_pass_through.pos;
@@
store_positions(fun, typ, pt_param, places)

@script:python collection2 depends on is_pass_through_member_candidate@ tag << is_pass_through_member_candidate.tag;
typ << is_pass_through_member_candidate.data_type;
fun << is_pass_through_member_candidate.pt_function; pt_param << is_pass_through_member_candidate.input;
places << is_pass_through_member_candidate.pos;
@@
store_positions(fun, typ, pt_param, places, tag)

@finalize:python@
if result:
result.insert(0, delimiter.join(("function", '"data type"', "parameter",
'"contained in"', '"source file"', "line", "column")))
print("\r\n".join(result))
else:
sys.stderr.write("No result for this analysis!\n")

Regards,
Markus