Skip to contentSkip to conversation

Query: Crash is coming during /prod/PID/stat and do_exit of same task

DISCUSSION Kohli, Gaurav 2 participants 4 messages Jan 9, 2018 – Jan 15, 2018 Arm Qualcomm mailing list

lore
Share & tools
lore
#0 lore raw

HI ,

We are seeing crash in do_task_stat while accessing stack pointer, It seems same task has already completed do_exit call. So it seems a race between them:

Below is the crash trace:
49750.534377] Kernel BUG at ffffff8e7a4c53a8 [verbose debug info unavailable]

[49750.534394] task: ffffffe7b4475580 task.stack: ffffffe7a5f0c000
[49750.534400] PC is at do_task_stat+0x740/0x908
[49750.534402] LR is at do_task_stat+0xa4/0x908
[49750.534403] pc : [<ffffff8e7a4c53a8>] lr : [<ffffff8e7a4c4d0c>] pstate: 80400145
[49750.534404] sp : ffffffe7a5f0fbd0

and here is stack trace on that core:

-000|user_stack_pointer(inline)
-000|do_task_stat(

    |    m = 0xFFFFFFE7A5CD7380,
    |    ns = 0xFFFFFF8E7C43C748,
    |  ?,
    |    task = 0xFFFFFFE80D8C2280,
    |  ?)
    |  tty_pgrp = 0
    |  ppid = 2084696064
    |  sid = 0
    |  mm = 0xFFFFFFE7B4424140
    |  tcomm = (84, 9, 71, 122, 142, 255, 255, 255, 48, 253, 240, 165, 231, 255, 255, 255)
    |  flags = 18446743969119403392

-001|proc_tgid_stat(
    |    m = 0xFFFFFFE7A5CD7380,
    |  ?,

Below are task stats which shows , process completed the do_exit call:
struct task_struct.flags -x 0xFFFFFFE80D8C2280
  flags = 0x40870c

crash_64> struct task_struct.exit_code -x 0xFFFFFFE80D8C2280 exit_code = 0x6

   struct task_struct.state -x 0xFFFFFFE80D8C2280
  state = 0x40

In our build both patches are there ,
fs/proc: report eip/esp in /prod/PID/stat for coredumping

and also  task.state has already set PF_DUMPCORE as it got the sigabrt signal.

Regards
Gaurav

-- Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.

#1 lore raw

Hi John, Ingo

As still we are seeing race between do_task_stat and do_exit of task, Can't we have to put more strict check in case, if stack pointer is NULL in below code :

                if (permitted && (task->flags & PF_DUMPCORE)) { eip = KSTK_EIP(task);
                        esp = KSTK_ESP(task);
                }

Regards
Gaurav

Quoted 45 lines · Gaurav

HI ,

We are seeing crash in do_task_stat while accessing stack pointer, It seems same task has already completed do_exit call. So it seems a race between them:

Below is the crash trace:
49750.534377] Kernel BUG at ffffff8e7a4c53a8 [verbose debug info unavailable]

[49750.534394] task: ffffffe7b4475580 task.stack: ffffffe7a5f0c000
[49750.534400] PC is at do_task_stat+0x740/0x908
[49750.534402] LR is at do_task_stat+0xa4/0x908
[49750.534403] pc : [<ffffff8e7a4c53a8>] lr : [<ffffff8e7a4c4d0c>] pstate: 80400145
[49750.534404] sp : ffffffe7a5f0fbd0

and here is stack trace on that core:

-000|user_stack_pointer(inline)
-000|do_task_stat(

    |    m = 0xFFFFFFE7A5CD7380,
    |    ns = 0xFFFFFF8E7C43C748,
    |  ?,
    |    task = 0xFFFFFFE80D8C2280,
    |  ?)
    |  tty_pgrp = 0
    |  ppid = 2084696064
    |  sid = 0
    |  mm = 0xFFFFFFE7B4424140
    |  tcomm = (84, 9, 71, 122, 142, 255, 255, 255, 48, 253, 240, 165, 231, 255, 255, 255)
    |  flags = 18446743969119403392

-001|proc_tgid_stat(
    |    m = 0xFFFFFFE7A5CD7380,
    |  ?,

Below are task stats which shows , process completed the do_exit call:
struct task_struct.flags -x 0xFFFFFFE80D8C2280
  flags = 0x40870c

crash_64> struct task_struct.exit_code -x 0xFFFFFFE80D8C2280 exit_code = 0x6

   struct task_struct.state -x 0xFFFFFFE80D8C2280
  state = 0x40

In our build both patches are there ,
fs/proc: report eip/esp in /prod/PID/stat for coredumping

Quoted 8 lines

and also  task.state has already set PF_DUMPCORE as it got the sigabrt signal.

Regards
Gaurav

-- Qualcomm India Private Limited, on behalf of Qualcomm Innovation
Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.

-- Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.

#2 lore raw
In reply to Kohli, Gaurav

Hello Gaurav.

Quoted 38 lines · Gaurav

We are seeing crash in do_task_stat while accessing stack pointer, It seems same task has already completed do_exit call.
So it seems a race between them:

Below is the crash trace:
49750.534377] Kernel BUG at ffffff8e7a4c53a8 [verbose debug info unavailable]

[49750.534394] task: ffffffe7b4475580 task.stack: ffffffe7a5f0c000
[49750.534400] PC is at do_task_stat+0x740/0x908
[49750.534402] LR is at do_task_stat+0xa4/0x908
[49750.534403] pc : [<ffffff8e7a4c53a8>] lr : [<ffffff8e7a4c4d0c>]
pstate: 80400145
[49750.534404] sp : ffffffe7a5f0fbd0

and here is stack trace on that core:

-000|user_stack_pointer(inline)
-000|do_task_stat(

    |    m = 0xFFFFFFE7A5CD7380,
    |    ns = 0xFFFFFF8E7C43C748,
    |  ?,
    |    task = 0xFFFFFFE80D8C2280,
    |  ?)
    |  tty_pgrp = 0
    |  ppid = 2084696064
    |  sid = 0
    |  mm = 0xFFFFFFE7B4424140
    |  tcomm = (84, 9, 71, 122, 142, 255, 255, 255, 48, 253, 240, 165,

231, 255, 255, 255)
    |  flags = 18446743969119403392
-001|proc_tgid_stat(
    |    m = 0xFFFFFFE7A5CD7380,
    |  ?,

Below are task stats which shows , process completed the do_exit call:
struct task_struct.flags -x 0xFFFFFFE80D8C2280
  flags = 0x40870c

Quoted 5 lines

crash_64> struct task_struct.exit_code -x 0xFFFFFFE80D8C2280 exit_code = 0x6

   struct task_struct.state -x 0xFFFFFFE80D8C2280
  state = 0x40

I am confused why this task is in the TASK_PARKED state. What kind of task is this?

Quoted 5 lines

In our build both patches are there ,
fs/proc: report eip/esp in /prod/PID/stat for coredumping

and also  task.state has already set PF_DUMPCORE as it got the sigabrt signal.

John Ogness

#3 lore raw
Quoted 41 lines · John Ogness

Hello Gaurav.

Quoted 38 lines · Gaurav

We are seeing crash in do_task_stat while accessing stack pointer, It seems same task has already completed do_exit call.
So it seems a race between them:

Below is the crash trace:
49750.534377] Kernel BUG at ffffff8e7a4c53a8 [verbose debug info unavailable]

[49750.534394] task: ffffffe7b4475580 task.stack: ffffffe7a5f0c000
[49750.534400] PC is at do_task_stat+0x740/0x908
[49750.534402] LR is at do_task_stat+0xa4/0x908
[49750.534403] pc : [<ffffff8e7a4c53a8>] lr : [<ffffff8e7a4c4d0c>]
pstate: 80400145
[49750.534404] sp : ffffffe7a5f0fbd0

and here is stack trace on that core:

-000|user_stack_pointer(inline)
-000|do_task_stat(

     |    m = 0xFFFFFFE7A5CD7380,
     |    ns = 0xFFFFFF8E7C43C748,
     |  ?,
     |    task = 0xFFFFFFE80D8C2280,
     |  ?)
     |  tty_pgrp = 0
     |  ppid = 2084696064
     |  sid = 0
     |  mm = 0xFFFFFFE7B4424140
     |  tcomm = (84, 9, 71, 122, 142, 255, 255, 255, 48, 253, 240, 165,

231, 255, 255, 255)
    |  flags = 18446743969119403392
-001|proc_tgid_stat(
    |    m = 0xFFFFFFE7A5CD7380,
    |  ?,

Below are task stats which shows , process completed the do_exit call:
struct task_struct.flags -x 0xFFFFFFE80D8C2280
  flags = 0x40870c

Quoted 7 lines
Quoted 5 lines

crash_64> struct task_struct.exit_code -x 0xFFFFFFE80D8C2280
  exit_code = 0x6

   struct task_struct.state -x 0xFFFFFFE80D8C2280 state = 0x40

I am confused why this task is in the TASK_PARKED state. What kind of task is this?

Hi John,

This is android HAL layer service and also before bug, i am seeing lot of service exited in logs also, although not seeing for this pid 6807

.452202:   <2> init: starting service 'limits-hal-1-0'...

 49749.460039:   <2> init: property_set("ro.boottime.limits-hal-1-0", "61591320967789") failed: property already set

 49749.607496:   <6> sh (2422): drop_caches: 3

 49750.281635:   <6> sh (2422): drop_caches: 3

 49750.533853:   <2> init: Untracked pid 6811 exited with status 0

And why it is parked , that is not clear as state is already updated of task.

Regards

Gaurav

Quoted 8 lines
Quoted 6 lines

In our build both patches are there ,
fs/proc: report eip/esp in /prod/PID/stat for coredumping

and also  task.state has already set PF_DUMPCORE as it got the sigabrt signal.

John Ogness

Signature · 2 lines
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project.