Skip to contentSkip to conversation

cfq-iosched.c: panic in cfq_dispatch_requests

DISCUSSION Lina Lu 2 participants 5 messages Mar 20, 2011 – Mar 22, 2011

lore
Share & tools
lore

Lina Lu

author
#0 lore raw

Hi Vivek,

I did some test with blkio weight policy these days. The kernel panic once in a while.

Quoted 1 line

From the calltrace, It 's in func cfq_dispatch_requests(+0x058D).

And I caught a divide by zero error with system monitor tools.

The panic happens when I first set 100 weight to two IO pids, then change the weight to 200, and begin to perform IO. The kernel panic accurately when the IO begin.

I try to find the divide 0 code in cfq_dispatch_requests. This function call the cfq_select_queue to select a queue, and cfq_select_queue call cfq_choose_cfqg, then choose_service_tree(). There is a expression of division in choose_service_tree.

        slice = group_slice * count /
                max_t(unsigned, cfqg->busy_queues_avg[cfqd->serving_prio],
                      cfq_group_busy_queues_wl(cfqd->serving_prio, cfqd, cfqg));

I don't know whether this code divide 0. Can you take a look of this issue?

Thanks
Lina

#1 lore raw
Quoted 7 lines · Lina Lu

Hi Vivek,

I did some test with blkio weight policy these days. The kernel panic once in a while.

Quoted 1 line

From the calltrace, It 's in func cfq_dispatch_requests(+0x058D).

And I caught a divide by zero error with system monitor tools.

Hi Lina,

Can you please paste the full backtrace here.

Quoted 4 lines

The panic happens when I first set 100 weight to two IO pids, then change the weight to 200, and begin to perform IO. The kernel panic accurately when the IO begin.

So you notice this only if weights are changed otherwise not?

Recently Justin fixed a bug in CFQ where crash can happen when weights are changed. This fix in Jens's block tree. Can you please apply this patch and see if it resolves your issue too.

commit 8184f93eced1e304721c2a55c00d87d5a14f8907

Author: Justin TerAvest <[email protected]>
Date:   Thu Mar 17 16:12:36 2011 +0100

cfq-iosched: Don't update group weights when on service tree

Thanks
Vivek

author
#2 lore raw
In reply to Lina Lu
Quoted 11 lines · Vivek Goyal
Quoted 7 lines · Lina Lu

Hi Vivek,

I did some test with blkio weight policy these days. The kernel panic once in a while.

From the calltrace, It 's in func cfq_dispatch_requests(+0x058D). And I caught a divide by zero error with system monitor tools.

Hi Lina,

Quoted 8 lines

Can you please paste the full backtrace here.

Quoted 4 lines

The panic happens when I first set 100 weight to two IO pids, then change the weight to 200, and begin to perform IO. The kernel panic accurately when the IO begin.

So you notice this only if weights are changed otherwise not?

Hi Vivek

Thanks for this patch! I did objdump and find place that divide by zero. It's in cfq_group_slice function, the st->total_weight is zero in my issue. As described in the patch, this is because I change the weight when the group on service tree.

Thanks
Lina

Quoted 5 lines

Recently Justin fixed a bug in CFQ where crash can happen when weights are changed. This fix in Jens's block tree. Can you please apply this patch and see if it resolves your issue too.

Quoted 8 lines

commit 8184f93eced1e304721c2a55c00d87d5a14f8907

Author: Justin TerAvest <[email protected]>
Date:   Thu Mar 17 16:12:36 2011 +0100

cfq-iosched: Don't update group weights when on service tree

Thanks
Vivek

@

#3 lore raw
Quoted 19 lines · Lina Lu
Quoted 18 lines · Vivek Goyal
Quoted 7 lines · Lina Lu

Hi Vivek,

I did some test with blkio weight policy these days. The kernel panic once in a while.

From the calltrace, It 's in func cfq_dispatch_requests(+0x058D). And I caught a divide by zero error with system monitor tools.

Hi Lina,

Can you please paste the full backtrace here.

Quoted 4 lines

The panic happens when I first set 100 weight to two IO pids, then change the weight to 200, and begin to perform IO. The kernel panic accurately when the IO begin.

Quoted 8 lines
Quoted 1 line

So you notice this only if weights are changed otherwise not?

Hi Vivek

Thanks for this patch! I did objdump and find place that divide by zero. It's in cfq_group_slice function, the st->total_weight is zero in my issue. As described in the patch, this is because I change the weight when the group on service tree.

Ok, so this patch did solve the issue you are facing?

Thanks
Vivek

author
#4 lore raw
In reply to Lina Lu
Quoted 22 lines · Vivek Goyal
Quoted 21 lines · Lina Lu
Quoted 19 lines · Vivek Goyal
Quoted 7 lines · Lina Lu

Hi Vivek,

I did some test with blkio weight policy these days. The kernel panic once in a while.

From the calltrace, It 's in func cfq_dispatch_requests(+0x058D). And I caught a divide by zero error with system monitor tools.

Hi Lina,

Can you please paste the full backtrace here.

Quoted 4 lines

The panic happens when I first set 100 weight to two IO pids, then change the weight to 200, and begin to perform IO. The kernel panic accurately when the IO begin.

So you notice this only if weights are changed otherwise not?

Quoted 9 lines
Quoted 6 lines

Hi Vivek

Thanks for this patch! I did objdump and find place that divide by zero. It's in cfq_group_slice function, the st->total_weight is zero in my issue. As described in the patch, this is because I change the weight when the group on service tree.

Ok, so this patch did solve the issue you are facing?

Yes, this patch did solve the issue.

And I will do more test these days, if there is any other problem, I will keep you know.

Thanks
Lina