Skip to contentSkip to conversation

CFQ is broken for CONFIG_BLK_CGROUP=y, CFQ_GROUP_IOSCHED=n

DISCUSSION Dmitry Monakhov 3 participants 6 messages Apr 28, 2010 Block layer

lore
Share & tools
lore
#0 lore raw

I've had an oops on kernel boot due to NULL pointer deference linux-2.6-block/for-next HEAD:7eaed1226ab411ee5dc8c34fc0d8034e4c98e3c6 I've enabled CONFIG_BLK_CGROUP, but not CFQ_GROUP_IOSCHED
In this case cfq_ref_get_cfqg() defined as
static inline struct cfq_group *cfq_ref_get_cfqg(struct cfq_group *cfqg)

{
        return NULL;
}

So following call trace is simply NOOP cfq_set_request()
rq->elevator_private3 = cfq_ref_get_cfqg(cfqq->cfqg);

Which later result in OOPS on bio insertion cfq_insert_request
-> blkiocg_update_io_add_stats(&(RQ_CFQG(rq))->blkg,...)

    -> spin_lock_irqsave(&blkg->stats_lock, flags);

Bad commit.

Quoted 1 line

From 7f1dc8a2d2f45fc557b27fd56115338b1d34fc24 Mon Sep 17 00:00:00 2001

From: Vivek Goyal <[email protected]>
Date: Wed, 21 Apr 2010 17:44:16 +0200
Subject: [PATCH] blkio: Fix blkio crash during rq stat update

#1 lore raw
Quoted 16 lines · Dmitry Monakhov

I've had an oops on kernel boot due to NULL pointer deference linux-2.6-block/for-next HEAD:7eaed1226ab411ee5dc8c34fc0d8034e4c98e3c6 I've enabled CONFIG_BLK_CGROUP, but not CFQ_GROUP_IOSCHED
In this case cfq_ref_get_cfqg() defined as
static inline struct cfq_group *cfq_ref_get_cfqg(struct cfq_group *cfqg)

{
        return NULL;
}

So following call trace is simply NOOP cfq_set_request()
rq->elevator_private3 = cfq_ref_get_cfqg(cfqq->cfqg);

Which later result in OOPS on bio insertion cfq_insert_request
-> blkiocg_update_io_add_stats(&(RQ_CFQG(rq))->blkg,...)

Quoted 7 lines
    -> spin_lock_irqsave(&blkg->stats_lock, flags);

Bad commit.

Quoted 1 line

From 7f1dc8a2d2f45fc557b27fd56115338b1d34fc24 Mon Sep 17 00:00:00 2001

From: Vivek Goyal <[email protected]>
Date: Wed, 21 Apr 2010 17:44:16 +0200
Subject: [PATCH] blkio: Fix blkio crash during rq stat update

Hi Dmitry,

Thanks for reporting the issue. I am cooking up a patch now.

Vivek

#2 lore raw
In reply to Dmitry Monakhov
Quoted 16 lines · Dmitry Monakhov

I've had an oops on kernel boot due to NULL pointer deference linux-2.6-block/for-next HEAD:7eaed1226ab411ee5dc8c34fc0d8034e4c98e3c6 I've enabled CONFIG_BLK_CGROUP, but not CFQ_GROUP_IOSCHED
In this case cfq_ref_get_cfqg() defined as
static inline struct cfq_group *cfq_ref_get_cfqg(struct cfq_group *cfqg)

{
        return NULL;
}

So following call trace is simply NOOP cfq_set_request()
rq->elevator_private3 = cfq_ref_get_cfqg(cfqq->cfqg);

Which later result in OOPS on bio insertion cfq_insert_request
-> blkiocg_update_io_add_stats(&(RQ_CFQG(rq))->blkg,...)

Quoted 7 lines
    -> spin_lock_irqsave(&blkg->stats_lock, flags);

Bad commit.

Quoted 1 line

From 7f1dc8a2d2f45fc557b27fd56115338b1d34fc24 Mon Sep 17 00:00:00 2001

From: Vivek Goyal <[email protected]>
Date: Wed, 21 Apr 2010 17:44:16 +0200
Subject: [PATCH] blkio: Fix blkio crash during rq stat update

Dmitry, this patch should fix the issue. Can you please give it a try.

Jens, I know you don't like this form of cfq_ref_get_cfqg(), but this seems to be the simplest solution to fix it.

cfq-iosched: fix cfq crash with CFQ_GROUP_IOSCHED=n

Dmitry reported an oops with CFQ when booted with BLK_CGROUP=y and CFQ_GROUP_IOSCHED=n. This patch fixes it.

We maintain a root group even if group ioscheduling is not enabled. Hence don't store NULL in elevator_private3 in that case.

---
 block/cfq-iosched.c |   11 ++++-------
 1 files changed, 4 insertions(+), 7 deletions(-)
block/cfq-iosched.c +4 -7
index 286008c..5aa5364 100644
@@ -1002,10 +1002,9 @@ static struct cfq_group *cfq_get_cfqg(struct cfq_data *cfqd, int create)
 	return cfqg;
 }
 
-static inline struct cfq_group *cfq_ref_get_cfqg(struct cfq_group *cfqg)
+static inline void cfq_ref_get_cfqg(struct cfq_group *cfqg)
 {
 	atomic_inc(&cfqg->ref);
-	return cfqg;
 }
 
 static void cfq_link_cfqq_cfqg(struct cfq_queue *cfqq, struct cfq_group *cfqg)
@@ -1092,10 +1091,7 @@ static struct cfq_group *cfq_get_cfqg(struct cfq_data *cfqd, int create)
 	return &cfqd->root_group;
 }
 
-static inline struct cfq_group *cfq_ref_get_cfqg(struct cfq_group *cfqg)
-{
-	return NULL;
-}
+static inline void cfq_ref_get_cfqg(struct cfq_group *cfqg) {}
 
 static inline void
 cfq_link_cfqq_cfqg(struct cfq_queue *cfqq, struct cfq_group *cfqg) {
@@ -3574,7 +3570,8 @@ new_queue:
 
 	rq->elevator_private = cic;
 	rq->elevator_private2 = cfqq;
-	rq->elevator_private3 = cfq_ref_get_cfqg(cfqq->cfqg);
+	rq->elevator_private3 = cfqq->cfqg;
+	cfq_ref_get_cfqg(cfqq->cfqg);
 	return 0;
 
 queue_fail:
Signature · 1 line
1.6.2.5
#3 lore raw
Quoted 19 lines · Vivek Goyal
Quoted 18 lines · Dmitry Monakhov

I've had an oops on kernel boot due to NULL pointer deference linux-2.6-block/for-next HEAD:7eaed1226ab411ee5dc8c34fc0d8034e4c98e3c6 I've enabled CONFIG_BLK_CGROUP, but not CFQ_GROUP_IOSCHED
In this case cfq_ref_get_cfqg() defined as
static inline struct cfq_group *cfq_ref_get_cfqg(struct cfq_group *cfqg)

{
        return NULL;
}

So following call trace is simply NOOP cfq_set_request()
rq->elevator_private3 = cfq_ref_get_cfqg(cfqq->cfqg);

Which later result in OOPS on bio insertion cfq_insert_request
-> blkiocg_update_io_add_stats(&(RQ_CFQG(rq))->blkg,...)

    -> spin_lock_irqsave(&blkg->stats_lock, flags);
Quoted 7 lines
Quoted 5 lines

Bad commit.

Quoted 1 line

From 7f1dc8a2d2f45fc557b27fd56115338b1d34fc24 Mon Sep 17 00:00:00 2001

From: Vivek Goyal <[email protected]>
Date: Wed, 21 Apr 2010 17:44:16 +0200
Subject: [PATCH] blkio: Fix blkio crash during rq stat update

Dmitry, this patch should fix the issue. Can you please give it a try.

Yes. the issue is gone.

Quoted 3 lines

Jens, I know you don't like this form of cfq_ref_get_cfqg(), but this seems to be the simplest solution to fix it.

IMHO following fix is simpler, and cleaner.

block/cfq-iosched.c +1 -1
index 286008c..0f3eb70 100644
@@ -1094,7 +1094,7 @@ static struct cfq_group *cfq_get_cfqg(struct cfq_data *cfqd, int create)
 
 static inline struct cfq_group *cfq_ref_get_cfqg(struct cfq_group *cfqg)
 {
-	return NULL;
+	return cfqg;
 }
 
 static inline void
#4 lore raw
Quoted 39 lines · Dmitry Monakhov
Quoted 19 lines · Vivek Goyal
Quoted 18 lines · Dmitry Monakhov

I've had an oops on kernel boot due to NULL pointer deference linux-2.6-block/for-next HEAD:7eaed1226ab411ee5dc8c34fc0d8034e4c98e3c6 I've enabled CONFIG_BLK_CGROUP, but not CFQ_GROUP_IOSCHED
In this case cfq_ref_get_cfqg() defined as
static inline struct cfq_group *cfq_ref_get_cfqg(struct cfq_group *cfqg)

{
        return NULL;
}

So following call trace is simply NOOP cfq_set_request()
rq->elevator_private3 = cfq_ref_get_cfqg(cfqq->cfqg);

Which later result in OOPS on bio insertion cfq_insert_request
-> blkiocg_update_io_add_stats(&(RQ_CFQG(rq))->blkg,...)

    -> spin_lock_irqsave(&blkg->stats_lock, flags);
Quoted 7 lines
Quoted 5 lines

Bad commit.

Quoted 1 line

From 7f1dc8a2d2f45fc557b27fd56115338b1d34fc24 Mon Sep 17 00:00:00 2001

From: Vivek Goyal <[email protected]>
Date: Wed, 21 Apr 2010 17:44:16 +0200
Subject: [PATCH] blkio: Fix blkio crash during rq stat update

Dmitry, this patch should fix the issue. Can you please give it a try.

Yes. the issue is gone.

Quoted 3 lines

Jens, I know you don't like this form of cfq_ref_get_cfqg(), but this seems to be the simplest solution to fix it.

IMHO following fix is simpler, and cleaner.

block/cfq-iosched.c +0 -0
index 286008c..0f3eb70 100644

block/cfq-iosched.c static struct cfq_group *cfq_get_cfqg(struct cfq_data *cfqd, int create) · line 1098

Quoted 6 lines
block/cfq-iosched.c +1 -1
@@ -1094,7 +1094,7 @@ static struct cfq_group *cfq_get_cfqg(struct cfq_data *cfqd, int create)
 
 static inline struct cfq_group *cfq_ref_get_cfqg(struct cfq_group *cfqg)
 {
-	return NULL;
+	return cfqg;
 }

Thanks Dmitry. I like it. simpler.

Vivek

#5 lore raw
In reply to Dmitry Monakhov
Quoted 27 lines · Dmitry Monakhov
Quoted 25 lines · Vivek Goyal
Quoted 23 lines · Dmitry Monakhov

I've had an oops on kernel boot due to NULL pointer deference linux-2.6-block/for-next HEAD:7eaed1226ab411ee5dc8c34fc0d8034e4c98e3c6 I've enabled CONFIG_BLK_CGROUP, but not CFQ_GROUP_IOSCHED
In this case cfq_ref_get_cfqg() defined as
static inline struct cfq_group *cfq_ref_get_cfqg(struct cfq_group *cfqg)

{
        return NULL;
}

So following call trace is simply NOOP cfq_set_request()
rq->elevator_private3 = cfq_ref_get_cfqg(cfqq->cfqg);

Which later result in OOPS on bio insertion cfq_insert_request
-> blkiocg_update_io_add_stats(&(RQ_CFQG(rq))->blkg,...)

    -> spin_lock_irqsave(&blkg->stats_lock, flags);

Bad commit.

Quoted 1 line

From 7f1dc8a2d2f45fc557b27fd56115338b1d34fc24 Mon Sep 17 00:00:00 2001

From: Vivek Goyal <[email protected]>
Date: Wed, 21 Apr 2010 17:44:16 +0200
Subject: [PATCH] blkio: Fix blkio crash during rq stat update

Quoted 6 lines
Quoted 1 line

Dmitry, this patch should fix the issue. Can you please give it a try.

Yes. the issue is gone.

Quoted 3 lines

Jens, I know you don't like this form of cfq_ref_get_cfqg(), but this seems to be the simplest solution to fix it.

IMHO following fix is simpler, and cleaner.

Agreed, that's much better. I'll add it, thanks.

Signature · 1 line
Jens Axboe