Skip to contentSkip to conversation

loop: queue_lock NULL pointer derefence in blk_throtl_exit

PATCH Applied Sergey Senozhatsky 3 participants 8 messages Jan 14, 2011 – Jan 21, 2011 Block layer

lore
Share & tools
lore

Sergey Senozhatsky

author
#0 lore raw

PATCH 1/1 loop: queue_lock NULL pointer derefence in blk_throtl_exit

Performing

$ sudo mount -o loop -o umask=0 /dev/sdb1 /mnt/
mount: wrong fs type, bad option, bad superblock on /dev/loop0,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so
$ sudo modprobe -r loop

results in oops:

 BUG: unable to handle kernel NULL pointer dereference at 0000000000000004
 IP: [<ffffffff812479d4>] do_raw_spin_lock+0x14/0x122
 Process modprobe (pid: 6189, threadinfo ffff88009a898000, task ffff880154a88000)
 Call Trace:
  [<ffffffff81486788>] _raw_spin_lock_irq+0x4a/0x51
  [<ffffffff8123404b>] ? blk_throtl_exit+0x3b/0xa0
  [<ffffffff8105b120>] ? cancel_delayed_work_sync+0xd/0xf
  [<ffffffff8123404b>] blk_throtl_exit+0x3b/0xa0
  [<ffffffff81229bc8>] blk_release_queue+0x21/0x65
  [<ffffffff8123bb06>] kobject_release+0x51/0x66
  [<ffffffff8123bab5>] ? kobject_release+0x0/0x66
  [<ffffffff8123ce1e>] kref_put+0x43/0x4d
  [<ffffffff8123ba27>] kobject_put+0x47/0x4b
  [<ffffffff8122717c>] blk_cleanup_queue+0x56/0x5b
  [<ffffffffa01c3824>] loop_exit+0x68/0x844 [loop]
  [<ffffffff8107cccc>] sys_delete_module+0x1e8/0x25b
  [<ffffffff814864c9>] ? trace_hardirqs_on_thunk+0x3a/0x3f
  [<ffffffff81002112>] system_call_fastpath+0x16/0x1b

because of an attempt to acquire NULL queue_lock. I added the same lines as in blk_queue_make_request - `fall back to embedded per-queue lock'.

Signed-off-by Sergey Senozhatsky <[email protected]>

---

 drivers/block/loop.c |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)
drivers/block/loop.c +3 -0
index 44e18c0..49e6a54 100644
@@ -1641,6 +1641,9 @@ out:
 
 static void loop_free(struct loop_device *lo)
 {
+	if (!lo->lo_queue->queue_lock)
+		lo->lo_queue->queue_lock = &lo->lo_queue->__queue_lock;
+
 	blk_cleanup_queue(lo->lo_queue);
 	put_disk(lo->lo_disk);
 	list_del(&lo->lo_list);
Applied
#1 lore raw
Quoted 31 lines · Sergey Senozhatsky

Performing

$ sudo mount -o loop -o umask=0 /dev/sdb1 /mnt/
mount: wrong fs type, bad option, bad superblock on /dev/loop0,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so
$ sudo modprobe -r loop

results in oops:

 BUG: unable to handle kernel NULL pointer dereference at 0000000000000004
 IP: [<ffffffff812479d4>] do_raw_spin_lock+0x14/0x122
 Process modprobe (pid: 6189, threadinfo ffff88009a898000, task ffff880154a88000)
 Call Trace:
  [<ffffffff81486788>] _raw_spin_lock_irq+0x4a/0x51
  [<ffffffff8123404b>] ? blk_throtl_exit+0x3b/0xa0
  [<ffffffff8105b120>] ? cancel_delayed_work_sync+0xd/0xf
  [<ffffffff8123404b>] blk_throtl_exit+0x3b/0xa0
  [<ffffffff81229bc8>] blk_release_queue+0x21/0x65
  [<ffffffff8123bb06>] kobject_release+0x51/0x66
  [<ffffffff8123bab5>] ? kobject_release+0x0/0x66
  [<ffffffff8123ce1e>] kref_put+0x43/0x4d
  [<ffffffff8123ba27>] kobject_put+0x47/0x4b
  [<ffffffff8122717c>] blk_cleanup_queue+0x56/0x5b
  [<ffffffffa01c3824>] loop_exit+0x68/0x844 [loop]
  [<ffffffff8107cccc>] sys_delete_module+0x1e8/0x25b
  [<ffffffff814864c9>] ? trace_hardirqs_on_thunk+0x3a/0x3f
  [<ffffffff81002112>] system_call_fastpath+0x16/0x1b
Quoted 3 lines

because of an attempt to acquire NULL queue_lock. I added the same lines as in blk_queue_make_request - `fall back to embedded per-queue lock'.

Queued up for 2.6.38, thanks.

Signature · 1 line
Jens Axboe
#2 lore raw
In reply to Sergey Senozhatsky
Quoted 47 lines · Sergey Senozhatsky

Performing

$ sudo mount -o loop -o umask=0 /dev/sdb1 /mnt/
mount: wrong fs type, bad option, bad superblock on /dev/loop0,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so
$ sudo modprobe -r loop

results in oops:

 BUG: unable to handle kernel NULL pointer dereference at 0000000000000004
 IP: [<ffffffff812479d4>] do_raw_spin_lock+0x14/0x122
 Process modprobe (pid: 6189, threadinfo ffff88009a898000, task ffff880154a88000)
 Call Trace:
  [<ffffffff81486788>] _raw_spin_lock_irq+0x4a/0x51
  [<ffffffff8123404b>] ? blk_throtl_exit+0x3b/0xa0
  [<ffffffff8105b120>] ? cancel_delayed_work_sync+0xd/0xf
  [<ffffffff8123404b>] blk_throtl_exit+0x3b/0xa0
  [<ffffffff81229bc8>] blk_release_queue+0x21/0x65
  [<ffffffff8123bb06>] kobject_release+0x51/0x66
  [<ffffffff8123bab5>] ? kobject_release+0x0/0x66
  [<ffffffff8123ce1e>] kref_put+0x43/0x4d
  [<ffffffff8123ba27>] kobject_put+0x47/0x4b
  [<ffffffff8122717c>] blk_cleanup_queue+0x56/0x5b
  [<ffffffffa01c3824>] loop_exit+0x68/0x844 [loop]
  [<ffffffff8107cccc>] sys_delete_module+0x1e8/0x25b
  [<ffffffff814864c9>] ? trace_hardirqs_on_thunk+0x3a/0x3f
  [<ffffffff81002112>] system_call_fastpath+0x16/0x1b

because of an attempt to acquire NULL queue_lock. I added the same lines as in blk_queue_make_request - `fall back to embedded per-queue lock'.

Signed-off-by Sergey Senozhatsky <[email protected]>

---

 drivers/block/loop.c |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)
drivers/block/loop.c +0 -0
index 44e18c0..49e6a54 100644

drivers/block/loop.c out: · line 1649

Quoted 9 lines
drivers/block/loop.c +3 -0
@@ -1641,6 +1641,9 @@ out:
 
 static void loop_free(struct loop_device *lo)
 {
+	if (!lo->lo_queue->queue_lock)
+		lo->lo_queue->queue_lock = &lo->lo_queue->__queue_lock;
+
 	blk_cleanup_queue(lo->lo_queue);
 	put_disk(lo->lo_disk);
 	list_del(&lo->lo_list);

IIUC, the issue here seems to be that we allocated a request queue but never initiliazed it and then called blk_cleanup_queue() on that.

I noticed in loop driver there is another blk_cleanup_queue() call in loop_alloc() if memory allocation fails. There also we will run into similar issue.

IMHO, a better place to fix this might be blk_release_queue() where we can default to q->__spin_lock if we are cleaning up an uninitialized queue. That way we shall not have to fix inidividual drivers. And it also does not sound right to expect a driver to make sure queue lock is initialized before it calls blk_cleanup_queue() on the queue.

Thanks
Vivek

author
#3 lore raw

Re: [PATCH] loop: queue_lock NULL pointer derefence in blk_throtl_exit (v2)

Performing

$ sudo mount -o loop -o umask=0 /dev/sdb1 /mnt/
mount: wrong fs type, bad option, bad superblock on /dev/loop0,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so
$ sudo modprobe -r loop

results in oops:

 BUG: unable to handle kernel NULL pointer dereference at 0000000000000004
 IP: [<ffffffff812479d4>] do_raw_spin_lock+0x14/0x122
 Process modprobe (pid: 6189, threadinfo ffff88009a898000, task ffff880154a88000)
 Call Trace:
  [<ffffffff81486788>] _raw_spin_lock_irq+0x4a/0x51
  [<ffffffff8123404b>] ? blk_throtl_exit+0x3b/0xa0
  [<ffffffff8105b120>] ? cancel_delayed_work_sync+0xd/0xf
  [<ffffffff8123404b>] blk_throtl_exit+0x3b/0xa0
  [<ffffffff81229bc8>] blk_release_queue+0x21/0x65
  [<ffffffff8123bb06>] kobject_release+0x51/0x66
  [<ffffffff8123bab5>] ? kobject_release+0x0/0x66
  [<ffffffff8123ce1e>] kref_put+0x43/0x4d
  [<ffffffff8123ba27>] kobject_put+0x47/0x4b
  [<ffffffff8122717c>] blk_cleanup_queue+0x56/0x5b
  [<ffffffffa01c3824>] loop_exit+0x68/0x844 [loop]
  [<ffffffff8107cccc>] sys_delete_module+0x1e8/0x25b
  [<ffffffff814864c9>] ? trace_hardirqs_on_thunk+0x3a/0x3f
  [<ffffffff81002112>] system_call_fastpath+0x16/0x1b

because of an attempt to acquire NULL queue_lock. I added the same lines as in blk_queue_make_request - `fall back to embedded per-queue lock'.

v2: According to comment by Vivek Goyal, queue_lock NULL check and fix moved out from loop driver code to blk_cleanup_queue, which is more general approach.

Signed-off-by Sergey Senozhatsky <[email protected]>

---

 block/blk-core.c |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)
block/blk-core.c +4 -0
index 2f4002f..45073ab 100644
@@ -458,6 +458,10 @@ void blk_put_queue(struct request_queue *q)
 
 void blk_cleanup_queue(struct request_queue *q)
 {
+	/* fall back to our embedded per-queue locks */
+	if (!q->queue_lock)
+		q->queue_lock = &q->__queue_lock;
+
 	/*
 	 * We know we have process context here, so we can be a little
 	 * cautious and ensure that pending block actions on this device
#4 lore raw

Re: [PATCH] loop: queue_lock NULL pointer derefence in blk_throtl_exit (v2)

Quoted 51 lines · Sergey Senozhatsky

Performing

$ sudo mount -o loop -o umask=0 /dev/sdb1 /mnt/
mount: wrong fs type, bad option, bad superblock on /dev/loop0,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so
$ sudo modprobe -r loop

results in oops:

 BUG: unable to handle kernel NULL pointer dereference at 0000000000000004
 IP: [<ffffffff812479d4>] do_raw_spin_lock+0x14/0x122
 Process modprobe (pid: 6189, threadinfo ffff88009a898000, task ffff880154a88000)
 Call Trace:
  [<ffffffff81486788>] _raw_spin_lock_irq+0x4a/0x51
  [<ffffffff8123404b>] ? blk_throtl_exit+0x3b/0xa0
  [<ffffffff8105b120>] ? cancel_delayed_work_sync+0xd/0xf
  [<ffffffff8123404b>] blk_throtl_exit+0x3b/0xa0
  [<ffffffff81229bc8>] blk_release_queue+0x21/0x65
  [<ffffffff8123bb06>] kobject_release+0x51/0x66
  [<ffffffff8123bab5>] ? kobject_release+0x0/0x66
  [<ffffffff8123ce1e>] kref_put+0x43/0x4d
  [<ffffffff8123ba27>] kobject_put+0x47/0x4b
  [<ffffffff8122717c>] blk_cleanup_queue+0x56/0x5b
  [<ffffffffa01c3824>] loop_exit+0x68/0x844 [loop]
  [<ffffffff8107cccc>] sys_delete_module+0x1e8/0x25b
  [<ffffffff814864c9>] ? trace_hardirqs_on_thunk+0x3a/0x3f
  [<ffffffff81002112>] system_call_fastpath+0x16/0x1b

because of an attempt to acquire NULL queue_lock. I added the same lines as in blk_queue_make_request - `fall back to embedded per-queue lock'.

v2: According to comment by Vivek Goyal, queue_lock NULL check and fix moved out from loop driver code to blk_cleanup_queue, which is more general approach.

Signed-off-by Sergey Senozhatsky <[email protected]>

---

 block/blk-core.c |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)
block/blk-core.c +0 -0
index 2f4002f..45073ab 100644

block/blk-core.c void blk_put_queue(struct request_queue *q) · line 464

Quoted 7 lines
block/blk-core.c +4 -0
@@ -458,6 +458,10 @@ void blk_put_queue(struct request_queue *q)
 
 void blk_cleanup_queue(struct request_queue *q)
 {
+	/* fall back to our embedded per-queue locks */
+	if (!q->queue_lock)
+		q->queue_lock = &q->__queue_lock;
+

Hi Sergey,

Can we expand a little bit on comment that why do we need to have q->queue_lock initialized here now. Basically in the past nobody tried to take q->queue_lock in blk_cleanup_queue() path hence things just worked. Now blk throttling code is new and it takes q->queue_lock hence we run into issues. This could be true for some other future code too.

Secondly currently blk throttle code seems to be the only user dependent on this lock initialization. So it might make sense to move this code closer to the actual call and blk_release_queue() might be even better place to do it atleast for now.

Thanks
Vivek

author
#5 lore raw

Re: [PATCH] loop: queue_lock NULL pointer derefence in blk_throtl_exit (v2)

Quoted 2 lines · Vivek Goyal

Hi Sergey,

Hi Vivek,

Quoted 11 lines

Can we expand a little bit on comment that why do we need to have q->queue_lock initialized here now. Basically in the past nobody tried to take q->queue_lock in blk_cleanup_queue() path hence things just worked. Now blk throttling code is new and it takes q->queue_lock hence we run into issues. This could be true for some other future code too.

Secondly currently blk throttle code seems to be the only user dependent on this lock initialization. So it might make sense to move this code closer to the actual call and blk_release_queue() might be even better place to do it atleast for now.

I'm afraid it's not safe to move NULL-check-and-fix out from blk_cleanup_queue, since we're performing elevator_exit(q->elevator) call, which may depend on q->queue_lock. For example, (./cfq-iosched.c: .elevator_exit_fn = cfq_exit_queue) cfq_exit_queue uses q->queue_lock.

Sergey

#6 lore raw

Re: [PATCH] loop: queue_lock NULL pointer derefence in blk_throtl_exit (v2)

Quoted 18 lines · Sergey Senozhatsky
Quoted 2 lines · Vivek Goyal

Hi Sergey,

Hi Vivek,

Quoted 11 lines

Can we expand a little bit on comment that why do we need to have q->queue_lock initialized here now. Basically in the past nobody tried to take q->queue_lock in blk_cleanup_queue() path hence things just worked. Now blk throttling code is new and it takes q->queue_lock hence we run into issues. This could be true for some other future code too.

Secondly currently blk throttle code seems to be the only user dependent on this lock initialization. So it might make sense to move this code closer to the actual call and blk_release_queue() might be even better place to do it atleast for now.

Quoted 4 lines

I'm afraid it's not safe to move NULL-check-and-fix out from blk_cleanup_queue, since we're performing elevator_exit(q->elevator) call, which may depend on q->queue_lock. For example, (./cfq-iosched.c: .elevator_exit_fn = cfq_exit_queue) cfq_exit_queue uses q->queue_lock.

ok, actually cfq_exit_queue() will be called only if elevator has been initilialzed and if elevator has been initialialized that means q->queue_lock also has been initilalized so that should not be a problem.

I am not very particular about this thing. So I will leave it to you.

Thanks
Vivek

author
#7 lore raw

PATCH 1/1 loop: queue_lock NULL pointer derefence in blk_throtl_exit (v3)

Performing

$ sudo mount -o loop -o umask=0 /dev/sdb1 /mnt/
mount: wrong fs type, bad option, bad superblock on /dev/loop0,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so
$ sudo modprobe -r loop

results in oops:

 BUG: unable to handle kernel NULL pointer dereference at 0000000000000004
 IP: [<ffffffff812479d4>] do_raw_spin_lock+0x14/0x122
 Process modprobe (pid: 6189, threadinfo ffff88009a898000, task
 ffff880154a88000)                                                        
 Call Trace:                                         
  [<ffffffff81486788>] _raw_spin_lock_irq+0x4a/0x51                         
  [<ffffffff8123404b>] ? blk_throtl_exit+0x3b/0xa0
  [<ffffffff8105b120>] ? cancel_delayed_work_sync+0xd/0xf
  [<ffffffff8123404b>] blk_throtl_exit+0x3b/0xa0   
  [<ffffffff81229bc8>] blk_release_queue+0x21/0x65
  [<ffffffff8123bb06>] kobject_release+0x51/0x66         
  [<ffffffff8123bab5>] ? kobject_release+0x0/0x66
  [<ffffffff8123ce1e>] kref_put+0x43/0x4d         
  [<ffffffff8123ba27>] kobject_put+0x47/0x4b    
  [<ffffffff8122717c>] blk_cleanup_queue+0x56/0x5b
  [<ffffffffa01c3824>] loop_exit+0x68/0x844 [loop]
  [<ffffffff8107cccc>] sys_delete_module+0x1e8/0x25b
  [<ffffffff814864c9>] ? trace_hardirqs_on_thunk+0x3a/0x3f
  [<ffffffff81002112>] system_call_fastpath+0x16/0x1b

because of an attempt to acquire NULL queue_lock.
Added the same lines as in blk_queue_make_request -
`fall back to embedded per-queue lock' - when call blk_release_queue on allocated but not initialized queue.

v2: According to comment by Vivek Goyal, queue_lock NULL check and fix moved out from loop driver code to blk_release_queue, which is more general approach.

Signed-off-by Sergey Senozhatsky <[email protected]>

---

 block/blk-sysfs.c |    6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)
block/blk-sysfs.c +6 -0
index 41fb691..b8ca1f7 100644
@@ -471,6 +471,12 @@ static void blk_release_queue(struct kobject *kobj)
 
 	blk_sync_queue(q);
 
+	/* It's possible that blk_release_queue will be called on allocated
+	 * but never initilalized queue. Fall back to our embedded per-queue 
+	 * locks in this case. */
+	if (!q->queue_lock)
+		q->queue_lock = &q->__queue_lock;
+
 	blk_throtl_exit(q);
 
 	if (rl->rq_pool)