System hangs after echo value > /sys/block/dm-0/queue/nr_requests

If you echo a value (any value; for example the default 128) to /sys/block/dm-0/queue/nr_requests the shell you are in hangs. After about 5 seconds, the whole system hangs 100%.

The system responds to interrupts (ping / sysrq) but that's it. For the rest it's dead in the water.

This is with Linux 2.6.0 and the deadline IO scheduler.

There is no oops or panic. With sysrq I managed to get a backtrace of the process doing the echo:

bash R 00000001 16 657 1 656 (NOTLB)
f6d45e60 00000082 c2432c80 00000001 00000003 00000000 c03ba4a0 00000000

       00000000 00000000 00000000 00000000 00000000 f738b680 f738b714 c01484d4
       f738b714 00000000 00000000 00000000 00000000 f738b680 00001000 00000000
Call Trace:
 [<c01484d4>] vmtruncate+0x79/0x112
 [<c016f0b2>] inode_setattr+0x106/0x17d
 [<c016f29b>] notify_change+0x119/0x16a
 [<c013b0a5>] unlock_page+0x15/0x57
 [<c01482c2>] do_wp_page+0x2c7/0x2f2
 [<c013ee4b>] buffered_rmqueue+0xc4/0x143
 [<c013ef71>] __alloc_pages+0xa7/0x317
 [<c01ff557>] queue_var_store+0x23/0x31
 [<c01ff690>] queue_requests_store+0x11b/0x17d
 [<c01ff751>] queue_attr_store+0x32/0x36
 [<c0185a3a>] flush_write_buffer+0x3b/0x47
 [<c0185aa0>] sysfs_write_file+0x5a/0x69
 [<c0185a46>] sysfs_write_file+0x0/0x69
 [<c0156300>] vfs_write+0xb0/0x119
 [<c015640e>] sys_write+0x42/0x63
 [<c010b20b>] syscall_call+0x7/0xb

A "show regs" shows:

SysRq : Show Regs

Pid: 6, comm: events/0
EIP: 0060:[<c01197b2>] CPU: 0
EIP is at smp_call_function+0x130/0x15b
EFLAGS: 00000297 Not tainted
EAX: 00000000 EBX: 00000001 ECX: 00000000 EDX: 00000002
ESI: 00000001 EDI: 00000297 EBP: f7fda580 DS: 007b ES: 007b
CR0: 8005003b CR2: 400c6fe0 CR3: 36c1d000 CR4: 000006d0

Call Trace:
 [<c0113780>] mce_checkregs+0x0/0x92
 [<c0113839>] do_mce_timer+0x27/0x2b
 [<c0113780>] mce_checkregs+0x0/0x92
 [<c01319f5>] worker_thread+0x1be/0x283
 [<c0113812>] do_mce_timer+0x0/0x2b
 [<c011f07d>] default_wake_function+0x0/0x12
 [<c011f07d>] default_wake_function+0x0/0x12
 [<c0131837>] worker_thread+0x0/0x283
 [<c0109269>] kernel_thread_helper+0x5/0xb

The sysrq reboot function doesn't work - it you use it, the kernel stops responding to sysrq requests as well.

Mike.

When life hands you lemons, grab the salt and pass the tequila.

hm, nice. It does the same thing for /sys/block/md0/queue/nr-requests.

With CONFIG_DEBUG_SPINLOCK enabled we go BUG in __wake_up():

Program received signal SIGTRAP, Trace/breakpoint trap.
__wake_up (q=0xcf299e90, mode=3, nr_exclusive=1) at include/asm/spinlock.h:137
137 BUG();
(gdb) bt
#0 __wake_up (q=0xcf299e90, mode=3, nr_exclusive=1) at include/asm/spinlock.h:137
#1 0xc02cc847 in queue_requests_store (q=0xcf299df8, page=0xe <Address 0xe out of bounds>, count=14)

    at drivers/block/ll_rw_blk.c:2843
#2  0xc02cc907 in queue_attr_store (kobj=0xcf299f68, attr=0xe, page=0xe <Address 0xe out of bounds>, 
    length=14) at drivers/block/ll_rw_blk.c:2892
#3  0xc01aa84f in flush_write_buffer (file=0xe, buffer=0xc0456100, count=14) at fs/sysfs/file.c:205
#4  0xc01aa8ac in sysfs_write_file (file=0xc0493550, buf=0xe <Address 0xe out of bounds>, count=2, 
    ppos=0xc851cf84) at fs/sysfs/file.c:233
#5  0xc016f458 in vfs_write (file=0xc0493550, buf=0x80b2d00 "128\n", count=4, pos=0xc851cf84)
    at fs/read_write.c:257

#6 0xc016f55e in sys_write (fd=14, buf=0xe <Address 0xe out of bounds>, count=14) at fs/read_write.c:293

Where queue_requests_store() does wake_up(&rl->wait[READ]);

It looks like nobody has called blk_init_queue() for this queue and the waitqueue head is uninitialised.

No md was in use on this machine: it was simply enabled in kernel config.

DM, MD, rd and loop use blk_alloc_queue and blk_queue_make_request to initialize their queue, because they only use the make_request_fn. The attached patch prevents the queue from being registered if only blk_alloc_queue was called.

Mike Christie

@@ -2902,7 +2902,7 @@ int blk_register_queue(struct gendisk *d
 
 	request_queue_t *q = disk->queue;
 
-	if (!q)
+	if (!q || !q->request_fn)
 		return -ENXIO;
 
 	q->kobj.parent = kobject_get(&disk->kobj);

@@ -2929,7 +2929,7 @@ void blk_unregister_queue(struct gendisk
 {
 	request_queue_t *q = disk->queue;
 
-	if (q) {
+	if (q && q->request_fn) {
 		elv_unregister_queue(q);
 
 		kobject_unregister(&q->kobj);

I'm fine with that patch since we don't have any attributes in there that apply to just remappers.

Jens Axboe