kmalloc(64) leak from IDE

3.1-rc6 Simon Kirby 4 participants 16 messages Sep 22, 2011 – Sep 30, 2011

Subsystems (2)

ACPI libata

lore

Share & tools

lore

Simon Kirby

author

2011-09-22 07:26 UTC #0 lore raw

All sorts of fun with 3.1-rc!

On an older x86 box still using the IDE code, I'm seeing a kmalloc(64) leak (according to slabtop) that basically OOM'd the box in a few days (640 MB of RAM). This has popped up since 2.6.36, which ran for a long time on this box with no problems. Issues seen on -rc5, so I rebuilt with CONFIG_DEBUG_KMEMLEAK on 9d037a777695993ec7437e5f451647dea7919d4c and /sys/kernel/debug/kmemleak filled up with size 64 traces involving IDE requests. Every trace seems to contain idedisk_prep_fn():

unreferenced object 0xe30c00c0 (size 64):
comm "md6_raid1", pid 255, jiffies 4294903935 (age 23889.704s) hex dump (first 32 bytes):

    00 00 00 00 00 00 00 ea 00 00 00 00 00 00 00 00  ................
    7e 00 00 00 20 00 00 00 01 00 00 00 00 00 00 00  ~... ...........
  backtrace:
    [<c1495c37>] kmemleak_alloc+0x27/0x50
    [<c10b2f4a>] kmem_cache_alloc_trace+0x8a/0x120
    [<c131fe47>] idedisk_prep_fn+0x37/0xf0
    [<c12758b3>] blk_peek_request+0xa3/0x1e0
    [<c1311f15>] __ide_requeue_and_plug+0x25/0x30
    [<c131257d>] do_ide_request+0x3d/0x4e0
    [<c1270ff4>] __blk_run_queue+0x14/0x20
    [<c127699c>] __make_request+0x21c/0x290
    [<c1274f26>] generic_make_request+0x1a6/0x490
    [<c127526c>] submit_bio+0x5c/0xd0
    [<c13a651b>] md_super_write+0x6b/0x80
    [<c13a67ec>] md_update_sb+0x2bc/0x540
    [<c13a7d51>] md_check_recovery+0x2c1/0x5f0
    [<c138abce>] raid1d+0x2e/0xd90
    [<c13a58d5>] md_thread+0xe5/0x110
    [<c1047df4>] kthread+0x74/0x80

unreferenced object 0xc1c3d900 (size 64):
comm "hardirq", pid 0, jiffies 5819438 (age 829.636s) hex dump (first 32 bytes):

    00 00 00 00 00 00 00 ea 00 00 00 00 00 00 00 00  ................
    7e 00 00 00 20 00 00 00 01 00 00 00 00 00 00 00  ~... ...........
  backtrace:
    [<c1495c37>] kmemleak_alloc+0x27/0x50
    [<c10b2f4a>] kmem_cache_alloc_trace+0x8a/0x120
    [<c131fe47>] idedisk_prep_fn+0x37/0xf0
    [<c12758b3>] blk_peek_request+0xa3/0x1e0
    [<c1311f15>] __ide_requeue_and_plug+0x25/0x30
    [<c1311f2f>] ide_requeue_and_plug+0xf/0x20
    [<c1311fb8>] ide_intr+0x78/0x1e0
    [<c1065c14>] handle_irq_event_percpu+0x54/0x1d0
    [<c1065dac>] handle_irq_event+0x1c/0x30
    [<c1067b8c>] handle_level_irq+0x4c/0xa0
    [<ffffffff>] 0xffffffff

idedisk_prep_fn() seems to allocate a command and return it as rq->special, but I'm not following what happens after that.

Simon-

Borislav Petkov

2011-09-22 08:48 UTC #1 lore raw

Quoted 51 lines · Simon Kirby

All sorts of fun with 3.1-rc!
On an older x86 box still using the IDE code, I'm seeing a kmalloc(64) leak (according to slabtop) that basically OOM'd the box in a few days (640 MB of RAM). This has popped up since 2.6.36, which ran for a long time on this box with no problems. Issues seen on -rc5, so I rebuilt with CONFIG_DEBUG_KMEMLEAK on 9d037a777695993ec7437e5f451647dea7919d4c and /sys/kernel/debug/kmemleak filled up with size 64 traces involving IDE requests. Every trace seems to contain idedisk_prep_fn():
unreferenced object 0xe30c00c0 (size 64):
comm "md6_raid1", pid 255, jiffies 4294903935 (age 23889.704s) hex dump (first 32 bytes):
    00 00 00 00 00 00 00 ea 00 00 00 00 00 00 00 00  ................
    7e 00 00 00 20 00 00 00 01 00 00 00 00 00 00 00  ~... ...........
  backtrace:
    [<c1495c37>] kmemleak_alloc+0x27/0x50
    [<c10b2f4a>] kmem_cache_alloc_trace+0x8a/0x120
    [<c131fe47>] idedisk_prep_fn+0x37/0xf0
    [<c12758b3>] blk_peek_request+0xa3/0x1e0
    [<c1311f15>] __ide_requeue_and_plug+0x25/0x30
    [<c131257d>] do_ide_request+0x3d/0x4e0
    [<c1270ff4>] __blk_run_queue+0x14/0x20
    [<c127699c>] __make_request+0x21c/0x290
    [<c1274f26>] generic_make_request+0x1a6/0x490
    [<c127526c>] submit_bio+0x5c/0xd0
    [<c13a651b>] md_super_write+0x6b/0x80
    [<c13a67ec>] md_update_sb+0x2bc/0x540
    [<c13a7d51>] md_check_recovery+0x2c1/0x5f0
    [<c138abce>] raid1d+0x2e/0xd90
    [<c13a58d5>] md_thread+0xe5/0x110
    [<c1047df4>] kthread+0x74/0x80
unreferenced object 0xc1c3d900 (size 64):
comm "hardirq", pid 0, jiffies 5819438 (age 829.636s) hex dump (first 32 bytes):
    00 00 00 00 00 00 00 ea 00 00 00 00 00 00 00 00  ................
    7e 00 00 00 20 00 00 00 01 00 00 00 00 00 00 00  ~... ...........
  backtrace:
    [<c1495c37>] kmemleak_alloc+0x27/0x50
    [<c10b2f4a>] kmem_cache_alloc_trace+0x8a/0x120
    [<c131fe47>] idedisk_prep_fn+0x37/0xf0
    [<c12758b3>] blk_peek_request+0xa3/0x1e0
    [<c1311f15>] __ide_requeue_and_plug+0x25/0x30
    [<c1311f2f>] ide_requeue_and_plug+0xf/0x20
    [<c1311fb8>] ide_intr+0x78/0x1e0
    [<c1065c14>] handle_irq_event_percpu+0x54/0x1d0
    [<c1065dac>] handle_irq_event+0x1c/0x30
    [<c1067b8c>] handle_level_irq+0x4c/0xa0
    [<ffffffff>] 0xffffffff

Quoted 2 lines

idedisk_prep_fn() seems to allocate a command and return it as rq->special, but I'm not following what happens after that.

AFAIR, that's blk_peek_request - it calls q->prep_rq_fn which is idedisk_prep_fn() and it unconditionally allocates those ide_cmd's without freeing them and I can imagine the upper layer requeue one request a couple of times back-to-back, leading to the leaks.

So maybe the following could work, it is a stab in the dark for all I know:

drivers/ide/ide-disk.c +6 -1

index 274798068a54..16f69be820c7 100644
@@ -435,7 +435,12 @@ static int idedisk_prep_fn(struct request_queue *q, struct request *rq)
 	if (!(rq->cmd_flags & REQ_FLUSH))
 		return BLKPREP_OK;
 
-	cmd = kzalloc(sizeof(*cmd), GFP_ATOMIC);
+	if (rq->special) {
+		cmd = rq->special;
+		memset(cmd, 0, sizeof(*cmd));
+	} else {
+		cmd = kzalloc(sizeof(*cmd), GFP_ATOMIC);
+	}
 
 	/* FIXME: map struct ide_taskfile on rq->cmd[] */
 	BUG_ON(cmd == NULL);

Can you rerun it with kmemleak enabled and check whether it still triggers?

Also, I'm sure you know IDE is deprecated, so what are the chances of moving this box to libata? Also, can you send me your .config pls?

Thanks.

Signature · 2 lines

Regards/Gruss,
    Boris.

Simon Kirby

author

2011-09-22 20:23 UTC #2 lore raw

Quoted 84 lines · Borislav Petkov

Quoted 51 lines · Simon Kirby
All sorts of fun with 3.1-rc!
On an older x86 box still using the IDE code, I'm seeing a kmalloc(64) leak (according to slabtop) that basically OOM'd the box in a few days (640 MB of RAM). This has popped up since 2.6.36, which ran for a long time on this box with no problems. Issues seen on -rc5, so I rebuilt with CONFIG_DEBUG_KMEMLEAK on 9d037a777695993ec7437e5f451647dea7919d4c and /sys/kernel/debug/kmemleak filled up with size 64 traces involving IDE requests. Every trace seems to contain idedisk_prep_fn():
unreferenced object 0xe30c00c0 (size 64):
comm "md6_raid1", pid 255, jiffies 4294903935 (age 23889.704s) hex dump (first 32 bytes):
    00 00 00 00 00 00 00 ea 00 00 00 00 00 00 00 00  ................
    7e 00 00 00 20 00 00 00 01 00 00 00 00 00 00 00  ~... ...........
  backtrace:
    [<c1495c37>] kmemleak_alloc+0x27/0x50
    [<c10b2f4a>] kmem_cache_alloc_trace+0x8a/0x120
    [<c131fe47>] idedisk_prep_fn+0x37/0xf0
    [<c12758b3>] blk_peek_request+0xa3/0x1e0
    [<c1311f15>] __ide_requeue_and_plug+0x25/0x30
    [<c131257d>] do_ide_request+0x3d/0x4e0
    [<c1270ff4>] __blk_run_queue+0x14/0x20
    [<c127699c>] __make_request+0x21c/0x290
    [<c1274f26>] generic_make_request+0x1a6/0x490
    [<c127526c>] submit_bio+0x5c/0xd0
    [<c13a651b>] md_super_write+0x6b/0x80
    [<c13a67ec>] md_update_sb+0x2bc/0x540
    [<c13a7d51>] md_check_recovery+0x2c1/0x5f0
    [<c138abce>] raid1d+0x2e/0xd90
    [<c13a58d5>] md_thread+0xe5/0x110
    [<c1047df4>] kthread+0x74/0x80
unreferenced object 0xc1c3d900 (size 64):
comm "hardirq", pid 0, jiffies 5819438 (age 829.636s) hex dump (first 32 bytes):
    00 00 00 00 00 00 00 ea 00 00 00 00 00 00 00 00  ................
    7e 00 00 00 20 00 00 00 01 00 00 00 00 00 00 00  ~... ...........
  backtrace:
    [<c1495c37>] kmemleak_alloc+0x27/0x50
    [<c10b2f4a>] kmem_cache_alloc_trace+0x8a/0x120
    [<c131fe47>] idedisk_prep_fn+0x37/0xf0
    [<c12758b3>] blk_peek_request+0xa3/0x1e0
    [<c1311f15>] __ide_requeue_and_plug+0x25/0x30
    [<c1311f2f>] ide_requeue_and_plug+0xf/0x20
    [<c1311fb8>] ide_intr+0x78/0x1e0
    [<c1065c14>] handle_irq_event_percpu+0x54/0x1d0
    [<c1065dac>] handle_irq_event+0x1c/0x30
    [<c1067b8c>] handle_level_irq+0x4c/0xa0
    [<ffffffff>] 0xffffffff
Quoted 2 lines
idedisk_prep_fn() seems to allocate a command and return it as rq->special, but I'm not following what happens after that.
AFAIR, that's blk_peek_request - it calls q->prep_rq_fn which is idedisk_prep_fn() and it unconditionally allocates those ide_cmd's without freeing them and I can imagine the upper layer requeue one request a couple of times back-to-back, leading to the leaks.
So maybe the following could work, it is a stab in the dark for all I know:
--
drivers/ide/ide-disk.c +6 -2
index 274798068a54..16f69be820c7 100644
@@ -435,7 +435,12 @@ static int idedisk_prep_fn(struct request_queue *q, struct request *rq)
 	if (!(rq->cmd_flags & REQ_FLUSH))
 		return BLKPREP_OK;
 
-	cmd = kzalloc(sizeof(*cmd), GFP_ATOMIC);
+	if (rq->special) {
+		cmd = rq->special;
+		memset(cmd, 0, sizeof(*cmd));
+	} else {
+		cmd = kzalloc(sizeof(*cmd), GFP_ATOMIC);
+	}
 
 	/* FIXME: map struct ide_taskfile on rq->cmd[] */
 	BUG_ON(cmd == NULL);
--

Quoted 2 lines

Can you rerun it with kmemleak enabled and check whether it still triggers?

Yes, that seems to have made it stop complaining about the IDE path. All I see from kmemleak now is:

unreferenced object 0xe7481a00 (size 256):
comm "swapper", pid 1, jiffies 4294892509 (age 515.560s) hex dump (first 32 bytes):

    00 00 00 28 ff ff ef ff 60 78 4e e7 00 02 00 00  ...(....`xN.....
    47 01 f8 0c f8 0c 01 08 00 00 00 00 0c 03 00 00  G...............
  backtrace:
    [<c1495c47>] kmemleak_alloc+0x27/0x50
    [<c10b3563>] __kmalloc+0xf3/0x1c0
    [<c149e0b0>] pci_acpi_scan_root+0x11e/0x272
    [<c149916b>] acpi_pci_root_add+0x163/0x256
    [<c12adddc>] acpi_device_probe+0x3a/0xf4
    [<c1302e38>] driver_probe_device+0x68/0x160
    [<c1302fb9>] __driver_attach+0x89/0x90
    [<c1302718>] bus_for_each_dev+0x48/0x70
    [<c1302cc9>] driver_attach+0x19/0x20
    [<c130213f>] bus_add_driver+0x17f/0x240
    [<c1303345>] driver_register+0x65/0x120
    [<c12af273>] acpi_bus_register_driver+0x3a/0x3f
    [<c16d421f>] acpi_pci_root_init+0x1b/0x2a
    [<c1001030>] do_one_initcall+0x30/0x160
    [<c16b920b>] kernel_init+0x78/0x10c
    [<c14a1b76>] kernel_thread_helper+0x6/0xd
unreferenced object 0xe74e7860 (size 16):
  comm "swapper", pid 1, jiffies 4294892509 (age 515.560s)
  hex dump (first 16 bytes):
    50 43 49 20 42 75 73 20 30 30 30 30 3a 30 30 00  PCI Bus 0000:00.
  backtrace:
    [<c1495c47>] kmemleak_alloc+0x27/0x50
    [<c10b3563>] __kmalloc+0xf3/0x1c0
    [<c1290cce>] kvasprintf+0x2e/0x50
    [<c1290d01>] kasprintf+0x11/0x20
    [<c149e0da>] pci_acpi_scan_root+0x148/0x272
    [<c149916b>] acpi_pci_root_add+0x163/0x256
    [<c12adddc>] acpi_device_probe+0x3a/0xf4
    [<c1302e38>] driver_probe_device+0x68/0x160
    [<c1302fb9>] __driver_attach+0x89/0x90
    [<c1302718>] bus_for_each_dev+0x48/0x70
    [<c1302cc9>] driver_attach+0x19/0x20
    [<c130213f>] bus_add_driver+0x17f/0x240
    [<c1303345>] driver_register+0x65/0x120
    [<c12af273>] acpi_bus_register_driver+0x3a/0x3f
    [<c16d421f>] acpi_pci_root_init+0x1b/0x2a
    [<c1001030>] do_one_initcall+0x30/0x160

...which is probably a separate, non-recurring leak.

Quoted 2 lines

Also, I'm sure you know IDE is deprecated, so what are the chances of moving this box to libata? Also, can you send me your .config pls?

Yeah, I was going to get around to that eventually. :) Config (and earlier kmemleak output) here: http://0x.ca/sim/ref/3.1-rc6-blue/

Simon-

Borislav Petkov

2011-09-23 07:21 UTC #3 lore raw

Quoted 1 line · Simon Kirby

Yes, that seems to have made it stop complaining about the IDE path.

Good, thanks for testing. It would be great if you left it running for a couple of days like this to see whether there aren't any other issues with the patch. I'll send it with a proper description to Dave soonish since this is a real bug.

Quoted 1 line

All I see from kmemleak now is:

Yep, not IDE-related. Adding linux-acpi.

Quoted 44 lines

unreferenced object 0xe7481a00 (size 256):
comm "swapper", pid 1, jiffies 4294892509 (age 515.560s) hex dump (first 32 bytes):

    00 00 00 28 ff ff ef ff 60 78 4e e7 00 02 00 00  ...(....`xN.....
    47 01 f8 0c f8 0c 01 08 00 00 00 00 0c 03 00 00  G...............
  backtrace:
    [<c1495c47>] kmemleak_alloc+0x27/0x50
    [<c10b3563>] __kmalloc+0xf3/0x1c0
    [<c149e0b0>] pci_acpi_scan_root+0x11e/0x272
    [<c149916b>] acpi_pci_root_add+0x163/0x256
    [<c12adddc>] acpi_device_probe+0x3a/0xf4
    [<c1302e38>] driver_probe_device+0x68/0x160
    [<c1302fb9>] __driver_attach+0x89/0x90
    [<c1302718>] bus_for_each_dev+0x48/0x70
    [<c1302cc9>] driver_attach+0x19/0x20
    [<c130213f>] bus_add_driver+0x17f/0x240
    [<c1303345>] driver_register+0x65/0x120
    [<c12af273>] acpi_bus_register_driver+0x3a/0x3f
    [<c16d421f>] acpi_pci_root_init+0x1b/0x2a
    [<c1001030>] do_one_initcall+0x30/0x160
    [<c16b920b>] kernel_init+0x78/0x10c
    [<c14a1b76>] kernel_thread_helper+0x6/0xd
unreferenced object 0xe74e7860 (size 16):
  comm "swapper", pid 1, jiffies 4294892509 (age 515.560s)
  hex dump (first 16 bytes):
    50 43 49 20 42 75 73 20 30 30 30 30 3a 30 30 00  PCI Bus 0000:00.
  backtrace:
    [<c1495c47>] kmemleak_alloc+0x27/0x50
    [<c10b3563>] __kmalloc+0xf3/0x1c0
    [<c1290cce>] kvasprintf+0x2e/0x50
    [<c1290d01>] kasprintf+0x11/0x20
    [<c149e0da>] pci_acpi_scan_root+0x148/0x272
    [<c149916b>] acpi_pci_root_add+0x163/0x256
    [<c12adddc>] acpi_device_probe+0x3a/0xf4
    [<c1302e38>] driver_probe_device+0x68/0x160
    [<c1302fb9>] __driver_attach+0x89/0x90
    [<c1302718>] bus_for_each_dev+0x48/0x70
    [<c1302cc9>] driver_attach+0x19/0x20
    [<c130213f>] bus_add_driver+0x17f/0x240
    [<c1303345>] driver_register+0x65/0x120
    [<c12af273>] acpi_bus_register_driver+0x3a/0x3f
    [<c16d421f>] acpi_pci_root_init+0x1b/0x2a
    [<c1001030>] do_one_initcall+0x30/0x160

Quoted 7 lines

...which is probably a separate, non-recurring leak.
Quoted 2 lines
Also, I'm sure you know IDE is deprecated, so what are the chances of moving this box to libata? Also, can you send me your .config pls?
Yeah, I was going to get around to that eventually. :) Config (and earlier kmemleak output) here: http://0x.ca/sim/ref/3.1-rc6-blue/

Ok, thanks.

Signature · 2 lines

Regards/Gruss,
    Boris.

Bernd Schubert

2011-09-23 15:58 UTC #4 lore raw

Quoted 60 lines · Borislav Petkov

Quoted 1 line · Simon Kirby

Yes, that seems to have made it stop complaining about the IDE path.

Quoted 1 line

All I see from kmemleak now is:

Yep, not IDE-related. Adding linux-acpi.

Quoted 46 lines

unreferenced object 0xe7481a00 (size 256):

   comm "swapper", pid 1, jiffies 4294892509 (age 515.560s)
   hex dump (first 32 bytes):
     00 00 00 28 ff ff ef ff 60 78 4e e7 00 02 00 00  ...(....`xN.....
     47 01 f8 0c f8 0c 01 08 00 00 00 00 0c 03 00 00  G...............
   backtrace:
     [<c1495c47>] kmemleak_alloc+0x27/0x50
     [<c10b3563>] __kmalloc+0xf3/0x1c0
     [<c149e0b0>] pci_acpi_scan_root+0x11e/0x272
     [<c149916b>] acpi_pci_root_add+0x163/0x256
     [<c12adddc>] acpi_device_probe+0x3a/0xf4
     [<c1302e38>] driver_probe_device+0x68/0x160
     [<c1302fb9>] __driver_attach+0x89/0x90
     [<c1302718>] bus_for_each_dev+0x48/0x70
     [<c1302cc9>] driver_attach+0x19/0x20
     [<c130213f>] bus_add_driver+0x17f/0x240
     [<c1303345>] driver_register+0x65/0x120
     [<c12af273>] acpi_bus_register_driver+0x3a/0x3f
     [<c16d421f>] acpi_pci_root_init+0x1b/0x2a
     [<c1001030>] do_one_initcall+0x30/0x160
     [<c16b920b>] kernel_init+0x78/0x10c
     [<c14a1b76>] kernel_thread_helper+0x6/0xd
unreferenced object 0xe74e7860 (size 16):
   comm "swapper", pid 1, jiffies 4294892509 (age 515.560s)
   hex dump (first 16 bytes):
     50 43 49 20 42 75 73 20 30 30 30 30 3a 30 30 00  PCI Bus 0000:00.
   backtrace:
     [<c1495c47>] kmemleak_alloc+0x27/0x50
     [<c10b3563>] __kmalloc+0xf3/0x1c0
     [<c1290cce>] kvasprintf+0x2e/0x50
     [<c1290d01>] kasprintf+0x11/0x20
     [<c149e0da>] pci_acpi_scan_root+0x148/0x272
     [<c149916b>] acpi_pci_root_add+0x163/0x256
     [<c12adddc>] acpi_device_probe+0x3a/0xf4
     [<c1302e38>] driver_probe_device+0x68/0x160
     [<c1302fb9>] __driver_attach+0x89/0x90
     [<c1302718>] bus_for_each_dev+0x48/0x70
     [<c1302cc9>] driver_attach+0x19/0x20
     [<c130213f>] bus_add_driver+0x17f/0x240
     [<c1303345>] driver_register+0x65/0x120
     [<c12af273>] acpi_bus_register_driver+0x3a/0x3f
     [<c16d421f>] acpi_pci_root_init+0x1b/0x2a
     [<c1001030>] do_one_initcall+0x30/0x160

...which is probably a separate, non-recurring leak.

Quoted 8 lines

Quoted 5 lines
Quoted 2 lines
Also, I'm sure you know IDE is deprecated, so what are the chances of moving this box to libata? Also, can you send me your .config pls?
Yeah, I was going to get around to that eventually. :) Config (and earlier kmemleak output) here: http://0x.ca/sim/ref/3.1-rc6-blue/
Ok, thanks.

I think I reported those already some time ago:

https://lkml.org/lkml/2011/6/21/95

Cheers,
Bernd

Bjorn Helgaas

2011-09-23 16:09 UTC #5 lore raw

Quoted 69 lines · Bernd Schubert

Quoted 68 lines · Borislav Petkov
Quoted 2 lines · Simon Kirby
Yes, that seems to have made it stop complaining about the IDE path.
Good, thanks for testing. It would be great if you left it running for a couple of days like this to see whether there aren't any other issues with the patch. I'll send it with a proper description to Dave soonish since this is a real bug.
Quoted 1 line
All I see from kmemleak now is:
Yep, not IDE-related. Adding linux-acpi.
Quoted 51 lines
unreferenced object 0xe7481a00 (size 256):
comm "swapper", pid 1, jiffies 4294892509 (age 515.560s) hex dump (first 32 bytes):
00 00 00 28 ff ff ef ff 60 78 4e e7 00 02 00 00 ...(....`xN..... 47 01 f8 0c f8 0c 01 08 00 00 00 00 0c 03 00 00 G............... backtrace:
    [<c1495c47>] kmemleak_alloc+0x27/0x50
    [<c10b3563>] __kmalloc+0xf3/0x1c0
    [<c149e0b0>] pci_acpi_scan_root+0x11e/0x272
    [<c149916b>] acpi_pci_root_add+0x163/0x256
    [<c12adddc>] acpi_device_probe+0x3a/0xf4
    [<c1302e38>] driver_probe_device+0x68/0x160
    [<c1302fb9>] __driver_attach+0x89/0x90
    [<c1302718>] bus_for_each_dev+0x48/0x70
    [<c1302cc9>] driver_attach+0x19/0x20
    [<c130213f>] bus_add_driver+0x17f/0x240
    [<c1303345>] driver_register+0x65/0x120
    [<c12af273>] acpi_bus_register_driver+0x3a/0x3f
    [<c16d421f>] acpi_pci_root_init+0x1b/0x2a
    [<c1001030>] do_one_initcall+0x30/0x160
    [<c16b920b>] kernel_init+0x78/0x10c
    [<c14a1b76>] kernel_thread_helper+0x6/0xd
unreferenced object 0xe74e7860 (size 16):
comm "swapper", pid 1, jiffies 4294892509 (age 515.560s) hex dump (first 16 bytes):
50 43 49 20 42 75 73 20 30 30 30 30 3a 30 30 00 PCI Bus 0000:00. backtrace:
    [<c1495c47>] kmemleak_alloc+0x27/0x50
    [<c10b3563>] __kmalloc+0xf3/0x1c0
    [<c1290cce>] kvasprintf+0x2e/0x50
    [<c1290d01>] kasprintf+0x11/0x20
    [<c149e0da>] pci_acpi_scan_root+0x148/0x272
    [<c149916b>] acpi_pci_root_add+0x163/0x256
    [<c12adddc>] acpi_device_probe+0x3a/0xf4
    [<c1302e38>] driver_probe_device+0x68/0x160
    [<c1302fb9>] __driver_attach+0x89/0x90
    [<c1302718>] bus_for_each_dev+0x48/0x70
    [<c1302cc9>] driver_attach+0x19/0x20
    [<c130213f>] bus_add_driver+0x17f/0x240
    [<c1303345>] driver_register+0x65/0x120
    [<c12af273>] acpi_bus_register_driver+0x3a/0x3f
    [<c16d421f>] acpi_pci_root_init+0x1b/0x2a
    [<c1001030>] do_one_initcall+0x30/0x160
...which is probably a separate, non-recurring leak.
Quoted 2 lines
Also, I'm sure you know IDE is deprecated, so what are the chances of moving this box to libata? Also, can you send me your .config pls?
Yeah, I was going to get around to that eventually. :) Config (and earlier kmemleak output) here: http://0x.ca/sim/ref/3.1-rc6-blue/

Quoted 6 lines

Quoted 2 lines
Ok, thanks.
I think I reported those already some time ago:
https://lkml.org/lkml/2011/6/21/95

Rats. And you even posted a patch
(https://lkml.org/lkml/2011/6/21/132). I was cc'd, but unfortunately to an old email address that no longer works. I'll follow up on it and make sure it's fixed (probably in 3.2 since it's minor and we're so late in 3.1).

Bjorn
--
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Bernd Schubert

2011-09-23 16:34 UTC #6 lore raw

Quoted 76 lines · Bjorn Helgaas

Quoted 74 lines · Bernd Schubert
Quoted 62 lines · Borislav Petkov
Quoted 2 lines · Simon Kirby
Yes, that seems to have made it stop complaining about the IDE path.
Good, thanks for testing. It would be great if you left it running for a couple of days like this to see whether there aren't any other issues with the patch. I'll send it with a proper description to Dave soonish since this is a real bug.
Quoted 1 line
All I see from kmemleak now is:
Yep, not IDE-related. Adding linux-acpi.
Quoted 46 lines
unreferenced object 0xe7481a00 (size 256):
   comm "swapper", pid 1, jiffies 4294892509 (age 515.560s)
   hex dump (first 32 bytes):
     00 00 00 28 ff ff ef ff 60 78 4e e7 00 02 00 00  ...(....`xN.....
     47 01 f8 0c f8 0c 01 08 00 00 00 00 0c 03 00 00  G...............
   backtrace:
     [<c1495c47>] kmemleak_alloc+0x27/0x50
     [<c10b3563>] __kmalloc+0xf3/0x1c0
     [<c149e0b0>] pci_acpi_scan_root+0x11e/0x272
     [<c149916b>] acpi_pci_root_add+0x163/0x256
     [<c12adddc>] acpi_device_probe+0x3a/0xf4
     [<c1302e38>] driver_probe_device+0x68/0x160
     [<c1302fb9>] __driver_attach+0x89/0x90
     [<c1302718>] bus_for_each_dev+0x48/0x70
     [<c1302cc9>] driver_attach+0x19/0x20
     [<c130213f>] bus_add_driver+0x17f/0x240
     [<c1303345>] driver_register+0x65/0x120
     [<c12af273>] acpi_bus_register_driver+0x3a/0x3f
     [<c16d421f>] acpi_pci_root_init+0x1b/0x2a
     [<c1001030>] do_one_initcall+0x30/0x160
     [<c16b920b>] kernel_init+0x78/0x10c
     [<c14a1b76>] kernel_thread_helper+0x6/0xd
unreferenced object 0xe74e7860 (size 16):
   comm "swapper", pid 1, jiffies 4294892509 (age 515.560s)
   hex dump (first 16 bytes):
     50 43 49 20 42 75 73 20 30 30 30 30 3a 30 30 00  PCI Bus 0000:00.
   backtrace:
     [<c1495c47>] kmemleak_alloc+0x27/0x50
     [<c10b3563>] __kmalloc+0xf3/0x1c0
     [<c1290cce>] kvasprintf+0x2e/0x50
     [<c1290d01>] kasprintf+0x11/0x20
     [<c149e0da>] pci_acpi_scan_root+0x148/0x272
     [<c149916b>] acpi_pci_root_add+0x163/0x256
     [<c12adddc>] acpi_device_probe+0x3a/0xf4
     [<c1302e38>] driver_probe_device+0x68/0x160
     [<c1302fb9>] __driver_attach+0x89/0x90
     [<c1302718>] bus_for_each_dev+0x48/0x70
     [<c1302cc9>] driver_attach+0x19/0x20
     [<c130213f>] bus_add_driver+0x17f/0x240
     [<c1303345>] driver_register+0x65/0x120
     [<c12af273>] acpi_bus_register_driver+0x3a/0x3f
     [<c16d421f>] acpi_pci_root_init+0x1b/0x2a
     [<c1001030>] do_one_initcall+0x30/0x160
...which is probably a separate, non-recurring leak.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Also, I'm sure you know IDE is deprecated, so what are the chances of moving this box to libata? Also, can you send me your .config pls?
Yeah, I was going to get around to that eventually. :) Config (and earlier kmemleak output) here: http://0x.ca/sim/ref/3.1-rc6-blue/
Ok, thanks.
I think I reported those already some time ago:

Quoted 7 lines

Quoted 1 line
https://lkml.org/lkml/2011/6/21/95
Rats. And you even posted a patch
(https://lkml.org/lkml/2011/6/21/132). I was cc'd, but unfortunately to an old email address that no longer works. I'll follow up on it and make sure it's fixed (probably in 3.2 since it's minor and we're so late in 3.1).

Ah great, I thought you are simply too busy to look into it. We are all just changing our mail addresses too often :)

Do you mind if I ping you directly once kernel.org is up again? We have a few test systems here, that fail to boot with recent kernels (just a system hard reset). The issue was introduced in between 2.6.27 and 2.6.32 and it is pci mmconfig related, probably due to a troublesome acpi bios. I filled a bugzilla some time ago and never got a responds on it. But I also need to update it with my latest finding...

Thanks,
Bernd

Bjorn Helgaas

2011-09-23 16:40 UTC #7 lore raw

Quoted 90 lines · Bernd Schubert

Quoted 78 lines · Bjorn Helgaas
Quoted 75 lines · Bernd Schubert
Quoted 62 lines · Borislav Petkov
Quoted 2 lines · Simon Kirby
Yes, that seems to have made it stop complaining about the IDE path.
Good, thanks for testing. It would be great if you left it running for a couple of days like this to see whether there aren't any other issues with the patch. I'll send it with a proper description to Dave soonish since this is a real bug.
Quoted 1 line
All I see from kmemleak now is:
Yep, not IDE-related. Adding linux-acpi.
Quoted 46 lines
unreferenced object 0xe7481a00 (size 256):
comm "swapper", pid 1, jiffies 4294892509 (age 515.560s) hex dump (first 32 bytes):
00 00 00 28 ff ff ef ff 60 78 4e e7 00 02 00 00 ...(....`xN..... 47 01 f8 0c f8 0c 01 08 00 00 00 00 0c 03 00 00 G............... backtrace:
    [<c1495c47>] kmemleak_alloc+0x27/0x50
    [<c10b3563>] __kmalloc+0xf3/0x1c0
    [<c149e0b0>] pci_acpi_scan_root+0x11e/0x272
    [<c149916b>] acpi_pci_root_add+0x163/0x256
    [<c12adddc>] acpi_device_probe+0x3a/0xf4
    [<c1302e38>] driver_probe_device+0x68/0x160
    [<c1302fb9>] __driver_attach+0x89/0x90
    [<c1302718>] bus_for_each_dev+0x48/0x70
    [<c1302cc9>] driver_attach+0x19/0x20
    [<c130213f>] bus_add_driver+0x17f/0x240
    [<c1303345>] driver_register+0x65/0x120
    [<c12af273>] acpi_bus_register_driver+0x3a/0x3f
    [<c16d421f>] acpi_pci_root_init+0x1b/0x2a
    [<c1001030>] do_one_initcall+0x30/0x160
    [<c16b920b>] kernel_init+0x78/0x10c
    [<c14a1b76>] kernel_thread_helper+0x6/0xd
unreferenced object 0xe74e7860 (size 16):
comm "swapper", pid 1, jiffies 4294892509 (age 515.560s) hex dump (first 16 bytes):
50 43 49 20 42 75 73 20 30 30 30 30 3a 30 30 00 PCI Bus 0000:00. backtrace:
    [<c1495c47>] kmemleak_alloc+0x27/0x50
    [<c10b3563>] __kmalloc+0xf3/0x1c0
    [<c1290cce>] kvasprintf+0x2e/0x50
    [<c1290d01>] kasprintf+0x11/0x20
    [<c149e0da>] pci_acpi_scan_root+0x148/0x272
    [<c149916b>] acpi_pci_root_add+0x163/0x256
    [<c12adddc>] acpi_device_probe+0x3a/0xf4
    [<c1302e38>] driver_probe_device+0x68/0x160
    [<c1302fb9>] __driver_attach+0x89/0x90
    [<c1302718>] bus_for_each_dev+0x48/0x70
    [<c1302cc9>] driver_attach+0x19/0x20
    [<c130213f>] bus_add_driver+0x17f/0x240
    [<c1303345>] driver_register+0x65/0x120
    [<c12af273>] acpi_bus_register_driver+0x3a/0x3f
    [<c16d421f>] acpi_pci_root_init+0x1b/0x2a
    [<c1001030>] do_one_initcall+0x30/0x160
...which is probably a separate, non-recurring leak.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Also, I'm sure you know IDE is deprecated, so what are the chances of moving this box to libata? Also, can you send me your .config pls?
Yeah, I was going to get around to that eventually. :) Config (and earlier kmemleak output) here: http://0x.ca/sim/ref/3.1-rc6-blue/
Ok, thanks.
I think I reported those already some time ago:
Quoted 7 lines
Quoted 1 line
https://lkml.org/lkml/2011/6/21/95
Rats. And you even posted a patch
(https://lkml.org/lkml/2011/6/21/132). I was cc'd, but unfortunately to an old email address that no longer works. I'll follow up on it and make sure it's fixed (probably in 3.2 since it's minor and we're so late in 3.1).
Ah great, I thought you are simply too busy to look into it. We are all just changing our mail addresses too often :)

Quoted 6 lines

Do you mind if I ping you directly once kernel.org is up again? We have a few test systems here, that fail to boot with recent kernels (just a system hard reset). The issue was introduced in between 2.6.27 and 2.6.32 and it is pci mmconfig related, probably due to a troublesome acpi bios. I filled a bugzilla some time ago and never got a responds on it. But I also need to update it with my latest finding...

Please do, and you don't need to wait for kernel.org to come back (though it'd be good to include any bugzilla URLs if you know them). Just email me and copy linux-pci (since it seems PCI MMCONFIG-related), and we can try to make some progress. It's very important to fix regressions as quickly as possible.

Bjorn
--
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Bernd Schubert

2011-09-23 17:49 UTC #8 lore raw

Quoted 99 lines · Bjorn Helgaas

Quoted 90 lines · Bernd Schubert
Quoted 78 lines · Bjorn Helgaas
Quoted 75 lines · Bernd Schubert
Quoted 62 lines · Borislav Petkov
Quoted 2 lines · Simon Kirby
Yes, that seems to have made it stop complaining about the IDE path.
Good, thanks for testing. It would be great if you left it running for a couple of days like this to see whether there aren't any other issues with the patch. I'll send it with a proper description to Dave soonish since this is a real bug.
Quoted 1 line
All I see from kmemleak now is:
Yep, not IDE-related. Adding linux-acpi.
Quoted 46 lines
unreferenced object 0xe7481a00 (size 256):
   comm "swapper", pid 1, jiffies 4294892509 (age 515.560s)
   hex dump (first 32 bytes):
     00 00 00 28 ff ff ef ff 60 78 4e e7 00 02 00 00  ...(....`xN.....
     47 01 f8 0c f8 0c 01 08 00 00 00 00 0c 03 00 00  G...............
   backtrace:
     [<c1495c47>] kmemleak_alloc+0x27/0x50
     [<c10b3563>] __kmalloc+0xf3/0x1c0
     [<c149e0b0>] pci_acpi_scan_root+0x11e/0x272
     [<c149916b>] acpi_pci_root_add+0x163/0x256
     [<c12adddc>] acpi_device_probe+0x3a/0xf4
     [<c1302e38>] driver_probe_device+0x68/0x160
     [<c1302fb9>] __driver_attach+0x89/0x90
     [<c1302718>] bus_for_each_dev+0x48/0x70
     [<c1302cc9>] driver_attach+0x19/0x20
     [<c130213f>] bus_add_driver+0x17f/0x240
     [<c1303345>] driver_register+0x65/0x120
     [<c12af273>] acpi_bus_register_driver+0x3a/0x3f
     [<c16d421f>] acpi_pci_root_init+0x1b/0x2a
     [<c1001030>] do_one_initcall+0x30/0x160
     [<c16b920b>] kernel_init+0x78/0x10c
     [<c14a1b76>] kernel_thread_helper+0x6/0xd
unreferenced object 0xe74e7860 (size 16):
   comm "swapper", pid 1, jiffies 4294892509 (age 515.560s)
   hex dump (first 16 bytes):
     50 43 49 20 42 75 73 20 30 30 30 30 3a 30 30 00  PCI Bus 0000:00.
   backtrace:
     [<c1495c47>] kmemleak_alloc+0x27/0x50
     [<c10b3563>] __kmalloc+0xf3/0x1c0
     [<c1290cce>] kvasprintf+0x2e/0x50
     [<c1290d01>] kasprintf+0x11/0x20
     [<c149e0da>] pci_acpi_scan_root+0x148/0x272
     [<c149916b>] acpi_pci_root_add+0x163/0x256
     [<c12adddc>] acpi_device_probe+0x3a/0xf4
     [<c1302e38>] driver_probe_device+0x68/0x160
     [<c1302fb9>] __driver_attach+0x89/0x90
     [<c1302718>] bus_for_each_dev+0x48/0x70
     [<c1302cc9>] driver_attach+0x19/0x20
     [<c130213f>] bus_add_driver+0x17f/0x240
     [<c1303345>] driver_register+0x65/0x120
     [<c12af273>] acpi_bus_register_driver+0x3a/0x3f
     [<c16d421f>] acpi_pci_root_init+0x1b/0x2a
     [<c1001030>] do_one_initcall+0x30/0x160
...which is probably a separate, non-recurring leak.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Also, I'm sure you know IDE is deprecated, so what are the chances of moving this box to libata? Also, can you send me your .config pls?
Yeah, I was going to get around to that eventually. :) Config (and earlier kmemleak output) here: http://0x.ca/sim/ref/3.1-rc6-blue/
Ok, thanks.
I think I reported those already some time ago:
Quoted 7 lines
Quoted 1 line
https://lkml.org/lkml/2011/6/21/95
Rats. And you even posted a patch
(https://lkml.org/lkml/2011/6/21/132). I was cc'd, but unfortunately to an old email address that no longer works. I'll follow up on it and make sure it's fixed (probably in 3.2 since it's minor and we're so late in 3.1).
Ah great, I thought you are simply too busy to look into it. We are all just changing our mail addresses too often :)
Quoted 6 lines
Do you mind if I ping you directly once kernel.org is up again? We have a few test systems here, that fail to boot with recent kernels (just a system hard reset). The issue was introduced in between 2.6.27 and 2.6.32 and it is pci mmconfig related, probably due to a troublesome acpi bios. I filled a bugzilla some time ago and never got a responds on it. But I also need to update it with my latest finding...

Quoted 5 lines

Please do, and you don't need to wait for kernel.org to come back (though it'd be good to include any bugzilla URLs if you know them). Just email me and copy linux-pci (since it seems PCI MMCONFIG-related), and we can try to make some progress. It's very important to fix regressions as quickly as possible.

Hmm, I tried to analyze it already and I don't have the text and attachments I sent to the bugzilla ready on my disk to send it again. So it would be really good if we could wait just a little bit longer (I posted it several months ago already). I also found some kind of workaround for us and need the systems right now for our own development...

Thanks,
Bernd

Simon Kirby

author

2011-09-23 17:38 UTC #9 lore raw

In reply to Borislav Petkov

Quoted 7 lines · Borislav Petkov

Quoted 1 line · Simon Kirby
Yes, that seems to have made it stop complaining about the IDE path.
Good, thanks for testing. It would be great if you left it running for a couple of days like this to see whether there aren't any other issues with the patch. I'll send it with a proper description to Dave soonish since this is a real bug.

It has been up 22 hours now without breaking. I suppose you're worried about the old command being used after it gets reused. Would it help to bisect the regression to see how it used to work? That would let us know how many stable kernels to patch, too.

Simon-

Borislav Petkov

2011-09-25 08:58 UTC #10 lore raw

Quoted 11 lines · Simon Kirby

Quoted 7 lines · Borislav Petkov
Quoted 1 line · Simon Kirby
Yes, that seems to have made it stop complaining about the IDE path.
Good, thanks for testing. It would be great if you left it running for a couple of days like this to see whether there aren't any other issues with the patch. I'll send it with a proper description to Dave soonish since this is a real bug.
It has been up 22 hours now without breaking.

Good.

Quoted 2 lines

I suppose you're worried about the old command being used after it gets reused.

Nah, we clear the old command before reusing it and when the request is done, it gets freed, AFAICT, but...

Quoted 2 lines

Would it help to bisect the regression to see how it used to work? That would let us know how many stable kernels to patch, too.

backporting it to -stable is a good point. I'll add the proper tagging to the patch.

Thanks.

Signature · 2 lines

Regards/Gruss,
    Boris.

Simon Kirby

author

2011-09-26 08:05 UTC #11 lore raw

Quoted 13 lines · Borislav Petkov

Quoted 11 lines · Simon Kirby
Quoted 7 lines · Borislav Petkov
Quoted 1 line · Simon Kirby
Yes, that seems to have made it stop complaining about the IDE path.
Good, thanks for testing. It would be great if you left it running for a couple of days like this to see whether there aren't any other issues with the patch. I'll send it with a proper description to Dave soonish since this is a real bug.
It has been up 22 hours now without breaking.

Quoted 7 lines

Good.
Quoted 2 lines
I suppose you're worried about the old command being used after it gets reused.
Nah, we clear the old command before reusing it and when the request is done, it gets freed, AFAICT, but...

Ok, good. It's still running without any problem, and no new leaks reported.

Quoted 5 lines

Quoted 2 lines
Would it help to bisect the regression to see how it used to work? That would let us know how many stable kernels to patch, too.
backporting it to -stable is a good point. I'll add the proper tagging to the patch.

Do you know in which version the issue started, then?

If not, all I have to start with is that it was fine on 2.6.36, and I can bisect it, if that would help.

Cheers,

Simon-

Borislav Petkov

2011-09-27 17:07 UTC #12 lore raw

(forgot to Cc linux-ide earlier, sorry)

Quoted 2 lines · Simon Kirby

Ok, good. It's still running without any problem, and no new leaks reported.

Ok.

[..]

Quoted 7 lines

Quoted 2 lines
backporting it to -stable is a good point. I'll add the proper tagging to the patch.
Do you know in which version the issue started, then?
If not, all I have to start with is that it was fine on 2.6.36, and I can bisect it, if that would help.

This is exactly the question: AFAICT, it could be that changes in the block layer at some point have caused the ide bust and since almost no one tests ide...

The patch adding the dynamic ide_cmd struct allocation is 395d8ef5bebe547a80737692f9789d2e36da16f2 from 2008 and I don't think it caused the issue then but I could also be remembering it wrong.

So I wouldn't bisect it but test stable trees after 2.6.36 to see whether they have the issue, and if so, only then the patch should be backported.

And this is not that easy now with k.org down but looking at

http://web.archive.org/web/20110725015737/http://kernel.org/

the only stable trees which need to be tested are 2.6.39 and 3.0.

How does that sound?

Signature · 2 lines

Regards/Gruss,
Boris.

Borislav Petkov

2011-09-29 09:27 UTC #13 lore raw

Quoted 30 lines · Borislav Petkov

(forgot to Cc linux-ide earlier, sorry)
Quoted 2 lines · Simon Kirby
Ok, good. It's still running without any problem, and no new leaks reported.
Ok.
[..]
Quoted 7 lines
Quoted 2 lines
backporting it to -stable is a good point. I'll add the proper tagging to the patch.
Do you know in which version the issue started, then?
If not, all I have to start with is that it was fine on 2.6.36, and I can bisect it, if that would help.
This is exactly the question: AFAICT, it could be that changes in the block layer at some point have caused the ide bust and since almost no one tests ide...
The patch adding the dynamic ide_cmd struct allocation is 395d8ef5bebe547a80737692f9789d2e36da16f2 from 2008 and I don't think it caused the issue then but I could also be remembering it wrong.
So I wouldn't bisect it but test stable trees after 2.6.36 to see whether they have the issue, and if so, only then the patch should be backported.

Quoted 7 lines

And this is not that easy now with k.org down but looking at
http://web.archive.org/web/20110725015737/http://kernel.org/
the only stable trees which need to be tested are 2.6.39 and 3.0.
How does that sound?

Btw, here's the patch, if you would like to test 2.6.39 once without it to see whether kmemleak screams and once with it, I'll add the stable tagging.

Thanks.

Quoted 1 line

From 96414ddbfecaaa3d265794c0792d816cf3c1e33d Mon Sep 17 00:00:00 2001

From: Borislav Petkov <[email protected]>
Date: Sun, 25 Sep 2011 13:38:04 +0200
Subject: [PATCH] ide-disk: Fix request requeuing

Simon Kirby reported that on his RAID setup with idedisk underneath the box OOMs after a couple of days of runtime. Running with CONFIG_DEBUG_KMEMLEAK pointed to idedisk_prep_fn() which unconditionally allocates an ide_cmd struct. However, ide_requeue_and_plug() can be called more than once per request, either from the request issue or the IRQ handler path and do blk_peek_request() ends up in idedisk_prep_fn() repeatedly, allocating a struct ide_cmd everytime and "forgetting" the previous pointer.

Make sure the code reuses the old allocated chunk.

Reported-and-tested-by Simon Kirby <[email protected]>

Link http://marc.info/?l=linux-kernel&m=131667641517919

Link http://lkml.kernel.org/r/[email protected] open here

Signed-off-by Borislav Petkov <[email protected]>

---
 drivers/ide/ide-disk.c |    7 ++++++-
 1 files changed, 6 insertions(+), 1 deletions(-)

drivers/ide/ide-disk.c +6 -1

index 274798068a54..16f69be820c7 100644
@@ -435,7 +435,12 @@ static int idedisk_prep_fn(struct request_queue *q, struct request *rq)
 	if (!(rq->cmd_flags & REQ_FLUSH))
 		return BLKPREP_OK;
 
-	cmd = kzalloc(sizeof(*cmd), GFP_ATOMIC);
+	if (rq->special) {
+		cmd = rq->special;
+		memset(cmd, 0, sizeof(*cmd));
+	} else {
+		cmd = kzalloc(sizeof(*cmd), GFP_ATOMIC);
+	}
 
 	/* FIXME: map struct ide_taskfile on rq->cmd[] */
 	BUG_ON(cmd == NULL);

Signature · 6 lines

1.7.5.3.401.gfb674


-- 
Regards/Gruss,
    Boris.

Simon Kirby

author

2011-09-29 22:45 UTC #14 lore raw

Quoted 90 lines · Borislav Petkov

Quoted 30 lines · Borislav Petkov
(forgot to Cc linux-ide earlier, sorry)
Quoted 2 lines · Simon Kirby
Ok, good. It's still running without any problem, and no new leaks reported.
Ok.
[..]
Quoted 7 lines
Quoted 2 lines
backporting it to -stable is a good point. I'll add the proper tagging to the patch.
Do you know in which version the issue started, then?
If not, all I have to start with is that it was fine on 2.6.36, and I can bisect it, if that would help.
This is exactly the question: AFAICT, it could be that changes in the block layer at some point have caused the ide bust and since almost no one tests ide...
The patch adding the dynamic ide_cmd struct allocation is 395d8ef5bebe547a80737692f9789d2e36da16f2 from 2008 and I don't think it caused the issue then but I could also be remembering it wrong.
So I wouldn't bisect it but test stable trees after 2.6.36 to see whether they have the issue, and if so, only then the patch should be backported.
Quoted 7 lines
And this is not that easy now with k.org down but looking at
http://web.archive.org/web/20110725015737/http://kernel.org/
the only stable trees which need to be tested are 2.6.39 and 3.0.
How does that sound?
Btw, here's the patch, if you would like to test 2.6.39 once without it to see whether kmemleak screams and once with it, I'll add the stable tagging.
Thanks.
--
Quoted 1 line
From 96414ddbfecaaa3d265794c0792d816cf3c1e33d Mon Sep 17 00:00:00 2001
From: Borislav Petkov <[email protected]>
Date: Sun, 25 Sep 2011 13:38:04 +0200
Subject: [PATCH] ide-disk: Fix request requeuing
Simon Kirby reported that on his RAID setup with idedisk underneath the box OOMs after a couple of days of runtime. Running with CONFIG_DEBUG_KMEMLEAK pointed to idedisk_prep_fn() which unconditionally allocates an ide_cmd struct. However, ide_requeue_and_plug() can be called more than once per request, either from the request issue or the IRQ handler path and do blk_peek_request() ends up in idedisk_prep_fn() repeatedly, allocating a struct ide_cmd everytime and "forgetting" the previous pointer.
Make sure the code reuses the old allocated chunk.
Reported-and-tested-by Simon Kirby <[email protected]>
Link http://marc.info/?l=linux-kernel&m=131667641517919
Link http://lkml.kernel.org/r/[email protected] open here
Signed-off-by Borislav Petkov <[email protected]>
---
 drivers/ide/ide-disk.c |    7 ++++++-
 1 files changed, 6 insertions(+), 1 deletions(-)
drivers/ide/ide-disk.c +6 -1
index 274798068a54..16f69be820c7 100644
@@ -435,7 +435,12 @@ static int idedisk_prep_fn(struct request_queue *q, struct request *rq)
 	if (!(rq->cmd_flags & REQ_FLUSH))
 		return BLKPREP_OK;
 
-	cmd = kzalloc(sizeof(*cmd), GFP_ATOMIC);
+	if (rq->special) {
+		cmd = rq->special;
+		memset(cmd, 0, sizeof(*cmd));
+	} else {
+		cmd = kzalloc(sizeof(*cmd), GFP_ATOMIC);
+	}
 
 	/* FIXME: map struct ide_taskfile on rq->cmd[] */
 	BUG_ON(cmd == NULL);
Signature · 1 line
1.7.5.3.401.gfb674

Tested against on 2.6.39 with and without this patch, and it definitely leaks without it and does not leak with it. 3.0 and 3.1-rc8 also seems good with the patch.

I tested on another IDE box (with an old Quantum Fireball 6.4GB!) and even with software RAID, I could not see the leak, so I had to use the original box, of course. The only difference I could see is the test box has piix and the production box has via82cxxx, but anyway...

Thanks!

Simon-

Borislav Petkov

2011-09-30 06:40 UTC #15 lore raw

Quoted 3 lines · Simon Kirby

Tested against on 2.6.39 with and without this patch, and it definitely leaks without it and does not leak with it. 3.0 and 3.1-rc8 also seems good with the patch.

Good job, thanks!

Quoted 1 line

I tested on another IDE box (with an old Quantum Fireball 6.4GB!) and

:-)

Quoted 3 lines

even with software RAID, I could not see the leak, so I had to use the original box, of course. The only difference I could see is the test box has piix and the production box has via82cxxx, but anyway...

It might be because the production box drive is reporting of being capable of doing write cache flushes, i.e. it should say something like

ideXX: cache flushes supported

in dmesg. idedisk_prep_fn() is contingent on that and is used only then AFAICT.

Thanks again for testing.

Signature · 2 lines

Regards/Gruss,
    Boris.