Skip to contentSkip to conversation

block: Clean up after deleting BHs

PATCH In review Avi Kivity 4 participants 12 messages Jun 23, 2009 – Jun 24, 2009

lore
Share & tools
lore

Avi Kivity

author
#0 lore raw

PATCH 1/1 block: Clean up after deleting BHs

Commit 6a7ad299 ("Call qemu_bh_delete at bdrv_aio_bh_cb") deletes emulated aio bottom halves to prevent endless accumulation. However, it leaves a stale ->bh pointer, which is then waited on when the aio is reused.

Zeroing the pointer fixes the issue, allowing vmdk format images to be used.

---
 block.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
block.c +2 -0
index aca5a6d..cefbe77 100644
@@ -1374,6 +1374,7 @@ static void bdrv_aio_cancel_em(BlockDriverAIOCB *blockacb)
 {
     BlockDriverAIOCBSync *acb = (BlockDriverAIOCBSync *)blockacb;
     qemu_bh_delete(acb->bh);
+    acb->bh = NULL;
     qemu_aio_release(acb);
 }
 
@@ -1391,6 +1392,7 @@ static void bdrv_aio_bh_cb(void *opaque)
     qemu_vfree(acb->bounce);
     acb->common.cb(acb->common.opaque, acb->ret);
     qemu_bh_delete(acb->bh);
+    acb->bh = NULL;
     qemu_aio_release(acb);
 }
 
Signature · 1 line
1.6.2.2
Acked-by on v1 1/1
#1 lore raw
Quoted 7 lines · Avi Kivity

Commit 6a7ad299 ("Call qemu_bh_delete at bdrv_aio_bh_cb") deletes emulated aio bottom halves to prevent endless accumulation.  However, it leaves a stale ->bh pointer, which is then waited on when the aio is reused.

Zeroing the pointer fixes the issue, allowing vmdk format images to be used.

It also fixes the qcow2 on Win32. At least "savevm" was broken.

Best regards,
Filip Navara

#2 lore raw
In reply to Avi Kivity
Quoted 5 lines · Avi Kivity

Commit 6a7ad299 ("Call qemu_bh_delete at bdrv_aio_bh_cb") deletes emulated aio bottom halves to prevent endless accumulation. However, it leaves a stale ->bh pointer, which is then waited on when the aio is reused.

Zeroing the pointer fixes the issue, allowing vmdk format images to be used.

What operations on vmdk images does this cause to fail? qemu-iotests seems to do fine on vmdk so it's nothing yet exercised by it.

Quoted 18 lines
---
 block.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
block.c +1 -0
index aca5a6d..cefbe77 100644
@@ -1374,6 +1374,7 @@ static void bdrv_aio_cancel_em(BlockDriverAIOCB *blockacb)
 {
     BlockDriverAIOCBSync *acb = (BlockDriverAIOCBSync *)blockacb;
     qemu_bh_delete(acb->bh);
+    acb->bh = NULL;
     qemu_aio_release(acb);
 }
 

block.c static void bdrv_aio_bh_cb(void *opaque) · line 1397

Quoted 6 lines
block.c +1 -0
@@ -1391,6 +1392,7 @@ static void bdrv_aio_bh_cb(void *opaque)
     qemu_vfree(acb->bounce);
     acb->common.cb(acb->common.opaque, acb->ret);
     qemu_bh_delete(acb->bh);
+    acb->bh = NULL;
     qemu_aio_release(acb);
 }

I think not having the state of the private acb area cleared over a free/realloc cycle is pretty dangerous. Wouldn't it be better to always clear that space in qemu_aio_get?

author
#3 lore raw
Quoted 6 lines · Christoph Hellwig
Quoted 4 lines · Avi Kivity

Commit 6a7ad299 ("Call qemu_bh_delete at bdrv_aio_bh_cb") deletes emulated aio bottom halves to prevent endless accumulation. However, it leaves a stale ->bh pointer, which is then waited on when the aio is reused.

Quoted 6 lines
Quoted 2 lines

Zeroing the pointer fixes the issue, allowing vmdk format images to be used.

What operations on vmdk images does this cause to fail? qemu-iotests seems to do fine on vmdk so it's nothing yet exercised by it.

Just starting qemu with a vmdk image hangs. I think the very first read triggers it.

Quoted 10 lines
Quoted 10 lines
block.c +1 -0
@@ -1374,6 +1374,7 @@ static void bdrv_aio_cancel_em(BlockDriverAIOCB *blockacb)
  {
      BlockDriverAIOCBSync *acb = (BlockDriverAIOCBSync *)blockacb;
      qemu_bh_delete(acb->bh);
+    acb->bh = NULL;
      qemu_aio_release(acb);
  }
 
Quoted 13 lines

block.c static void bdrv_aio_bh_cb(void *opaque) · line 1398

Quoted 8 lines
block.c +1 -0
@@ -1391,6 +1392,7 @@ static void bdrv_aio_bh_cb(void *opaque)
      qemu_vfree(acb->bounce);
      acb->common.cb(acb->common.opaque, acb->ret);
      qemu_bh_delete(acb->bh);
+    acb->bh = NULL;
      qemu_aio_release(acb);
  }
 

I think not having the state of the private acb area cleared over a free/realloc cycle is pretty dangerous. Wouldn't it be better to always clear that space in qemu_aio_get?

Maybe, but that's a bigger change. Let's start with this (in stable- too) and rework aio later.

Signature · 1 line
error compiling committee.c: too many arguments to function
#4 lore raw
Quoted 14 lines · Avi Kivity
Quoted 13 lines · Christoph Hellwig
Quoted 9 lines · Avi Kivity

Commit 6a7ad299 ("Call qemu_bh_delete at bdrv_aio_bh_cb") deletes emulated
aio bottom halves to prevent endless accumulation.  However, it leaves a stale ->bh pointer, which is then waited on when the aio is reused.

Zeroing the pointer fixes the issue, allowing vmdk format images to be used.

Quoted 6 lines
Quoted 3 lines

What operations on vmdk images does this cause to fail?  qemu-iotests seems to do fine on vmdk so it's nothing yet exercised by it.

Just starting qemu with a vmdk image hangs.  I think the very first read triggers it.

Actually I think it's the second read ;-)

Quoted 26 lines
Quoted 12 lines
Quoted 12 lines
block.c +1 -0
@@ -1374,6 +1374,7 @@ static void bdrv_aio_cancel_em(BlockDriverAIOCB
 *blockacb)
  {
      BlockDriverAIOCBSync *acb = (BlockDriverAIOCBSync *)blockacb;
      qemu_bh_delete(acb->bh);
+    acb->bh = NULL;
      qemu_aio_release(acb);
  }
Quoted 13 lines

block.c static void bdrv_aio_bh_cb(void *opaque) · line 1398

Quoted 8 lines
block.c +1 -0
@@ -1391,6 +1392,7 @@ static void bdrv_aio_bh_cb(void *opaque)
      qemu_vfree(acb->bounce);
      acb->common.cb(acb->common.opaque, acb->ret);
      qemu_bh_delete(acb->bh);
+    acb->bh = NULL;
      qemu_aio_release(acb);
  }
 

I think not having the state of the private acb area cleared over a free/realloc cycle is pretty dangerous.  Wouldn't it be better to always clear that space in qemu_aio_get?

Quoted 3 lines

Maybe, but that's a bigger change.  Let's start with this (in stable- too) and rework aio later.

Agreed, let's get this in, the win32 builds are seriously affected by the bug due to the absence of AIO on the platform.

Best regards,
Filip Navara

author
#5 lore raw
Quoted 7 lines · Filip Navara
Quoted 3 lines

Just starting qemu with a vmdk image hangs. I think the very first read triggers it.

Actually I think it's the second read ;-)

Ugh, off-by-one error.

Quoted 4 lines

Agreed, let's get this in, the win32 builds are seriously affected by the bug due to the absence of AIO on the platform.

Given that it has both threads and true AIO, this could easily be changed.

Signature · 2 lines
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.
#6 lore raw
In reply to Avi Kivity
Quoted 7 lines · Avi Kivity
Quoted 4 lines

I think not having the state of the private acb area cleared over a free/realloc cycle is pretty dangerous. Wouldn't it be better to always clear that space in qemu_aio_get?

Maybe, but that's a bigger change. Let's start with this (in stable- too) and rework aio later.

It's actually smaller - half the size to be exact :)

block.c +1 -0
Index: qemu/block.c
===================================================================
@@ -1515,6 +1515,7 @@ void *qemu_aio_get(AIOPool *pool, BlockD
     acb->bs = bs;
     acb->cb = cb;
     acb->opaque = opaque;
+    memset(acb + 1, 0, pool->aiocb_size - sizeof(BlockDriverAIOCB));
     return acb;
 }
 
author
#7 lore raw
Quoted 20 lines · Christoph Hellwig
Quoted 8 lines · Avi Kivity
Quoted 5 lines

I think not having the state of the private acb area cleared over a free/realloc cycle is pretty dangerous. Wouldn't it be better to always clear that space in qemu_aio_get?

Maybe, but that's a bigger change. Let's start with this (in stable- too) and rework aio later.

It's actually smaller - half the size to be exact :)

Index: qemu/block.c
===================================================================

block.c void *qemu_aio_get(AIOPool *pool, BlockD · line 1521

Quoted 7 lines
block.c +1 -0
@@ -1515,6 +1515,7 @@ void *qemu_aio_get(AIOPool *pool, BlockD
      acb->bs = bs;
      acb->cb = cb;
      acb->opaque = opaque;
+    memset(acb + 1, 0, pool->aiocb_size - sizeof(BlockDriverAIOCB));
      return acb;
  }
 

I meant in an omg we have to look at all the consequences way rather than byte count. This patch is somewhat less local.

Signature · 2 lines
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.
#8 lore raw
Quoted 2 lines · Avi Kivity

I meant in an omg we have to look at all the consequences way rather than byte count. This patch is somewhat less local.

My patch makes sure a re-used acb is in the same state as a newly allocated one. I think that's much more sensible than messing around with some state during freeing some acbs.

author
#9 lore raw
Quoted 11 lines · Christoph Hellwig
Quoted 3 lines · Avi Kivity

I meant in an omg we have to look at all the consequences way rather than byte count. This patch is somewhat less local.

My patch makes sure a re-used acb is in the same state as a newly allocated one. I think that's much more sensible than messing around with some state during freeing some acbs.

What if it has pointers to allocated stuff it was planning to reuse? You're leaking memory.

Signature · 2 lines
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.
#10 lore raw
In reply to Christoph Hellwig
Quoted 6 lines · Christoph Hellwig
Quoted 5 lines · Avi Kivity
Quoted 4 lines

I think not having the state of the private acb area cleared over a free/realloc cycle is pretty dangerous.  Wouldn't it be better to always clear that space in qemu_aio_get?

Quoted 6 lines
Quoted 2 lines

Maybe, but that's a bigger change.  Let's start with this (in stable- too) and rework aio later.

It's actually smaller - half the size to be exact :)

This one solves the issues on Win32 as well. I have no preference for either patch as long as one of them is applied.

Best regards,
Filip Navara

Acked-by on v1 1/1
#11 lore raw
In reply to Avi Kivity
Quoted 26 lines · Avi Kivity

Commit 6a7ad299 ("Call qemu_bh_delete at bdrv_aio_bh_cb") deletes emulated aio bottom halves to prevent endless accumulation. However, it leaves a stale ->bh pointer, which is then waited on when the aio is reused.

Zeroing the pointer fixes the issue, allowing vmdk format images to be used.

---
block.c | 2 ++

1 files changed, 2 insertions(+), 0 deletions(-)
block.c +1 -0
index aca5a6d..cefbe77 100644
@@ -1374,6 +1374,7 @@ static void bdrv_aio_cancel_em(BlockDriverAIOCB
 *blockacb)
 {
 BlockDriverAIOCBSync *acb = (BlockDriverAIOCBSync *)blockacb;
 qemu_bh_delete(acb->bh);
+ acb->bh = NULL;
 qemu_aio_release(acb);
 }

block.c static void bdrv_aio_bh_cb(void *opaque) · line 1398

Quoted 7 lines
block.c +1 -0
@@ -1391,6 +1392,7 @@ static void bdrv_aio_bh_cb(void *opaque)
 qemu_vfree(acb->bounce);
 acb->common.cb(acb->common.opaque, acb->ret);
 qemu_bh_delete(acb->bh);
+ acb->bh = NULL;
 qemu_aio_release(acb);
 }
 

Thanks, your patch fixes the dma issues in linux guests when booting with a block device without aio support (vmdk, vvfat, vpc, nbd, ...):

without patch (many DMA errors in the guest, long boot time):

[    5.374675] hda: QEMU HARDDISK, ATA DISK drive
[    6.044267] hda: host max PIO4 wanted PIO255(auto-tune) selected PIO2
[    6.045096] hda: MWDMA2 mode selected
[    7.555726] hda: max request size: 512KiB
[    7.556193] hda: 65536 sectors (33 MB) w/256KiB Cache, CHS=65/255/63
[    7.569253] hda: cache flushes supported
[    7.569909]  hda:<6>ide-cd driver 5.00
[   27.568145] hda: dma_timer_expiry: DMA status (0x21)
[   37.568221] hda: DMA timeout error
[   37.568838] hda: dma timeout error: status=0xd8 { Busy }
[   37.572195] hda: DMA disabled
[   57.624211] hda: dma_timer_expiry: DMA status (0x21)
[   67.624221] hda: DMA timeout error
[   67.624842] hda: dma timeout error: status=0xd8 { Busy }
[   67.628192] hda: DMA disabled
[   87.872210] hda: dma_timer_expiry: DMA status (0x21)
[   97.872209] hda: DMA timeout error
[   97.872826] hda: dma timeout error: status=0xd8 { Busy }
[   97.876184] hda: DMA disabled
[  117.928209] hda: dma_timer_expiry: DMA status (0x21)
[  127.928149] hda: DMA timeout error
[  127.928815] hda: dma timeout error: status=0xd8 { Busy }
[  127.932120] hda: DMA disabled

with patch (no DMA errors in the guest, fast boot):

[    5.313278] hda: QEMU HARDDISK, ATA DISK drive
[    5.984435] hda: host max PIO4 wanted PIO255(auto-tune) selected PIO2
[    5.985305] hda: MWDMA2 mode selected
[    7.447866] hda: max request size: 512KiB
[    7.448103] hda: 65536 sectors (33 MB) w/256KiB Cache, CHS=65/255/63
[    7.461297] hda: cache flushes supported
[    7.461948]  hda: unknown partition table

I hope your patch will be applied to QEMU master soon.

Regards
Stefan