Skip to contentSkip to conversation

ATAPI migration fix/hack

PATCH Applied Dr. David Alan Gilbert (git) 4 participants 15 messages Dec 9, 2014 – Feb 2, 2015

lore
Share & tools
lore
#0 lore raw

cover ATAPI migration fix/hack

From: "Dr. David Alan Gilbert" <[email protected]>

This pair of patches fixes a problem where IDE/ATAPI cdrom reads get lost/corrupted over migration.

The first of the patches (restore atapi_dma flag) is a simple fix that I think is safe; it no longer causes corruption in the case we saw, but does still trigger a long timeout.

The second is a hack; it throws a medium error causing the guest to retry the command in the case where migration happens just between the IDE/ATAPI command being submitted
and the bmdma being finished. This recovers a lot
faster than the timeout.

Only tried on Linux guests so far; I think it might be possible to replace both of these by reparsing the command buffer for ATAPI; I'm just not confident I know when that's safe to do, and I wanted to see how disgusted people were by the 2nd hack.

Dave

Dr. David Alan Gilbert (2):
Restore atapi_dma flag across migration
atapi migration: Throw recoverable error to avoid recovery

 hw/ide/atapi.c    | 17 +++++++++++++++++
 hw/ide/core.c     |  1 +
 hw/ide/internal.h |  2 ++
 hw/ide/pci.c      | 11 +++++++++++
 4 files changed, 31 insertions(+)
Signature · 1 line
2.1.0
#1 lore raw

PATCH 1/2 Restore atapi_dma flag across migration

From: "Dr. David Alan Gilbert" <[email protected]>

If a migration happens just after the guest has kicked off an ATAPI command and kicked off DMA, we lose the atapi_dma flag, and the destination tries to complete the command as PIO rather than DMA. This upsets Linux; modern libata based kernels stumble and recover OK, older kernels end up passing bad data to userspace.

---
 hw/ide/core.c | 1 +
 1 file changed, 1 insertion(+)
hw/ide/core.c +1 -0
index d4af5e2..ac3f015 100644
@@ -2417,6 +2417,7 @@ static int ide_drive_pio_post_load(void *opaque, int version_id)
     s->end_transfer_func = transfer_end_table[s->end_transfer_fn_idx];
     s->data_ptr = s->io_buffer + s->cur_io_buffer_offset;
     s->data_end = s->data_ptr + s->cur_io_buffer_len;
+    s->atapi_dma = s->feature & 1; /* as per cmd_packet */
 
     return 0;
 }
Signature · 1 line
2.1.0
Reviewed-by on v1 1/2
#2 lore raw

Re: [Qemu-devel] [PATCH 1/2] Restore atapi_dma flag across migration

Quoted 20 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

If a migration happens just after the guest has kicked off an ATAPI command and kicked off DMA, we lose the atapi_dma flag, and the destination tries to complete the command as PIO rather than DMA. This upsets Linux; modern libata based kernels stumble and recover OK, older kernels end up passing bad data to userspace.

---
  hw/ide/core.c | 1 +
  1 file changed, 1 insertion(+)
hw/ide/core.c +0 -0
index d4af5e2..ac3f015 100644

hw/ide/core.c static int ide_drive_pio_post_load(void *opaque, int version_id) · line 2423

Quoted 7 lines
hw/ide/core.c +1 -0
@@ -2417,6 +2417,7 @@ static int ide_drive_pio_post_load(void *opaque, int version_id)
      s->end_transfer_func = transfer_end_table[s->end_transfer_fn_idx];
      s->data_ptr = s->io_buffer + s->cur_io_buffer_offset;
      s->data_end = s->data_ptr + s->cur_io_buffer_len;
+    s->atapi_dma = s->feature & 1; /* as per cmd_packet */
 
      return 0;
  }

This looks fine to me; though perhaps a more comprehensive fix might prevent us needing this at all. As the code exists today, it's definitely an error to lose this value, as you've noticed.

#3 lore raw

PATCH 2/2 atapi migration: Throw recoverable error to avoid recovery

In reply to Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

(With the previous atapi_dma flag recovery)
If migration happens between the ATAPI command being written and the bmdma being started, the DMA is dropped. Eventually the guest times out and recovers, but that can take many seconds.
(This is rare, on a pingpong reading the CD continuously I hit this about ~1/30-1/50 migrates)

I don't think we've got enough state to be able to recover safely at this point, so I throw a 'medium error, no seek complete' that I'm assuming guests will try and recover from an apparently dirty CD.

OK, it's a hack, the real solution is probably to push a lot of ATAPI state into the migration stream, but this is a fix that works with no stream changes. Tested only on Linux (both RHEL5 (pre-libata) and RHEL7).

---
 hw/ide/atapi.c    | 17 +++++++++++++++++
 hw/ide/internal.h |  2 ++
 hw/ide/pci.c      | 11 +++++++++++
 3 files changed, 30 insertions(+)
hw/ide/atapi.c +17 -0
index c63b7e5..e17799c 100644
@@ -394,6 +394,23 @@ static void ide_atapi_cmd_read(IDEState *s, int lba, int nb_sectors,
     }
 }
 
+
+/* Called by *_restart_bh when the transfer function points
+ * to ide_atapi_cmd
+ */
+void ide_atapi_dma_restart(IDEState *s)
+{
+    /*
+     * I'm not sure we have enough stored to restart the command
+     * safely, so give the guest an error it should recover from.
+     * I'm assuming most guests will try to recover from something
+     * listed as a medium error on a CD; it seems to work on Linux.
+     * This would be more of a problem if we did any other type of
+     * DMA operation.
+     */
+    ide_atapi_cmd_error(s, MEDIUM_ERROR, ASC_NO_SEEK_COMPLETE);
+}
+
 static inline uint8_t ide_atapi_set_profile(uint8_t *buf, uint8_t *index,
                                             uint16_t profile)
 {
hw/ide/internal.h +2 -0
index 8a3eca4..8b65285 100644
@@ -289,6 +289,7 @@ typedef struct IDEDMAOps IDEDMAOps;
 #define ATAPI_INT_REASON_TAG            0xf8
 
 /* same constants as bochs */
+#define ASC_NO_SEEK_COMPLETE                 0x02
 #define ASC_ILLEGAL_OPCODE                   0x20
 #define ASC_LOGICAL_BLOCK_OOR                0x21
 #define ASC_INV_FIELD_IN_CMD_PACKET          0x24
@@ -529,6 +530,7 @@ void ide_dma_error(IDEState *s);
 
 void ide_atapi_cmd_ok(IDEState *s);
 void ide_atapi_cmd_error(IDEState *s, int sense_key, int asc);
+void ide_atapi_dma_restart(IDEState *s);
 void ide_atapi_io_error(IDEState *s, int ret);
 
 void ide_ioport_write(void *opaque, uint32_t addr, uint32_t val);
hw/ide/pci.c +11 -0
index bee5ad3..e3f2054 100644
@@ -235,6 +235,17 @@ static void bmdma_restart_bh(void *opaque)
         }
     } else if (error_status & IDE_RETRY_FLUSH) {
         ide_flush_cache(bmdma_active_if(bm));
+    } else {
+        IDEState *s = bmdma_active_if(bm);
+
+        /*
+         * We've not got any bits to tell us about ATAPI - but
+         * we do have the end_transfer_func that tells us what
+         * we're trying to do.
+         */
+        if (s->end_transfer_func == ide_atapi_cmd) {
+            ide_atapi_dma_restart(s);
+        }
     }
 }
 
Signature · 1 line
2.1.0
#4 lore raw

Re: [Qemu-devel] [PATCH 2/2] atapi migration: Throw recoverable error to avoid recovery

Quoted 41 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

(With the previous atapi_dma flag recovery)
If migration happens between the ATAPI command being written and the bmdma being started, the DMA is dropped. Eventually the guest times out and recovers, but that can take many seconds.
(This is rare, on a pingpong reading the CD continuously I hit this about ~1/30-1/50 migrates)

I don't think we've got enough state to be able to recover safely at this point, so I throw a 'medium error, no seek complete' that I'm assuming guests will try and recover from an apparently dirty CD.

OK, it's a hack, the real solution is probably to push a lot of ATAPI state into the migration stream, but this is a fix that works with no stream changes. Tested only on Linux (both RHEL5 (pre-libata) and RHEL7).

---
  hw/ide/atapi.c    | 17 +++++++++++++++++
  hw/ide/internal.h |  2 ++
  hw/ide/pci.c      | 11 +++++++++++
  3 files changed, 30 insertions(+)
hw/ide/atapi.c +7 -0
index c63b7e5..e17799c 100644
@@ -394,6 +394,23 @@ static void ide_atapi_cmd_read(IDEState *s, int lba, int nb_sectors,
      }
  }
 
+
+/* Called by *_restart_bh when the transfer function points
+ * to ide_atapi_cmd
+ */
+void ide_atapi_dma_restart(IDEState *s)
+{
+    /*

hw/ide/atapi.c static void ide_atapi_cmd_read(IDEState *s, int lba, int nb_sectors, · line 413

Quoted 10 lines
hw/ide/atapi.c +10 -0
+     * I'm not sure we have enough stored to restart the command
+     * safely, so give the guest an error it should recover from.
+     * I'm assuming most guests will try to recover from something
+     * listed as a medium error on a CD; it seems to work on Linux.
+     * This would be more of a problem if we did any other type of
+     * DMA operation.
+     */
+    ide_atapi_cmd_error(s, MEDIUM_ERROR, ASC_NO_SEEK_COMPLETE);
+}
+

Is this safe for non-data commands? Can we even get there in such a case?

Quoted 32 lines
hw/ide/atapi.c +0 -0
  static inline uint8_t ide_atapi_set_profile(uint8_t *buf, uint8_t *index,
                                              uint16_t profile)
  {
hw/ide/internal.h +2 -0
index 8a3eca4..8b65285 100644
@@ -289,6 +289,7 @@ typedef struct IDEDMAOps IDEDMAOps;
  #define ATAPI_INT_REASON_TAG            0xf8
 
  /* same constants as bochs */
+#define ASC_NO_SEEK_COMPLETE                 0x02
  #define ASC_ILLEGAL_OPCODE                   0x20
  #define ASC_LOGICAL_BLOCK_OOR                0x21
  #define ASC_INV_FIELD_IN_CMD_PACKET          0x24
@@ -529,6 +530,7 @@ void ide_dma_error(IDEState *s);
 
  void ide_atapi_cmd_ok(IDEState *s);
  void ide_atapi_cmd_error(IDEState *s, int sense_key, int asc);
+void ide_atapi_dma_restart(IDEState *s);
  void ide_atapi_io_error(IDEState *s, int ret);
 
  void ide_ioport_write(void *opaque, uint32_t addr, uint32_t val);
hw/ide/pci.c +1 -0
index bee5ad3..e3f2054 100644
@@ -235,6 +235,17 @@ static void bmdma_restart_bh(void *opaque)
          }
      } else if (error_status & IDE_RETRY_FLUSH) {
          ide_flush_cache(bmdma_active_if(bm));
+    } else {

hw/ide/pci.c static void bmdma_restart_bh(void *opaque) · line 248

Quoted 10 lines
hw/ide/pci.c +10 -0
+        IDEState *s = bmdma_active_if(bm);
+
+        /*
+         * We've not got any bits to tell us about ATAPI - but
+         * we do have the end_transfer_func that tells us what
+         * we're trying to do.
+         */
+        if (s->end_transfer_func == ide_atapi_cmd) {
+            ide_atapi_dma_restart(s);
+        }

OK, so when the restart routines get invoked we add a hook to see if we were in the middle of an ATAPI command and acknowledge that we don't know how to properly handle this.

Isn't this going to run on every vmstate change, though? I think we don't clear out end_transfer_func on success, so this might fire off more than we want it to, although I guess end_transfer_func is usually going to get set to ide_atapi_cmd_reply_end if it finishes normally ...

hw/ide/pci.c static void bmdma_restart_bh(void *opaque) · line 251

Quoted 4 lines
hw/ide/pci.c +0 -0
      }
  }
 

Indeed a hack, but it's probably appropriate: if our code cannot in fact handle ATAPI migration, throwing an error or disabling migration is the correct thing to do, but I don't think users would be very happy with the second option. I feel that this is an OK workaround because it should not introduce spurious errors or retries for cases where we manage to avoid migrating in the middle of the loop. This will at least let the currently broken case limp along until we fix it more properly.

What makes me the most curious is how this plays out in Windows if this case is triggered. Throw a trace around the fake error and see if you can't observe it getting called during a pingpong test while Windows reads a CD.

#5 lore raw

Re: [Qemu-devel] [PATCH 2/2] atapi migration: Throw recoverable error to avoid recovery

Quoted 48 lines · * John Snow ([email protected]) wrote:
Quoted 45 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

(With the previous atapi_dma flag recovery)
If migration happens between the ATAPI command being written and the bmdma being started, the DMA is dropped. Eventually the guest times out and recovers, but that can take many seconds.
(This is rare, on a pingpong reading the CD continuously I hit this about ~1/30-1/50 migrates)

I don't think we've got enough state to be able to recover safely at this point, so I throw a 'medium error, no seek complete' that I'm assuming guests will try and recover from an apparently dirty CD.

OK, it's a hack, the real solution is probably to push a lot of ATAPI state into the migration stream, but this is a fix that works with no stream changes. Tested only on Linux (both RHEL5 (pre-libata) and RHEL7).

---
 hw/ide/atapi.c    | 17 +++++++++++++++++
 hw/ide/internal.h |  2 ++
 hw/ide/pci.c      | 11 +++++++++++
 3 files changed, 30 insertions(+)
hw/ide/atapi.c +11 -0
index c63b7e5..e17799c 100644
@@ -394,6 +394,23 @@ static void ide_atapi_cmd_read(IDEState *s, int lba, int nb_sectors,
     }
 }
 
+
+/* Called by *_restart_bh when the transfer function points
+ * to ide_atapi_cmd
+ */
+void ide_atapi_dma_restart(IDEState *s)
+{
+    /*
+     * I'm not sure we have enough stored to restart the command
+     * safely, so give the guest an error it should recover from.
+     * I'm assuming most guests will try to recover from something
+     * listed as a medium error on a CD; it seems to work on Linux.
Quoted 8 lines

hw/ide/atapi.c static void ide_atapi_cmd_read(IDEState *s, int lba, int nb_sectors, · line 413

Quoted 6 lines
hw/ide/atapi.c +6 -0
+     * This would be more of a problem if we did any other type of
+     * DMA operation.
+     */
+    ide_atapi_cmd_error(s, MEDIUM_ERROR, ASC_NO_SEEK_COMPLETE);
+}
+

Is this safe for non-data commands? Can we even get there in such a case?

See below.

Quoted 38 lines
Quoted 38 lines
hw/ide/atapi.c +0 -0
 static inline uint8_t ide_atapi_set_profile(uint8_t *buf, uint8_t *index,
                                             uint16_t profile)
 {
hw/ide/internal.h +2 -0
index 8a3eca4..8b65285 100644
@@ -289,6 +289,7 @@ typedef struct IDEDMAOps IDEDMAOps;
 #define ATAPI_INT_REASON_TAG            0xf8
 
 /* same constants as bochs */
+#define ASC_NO_SEEK_COMPLETE                 0x02
 #define ASC_ILLEGAL_OPCODE                   0x20
 #define ASC_LOGICAL_BLOCK_OOR                0x21
 #define ASC_INV_FIELD_IN_CMD_PACKET          0x24
@@ -529,6 +530,7 @@ void ide_dma_error(IDEState *s);
 
 void ide_atapi_cmd_ok(IDEState *s);
 void ide_atapi_cmd_error(IDEState *s, int sense_key, int asc);
+void ide_atapi_dma_restart(IDEState *s);
 void ide_atapi_io_error(IDEState *s, int ret);
 
 void ide_ioport_write(void *opaque, uint32_t addr, uint32_t val);
hw/ide/pci.c +7 -0
index bee5ad3..e3f2054 100644
@@ -235,6 +235,17 @@ static void bmdma_restart_bh(void *opaque)
         }
     } else if (error_status & IDE_RETRY_FLUSH) {
         ide_flush_cache(bmdma_active_if(bm));
+    } else {
+        IDEState *s = bmdma_active_if(bm);
+
+        /*
+         * We've not got any bits to tell us about ATAPI - but
+         * we do have the end_transfer_func that tells us what
+         * we're trying to do.
Quoted 8 lines

hw/ide/pci.c static void bmdma_restart_bh(void *opaque) · line 248

Quoted 4 lines
hw/ide/pci.c +4 -0
+         */
+        if (s->end_transfer_func == ide_atapi_cmd) {
+            ide_atapi_dma_restart(s);
+        }

OK, so when the restart routines get invoked we add a hook to see if we were in the middle of an ATAPI command and acknowledge that we don't know how to properly handle this.

As to your qeustion above about non-data commands; hmm probably - but how do I guard it? I guess I could check for the atapi_dma flag the previous patch fixed.
(This is all probably still broken for non-DMA atapi transfers)

Quoted 1 line

Isn't this going to run on every vmstate change, though?

There aren't many - only starting/stopping the CPU does it; and bmdma_restart_cb guards it by 'if (!running)' exit, so it'll only do it when the CPU starts running again.

Quoted 4 lines

I think we don't
clear out end_transfer_func on success, so this might fire off more than we want it to, although I guess end_transfer_func is usually going to get set to ide_atapi_cmd_reply_end if it finishes normally ...

Right, or if ide_transfer_stop is called.

Quoted 13 lines

hw/ide/pci.c static void bmdma_restart_bh(void *opaque) · line 251

Quoted 4 lines
hw/ide/pci.c +0 -0
     }
 }
 

Indeed a hack, but it's probably appropriate: if our code cannot in fact handle ATAPI migration, throwing an error or disabling migration is the correct thing to do, but I don't think users would be very happy with the second option. I feel that this is an OK workaround because it should not introduce spurious errors or retries for cases where we manage to avoid migrating in the middle of the loop. This will at least let the currently broken case limp along until we fix it more properly.

Quoted 3 lines

What makes me the most curious is how this plays out in Windows if this case is triggered. Throw a trace around the fake error and see if you can't observe it getting called during a pingpong test while Windows reads a CD.

Yeh, I'm going to figure out how to try that.

Dave
--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

#6 lore raw

Re: [Qemu-devel] [PATCH 2/2] atapi migration: Throw recoverable error to avoid recovery

In reply to John Snow
Quoted 3 lines · * John Snow ([email protected]) wrote:

What makes me the most curious is how this plays out in Windows if this case is triggered. Throw a trace around the fake error and see if you can't observe it getting called during a pingpong test while Windows reads a CD.

I've not managed to trigger it on Windows (2012).
I've run InfraRecorder's track test with heavy migration and it doesn't seem to trigger this case, or cause any extra errors. (Curiously it does complain about not being able to read extended error information and say our CDROM is giving about 35% C2 errors - even without migration; but I think this is just bogus junk it's picking up, although feel free to polish the scratches out of our virtual CD).

Dave

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

#7 lore raw

Re: [Qemu-devel] [PATCH 2/2] atapi migration: Throw recoverable error to avoid recovery

In reply to Dr. David Alan Gilbert (git)
Quoted 11 lines · Dr. David Alan Gilbert (git)

(With the previous atapi_dma flag recovery)
If migration happens between the ATAPI command being written and the bmdma being started, the DMA is dropped. Eventually the guest times out and recovers, but that can take many seconds.
(This is rare, on a pingpong reading the CD continuously I hit this about ~1/30-1/50 migrates)

I don't think we've got enough state to be able to recover safely at this point, so I throw a 'medium error, no seek complete' that I'm assuming guests will try and recover from an apparently dirty CD.

What is the part of the state that is not being migrated? Also, what is the ATAPI command that is being run?

Paolo

#8 lore raw

Re: [Qemu-devel] [PATCH 2/2] atapi migration: Throw recoverable error to avoid recovery

Quoted 10 lines · * Paolo Bonzini ([email protected]) wrote:
Quoted 7 lines · Dr. David Alan Gilbert (git)

(With the previous atapi_dma flag recovery)
If migration happens between the ATAPI command being written and the bmdma being started, the DMA is dropped. Eventually the guest times out and recovers, but that can take many seconds.
(This is rare, on a pingpong reading the CD continuously I hit this about ~1/30-1/50 migrates)

Quoted 7 lines
Quoted 4 lines

I don't think we've got enough state to be able to recover safely at this point, so I throw a 'medium error, no seek complete' that I'm assuming guests will try and recover from an apparently dirty CD.

Also, what is the ATAPI command that is being run?

The command in my test case is:
28 00 00 06 df b8 00 00 40 00 00 00
so just a standard READ (10); I guess it's possible that other OSs/operations maybe doing different READ variants.

Quoted 1 line

What is the part of the state that is not being migrated?

I can't see lba, io_buffer_index, io_buffer_size or cd_sector_size being passed to a VMSTATE_ macro; and having noticed that those are missing I didn't dig much deeper.
(I'll admit to not really understanding the interaction between the core.c, atapi.c and pci.c dma code properly to know if it can be reconstructed).

Dave

Quoted 1 line

Paolo

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

#9 lore raw

Re: [Qemu-devel] [PATCH 2/2] atapi migration: Throw recoverable error to avoid recovery

Quoted 24 lines · * Dr. David Alan Gilbert ([email protected]) wrote:
Quoted 10 lines · * Paolo Bonzini ([email protected]) wrote:
Quoted 7 lines · Dr. David Alan Gilbert (git)

(With the previous atapi_dma flag recovery)
If migration happens between the ATAPI command being written and the bmdma being started, the DMA is dropped. Eventually the guest times out and recovers, but that can take many seconds.
(This is rare, on a pingpong reading the CD continuously I hit this about ~1/30-1/50 migrates)

Quoted 7 lines
Quoted 4 lines

I don't think we've got enough state to be able to recover safely at this point, so I throw a 'medium error, no seek complete' that I'm assuming guests will try and recover from an apparently dirty CD.

Also, what is the ATAPI command that is being run?

The command in my test case is:
28 00 00 06 df b8 00 00 40 00 00 00
so just a standard READ (10); I guess it's possible that other OSs/operations maybe doing different READ variants.

Quoted 7 lines
Quoted 1 line

What is the part of the state that is not being migrated?

I can't see lba, io_buffer_index, io_buffer_size or cd_sector_size being passed to a VMSTATE_ macro; and having noticed that those are missing I didn't dig much deeper.
(I'll admit to not really understanding the interaction between the core.c, atapi.c and pci.c dma code properly to know if it can be reconstructed).

Actually, another argument for this is that even if we come along and add the extra state as a subsection, we can still do this as a fallback when receiving older streams.

Dave
--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

#10 lore raw

Re: [Qemu-devel] [PATCH 2/2] atapi migration: Throw recoverable error to avoid recovery

Quoted 33 lines · Dr. David Alan Gilbert
Quoted 24 lines · * Dr. David Alan Gilbert ([email protected]) wrote:
Quoted 10 lines · * Paolo Bonzini ([email protected]) wrote:
Quoted 7 lines · Dr. David Alan Gilbert (git)

(With the previous atapi_dma flag recovery)
If migration happens between the ATAPI command being written and the bmdma being started, the DMA is dropped. Eventually the guest times out and recovers, but that can take many seconds.
(This is rare, on a pingpong reading the CD continuously I hit this about ~1/30-1/50 migrates)

Quoted 7 lines
Quoted 4 lines

I don't think we've got enough state to be able to recover safely at this point, so I throw a 'medium error, no seek complete' that I'm assuming guests will try and recover from an apparently dirty CD.

Also, what is the ATAPI command that is being run?

The command in my test case is:
28 00 00 06 df b8 00 00 40 00 00 00
so just a standard READ (10); I guess it's possible that other OSs/operations maybe doing different READ variants.

Quoted 7 lines
Quoted 1 line

What is the part of the state that is not being migrated?

I can't see lba, io_buffer_index, io_buffer_size or cd_sector_size being passed to a VMSTATE_ macro; and having noticed that those are missing I didn't dig much deeper.
(I'll admit to not really understanding the interaction between the core.c, atapi.c and pci.c dma code properly to know if it can be reconstructed).

Quoted 4 lines

Actually, another argument for this is that even if we come along and add the extra state as a subsection, we can still do this as a fallback when receiving older streams.

Fair enough. I vote we give this hack a shot for now, then. I can always put this in my basket of things to look at ($later) after I have finished ($other_things).

Reviewed-by on v1 2/2
#11 lore raw

Re: [Qemu-devel] [PATCH 2/2] atapi migration: Throw recoverable error to avoid recovery

In reply to Dr. David Alan Gilbert (git)
Quoted 87 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

(With the previous atapi_dma flag recovery)
If migration happens between the ATAPI command being written and the bmdma being started, the DMA is dropped. Eventually the guest times out and recovers, but that can take many seconds.
(This is rare, on a pingpong reading the CD continuously I hit this about ~1/30-1/50 migrates)

I don't think we've got enough state to be able to recover safely at this point, so I throw a 'medium error, no seek complete' that I'm assuming guests will try and recover from an apparently dirty CD.

OK, it's a hack, the real solution is probably to push a lot of ATAPI state into the migration stream, but this is a fix that works with no stream changes. Tested only on Linux (both RHEL5 (pre-libata) and RHEL7).

---
  hw/ide/atapi.c    | 17 +++++++++++++++++
  hw/ide/internal.h |  2 ++
  hw/ide/pci.c      | 11 +++++++++++
  3 files changed, 30 insertions(+)
hw/ide/atapi.c +17 -0
index c63b7e5..e17799c 100644
@@ -394,6 +394,23 @@ static void ide_atapi_cmd_read(IDEState *s, int lba, int nb_sectors,
      }
  }
 
+
+/* Called by *_restart_bh when the transfer function points
+ * to ide_atapi_cmd
+ */
+void ide_atapi_dma_restart(IDEState *s)
+{
+    /*
+     * I'm not sure we have enough stored to restart the command
+     * safely, so give the guest an error it should recover from.
+     * I'm assuming most guests will try to recover from something
+     * listed as a medium error on a CD; it seems to work on Linux.
+     * This would be more of a problem if we did any other type of
+     * DMA operation.
+     */
+    ide_atapi_cmd_error(s, MEDIUM_ERROR, ASC_NO_SEEK_COMPLETE);
+}
+
  static inline uint8_t ide_atapi_set_profile(uint8_t *buf, uint8_t *index,
                                              uint16_t profile)
  {
hw/ide/internal.h +2 -0
index 8a3eca4..8b65285 100644
@@ -289,6 +289,7 @@ typedef struct IDEDMAOps IDEDMAOps;
  #define ATAPI_INT_REASON_TAG            0xf8
 
  /* same constants as bochs */
+#define ASC_NO_SEEK_COMPLETE                 0x02
  #define ASC_ILLEGAL_OPCODE                   0x20
  #define ASC_LOGICAL_BLOCK_OOR                0x21
  #define ASC_INV_FIELD_IN_CMD_PACKET          0x24
@@ -529,6 +530,7 @@ void ide_dma_error(IDEState *s);
 
  void ide_atapi_cmd_ok(IDEState *s);
  void ide_atapi_cmd_error(IDEState *s, int sense_key, int asc);
+void ide_atapi_dma_restart(IDEState *s);
  void ide_atapi_io_error(IDEState *s, int ret);
 
  void ide_ioport_write(void *opaque, uint32_t addr, uint32_t val);
hw/ide/pci.c +4 -0
index bee5ad3..e3f2054 100644
@@ -235,6 +235,17 @@ static void bmdma_restart_bh(void *opaque)
          }
      } else if (error_status & IDE_RETRY_FLUSH) {
          ide_flush_cache(bmdma_active_if(bm));
+    } else {
+        IDEState *s = bmdma_active_if(bm);
+
+        /*

hw/ide/pci.c static void bmdma_restart_bh(void *opaque) · line 251

Quoted 10 lines
hw/ide/pci.c +7 -0
+         * We've not got any bits to tell us about ATAPI - but
+         * we do have the end_transfer_func that tells us what
+         * we're trying to do.
+         */
+        if (s->end_transfer_func == ide_atapi_cmd) {
+            ide_atapi_dma_restart(s);
+        }
      }
  }
 

I think this workaround is sensible until we develop a more precise fix.

Applied
#12 lore raw

Re: [Qemu-devel] [PATCH 2/2] atapi migration: Throw recoverable error to avoid recovery

Am 16.01.2015 um 18:28 hat John Snow geschrieben:

Quoted 22 lines · Dr. David Alan Gilbert (git)
Quoted 19 lines

From: "Dr. David Alan Gilbert" <[email protected]>

(With the previous atapi_dma flag recovery)
If migration happens between the ATAPI command being written and the bmdma being started, the DMA is dropped. Eventually the guest times out and recovers, but that can take many seconds.
(This is rare, on a pingpong reading the CD continuously I hit this about ~1/30-1/50 migrates)

I don't think we've got enough state to be able to recover safely at this point, so I throw a 'medium error, no seek complete' that I'm assuming guests will try and recover from an apparently dirty CD.

OK, it's a hack, the real solution is probably to push a lot of ATAPI state into the migration stream, but this is a fix that works with no stream changes. Tested only on Linux (both RHEL5 (pre-libata) and RHEL7).

Quoted 5 lines
Quoted 1 line

I think this workaround is sensible until we develop a more precise fix.

Thanks, applied all to the block branch.

Kevin

#13 lore raw
In reply to Dr. David Alan Gilbert (git)

Oops, forgot to include Kevin and Stefan on cc for this.

Dave

Quoted 27 lines · * Dr. David Alan Gilbert (git) ([email protected]) wrote:

From: "Dr. David Alan Gilbert" <[email protected]>

This pair of patches fixes a problem where IDE/ATAPI cdrom reads get lost/corrupted over migration.

The first of the patches (restore atapi_dma flag) is a simple fix that I think is safe; it no longer causes corruption in the case we saw, but does still trigger a long timeout.

The second is a hack; it throws a medium error causing the guest to retry the command in the case where migration happens just between the IDE/ATAPI command being submitted
and the bmdma being finished. This recovers a lot
faster than the timeout.

Only tried on Linux guests so far; I think it might be possible to replace both of these by reparsing the command buffer for ATAPI; I'm just not confident I know when that's safe to do, and I wanted to see how disgusted people were by the 2nd hack.

Dave

Dr. David Alan Gilbert (2):
Restore atapi_dma flag across migration
atapi migration: Throw recoverable error to avoid recovery

Quoted 10 lines
 hw/ide/atapi.c    | 17 +++++++++++++++++
 hw/ide/core.c     |  1 +
 hw/ide/internal.h |  2 ++
 hw/ide/pci.c      | 11 +++++++++++
 4 files changed, 31 insertions(+)
Signature · 1 line
2.1.0

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

#14 lore raw
Quoted 2 lines · * Dr. David Alan Gilbert ([email protected]) wrote:

Oops, forgot to include Kevin and Stefan on cc for this.

Ping;
John R-b'd the two patches:
https://lists.gnu.org/archive/html/qemu-devel/2014-12/msg01133.html
http://lists.gnu.org/archive/html/qemu-devel/2015-01/msg01956.html

although hmm, possibly before I added both of you on the cc.

Dave

Quoted 37 lines

Dave

Quoted 33 lines · * Dr. David Alan Gilbert (git) ([email protected]) wrote:

From: "Dr. David Alan Gilbert" <[email protected]>

This pair of patches fixes a problem where IDE/ATAPI cdrom reads get lost/corrupted over migration.

The first of the patches (restore atapi_dma flag) is a simple fix that I think is safe; it no longer causes corruption in the case we saw, but does still trigger a long timeout.

The second is a hack; it throws a medium error causing the guest to retry the command in the case where migration happens just between the IDE/ATAPI command being submitted
and the bmdma being finished. This recovers a lot
faster than the timeout.

Only tried on Linux guests so far; I think it might be possible to replace both of these by reparsing the command buffer for ATAPI; I'm just not confident I know when that's safe to do, and I wanted to see how disgusted people were by the 2nd hack.

Dave

Dr. David Alan Gilbert (2):
Restore atapi_dma flag across migration
atapi migration: Throw recoverable error to avoid recovery

 hw/ide/atapi.c    | 17 +++++++++++++++++
 hw/ide/core.c     |  1 +
 hw/ide/internal.h |  2 ++
 hw/ide/pci.c      | 11 +++++++++++
 4 files changed, 31 insertions(+)
Quoted 6 lines
Quoted 4 lines
Signature · 1 line
2.1.0

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK