RFC Deterministic replay and reverse execution

This set of patches is related to the reverse execution and deterministic replay of qemu execution Our implementation of deterministic replay can be used for deterministic and reverse debugging of guest code through gdb remote interface.

Execution recording writes non-deterministic events log, which can be later used for replaying the execution anywhere and for unlimited number of times. It also supports checkpointing for faster rewinding during reverse debugging. Execution replaying reads the log and replays all non-deterministic events including external input, hardware clocks, and interrupts.

Reverse execution has the following features:
* Deterministically replays whole system execution and all contents of the memory, state of the hadrware devices, clocks, and screen of the VM.
* Writes execution log into the file for latter replaying for multiple times on different machines.
* Supports i386, x86_64, and ARM hardware platforms.
* Performs deterministic replay of all operations with keyboard, mouse, network adapters, audio devices, serial interfaces, and physical USB devices connected to the emulator.
* Provides support for gdb reverse debugging commands like reverse-step and reverse-continue.
* Supports auto-checkpointing for convenient reverse debugging.

Usage of the record/replay:
* First, record the execution, by adding '-record fname=replay.bin' to the command line.
* Then you can replay it for the multiple times by using another command line option: '-replay fname=replay.bin'
* Virtual machine should have at least one virtual disk, which is used to store checkpoints. If you want to enable automatic checkpointing, simply add ',period=XX' to record options, where XX is the checkpointing period in seconds.
* Using of the network adapters in record/replay mode is possible with the following command-line options:
- '-net user' (or another host adapter) in record mode
- '-net replay' in replay mode. Every host network adapter should be replaced by 'replay' when replaying the execution.
* Reverse debugging can be used through gdb remote interface. reverse-stepi and reverse-continue commands are supported. Other reverse commands should also work, because they reuse these ones.
* Monitor is extended by the following commands:
- replay_info - prints information about replay mode and current step (number of instructions executed)
- replay_break - sets "breakpoint" at the specified instructions count.
- replay_seek - rewinds (using the checkpoints, if possible) to the specified step of replay log.

Paper with short description of deterministic replay implementation:
http://www.computer.org/csdl/proceedings/csmr/2012/4666/00/4666a553-abs.html

Modifications of qemu include:
* adding missed fields of the virtual devices' states to the vmstate structures to allow deterministic saving and restoring the VM state
* adding virtual clock-based timers to vmstate structures, because virtual clock is the part of the virtual machine state
* modification of block layer to support automatic creation of the overlay files to store the changes and snapshots while recording
* disabling of system reset while loading VM state to avoid generating of interrupts by reset handlers
* adding warpers for clock and time functions to save their return values in the log
* saving different asynchronous events (e.g. system shutdown) into the log
* synchronization of the bottom halves execution
* synchronization of the threads from thread pool
* recording/replaying user input (mouse and keyboard), input from virtual serial ports, incoming network packets, input from connected USB devices
* adding HMP/QMP commands to monitor for controlling replay execution

v4 changes:
* Updated block drivers to support new bdrv_open interface.
* Moved migration patches into separate series (as suggested by Paolo Bonzini)
* Fixed a bug in replay_break operation.
* Fixed rtl8139 migration for replay.
* Fixed 'period' parameter processing for record mode.
* Fixed bug in 'reverse-stepi' implementation.
* Fixed replay without making any snapshots (even the starting one).
* Moved core replay patches into the separate series.
* Fixed reverse step and reverse continue support.

v3 changes:
* Fixed bug with replay of the aio write operations.
* Added virtual clock based on replay icount.
* Removed duplicated saving of interrupt_request CPU field.
* Fixed some coding style issues.
* Renamed QMP commands for controlling reverse execution (as suggested by Eric Blake)
* Replay mode and submode implemented as QAPI enumerations (as suggested by Eric Blake)
* Added description and example for replay-info command (as suggested by Eric Blake)
* Added information about the current breakpoint to the output of replay-info (as suggested by Eric Blake)
* Updated version id for HPET vmstate (as suggested by Paolo Bonzini)
* Removed static fields from parallel vmstate (as suggested by Paolo Bonzini)
* New vmstate fields for mc146818rtc, pckbd, kvmapic, serial, fdc, rtl8139 moved to subsection (as suggested by Paolo Bonzini)
* Disabled textmode cursor blinking, when virtual machine is stopped (as suggested by Paolo Bonzini)
* Extracted saving of exception_index to separate patch (as suggested by Paolo Bonzini)

v2 changes:
* Patches are split to be reviewable and bisectable (as suggested by Kirill Batuzov)
* Added QMP versions of replay commands (as suggested by Eric Blake)
* Removed some optional features of replay to make patches cleaner
* Minor changes and code cleanup were made

---

Paolo Bonzini (4):

      From 7abf2f72777958d395cfd01d97fe707cc06152b5 Mon Sep 17 00:00:00 2001
      From 185a3a47d08857a66332ae862b372a153ce92bb9 Mon Sep 17 00:00:00 2001
      From a0cb9e80ba0de409b5ad556109a1c71ce4d8ce19 Mon Sep 17 00:00:00 2001
      From 04bbd21134dd2c6b7309a7f5f2b780aae2757003 Mon Sep 17 00:00:00 2001

Pavel Dovgalyuk (27):
cpu-exec: fix cpu_exec_nocache
acpi: accurate overflow check
replay: global variables and function stubs
sysemu: system functions for replay
replay: internal functions for replay log
cpu-exec: reset exception_index correctly
icount: implement icount requesting
icount: improve enable/disable ticks
replay: introduce icount event
i386: do not cross the pages boundaries in replay mode
cpu-exec: allow temporary disabling icount
cpu-exec: invalidate nocache translation if they are interrupted
replay: interrupts and exceptions
replay: asynchronous events infrastructure
cpu: replay instructions sequence
replay: recording and replaying clock ticks
replay: recording and replaying different timers
timer: introduce new QEMU_CLOCK_VIRTUAL_RT clock
cpus: make icount warp deterministic in replay mode
replay: shutdown event
replay: checkpoints
replay: bottom halves
replay: replay aio requests
replay: thread pool
replay: initialization and deinitialization
replay: command line options
replay: recording of the user input

 Makefile.target                |    1 
 async.c                        |   46 +++++-
 block.c                        |   92 +++++++++++-
 block/block-backend.c          |   30 ++++
 block/qcow2.c                  |    4 +
 block/raw-posix.c              |    6 +
 block/raw-win32.c              |    4 -
 cpu-exec.c                     |   50 +++++-
 cpus.c                         |   97 +++++++++---
 dma-helpers.c                  |   10 +
 exec.c                         |    1 
 hw/acpi/core.c                 |    7 +
 hw/block/virtio-blk.c          |   10 +
 hw/ide/ahci.c                  |    4 -
 hw/ide/atapi.c                 |   10 +
 hw/ide/core.c                  |   18 ++
 hw/timer/arm_timer.c           |    2 
 hw/timer/mc146818rtc.c         |   10 +
 hw/timer/pl031.c               |   10 +
 hw/usb/hcd-uhci.c              |    2 
 include/block/aio.h            |   18 ++
 include/block/block.h          |   15 ++
 include/block/thread-pool.h    |    4 -
 include/exec/exec-all.h        |    8 +
 include/exec/gen-icount.h      |    6 -
 include/qemu-common.h          |    3 
 include/qemu/main-loop.h       |    1 
 include/qemu/timer.h           |   34 ++++
 include/qom/cpu.h              |   10 +
 include/sysemu/block-backend.h |   10 +
 include/sysemu/cpus.h          |    1 
 include/sysemu/sysemu.h        |    1 
 include/ui/input.h             |    2 
 main-loop.c                    |   10 +
 qapi-schema.json               |   32 ++++
 qemu-io-cmds.c                 |    2 
 qemu-options.hx                |   27 +++
 qemu-timer.c                   |   51 +++++-
 replay/Makefile.objs           |    5 +
 replay/replay-events.c         |  292 +++++++++++++++++++++++++++++++++++++
 replay/replay-input.c          |  108 ++++++++++++++
 replay/replay-internal.c       |  155 ++++++++++++++++++++
 replay/replay-internal.h       |  132 +++++++++++++++++
 replay/replay-time.c           |  191 ++++++++++++++++++++++++
 replay/replay.c                |  314 ++++++++++++++++++++++++++++++++++++++++
 replay/replay.h                |  119 +++++++++++++++
 stubs/Makefile.objs            |    1 
 stubs/replay.c                 |   42 +++++
 target-alpha/translate.c       |   10 +
 target-arm/translate-a64.c     |    6 -
 target-arm/translate.c         |    6 -
 target-cris/translate.c        |    2 
 target-i386/cpu.h              |    3 
 target-i386/translate.c        |   62 +++++---
 target-lm32/translate.c        |   10 +
 target-m68k/translate.c        |    2 
 target-microblaze/translate.c  |    2 
 target-mips/translate.c        |   26 ++-
 target-moxie/translate.c       |    2 
 target-openrisc/translate.c    |    2 
 target-ppc/cpu.h               |   13 +-
 target-ppc/translate.c         |   12 +-
 target-ppc/translate_init.c    |  271 +++++++++++++++++------------------
 target-s390x/translate.c       |    2 
 target-sh4/translate.c         |    2 
 target-sparc/translate.c       |    2 
 target-tricore/translate.c     |    2 
 target-unicore32/translate.c   |    2 
 target-xtensa/translate.c      |    2 
 tests/test-thread-pool.c       |    7 +
 thread-pool.c                  |   49 ++++--
 trace-events                   |    2 
 translate-all.c                |   19 ++
 ui/input.c                     |   80 ++++++++--
 util/iov.c                     |    4 +
 vl.c                           |  112 +++++++++++++-
 76 files changed, 2365 insertions(+), 357 deletions(-)
 create mode 100755 replay/Makefile.objs
 create mode 100755 replay/replay-events.c
 create mode 100755 replay/replay-input.c
 create mode 100755 replay/replay-internal.c
 create mode 100755 replay/replay-internal.h
 create mode 100755 replay/replay-time.c
 create mode 100755 replay/replay.c
 create mode 100755 replay/replay.h
 create mode 100755 stubs/replay.c

Pavel Dovgalyuk