Introduce attributes for timers subsystem and remove QEMU_CLOCK_VIRTUAL_EXT clock type
v3 cover Introduce attributes for timers subsystem and remove QEMU_CLOCK_VIRTUAL_EXT clock type
Recent patches from series [PATCH v6] "Fixing record/replay and adding reverse debugging" introduced new clock type QEMU_CLOCK_VIRTUAL_EXT and replaced virtual timers in some external subsystems with it.
This resulted in small change to existing behavior, which I consider to be unacceptable.
Processing of virtual timers, belonging to new clock type, was kicked off to main loop, which made them asynchronous with vCPU thread and, in icount mode, with whole guest execution. This breaks expected determinism in non-record/replay icount mode of emulation where these "external subsystems" are isolated from host (i.e. they external only to guest core, not to emulation environment).
Example for slirp ("user" backend for network device):
User runs qemu in icount mode with rtc clock=vm without any external communication interfaces but with "-netdev user,restrict=on". It expects deterministic execution, because network services are emulated inside qemu and isolated from host. There are no reasons to get reply from DHCP server with different delay or something like that.
These series of patches revert those commits and reimplement their modifications in a better way.
Current implementation of timers/clock processing is confusing (at least for me) because of exceptions from design concept behind them, which already introduced by icount mode (which adds QEMU_CLOCK_VIRTUAL_RT). Adding QEMU_CLOCK_VIRTUAL_EXT just made things even more complicated. I consider these "alternative" virtual clocks to be some kind of hacks being convinient only to authors of relevant qemu features. Lets don't touch fundamental clock types and keep them orthogonal to special cases of timers handling.
As far as I understand, original intention of author was just to make optimization in replay log by avoiding storing extra events which don't change guest state directly. Indeed, for example, ipv6 icmp timer in slirp does things which external to guest core and ends with sending network packet to guest, but record/replay will anyway catch event, corresponding to packet reception in guest network frontend, and store it to replay log, so there are no need in making checkpoint for corresponding clock when that timer fires. If so, then we just need to skip checkpoints for clock values, when only these specific timers are run. It is individual timers which are specific, not clock.
Adding some kind of attribute/flag/property to individual timer allows any special qemu feature (such as record/replay) to inspect it and handle as needed. This design achieves less dependencies, more transparency and has more intuitive and clear sense. For record/replay feature it's achieved with 'EXTERNAL' attribute. The only drawback is that it required to add one extra mutex unlock/lock pair for virtual clock type in timerlist_run_timers() function (see patch 3), but it's being added only when qemu runs in record/replay mode.
Finally, this patch series optimizes checkpointing for all other clocks it applies to.
v3 changes:
- fixed failed build in last patch
- attributes has been refactored and restricted to be used only within qemu-timer (as suggested by Stefan Hajnoczi and Paolo Bonzini)
v2 changes:
- timers/aio interface refactored and improved, addition to couroutines removed (as Paolo Bonzini suggested)
- fixed wrong reimplementation in qemu-timer.c caused race conditions
- added bonus patch which optimizes checkpointing for other clocks
P.S. I've tried to test record/replay with slirp, but in replay mode qemu stucks at guest linux boot after "Configuring network interfaces..." message, where DHCP communication takes place. It's broken in a same way both in master and master with reverted commits being fixed.
P.S.2. I wasn't able to test these patches on purely clean master branch because of bugs https://bugs.launchpad.net/qemu/+bug/1790460 and https://bugs.launchpad.net/qemu/+bug/1795369, which workarounded by several not-accepted (yet?) modifications.
Artem Pisarenko (4):
Revert some patches from recent [PATCH v6] "Fixing record/replay and
adding reverse debugging"
Introduce attributes to qemu timer subsystem
Restores record/replay behavior related to special virtual clock
processing for timers used in external subsystems.
Optimize record/replay checkpointing for all clocks it applies to
include/block/aio.h | 59 ++++++++++++++++++--- include/qemu/timer.h | 128 +++++++++++++++++++++++----------------------- slirp/ip6_icmp.c | 9 ++-- tests/ptimer-test-stubs.c | 13 +++-- ui/input.c | 9 ++-- util/qemu-timer.c | 95 +++++++++++++++++++++++----------- 6 files changed, 201 insertions(+), 112 deletions(-)
Signature · 1 line
2.7.4
PATCH v3 1/4 Revert some patches from recent [PATCH v6] "Fixing record/replay and adding reverse debugging"
That patch series introduced new virtual clock type for use in external subsystems. It breaks desired behavior in non-record/replay usage scenarios.
This reverts commit 87f4fe7653baf55b5c2f2753fe6003f473c07342. This reverts commit 775a412bf83f6bc0c5c02091ee06cf649b34c593. This reverts commit 9888091404a702d7ec79d51b088d994b9fc121bd.
--- include/qemu/timer.h | 9 --------- slirp/ip6_icmp.c | 7 +++---- ui/input.c | 8 ++++---- util/qemu-timer.c | 2 -- 4 files changed, 7 insertions(+), 19 deletions(-)
include/qemu/timer.h +0 -9
@@ -42,14 +42,6 @@* In icount mode, this clock counts nanoseconds while the virtual * machine is running. It is used to increase @QEMU_CLOCK_VIRTUAL * while the CPUs are sleeping and thus not executing instructions. -* -* @QEMU_CLOCK_VIRTUAL_EXT: virtual clock for external subsystems -* -* The virtual clock only runs during the emulation. It stops -* when the virtual machine is stopped. The timers for this clock -* do not recorded in rr mode, therefore this clock could be used -* for the subsystems that operate outside the guest core. -* */ typedef enum {
@@ -57,7 +49,6 @@ typedef enum {QEMU_CLOCK_VIRTUAL =1 ,QEMU_CLOCK_HOST =2 ,QEMU_CLOCK_VIRTUAL_RT =3 , -QEMU_CLOCK_VIRTUAL_EXT =4 , QEMU_CLOCK_MAX } QEMUClockType;
slirp/ip6_icmp.c +3 -4
@@ -17,7 +17,7 @@ static void ra_timer_handler(void *opaque) {Slirp *slirp = opaque;timer_mod (slirp->ra_timer, -qemu_clock_get_ms (QEMU_CLOCK_VIRTUAL_EXT ) + NDP_Interval); +qemu_clock_get_ms (QEMU_CLOCK_VIRTUAL ) + NDP_Interval);ndp_send_ra (slirp); }
@@ -27,10 +27,9 @@ void icmp6_init(Slirp *slirp)return ; } - slirp->ra_timer =timer_new_ms (QEMU_CLOCK_VIRTUAL_EXT , - ra_timer_handler, slirp); + slirp->ra_timer =timer_new_ms (QEMU_CLOCK_VIRTUAL , ra_timer_handler, slirp);timer_mod (slirp->ra_timer, -qemu_clock_get_ms (QEMU_CLOCK_VIRTUAL_EXT ) + NDP_Interval); +qemu_clock_get_ms (QEMU_CLOCK_VIRTUAL ) + NDP_Interval); }void icmp6_cleanup (Slirp *slirp)
ui/input.c +4 -4
@@ -271,7 +271,7 @@ static void qemu_input_queue_process(void *opaque) item =QTAILQ_FIRST (queue);switch (item->type) {case QEMU_INPUT_QUEUE_DELAY : -timer_mod (item->timer,qemu_clock_get_ms (QEMU_CLOCK_VIRTUAL_EXT ) +timer_mod (item->timer,qemu_clock_get_ms (QEMU_CLOCK_VIRTUAL ) + item->delay_ms);return ;case QEMU_INPUT_QUEUE_EVENT :
@@ -301,7 +301,7 @@ static void qemu_input_queue_delay(struct QemuInputEventQueueHead *queue, queue_count++;if (start_timer) { -timer_mod (item->timer,qemu_clock_get_ms (QEMU_CLOCK_VIRTUAL_EXT ) +timer_mod (item->timer,qemu_clock_get_ms (QEMU_CLOCK_VIRTUAL ) + item->delay_ms); } }
@@ -448,8 +448,8 @@ void qemu_input_event_send_key_delay(uint32_t delay_ms) }if (!kbd_timer) { - kbd_timer =timer_new_ms (QEMU_CLOCK_VIRTUAL_EXT , - qemu_input_queue_process, &kbd_queue); + kbd_timer =timer_new_ms (QEMU_CLOCK_VIRTUAL , qemu_input_queue_process, + &kbd_queue); }if (queue_count < queue_limit) { qemu_input_queue_delay(&kbd_queue, kbd_timer,
util/qemu-timer.c +0 -2
@@ -496,7 +496,6 @@ bool timerlist_run_timers(QEMUTimerList *timer_list)switch (timer_list->clock->type) {case QEMU_CLOCK_REALTIME : -case QEMU_CLOCK_VIRTUAL_EXT :break ;default :case QEMU_CLOCK_VIRTUAL :
@@ -598,7 +597,6 @@ int64_t qemu_clock_get_ns(QEMUClockType type)return get_clock ();default :case QEMU_CLOCK_VIRTUAL : -case QEMU_CLOCK_VIRTUAL_EXT :if (use_icount) {return cpu_get_icount (); }else {
Signature · 1 line
2.7.4
PATCH v3 2/4 Introduce attributes to qemu timer subsystem
In reply to Artem PisarenkoAttributes are simple flags, associated with individual timers for their whole lifetime.
They intended to be used to mark individual timers for special handling by various qemu features which have integration into qemu-timer.
New/init functions family in timer interface updated and refactored (new 'attribute' argument added, timer_list replaced with timer_list_group+type combinations, comments improved to avoid info duplication). Also existing aio interface extended with attribute-enabled variants of functions, which create/initialize timers.
---
Notes:
v3:
- attributes has been properly incapsulated to qemu-timer (as suggested by Stefan Hajnoczi)
- attributes definition and docs refactored to avoid extra enum and use simple macros with explicit bit positions (as suggested by Stefan Hajnoczi and Paolo Bonzini)
- fixed old "QEMU_TIMER_ATTR(id)" notation (in comments) left from initial patch version
v2:
- timer creation/initialize functions reworked and and their unnecessary variants removed (as Paolo Bonzini suggested)
- also their comments improved to avoid info duplication
include/block/aio.h | 59 ++++++++++++++++++++++--- include/qemu/timer.h | 110 +++++++++++++++++++++++----------------------- tests/ptimer-test-stubs.c | 13 ++++-- util/qemu-timer.c | 13 ++++-- 4 files changed, 125 insertions(+), 70 deletions(-)
include/block/aio.h +52 -7
@@ -388,6 +388,32 @@ struct LinuxAioState *aio_setup_linux_aio(AioContext *ctx, Error **errp);struct LinuxAioState *aio_get_linux_aio (AioContext *ctx);/** +* aio_timer_new_with_attrs: +* @ctx: the aio context +* @type: the clock type +* @scale: the scale +* @attributes: 0, or one to multiple OR'ed QEMU_TIMER_ATTR_<id> values +* to assign +* @cb: the callback to call on timer expiry +* @opaque: the opaque pointer to pass to the callback +* +* Allocate a new timer (with attributes) attached to the context @ctx. +* The function is responsible for memory allocation. +* +* The preferred interface is aio_timer_init or aio_timer_init_with_attrs. +* Use that unless you really need dynamic memory allocation. +* +* Returns: a pointer to the new timer +*/ +static inline QEMUTimer *aio_timer_new_with_attrs (AioContext *ctx, +QEMUClockType type, +int scale,int attributes, +QEMUTimerCB *cb,void *opaque) +{ +return timer_new_full (&ctx->tlg, type, scale, attributes, cb, opaque); +} + +/** * aio_timer_new: * @ctx: the aio context * @type: the clock type
@@ -396,10 +422,7 @@ struct LinuxAioState *aio_get_linux_aio(AioContext *ctx);* @opaque: the opaque pointer to pass to the callback * * Allocate a new timer attached to the context @ctx. -* The function is responsible for memory allocation. -* -* The preferred interface is aio_timer_init. Use that -* unless you really need dynamic memory allocation. +* See aio_timer_new_with_attrs for details. * * Returns: a pointer to the new timer */
@@ -407,7 +430,29 @@ static inline QEMUTimer *aio_timer_new(AioContext *ctx, QEMUClockType type,int scale, QEMUTimerCB *cb, void *opaque) { - return timer_new_tl(ctx->tlg.tl[type], scale, cb, opaque); + return timer_new_full(&ctx->tlg, type, scale, 0, cb, opaque); +} + +/** +* aio_timer_init_with_attrs: +* @ctx: the aio context +* @ts: the timer +* @type: the clock type +* @scale: the scale +* @attributes: 0, or one to multiple OR'ed QEMU_TIMER_ATTR_<id> values +* to assign +* @cb: the callback to call on timer expiry +* @opaque: the opaque pointer to pass to the callback +* +* Initialise a new timer (with attributes) attached to the context @ctx. +* The caller is responsible for memory allocation. +*/ +static inline void aio_timer_init_with_attrs (AioContext *ctx, +QEMUTimer *ts,QEMUClockType type, +int scale,int attributes, +QEMUTimerCB *cb,void *opaque) +{ +timer_init_full (ts, &ctx->tlg, type, scale, attributes, cb, opaque); }/**
@@ -420,14 +465,14 @@ static inline QEMUTimer *aio_timer_new(AioContext *ctx, QEMUClockType type,* @opaque: the opaque pointer to pass to the callback * * Initialise a new timer attached to the context @ctx. -* The caller is responsible for memory allocation. +* See aio_timer_init_with_attrs for details. */ static inline void aio_timer_init (AioContext *ctx,QEMUTimer *ts,QEMUClockType type,int scale,QEMUTimerCB *cb,void *opaque) { -timer_init_tl (ts, ctx->tlg.tl[type], scale, cb, opaque); +timer_init_full (ts, &ctx->tlg, type, scale,0 , cb, opaque); }/**
include/qemu/timer.h +55 -55
218 lines not on this page · raw message
tests/ptimer-test-stubs.c +9 -4
@@ -34,14 +34,19 @@ int64_t ptimer_test_time_ns;int use_icount =1 ;bool qtest_allowed; -void timer_init_tl (QEMUTimer *ts, -QEMUTimerList *timer_list,int scale, -QEMUTimerCB *cb,void *opaque) +void timer_init_full (QEMUTimer *ts, +QEMUTimerListGroup *timer_list_group,QEMUClockType type, +int scale,int attributes, +QEMUTimerCB *cb,void *opaque) { - ts->timer_list = timer_list; +if (!timer_list_group) { + timer_list_group = &main_loop_tlg; + } + ts->timer_list = timer_list_group->tl[type]; ts->cb = cb; ts->opaque = opaque; ts->scale = scale; + ts->attributes = attributes; ts->expire_time =-1 ; }
util/qemu-timer.c +9 -4
@@ -339,14 +339,19 @@ int qemu_poll_ns(GPollFD *fds, guint nfds, int64_t timeout) } -void timer_init_tl (QEMUTimer *ts, -QEMUTimerList *timer_list,int scale, -QEMUTimerCB *cb,void *opaque) +void timer_init_full (QEMUTimer *ts, +QEMUTimerListGroup *timer_list_group,QEMUClockType type, +int scale,int attributes, +QEMUTimerCB *cb,void *opaque) { - ts->timer_list = timer_list; +if (!timer_list_group) { + timer_list_group = &main_loop_tlg; + } + ts->timer_list = timer_list_group->tl[type]; ts->cb = cb; ts->opaque = opaque; ts->scale = scale; + ts->attributes = attributes; ts->expire_time =-1 ; }
Signature · 1 line
2.7.4
Re: [Qemu-devel] [PATCH v3 2/4] Introduce attributes to qemu timer subsystem
Quoted 18 lines · Artem Pisarenko
Attributes are simple flags, associated with individual timers for their whole lifetime.
They intended to be used to mark individual timers for special handling by various qemu features which have integration into qemu-timer.
New/init functions family in timer interface updated and refactored (new 'attribute' argument added,timer_listreplaced with timer_list_group+type combinations, comments improved to avoid info duplication). Also existing aio interface extended with attribute-enabled variants of functions, which create/initialize timers.Signed-off-by Artem Pisarenko <[email protected]>---
Notes:
v3:
- attributes has been properly incapsulated to qemu-timer (as suggested by Stefan Hajnoczi)
- attributes definition and docs refactored to avoid extra enum and use simple macros with explicit bit positions (as suggested by Stefan Hajnoczi and Paolo Bonzini)
- fixed old "QEMU_TIMER_ATTR(id)" notation (in comments) left from initial patch versionv2:
- timer creation/initialize functions reworked and and their unnecessary variants removed (as Paolo Bonzini suggested)
- also their comments improved to avoid info duplication
Quoted 5 lines
include/block/aio.h | 59 ++++++++++++++++++++++--- include/qemu/timer.h | 110 +++++++++++++++++++++++----------------------- tests/ptimer-test-stubs.c | 13 ++++-- util/qemu-timer.c | 13 ++++-- 4 files changed, 125 insertions(+), 70 deletions(-)
PATCH v3 3/4 Restores record/replay behavior related to special virtual clock processing for timers used in external subsystems.
In reply to Artem PisarenkoAdds EXTERNAL attribute definition to qemu timers subsystem and assigns it to virtual clock timers, used in slirp (ICMP IPv6) and ui (key queue). Virtual clock processing in rr mode reimplemented using this attribute.
--- include/qemu/timer.h | 11 ++++++++++- slirp/ip6_icmp.c | 4 +++- ui/input.c | 5 +++-- util/qemu-timer.c | 50 +++++++++++++++++++++++++++++++++++++++++++------- 4 files changed, 59 insertions(+), 11 deletions(-)
include/qemu/timer.h +10 -1
@@ -2,6 +2,7 @@#define QEMU_TIMER_H #include "qemu-common.h" +#include "qemu/bitops.h" #include "qemu/notify.h" #include "qemu/host-utils.h"
@@ -59,9 +60,17 @@ typedef enum {* Each attribute corresponds to one bit. Attributes modify the processing * of timers when they fire. * -* No attributes defined currently. +* The following attributes are available: +* +* QEMU_TIMER_ATTR_EXTERNAL: drives external subsystem +* +* Timers with this attribute do not recorded in rr mode, therefore it could be +* used for the subsystems that operate outside the guest core. Applicable only +* with virtual clock type. */ +#define QEMU_TIMER_ATTR_EXTERNAL BIT(0) +typedef struct QEMUTimerList QEMUTimerList ;struct QEMUTimerListGroup {
slirp/ip6_icmp.c +3 -1
@@ -27,7 +27,9 @@ void icmp6_init(Slirp *slirp)return ; } - slirp->ra_timer =timer_new_ms (QEMU_CLOCK_VIRTUAL , ra_timer_handler, slirp); + slirp->ra_timer =timer_new_full (NULL ,QEMU_CLOCK_VIRTUAL , +SCALE_MS ,QEMU_TIMER_ATTR_EXTERNAL , + ra_timer_handler, slirp);timer_mod (slirp->ra_timer,qemu_clock_get_ms (QEMU_CLOCK_VIRTUAL ) + NDP_Interval); }
ui/input.c +3 -2
@@ -448,8 +448,9 @@ void qemu_input_event_send_key_delay(uint32_t delay_ms) }if (!kbd_timer) { - kbd_timer =timer_new_ms (QEMU_CLOCK_VIRTUAL , qemu_input_queue_process, - &kbd_queue); + kbd_timer =timer_new_full (NULL ,QEMU_CLOCK_VIRTUAL , +SCALE_MS ,QEMU_TIMER_ATTR_EXTERNAL , + qemu_input_queue_process, &kbd_queue); }if (queue_count < queue_limit) { qemu_input_queue_delay(&kbd_queue, kbd_timer,
util/qemu-timer.c +43 -7
82 lines not on this page · raw message
Signature · 1 line
2.7.4
PATCH v3 4/4 Optimize record/replay checkpointing for all clocks it applies to
In reply to Artem PisarenkoRemoves redundant checkpoints in replay log when there are no expired timers in timers list, associated with corresponding clock (i.e. no rr events associated with current clock value). This also improves performance in rr mode.
---
Notes:
v3:
- fixed compiler warning caused non-debug build to fail
include/qemu/timer.h | 2 +- util/qemu-timer.c | 62 +++++++++++++++++++++++++--------------------------- 2 files changed, 31 insertions(+), 33 deletions(-)
include/qemu/timer.h +1 -1
@@ -65,7 +65,7 @@ typedef enum {* QEMU_TIMER_ATTR_EXTERNAL: drives external subsystem * * Timers with this attribute do not recorded in rr mode, therefore it could be -* used for the subsystems that operate outside the guest core. Applicable only +* used for the subsystems that operate outside the guest core. Relevant only * with virtual clock type. */
util/qemu-timer.c +30 -32
86 lines not on this page · raw message
Signature · 1 line
2.7.4
Re: [Qemu-devel] [PATCH v3 4/4] Optimize record/replay checkpointing for all clocks it applies to
Quoted 61 lines
From: Artem Pisarenko [mailto:[email protected]]
Removes redundant checkpoints in replay log when there are no expired timers in timers list, associated with corresponding clock (i.e. no rr events associated with current clock value). This also improves performance in rr mode.Signed-off-by Artem Pisarenko <[email protected]>---
Notes:
v3:
- fixed compiler warning caused non-debug build to failinclude/qemu/timer.h | 2 +- util/qemu-timer.c | 62 +++++++++++++++++++++++++--------------------------- 2 files changed, 31 insertions(+), 33 deletions(-)include/qemu/timer.h +1
-1@@ -65,7 +65,7 @@ typedef enum {* QEMU_TIMER_ATTR_EXTERNAL: drives external subsystem * * Timers with this attribute do not recorded in rr mode, therefore it could be -* used for the subsystems that operate outside the guest core. Applicable only +* used for the subsystems that operate outside the guest core. Relevant only * with virtual clock type. */ util/qemu-timer.c +4
-13@@ -490,6 +490,7 @@ bool timerlist_run_timers(QEMUTimerList *timer_list)QEMUTimerCB *cb;void *opaque;bool need_replay_checkpoint = false; +ReplayCheckpoint replay_checkpoint_id;if (!atomic_read (&timer_list->active_timers)) {return false;@@ -500,43 +501,40 @@ bool timerlist_run_timers(QEMUTimerList *timer_list) goto out; } -switch (timer_list->clock->type) { -case QEMU_CLOCK_REALTIME : -break ; -default : -case QEMU_CLOCK_VIRTUAL : -if (replay_mode !=REPLAY_MODE_NONE ) { -/* Checkpoint for virtual clock is redundant in cases where -* it's being triggered with only non-EXTERNAL timers, because -* these timers don't change guest state directly. -* Since it has conditional dependence on specific timers, it is -* subject to race conditions and requires special handling. -* See below. -*/ +if (replay_mode !=REPLAY_MODE_NONE ) { +/* Postpone actual checkpointing to timer list processing +* to properly check if we actually need it.
util/qemu-timer.c bool timerlist_run_timers(QEMUTimerList *timer_list) · line 516
Quoted 10 lines
util/qemu-timer.c +9
-0+*/ +switch (timer_list->clock->type) { +case QEMU_CLOCK_VIRTUAL : need_replay_checkpoint = true; + replay_checkpoint_id =CHECKPOINT_CLOCK_VIRTUAL ; +break ; +case QEMU_CLOCK_HOST : + need_replay_checkpoint = true; + replay_checkpoint_id =CHECKPOINT_CLOCK_HOST ; +break ;
This is wrong at least for QEMU_CLOCK_HOST.
Quoted 31 lines
util/qemu-timer.c +8
-18+case QEMU_CLOCK_VIRTUAL_RT : + need_replay_checkpoint = true; + replay_checkpoint_id =CHECKPOINT_CLOCK_VIRTUAL_RT ; +break ; +default : +break ; } -break ; -case QEMU_CLOCK_HOST : -if (!replay_checkpoint (CHECKPOINT_CLOCK_HOST )) { - goto out; - } -break ; -case QEMU_CLOCK_VIRTUAL_RT : -if (!replay_checkpoint (CHECKPOINT_CLOCK_VIRTUAL_RT )) { - goto out; - } -break ; }/* -* Extract expired timers from active timers list and and process them. +* Extract expired timers from active timers list and and process them, +* taking into account checkpointing required in rr mode. * -* In rr mode we need "filtered" checkpointing for virtual clock. -* Checkpoint must be replayed before any non-EXTERNAL timer has been -* processed and only one time (virtual clock value stays same). But these -* timers may appear in the timers list while it being processed, so this -* must be checked until we finally decide that "no timers left - we are -* done".
util/qemu-timer.c bool timerlist_run_timers(QEMUTimerList *timer_list) · line 539
Quoted 10 lines
util/qemu-timer.c +8
-0+* Checkpoint must be replayed before any timer has been processed +* and only one time. But new timers may appear in the timers list while +* it's being processed, so this must be checked until we finally decide +* that "no timers left - we are done" (to avoid skipping checkpoint due to +* possible races). +* Also checkpoint for virtual clock is redundant in cases where it's being +* triggered with only non-EXTERNAL timers, because these timers don't +* change guest state directly. */ current_time =qemu_clock_get_ns (timer_list->clock->type);
Reading the host clock here is not protected by the checkpoint. Therefore it may incur the inconsistency when replaying the execution.
Quoted 11 lines
util/qemu-timer.c +1
-2qemu_mutex_lock (&timer_list->active_timers_lock);@@ -552,7 +550,7 @@ bool timerlist_run_timers(QEMUTimerList *timer_list)/* once we got here, checkpoint clock only once */ need_replay_checkpoint = false;qemu_mutex_unlock (&timer_list->active_timers_lock); -if (!replay_checkpoint (CHECKPOINT_CLOCK_VIRTUAL )) { +if (!replay_checkpoint (replay_checkpoint_id)) { goto out; }qemu_mutex_lock (&timer_list->active_timers_lock); --
Quoted 1 line
2.7.4
Pavel Dovgalyuk
Re: [Qemu-devel] [PATCH v3 4/4] Optimize record/replay checkpointing for all clocks it applies to
Quoted 5 lines
…
This is wrong at least forQEMU_CLOCK_HOST.
…
Reading the host clock here is not protected by the checkpoint. Therefore it may incur the inconsistency when replaying the execution.
That's why I didn't like idea of this patch and asked for any possible side effects beforehand. So, here is one.
I have no idea how to fix it properly and would like to recall this particular patch from these series (remember, it doesn't match series goal). Lets leave this patch/issue for separate resolution, although I'm not very interested in it.
Re: [Qemu-devel] [PATCH v3 4/4] Optimize record/replay checkpointing for all clocks it applies to
Quoted 11 lines · Artem Pisarenko
Quoted 4 lines
This is wrong at least for
QEMU_CLOCK_HOST.
…
Reading the host clock here is not protected by the checkpoint. Therefore it may incur the inconsistency when replaying the execution.That's why I didn't like idea of this patch and asked for any possible side effects beforehand. So, here is one.
I have no idea how to fix it properly and would like to recall this particular patch from these series (remember, it doesn't match series goal). Lets leave this patch/issue for separate resolution, although I'm not very interested in it.
Yup, I'll send v2 of the pull request with this patch removed.
Paolo
PATCH v3 1/1 Optimize record/replay checkpointing for all clocks it applies to
In reply to Artem PisarenkoRemoves redundant checkpoints in replay log when there are no expired timers in timers list, associated with corresponding clock (i.e. no rr events associated with current clock value). This also improves performance in rr mode.
---
Oops, forgot to commit this fix
v3:
- fixed compiler warning caused non-debug build to fail
include/qemu/timer.h | 2 +- util/qemu-timer.c | 62 +++++++++++++++++++++++++--------------------------- 2 files changed, 31 insertions(+), 33 deletions(-)
include/qemu/timer.h +1 -1
@@ -65,7 +65,7 @@ typedef enum {* QEMU_TIMER_ATTR_EXTERNAL: drives external subsystem * * Timers with this attribute do not recorded in rr mode, therefore it could be -* used for the subsystems that operate outside the guest core. Applicable only +* used for the subsystems that operate outside the guest core. Relevant only * with virtual clock type. */
util/qemu-timer.c +30 -32
86 lines not on this page · raw message
Signature · 1 line
2.7.4
Re: [Qemu-devel] [PATCH v3] Optimize record/replay checkpointing for all clocks it applies to
Quoted 10 lines · Artem Pisarenko
Removes redundant checkpoints in replay log when there are no expired timers in timers list, associated with corresponding clock (i.e. no rr events associated with current clock value). This also improves performance in rr mode.
Signed-off-by Artem Pisarenko <[email protected]>---
Oops, forgot to commit this fix
v3:
- fixed compiler warning caused non-debug build to fail
We can also move the switch statement to a separate function, it simplifies the code:
util/qemu-timer.c +0 -0
@@ -482,6 +482,26 @@ bool timer_expired(QEMUTimer *timer_head, int64_t
current_time)
return timer_expired_ns(timer_head, current_time * timer_head->scale);
}
+static bool timer_checkpoint(QEMUClockType clock) +{
+ if (replay_mode != REPLAY_MODE_NONE) {
+ switch (clock) {
+ case QEMU_CLOCK_VIRTUAL:
+ return replay_checkpoint(CHECKPOINT_CLOCK_VIRTUAL);
+ case QEMU_CLOCK_HOST:
+ return replay_checkpoint(CHECKPOINT_CLOCK_HOST);
+ case QEMU_CLOCK_VIRTUAL_RT:
+ return replay_checkpoint(CHECKPOINT_CLOCK_VIRTUAL_RT);
+ default:
+ /* QEMU_CLOCK_REALTIME is external to the emulation and does
+ * not need checkpointing.
+ */
+ break;
+ }
+ }
+ return true;
+}
+
bool timerlist_run_timers(QEMUTimerList *timer_list)
{
QEMUTimer *ts;
@@ -489,8 +509,7 @@ bool timerlist_run_timers(QEMUTimerList *timer_list)
bool progress = false;
QEMUTimerCB *cb;
void *opaque;
- bool need_replay_checkpoint = false;
- ReplayCheckpoint replay_checkpoint_id;
+ bool need_replay_checkpoint = true; if (!atomic_read(&timer_list->active_timers)) {
return false;
@@ -501,28 +520,6 @@ bool timerlist_run_timers(QEMUTimerList *timer_list)
goto out;
}- if (replay_mode != REPLAY_MODE_NONE) {
- /* Postpone actual checkpointing to timer list processing
- * to properly check if we actually need it.
- */
- switch (timer_list->clock->type) {
- case QEMU_CLOCK_VIRTUAL:
- need_replay_checkpoint = true; - replay_checkpoint_id = CHECKPOINT_CLOCK_VIRTUAL; - break; - case QEMU_CLOCK_HOST: - need_replay_checkpoint = true; - replay_checkpoint_id = CHECKPOINT_CLOCK_HOST; - break; - case QEMU_CLOCK_VIRTUAL_RT: - need_replay_checkpoint = true; - replay_checkpoint_id = CHECKPOINT_CLOCK_VIRTUAL_RT; - break;
- default:
- break;
- }
- }
-
/*
* Extract expired timers from active timers list and and process them,
* taking into account checkpointing required in rr mode.
@@ -545,11 +542,11 @@ bool timerlist_run_timers(QEMUTimerList *timer_list)
break;
}
if (need_replay_checkpoint
&& !(ts->attributes & QEMU_TIMER_ATTR_EXTERNAL)) {
/* once we got here, checkpoint clock only once */
need_replay_checkpoint = false;
qemu_mutex_unlock(&timer_list->active_timers_lock);
- if (!replay_checkpoint(replay_checkpoint_id)) {
+ if (!timer_checkpoint(timer_list->clock->type)) {
goto out;
}
qemu_mutex_lock(&timer_list->active_timers_lock);No need to do anything on your part.
Paolo
Re: [Qemu-devel] [PATCH v3] Optimize record/replay checkpointing for all clocks it applies to
Quoted 3 lines
We can also move the switch statement to a separate function, it simplifies the code:
...
When I prepared this patch my intuition said me to add note in advance:
"Paolo, please, don't try to move this to a separate function. I've tried it already. It cannot be done correct, look nice and not decrease performancy at the same time". But I've ignored it... :) Change you did is correct and nice, but (compared to my version) it adds extra unlock/lock pair for running each timer list when it isn't empty and in non-rr mode (where we would ignore checkpoints and execution flow would bypass whole "if (need_replay_checkpoint) {...}" block).
Maybe you're aware of it, but I don't think that such change worth it.
Signature · 2 lines
С уважением, Артем Писаренко
Re: [Qemu-devel] [PATCH v3] Optimize record/replay checkpointing for all clocks it applies to
Quoted 4 lines · Artem Pisarenko
Quoted 3 lines
We can also move the switch statement to a separate function, it simplifies the code:
...
Quoted 9 lines
When I prepared this patch my intuition said me to add note in advance:
"Paolo, please, don't try to move this to a separate function. I've tried it already. It cannot be done correct, look nice and not decrease performancy at the same time". But I've ignored it... :) Change you did is correct and nice, but (compared to my version) it adds extra unlock/lock pair for running each timer list when it isn't empty and in non-rr mode (where we would ignore checkpoints and execution flow would bypass whole "if (need_replay_checkpoint) {...}" block). Maybe you're aware of it, but I don't think that such change worth it.
No, you're right. The if should remain in the caller, or need_replay_checkpoint must be initialized with replay_mode.
Paolo
Re: [Qemu-devel] [PATCH v3] Optimize record/replay checkpointing for all clocks it applies to
Quoted 16 lines
чт, 18 окт. 2018 г., 20:31 Paolo Bonzini:
Quoted 4 lines · Artem Pisarenko
Quoted 3 lines
We can also move the switch statement to a separate function, it simplifies the code:
...Quoted 9 lines
When I prepared this patch my intuition said me to add note in advance:
"Paolo, please, don't try to move this to a separate function. I've tried it already. It cannot be done correct, look nice and not decrease performancy at the same time". But I've ignored it... :) Change you did is correct and nice, but (compared to my version) it adds extra unlock/lock pair for running each timer list when it isn't empty and in non-rr mode (where we would ignore checkpoints and execution flow would bypass whole "if (need_replay_checkpoint) {...}" block). Maybe you're aware of it, but I don't think that such change worth it.
Quoted 2 lines
No, you're right. The if should remain in the caller, or
need_replay_checkpointmust be initialized withreplay_mode.
If initialize 'need_replay_checkpoint', then it should also account for clock != QEMU_CLOCK_REALTIME. And here we come to what if+switch block actually (mostly) does in my version. Finally, you will get duplication of this whole condition usage between source function and extracted function, which isn't nice.
Why do you want to split up such tightly coupled code?
Re: [Qemu-devel] [PATCH v3] Optimize record/replay checkpointing for all clocks it applies to
Quoted 6 lines · Artem Pisarenko
Quoted 3 lines
No, you're right. The if should remain in the caller, or
need_replay_checkpointmust be initialized withreplay_mode.If initialize '
need_replay_checkpoint', then it should also account for clock !=QEMU_CLOCK_REALTIME.
Or you just get a unlock/lock pair for QEMU_CLOCK_REALTIME (which should really never happen if e.g. you have no UI).
Quoted 5 lines
And here we come to what if+switch block
actually (mostly) does in my version. Finally, you will get duplication of this whole condition usage between source function and extracted function, which isn't nice.
Why do you want to split up such tightly coupled code?
Because it's *too* coupled and not very readable.
Paolo
Re: [Qemu-devel] [PATCH v3] Optimize record/replay checkpointing for all clocks it applies to
Quoted 11 lines
чт, 18 окт. 2018 г., 23:25 Paolo Bonzini:
Quoted 6 lines · Artem Pisarenko
Quoted 3 lines
No, you're right. The if should remain in the caller, or
need_replay_checkpointmust be initialized withreplay_mode.If initialize '
need_replay_checkpoint', then it should also account for clock !=QEMU_CLOCK_REALTIME.Or you just get a unlock/lock pair for
QEMU_CLOCK_REALTIME(which should really never happen if e.g. you have no UI).
And still have duplication (just smaller in this case).
Quoted 3 lines
Quoted 1 line
Why do you want to split up such tightly coupled code?
Because it's *too* coupled and not very readable.
Tightly coupled code in my understanding is property having its roots in design, which usually has wider context than piece of code in question. So we canot avoid this by definition (limiting changes only to this code). Trying to improve it is just like a playing with bubble wrap - pressing each bubble causes another bubble to pop up.
In our particular case it's because of overall rr design, qemu architecture and a way how rr integrated in timers. The best we can do in this case is to localise/quarantine ugly aspects as much as possible, carefully and plenteously comment and try don't touch them... never... I consider 'timerlist_run_timers()' is already totally infected and we're just late to save anyone of its residents (by isolating others).
Sorry, I forgot to fix long lines in commit messages. Do I need to submit v4 ?
Signature · 2 lines
С уважением, Артем Писаренко
Quoted 3 lines · Artem Pisarenko
Sorry, I forgot to fix long lines in commit messages. Do I need to submit v4 ?
No need, don't worry.
Paolo
PATCH v3 4/4 Optimize record/replay checkpointing for all clocks it applies to
In reply to Artem PisarenkoRemoves redundant checkpoints in replay log when there are no expired timers in timers list, associated with corresponding clock (i.e. no rr events associated with current clock value). This also improves performance in rr mode.
---
Oops and again oops. Now it finally should be fine...
v3:
- fixed compiler warning caused non-debug build to fail
include/qemu/timer.h | 2 +- util/qemu-timer.c | 62 +++++++++++++++++++++++++--------------------------- 2 files changed, 31 insertions(+), 33 deletions(-)
include/qemu/timer.h +1 -1
@@ -65,7 +65,7 @@ typedef enum {* QEMU_TIMER_ATTR_EXTERNAL: drives external subsystem * * Timers with this attribute do not recorded in rr mode, therefore it could be -* used for the subsystems that operate outside the guest core. Applicable only +* used for the subsystems that operate outside the guest core. Relevant only * with virtual clock type. */
util/qemu-timer.c +30 -32
86 lines not on this page · raw message
Signature · 1 line
2.7.4
Hi,
It seems, that this approach is not always correct.
Now timerlist_deadline_ns uses all virtual timers for deadline calculation (including external ones).qemu_start_warp_timer uses the deadline for setting warp timer (which should be deterministic). Therefore warp timer may become non-deterministic and replay may behave differently compared to recording phase.
We have to rollback these or improve somehow to avoid non-determinism.
Pavel Dovgalyuk
Quoted 90 lines
-----Original Message-----
From: Artem Pisarenko [mailto:[email protected]]
Sent: Thursday, October 18, 2018 2:04 PM
To: [email protected]
Cc: Pavel Dovgalyuk; Paolo Bonzini; Artem Pisarenko
Subject: [PATCH v3 0/4] Introduce attributes for timers subsystem and removeQEMU_CLOCK_VIRTUAL_EXTclock typeRecent patches from series [PATCH v6] "Fixing record/replay and adding reverse debugging" introduced new clock type
QEMU_CLOCK_VIRTUAL_EXTand replaced virtual timers in some external subsystems with it.
This resulted in small change to existing behavior, which I consider to be unacceptable. Processing of virtual timers, belonging to new clock type, was kicked off to main loop, which made them asynchronous with vCPU thread and, in icount mode, with whole guest execution. This breaks expected determinism in non-record/replay icount mode of emulation where these "external subsystems" are isolated from host (i.e. they external only to guest core, not to emulation environment).Example for slirp ("user" backend for network device):
User runs qemu in icount mode with rtc clock=vm without any external communication interfaces but with "-netdev user,restrict=on". It expects deterministic execution, because network services are emulated inside qemu and isolated from host. There are no reasons to get reply from DHCP server with different delay or something like that.These series of patches revert those commits and reimplement their modifications in a better way.
Current implementation of timers/clock processing is confusing (at least for me) because of exceptions from design concept behind them, which already introduced by icount mode (which adds
QEMU_CLOCK_VIRTUAL_RT). AddingQEMU_CLOCK_VIRTUAL_EXTjust made things even more complicated. I consider these "alternative" virtual clocks to be some kind of hacks being convinient only to authors of relevant qemu features. Lets don't touch fundamental clock types and keep them orthogonal to special cases of timers handling.As far as I understand, original intention of author was just to make optimization in replay log by avoiding storing extra events which don't change guest state directly. Indeed, for example, ipv6 icmp timer in slirp does things which external to guest core and ends with sending network packet to guest, but record/replay will anyway catch event, corresponding to packet reception in guest network frontend, and store it to replay log, so there are no need in making checkpoint for corresponding clock when that timer fires.
If so, then we just need to skip checkpoints for clock values, when only these specific timers are run. It is individual timers which are specific, not clock.
Adding some kind of attribute/flag/property to individual timer allows any special qemu feature (such as record/replay) to inspect it and handle as needed. This design achieves less dependencies, more transparency and has more intuitive and clear sense. For record/replay feature it's achieved with 'EXTERNAL' attribute. The only drawback is that it required to add one extra mutex unlock/lock pair for virtual clock type intimerlist_run_timers()function (see patch 3), but it's being added only when qemu runs in record/replay mode.Finally, this patch series optimizes checkpointing for all other clocks it applies to.
v3 changes:
- fixed failed build in last patch
- attributes has been refactored and restricted to be used only within qemu-timer (as
suggested by Stefan Hajnoczi and Paolo Bonzini)v2 changes:
- timers/aio interface refactored and improved, addition to couroutines removed (as Paolo
Bonzini suggested)
- fixed wrong reimplementation inqemu-timer.ccaused race conditions
- added bonus patch which optimizes checkpointing for other clocks
P.S. I've tried to test record/replay with slirp, but in replay mode qemu stucks at guest linux boot after "Configuring network interfaces..." message, where DHCP communication takes place. It's broken in a same way both in master and master with reverted commits being fixed.P.S.2. I wasn't able to test these patches on purely clean master branch because of bugs
https://bugs.launchpad.net/qemu/+bug/1790460 and https://bugs.launchpad.net/qemu/+bug/1795369,
which workarounded by several not-accepted (yet?) modifications.Artem Pisarenko (4):
Revert some patches from recent [PATCH v6] "Fixing record/replay and
adding reverse debugging"
Introduce attributes to qemu timer subsystem
Restores record/replay behavior related to special virtual clock
processing for timers used in external subsystems.
Optimize record/replay checkpointing for all clocks it applies toinclude/block/aio.h | 59 ++++++++++++++++++--- include/qemu/timer.h | 128 +++++++++++++++++++++++----------------------- slirp/ip6_icmp.c | 9 ++-- tests/ptimer-test-stubs.c | 13 +++-- ui/input.c | 9 ++-- util/qemu-timer.c | 95 +++++++++++++++++++++++----------- 6 files changed, 201 insertions(+), 112 deletions(-)--
2.7.4
Quoted 3 lines · Pavel Dovgalyuk
Now
timerlist_deadline_nsuses all virtual timers for deadline calculation (including external ones).qemu_start_warp_timeruses the deadline for setting warp timer (which should be deterministic).
Can you introduce a variant of qemu_clock_deadline_ns_all that ignores external timers, and use it in qemu_start_warp_timer?
timerlist_deadline_ns can be made static, so adding a bool argument to it is not a big issue for API cleanliness.
In fact, qemu_clock_deadline_ns_all is only ever used with QEMU_CLOCK_VIRTUAL, so you could also change it to
uint64_t virtual_clock_deadline_ns(bool include_external);
Paolo
Quoted 10 lines
From: Paolo Bonzini [mailto:[email protected]]
Quoted 1 line · Pavel Dovgalyuk
Now
timerlist_deadline_nsuses all virtual timers for deadline calculation (includingexternal
Quoted 2 lines
ones).
qemu_start_warp_timeruses the deadline for setting warp timer (which should bedeterministic).
Can you introduce a variant of
qemu_clock_deadline_ns_allthat ignores external timers, and use it inqemu_start_warp_timer?
Ok.
Quoted 7 lines
timerlist_deadline_nscan be made static, so adding a bool argument to it is not a big issue for API cleanliness.In fact,
qemu_clock_deadline_ns_allis only ever used withQEMU_CLOCK_VIRTUAL, so you could also change it to
uint64_tvirtual_clock_deadline_ns(bool include_external);
I started making changes and it seems that this parameter should never be true, because this function is called either from cpus.c or from the tests.
Pavel Dovgalyuk
Quoted 8 lines
It seems, that this approach is not always correct.
Nowtimerlist_deadline_nsuses all virtual timers for deadline calculation (including external ones).qemu_start_warp_timeruses the deadline for setting warp timer (which should be deterministic). Therefore warp timer may become non-deterministic and replay may behave differently compared to recording phase.We have to rollback these or improve somehow to avoid non-determinism.
I dont understand how this approach would even introduce non-determinism. I'm not sure about aspects of timers subsystem you mentioned, assuming that maybe we missed something when tried to optimize. But this has nothing to do with determinism as long as we treat all virtual clock timers as deterministic, regardless of EXTERNAL attribute being set or not. They are intended to be used as such by design, aren't? This attribute was introduced purely to avoid extra events in log.
Artem
Quoted 13 lines
From: Artem Pisarenko [mailto:[email protected]]
Quoted 2 lines
It seems, that this approach is not always correct.
Nowtimerlist_deadline_nsuses all virtual timers for deadline calculation (includingexternal
Quoted 2 lines
ones).
qemu_start_warp_timeruses the deadline for setting warp timer (which should bedeterministic).
Quoted 1 line
Therefore warp timer may become non-deterministic and replay may behave differently compared
to
Quoted 3 lines
recording phase.
We have to rollback these or improve somehow to avoid non-determinism.
Quoted 8 lines
I dont understand how this approach would even introduce non-determinism. I'm not sure about aspects of timers subsystem you mentioned, assuming that maybe we missed something when tried to optimize. But this has nothing to do with determinism as long as we treat all virtual clock timers as deterministic, regardless of EXTERNAL attribute being set or not. They are intended to be used as such by design, aren't? This attribute was introduced purely to avoid extra events in log.
Right, but deadline calculation and icount warp events (that are written into the log too) depend on the active virtual timers. Therefore external ones may affect icount warp operation sequence.
Pavel Dovgalyuk
ср, 16 янв. 2019 г. в 12:15, Pavel Dovgalyuk <[email protected]>:
Quoted 23 lines
Quoted 14 lines
From: Artem Pisarenko [mailto:[email protected]]
Quoted 2 lines
It seems, that this approach is not always correct.
Nowtimerlist_deadline_nsuses all virtual timers for deadline calculation (includingexternal
Quoted 2 lines
ones).
qemu_start_warp_timeruses the deadline for setting warp timer (which should bedeterministic).
Quoted 1 line
Therefore warp timer may become non-deterministic and replay may behave differently compared
to
Quoted 3 lines
recording phase.
We have to rollback these or improve somehow to avoid non-determinism.
Quoted 8 lines
I dont understand how this approach would even introduce non-determinism. I'm not sure about aspects of timers subsystem you mentioned, assuming that maybe we missed something when tried to optimize. But this has nothing to do with determinism as long as we treat all virtual clock timers as deterministic, regardless of EXTERNAL attribute being set or not. They are intended to be used as such by design, aren't? This attribute was introduced purely to avoid extra events in log.
Quoted 6 lines
Right, but deadline calculation and icount warp events (that are written into the log too) depend on the active virtual timers. Therefore external ones may affect icount warp operation sequence.
Pavel Dovgalyuk
Sorry, but I still don't understand the source of non-determinism.
Quoted 1 line
From what you said I may understand that replay module actually has
some additional non-trivial (for me) dependencies on EXTERNAL attribute other than one, introduced in 'timerlist_run_timers()' (i.e. it expects deadline calculations somewhere else to match decisions made in this function, or something like that). I would agree that these patch series might introduce some incorrect behavior. But it must be identical in all emulations running in same conditions, because deadline, warp timer and their effects are all dependent only on virtual timers, and, therefore, are deterministic too. Of course, it needs to be fixed. Did you find solution ?
Quoted 26 lines
From: Artem Pisarenko [mailto:[email protected]] ср, 16 янв. 2019 г. в 12:15, Pavel Dovgalyuk <[email protected]>:
Quoted 9 lines
Quoted 9 lines
From: Artem Pisarenko [mailto:[email protected]]
Quoted 2 lines
It seems, that this approach is not always correct.
Nowtimerlist_deadline_nsuses all virtual timers for deadline calculation (includingexternal
Quoted 2 lines
ones).
qemu_start_warp_timeruses the deadline for setting warp timer (which should bedeterministic).
Quoted 1 line
Therefore warp timer may become non-deterministic and replay may behave differently
compared
Quoted 14 lines
Quoted 13 lines
to
Quoted 3 lines
recording phase.
We have to rollback these or improve somehow to avoid non-determinism.
I dont understand how this approach would even introduce non-determinism. I'm not sure about aspects of timers subsystem you mentioned, assuming that maybe we missed something when tried to optimize. But this has nothing to do with determinism as long as we treat all virtual clock timers as deterministic, regardless of EXTERNAL attribute being set or not. They are intended to be used as such by design, aren't? This attribute was introduced purely to avoid extra events in log.
Quoted 8 lines
Quoted 6 lines
Right, but deadline calculation and icount warp events (that are written into the log too) depend on the active virtual timers. Therefore external ones may affect icount warp operation sequence.
Pavel Dovgalyuk
Sorry, but I still don't understand the source of non-determinism.
The source is the external timers, that depend on "outer world", e.g., slirp. These timers are not recorded/replayed, but may affect the icount warping (which should remain deterministic).
Quoted 10 lines
From what you said I may understand that replay module actually has some additional non-trivial (for me) dependencies on EXTERNAL attribute other than one, introduced in '
timerlist_run_timers()' (i.e. it expects deadline calculations somewhere else to match decisions made in this function, or something like that). I would agree that these patch series might introduce some incorrect behavior. But it must be identical in all emulations running in same conditions, because deadline, warp timer and their effects are all dependent only on virtual timers, and, therefore, are deterministic too. Of course, it needs to be fixed. Did you find solution ?
Please check the new version of the series.
Patch 22
Pavel Dovgalyuk
Quoted 3 lines
The source is the external timers, that depend on "outer world", e.g., slirp. These timers are not recorded/replayed, but may affect the icount warping (which should remain deterministic).
I thought we agreed earlier in this discussion that external timers are deterministic by design, because they are subset of virtual clock timers. Regarding slirp module, although it interfaces "outer world" (except when 'restrict' option is on), it still uses IPv6 timer deterministically. You may check it yourself. Otherwise it shouldn't use timer of 'virtual clock' type for this purpose and this needs to be fixed.
So I still don't understand. And I'm not asking you to explain. I'm just hinting you that your fix may finally happen to be incorrect in case if it based on wrong assertion I'm talking about.
Quoted 1 line
Please check the new version of the series.
Sorry, I'm too busy right now for deep analysis. I've just tried to briefly understand issue you pointed and provide minimal feedback in order to prevent possible wrong decisions. I'll return to this work later.