Skip to content

iscsi: fix hang after abrupt target disconnect

Konstantin Nigmatullin qemu-devel 1 version New

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest5 patches 0 repliesNew

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/5iscsi: detect dead iSCSI sessions via local NOP failure countingblock/iscsi.c+24 -20 replies
2/5iscsi: stop retrying timed-out iSCSI commandsblock/iscsi.c+11 -90 replies
3/5iscsi: cancel in-flight iSCSI tasks when the session diesblock/iscsi.c+7 -00 replies
4/5iscsi: fail new iSCSI I/O while the transport is known downblock/iscsi.c+58 -10 replies
5/5iscsi: avoid null iSCSI error strings after cancel or timeoutblock/iscsi.c+14 -00 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH 0/5] iscsi: fix hang after abrupt target disconnect

When an iSCSI target disappears (for example tgtd is killed), QEMU can leave guest I/O stuck and then fail to exit on SIGINT because virtio_scsi_dataplane_stop blocks forever in blk_drain_all().

This is a stack of problems in block/iscsi.c, not only the documented default command timeout of 0:

1. NOP dead-session detection never fired in practice because iscsi_get_nops_in_flight() stayed at 0 on a dead TCP session.
2. SCSI_STATUS_TIMEOUT was retried like BUSY (up to seven full timeouts), so drain stayed busy for minutes even with -iscsi timeout=N.
3. In-flight coroutines were not cancelled locally, so with timeout=0 waiters never woke.
4. After one failure, queued I/O restarted another long wait, so drain never stayed idle (permanent hang).

This series addresses those without changing the default timeout value.

Verified with the GitLab #3067 recipe (default timeout=0). After killing tgtd and unmounting, the guest correctly sees I/O errors, and Ctrl-C makes QEMU exit cleanly instead of hanging in blk_drain_all:

(initramfs) umount /mnt
qemu-system-x86_64: terminating on signal 2
qemu-system-x86_64: iSCSI: NOP timeout. Reconnecting...
qemu-system-x86_64: iSCSI SYNCHRONIZECACHE10 failed: SENSE KEY:UNIT_ATTENTION(6) ASCQ:BUS_RESET(0x2900)

Residual host messages such as NOP timeout / SYNCHRONIZECACHE10 UNIT_ATTENTION(BUS_RESET) are expected while in-flight flush fails on a dead session during reconnect; they are cosmetic and not a correctness issue for this series.

Konstantin Nigmatullin (5):
iscsi: detect dead iSCSI sessions via local NOP failure counting
iscsi: stop retrying timed-out iSCSI commands
iscsi: cancel in-flight iSCSI tasks when the session dies
iscsi: fail new iSCSI I/O while the transport is known down
iscsi: avoid null iSCSI error strings after cancel or timeout

 block/iscsi.c | 124 +++++++++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 113 insertions(+), 11 deletions(-)
Signature · 1 line
2.53.0