scsi: infinite guest hangs with scsi-disk
cover scsi: infinite guest hangs with scsi-disk
Hi all,
one of our customers reported an infinite guest hang following an FC link loss when using scsi-disk. Problem is that scsi-disk issues SG_IO command with a timeout of UINT_MAX, which essentially signals 'no timeout' to the host kernel. So if the command gets lost eg during an unexpected link loss the HBA driver will never attempt to abort or return the command. Hence the guest will hang forever, and the only way to resolve things is to reboot the host.
To solve it this patchset adds an 'io_timeout' parameter to scsi-disk and scsi-generic, which allows the admin to specify a command timeout for SG_IO request. It is initialized to 30 seconds to avoid the infinite hang as mentioned above.
As usual, comments and reviews are welcome.
Hannes Reinecke (3):
virtio-scsi: trace events
scsi: make io_timeout configurable
scsi: add tracing for SG_IO commands
hw/scsi/scsi-disk.c | 9 ++++++--- hw/scsi/scsi-generic.c | 25 ++++++++++++++++++------- hw/scsi/trace-events | 13 +++++++++++++ hw/scsi/virtio-scsi.c | 30 +++++++++++++++++++++++++++++- include/hw/scsi/scsi.h | 4 +++- 5 files changed, 69 insertions(+), 12 deletions(-)
Signature · 1 line
2.16.4
PATCH 1/3 virtio-scsi: trace events
Add trace events for virtio command and response tracing.
--- hw/scsi/trace-events | 9 +++++++++ hw/scsi/virtio-scsi.c | 30 +++++++++++++++++++++++++++++- 2 files changed, 38 insertions(+), 1 deletion(-)
hw/scsi/trace-events +9 -0
@@ -294,6 +294,15 @@ lsi_awoken(void) "Woken by SIGP" lsi_reg_read(const char *name, int offset, uint8_t ret) "Read reg %s 0x%x = 0x%02x" lsi_reg_write(const char *name, int offset, uint8_t val) "Write reg %s 0x%x = 0x%02x" +# virtio-scsi.c +virtio_scsi_cmd_req(int lun, uint32_t tag, uint8_t cmd) "virtio_scsi_cmd_req lun=%u tag=0x%x cmd=0x%x" +virtio_scsi_cmd_resp(int lun, uint32_t tag, int response, uint8_t status) "virtio_scsi_cmd_resp lun=%u tag=0x%x response=%d status=0x%x" +virtio_scsi_tmf_req(int lun, uint32_t tag, int subtype) "virtio_scsi_tmf_req lun=%u tag=0x%x subtype=%d" +virtio_scsi_tmf_resp(int lun, uint32_t tag, int response) "virtio_scsi_tmf_resp lun=%u tag=0x%x response=%d" +virtio_scsi_an_req(int lun, uint32_t event_requested) "virtio_scsi_an_req lun=%u event_requested=0x%x" +virtio_scsi_an_resp(int lun, int response) "virtio_scsi_an_resp lun=%u response=%d" +virtio_scsi_event(int lun, int event, int reason) "virtio_scsi_event lun=%u event=%d reason=%d" + # scsi-disk.c scsi_disk_check_condition(uint32_t tag, uint8_t key, uint8_t asc, uint8_t ascq) "Command complete tag=0x%x sense=%d/%d/%d" scsi_disk_read_complete(uint32_t tag, size_t size) "Data ready tag=0x%x len=%zd"
hw/scsi/virtio-scsi.c +29 -1
@@ -27,6 +27,7 @@#include "scsi/constants.h" #include "hw/virtio/virtio-bus.h" #include "hw/virtio/virtio-access.h" +#include "trace.h" static inline int virtio_scsi_get_lun (uint8_t *lun) {
@@ -239,7 +240,11 @@ static void virtio_scsi_cancel_notify(Notifier *notifier, void *data) notifier);if (--n->tmf_req->remaining ==0 ) { -virtio_scsi_complete_req (n->tmf_req); +VirtIOSCSIReq *req = n->tmf_req; + +trace_virtio_scsi_tmf_resp (virtio_scsi_get_lun (req->req.tmf.lun), + req->req.tmf.tag, req->resp.tmf.response); +virtio_scsi_complete_req (req); }g_free (n); }
@@ -273,6 +278,9 @@ static int virtio_scsi_do_tmf(VirtIOSCSI *s, VirtIOSCSIReq *req) req->req.tmf.subtype =virtio_tswap32 (VIRTIO_DEVICE (s), req->req.tmf.subtype); +trace_virtio_scsi_tmf_req (virtio_scsi_get_lun (req->req.tmf.lun), + req->req.tmf.tag, req->req.tmf.subtype); +switch (req->req.tmf.subtype) {case VIRTIO_SCSI_T_TMF_ABORT_TASK :case VIRTIO_SCSI_T_TMF_QUERY_TASK :
@@ -422,11 +430,23 @@ static void virtio_scsi_handle_ctrl_req(VirtIOSCSI *s, VirtIOSCSIReq *req)virtio_scsi_bad_req (req);return ; }else { + req->req.an.event_requested = +virtio_tswap32 (VIRTIO_DEVICE (s), req->req.an.event_requested); +trace_virtio_scsi_an_req (virtio_scsi_get_lun (req->req.an.lun), + req->req.an.event_requested); req->resp.an.event_actual =0 ; req->resp.an.response =VIRTIO_SCSI_S_OK ; } }if (r ==0 ) { +if (type ==VIRTIO_SCSI_T_TMF ) +trace_virtio_scsi_tmf_resp (virtio_scsi_get_lun (req->req.tmf.lun), + req->req.tmf.tag, + req->resp.tmf.response); +else if (type ==VIRTIO_SCSI_T_AN_QUERY || + type ==VIRTIO_SCSI_T_AN_SUBSCRIBE ) +trace_virtio_scsi_an_resp (virtio_scsi_get_lun (req->req.an.lun), + req->resp.an.response);virtio_scsi_complete_req (req); }else {assert (r == -EINPROGRESS );
@@ -462,6 +482,10 @@ static void virtio_scsi_handle_ctrl(VirtIODevice *vdev, VirtQueue *vq)static void virtio_scsi_complete_cmd_req (VirtIOSCSIReq *req) { +trace_virtio_scsi_cmd_resp (virtio_scsi_get_lun (req->req.cmd.lun), + req->req.cmd.tag, + req->resp.cmd.response, + req->resp.cmd.status);/* Sense data is not in req->resp and is copied separately * in virtio_scsi_command_complete. */
@@ -559,6 +583,8 @@ static int virtio_scsi_handle_cmd_req_prepare(VirtIOSCSI *s, VirtIOSCSIReq *req)return -EINVAL ; } } +trace_virtio_scsi_cmd_req (virtio_scsi_get_lun (req->req.cmd.lun), + req->req.cmd.tag, req->req.cmd.cdb[0 ]); d =virtio_scsi_device_find (s, req->req.cmd.lun);if (!d) {
@@ -758,6 +784,8 @@ void virtio_scsi_push_event(VirtIOSCSI *s, SCSIDevice *dev, } evt->lun[3 ] = dev->lun &0xFF ; } +trace_virtio_scsi_event (virtio_scsi_get_lun (evt->lun), event, reason); +virtio_scsi_complete_req (req); }
Signature · 1 line
2.16.4
PATCH 2/3 scsi: make io_timeout configurable
In reply to Hannes ReineckeThe current code sets an infinite timeout on SG_IO requests, causing the guest to stall if the host experiences a frame loss.
This patch adds an 'io_timeout' parameter for SCSIDevice to make the SG_IO timeout configurable, and also shortens the default timeout to 30 seconds to avoid infinite stalls.
--- hw/scsi/scsi-disk.c | 6 ++++-- hw/scsi/scsi-generic.c | 17 +++++++++++------ include/hw/scsi/scsi.h | 4 +++- 3 files changed, 18 insertions(+), 9 deletions(-)
hw/scsi/scsi-disk.c +4 -2
@@ -2604,7 +2604,7 @@ static int get_device_type(SCSIDiskState *s) cmd[4 ] =sizeof (buf); ret =scsi_SG_IO_FROM_DEV (s->qdev.conf.blk, cmd,sizeof (cmd), - buf,sizeof (buf)); + buf,sizeof (buf), s->qdev.io_timeout);if (ret <0 ) {return -1 ; }
@@ -2765,7 +2765,7 @@ static BlockAIOCB *scsi_block_do_sgio(SCSIBlockReq *req,/* The rest is as in scsi-generic.c. */ io_header->mx_sb_len =sizeof (r->req.sense); io_header->sbp = r->req.sense; - io_header->timeout =UINT_MAX ; + io_header->timeout = s->qdev.io_timeout *1000 ; io_header->usr_ptr = r; io_header->flags |=SG_FLAG_DIRECT_IO ;
@@ -3083,6 +3083,8 @@ static Property scsi_block_properties[] = { DEFAULT_MAX_IO_SIZE),DEFINE_PROP_INT32 ("scsi_version" , SCSIDiskState, qdev.default_scsi_version,-1 ), +DEFINE_PROP_UINT32 ("io_timeout" , SCSIDiskState, qdev.io_timeout, +DEFAULT_IO_TIMEOUT ), DEFINE_PROP_END_OF_LIST(), };
hw/scsi/scsi-generic.c +11 -6
@@ -114,6 +114,8 @@ static int execute_command(BlockBackend *blk, SCSIGenericReq *r, int direction, BlockCompletionFunc *complete) { + SCSIDevice *s = r->req.dev; + r->io_header.interface_id ='S' ; r->io_header.dxfer_direction = direction; r->io_header.dxferp = r->buf;
@@ -122,7 +124,7 @@ static int execute_command(BlockBackend *blk, r->io_header.cmd_len = r->req.cmd.len; r->io_header.mx_sb_len =sizeof (r->req.sense); r->io_header.sbp = r->req.sense; - r->io_header.timeout =MAX_UINT ; + r->io_header.timeout = s->io_timeout *1000 ; r->io_header.usr_ptr = r; r->io_header.flags |=SG_FLAG_DIRECT_IO ;
@@ -505,7 +507,7 @@ static int read_naa_id(const uint8_t *p, uint64_t *p_wwn) }int scsi_SG_IO_FROM_DEV (BlockBackend *blk,uint8_t *cmd,uint8_t cmd_size, -uint8_t *buf,uint8_t buf_size) +uint8_t *buf,uint8_t buf_size,uint32_t timeout) {sg_io_hdr_t io_header;uint8_t sensebuf[8 ];
@@ -520,7 +522,7 @@ int scsi_SG_IO_FROM_DEV(BlockBackend *blk, uint8_t *cmd, uint8_t cmd_size, io_header.cmd_len = cmd_size; io_header.mx_sb_len =sizeof (sensebuf); io_header.sbp = sensebuf; - io_header.timeout =6000 ;/* XXX */ + io_header.timeout = timeout *1000 ; ret =blk_ioctl (blk,SG_IO , &io_header);if (ret <0 || io_header.driver_status || io_header.host_status) {
@@ -550,7 +552,7 @@ static void scsi_generic_set_vpd_bl_emulation(SCSIDevice *s) cmd[4 ] =sizeof (buf); ret =scsi_SG_IO_FROM_DEV (s->conf.blk, cmd,sizeof (cmd), - buf,sizeof (buf)); + buf,sizeof (buf), s->io_timeout);if (ret <0 ) {/* * Do not assume anything if we can't retrieve the
@@ -586,7 +588,7 @@ static void scsi_generic_read_device_identification(SCSIDevice *s) cmd[4 ] =sizeof (buf); ret =scsi_SG_IO_FROM_DEV (s->conf.blk, cmd,sizeof (cmd), - buf,sizeof (buf)); + buf,sizeof (buf), s->io_timeout);if (ret <0 ) {return ; }
@@ -637,7 +639,7 @@ static int get_stream_blocksize(BlockBackend *blk) cmd[0 ] =MODE_SENSE ; cmd[4 ] =sizeof (buf); - ret =scsi_SG_IO_FROM_DEV (blk, cmd,sizeof (cmd), buf,sizeof (buf)); + ret =scsi_SG_IO_FROM_DEV (blk, cmd,sizeof (cmd), buf,sizeof (buf),6 );if (ret <0 ) {return -1 ; }
@@ -727,6 +729,7 @@ static void scsi_generic_realize(SCSIDevice *s, Error **errp)/* Only used by scsi-block, but initialize it nevertheless to be clean. */ s->default_scsi_version =-1 ; + s->io_timeout =DEFAULT_IO_TIMEOUT ;scsi_generic_read_device_inquiry (s); }
@@ -750,6 +753,8 @@ static SCSIRequest *scsi_new_request(SCSIDevice *d, uint32_t tag, uint32_t lun,static Property scsi_generic_properties[] = {DEFINE_PROP_DRIVE ("drive" , SCSIDevice, conf.blk),DEFINE_PROP_BOOL ("share-rw" , SCSIDevice, conf.share_rw, false), +DEFINE_PROP_UINT32 ("io_timeout" , SCSIDevice, io_timeout, +DEFAULT_IO_TIMEOUT ),DEFINE_PROP_END_OF_LIST (), };
include/hw/scsi/scsi.h +3 -1
@@ -18,6 +18,7 @@ typedef struct SCSIReqOps SCSIReqOps;#define SCSI_SENSE_BUF_SIZE_OLD 96#define SCSI_SENSE_BUF_SIZE 252 +#define DEFAULT_IO_TIMEOUT 30struct SCSIRequest {SCSIBus *bus;
@@ -84,6 +85,7 @@ struct SCSIDeviceuint64_t port_wwn;int scsi_version;int default_scsi_version; +uint32_t io_timeout;bool needs_vpd_bl_emulation;bool hba_supports_iothread; };
@@ -188,7 +190,7 @@ void scsi_device_unit_attention_reported(SCSIDevice *dev);void scsi_generic_read_device_inquiry (SCSIDevice *dev);int scsi_device_get_sense (SCSIDevice *dev,uint8_t *buf,int len,bool fixed);int scsi_SG_IO_FROM_DEV (BlockBackend *blk,uint8_t *cmd,uint8_t cmd_size, -uint8_t *buf,uint8_t buf_size); +uint8_t *buf,uint8_t buf_size,uint32_t timeout);SCSIDevice *scsi_device_find (SCSIBus *bus,int channel,int target,int lun);/* scsi-generic.c. */
Signature · 1 line
2.16.4
Re: [PATCH 2/3] scsi: make io_timeout configurable
Quoted 6 lines · Hannes Reinecke
The current code sets an infinite timeout on
SG_IOrequests, causing the guest to stall if the host experiences a frame loss.
This patch adds an 'io_timeout' parameter for SCSIDevice to make theSG_IOtimeout configurable, and also shortens the default timeout to 30 seconds to avoid infinite stalls.
Hannes, could 30 seconds be a bit too short for tape drives?
Paolo
Re: [PATCH 2/3] scsi: make io_timeout configurable
Quoted 10 lines · Paolo Bonzini
Quoted 6 lines · Hannes Reinecke
The current code sets an infinite timeout on
SG_IOrequests, causing the guest to stall if the host experiences a frame loss.
This patch adds an 'io_timeout' parameter for SCSIDevice to make theSG_IOtimeout configurable, and also shortens the default timeout to 30 seconds to avoid infinite stalls.Hannes, could 30 seconds be a bit too short for tape drives?
It would, but then anyone attempting to use tapes via qemu emulation deserves to suffer.
Tapes are bitchy even when used normally, so attempting to use them under qemu emulation will land you with lots of unhappy experiences, where the timeout is the least of your problems. I sincerely doubt anyone will be using tapes here.
Not in real-world scenarios.
Cheers,
Hannes
Signature · 4 lines
Dr. Hannes Reinecke Kernel Storage Architect [email protected] +49 911 74053 688 SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
Re: [PATCH 2/3] scsi: make io_timeout configurable
Quoted 4 lines · Hannes Reinecke
It would, but then anyone attempting to use tapes via qemu emulation deserves to suffer.
Tapes are bitchy even when used normally, so attempting to use them under qemu emulation will land you with lots of unhappy experiences, where the timeout is the least of your problems. I sincerely doubt anyone will be using tapes here.
Not in real-world scenarios.
Hmm, I have customers that disagree. Probably the timeout should be kept infinite for tapes.
Paolo
Re: [PATCH 2/3] scsi: make io_timeout configurable
In reply to Hannes ReineckeHi Hannes,
Quoted 28 lines · Hannes Reinecke
The current code sets an infinite timeout on
SG_IOrequests, causing the guest to stall if the host experiences a frame loss.
This patch adds an 'io_timeout' parameter for SCSIDevice to make theSG_IOtimeout configurable, and also shortens the default timeout to 30 seconds to avoid infinite stalls.Signed-off-by Hannes Reinecke <[email protected]>--- hw/scsi/scsi-disk.c | 6 ++++-- hw/scsi/scsi-generic.c | 17 +++++++++++------ include/hw/scsi/scsi.h | 4 +++- 3 files changed, 18 insertions(+), 9 deletions(-)int scsi_SG_IO_FROM_DEV(BlockBackend *blk,
uint8_t*cmd,uint8_tcmd_size,hw/scsi/scsi-generic.c +2
-2- uint8_t *buf, uint8_t buf_size) + uint8_t *buf, uint8_t buf_size, uint32_t timeout) { sg_io_hdr_t io_header;uint8_t sensebuf[8 ];@@ -520,7 +522,7 @@ int scsi_SG_IO_FROM_DEV(BlockBackend *blk, uint8_t *cmd, uint8_t cmd_size, io_header.cmd_len = cmd_size; io_header.mx_sb_len =sizeof (sensebuf); io_header.sbp = sensebuf; - io_header.timeout =6000 ;/* XXX */ + io_header.timeout = timeout *1000 ;
hw/scsi/scsi-generic.c static int get_stream_blocksize(BlockBackend *blk) · line 642
Quoted 5 lines
hw/scsi/scsi-generic.c +1
-1@@ -637,7 +639,7 @@ static int get_stream_blocksize(BlockBackend *blk) cmd[0 ] =MODE_SENSE ; cmd[4 ] =sizeof (buf); - ret =scsi_SG_IO_FROM_DEV (blk, cmd,sizeof (cmd), buf,sizeof (buf)); + ret =scsi_SG_IO_FROM_DEV (blk, cmd,sizeof (cmd), buf,sizeof (buf),6 );
Why is this timeout hardcoded? Due to the /* XXX */ comment?
Re: [PATCH 2/3] scsi: make io_timeout configurable
Quoted 32 lines · Philippe Mathieu-Daudé
Hi Hannes,
Quoted 28 lines · Hannes Reinecke
The current code sets an infinite timeout on
SG_IOrequests, causing the guest to stall if the host experiences a frame loss.
This patch adds an 'io_timeout' parameter for SCSIDevice to make theSG_IOtimeout configurable, and also shortens the default timeout to 30 seconds to avoid infinite stalls.Signed-off-by Hannes Reinecke <[email protected]>---
hw/scsi/scsi-disk.c | 6 ++++--
hw/scsi/scsi-generic.c| 17 +++++++++++------
include/hw/scsi/scsi.h| 4 +++-3 files changed, 18 insertions(+), 9 deletions(-)int scsi_SG_IO_FROM_DEV(BlockBackend *blk,
uint8_t*cmd,uint8_tcmd_size, -uint8_t*buf,uint8_tbuf_size)
+uint8_t*buf,uint8_tbuf_size,uint32_ttimeout)
{
sg_io_hdr_tio_header;
uint8_tsensebuf[8];hw/scsi/scsi-generic.c +1
-1@@ -520,7 +522,7 @@ int scsi_SG_IO_FROM_DEV(BlockBackend *blk, uint8_t *cmd, uint8_t cmd_size, io_header.cmd_len = cmd_size; io_header.mx_sb_len = sizeof(sensebuf); io_header.sbp = sensebuf; - io_header.timeout = 6000; /* XXX */ + io_header.timeout = timeout * 1000;
Quoted 9 lines
hw/scsi/scsi-generic.c static int get_stream_blocksize(BlockBackend *blk) · line 642
Quoted 6 lines
hw/scsi/scsi-generic.c +1
-1@@ -637,7 +639,7 @@ static int get_stream_blocksize(BlockBackend *blk) cmd[0] = MODE_SENSE; cmd[4] = sizeof(buf); - ret = scsi_SG_IO_FROM_DEV(blk, cmd, sizeof(cmd), buf, sizeof(buf)); + ret = scsi_SG_IO_FROM_DEV(blk, cmd, sizeof(cmd), buf,sizeof (buf),6 );Why is this timeout hardcoded? Due to the /* XXX */ comment?
60 seconds is the default command timeout on linux.
And the problem is that the guest might set a command timeout on the comands being send from the guests user space, but the guest is unable to communicate this timeout to the host.
Really, one should fix up the virtio spec here to allow for a 'timeout' field. But in the absence of that being able to configure it is the next best attempt.
Cheers,
Hannes
Signature · 4 lines
Dr. Hannes Reinecke Kernel Storage Architect [email protected] +49 911 74053 688 SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
Re: [PATCH 2/3] scsi: make io_timeout configurable
In reply to Philippe Mathieu-DaudéQuoted 10 lines · Philippe Mathieu-Daudé
hw/scsi/scsi-generic.c static int get_stream_blocksize(BlockBackend *blk) · line 642
Quoted 8 lines
hw/scsi/scsi-generic.c +1
-1@@ -637,7 +639,7 @@ static int get_stream_blocksize(BlockBackend *blk) cmd[0] = MODE_SENSE; cmd[4] = sizeof(buf); - ret = scsi_SG_IO_FROM_DEV(blk, cmd, sizeof(cmd), buf, sizeof(buf)); + ret = scsi_SG_IO_FROM_DEV(blk, cmd, sizeof(cmd), buf,sizeof (buf),6 );Why is this timeout hardcoded? Due to the /* XXX */ comment?
This command is only invoked at startup and involves no I/O, so 6 seconds should be plenty.
Paolo
PATCH 3/3 scsi: add tracing for SG_IO commands
In reply to Hannes ReineckeAdd tracepoints for SG_IO commands to allow for debugging of SG_IO commands.
--- hw/scsi/scsi-disk.c | 3 ++- hw/scsi/scsi-generic.c | 8 +++++++- hw/scsi/trace-events | 4 ++++ 3 files changed, 13 insertions(+), 2 deletions(-)
hw/scsi/scsi-disk.c +2 -1
@@ -2768,7 +2768,8 @@ static BlockAIOCB *scsi_block_do_sgio(SCSIBlockReq *req, io_header->timeout = s->qdev.io_timeout *1000 ; io_header->usr_ptr = r; io_header->flags |=SG_FLAG_DIRECT_IO ; - +trace_scsi_disk_aio_sgio_command (r->req.tag, req->cdb[0 ], lba, + nb_logical_blocks, io_header->timeout); aiocb =blk_aio_ioctl (s->qdev.conf.blk,SG_IO , io_header, cb, opaque);assert (aiocb !=NULL );return aiocb;
hw/scsi/scsi-generic.c +7 -1
@@ -128,6 +128,8 @@ static int execute_command(BlockBackend *blk, r->io_header.usr_ptr = r; r->io_header.flags |=SG_FLAG_DIRECT_IO ; +trace_scsi_generic_aio_sgio_command (r->req.tag, r->req.cmd.buf[0 ], + r->io_header.timeout); r->req.aiocb =blk_aio_ioctl (blk,SG_IO , &r->io_header, complete, r);if (r->req.aiocb ==NULL ) {return -EIO ;
@@ -524,8 +526,12 @@ int scsi_SG_IO_FROM_DEV(BlockBackend *blk, uint8_t *cmd, uint8_t cmd_size, io_header.sbp = sensebuf; io_header.timeout = timeout *1000 ; +trace_scsi_generic_ioctl_sgio_command (cmd[0 ], io_header.timeout); ret =blk_ioctl (blk,SG_IO , &io_header); -if (ret <0 || io_header.driver_status || io_header.host_status) { +if (ret <0 || io_header.status || + io_header.driver_status || io_header.host_status) { +trace_scsi_generic_ioctl_sgio_done (cmd[0 ], ret, io_header.status, + io_header.host_status);return -1 ; }return 0 ;
hw/scsi/trace-events +4 -0
@@ -331,6 +331,7 @@ scsi_disk_emulate_command_UNKNOWN(int cmd, const char *name) "Unknown SCSI comma scsi_disk_dma_command_READ(uint64_t lba, uint32_t len) "Read (sector %" PRId64 ", count %u)" scsi_disk_dma_command_WRITE(const char *cmd, uint64_t lba, int len) "Write %s(sector %" PRId64 ", count %u)" scsi_disk_new_request(uint32_t lun, uint32_t tag, const char *line) "Command: lun=%d tag=0x%x data=%s" +scsi_disk_aio_sgio_command(uint32_t tag, uint8_t cmd, uint64_t lba, int len, uint32_t timeout) "disk aio sgio: tag=0x%x cmd=0x%x (sector %" PRId64 ", count %d) timeout=%u" # scsi-generic.c scsi_generic_command_complete_noio(void *req, uint32_t tag, int statuc) "Command complete %p tag=0x%x status=%d"
@@ -342,3 +343,6 @@ scsi_generic_write_data(uint32_t tag) "scsi_write_data tag=0x%x" scsi_generic_send_command(const char *line) "Command: data=%s" scsi_generic_realize_type(int type) "device type %d" scsi_generic_realize_blocksize(int blocksize) "block size %d" +scsi_generic_aio_sgio_command(uint32_t tag, uint8_t cmd, uint32_t timeout) "generic aio sgio: tag=0x%x cmd=0x%x timeout=%u" +scsi_generic_ioctl_sgio_command(uint8_t cmd, uint32_t timeout) "generic ioctl sgio: cmd=0x%x timeout=%u" +scsi_generic_ioctl_sgio_done(uint8_t cmd, int ret, uint8_t status, uint8_t host_status) "generic ioctl sgio: cmd=0x%x ret=%d status=0x%x host_status=0x%x"
Signature · 1 line
2.16.4
Patchew URL: https://patchew.org/QEMU/[email protected]/
Patchew URL: https://patchew.org/QEMU/[email protected]/
Hi,
This series seems to have some coding style problems. See output below for more information:
Message-id: [email protected]
Type: series
Subject: [PATCH 0/3] scsi: infinite guest hangs with scsi-disk
=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
- [tag update] patchew/[email protected] -> patchew/[email protected] - [tag update] patchew/[email protected] -> patchew/[email protected] * [new tag] patchew/[email protected] -> patchew/[email protected]
Switched to a new branch 'test'
350bcf1 scsi: add tracing for SG_IO commands
7ecf5b6 scsi: make io_timeout configurable
450c008 virtio-scsi: trace events
=== OUTPUT BEGIN ===
1/3 Checking commit 450c008843e5 (virtio-scsi: trace events)
ERROR: trailing whitespace
#116: FILE: hw/scsi/virtio-scsi.c:797:
+ $
total: 1 errors, 0 warnings, 92 lines checked
Patch 1/3 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS.
2/3 Checking commit 7ecf5b611a5b (scsi: make io_timeout configurable) 3/3 Checking commit 350bcf121178 (scsi: add tracing for SG_IO commands) === OUTPUT END ===
Test command exited with code: 1
The full log is available at
http://patchew.org/logs/[email protected]/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]
Quoted 19 lines · Hannes Reinecke
Hi all,
one of our customers reported an infinite guest hang following an FC link loss when using scsi-disk. Problem is that scsi-disk issues
SG_IOcommand with a timeout ofUINT_MAX, which essentially signals 'no timeout' to the host kernel. So if the command gets lost eg during an unexpected link loss the HBA driver will never attempt to abort or return the command. Hence the guest will hang forever, and the only way to resolve things is to reboot the host.To solve it this patchset adds an '
io_timeout' parameter to scsi-disk and scsi-generic, which allows the admin to specify a command timeout forSG_IOrequest. It is initialized to 30 seconds to avoid the infinite hang as mentioned above.As usual, comments and reviews are welcome.
Hannes Reinecke (3):
virtio-scsi: trace events
scsi: makeio_timeoutconfigurable
scsi: add tracing forSG_IOcommands
Quoted 7 lines
hw/scsi/scsi-disk.c | 9 ++++++--- hw/scsi/scsi-generic.c | 25 ++++++++++++++++++------- hw/scsi/trace-events | 13 +++++++++++++ hw/scsi/virtio-scsi.c | 30 +++++++++++++++++++++++++++++- include/hw/scsi/scsi.h | 4 +++- 5 files changed, 69 insertions(+), 12 deletions(-)
The UINT_MAX timeout predates me, but I think the idea was to make it sort of like NFS's hard option. Without a timeout you cannot be quite sure if/when the command will stay in some buffer of the HBA or the SAN or the target, and there could be unintended reordering of writes.
Though I guess at some point you'll anyway restart the VM on another host and the same reordering can happen, so I've queued the patch.
Paolo