spapr_vscsi: fix silent data corruption above 128 KiB
cover spapr_vscsi: fix silent data corruption above 128 KiB
struct vscsi_req::cur_desc_offset is uint16_t. scsi-disk delivers a command in SCSI_DMA_BUF_SIZE (128 KiB) chunks, so vscsi_srp_direct_data() adds 0x20000 to it on the first chunk and it wraps back to zero. Every following chunk of the same command is then DMA'd over the start of the guest buffer:
the tail of the transfer overwrites the head, and the rest is never written at all. The command completes with status 0 and no sense data, so the guest has no way to notice.
Linux never hits it. Its ibmvscsi initiator uses INDIRECT descriptors for anything multi-page, and that path resets cur_desc_offset at every descriptor boundary, so the counter never reaches 65536. A guest that describes the whole transfer with one DIRECT descriptor and issues 256 KiB commands hits it on the first command. The field has been uint16_t since it was introduced in 2013.
Patch 1 widens it. struct srp_direct_buf.len is already uint32_t and vscsi_fetch_desc() already takes buf_offset as unsigned, so that is the whole change. The VMSTATE entries for cur_desc_num/cur_desc_offset are commented out upstream, so the migration format is unaffected.
Patch 2 adds the reproducer. It is a bare-metal PowerPC payload booted with -kernel on -M pseries that speaks CRQ and SRP directly, with no guest operating system involved, so a failure is attributable to one function in QEMU rather than to something a kernel did on the way there. The fix comes first, so the tree is never left with a failing test, but the test applies on its own if a reviewer wants to put it first and watch it fail.
Testing. Against unpatched master the test reports 2 passed, 2 failed: the two controls hold -- the CRQ registers and the 256 KiB READ_10 completes with SRP status 0 -- and exactly the two assertions about where the data landed fail. With patch 1 it reports 4 passed, 0 failed. Both runs use the same payload and the same script; only qemu-system-ppc64 differs. Each patch was also confirmed to apply and compile on its own against master.
The committed ELF is 920 bytes and its source rebuilds to it byte for byte; "make check-reproducible" is there so the .S and the .elf cannot drift apart unnoticed. tests/multiboot is the precedent for committing a built payload and running it from a standalone script rather than a meson target.
The bug was found while trying to boot AIX 5.3 on -M pseries, where liblvm reads a volume group descriptor area in one 256 KiB command and gets the second half of it at offset 0. That is not needed to see the problem and is not part of this series; the reproducer needs no guest at all.
Bernard Ladenthin (2):
spapr_vscsi: fix data corruption on transfers above 128 KiB tests/spapr: add bare-metal reproducers for pseries machine behaviour
MAINTAINERS | 1 + hw/scsi/spapr_vscsi.c | 9 +- tests/spapr/Makefile | 62 ++++++++++ tests/spapr/link.ld | 3 + tests/spapr/run_test.sh | 108 ++++++++++++++++ tests/spapr/vscsibig.S | 285 +++++++++++++++++++++++++++++++++++++++++++ tests/spapr/vscsibig.elf | Bin 0 -> 920 bytes 7 files changed, 467 insertions(+), 1 deletion(-) create mode 100644 tests/spapr/Makefile create mode 100644 tests/spapr/link.ld create mode 100755 tests/spapr/run_test.sh create mode 100644 tests/spapr/vscsibig.S create mode 100644 tests/spapr/vscsibig.elf
--
2.47.3
PATCH 1/2 spapr_vscsi: fix data corruption on transfers above 128 KiB
scsi-disk hands the HBA at most SCSI_DMA_BUF_SIZE (128 KiB) per vscsi_transfer_data() callback, so a single SCSI command carrying more than that arrives in several chunks. vscsi_srp_direct_data() remembers how far into the descriptor it already is in req->cur_desc_offset -- but that field is uint16_t, so the very first 128 KiB chunk does
req->cur_desc_offset += 0x20000; /* 0x20000 & 0xFFFF == 0 */
and wraps straight back to zero. Every following chunk of the same command is then fetched with buf_offset 0 and DMA'd over the START of the guest buffer: the tail of the transfer overwrites the head, and the rest of the buffer is never written at all. The command still completes with status 0 and no sense data, so the guest cannot notice.
Linux never hits this. Its ibmvscsi initiator uses INDIRECT descriptors for anything multi-page, and the indirect path resets cur_desc_offset to 0 at every descriptor boundary, so the counter never reaches 65536. A guest that describes the whole transfer with a single DIRECT descriptor and issues 256 KiB commands does hit it on the first command.
Measured with a READ_10 of 2098 sectors, traced with spapr_vscsi_transfer_data and spapr_vscsi_fetch_desc_done:
queue_cmd tag 0x2 CMD 0x28=READ_10 LUN 0 ret: 262144
transfer_data tag=0x2 len=0x20000
fetch_desc cur=0 offs=0x0 ret { va=0x84d28 len=0x40000 }
transfer_data tag=0x2 len=0x20000
fetch_desc cur=0 offs=0x0 ret { va=0x84d28 len=0x40000 }
cmd_complete tag=0x2 status=0x0Same tag, same request, same target address twice. The guest buffer ends up holding the second half of the transfer at offset 0 and zeroes above it.
struct srp_direct_buf.len is already uint32_t and vscsi_fetch_desc() already takes buf_offset as unsigned, so widening the field is all that is required. The VMSTATE entries for cur_desc_num/cur_desc_offset are commented out upstream, so the migration format is unaffected.
---
This is my first QEMU submission. Corrections on anything I got wrong in the process are welcome.
hw/scsi/spapr_vscsi.c +8 -1
@@ -86,7 +86,14 @@ typedef struct vscsi_req {uint16_t total_desc;uint16_t cdb_offset;uint16_t cur_desc_num; -uint16_t cur_desc_offset; +/* +* Byte offset reached inside the current descriptor. Must be at +* least 32 bit: scsi-disk delivers a command in SCSI_DMA_BUF_SIZE +* (128 KiB) chunks, so a DIRECT descriptor larger than 64 KiB would +* wrap a uint16_t back to zero and make every chunk after the first +* overwrite the head of the guest buffer. +*/ +uint32_t cur_desc_offset; } vscsi_req;#define TYPE_VIO_SPAPR_VSCSI_DEVICE "spapr-vscsi"
PATCH 2/2 tests/spapr: add bare-metal reproducers for pseries machine behaviour
In reply to Bernard LadenthinThese are small PowerPC payloads booted with -kernel on -M pseries. They drive the hypervisor interface directly -- hypercalls, CRQ, SRP -- so no guest operating system is involved and a failure is attributable to one function in QEMU rather than to something a kernel did on the way there.
vscsibig.S is the first. It registers a CRQ, identity-maps its pages with H_PUT_TCE, clears the power-on unit attention, then issues a single 256 KiB READ_10 described by one DIRECT descriptor and checks where the data landed. The buffer is poisoned with 0xeeeeeeee first, which separates "the DMA never happened" from "the disk really holds zeros".
Following tests/multiboot, the built payload is committed so the test can be run without a PowerPC cross toolchain, and run_test.sh is a standalone script rather than a meson target. The ELF is 920 bytes: linking with -N keeps the loader from page-aligning the load segment, which would otherwise pad a 584-byte payload out to 65 KiB of zeros.
"make check-reproducible" rebuilds from source and compares against the committed binary, so the .S and the .elf cannot drift apart unnoticed.
--- MAINTAINERS | 1 + tests/spapr/Makefile | 62 +++++++++ tests/spapr/link.ld | 3 + tests/spapr/run_test.sh | 108 +++++++++++++++ tests/spapr/vscsibig.S | 285 +++++++++++++++++++++++++++++++++++++++ tests/spapr/vscsibig.elf | Bin 0 -> 920 bytes 6 files changed, 459 insertions(+) create mode 100644 tests/spapr/Makefile create mode 100644 tests/spapr/link.ld create mode 100644 tests/spapr/run_test.sh create mode 100644 tests/spapr/vscsibig.S create mode 100644 tests/spapr/vscsibig.elf
MAINTAINERS +1 -0
@@ -1658,6 +1658,7 @@ F: pc-bios/slof.bin F: docs/system/ppc/pseries.rst F: docs/specs/ppc-spapr-* F: tests/qtest/spapr* +F: tests/spapr/ F: tests/qtest/libqos/*spapr* F: tests/qtest/rtas* F: tests/qtest/libqos/rtas*
tests/spapr/Makefile new file +62 -0
62 lines not on this page · raw message
tests/spapr/link.ld new file +3 -0
@@ -0,0 +1,3 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +ENTRY(_start) +SECTIONS { . = 0x400000; .text : { *(.text) } }
tests/spapr/run_test.sh new file +108 -0
108 lines not on this page · raw message
tests/spapr/vscsibig.S new file +285 -0
285 lines not on this page · raw message
tests/spapr/vscsibig.elf new file binary +0 -0
Signature · 1 line
2.49.0.windows.1