block/nvme: Fix VFIO_MAP_DMA failed: No space left on device
PATCH v3 1/1 block/nvme: Fix VFIO_MAP_DMA failed: No space left on device
When the NVMe block driver was introduced (see commit bdd6a90a9e5, January 2018), Linux VFIO_IOMMU_MAP_DMA ioctl was only returning -ENOMEM in case of error. The driver was correctly handling the error path to recycle its volatile IOVA mappings.
To fix CVE-2019-3882, Linux commit 492855939bdb ("vfio/type1: Limit DMA mappings per container", April 2019) added the -ENOSPC error to signal the user exhausted the DMA mappings available for a container.
The block driver started to mis-behave:
qemu-system-x86_64: VFIO_MAP_DMA failed: No space left on device (qemu)
(qemu) info status
VM status: paused (io-error)
(qemu) c
VFIO_MAP_DMA failed: No space left on device
(qemu) c
VFIO_MAP_DMA failed: No space left on device
(The VM is not resumable from here, hence stuck.)
Fix by handling the new -ENOSPC error (when DMA mappings are exhausted) without any distinction to the current -ENOMEM error, so we don't change the behavior on old kernels where the CVE-2019-3882 fix is not present.
An easy way to reproduce this bug is to restrict the DMA mapping limit (65535 by default) when loading the VFIO IOMMU module:
# modprobe vfio_iommu_type1 dma_entry_limit=666
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/65
---
v3: Reworded (Fam)
v2: KISS checking both errors undistinguishedly (Maxim)
--- block/nvme.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+)
block/nvme.c +22 -0
@@ -1030,7 +1030,29 @@ try_map: r =qemu_vfio_dma_map (s->vfio, qiov->iov[i].iov_base, len, true, &iova); +if (r == -ENOSPC ) { +/* +* In addition to the -ENOMEM error, the VFIO_IOMMU_MAP_DMA +* ioctl returns -ENOSPC to signal the user exhausted the DMA +* mappings available for a container since Linux kernel commit +* 492855939bdb ("vfio/type1: Limit DMA mappings per container", +* April 2019, see CVE-2019-3882). +* +* This block driver already handles this error path by checking +* for the -ENOMEM error, so we directly replace -ENOSPC by +* -ENOMEM. Beside, -ENOSPC has a specific meaning for blockdev +* coroutines: it triggers BLOCKDEV_ON_ERROR_ENOSPC and +* BLOCK_ERROR_ACTION_STOP which stops the VM, asking the operator +* to add more storage to the blockdev. Not something we can do +* easily with an IOMMU :) +*/ + r = -ENOMEM ; + }if (r == -ENOMEM && retry) { +/* +* We exhausted the DMA mappings available for our container: +* recycle the volatile IOVA mappings. +*/ retry = false;trace_nvme_dma_flush_queue_wait (s);if (s->dma_map_count) {
Signature · 1 line
2.31.1
Stefan Hajnoczi
AppliedQuoted 39 lines · Philippe Mathieu-Daudé
When the NVMe block driver was introduced (see commit
bdd6a90a9e5, January 2018), LinuxVFIO_IOMMU_MAP_DMAioctl was only returning-ENOMEMin case of error. The driver was correctly handling the error path to recycle its volatile IOVA mappings.To fix CVE-2019-3882, Linux commit
492855939bdb("vfio/type1: Limit DMA mappings per container", April 2019) added the-ENOSPCerror to signal the user exhausted the DMA mappings available for a container.The block driver started to mis-behave:
qemu-system-x86_64:
VFIO_MAP_DMAfailed: No space left on device (qemu)
(qemu) info status
VM status: paused (io-error)
(qemu) c
VFIO_MAP_DMAfailed: No space left on device
(qemu) c
VFIO_MAP_DMAfailed: No space left on device(The VM is not resumable from here, hence stuck.)
Fix by handling the new
-ENOSPCerror (when DMA mappings are exhausted) without any distinction to the current-ENOMEMerror, so we don't change the behavior on old kernels where the CVE-2019-3882 fix is not present.An easy way to reproduce this bug is to restrict the DMA mapping limit (65535 by default) when loading the VFIO IOMMU module:
# modprobe vfio_iommu_type1 dma_entry_limit=666Cc Fam Zheng <[email protected]>Cc Maxim Levitsky <[email protected]>Cc Alex Williamson <[email protected]>Reported-by Michal Prívozník <[email protected]>Fixes bdd6a90a9e5 ("block: Add VFIO based NVMe driver")
Quoted 8 lines
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/65
Signed-off-by Philippe Mathieu-Daudé <[email protected]>---
v3: Reworded (Fam)
v2: KISS checking both errors undistinguishedly (Maxim)--- block/nvme.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+)
Thanks, applied to my block tree:
https://gitlab.com/stefanha/qemu/commits/block
Stefan