util/vfio-helpers: Generic code strengthening
v3 cover util/vfio-helpers: Generic code strengthening
v3:
- Extract reviewed patches from
"util/vfio-helpers: Allow using multiple MSIX IRQs"
- Added "Assert offset is aligned to page size"
which would have helped debugging:
"block/nvme: Fix use of write-only doorbells page on Aarch64 arch"
Missing review: 7
Based-on: <[email protected]>
Philippe Mathieu-Daudé (7):
util/vfio-helpers: Improve reporting unsupported IOMMU type util/vfio-helpers: Trace PCI I/O config accesses util/vfio-helpers: Trace PCI BAR region info util/vfio-helpers: Trace where BARs are mapped util/vfio-helpers: Improve DMA trace events util/vfio-helpers: Convert vfio_dump_mapping to trace events util/vfio-helpers: Assert offset is aligned to page size
util/vfio-helpers.c | 43 ++++++++++++++++++++++++++----------------- util/trace-events | 10 ++++++++-- 2 files changed, 34 insertions(+), 19 deletions(-)
Signature · 1 line
2.26.2
PATCH v3 1/7 util/vfio-helpers: Improve reporting unsupported IOMMU type
Change the confuse "VFIO IOMMU check failed" error message by the explicit "VFIO IOMMU Type1 is not supported" once.
Example on POWER:
$ qemu-system-ppc64 -drive if=none,id=nvme0,file=nvme://0001:01:00.0/1,format=raw qemu-system-ppc64: -drive if=none,id=nvme0,file=nvme://0001:01:00.0/1,format=raw: VFIO IOMMU Type1 is not supported
--- util/vfio-helpers.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
util/vfio-helpers.c +1 -1
@@ -300,7 +300,7 @@ static int qemu_vfio_init_pci(QEMUVFIOState *s, const char *device, }if (!ioctl (s->container,VFIO_CHECK_EXTENSION ,VFIO_TYPE1_IOMMU )) { -error_setg_errno (errp, errno,"VFIO IOMMU check failed" ); +error_setg_errno (errp, errno,"VFIO IOMMU Type1 is not supported" ); ret = -EINVAL ; goto fail_container; }
Signature · 1 line
2.26.2
PATCH v3 2/7 util/vfio-helpers: Trace PCI I/O config accesses
In reply to Philippe Mathieu-DaudéWe sometime get kernel panic with some devices on Aarch64 hosts. Alex Williamson suggests it might be broken PCIe root complex. Add trace event to record the latest I/O access before crashing. In case, assert our accesses are aligned.
--- util/vfio-helpers.c | 8 ++++++++ util/trace-events | 2 ++ 2 files changed, 10 insertions(+)
util/vfio-helpers.c +8 -0
@@ -227,6 +227,10 @@ static int qemu_vfio_pci_read_config(QEMUVFIOState *s, void *buf, {int ret; +trace_qemu_vfio_pci_read_config (buf, ofs, size, + s->config_region_info.offset, + s->config_region_info.size); +assert (QEMU_IS_ALIGNED (s->config_region_info.offset + ofs, size));do { ret =pread (s->device, buf, size, s->config_region_info.offset + ofs); }while (ret ==-1 && errno ==EINTR );
@@ -237,6 +241,10 @@ static int qemu_vfio_pci_write_config(QEMUVFIOState *s, void *buf, int size, int {int ret; +trace_qemu_vfio_pci_write_config (buf, ofs, size, + s->config_region_info.offset, + s->config_region_info.size); +assert (QEMU_IS_ALIGNED (s->config_region_info.offset + ofs, size));do { ret =pwrite (s->device, buf, size, s->config_region_info.offset + ofs); }while (ret ==-1 && errno ==EINTR );
util/trace-events +2 -0
@@ -85,3 +85,5 @@ qemu_vfio_new_mapping(void *s, void *host, size_t size, int index, uint64_t iova qemu_vfio_do_mapping(void *s, void *host, size_t size, uint64_t iova) "s %p host %p size 0x%zx iova 0x%"PRIx64 qemu_vfio_dma_map(void *s, void *host, size_t size, bool temporary, uint64_t *iova) "s %p host %p size 0x%zx temporary %d iova %p" qemu_vfio_dma_unmap(void *s, void *host) "s %p host %p" +qemu_vfio_pci_read_config(void *buf, int ofs, int size, uint64_t region_ofs, uint64_t region_size) "read cfg ptr %p ofs 0x%x size 0x%x (region addr 0x%"PRIx64" size 0x%"PRIx64")" +qemu_vfio_pci_write_config(void *buf, int ofs, int size, uint64_t region_ofs, uint64_t region_size) "write cfg ptr %p ofs 0x%x size 0x%x (region addr 0x%"PRIx64" size 0x%"PRIx64")"
Signature · 1 line
2.26.2
PATCH v3 3/7 util/vfio-helpers: Trace PCI BAR region info
In reply to Philippe Mathieu-DaudéFor debug purpose, trace BAR regions info.
--- util/vfio-helpers.c | 8 ++++++++ util/trace-events | 1 + 2 files changed, 9 insertions(+)
util/vfio-helpers.c +8 -0
@@ -136,6 +136,7 @@ static inline void assert_bar_index_valid(QEMUVFIOState *s, int index)static int qemu_vfio_pci_init_bar (QEMUVFIOState *s,int index,Error **errp) { + g_autofree char *barname = NULL;assert_bar_index_valid (s, index); s->bar_region_info[index] = (struct vfio_region_info ) { .index =VFIO_PCI_BAR0_REGION_INDEX + index,
@@ -145,6 +146,10 @@ static int qemu_vfio_pci_init_bar(QEMUVFIOState *s, int index, Error **errp)error_setg_errno (errp, errno,"Failed to get BAR region info" );return -errno; } + barname =g_strdup_printf ("bar[%d]" , index); +trace_qemu_vfio_region_info (barname, s->bar_region_info[index].offset, + s->bar_region_info[index].size, + s->bar_region_info[index].cap_offset);return 0 ; }
@@ -416,6 +421,9 @@ static int qemu_vfio_init_pci(QEMUVFIOState *s, const char *device, ret = -errno; goto fail; } +trace_qemu_vfio_region_info ("config" , s->config_region_info.offset, + s->config_region_info.size, + s->config_region_info.cap_offset);for (i =0 ; i <ARRAY_SIZE (s->bar_region_info); i++) { ret =qemu_vfio_pci_init_bar (s, i, errp);
util/trace-events +1 -0
@@ -87,3 +87,4 @@ qemu_vfio_dma_map(void *s, void *host, size_t size, bool temporary, uint64_t *io qemu_vfio_dma_unmap(void *s, void *host) "s %p host %p" qemu_vfio_pci_read_config(void *buf, int ofs, int size, uint64_t region_ofs, uint64_t region_size) "read cfg ptr %p ofs 0x%x size 0x%x (region addr 0x%"PRIx64" size 0x%"PRIx64")" qemu_vfio_pci_write_config(void *buf, int ofs, int size, uint64_t region_ofs, uint64_t region_size) "write cfg ptr %p ofs 0x%x size 0x%x (region addr 0x%"PRIx64" size 0x%"PRIx64")" +qemu_vfio_region_info(const char *desc, uint64_t region_ofs, uint64_t region_size, uint32_t cap_offset) "region '%s' addr 0x%"PRIx64" size 0x%"PRIx64" cap_ofs 0x%"PRIx32
Signature · 1 line
2.26.2
PATCH v3 4/7 util/vfio-helpers: Trace where BARs are mapped
In reply to Philippe Mathieu-DaudéFor debugging purpose, trace where a BAR is mapped.
--- util/vfio-helpers.c | 2 ++ util/trace-events | 1 + 2 files changed, 3 insertions(+)
util/vfio-helpers.c +2 -0
@@ -166,6 +166,8 @@ void *qemu_vfio_pci_map_bar(QEMUVFIOState *s, int index, p =mmap (NULL ,MIN (size, s->bar_region_info[index].size - offset), prot,MAP_SHARED , s->device, s->bar_region_info[index].offset + offset); +trace_qemu_vfio_pci_map_bar (index, s->bar_region_info[index].offset , + size, offset, p);if (p ==MAP_FAILED ) {error_setg_errno (errp, errno,"Failed to map BAR region" ); p =NULL ;
util/trace-events +1 -0
@@ -88,3 +88,4 @@ qemu_vfio_dma_unmap(void *s, void *host) "s %p host %p" qemu_vfio_pci_read_config(void *buf, int ofs, int size, uint64_t region_ofs, uint64_t region_size) "read cfg ptr %p ofs 0x%x size 0x%x (region addr 0x%"PRIx64" size 0x%"PRIx64")" qemu_vfio_pci_write_config(void *buf, int ofs, int size, uint64_t region_ofs, uint64_t region_size) "write cfg ptr %p ofs 0x%x size 0x%x (region addr 0x%"PRIx64" size 0x%"PRIx64")" qemu_vfio_region_info(const char *desc, uint64_t region_ofs, uint64_t region_size, uint32_t cap_offset) "region '%s' addr 0x%"PRIx64" size 0x%"PRIx64" cap_ofs 0x%"PRIx32 +qemu_vfio_pci_map_bar(int index, uint64_t region_ofs, uint64_t region_size, int ofs, void *host) "map region bar#%d addr 0x%"PRIx64" size 0x%"PRIx64" ofs 0x%x host %p"
Signature · 1 line
2.26.2
PATCH v3 5/7 util/vfio-helpers: Improve DMA trace events
In reply to Philippe Mathieu-DaudéFor debugging purpose, trace where DMA regions are mapped.
--- util/vfio-helpers.c | 3 ++- util/trace-events | 5 +++-- 2 files changed, 5 insertions(+), 3 deletions(-)
util/vfio-helpers.c +2 -1
@@ -627,7 +627,7 @@ static int qemu_vfio_do_mapping(QEMUVFIOState *s, void *host, size_t size, .vaddr = (uintptr_t)host, .size = size, }; -trace_qemu_vfio_do_mapping (s, host, size, iova); +trace_qemu_vfio_do_mapping (s, host, iova, size);if (ioctl (s->container,VFIO_IOMMU_MAP_DMA , &dma_map)) {error_report ("VFIO_MAP_DMA failed: %s" ,strerror (errno));
@@ -783,6 +783,7 @@ int qemu_vfio_dma_map(QEMUVFIOState *s, void *host, size_t size, } } } +trace_qemu_vfio_dma_mapped (s, host, iova0, size);if (iova) { *iova = iova0; }
util/trace-events +3 -2
@@ -82,8 +82,9 @@ qemu_vfio_ram_block_added(void *s, void *p, size_t size) "s %p host %p size 0x%z qemu_vfio_ram_block_removed(void *s, void *p, size_t size) "s %p host %p size 0x%zx" qemu_vfio_find_mapping(void *s, void *p) "s %p host %p" qemu_vfio_new_mapping(void *s, void *host, size_t size, int index, uint64_t iova) "s %p host %p size 0x%zx index %d iova 0x%"PRIx64 -qemu_vfio_do_mapping(void *s, void *host, size_t size, uint64_t iova) "s %p host %p size 0x%zx iova 0x%"PRIx64 -qemu_vfio_dma_map(void *s, void *host, size_t size, bool temporary, uint64_t *iova) "s %p host %p size 0x%zx temporary %d iova %p" +qemu_vfio_do_mapping(void *s, void *host, uint64_t iova, size_t size) "s %p host %p <-> iova 0x%"PRIx64 " size 0x%zx" +qemu_vfio_dma_map(void *s, void *host, size_t size, bool temporary, uint64_t *iova) "s %p host %p size 0x%zx temporary %d &iova %p" +qemu_vfio_dma_mapped(void *s, void *host, uint64_t iova, size_t size) "s %p host %p <-> iova 0x%"PRIx64" size 0x%zx" qemu_vfio_dma_unmap(void *s, void *host) "s %p host %p" qemu_vfio_pci_read_config(void *buf, int ofs, int size, uint64_t region_ofs, uint64_t region_size) "read cfg ptr %p ofs 0x%x size 0x%x (region addr 0x%"PRIx64" size 0x%"PRIx64")" qemu_vfio_pci_write_config(void *buf, int ofs, int size, uint64_t region_ofs, uint64_t region_size) "write cfg ptr %p ofs 0x%x size 0x%x (region addr 0x%"PRIx64" size 0x%"PRIx64")"
Signature · 1 line
2.26.2
PATCH v3 6/7 util/vfio-helpers: Convert vfio_dump_mapping to trace events
In reply to Philippe Mathieu-DaudéThe QEMU_VFIO_DEBUG definition is only modifiable at build-time. Trace events can be enabled at run-time. As we prefer the latter, convert qemu_vfio_dump_mappings() to use trace events instead of fprintf().
--- util/vfio-helpers.c | 19 ++++--------------- util/trace-events | 1 + 2 files changed, 5 insertions(+), 15 deletions(-)
util/vfio-helpers.c +4 -15
@@ -521,23 +521,12 @@ QEMUVFIOState *qemu_vfio_open_pci(const char *device, Error **errp)return s; } -static void qemu_vfio_dump_mapping (IOVAMapping *m) -{ -if (QEMU_VFIO_DEBUG ) { -printf (" vfio mapping %p %" PRIx64" to %" PRIx64"\n" , m->host, - (uint64_t )m->size, (uint64_t )m->iova); - } -} -static void qemu_vfio_dump_mappings (QEMUVFIOState *s) { -int i; - -if (QEMU_VFIO_DEBUG ) { -printf ("vfio mappings\n" ); -for (i =0 ; i < s->nr_mappings; ++i) { -qemu_vfio_dump_mapping (&s->mappings[i]); - } +for (int i =0 ; i < s->nr_mappings; ++i) { +trace_qemu_vfio_dump_mapping (s->mappings[i].host, + s->mappings[i].iova, + s->mappings[i].size); } }
util/trace-events +1 -0
@@ -80,6 +80,7 @@ qemu_mutex_unlock(void *mutex, const char *file, const int line) "released mutex qemu_vfio_dma_reset_temporary(void *s) "s %p" qemu_vfio_ram_block_added(void *s, void *p, size_t size) "s %p host %p size 0x%zx" qemu_vfio_ram_block_removed(void *s, void *p, size_t size) "s %p host %p size 0x%zx" +qemu_vfio_dump_mapping(void *host, uint64_t iova, size_t size) "vfio mapping %p to iova 0x%08" PRIx64 " size 0x%zx" qemu_vfio_find_mapping(void *s, void *p) "s %p host %p" qemu_vfio_new_mapping(void *s, void *host, size_t size, int index, uint64_t iova) "s %p host %p size 0x%zx index %d iova 0x%"PRIx64 qemu_vfio_do_mapping(void *s, void *host, uint64_t iova, size_t size) "s %p host %p <-> iova 0x%"PRIx64 " size 0x%zx"
Signature · 1 line
2.26.2
PATCH v3 7/7 util/vfio-helpers: Assert offset is aligned to page size
In reply to Philippe Mathieu-Daudémmap(2) states:
'offset' must be a multiple of the page size as returned
by sysconf(_SC_PAGE_SIZE).
Add an assertion to be sure we don't break this contract.
--- util/vfio-helpers.c | 1 + 1 file changed, 1 insertion(+)
util/vfio-helpers.c +1 -0
@@ -162,6 +162,7 @@ void *qemu_vfio_pci_map_bar(QEMUVFIOState *s, int index, Error **errp) {void *p; +assert (QEMU_IS_ALIGNED (offset, qemu_real_host_page_size));assert_bar_index_valid (s, index); p = mmap(NULL, MIN(size, s->bar_region_info[index].size - offset), prot, MAP_SHARED,
Signature · 1 line
2.26.2
Stefan Hajnoczi
AppliedQuoted 20 lines · Philippe Mathieu-Daudé
v3:
- Extract reviewed patches from
"util/vfio-helpers: Allow using multiple MSIX IRQs"
- Added "Assert offset is aligned to page size"
which would have helped debugging:
"block/nvme: Fix use of write-only doorbells page on Aarch64 arch"Missing review: 7
Based-on: <[email protected]>
Philippe Mathieu-Daudé (7):
util/vfio-helpers: Improve reporting unsupported IOMMU type util/vfio-helpers: Trace PCI I/O config accesses util/vfio-helpers: Trace PCI BAR region info util/vfio-helpers: Trace where BARs are mapped util/vfio-helpers: Improve DMA trace events util/vfio-helpers: Convertvfio_dump_mappingto trace events util/vfio-helpers: Assert offset is aligned to page size
Quoted 8 lines
util/vfio-helpers.c | 43 ++++++++++++++++++++++++++----------------- util/trace-events | 10 ++++++++-- 2 files changed, 34 insertions(+), 19 deletions(-)Signature · 1 line
2.26.2
Thanks, applied to my block tree:
https://gitlab.com/stefanha/qemu/commits/block
Stefan
Hi Philippe,
Quoted 10 lines · Philippe Mathieu-Daudé
v3:
- Extract reviewed patches from
"util/vfio-helpers: Allow using multiple MSIX IRQs"
- Added "Assert offset is aligned to page size"
which would have helped debugging:
"block/nvme: Fix use of write-only doorbells page on Aarch64 arch"Missing review: 7
Based-on: <[email protected]>
Thanks
Eric
Quoted 14 lines
Philippe Mathieu-Daudé (7):
util/vfio-helpers: Improve reporting unsupported IOMMU type util/vfio-helpers: Trace PCI I/O config accesses util/vfio-helpers: Trace PCI BAR region info util/vfio-helpers: Trace where BARs are mapped util/vfio-helpers: Improve DMA trace events util/vfio-helpers: Convertvfio_dump_mappingto trace events util/vfio-helpers: Assert offset is aligned to page sizeutil/vfio-helpers.c | 43 ++++++++++++++++++++++++++----------------- util/trace-events | 10 ++++++++-- 2 files changed, 34 insertions(+), 19 deletions(-)