virtio disk host cpu overhead?
I've been testing a new samsung 960 evo nvme drive i just got, and running tests on it in a guest shows a minimum of double the cpu time on the host as it does on the guest. Somehow a guest running a single thread iozone or fio test can use 4+ cores on the host side at 50-100%.
I have debian's 4.9.13 kernel installed on the host (4.9.2 on the guest), and kvm/qemu 2.8.0. I'm using libvirt and virt-manager to set up the guest, with a lvm volume set up on the nvme drive, and exporting a couple volumes to the guest which it then either uses lvm on, or testing is done raw on a secondary volume. (I've tried both)
Host is a consumer box with an AMD 970 board + FX-8320, and 32GB ram. Guest is currently allocated 8 threads, and 1GB ram which it is currently only using 111MB of while fio is running.
I'm curious to know if this is expected behavior, misconfiguration on my end, or a misunderstanding of how things work?
I'd appreciate any insight or help anyone is able and willing to provide.
Signature · 2 lines
Thomas Fjellstrom [email protected]
Quoted 4 lines · Thomas Fjellstrom
I've been testing a new samsung 960 evo nvme drive i just got, and running tests on it in a guest shows a minimum of double the cpu time on the host as it does on the guest. Somehow a guest running a single thread iozone or fio test can use 4+ cores on the host side at 50-100%.
Are you using -drive aio=native?
Are you pinning threads on the host?
If the answer to both questions is no, then QEMU uses a thread pool for disk I/O and those threads could be scheduled on any host CPU.
Stefan
Quoted 7 lines · MST Stefan Hajnoczi
Quoted 4 lines · Thomas Fjellstrom
I've been testing a new samsung 960 evo nvme drive i just got, and running tests on it in a guest shows a minimum of double the cpu time on the host as it does on the guest. Somehow a guest running a single thread iozone or fio test can use 4+ cores on the host side at 50-100%.
Are you using -drive aio=native?
Yes.
Quoted 1 line
Are you pinning threads on the host?
No pinning by default.
Quoted 2 lines
If the answer to both questions is no, then QEMU uses a thread pool for disk I/O and those threads could be scheduled on any host CPU.
But should they be pegging 4 or more cores, when the host can do the same work with a single core? I don't mind it spreading work out. But using 4x the cpu time to do the same work is a bit much.
Quoted 1 line
Stefan
Signature · 2 lines
Thomas Fjellstrom [email protected]
Paolo Bonzini
maintainerQuoted 5 lines · Thomas Fjellstrom
Quoted 2 lines
If the answer to both questions is no, then QEMU uses a thread pool for disk I/O and those threads could be scheduled on any host CPU.
But should they be pegging 4 or more cores, when the host can do the same work with a single core? I don't mind it spreading work out. But using 4x the cpu time to do the same work is a bit much.
I would expect 2x, but not really more than that. Which threads are doing all the work, beyond the VCPU thread and the host I/O thread?
Paolo