system: Try hardware accelerators (KVM, HVF) before software one (TCG)
PATCH 1/1 system: Try hardware accelerators (KVM, HVF) before software one (TCG)
As Daniel suggested [*]:
Quoted 3 lines
We should consider to rank HVF above TCG, on the basis that HW acceleration is faster and should provide a host<->guest security boundary that we don't claim for TCG
[*] https://lore.kernel.org/qemu-devel/[email protected]/
--- system/vl.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
system/vl.c +2 -3
@@ -2379,7 +2379,6 @@ static void configure_accelerators(const char *progname)/* Select the default accelerator */ bool have_tcg =accel_find ("tcg" );bool have_kvm =accel_find ("kvm" ); -bool have_hvf =accel_find ("hvf" );if (have_tcg && have_kvm) {if (g_str_has_suffix (progname,"kvm" )) {
@@ -2390,10 +2389,10 @@ static void configure_accelerators(const char *progname) } }else if (have_kvm) { accelerators ="kvm" ; + }else if (accel_find ("hvf" )) { + accelerators ="hvf" ; }else if (have_tcg) { accelerators ="tcg" ; - }else if (have_hvf) { - accelerators ="hvf" ; }else {error_report ("No accelerator selected and" " no default accelerator available" );
Signature · 1 line
2.47.1
Quoted 7 lines · Philippe Mathieu-Daudé
As Daniel suggested [*]:
Quoted 3 lines
We should consider to rank HVF above TCG, on the basis that HW acceleration is faster and should provide a host<->guest security boundary that we don't claim for TCG
Note, my statement above was on the basis that HVF passes all our functional tests, thus indicating a decent level of confidence in the correctness of the HVF impl.
If anyone knows any show stopper problems with HVF that would justify blocking its promotion ahead of TCG.... say now.
Quoted 5 lines
Suggested-by Daniel P. Berrangé <[email protected]>Signed-off-by Philippe Mathieu-Daudé <[email protected]>--- system/vl.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Quoted 7 lines · Daniel P. Berrangé
Quoted 6 lines · Philippe Mathieu-Daudé
As Daniel suggested [*]:
Quoted 3 lines
We should consider to rank HVF above TCG, on the basis that HW acceleration is faster and should provide a host<->guest security boundary that we don't claim for TCG
Quoted 5 lines
Quoted 1 line
Note, my statement above was on the basis that HVF passes all our functional tests, thus indicating a decent level of confidence in the correctness of the HVF impl.
Indeed, I forgot about that, and only tested in my 'HVF-only' directory before posting, but ...
Quoted 2 lines
If anyone knows any show stopper problems with HVF that would justify blocking its promotion ahead of TCG.... say now.
... here we go:
3/15 qemu:qtest+qtest-aarch64 / qtest-aarch64/migration-test
ERROR 0.88s killed by signal 11 SIGSEGV
Quoted 11 lines
Quoted 5 lines
Suggested-by Daniel P. Berrangé <[email protected]>Signed-off-by Philippe Mathieu-Daudé <[email protected]>--- system/vl.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)Reviewed-by Daniel P. Berrangé <[email protected]>With regards,
Daniel
Quoted 17 lines · Philippe Mathieu-Daudé
Quoted 7 lines · Daniel P. Berrangé
Quoted 6 lines · Philippe Mathieu-Daudé
As Daniel suggested [*]:
Quoted 3 lines
We should consider to rank HVF above TCG, on the basis that HW acceleration is faster and should provide a host<->guest security boundary that we don't claim for TCG
Quoted 5 lines
Quoted 1 line
Note, my statement above was on the basis that HVF passes all our functional tests, thus indicating a decent level of confidence in the correctness of the HVF impl.
Indeed, I forgot about that, and only tested in my 'HVF-only' directory before posting, but ...
Quoted 7 lines
Quoted 2 lines
If anyone knows any show stopper problems with HVF that would justify blocking its promotion ahead of TCG.... say now.
... here we go:
3/15 qemu:qtest+qtest-aarch64 / qtest-aarch64/migration-test ERROR
0.88s killed by signal 11 SIGSEGV
Hmm.. I think migration-test specifies either kvm or tcg in all its tests, so I don't yet know why this patch can affect it..
migrate_start():
cmd_source = g_strdup_printf("-accel kvm%s -accel tcg "
"-machine %s,%s "
...);May need a closer look on the crashed stack.
Signature · 1 line
Peter Xu
Quoted 7 lines · Daniel P. Berrangé
Quoted 6 lines · Philippe Mathieu-Daudé
As Daniel suggested [*]:
Quoted 3 lines
We should consider to rank HVF above TCG, on the basis that HW acceleration is faster and should provide a host<->guest security boundary that we don't claim for TCG
Quoted 9 lines
Quoted 1 line
Note, my statement above was on the basis that HVF passes all our functional tests, thus indicating a decent level of confidence in the correctness of the HVF impl.
If anyone knows any show stopper problems with HVF that would justify blocking its promotion ahead of TCG.... say now.
I don't know about showstoppers, but:
1. As far as I'm aware there are/were problems with the virtual IOMMU devices in HVF. It's been a while (~half a year?) since I tried them, but I had problems getting guests booted with intel_iommu etc.
2. I think there might also be a few remaining edge cases where the x86 instruction emulation on fault/trap is incomplete. Most notably, MMIO using SSE/AVX/etc. instructions will, I think, fail. In practice this is a fairly obscure use case - I'm not aware of any guest OS that actually performs MMIO using these instructions. I have a patch kicking around that adds support for missing 64-bit variants of common scalar arithmetic instructions with memory operands. I can dig that up and post it - do we have a good way of adding tests for this kind of thing?
3. As far as I'm aware, there's no CI happening on HVF? Or has that changed? macOS is notoriously a pain in the rear in terms of CI thanks to its licensing, and the big cloud CI platforms tend to run it in a VM which in turn typically doesn't support nested HVF. I've been working on an on-prem solution to provisioning bare-metal Macs to run clean-slate OS images for CI. This has been a side project though and I haven't had the resources to focus on that project to see it through. It might be possible to do this in the cloud on Amazon's EC2 Mac Minis as well, but those aren't exactly cheap.
Quoted 21 lines
Quoted 5 lines
Suggested-by Daniel P. Berrangé <[email protected]>Signed-off-by Philippe Mathieu-Daudé <[email protected]>--- system/vl.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)Reviewed-by Daniel P. Berrangé <[email protected]>With regards,
Daniel
--
|: https://berrange.com -o-
https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-
https://fstop138.berrange.com :|
|: https://entangle-photo.org -o-
https://www.instagram.com/dberrange :|
Quoted 19 lines · Phil Dennis-Jordan
Quoted 7 lines · Daniel P. Berrangé
Quoted 6 lines · Philippe Mathieu-Daudé
As Daniel suggested [*]:
Quoted 3 lines
We should consider to rank HVF above TCG, on the basis that HW acceleration is faster and should provide a host<->guest security boundary that we don't claim for TCG
Quoted 9 lines
Quoted 1 line
Note, my statement above was on the basis that HVF passes all our functional tests, thus indicating a decent level of confidence in the correctness of the HVF impl.
If anyone knows any show stopper problems with HVF that would justify blocking its promotion ahead of TCG.... say now.
Quoted 5 lines
I don't know about showstoppers, but:
1. As far as I'm aware there are/were problems with the virtual IOMMU devices in HVF. It's been a while (~half a year?) since I tried them, but I had problems getting guests booted with
intel_iommuetc.
I think that vIOMMU is niche enough that we can merely consider it a nice-to-fix bug, and not block promoting HVF.
Quoted 8 lines
2. I think there might also be a few remaining edge cases where the x86 instruction emulation on fault/trap is incomplete. Most notably, MMIO using SSE/AVX/etc. instructions will, I think, fail. In practice this is a fairly obscure use case - I'm not aware of any guest OS that actually performs MMIO using these instructions. I have a patch kicking around that adds support for missing 64-bit variants of common scalar arithmetic instructions with memory operands. I can dig that up and post it - do we have a good way of adding tests for this kind of thing?
Not sure how best to test this, other than finding a guest OS that exhibits this ? Others probably have better suggestions...
Quoted 9 lines
3. As far as I'm aware, there's no CI happening on HVF? Or has that changed? macOS is notoriously a pain in the rear in terms of CI thanks to its licensing, and the big cloud CI platforms tend to run it in a VM which in turn typically doesn't support nested HVF. I've been working on an on-prem solution to provisioning bare-metal Macs to run clean-slate OS images for CI. This has been a side project though and I haven't had the resources to focus on that project to see it through. It might be possible to do this in the cloud on Amazon's EC2 Mac Minis as well, but those aren't exactly cheap.
The only CI we have is running under Cirrus CI which uses VMs on real mac aarch64 hardware, but I don't think we can test HVF there.
Mostly we rely on regular contributors periodically running tests and reporting on problems. This is not ideal, but also not a blocker for enabling HVF - it just means macOS isn't a full tier 1 platform for us.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Quoted 23 lines · Daniel P. Berrangé
Quoted 1 line · Phil Dennis-Jordan
On Fri, 3 Jan 2025 at 16:16, Daniel P. Berrangé <[email protected]>
Quoted 20 lines · wrote:
Quoted 8 lines · Philippe Mathieu-Daudé
Quoted 6 lines
As Daniel suggested [*]:
Quoted 3 lines
We should consider to rank HVF above TCG, on the basis that HW acceleration is faster and should provide a host<->guest security boundary that we don't claim for TCG
Quoted 9 lines
Quoted 1 line
Note, my statement above was on the basis that HVF passes all our functional tests, thus indicating a decent level of confidence in the correctness of the HVF impl.
If anyone knows any show stopper problems with HVF that would justify blocking its promotion ahead of TCG.... say now.
I don't know about showstoppers, but:
Quoted 8 lines
Quoted 2 lines
1. As far as I'm aware there are/were problems with the virtual IOMMU devices in HVF. It's been a while (~half a year?) since I tried them,
but I
Quoted 1 line
had problems getting guests booted with
intel_iommuetc.I think that vIOMMU is niche enough that we can merely consider it a nice-to-fix bug, and not block promoting HVF.
Yes, I assume those are mainly useful with very large numbers of vCPUs and/or PCI device passthrough? Neither of those are relevant on macOS hosts. (One other thing they're useful with is device driver development/testing/debugging - real Macs come with IOMMUs so PCI/Thunderbolt device drivers MUST translate addresses for DMA correctly - but that's very niche indeed.)
Quoted 5 lines
2. I think there might also be a few remaining edge cases where the x86
Quoted 1 line
instruction emulation on fault/trap is incomplete. Most notably, MMIO
using
Quoted 1 line
SSE/AVX/etc. instructions will, I think, fail. In practice this is a
fairly
Quoted 9 lines
Quoted 5 lines
obscure use case - I'm not aware of any guest OS that actually performs MMIO using these instructions. I have a patch kicking around that adds support for missing 64-bit variants of common scalar arithmetic instructions with memory operands. I can dig that up and post it - do we have a good way of adding tests for this kind of thing?
Not sure how best to test this, other than finding a guest OS that exhibits this ? Others probably have better suggestions...
I encountered it while trying the aforementioned vIOMMU; macOS guests seemed to use 64-bit instructions for accessing some of the registers.
I've just checked, and there actually seem to be some instruction emulation tests in the kvm-unit-tests project; those are designed for KVM, but I expect it should be possible to tweak some of them so they're useful for testing HVF. kvm-unit-tests don't seem to be part of QEMU's regular test suite at the moment though, as far as I can tell?
Quoted 8 lines
Quoted 3 lines
3. As far as I'm aware, there's no CI happening on HVF? Or has that changed? macOS is notoriously a pain in the rear in terms of CI thanks to its licensing, and the big cloud CI platforms tend to run it in a VM
which
Quoted 4 lines
in turn typically doesn't support nested HVF. I've been working on an on-prem solution to provisioning bare-metal Macs to run clean-slate OS images for CI. This has been a side project though and I haven't had the resources to focus on that project to see it through. It might be
Quoted 8 lines
possible
Quoted 1 line
to do this in the cloud on Amazon's EC2 Mac Minis as well, but those
aren't
Quoted 1 line
exactly cheap.
The only CI we have is running under Cirrus CI which uses VMs on real mac aarch64 hardware, but I don't think we can test HVF there.
Thanks for confirming.
Quoted 5 lines
Mostly we rely on regular contributors periodically running tests and reporting on problems. This is not ideal, but also not a blocker for enabling HVF - it just means macOS isn't a full tier 1 platform for us.
OK, that makes sense, thanks. At least this gives me a better idea of what will be covered by CI and what won't when I'm reviewing patches.
Phil
Quoted 6 lines · Philippe Mathieu-Daudé
As Daniel suggested [*]:
Quoted 3 lines
We should consider to rank HVF above TCG, on the basis that HW acceleration is faster and should provide a host<->guest security boundary that we don't claim for TCG
Quoted 6 lines
[*] https://lore.kernel.org/qemu-devel/[email protected]/
Suggested-by Daniel P. Berrangé <[email protected]>Signed-off-by Philippe Mathieu-Daudé <[email protected]>--- system/vl.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
Do nvmm and/or whpx deserve the same consideration?
r~