Skip to contentSkip to conversation

Accelerated read from WC mem (i915_memcpy_from_wc()) may not work in virtualization world

DISCUSSION Du, Changbin 2 participants 5 messages Dec 21, 2017 – Dec 22, 2017

lore
Share & tools
lore
#0 lore raw

Hi Chris,
Our QA reported a problem caused by movntdqa instructions. Currently, the KVM hypervisor doesn't support VEX-prefix instructions emulation. If users passthrough a GPU to guest with vfio option 'x-no-mmap=on', then all access to the BARs will be trapped and emulated. The KVM hypervisor would raise an inertal error to qemu which cause the guest killed. (Since 'movntdqa' ins is not supported.)

One possible solution is that disable this optimization at i915_memcpy_init_early. This require us to identify the CPUID to check if it is running in guest mode.

What do you think? Thanks.

Signature · 6 lines
Thanks,
Changbin Du
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#1 lore raw

Quoting Du, Changbin (2017-12-21 09:52:16)

Quoted 9 lines

Hi Chris,
Our QA reported a problem caused by movntdqa instructions. Currently, the KVM hypervisor doesn't support VEX-prefix instructions emulation. If users passthrough a GPU to guest with vfio option 'x-no-mmap=on', then all access to the BARs will be trapped and emulated. The KVM hypervisor would raise an inertal error to qemu which cause the guest killed. (Since 'movntdqa' ins is not supported.)

One possible solution is that disable this optimization at i915_memcpy_init_early. This require us to identify the CPUID to check if it is running in guest mode.

Is this mode not detected by intel_vgpu_active() ?

If we need to disable movntdqa, we need to disable it. But not by directly checking CPUID, it should be already decoded into a cpu feature flag -- assuming we need more than intel_vgpu_active and possibly rearranging the init order.
-Chris
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#2 lore raw
Quoted 8 lines · Chris Wilson

Quoting Du, Changbin (2017-12-21 09:52:16)

Quoted 7 lines

Hi Chris,
Our QA reported a problem caused by movntdqa instructions. Currently, the KVM hypervisor doesn't support VEX-prefix instructions emulation. If users passthrough a GPU to guest with vfio option 'x-no-mmap=on', then all access to the BARs will be trapped and emulated. The KVM hypervisor would raise an inertal error to qemu which cause the guest killed. (Since 'movntdqa' ins is not supported.)

Quoted 5 lines
Quoted 2 lines

One possible solution is that disable this optimization at i915_memcpy_init_early. This require us to identify the CPUID to check if it is running in guest mode.

Is this mode not detected by intel_vgpu_active() ?

Here intel_vgpu_active() is not ready to use. i915_memcpy_init_early is executed before MMIO mapped.

Quoted 5 lines

If we need to disable movntdqa, we need to disable it. But not by directly checking CPUID, it should be already decoded into a cpu feature flag -- assuming we need more than intel_vgpu_active and possibly rearranging the init order.
-Chris

Yes, for Intel CPU, Bit 31 of ECX of CPUID leaf 0x1 is Hypervisor Present Bit. So I will write a patch to disable it in hypervisor guest if you don't have objections. Thanks.

Signature · 6 lines
Thanks,
Changbin Du
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#3 lore raw
In reply to Du, Changbin

Quoting Du, Changbin (2017-12-21 09:52:16)

Quoted 9 lines

Hi Chris,
Our QA reported a problem caused by movntdqa instructions. Currently, the KVM hypervisor doesn't support VEX-prefix instructions emulation. If users passthrough a GPU to guest with vfio option 'x-no-mmap=on', then all access to the BARs will be trapped and emulated. The KVM hypervisor would raise an inertal error to qemu which cause the guest killed. (Since 'movntdqa' ins is not supported.)

One possible solution is that disable this optimization at i915_memcpy_init_early. This require us to identify the CPUID to check if it is running in guest mode.

But do note we are already checking the CPUID/cpuflags for support; is the hv not correcting them?
-Chris
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#4 lore raw
Quoted 8 lines · Chris Wilson

Quoting Du, Changbin (2017-12-21 09:52:16)

Quoted 7 lines

Hi Chris,
Our QA reported a problem caused by movntdqa instructions. Currently, the KVM hypervisor doesn't support VEX-prefix instructions emulation. If users passthrough a GPU to guest with vfio option 'x-no-mmap=on', then all access to the BARs will be trapped and emulated. The KVM hypervisor would raise an inertal error to qemu which cause the guest killed. (Since 'movntdqa' ins is not supported.)

Quoted 6 lines
Quoted 2 lines

One possible solution is that disable this optimization at i915_memcpy_init_early. This require us to identify the CPUID to check if it is running in guest mode.

But do note we are already checking the CPUID/cpuflags for support; is the hv not correcting them?
-Chris

The CPU does support SSE, but just the KVM ins emulator doesn't. So it is not a problem if the movntdqa runs on HW directly. But if it is emulated, then run into failure.

Signature · 6 lines
Thanks,
Changbin Du
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx