Revert "x86/hvm: simplify 'mmio_direct' check in epte_get_entry_emt()"
PATCH 1/1 Revert "x86/hvm: simplify 'mmio_direct' check in epte_get_entry_emt()"
This reverts commit 81fd0d3ca4b2cd309403c6e8da662c325dd35750.
Original commit only takes into account the APIC access page being a 'special' page, but when running a PVH dom0 there are other pages that also fulfill the 'special' page check but shouldn't have it's cache type set to WB.
For example the ACPI regions are identity mapped into the guest but are also Xen heap pages, and forcing those to WB cache type is wrong.
I've discovered this while trying to boot a PVH dom0, which fail to boot with this commit applied.
Revert the commit while this is sorted out: either we settle that the current code is correct, or we modify the way ACPI regions are mapped into a PVH dom0.
---
Cc: Paul Durrant <[email protected]>
--- xen/arch/x86/hvm/mtrr.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-)
xen/arch/x86/hvm/mtrr.c +11 -4
@@ -815,13 +815,23 @@ int epte_get_entry_emt(struct domain *d, unsigned long gfn, mfn_t mfn,return -1 ; } +if ( direct_mmio ) + { +if ( (mfn_x (mfn) ^mfn_x (d->arch.hvm.vmx.apic_access_mfn)) >> order ) +return MTRR_TYPE_UNCACHABLE ; +if ( order ) +return -1 ; + *ipat =1 ; +return MTRR_TYPE_WRBACK ; + } +if ( !mfn_valid (mfn) ) { *ipat =1 ;return MTRR_TYPE_UNCACHABLE ; } -if ( !direct_mmio && !is_iommu_enabled (d) && !cache_flush_permitted (d) ) +if ( !is_iommu_enabled (d) && !cache_flush_permitted (d) ) { *ipat =1 ;return MTRR_TYPE_WRBACK ;
@@ -838,9 +848,6 @@ int epte_get_entry_emt(struct domain *d, unsigned long gfn, mfn_t mfn, } } -if ( direct_mmio ) -return MTRR_TYPE_UNCACHABLE ; - gmtrr_mtype =hvm_get_mem_pinned_cacheattr (d,_gfn (gfn), order);if ( gmtrr_mtype >=0 ) {
Signature · 1 line
2.28.0
Quoted 10 lines
-----Original Message-----
From: Roger Pau Monne <[email protected]>
Sent: 07 September 2020 18:09
To: [email protected]
Cc: Roger Pau Monne <[email protected]>; Jan Beulich <[email protected]>; Andrew Cooper <[email protected]>; Wei Liu <[email protected]>; Paul Durrant <[email protected]>
Subject: [PATCH] Revert "x86/hvm: simplify 'mmio_direct' check inepte_get_entry_emt()"This reverts commit
81fd0d3ca4b2cd309403c6e8da662c325dd35750.
Quoted 7 lines
Original commit only takes into account the APIC access page being a 'special' page, but when running a PVH dom0 there are other pages that also fulfill the 'special' page check but shouldn't have it's cache type set to WB.
For example the ACPI regions are identity mapped into the guest but are also Xen heap pages, and forcing those to WB cache type is wrong.
I don't understand why reversion of this patch alone is sufficient then...
Quoted 34 lines
I've discovered this while trying to boot a PVH dom0, which fail to boot with this commit applied.
Revert the commit while this is sorted out: either we settle that the current code is correct, or we modify the way ACPI regions are mapped into a PVH dom0.
Signed-off-by Roger Pau Monné <[email protected]>---
Cc: Paul Durrant <[email protected]>--- xen/arch/x86/hvm/mtrr.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-)xen/arch/x86/hvm/mtrr.c +10
-0@@ -815,13 +815,23 @@ int epte_get_entry_emt(struct domain *d, unsigned long gfn, mfn_t mfn,return -1 ; } +if ( direct_mmio ) + { +if ( (mfn_x (mfn) ^mfn_x (d->arch.hvm.vmx.apic_access_mfn)) >> order ) +return MTRR_TYPE_UNCACHABLE ; +if ( order ) +return -1 ; + *ipat =1 ; +return MTRR_TYPE_WRBACK ; + } +if ( !mfn_valid (mfn) )
xen/arch/x86/hvm/mtrr.c int epte_get_entry_emt(struct domain *d, unsigned long gfn, mfn_t mfn, · line 837
Quoted 10 lines
xen/arch/x86/hvm/mtrr.c +1
-1{ *ipat =1 ;return MTRR_TYPE_UNCACHABLE ; } -if ( !direct_mmio && !is_iommu_enabled (d) && !cache_flush_permitted (d) ) +if ( !is_iommu_enabled (d) && !cache_flush_permitted (d) ) { *ipat =1 ;return MTRR_TYPE_WRBACK ;
... since just below this hunk, commit ca24b2ffdbd9 "x86/hvm: set 'ipat' in EPT for special pages" introduced the check:
+ for ( i = 0; i < (1ul << order); i++ )
+ {
+ if ( is_special_page(mfn_to_page(mfn_add(mfn, i))) )
+ {
+ if ( order )
+ return -1;
+ *ipat = 1;
+ return MTRR_TYPE_WRBACK;
+ }
+ }
+
So, would that not be catching the ACPI regions?
Also, why is it ok for ACPI regions to be WB if the IOMMU is not enabled? I know that this will never be the case for PVH dom0 but why do domUs also have to suffer? I think we need a more targeted patch.
Paul
Quoted 12 lines
xen/arch/x86/hvm/mtrr.c +0
-4@@ -838,9 +848,6 @@ int epte_get_entry_emt(struct domain *d, unsigned long gfn, mfn_t mfn, } } -if ( direct_mmio ) -return MTRR_TYPE_UNCACHABLE ; - gmtrr_mtype =hvm_get_mem_pinned_cacheattr (d,_gfn (gfn), order);if ( gmtrr_mtype >=0 ) { --2.28.0
Quoted 59 lines · Paul Durrant
Quoted 10 lines
-----Original Message-----
From: Roger Pau Monne <[email protected]>
Sent: 07 September 2020 18:09
To: [email protected]
Cc: Roger Pau Monne <[email protected]>; Jan Beulich <[email protected]>; Andrew Cooper <[email protected]>; Wei Liu <[email protected]>; Paul Durrant <[email protected]>
Subject: [PATCH] Revert "x86/hvm: simplify 'mmio_direct' check inepte_get_entry_emt()"This reverts commit
81fd0d3ca4b2cd309403c6e8da662c325dd35750.Quoted 7 lines
Original commit only takes into account the APIC access page being a 'special' page, but when running a PVH dom0 there are other pages that also fulfill the 'special' page check but shouldn't have it's cache type set to WB.
For example the ACPI regions are identity mapped into the guest but are also Xen heap pages, and forcing those to WB cache type is wrong.
I don't understand why reversion of this patch alone is sufficient then...
Quoted 39 lines
I've discovered this while trying to boot a PVH dom0, which fail to boot with this commit applied.
Revert the commit while this is sorted out: either we settle that the current code is correct, or we modify the way ACPI regions are mapped into a PVH dom0.
Signed-off-by Roger Pau Monné <[email protected]>---
Cc: Paul Durrant <[email protected]>--- xen/arch/x86/hvm/mtrr.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-)xen/arch/x86/hvm/mtrr.c +10
-0@@ -815,13 +815,23 @@ int epte_get_entry_emt(struct domain *d, unsigned long gfn, mfn_t mfn,return -1 ; } +if ( direct_mmio ) + { +if ( (mfn_x (mfn) ^mfn_x (d->arch.hvm.vmx.apic_access_mfn)) >> order ) +return MTRR_TYPE_UNCACHABLE ; +if ( order ) +return -1 ; + *ipat =1 ; +return MTRR_TYPE_WRBACK ; + } +if ( !mfn_valid (mfn) ) { *ipat =1 ;return MTRR_TYPE_UNCACHABLE ; }
Quoted 7 lines
xen/arch/x86/hvm/mtrr.c int epte_get_entry_emt(struct domain *d, unsigned long gfn, mfn_t mfn, · line 837
Quoted 5 lines
xen/arch/x86/hvm/mtrr.c +1
-1-if ( !direct_mmio && !is_iommu_enabled (d) && !cache_flush_permitted (d) ) +if ( !is_iommu_enabled (d) && !cache_flush_permitted (d) ) { *ipat =1 ;return MTRR_TYPE_WRBACK ;... since just below this hunk, commit
ca24b2ffdbd9"x86/hvm: set 'ipat' in EPT for special pages" introduced the check:
ACPI identity mapped regions in a PVH dom0 don't even get there since they are handled by the 'if ( direct_mmio )' chunk above.
Quoted 12 lines
xen/arch/x86/hvm/mtrr.c +11
-0+for ( i =0 ; i < (1ul << order); i++ ) + { +if (is_special_page (mfn_to_page (mfn_add (mfn, i))) ) + { +if ( order ) +return -1 ; + *ipat =1 ; +return MTRR_TYPE_WRBACK ; + } + } +
Quoted 1 line
So, would that not be catching the ACPI regions?
No, because on a PVH dom0 ACPI regions are MMIO identity mapped, and thus direct_mmio is true for them, and hence they get handled by the first chunk that the original patch removed.
Quoted 2 lines
Also, why is it ok for ACPI regions to be WB if the IOMMU is not enabled? I know that this will never be the case for PVH dom0 but why do domUs also have to suffer? I think we need a more targeted patch.
domUs don't have ACPI tables mapped as MMIO, as the ACPI tables in that case are created by the toolstack and reside in a RAM region.
I'm not completely convinced that using UC unconditionally for ACPI identity mapped regions is fully correct, I will think of a better way to handled them but in the meantime we need to keep this working.
Thanks, Roger.
Quoted 9 lines · Roger Pau Monne
This reverts commit
81fd0d3ca4b2cd309403c6e8da662c325dd35750.Original commit only takes into account the APIC access page being a 'special' page, but when running a PVH dom0 there are other pages that also fulfill the 'special' page check but shouldn't have it's cache type set to WB.
For example the ACPI regions are identity mapped into the guest but are also Xen heap pages, and forcing those to WB cache type is wrong.
Why is this wrong? Xen heap pages are RAM ones. Quite the opposite, _not_ forcing them to WB is going to cause cachability conflicts, unless their direct mapping entries also got changed to UC (which I think we want to avoid if at all possible). The cachability we select here is the one to be used by the hardware, not the one perceived by a domain (be it Dom0 or a DomU).
Quoted 2 lines
I've discovered this while trying to boot a PVH dom0, which fail to boot with this commit applied.
And what exactly is the deeper reason of the failure?
Jan