xen: reserve RMRR to avoid conflicting MMIO/RAM
v6 cover xen: reserve RMRR to avoid conflicting MMIO/RAM
v6:
* Jan cleanup to regenerate one patch to replace the two original patches then rebase the others.
* Refine all patches short/long logs.
* Replace those info with 'reserved device memory maps' and s/RMRR/RDM.
* Fix some code styles.
* Fix one interation error when we grab e820 info
This is not valid when j == nr - 1 (last iteration).
* Cleanup some codes.
* One comment to introduce a new field, nr_reserved_device_memory_map libxc/hvm_info_table, is still pending to wait the tools maintainer's further comment.
v5:
* Add patch #2 to introduce a global count, acpi_rmrr_unit_entries.
* Refine hypercall return value to make sure the caller can distinguish clearly between "xen filled in N entries" and "xen said you need N entries for all information".
* Refine some structures
* Then Rebase
v4:
* Drop the original patch #1. Instead, we use acpi_rmrr_units to get rmrr info directly.
* Refine the hypercall definition to make sure we can use it safely.
* Introduce introduce nr_reserved_device_memory_map in hvm_info_table, then we can avoid issue unnecessary hypercall, even we can know current RMRR entries to issue hypercall one time.
* Cleanup and rebase
v3:
* Use XENMEM_reserved_device_memory_map to replace XENMEM_RMRR_memory_map
* Then rebase all patches
v2:
* Don't use e820map to define RMRR maps directly to avoid any confusion.
* In patch #3 we introduce construct_rmrr_e820_maps() to check if we can insert RMRR maps and then we will sort all e820 entries.
* Clean patch #4
* In patch #5 we reuse check_mmio_hole() to check if current mmio range is fine to RMRR maps. If not, we just issue error to notify the user since mostly mmio should be configured again.
While we work for supporting RMRR mapping for Windows GFX driver in case shared table,
http://osdir.com/ml/general/2014-07/msg55347.html
http://osdir.com/ml/general/2014-07/msg55348.html
we realize we should reserve RMRR range to avoid any potential MMIO/RAM overlap with our discussion so here these preliminary patches are intended to cover this.
----------------------------------------------------------------
Jan Beulich (1):
introduce XENMEM_reserved_device_memory_map
Tiejun Chen (6):
tools/libxc: introduce hypercall for xc_reserved_device_memory_map tools/libxc: check if mmio BAR is out of reserved device memory maps libxc/hvm_info_table: introduce a new field nr_reserved_device_memory_map
hvmloader: introduce hypercall for xc_reserved_device_memory_map
hvmloader: check to reserved device memory maps in e820 xen/vtd: make USB RMRR mapping safe
tools/firmware/hvmloader/e820.c | 100 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ tools/firmware/hvmloader/util.c | 22 ++++++++++++++++++++++ tools/firmware/hvmloader/util.h | 3 +++ tools/libxc/xc_domain.c | 29 +++++++++++++++++++++++++++++ tools/libxc/xc_hvm_build_x86.c | 82 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-- tools/libxc/xenctrl.h | 4 ++++ xen/common/compat/memory.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++++ xen/common/memory.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++++ xen/drivers/passthrough/iommu.c | 10 ++++++++++ xen/drivers/passthrough/vtd/dmar.c | 17 +++++++++++++++++ xen/drivers/passthrough/vtd/extern.h | 1 + xen/drivers/passthrough/vtd/iommu.c | 9 +-------- xen/include/public/hvm/hvm_info_table.h | 3 +++ xen/include/public/memory.h | 24 +++++++++++++++++++++++- xen/include/xen/iommu.h | 4 ++++ xen/include/xlat.lst | 3 ++- 16 files changed, 400 insertions(+), 12 deletions(-)
Thanks
Tiejun
PATCH v6 1/7 introduce XENMEM_reserved_device_memory_map
From: Jan Beulich <[email protected]>
This is a prerequisite for punching holes into HVM and PVH guests' P2M to allow passing through devices that are associated with (on VT-d) RMRRs.
xen/common/compat/memory.c +52 -0
@@ -15,6 +15,35 @@ CHECK_TYPE(domid); CHECK_mem_access_op; +struct get_reserved_device_memory { +struct compat_mem_reserved_device_memory_map map; +unsigned int used_entries; +}; + +static int get_reserved_device_memory (xen_pfn_t start, +xen_ulong_t nr,void *ctxt) +{ +struct get_reserved_device_memory *grdm = ctxt; + +if ( grdm->used_entries < grdm->map.nr_entries ) + { +struct compat_mem_reserved_device_memory rdm = { + .start_pfn = start, .nr_pages = nr + }; + +if ( rdm.start_pfn != start || rdm.nr_pages != nr ) +return -ERANGE ; + +if (__copy_to_compat_offset (grdm->map.buffer, grdm->used_entries, + &rdm,1 ) ) +return -EFAULT ; + } + + ++grdm->used_entries; + +return 0 ; +} +int compat_memory_op (unsigned int cmd,XEN_GUEST_HANDLE_PARAM (void ) compat) {int split, op = cmd &MEMOP_CMD_MASK ;
@@ -272,6 +301,29 @@ int compat_memory_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) compat)break ; } +#ifdef HAS_PASSTHROUGH +case XENMEM_reserved_device_memory_map: + { +struct get_reserved_device_memory grdm; + +if (copy_from_guest (&grdm.map, compat,1 ) || + !compat_handle_okay (grdm.map.buffer, grdm.map.nr_entries) ) +return -EFAULT ; + + grdm.used_entries =0 ; + rc =iommu_get_reserved_device_memory (get_reserved_device_memory, + &grdm); + +if ( !rc && grdm.map.nr_entries < grdm.used_entries ) + rc = -ENOBUFS ; + grdm.map.nr_entries = grdm.used_entries; +if (__copy_to_guest (compat, &grdm.map,1 ) ) + rc = -EFAULT ; + +return rc; + } +#endif +default :return compat_arch_memory_op (cmd, compat); }
xen/common/memory.c +49 -0
61 lines not on this page · raw message
xen/drivers/passthrough/iommu.c +10 -0
@@ -344,6 +344,16 @@ void iommu_crash_shutdown(void) iommu_enabled = iommu_intremap =0 ; } +int iommu_get_reserved_device_memory (iommu_grdm_t *func,void *ctxt) +{ +const struct iommu_ops *ops =iommu_get_ops (); + +if ( !iommu_enabled || !ops->get_reserved_device_memory ) +return 0 ; + +return ops->get_reserved_device_memory (func, ctxt); +} +bool_t iommu_has_feature (struct domain *d,enum iommu_feature feature) {const struct hvm_iommu *hd =domain_hvm_iommu (d);
xen/drivers/passthrough/vtd/dmar.c +17 -0
@@ -893,3 +893,20 @@ int platform_supports_x2apic(void)unsigned int mask =ACPI_DMAR_INTR_REMAP |ACPI_DMAR_X2APIC_OPT_OUT ;return cpu_has_x2apic && ((dmar_flags & mask) ==ACPI_DMAR_INTR_REMAP ); } + +int intel_iommu_get_reserved_device_memory (iommu_grdm_t *func,void *ctxt) +{ +struct acpi_rmrr_unit *rmrr; +int rc =0 ; + +list_for_each_entry (rmrr, &acpi_rmrr_units, list) + { + rc =func (PFN_DOWN (rmrr->base_address), +PFN_UP (rmrr->end_address) -PFN_DOWN (rmrr->base_address), + ctxt); +if ( rc ) +break ; + } + +return rc; +}
xen/drivers/passthrough/vtd/extern.h +1 -0
@@ -75,6 +75,7 @@ int domain_context_mapping_one(struct domain *domain, struct iommu *iommu, u8 bus, u8 devfn, const struct pci_dev *);int domain_context_unmap_one (struct domain *domain,struct iommu *iommu,u8 bus,u8 devfn); +int intel_iommu_get_reserved_device_memory (iommu_grdm_t *func,void *ctxt);unsigned int io_apic_read_remap_rte (unsigned int apic,unsigned int reg); void io_apic_write_remap_rte(unsigned int apic,
xen/drivers/passthrough/vtd/iommu.c +1 -0
@@ -2485,6 +2485,7 @@ const struct iommu_ops intel_iommu_ops = { .crash_shutdown = vtd_crash_shutdown, .iotlb_flush = intel_iommu_iotlb_flush, .iotlb_flush_all = intel_iommu_iotlb_flush_all, + .get_reserved_device_memory = intel_iommu_get_reserved_device_memory, .dump_p2m_table = vtd_dump_p2m_table, };
xen/include/public/memory.h +23 -1
@@ -523,7 +523,29 @@ DEFINE_XEN_GUEST_HANDLE(xen_mem_sharing_op_t);#endif /* defined(__XEN__) || defined(__XEN_TOOLS__) */ -/* Next available subop number is 26 */ +/* +* For legacy reasons, some devices must be configured with special memory +* regions to function correctly. The guest must avoid using any of these +* regions. +*/ +#define XENMEM_reserved_device_memory_map 26 +struct xen_mem_reserved_device_memory { +xen_pfn_t start_pfn; +xen_ulong_t nr_pages; +}; +typedef struct xen_mem_reserved_device_memory xen_mem_reserved_device_memory_t ; +DEFINE_XEN_GUEST_HANDLE (xen_mem_reserved_device_memory_t); + +struct xen_mem_reserved_device_memory_map { +/* IN/OUT */ +unsigned int nr_entries; +/* OUT */ +XEN_GUEST_HANDLE (xen_mem_reserved_device_memory_t) buffer; +}; +typedef struct xen_mem_reserved_device_memory_map xen_mem_reserved_device_memory_map_t ; +DEFINE_XEN_GUEST_HANDLE (xen_mem_reserved_device_memory_map_t); + +/* Next available subop number is 27 */ #endif /* __XEN_PUBLIC_MEMORY_H__ */
xen/include/xen/iommu.h +4 -0
@@ -120,6 +120,8 @@ void iommu_dt_domain_destroy(struct domain *d);struct page_info ; +typedef int iommu_grdm_t (xen_pfn_t start,xen_ulong_t nr,void *ctxt); +struct iommu_ops {int (*init)(struct domain *d);void (*hwdom_init)(struct domain *d);
@@ -156,12 +158,14 @@ struct iommu_ops {void (*crash_shutdown)(void );void (*iotlb_flush)(struct domain *d,unsigned long gfn,unsigned int page_count);void (*iotlb_flush_all)(struct domain *d); +int (*get_reserved_device_memory)(iommu_grdm_t *,void *);void (*dump_p2m_table)(struct domain *d); };void iommu_suspend (void );void iommu_resume (void );void iommu_crash_shutdown (void ); +int iommu_get_reserved_device_memory (iommu_grdm_t *,void *);void iommu_share_p2m_table (struct domain *d);
xen/include/xlat.lst +2 -1
@@ -60,7 +60,8 @@ ! memory_exchange memory.h ! memory_map memory.h ! memory_reservation memory.h -? mem_access_op memory.h +? mem_access_op memory.h +! mem_reserved_device_memory_map memory.h ! pod_target memory.h ! remove_from_physmap memory.h ? physdev_eoi physdev.h
Signature · 1 line
1.9.1
Re: [v6][PATCH 1/7] introduce XENMEM_reserved_device_memory_map
Quoted 10 lines
From: Chen, Tiejun
Sent: Tuesday, September 09, 2014 10:50 PMFrom: Jan Beulich <[email protected]>
This is a prerequisite for punching holes into HVM and PVH guests' P2M to allow passing through devices that are associated with (on VT-d) RMRRs.
Signed-off-by Jan Beulich <[email protected]>
Did you change anything on top of Jan's original version? If yes, also need your sign-off.
Quoted 314 lines
xen/common/compat/memory.c +29
-0@@ -15,6 +15,35 @@ CHECK_TYPE(domid); CHECK_mem_access_op; +struct get_reserved_device_memory { +struct compat_mem_reserved_device_memory_map map; +unsigned int used_entries; +}; + +static int get_reserved_device_memory (xen_pfn_t start, +xen_ulong_t nr,void *ctxt) +{ +struct get_reserved_device_memory *grdm = ctxt; + +if ( grdm->used_entries < grdm->map.nr_entries ) + { +struct compat_mem_reserved_device_memory rdm = { + .start_pfn = start, .nr_pages = nr + }; + +if ( rdm.start_pfn != start || rdm.nr_pages != nr ) +return -ERANGE ; + +if (__copy_to_compat_offset (grdm->map.buffer, grdm->used_entries, + &rdm,1 ) ) +return -EFAULT ; + } + + ++grdm->used_entries; + +return 0 ; +} +int compat_memory_op (unsigned int cmd,XEN_GUEST_HANDLE_PARAM (void ) compat){ int split, op = cmd & MEMOP_CMD_MASK;xen/common/compat/memory.c +23
-0@@ -272,6 +301,29 @@ int compat_memory_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) compat) break; } +#ifdef HAS_PASSTHROUGH +case XENMEM_reserved_device_memory_map: + { +struct get_reserved_device_memory grdm; + +if (copy_from_guest (&grdm.map, compat,1 ) || + !compat_handle_okay (grdm.map.buffer, grdm.map.nr_entries) ) +return -EFAULT ; + + grdm.used_entries =0 ; + rc =iommu_get_reserved_device_memory (get_reserved_device_memory, + &grdm); + +if ( !rc && grdm.map.nr_entries < grdm.used_entries ) + rc = -ENOBUFS ; + grdm.map.nr_entries = grdm.used_entries; +if (__copy_to_guest (compat, &grdm.map,1 ) ) + rc = -EFAULT ; + +return rc; + } +#endif +default: return compat_arch_memory_op(cmd, compat); }xen/common/memory.c +26
-0@@ -695,6 +695,32 @@ out:return rc; } +struct get_reserved_device_memory { +struct xen_mem_reserved_device_memory_map map; +unsigned int used_entries; +}; + +static int get_reserved_device_memory (xen_pfn_t start, +xen_ulong_t nr,void *ctxt) +{ +struct get_reserved_device_memory *grdm = ctxt; + +if ( grdm->used_entries < grdm->map.nr_entries ) + { +struct xen_mem_reserved_device_memory rdm = { + .start_pfn = start, .nr_pages = nr + }; + +if (__copy_to_guest_offset (grdm->map.buffer, grdm->used_entries, + &rdm,1 ) ) +return -EFAULT ; + } + + ++grdm->used_entries; + +return 0 ; +} +long do_memory_op (unsigned long cmd,XEN_GUEST_HANDLE_PARAM (void ) arg){ struct domain *d;xen/common/memory.c +23
-0@@ -969,6 +995,29 @@ long do_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg) break; +#ifdef HAS_PASSTHROUGH +case XENMEM_reserved_device_memory_map: + { +struct get_reserved_device_memory grdm; + +if (copy_from_guest (&grdm.map, arg,1 ) || + !guest_handle_okay (grdm.map.buffer, grdm.map.nr_entries) ) +return -EFAULT ; + + grdm.used_entries =0 ; + rc =iommu_get_reserved_device_memory (get_reserved_device_memory, + &grdm); + +if ( !rc && grdm.map.nr_entries < grdm.used_entries ) + rc = -ENOBUFS ; + grdm.map.nr_entries = grdm.used_entries; +if (__copy_to_guest (arg, &grdm.map,1 ) ) + rc = -EFAULT ; + +break ; + } +#endif +default: rc = arch_memory_op(cmd, arg); break;xen/drivers/passthrough/iommu.c +10
-0@@ -344,6 +344,16 @@ void iommu_crash_shutdown(void) iommu_enabled = iommu_intremap =0 ; } +int iommu_get_reserved_device_memory (iommu_grdm_t *func,void *ctxt) +{ +const struct iommu_ops *ops =iommu_get_ops (); + +if ( !iommu_enabled || !ops->get_reserved_device_memory ) +return 0 ; + +return ops->get_reserved_device_memory (func, ctxt); +} +bool_t iommu_has_feature (struct domain *d,enum iommu_feature feature) {const struct hvm_iommu *hd =domain_hvm_iommu (d);xen/drivers/passthrough/vtd/dmar.c +0
-0@@ -893,3 +893,20 @@ int platform_supports_x2apic(void)unsigned int mask =ACPI_DMAR_INTR_REMAP |ACPI_DMAR_X2APIC_OPT_OUT ; return cpu_has_x2apic && ((dmar_flags & mask) ==
ACPI_DMAR_INTR_REMAP);
}xen/drivers/passthrough/vtd/dmar.c +2
-0+ +int intel_iommu_get_reserved_device_memory(iommu_grdm_t *func, void*ctxt)
xen/drivers/passthrough/vtd/dmar.c +8
-0+{ +struct acpi_rmrr_unit *rmrr; +int rc =0 ; + +list_for_each_entry (rmrr, &acpi_rmrr_units, list) + { + rc = func(PFN_DOWN(rmrr->base_address), + PFN_UP(rmrr->end_address) -
PFN_DOWN(rmrr->base_address),xen/drivers/passthrough/vtd/dmar.c +7
-0+ ctxt); +if ( rc ) +break ; + } + +return rc; +}xen/drivers/passthrough/vtd/extern.h +1
-0@@ -75,6 +75,7 @@ int domain_context_mapping_one(struct domain *domain, struct iommu *iommu, u8 bus, u8 devfn, const struct pci_dev *);int domain_context_unmap_one (struct domain *domain,struct iommu *iommu,u8 bus,u8 devfn); +int intel_iommu_get_reserved_device_memory(iommu_grdm_t *func, void*ctxt);
unsigned int
io_apic_read_remap_rte(unsigned int apic, unsigned int reg); void io_apic_write_remap_rte(unsigned int apic,xen/drivers/passthrough/vtd/iommu.c +1
-0@@ -2485,6 +2485,7 @@ const struct iommu_ops intel_iommu_ops = { .crash_shutdown = vtd_crash_shutdown, .iotlb_flush = intel_iommu_iotlb_flush, .iotlb_flush_all = intel_iommu_iotlb_flush_all, + .get_reserved_device_memory = intel_iommu_get_reserved_device_memory, .dump_p2m_table = vtd_dump_p2m_table, };xen/include/public/memory.h +23
-1@@ -523,7 +523,29 @@DEFINE_XEN_GUEST_HANDLE (xen_mem_sharing_op_t);#endif /* defined(__XEN__) || defined(__XEN_TOOLS__) */ -/* Next available subop number is 26 */ +/* +* For legacy reasons, some devices must be configured with special memory +* regions to function correctly. The guest must avoid using any of these +* regions. +*/ +#define XENMEM_reserved_device_memory_map 26 +struct xen_mem_reserved_device_memory { +xen_pfn_t start_pfn; +xen_ulong_t nr_pages; +}; +typedef struct xen_mem_reserved_device_memory xen_mem_reserved_device_memory_t ; +DEFINE_XEN_GUEST_HANDLE (xen_mem_reserved_device_memory_t); + +struct xen_mem_reserved_device_memory_map { +/* IN/OUT */ +unsigned int nr_entries; +/* OUT */ +XEN_GUEST_HANDLE (xen_mem_reserved_device_memory_t) buffer; +}; +typedef struct xen_mem_reserved_device_memory_map xen_mem_reserved_device_memory_map_t ; +DEFINE_XEN_GUEST_HANDLE (xen_mem_reserved_device_memory_map_t); + +/* Next available subop number is 27 */ #endif /*
__XEN_PUBLIC_MEMORY_H__*/xen/include/xen/iommu.h +4
-0@@ -120,6 +120,8 @@ void iommu_dt_domain_destroy(struct domain *d);struct page_info ; +typedef int iommu_grdm_t (xen_pfn_t start,xen_ulong_t nr,void *ctxt); +struct iommu_ops {int (*init)(struct domain *d);void (*hwdom_init)(struct domain *d);@@ -156,12 +158,14 @@ struct iommu_ops {void (*crash_shutdown)(void );void (*iotlb_flush)(struct domain *d,unsigned long gfn,unsigned int page_count);void (*iotlb_flush_all)(struct domain *d); +int (*get_reserved_device_memory)(iommu_grdm_t *,void *);void (*dump_p2m_table)(struct domain *d); };void iommu_suspend (void );void iommu_resume (void );void iommu_crash_shutdown (void ); +int iommu_get_reserved_device_memory (iommu_grdm_t *,void *);void iommu_share_p2m_table (struct domain *d);xen/include/xlat.lst +2
-2@@ -60,7 +60,8 @@ ! memory_exchange memory.h ! memory_map memory.h ! memory_reservation memory.h -? mem_access_op memory.h +? mem_access_op memory.h +! mem_reserved_device_memory_map memory.h ! pod_target memory.h ! remove_from_physmap memory.h ? physdev_eoi physdev.h --1.9.1
PATCH v6 2/7 tools/libxc: introduce hypercall for xc_reserved_device_memory_map
In reply to Tiejun ChenWe will introduce that hypercall xc_reserved_device_memory_map to libxc.
tools/libxc/xc_domain.c +29 -0
@@ -649,6 +649,35 @@ int xc_domain_set_memory_map(xc_interface *xch,return rc; } + +int xc_reserved_device_memory_map (xc_interface *xch, +struct xen_mem_reserved_device_memory entries[], +uint32_t *max_entries) +{ +int rc; +struct xen_mem_reserved_device_memory_map memmap = { + .nr_entries = *max_entries + }; +DECLARE_HYPERCALL_BOUNCE (entries, +sizeof (struct xen_mem_reserved_device_memory ) * + *max_entries,XC_HYPERCALL_BUFFER_BOUNCE_OUT ); + +if (xc_hypercall_bounce_pre (xch, entries) ) +return -1 ; + +set_xen_guest_handle (memmap.buffer, entries); + + rc =do_memory_op (xch, XENMEM_reserved_device_memory_map, + &memmap,sizeof (memmap)); + +xc_hypercall_bounce_post (xch, entries); + +if ( errno ==ENOBUFS ) + *max_entries = memmap.nr_entries; + +return rc ? -errno : memmap.nr_entries; +} +int xc_get_machine_memory_map (xc_interface *xch,struct e820entry entries[],uint32_t max_entries)
tools/libxc/xenctrl.h +4 -0
@@ -1270,6 +1270,10 @@ int xc_domain_set_memory_map(xc_interface *xch,int xc_get_machine_memory_map (xc_interface *xch,struct e820entry entries[],uint32_t max_entries); + +int xc_reserved_device_memory_map (xc_interface *xch, +struct xen_mem_reserved_device_memory entries[], +uint32_t *max_entries);#endif int xc_domain_set_time_offset(xc_interface *xch, uint32_t domid,
Signature · 1 line
1.9.1
Re: [v6][PATCH 2/7] tools/libxc: introduce hypercall for xc_reserved_device_memory_map
tools/libxc/xc_domain.c int xc_domain_set_memory_map(xc_interface *xch, · line 679
Quoted 6 lines
Quoted 1 line
Quoted 1 line · On 10.09.14 at 07:49, <[email protected]> wrote:
tools/libxc/xc_domain.c +5
-0+if ( errno ==ENOBUFS ) + *max_entries = memmap.nr_entries; + +return rc ? -errno : memmap.nr_entries; +}
Isn't libxc generally aiming at using POSIX style return values, i.e. -1 one error with errno indicating the details? Furthermore you're even inconsistent with yourself, as you return -1 in an earlier error path.
Jan
Re: [v6][PATCH 2/7] tools/libxc: introduce hypercall for xc_reserved_device_memory_map
Quoted 9 lines · Jan Beulich
tools/libxc/xc_domain.c int xc_domain_set_memory_map(xc_interface *xch, · line 679
Quoted 6 lines
Quoted 1 line
Quoted 1 line · On 10.09.14 at 07:49, <[email protected]> wrote:
tools/libxc/xc_domain.c +5
-0+if ( errno ==ENOBUFS ) + *max_entries = memmap.nr_entries; + +return rc ? -errno : memmap.nr_entries; +}Isn't libxc generally aiming at using POSIX style return values, i.e. -1 one error with errno indicating the details?
We are trying to move that way, yes. It's an unholy mess right now but lets at least try not to add to it!
Quoted 2 lines
Furthermore you're even
inconsistent with yourself, as you return -1 in an earlier error path.
Re: [v6][PATCH 2/7] tools/libxc: introduce hypercall for xc_reserved_device_memory_map
In reply to Jan BeulichQuoted 9 lines · Jan Beulich
tools/libxc/xc_domain.c int xc_domain_set_memory_map(xc_interface *xch, · line 679
Quoted 6 lines
Quoted 1 line
Quoted 1 line · On 10.09.14 at 07:49, <[email protected]> wrote:
tools/libxc/xc_domain.c +5
-0+if ( errno ==ENOBUFS ) + *max_entries = memmap.nr_entries; + +return rc ? -errno : memmap.nr_entries; +}Isn't libxc generally aiming at using POSIX style return values, i.e. -1 one error with errno indicating the details? Furthermore you're even inconsistent with yourself, as you return -1 in an earlier error path.
Furthermore, this in now the 3rd time I have commented on this problem specifically, across different versions of this series.
Please address all outstanding comments before reposting.
~Andrew
Re: [v6][PATCH 2/7] tools/libxc: introduce hypercall for xc_reserved_device_memory_map
In reply to Jan BeulichQuoted 11 lines · Jan Beulich
tools/libxc/xc_domain.c int xc_domain_set_memory_map(xc_interface *xch, · line 679
Quoted 6 lines
Quoted 1 line
Quoted 1 line · On 10.09.14 at 07:49, <[email protected]> wrote:
tools/libxc/xc_domain.c +5
-0+if ( errno ==ENOBUFS ) + *max_entries = memmap.nr_entries; + +return rc ? -errno : memmap.nr_entries; +}Isn't libxc generally aiming at using POSIX style return values, i.e. -1 one error with errno indicating the details? Furthermore you're even inconsistent with yourself, as you return -1 in an earlier error path.
tools/libxc/xc_domain.c:
int xc_get_machine_memory_map(xc_interface *xch,
struct e820entry entries[],
uint32_t max_entries)
{
int rc;
struct xen_memory_map memmap = {
.nr_entries = max_entries
};
DECLARE_HYPERCALL_BOUNCE(entries, sizeof(struct e820entry) *
max_entries,
XC_HYPERCALL_BUFFER_BOUNCE_OUT);
if ( !entries || xc_hypercall_bounce_pre(xch, entries) ||
max_entries <= 1)
return -1;
set_xen_guest_handle(memmap.buffer, entries);
rc = do_memory_op(xch, XENMEM_machine_memory_map, &memmap, sizeof(memmap));
xc_hypercall_bounce_post(xch, entries);
return rc ? rc : memmap.nr_entries;
}Here I just change last line as follows:
return rc ? -errno : memmap.nr_entries;
Because if I don't return '-errno', how will the caller know the detailed error?
rc = xc_reserved_device_memory_map(xch, xmrdm, &nr_entries);
if ( rc < 0 )
{
/* DRM doesn't exist. */
if ( rc == -ENOENT )
rc = 0;Or are you saying we should use errno ditectly,
+
+ rc = xc_reserved_device_memory_map(xch, xmrdm, &nr_entries);
+ if (rc < 0 ) {
+ switch (errno) {
+ case ENOENT: /* RDM doesn't exist. */
Right?
Thanks
Tiejun
Re: [v6][PATCH 2/7] tools/libxc: introduce hypercall for xc_reserved_device_memory_map
Review
Quoted 8 lines
Quoted 1 line
Quoted 1 line · On 12.09.14 at 04:43, <[email protected]> wrote:
Or are you saying we should use errno ditectly,
(quoted diff) +5
-0+ + rc = xc_reserved_device_memory_map(xch, xmrdm, &nr_entries); + if (rc < 0 ) { + switch (errno) { + case ENOENT: /* RDM doesn't exist. */
Yes, of course.
Jan
PATCH v6 3/7 tools/libxc: check if mmio BAR is out of reserved device memory maps
In reply to Tiejun ChenWe need to avoid allocating MMIO BAR conflicting to all reserved device memory range.
tools/libxc/xc_hvm_build_x86.c +71 -0
@@ -239,6 +239,73 @@ static int check_mmio_hole(uint64_t start, uint64_t memsize,return 1 ; } +/* +* Check whether there exists mmio overplap with the reserved device +* memory map +*/ +static int check_rdm_overlap (xc_interface *xch,uint64_t mmio_start, +uint64_t mmio_size) +{ +struct xen_mem_reserved_device_memory *map =NULL ; +uint64_t rdm_start =0 , rdm_end =0 ; +unsigned int i =0 ; +int rc =0 ; +/* Assume we have one entry if not enough we'll expand.*/ +uint32_t nr_entries =1 ; + +/* We should check if mmio range is out of RDM mapping. */ +if ( (map =malloc (nr_entries * +sizeof (xen_mem_reserved_device_memory_t))) ==NULL ) + { +PERROR ("Could not allocate memory for map." ); +return -1 ; + } + rc =xc_reserved_device_memory_map (xch, map, &nr_entries); +if ( rc <0 ) + { +/* DRM doesn't exist. */ +if ( rc == -ENOENT ) + rc =0 ; +else if ( rc == -ENOBUFS ) + { +free (map); +/* Now we need more space to map all RDM mappings. */ +if ( (map =malloc (nr_entries * +sizeof (xen_mem_reserved_device_memory_t))) ==NULL ) + { +PERROR ("Could not allocate memory for map." ); +return -1 ; + } + rc =xc_reserved_device_memory_map (xch, map, &nr_entries); +if ( rc <0 ) + { +PERROR ("Could not get DRM info on domain" ); +free (map); +return rc; + } + } +else +PERROR ("Could not get RDM info on domain" ); + } + +for ( i =0 ; i < rc; i++ ) + { + rdm_start = map[i].start_pfn <<PAGE_SHIFT ; + rdm_end = rdm_start + map[i].nr_pages *PAGE_SIZE ; +if (check_mmio_hole (rdm_start, map[i].nr_pages *PAGE_SIZE , + mmio_start, mmio_size) ) + { +PERROR ("MMIO: [%lx]<->[%lx] overlap DRM [%lx]<->[%lx]\n" , + mmio_start, (mmio_start + mmio_size), rdm_start, rdm_end); +free (map); +return -1 ; + } + } + +free (map); +return rc; +} +static int setup_guest (xc_interface *xch,uint32_t dom,struct xc_hvm_build_args *args,char *image,unsigned long image_size)
@@ -300,6 +367,10 @@ static int setup_guest(xc_interface *xch, goto error_out; } + rc =check_rdm_overlap (xch, mmio_start, mmio_start); +if ( rc <0 ) + goto error_out; +for ( i =0 ; i < nr_pages; i++ ) page_array[i] = i;for ( i = mmio_start >>PAGE_SHIFT ; i < nr_pages; i++ )
Signature · 1 line
1.9.1
Re: [v6][PATCH 3/7] tools/libxc: check if mmio BAR is out of reserved device memory maps
Quoted 5 lines
From: Chen, Tiejun
Sent: Tuesday, September 09, 2014 10:50 PMWe need to avoid allocating MMIO BAR conflicting to all reserved device memory range.
besides checking MMIO BAR confliction, you also need check guest memory confliction with reserved device memory ranges.
Quoted 10 lines
Signed-off-by Tiejun Chen <[email protected]>tools/libxc/xc_hvm_build_x86.c +0
-0@@ -239,6 +239,73 @@ static int check_mmio_hole(uint64_t start, uint64_t memsize, return 1;
tools/libxc/xc_hvm_build_x86.c static int check_mmio_hole(uint64_t start, uint64_t · line 250
Quoted 10 lines
tools/libxc/xc_hvm_build_x86.c +8
-0} +/* +* Check whether there exists mmio overplap with the reserved device +* memory map +*/ +static int check_rdm_overlap (xc_interface *xch,uint64_t mmio_start, +uint64_t mmio_size) +{ +struct xen_mem_reserved_device_memory *map =NULL ;
If there are multiple callers of this call, better to move the structure out of this function to avoid multiple calls.
Quoted 67 lines
tools/libxc/xc_hvm_build_x86.c +30
-0+uint64_t rdm_start =0 , rdm_end =0 ; +unsigned int i =0 ; +int rc =0 ; +/* Assume we have one entry if not enough we'll expand.*/ +uint32_t nr_entries =1 ; + +/* We should check if mmio range is out of RDM mapping. */ +if ( (map =malloc (nr_entries * +sizeof (xen_mem_reserved_device_memory_t))) ==NULL ) + { +PERROR ("Could not allocate memory for map." ); +return -1 ; + } + rc =xc_reserved_device_memory_map (xch, map, &nr_entries); +if ( rc <0 ) + { +/* DRM doesn't exist. */ +if ( rc == -ENOENT ) + rc =0 ; +else if ( rc == -ENOBUFS ) + { +free (map); +/* Now we need more space to map all RDM mappings. */ +if ( (map =malloc (nr_entries * +sizeof (xen_mem_reserved_device_memory_t))) ==NULL ) + { +PERROR ("Could not allocate memory for map." ); +return -1 ; + } + rc = xc_reserved_device_memory_map(xch, map,
&nr_entries);tools/libxc/xc_hvm_build_x86.c +20
-0+if ( rc <0 ) + { +PERROR ("Could not get DRM info on domain" ); +free (map); +return rc; + } + } +else +PERROR ("Could not get RDM info on domain" ); + } + +for ( i =0 ; i < rc; i++ ) + { + rdm_start = map[i].start_pfn <<PAGE_SHIFT ; + rdm_end = rdm_start + map[i].nr_pages *PAGE_SIZE ; +if (check_mmio_hole (rdm_start, map[i].nr_pages *PAGE_SIZE , + mmio_start, mmio_size) ) + { + PERROR("MMIO: [%lx]<->[%lx] overlap DRM [%lx]<->[%lx]\n", + mmio_start, (mmio_start + mmio_size), rdm_start,
rdm_end);tools/libxc/xc_hvm_build_x86.c +9
-0+free (map); +return -1 ; + } + } + +free (map); +return rc; +} +static int setup_guest (xc_interface *xch,uint32_t dom,struct xc_hvm_build_args *args,char *image,unsigned long image_size)
tools/libxc/xc_hvm_build_x86.c static int setup_guest(xc_interface *xch, · line 370
Quoted 4 lines
tools/libxc/xc_hvm_build_x86.c +1
-0@@ -300,6 +367,10 @@ static int setup_guest(xc_interface *xch, goto error_out; } + rc =check_rdm_overlap (xch, mmio_start, mmio_start);
mmio_start -> mmio_size.
Quoted 8 lines
tools/libxc/xc_hvm_build_x86.c +3
-1+if ( rc <0 ) + goto error_out; +for ( i =0 ; i < nr_pages; i++ ) page_array[i] = i;for ( i = mmio_start >>PAGE_SHIFT ; i < nr_pages; i++ ) --1.9.1
Re: [v6][PATCH 3/7] tools/libxc: check if mmio BAR is out of reserved device memory maps
Quoted 8 lines · Kevin
Quoted 5 lines
From: Chen, Tiejun
Sent: Tuesday, September 09, 2014 10:50 PMWe need to avoid allocating MMIO BAR conflicting to all reserved device memory range.
besides checking MMIO BAR confliction, you also need check guest memory confliction with reserved device memory ranges.
Kexin,
In patch #6, hvmloader: check to reserved device memory maps in e820, we'll lookup the entire e820 table to check if RMRR is overlapping with all recorded ranges in e820, so I think that already cover this case, right?
Quoted 13 lines
Quoted 13 lines
Signed-off-by Tiejun Chen <[email protected]>tools/libxc/xc_hvm_build_x86.c +0
-0@@ -239,6 +239,73 @@ static int check_mmio_hole(uint64_t start, uint64_t memsize, return 1; }
Quoted 11 lines
tools/libxc/xc_hvm_build_x86.c static int check_mmio_hole(uint64_t start, uint64_t memsize, · line 249
Quoted 8 lines
tools/libxc/xc_hvm_build_x86.c +8
-0+/* +* Check whether there exists mmio overplap with the reserved device +* memory map +*/ +static int check_rdm_overlap (xc_interface *xch,uint64_t mmio_start, +uint64_t mmio_size) +{ +struct xen_mem_reserved_device_memory *map =NULL ;If there are multiple callers of this call, better to move the structure out of this function to avoid multiple calls.
Good point.
+/* Record reserved device memory. */
+static struct xen_mem_reserved_device_memory *xmrdm = NULL; +
Quoted 6 lines
Quoted 6 lines
tools/libxc/xc_hvm_build_x86.c +5
-0+uint64_t rdm_start =0 , rdm_end =0 ; +unsigned int i =0 ; +int rc =0 ; +/* Assume we have one entry if not enough we'll expand.*/ +uint32_t nr_entries =1 ;
[snip]
Quoted 5 lines
Quoted 3 lines
}+ rc =
check_rdm_overlap(xch, mmio_start, mmio_start);
mmio_start->mmio_size.
Fixed and thanks a lot.
Thanks
Tiejun
Quoted 11 lines
Quoted 9 lines
tools/libxc/xc_hvm_build_x86.c +3
-1+if ( rc <0 ) + goto error_out; +for ( i =0 ; i < nr_pages; i++ ) page_array[i] = i;for ( i = mmio_start >>PAGE_SHIFT ; i < nr_pages; i++ ) --1.9.1
Re: [v6][PATCH 3/7] tools/libxc: check if mmio BAR is out of reserved device memory maps
Quoted 10 lines
From: Chen, Tiejun
Sent: Wednesday, September 10, 2014 6:15 PMQuoted 6 lines · Kevin
Quoted 5 lines
From: Chen, Tiejun
Sent: Tuesday, September 09, 2014 10:50 PMWe need to avoid allocating MMIO BAR conflicting to all reserved device memory range.
Quoted 9 lines
Quoted 2 lines
besides checking MMIO BAR confliction, you also need check guest memory confliction with reserved device memory ranges.
Kexin,
In patch #6, hvmloader: check to reserved device memory maps in e820, we'll lookup the entire e820 table to check if RMRR is overlapping with all recorded ranges in e820, so I think that already cover this case, right?
yes, forgot my comment then. My comment was still based on the assumption that we'll both detect and fix confliction, so 'fix' means we need do something before P2M is allocated. But since the decision now is only for detection, yes patch #6 is enough then.
Thanks
Kevin
Re: [v6][PATCH 3/7] tools/libxc: check if mmio BAR is out of reserved device memory maps
In reply to Tiejun ChenQuoted 15 lines
Quoted 1 line
Quoted 1 line · On 10.09.14 at 07:49, <[email protected]> wrote:
tools/libxc/xc_hvm_build_x86.c +14
-0+static int check_rdm_overlap (xc_interface *xch,uint64_t mmio_start, +uint64_t mmio_size) +{ +struct xen_mem_reserved_device_memory *map =NULL ; +uint64_t rdm_start =0 , rdm_end =0 ; +unsigned int i =0 ; +int rc =0 ; +/* Assume we have one entry if not enough we'll expand.*/ +uint32_t nr_entries =1 ; + +/* We should check if mmio range is out of RDM mapping. */ +if ( (map =malloc (nr_entries * +sizeof (xen_mem_reserved_device_memory_t))) ==NULL ) + {
tools/libxc/xc_hvm_build_x86.c static int check_mmio_hole(uint64_t start, uint64_t memsize, · line 269
Quoted 10 lines
tools/libxc/xc_hvm_build_x86.c +10
-0+PERROR ("Could not allocate memory for map." ); +return -1 ; + } + rc =xc_reserved_device_memory_map (xch, map, &nr_entries); +if ( rc <0 ) + { +/* DRM doesn't exist. */ +if ( rc == -ENOENT ) + rc =0 ; +else if ( rc == -ENOBUFS )
I think you'd be better off handling both if() levels we're in here with a single switch().
tools/libxc/xc_hvm_build_x86.c static int check_mmio_hole(uint64_t start, uint64_t memsize, · line 282
Quoted 13 lines
tools/libxc/xc_hvm_build_x86.c +13
-0+ { +free (map); +/* Now we need more space to map all RDM mappings. */ +if ( (map =malloc (nr_entries * +sizeof (xen_mem_reserved_device_memory_t))) ==NULL ) + { +PERROR ("Could not allocate memory for map." ); +return -1 ; + } + rc =xc_reserved_device_memory_map (xch, map, &nr_entries); +if ( rc <0 ) + { +PERROR ("Could not get DRM info on domain" );
Please change to RDM here and elsewhere.
tools/libxc/xc_hvm_build_x86.c static int check_mmio_hole(uint64_t start, uint64_t memsize, · line 288
Quoted 6 lines
tools/libxc/xc_hvm_build_x86.c +6
-0+free (map); +return rc; + } + } +else +PERROR ("Could not get RDM info on domain" );
Not returning here means looping for almost 4G iterations below.
tools/libxc/xc_hvm_build_x86.c static int check_mmio_hole(uint64_t start, uint64_t memsize, · line 295
Quoted 7 lines
tools/libxc/xc_hvm_build_x86.c +7
-0+ } + +for ( i =0 ; i < rc; i++ ) + { + rdm_start = map[i].start_pfn <<PAGE_SHIFT ; + rdm_end = rdm_start + map[i].nr_pages *PAGE_SIZE ; + if ( check_mmio_hole(rdm_start, map[i].nr_pages * PAGE_SIZE,
First of all please be consistent: Either always use "<< PAGE_SHIFT" or always use "* PAGE_SIZE". And then what you're doing here won't do what you intend to in a 32-bit toolstack (due to the lack of casts). Plus finally I think using XC_PAGE_* would be the better choice here even if PAGE_* are #define-d as aliases thereof.
Quoted 1 line
+
mmio_start,mmio_size) )
Can you really use check_mmio_hole() here? I'm not certain that its two one-offs are intentional (to allow for a gap between MMIO and RAM), but these are certainly harmful to you.
Review
Quoted 2 lines
(quoted diff) +2
-0+ { + PERROR("MMIO: [%lx]<->[%lx] overlap DRM [%lx]<->[%lx]\n",
Please use a more conventional representation here, e.g.
"MMIO [%lx,%lx) overlaps RDM [%lx,%lx)\n"
Jan
Re: [v6][PATCH 3/7] tools/libxc: check if mmio BAR is out of reserved device memory maps
Quoted 20 lines · Jan Beulich
Quoted 20 lines
Quoted 1 line
Quoted 1 line · On 10.09.14 at 07:49, <[email protected]> wrote:
tools/libxc/xc_hvm_build_x86.c +19
-0+static int check_rdm_overlap (xc_interface *xch,uint64_t mmio_start, +uint64_t mmio_size) +{ +struct xen_mem_reserved_device_memory *map =NULL ; +uint64_t rdm_start =0 , rdm_end =0 ; +unsigned int i =0 ; +int rc =0 ; +/* Assume we have one entry if not enough we'll expand.*/ +uint32_t nr_entries =1 ; + +/* We should check if mmio range is out of RDM mapping. */ +if ( (map =malloc (nr_entries * +sizeof (xen_mem_reserved_device_memory_t))) ==NULL ) + { +PERROR ("Could not allocate memory for map." ); +return -1 ; + } + rc =xc_reserved_device_memory_map (xch, map, &nr_entries); +if ( rc <0 )
Quoted 8 lines
tools/libxc/xc_hvm_build_x86.c static int check_mmio_hole(uint64_t start, uint64_t memsize, · line 269
Quoted 5 lines
tools/libxc/xc_hvm_build_x86.c +5
-0+ { +/* DRM doesn't exist. */ +if ( rc == -ENOENT ) + rc =0 ; +else if ( rc == -ENOBUFS )I think you'd be better off handling both
if()levels we're in here with a singleswitch().
Okay.
Quoted 15 lines
Quoted 4 lines
tools/libxc/xc_hvm_build_x86.c +3
-0+ { +free (map); +/* Now we need more space to map all RDM mappings. */ tools/libxc/xc_hvm_build_x86.c static int check_mmio_hole(uint64_t start, uint64_t memsize, · line 282
Quoted 10 lines
tools/libxc/xc_hvm_build_x86.c +10
-0+if ( (map =malloc (nr_entries * +sizeof (xen_mem_reserved_device_memory_t))) ==NULL ) + { +PERROR ("Could not allocate memory for map." ); +return -1 ; + } + rc =xc_reserved_device_memory_map (xch, map, &nr_entries); +if ( rc <0 ) + { +PERROR ("Could not get DRM info on domain" );
Quoted 1 line
Please change to RDM here and elsewhere.
Okay.
Quoted 24 lines
tools/libxc/xc_hvm_build_x86.c static int check_mmio_hole(uint64_t start, uint64_t memsize, · line 288
Quoted 7 lines
tools/libxc/xc_hvm_build_x86.c +6
-0+free (map); +return rc; + } + } +else +PERROR ("Could not get RDM info on domain" );Not returning here means looping for almost 4G iterations below.
tools/libxc/xc_hvm_build_x86.c static int check_mmio_hole(uint64_t start, uint64_t memsize, · line 295
Quoted 7 lines
tools/libxc/xc_hvm_build_x86.c +7
-0+ } + +for ( i =0 ; i < rc; i++ ) + { + rdm_start = map[i].start_pfn <<PAGE_SHIFT ; + rdm_end = rdm_start + map[i].nr_pages *PAGE_SIZE ; + if ( check_mmio_hole(rdm_start, map[i].nr_pages * PAGE_SIZE,First of all please be consistent: Either always use "<<
PAGE_SHIFT" or always use "*PAGE_SIZE". And then what you're doing here won't do what you intend to in a 32-bit toolstack (due to the lack of casts). Plus finally I think using XC_PAGE_* would be the better choice here even if PAGE_* are #define-d as aliases thereof.
Quoted 5 lines
Quoted 1 line
+
mmio_start,mmio_size) )Can you really use
check_mmio_hole()here? I'm not certain that its two one-offs are intentional (to allow for a gap between MMIO and RAM), but these are certainly harmful to you.
I don't understand what you mean here. But here we should check if RMRR is overlapping the whole MMIO,
/*
* Check whether there exists mmio hole in the specified memory range.
* Returns 1 if exists, else returns 0.
*/
static int check_mmio_hole(uint64_t start, uint64_t memsize,
uint64_t mmio_start, uint64_t mmio_size)
{
if ( start + memsize <= mmio_start || start >= mmio_start + mmio_size )
return 0;
else
return 1;
}Quoted 7 lines
Review
Quoted 3 lines
(quoted diff) +2
-0+ { + PERROR("MMIO: [%lx]<->[%lx] overlap DRM [%lx]<->[%lx]\n",Please use a more conventional representation here, e.g.
"MMIO [%lx,%lx) overlaps RDM [%lx,%lx)\n"
Okay.
I tried to address your comments but I assume its correct to use 'errno' directly as I'm asking in another email:
@@ -54,6 +54,9 @@
#define VGA_HOLE_SIZE (0x20)
+/* Record reserved device memory. */
+static struct xen_mem_reserved_device_memory *xmrdm = NULL;
+
static int modules_init(struct xc_hvm_build_args *args,
uint64_t vend, struct elf_binary *elf, uint64_t *mstart_out, uint64_t *mend_out)
@@ -239,6 +242,70 @@ static int check_mmio_hole(uint64_t start, uint64_t memsize,
return 1;
}
+/*
+ * Check whether there exists mmio overplap with the reserved device
+ * memory map
+ */
+static int check_rdm_overlap(xc_interface *xch, uint64_t mmio_start,
+ uint64_t mmio_size)
+{
+ uint64_t rdm_start = 0, rdm_end = 0; + unsigned int i = 0; + int rc = 0; + /* Assume we have one entry if not enough we'll expand.*/ + uint32_t nr_entries = 1;
+
+ /* We should check if mmio range is out of RDM mapping. */
+ if (!xmrdm) {
+ if ((xmrdm = malloc(nr_entries *
+ sizeof(xen_mem_reserved_device_memory_t))) == NULL) {
+ PERROR("Could not allocate memory for map.");
+ return -1;
+ }
+
+ rc = xc_reserved_device_memory_map(xch, xmrdm, &nr_entries);
+ if (rc < 0 ) {
+ switch (errno) {
+ case ENOENT: /* RDM doesn't exist. */
+ rc = 0;
+ break;
+ case ENOBUFS: /* Need more space */
+ free(xmrdm);
+ /* Now we need more space to map all RDM mappings. */
+ if ((xmrdm = malloc(nr_entries *
+ sizeof(xen_mem_reserved_device_memory_t))) == NULL) {
+ PERROR("Could not allocate memory for map.");
+ return -1;
+ }
+ rc = xc_reserved_device_memory_map(xch, xmrdm, &nr_entries);
+ if (rc) {
+ PERROR("Could not get RDM info on domain");
+ free(xmrdm);
+ return rc;+ }
+ break;
+ default: /* Failed to get RDM */
+ PERROR("Could not get RDM info on domain");
+ return rc;
+ }
+ }
+ }
+
+ for (i = 0; i < rc; i++) {
+ rdm_start = xmrdm[i].start_pfn << XC_PAGE_SHIFT; + rdm_end = rdm_start + (xmrdm[i].nr_pages << XC_PAGE_SHIFT); + if (check_mmio_hole(rdm_start, (xmrdm[i].nr_pages << XC_PAGE_SHIFT),
+ mmio_start, mmio_size) ) {
+ PERROR("MMIO: [%lx,%lx) overlap RDM [%lx,%lx)\n",
+ mmio_start, (mmio_start + mmio_size), rdm_start, rdm_end);
+ free(xmrdm);
+ return -1;
+ }
+ }
+
+ return rc;
+}
+
static int setup_guest(xc_interface *xch,
uint32_t dom, struct xc_hvm_build_args *args, char *image, unsigned long image_size)
Thanks
Tiejun
Re: [v6][PATCH 3/7] tools/libxc: check if mmio BAR is out of reserved device memory maps
In reply to Jan BeulichQuoted 7 lines
Quoted 3 lines
Quoted 2 lines · On 11.09.14 at 17:38, <[email protected]> wrote:
Quoted 1 line · On 10.09.14 at 07:49, <[email protected]> wrote:
+
mmio_start,mmio_size) )Can you really use
check_mmio_hole()here? I'm not certain that its two one-offs are intentional (to allow for a gap between MMIO and RAM), but these are certainly harmful to you.
Oops - I was sure I deleted this after closer looking. Please disregard this.
However, the while thing here seems to get done in the wrong place anyway: The (mmio_start,mmio_size) tuple here doesn't represent any actual MMIO assignments, just the hole below 4G that's being left available for such. Instead you'd need to avoid the specific regions in hvmloader's pci_setup().
Plus, considering migration, all of this isn't going to help if on the destination host (in order for subsequent device hotplug to work) different RMRR regions need to be taken care of.
All in all I continue thinking that we'd be best off disallowing assignment of devices associated with RMRR regions when the 1:1 mappings needed collide with something already there, perhaps with a 2-level operator override to allow USB ones (possibly making the current USB-specific code conditional) or anything at their own risk.
Jan
PATCH v6 4/7 libxc/hvm_info_table: introduce a new field nr_reserved_device_memory_map
In reply to Tiejun ChenIn hvm_info_table this field represents the number of all reserved device memory maps. It will be convenient to expose such a information to VM. While building hvm info, libxc is responsible for constructing this number after check_rdm_overlap().
tools/libxc/xc_hvm_build_x86.c +9 -2
@@ -89,7 +89,8 @@ static int modules_init(struct xc_hvm_build_args *args, }static void build_hvm_info (void *hvm_info_page,uint64_t mem_size, -uint64_t mmio_start,uint64_t mmio_size) +uint64_t mmio_start,uint64_t mmio_size, +unsigned int num) {struct hvm_info_table *hvm_info = (struct hvm_info_table *) (((unsigned char *)hvm_info_page) +HVM_INFO_OFFSET );
@@ -119,6 +120,9 @@ static void build_hvm_info(void *hvm_info_page, uint64_t mem_size, hvm_info->high_mem_pgend = highmem_end >>PAGE_SHIFT ; hvm_info->reserved_mem_pgstart =ioreq_server_pfn (0 ); +/* Reserved device memory map number. */ + hvm_info->nr_reserved_device_memory_map = num; +/* Finish with the checksum. */ for ( i =0 , sum =0 ; i < hvm_info->length; i++ ) sum += ((uint8_t *)hvm_info)[i];
@@ -329,6 +333,7 @@ static int setup_guest(xc_interface *xch,int claim_enabled = args->claim_enabled;xen_pfn_t special_array[NR_SPECIAL_PAGES ];xen_pfn_t ioreq_server_array[NR_IOREQ_SERVER_PAGES ]; +unsigned int num_reserved =0 ;if ( nr_pages > target_pages ) pod_mode = XENMEMF_populate_on_demand;
@@ -371,6 +376,8 @@ static int setup_guest(xc_interface *xch,if ( rc <0 ) goto error_out; + num_reserved = rc; +for ( i =0 ; i < nr_pages; i++ ) page_array[i] = i;for ( i = mmio_start >>PAGE_SHIFT ; i < nr_pages; i++ )
@@ -540,7 +547,7 @@ static int setup_guest(xc_interface *xch, xch, dom, PAGE_SIZE, PROT_READ | PROT_WRITE, HVM_INFO_PFN)) == NULL ) goto error_out; -build_hvm_info (hvm_info_page, v_end, mmio_start, mmio_size); +build_hvm_info (hvm_info_page, v_end, mmio_start, mmio_size, num_reserved);munmap (hvm_info_page,PAGE_SIZE );/* Allocate and clear special pages. */
xen/include/public/hvm/hvm_info_table.h +3 -0
@@ -65,6 +65,9 @@ struct hvm_info_table {*/ uint32_t high_mem_pgend; +/* How many reserved device memory maps does we have? */ +uint32_t nr_reserved_device_memory_map; +/* Bitmap of which CPUs are online at boot time. */ uint8_t vcpu_online[(HVM_MAX_VCPUS +7 )/8 ]; };
Signature · 1 line
1.9.1
Re: [v6][PATCH 4/7] libxc/hvm_info_table: introduce a new field nr_reserved_device_memory_map
Quoted 7 lines
From: Chen, Tiejun
Sent: Tuesday, September 09, 2014 10:50 PMIn
hvm_info_tablethis field represents the number of all reserved device memory maps. It will be convenient to expose such a information to VM. While building hvm info, libxc is responsible for constructing this number aftercheck_rdm_overlap().
Agree with Jan that putting the entry number here looks dirty. If we really want to go this way, I prefer to put the whole reserved entries so it removes the call completely from hvmloader. But since it's a dynamic structure, so it's not a good option here.
Quoted 72 lines
Signed-off-by Tiejun Chen <[email protected]>tools/libxc/xc_hvm_build_x86.c +5
-1@@ -89,7 +89,8 @@ static int modules_init(struct xc_hvm_build_args *args, }static void build_hvm_info (void *hvm_info_page,uint64_t mem_size, -uint64_t mmio_start,uint64_t mmio_size) +uint64_t mmio_start,uint64_t mmio_size, +unsigned int num) {struct hvm_info_table *hvm_info = (struct hvm_info_table *) (((unsigned char *)hvm_info_page) +HVM_INFO_OFFSET );@@ -119,6 +120,9 @@ static void build_hvm_info(void *hvm_info_page,uint64_t mem_size, hvm_info->high_mem_pgend = highmem_end >> PAGE_SHIFT; hvm_info->reserved_mem_pgstart =ioreq_server_pfn (0 ); +/* Reserved device memory map number. */ + hvm_info->nr_reserved_device_memory_map = num; +/* Finish with the checksum. */ for ( i =0 , sum =0 ; i < hvm_info->length; i++ )sum += ((uint8_t *)hvm_info)[i];tools/libxc/xc_hvm_build_x86.c +4
-1@@ -329,6 +333,7 @@ static int setup_guest(xc_interface *xch,int claim_enabled = args->claim_enabled;xen_pfn_t special_array[NR_SPECIAL_PAGES ];xen_pfn_t ioreq_server_array[NR_IOREQ_SERVER_PAGES ]; +unsigned int num_reserved =0 ;if ( nr_pages > target_pages ) pod_mode = XENMEMF_populate_on_demand;@@ -371,6 +376,8 @@ static int setup_guest(xc_interface *xch,if ( rc <0 ) goto error_out; + num_reserved = rc; +for ( i =0 ; i < nr_pages; i++ ) page_array[i] = i;for ( i = mmio_start >>PAGE_SHIFT ; i < nr_pages; i++ )@@ -540,7 +547,7 @@ static int setup_guest(xc_interface *xch, xch, dom, PAGE_SIZE, PROT_READ | PROT_WRITE, HVM_INFO_PFN)) == NULL ) goto error_out; -build_hvm_info (hvm_info_page, v_end, mmio_start, mmio_size); +build_hvm_info (hvm_info_page, v_end, mmio_start, mmio_size, num_reserved);munmap (hvm_info_page,PAGE_SIZE );/* Allocate and clear special pages. */xen/include/public/hvm/hvm_info_table.h +3
-1@@ -65,6 +65,9 @@ struct hvm_info_table {*/ uint32_t high_mem_pgend; +/* How many reserved device memory maps does we have? */ +uint32_t nr_reserved_device_memory_map; +/* Bitmap of which CPUs are online at boot time. */ uint8_t vcpu_online[(HVM_MAX_VCPUS +7 )/8 ]; }; --1.9.1
Re: [v6][PATCH 4/7] libxc/hvm_info_table: introduce a new field nr_reserved_device_memory_map
Quoted 8 lines · Kevin
Quoted 7 lines
From: Chen, Tiejun
Sent: Tuesday, September 09, 2014 10:50 PMIn
hvm_info_tablethis field represents the number of all reserved device memory maps. It will be convenient to expose such a information to VM. While building hvm info, libxc is responsible for constructing this number aftercheck_rdm_overlap().
Quoted 4 lines
Agree with Jan that putting the entry number here looks dirty. If we really want to go this way, I prefer to put the whole reserved entries so it removes the call completely from hvmloader. But since it's a dynamic structure, so it's not a good option here.
Looks most guys don't refer to introducing such a thing, I will remove this.
Thanks
Tiejun
Quoted 75 lines
Quoted 73 lines
Signed-off-by Tiejun Chen <[email protected]>tools/libxc/xc_hvm_build_x86.c +5
-1@@ -89,7 +89,8 @@ static int modules_init(struct xc_hvm_build_args *args, }static void build_hvm_info (void *hvm_info_page,uint64_t mem_size, -uint64_t mmio_start,uint64_t mmio_size) +uint64_t mmio_start,uint64_t mmio_size, +unsigned int num) {struct hvm_info_table *hvm_info = (struct hvm_info_table *) (((unsigned char *)hvm_info_page) +HVM_INFO_OFFSET );@@ -119,6 +120,9 @@ static void build_hvm_info(void *hvm_info_page,uint64_t mem_size, hvm_info->high_mem_pgend = highmem_end >> PAGE_SHIFT; hvm_info->reserved_mem_pgstart =ioreq_server_pfn (0 ); +/* Reserved device memory map number. */ + hvm_info->nr_reserved_device_memory_map = num; +/* Finish with the checksum. */ for ( i =0 , sum =0 ; i < hvm_info->length; i++ )sum += ((uint8_t *)hvm_info)[i];tools/libxc/xc_hvm_build_x86.c +4
-1@@ -329,6 +333,7 @@ static int setup_guest(xc_interface *xch,int claim_enabled = args->claim_enabled;xen_pfn_t special_array[NR_SPECIAL_PAGES ];xen_pfn_t ioreq_server_array[NR_IOREQ_SERVER_PAGES ]; +unsigned int num_reserved =0 ;if ( nr_pages > target_pages ) pod_mode = XENMEMF_populate_on_demand;@@ -371,6 +376,8 @@ static int setup_guest(xc_interface *xch,if ( rc <0 ) goto error_out; + num_reserved = rc; +for ( i =0 ; i < nr_pages; i++ ) page_array[i] = i;for ( i = mmio_start >>PAGE_SHIFT ; i < nr_pages; i++ )@@ -540,7 +547,7 @@ static int setup_guest(xc_interface *xch, xch, dom, PAGE_SIZE, PROT_READ | PROT_WRITE, HVM_INFO_PFN)) == NULL ) goto error_out; -build_hvm_info (hvm_info_page, v_end, mmio_start, mmio_size); +build_hvm_info (hvm_info_page, v_end, mmio_start, mmio_size, num_reserved);munmap (hvm_info_page,PAGE_SIZE );/* Allocate and clear special pages. */xen/include/public/hvm/hvm_info_table.h +3
-1@@ -65,6 +65,9 @@ struct hvm_info_table {*/ uint32_t high_mem_pgend; +/* How many reserved device memory maps does we have? */ +uint32_t nr_reserved_device_memory_map; +/* Bitmap of which CPUs are online at boot time. */ uint8_t vcpu_online[(HVM_MAX_VCPUS +7 )/8 ]; }; --1.9.1
PATCH v6 5/7 hvmloader: introduce hypercall for xc_reserved_device_memory_map
In reply to Tiejun ChenWe will introduce that hypercall xc_reserved_device_memory_map to hvmloader.
tools/firmware/hvmloader/util.c +22 -0
@@ -828,6 +828,28 @@ int hpet_exists(unsigned long hpet_base)return ((hpet_id >>16 ) ==0x8086 ); } +int get_reserved_device_memory_map (struct xen_mem_reserved_device_memory entries[], +uint32_t max_entries) +{ +static int map_done =0 ; +struct xen_mem_reserved_device_memory_map memmap = { + .nr_entries = max_entries + }; + +if ( map_done ) +return 0 ; + +set_xen_guest_handle (memmap.buffer, entries); + +if (hypercall_memory_op (XENMEM_reserved_device_memory_map, + &memmap) !=0 ) +BUG (); + + map_done =1 ; + +return 0 ; +} +/* * Local variables: * mode: C
tools/firmware/hvmloader/util.h +3 -0
@@ -241,6 +241,9 @@ int build_e820_table(struct e820entry *e820, unsigned int bios_image_base);void dump_e820_table (struct e820entry *e820,unsigned int nr); +#include <xen/memory.h> +int get_reserved_device_memory_map (struct xen_mem_reserved_device_memory entries[], +uint32_t max_entries);#ifndef NDEBUG void perform_tests (void );#else
Signature · 1 line
1.9.1
Re: [v6][PATCH 5/7] hvmloader: introduce hypercall for xc_reserved_device_memory_map
Quoted 13 lines
From: Chen, Tiejun
Sent: Tuesday, September 09, 2014 10:50 PMWe will introduce that hypercall
xc_reserved_device_memory_mapto hvmloader.Signed-off-by Tiejun Chen <[email protected]>tools/firmware/hvmloader/util.c +0
-0
tools/firmware/hvmloader/util.c int hpet_exists(unsigned long hpet_base) · line 833
Quoted 6 lines
tools/firmware/hvmloader/util.c +2
-0@@ -828,6 +828,28 @@ int hpet_exists(unsigned long hpet_base)return ((hpet_id >>16 ) ==0x8086 ); } +int get_reserved_device_memory_map (struct xen_mem_reserved_device_memory entries[], +uint32_t max_entries)
usually claim as a pointer instead of array as the parameter.
tools/firmware/hvmloader/util.c int hpet_exists(unsigned long hpet_base) · line 834
Quoted 2 lines
tools/firmware/hvmloader/util.c +2
-0+{ +static int map_done =0 ;
don't see much value of the static variable here. Better for the caller to judge whether multiple calls are required.
Quoted 39 lines
tools/firmware/hvmloader/util.c +18
-0+struct xen_mem_reserved_device_memory_map memmap = { + .nr_entries = max_entries + }; + +if ( map_done ) +return 0 ; + +set_xen_guest_handle (memmap.buffer, entries); + +if (hypercall_memory_op (XENMEM_reserved_device_memory_map, + &memmap) !=0 ) +BUG (); + + map_done =1 ; + +return 0 ; +} +/* * Local variables: * mode: C
tools/firmware/hvmloader/util.h +3
-0@@ -241,6 +241,9 @@ int build_e820_table(struct e820entry *e820, unsigned int bios_image_base);void dump_e820_table (struct e820entry *e820,unsigned int nr); +#include <xen/memory.h> +int get_reserved_device_memory_map (struct xen_mem_reserved_device_memory entries[], +uint32_t max_entries);#ifndef NDEBUG void perform_tests (void );#else
--
1.9.1
Re: [v6][PATCH 5/7] hvmloader: introduce hypercall for xc_reserved_device_memory_map
Quoted 16 lines · Kevin
Quoted 16 lines
From: Chen, Tiejun
Sent: Tuesday, September 09, 2014 10:50 PMWe will introduce that hypercall
xc_reserved_device_memory_mapto hvmloader.Signed-off-by Tiejun Chen <[email protected]>tools/firmware/hvmloader/util.c +0
-0@@ -828,6 +828,28 @@ int hpet_exists(unsigned long hpet_base)return ((hpet_id >>16 ) ==0x8086 ); }
Quoted 5 lines
tools/firmware/hvmloader/util.c int hpet_exists(unsigned long hpet_base) · line 833
Quoted 3 lines
tools/firmware/hvmloader/util.c +2
-0+int get_reserved_device_memory_map (struct xen_mem_reserved_device_memory entries[], +uint32_t max_entries)usually claim as a pointer instead of array as the parameter.
But I see there are some existing similar things like,
tools/libxc/xenctrl.h:
int xc_domain_set_memory_map(xc_interface *xch,
uint32_t domid,
struct e820entry entries[],
uint32_t nr_entries);int xc_get_machine_memory_map(xc_interface *xch,
struct e820entry entries[],
uint32_t max_entries);Quoted 6 lines
tools/firmware/hvmloader/util.c int hpet_exists(unsigned long hpet_base) · line 834
Quoted 3 lines
tools/firmware/hvmloader/util.c +2
-0+{ +static int map_done =0 ;don't see much value of the static variable here. Better for the caller to judge whether multiple calls are required.
Originally this intends to indicate if we call this hypercall in case when we already know that appropriate entries number as we introduce that new field nr_reserved_device_memory_map in hvm_info_table. If so, we just need to call one time.
But looks you guys think we shouldn't live with nr_reserved_device_memory_map, so this should be gone.
Thanks
Tiejun
Quoted 42 lines
Quoted 40 lines
tools/firmware/hvmloader/util.c +18
-0+struct xen_mem_reserved_device_memory_map memmap = { + .nr_entries = max_entries + }; + +if ( map_done ) +return 0 ; + +set_xen_guest_handle (memmap.buffer, entries); + +if (hypercall_memory_op (XENMEM_reserved_device_memory_map, + &memmap) !=0 ) +BUG (); + + map_done =1 ; + +return 0 ; +} +/* * Local variables: * mode: C
tools/firmware/hvmloader/util.h +3
-0@@ -241,6 +241,9 @@ int build_e820_table(struct e820entry *e820, unsigned int bios_image_base);void dump_e820_table (struct e820entry *e820,unsigned int nr); +#include <xen/memory.h> +int get_reserved_device_memory_map (struct xen_mem_reserved_device_memory entries[], +uint32_t max_entries);#ifndef NDEBUG void perform_tests (void );#else
--
1.9.1
Re: [v6][PATCH 5/7] hvmloader: introduce hypercall for xc_reserved_device_memory_map
In reply to Tian, KevinQuoted 17 lines
Quoted 17 lines
Quoted 1 line · On 10.09.14 at 23:41, <[email protected]> wrote:
From: Chen, Tiejun
Sent: Tuesday, September 09, 2014 10:50 PMWe will introduce that hypercall
xc_reserved_device_memory_mapto hvmloader.Signed-off-by Tiejun Chen <[email protected]>tools/firmware/hvmloader/util.c +0
-0@@ -828,6 +828,28 @@ int hpet_exists(unsigned long hpet_base)return ((hpet_id >>16 ) ==0x8086 ); }
Quoted 5 lines
tools/firmware/hvmloader/util.c int hpet_exists(unsigned long hpet_base) · line 833
Quoted 3 lines
tools/firmware/hvmloader/util.c +2
-0+int get_reserved_device_memory_map (struct xen_mem_reserved_device_memory entries[], +uint32_t max_entries)usually claim as a pointer instead of array as the parameter.
I see nothing wrong with the array syntax; if what is being referred to in fact is more than one object, I'd actually slightly favor this syntax. All that keeping in mind that both forms mean _exactly_ the same.
Jan
Re: [v6][PATCH 5/7] hvmloader: introduce hypercall for xc_reserved_device_memory_map
In reply to Tiejun ChenQuoted 3 lines
Quoted 1 line
Quoted 1 line · On 10.09.14 at 07:49, <[email protected]> wrote:
We will introduce that hypercall
xc_reserved_device_memory_mapto hvmloader.
Title and text: What does a libxc name do here?
Also I don't think this warrants a separate patch - introduce the function along with the code needing it.
Quoted 16 lines
Signed-off-by Tiejun Chen <[email protected]>tools/firmware/hvmloader/util.c +6
-0@@ -828,6 +828,28 @@ int hpet_exists(unsigned long hpet_base)return ((hpet_id >>16 ) ==0x8086 ); } +int get_reserved_device_memory_map (struct xen_mem_reserved_device_memory entries[], +uint32_t max_entries) +{ +static int map_done =0 ; +struct xen_mem_reserved_device_memory_map memmap = { + .nr_entries = max_entries
tools/firmware/hvmloader/util.c int hpet_exists(unsigned long hpet_base) · line 846
Quoted 10 lines
tools/firmware/hvmloader/util.c +10
-0+ }; + +if ( map_done ) +return 0 ; + +set_xen_guest_handle (memmap.buffer, entries); + +if (hypercall_memory_op (XENMEM_reserved_device_memory_map, + &memmap) !=0 ) +BUG ();
I don't think there's any harm in not BUG()ing here - just return the error to the caller (and let it assume there are no entries). In the worst case guest startup will subsequently fail (instead of here).
tools/firmware/hvmloader/util.c int hpet_exists(unsigned long hpet_base) · line 848
Quoted 2 lines
tools/firmware/hvmloader/util.c +2
-0+ + map_done =1 ;
The map_done variable if completely bogus here, but I think someone else already pointed this out.
Jan
Re: [v6][PATCH 5/7] hvmloader: introduce hypercall for xc_reserved_device_memory_map
Quoted 5 lines · Jan Beulich
Quoted 3 lines
Quoted 1 line
Quoted 1 line · On 10.09.14 at 07:49, <[email protected]> wrote:
We will introduce that hypercall
xc_reserved_device_memory_mapto hvmloader.Title and text: What does a libxc name do here?
I need to rename this.
Quoted 3 lines
Also I don't think this warrants a separate patch - introduce the function along with the code needing it.
I will squash the following patch into this.
Quoted 23 lines
Quoted 23 lines
Signed-off-by Tiejun Chen <[email protected]>tools/firmware/hvmloader/util.c +12
-0@@ -828,6 +828,28 @@ int hpet_exists(unsigned long hpet_base)return ((hpet_id >>16 ) ==0x8086 ); } +int get_reserved_device_memory_map (struct xen_mem_reserved_device_memory entries[], +uint32_t max_entries) +{ +static int map_done =0 ; +struct xen_mem_reserved_device_memory_map memmap = { + .nr_entries = max_entries + }; + +if ( map_done ) +return 0 ; + +set_xen_guest_handle (memmap.buffer, entries);
Quoted 8 lines
tools/firmware/hvmloader/util.c int hpet_exists(unsigned long hpet_base) · line 846
Quoted 4 lines
tools/firmware/hvmloader/util.c +4
-0+ +if (hypercall_memory_op (XENMEM_reserved_device_memory_map, + &memmap) !=0 ) +BUG ();I don't think there's any harm in not BUG()ing here - just return the error to the caller (and let it assume there are no entries). In the worst case guest startup will subsequently fail (instead of here).
Okay so just return rc.
Quoted 7 lines
tools/firmware/hvmloader/util.c int hpet_exists(unsigned long hpet_base) · line 848
Quoted 3 lines
tools/firmware/hvmloader/util.c +2
-0+ + map_done =1 ;The
map_donevariable if completely bogus here, but I think someone else already pointed this out.
Yes. I will remove this.
Thanks
Tiejun
PATCH v6 6/7 hvmloader: check to reserved device memory maps in e820
In reply to Tiejun ChenWe need to check to reserve all reserved device memory maps in e820 to avoid any potential guest memory conflict.
tools/firmware/hvmloader/e820.c +100 -0
@@ -68,12 +68,89 @@ void dump_e820_table(struct e820entry *e820, unsigned int nr) } } +static unsigned int construct_rdm_e820_maps (unsigned int nr, +uint32_t nr_map, +struct xen_mem_reserved_device_memory *map, +struct e820entry *e820) +{ +unsigned int i =0 , j =0 , m =0 , sum_nr = nr + nr_map; +uint64_t start, end, rdm_start, rdm_end; +unsigned int insert =0 , do_insert =0 ; + + do_real_construct: +for ( i =0 ; i < nr_map; i++ ) + { + rdm_start = map[i].start_pfn <<PAGE_SHIFT ; + rdm_end = rdm_start + map[i].nr_pages *PAGE_SIZE ; + +for ( j =0 ; j < nr -1 ; j++ ) + { + end = e820[j].addr + e820[j].size; + start = e820[j+1 ].addr; + +/* Between those existing e820 entries. */ +if ( (rdm_start > end) && (rdm_end < start) ) + { +if ( do_insert ) + { +/* Move to free this entry. */ +for ( m = sum_nr -1 ; m > j; m-- ) + { + e820[m].addr = e820[m-1 ].addr; + e820[m].size = e820[m-1 ].size; + e820[m].type = e820[m-1 ].type; + } + +/* Then fill RMRR into that entry. */ + e820[j+1 ].addr = rdm_start; + e820[j+1 ].size = rdm_end - rdm_start; + e820[j+1 ].type =E820_RESERVED ; + nr++; + } + insert++; + } +/* Already at the end. */ +else if ( (rdm_start > end) && !start ) + { +if ( do_insert ) + { + e820[nr].addr = rdm_start; + e820[nr].size = rdm_end - rdm_start; + e820[nr].type =E820_RESERVED ; + nr++; + } + insert++; + } + } + } + +/* Just return if done. */ +if ( do_insert ) +return nr; + +/* Fine to construct RDM mappings into e820. */ +if ( insert == nr_map ) + { + do_insert =1 ; + goto do_real_construct; + } +/* Overlap. */ +else + { +printf ("RDM overlap with those existing e820 entries!\n" ); +printf ("So we don't construct RDM mapping in e820!\n" ); + } + +return nr; +}/* Create an E820 table based on memory parameters provided in hvm_info. */ int build_e820_table (struct e820entry *e820,unsigned int lowmem_reserved_base,unsigned int bios_image_base) {unsigned int nr =0 ; +struct xen_mem_reserved_device_memory *map =NULL ; +int rc;if ( !lowmem_reserved_base ) lowmem_reserved_base =0xA0000 ;
@@ -169,6 +246,29 @@ int build_e820_table(struct e820entry *e820, nr++; } +/* We'd better reserve RDM maps for each VM to avoid potential +* memory conflict. +*/ +if ( hvm_info->nr_reserved_device_memory_map ) + { +if ( !map ) + map =mem_alloc (hvm_info->nr_reserved_device_memory_map * +sizeof (struct xen_mem_reserved_device_memory ),0 ); +if ( map ) + { + rc =get_reserved_device_memory_map (map, + hvm_info->nr_reserved_device_memory_map); +if ( !rc ) + { + nr =construct_rdm_e820_maps (nr, + hvm_info->nr_reserved_device_memory_map, + map, e820); + } + } +else +printf ("No space to get device reserved memory maps!\n" ); + } +return nr; }
Signature · 1 line
1.9.1
Re: [v6][PATCH 6/7] hvmloader: check to reserved device memory maps in e820
tools/firmware/hvmloader/e820.c void dump_e820_table(struct e820entry *e820, unsigned int nr) · line 76
Quoted 7 lines
Quoted 1 line
Quoted 1 line · On 10.09.14 at 07:49, <[email protected]> wrote:
tools/firmware/hvmloader/e820.c +6
-0+static unsigned int construct_rdm_e820_maps (unsigned int nr, +uint32_t nr_map, +struct xen_mem_reserved_device_memory *map, +struct e820entry *e820) +{ +unsigned int i =0 , j =0 , m =0 , sum_nr = nr + nr_map;
Some or all of the initializers here are pointless.
tools/firmware/hvmloader/e820.c void dump_e820_table(struct e820entry *e820, unsigned int nr) · line 84
Quoted 8 lines
tools/firmware/hvmloader/e820.c +8
-0+uint64_t start, end, rdm_start, rdm_end; +unsigned int insert =0 , do_insert =0 ; + + do_real_construct: +for ( i =0 ; i < nr_map; i++ ) + { + rdm_start = map[i].start_pfn <<PAGE_SHIFT ; + rdm_end = rdm_start + map[i].nr_pages *PAGE_SIZE ;
Same comment as for an earlier patch.
tools/firmware/hvmloader/e820.c void dump_e820_table(struct e820entry *e820, unsigned int nr) · line 89
Quoted 5 lines
tools/firmware/hvmloader/e820.c +5
-0+ +for ( j =0 ; j < nr -1 ; j++ ) + { + end = e820[j].addr + e820[j].size; + start = e820[j+1 ].addr;
So now you don't check the last region at all? You just need to make this assignment conditional upon whether you're in the last iteration.
tools/firmware/hvmloader/e820.c void dump_e820_table(struct e820entry *e820, unsigned int nr) · line 102
Quoted 13 lines
tools/firmware/hvmloader/e820.c +13
-0+ +/* Between those existing e820 entries. */ +if ( (rdm_start > end) && (rdm_end < start) ) + { +if ( do_insert ) + { +/* Move to free this entry. */ +for ( m = sum_nr -1 ; m > j; m-- ) + { + e820[m].addr = e820[m-1 ].addr; + e820[m].size = e820[m-1 ].size; + e820[m].type = e820[m-1 ].type; + }
You know of the memmove() function, don't you?
tools/firmware/hvmloader/e820.c void dump_e820_table(struct e820entry *e820, unsigned int nr) · line 113
Quoted 11 lines
tools/firmware/hvmloader/e820.c +11
-0+ +/* Then fill RMRR into that entry. */ + e820[j+1 ].addr = rdm_start; + e820[j+1 ].size = rdm_end - rdm_start; + e820[j+1 ].type =E820_RESERVED ; + nr++; + } + insert++; + } +/* Already at the end. */ +else if ( (rdm_start > end) && !start )
How would !start represent the end of anything?
Quoted 23 lines
tools/firmware/hvmloader/e820.c +23
-0+ { +if ( do_insert ) + { + e820[nr].addr = rdm_start; + e820[nr].size = rdm_end - rdm_start; + e820[nr].type =E820_RESERVED ; + nr++; + } + insert++; + } + } + } + +/* Just return if done. */ +if ( do_insert ) +return nr; + +/* Fine to construct RDM mappings into e820. */ +if ( insert == nr_map ) + { + do_insert =1 ; + goto do_real_construct; + }
tools/firmware/hvmloader/e820.c void dump_e820_table(struct e820entry *e820, unsigned int nr) · line 146
Quoted 10 lines
tools/firmware/hvmloader/e820.c +9
-0+/* Overlap. */ + else + { +printf ("RDM overlap with those existing e820 entries!\n" ); +printf ("So we don't construct RDM mapping in e820!\n" ); + } + +return nr; +}/* Create an E820 table based on memory parameters provided in hvm_info. */
Blank line missing above.
Jan
Re: [v6][PATCH 6/7] hvmloader: check to reserved device memory maps in e820
Quoted 23 lines
tools/firmware/hvmloader/e820.c void dump_e820_table(struct e820entry *e820, unsigned int nr)
Quoted 5 lines
Quoted 2 lines · On 11.09.14 at 17:57, <[email protected]> wrote:
Quoted 1 line · On 10.09.14 at 07:49, <[email protected]> wrote:
tools/firmware/hvmloader/e820.c +3
-0+static unsigned int construct_rdm_e820_maps(unsigned int nr, + uint32_t nr_map, + struct
xen_mem_reserved_device_memory*map,tools/firmware/hvmloader/e820.c void dump_e820_table(struct e820entry *e820, unsigned int nr) · line 76
Quoted 3 lines
tools/firmware/hvmloader/e820.c +3
-0+ struct e820entry *e820) +{ +unsigned int i =0 , j =0 , m =0 , sum_nr = nr + nr_map;Some or all of the initializers here are pointless.
tools/firmware/hvmloader/e820.c void dump_e820_table(struct e820entry *e820, unsigned int nr) · line 84
Quoted 8 lines
tools/firmware/hvmloader/e820.c +8
-0+uint64_t start, end, rdm_start, rdm_end; +unsigned int insert =0 , do_insert =0 ; + + do_real_construct: +for ( i =0 ; i < nr_map; i++ ) + { + rdm_start = map[i].start_pfn <<PAGE_SHIFT ; + rdm_end = rdm_start + map[i].nr_pages *PAGE_SIZE ;Same comment as for an earlier patch.
Quoted 8 lines
tools/firmware/hvmloader/e820.c void dump_e820_table(struct e820entry *e820, unsigned int nr) · line 89
Quoted 5 lines
tools/firmware/hvmloader/e820.c +5
-0+ +for ( j =0 ; j < nr -1 ; j++ ) + { + end = e820[j].addr + e820[j].size; + start = e820[j+1 ].addr;So now you don't check the last region at all? You just need to make this assignment conditional upon whether you're in the last iteration.
Furthermore
- I don't think an overlap with an E820_RESERVED or E820_UNUSABLE region would be a problem (you'd just need to extend the former or amend the latter)
- you only check here rather than making sure no RAM gets placed into these address regions in the first place (or rather than relocating the RAM as is done for at least some MMIO/RAM overlaps)
Jan
Re: [v6][PATCH 6/7] hvmloader: check to reserved device memory maps in e820
In reply to Jan BeulichQuoted 9 lines · Jan Beulich
tools/firmware/hvmloader/e820.c void dump_e820_table(struct e820entry *e820, unsigned int nr) · line 76
Quoted 7 lines
Quoted 1 line
Quoted 1 line · On 10.09.14 at 07:49, <[email protected]> wrote:
tools/firmware/hvmloader/e820.c +6
-0+static unsigned int construct_rdm_e820_maps (unsigned int nr, +uint32_t nr_map, +struct xen_mem_reserved_device_memory *map, +struct e820entry *e820) +{ +unsigned int i =0 , j =0 , m =0 , sum_nr = nr + nr_map;Some or all of the initializers here are pointless.
unsigned int i, j, m, sum_nr = nr + nr_map;
uint64_t start, end, rdm_start, rdm_end;
unsigned int insert, do_insert;Quoted 11 lines
tools/firmware/hvmloader/e820.c void dump_e820_table(struct e820entry *e820, unsigned int nr) · line 84
Quoted 9 lines
tools/firmware/hvmloader/e820.c +8
-0+uint64_t start, end, rdm_start, rdm_end; +unsigned int insert =0 , do_insert =0 ; + + do_real_construct: +for ( i =0 ; i < nr_map; i++ ) + { + rdm_start = map[i].start_pfn <<PAGE_SHIFT ; + rdm_end = rdm_start + map[i].nr_pages *PAGE_SIZE ;Same comment as for an earlier patch.
rdm_start = map[i].start_pfn << XC_PAGE_SHIFT;
rdm_end = rdm_start + map[i].nr_pages << XC_PAGE_SHIFT;But there's no such definition in tools/firmware/, and this is defined in xenctl.h. But if I include xenctl.h here, something else is redefined. So here I have to define this, XC_PAGE_SHIFTseparately.
Quoted 9 lines
tools/firmware/hvmloader/e820.c void dump_e820_table(struct e820entry *e820, unsigned int nr) · line 89
Quoted 6 lines
tools/firmware/hvmloader/e820.c +5
-0+ +for ( j =0 ; j < nr -1 ; j++ ) + { + end = e820[j].addr + e820[j].size; + start = e820[j+1 ].addr;So now you don't check the last region at all? You just need to make this assignment conditional upon whether you're in the last iteration.
I remember we discussed this previously.
"
Quoted 2 lines
And this would skip the last region - I'm not sure that's correct.
No. Here we just want to know where we should insert a entry, so
for ( j = 0; j < nr - 1; j++ )
{
end = e820[j].addr + e820[j].size;
start = e820[j+1].addr;For example, nr = 3,
for ( j = 0; j < 2; j++ )
So
#1,
end = e820[0].addr + e820[0].size;
start = e820[1].addr;#2,
end = e820[1].addr + e820[1].size;
start = e820[2].addr;"
I didn't you have further comments there.
Quoted 15 lines
Quoted 4 lines
tools/firmware/hvmloader/e820.c +3
-0+ +/* Between those existing e820 entries. */ +if ( (rdm_start > end) && (rdm_end < start) )tools/firmware/hvmloader/e820.c void dump_e820_table(struct e820entry *e820, unsigned int nr) · line 102
Quoted 10 lines
tools/firmware/hvmloader/e820.c +10
-0+ { +if ( do_insert ) + { +/* Move to free this entry. */ +for ( m = sum_nr -1 ; m > j; m-- ) + { + e820[m].addr = e820[m-1 ].addr; + e820[m].size = e820[m-1 ].size; + e820[m].type = e820[m-1 ].type; + }
Quoted 1 line
You know of the
memmove()function, don't you?
I will try this.
Quoted 13 lines
tools/firmware/hvmloader/e820.c void dump_e820_table(struct e820entry *e820, unsigned int nr) · line 113
Quoted 12 lines
tools/firmware/hvmloader/e820.c +11
-0+ +/* Then fill RMRR into that entry. */ + e820[j+1 ].addr = rdm_start; + e820[j+1 ].size = rdm_end - rdm_start; + e820[j+1 ].type =E820_RESERVED ; + nr++; + } + insert++; + } +/* Already at the end. */ +else if ( (rdm_start > end) && !start )
Quoted 1 line
How would !start represent the end of anything?
end = e820[j].addr + e820[j].size;
start = e820[j+1].addr;
so if end = 0xXXXXXXXX and start = 0, does this mean we already are at the end of all valid e820 entries?
Quoted 28 lines
Quoted 28 lines
tools/firmware/hvmloader/e820.c +27
-0+ { +if ( do_insert ) + { + e820[nr].addr = rdm_start; + e820[nr].size = rdm_end - rdm_start; + e820[nr].type =E820_RESERVED ; + nr++; + } + insert++; + } + } + } + +/* Just return if done. */ +if ( do_insert ) +return nr; + +/* Fine to construct RDM mappings into e820. */ +if ( insert == nr_map ) + { + do_insert =1 ; + goto do_real_construct; + } +/* Overlap. */ +else + { +printf ("RDM overlap with those existing e820 entries!\n" );
Quoted 9 lines
tools/firmware/hvmloader/e820.c void dump_e820_table(struct e820entry *e820, unsigned int nr) · line 146
Quoted 6 lines
tools/firmware/hvmloader/e820.c +5
-0+printf ("So we don't construct RDM mapping in e820!\n" ); + } + +return nr; +}/* Create an E820 table based on memory parameters provided in hvm_info. */ Blank line missing above.
Will fixed.
Thanks
Tiejun
Re: [v6][PATCH 6/7] hvmloader: check to reserved device memory maps in e820
Quoted 9 lines
Quoted 1 line
Quoted 1 line · On 12.09.14 at 08:28, <[email protected]> wrote:
Quoted 7 lines · Jan Beulich
Quoted 6 lines
Quoted 1 line
Quoted 1 line · On 10.09.14 at 07:49, <[email protected]> wrote:
+
do_real_construct:tools/firmware/hvmloader/e820.c +4
-0+for ( i =0 ; i < nr_map; i++ ) + { + rdm_start = map[i].start_pfn <<PAGE_SHIFT ; + rdm_end = rdm_start + map[i].nr_pages *PAGE_SIZE ;
Quoted 8 lines
Quoted 1 line
Same comment as for an earlier patch.
rdm_start = map[i].start_pfn << XC_PAGE_SHIFT; rdm_end = rdm_start + map[i].nr_pages << XC_PAGE_SHIFT;But there's no such definition in
tools/firmware/, and this is defined inxenctl.h. But if I includexenctl.hhere, something else is redefined. So here I have to define this,XC_PAGE_SHIFTseparately.
Iirc the earlier comment was on a libxc change. Obviously using its definitions in hvmloader is forbidden. Out of the three things that were pointed out wrong there, two still apply (and I think you should have the smarts to filter out the one that doesn't rather than me having to re-explain the same issue here again).
Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
tools/firmware/hvmloader/e820.c +11
-0+ +/* Then fill RMRR into that entry. */ + e820[j+1 ].addr = rdm_start; + e820[j+1 ].size = rdm_end - rdm_start; + e820[j+1 ].type =E820_RESERVED ; + nr++; + } + insert++; + } +/* Already at the end. */ +else if ( (rdm_start > end) && !start )
Quoted 7 lines
Quoted 1 line
How would !start represent the end of anything?
end = e820[j].addr + e820[j].size;
start = e820[j+1].addr;so if end = 0xXXXXXXXX and start = 0, does this mean we already are at the end of all valid e820 entries?
How would start validly be zero (i.e. other than incidentally due to reading from an uninitialized entry)?
Jan
PATCH v6 7/7 xen/vtd: make USB RMRR mapping safe
In reply to Tiejun ChenWe already reserve all RMRR mappings so USB should be safe with RMRR.
xen/drivers/passthrough/vtd/iommu.c +0 -8
@@ -2273,16 +2273,8 @@ static int intel_iommu_assign_device(if ( ret ) goto done; -/* FIXME: Because USB RMRR conflicts with guest bios region, -* ignore USB RMRR temporarily. -*/ seg = pdev->seg; bus = pdev->bus; -if (is_usb_device (seg, bus, pdev->devfn) ) - { - ret =0 ; - goto done; - }/* Setup rmrr identity mapping */ for_each_rmrr_device ( rmrr, bdf, i )
Signature · 1 line
1.9.1
Re: [v6][PATCH 7/7] xen/vtd: make USB RMRR mapping safe
Quoted 10 lines
Quoted 1 line
Quoted 1 line · On 10.09.14 at 07:49, <[email protected]> wrote:
xen/drivers/passthrough/vtd/iommu.c +0
-3@@ -2273,16 +2273,8 @@ static int intel_iommu_assign_device(if ( ret ) goto done; -/* FIXME: Because USB RMRR conflicts with guest bios region, -* ignore USB RMRR temporarily. -*/
xen/drivers/passthrough/vtd/iommu.c static int intel_iommu_assign_device( · line 2280
Quoted 10 lines
xen/drivers/passthrough/vtd/iommu.c +0
-5seg = pdev->seg; bus = pdev->bus; -if (is_usb_device (seg, bus, pdev->devfn) ) - { - ret =0 ; - goto done; - }/* Setup rmrr identity mapping */ for_each_rmrr_device ( rmrr, bdf, i )
This orphans is_usb_device(), which therefore the patch should remove at once.
Jan
Quoted 3 lines
From: Chen, Tiejun
Sent: Tuesday, September 09, 2014 10:50 PM
currently the confliction is detected absolutely. Do we need a way to allow the confliction if there is no device assigned at all?
Quoted 100 lines
v6:
* Jan cleanup to regenerate one patch to replace the two original patches then rebase the others.
* Refine all patches short/long logs.
* Replace those info with 'reserved device memory maps' and s/RMRR/RDM.
* Fix some code styles.
* Fix one interation error when we grab e820 info
This is not valid when j == nr - 1 (last iteration).
* Cleanup some codes.
* One comment to introduce a new field,nr_reserved_device_memory_maplibxc/hvm_info_table, is still pending to wait the tools maintainer's further comment.v5:
* Add patch #2 to introduce a global count,
acpi_rmrr_unit_entries.
* Refine hypercall return value to make sure the caller can distinguish clearly between "xen filled in N entries" and "xen said you need N entries for all information".
* Refine some structures
* Then Rebasev4:
* Drop the original patch #1. Instead, we use
acpi_rmrr_unitsto get rmrr info directly.
* Refine the hypercall definition to make sure we can use it safely.
* Introduce introducenr_reserved_device_memory_mapinhvm_info_table, then we can avoid issue unnecessary hypercall, even we can know current RMRR entries to issue hypercall one time.
* Cleanup and rebasev3:
* Use
XENMEM_reserved_device_memory_mapto replaceXENMEM_RMRR_memory_map
* Then rebase all patchesv2:
* Don't use e820map to define RMRR maps directly to avoid any confusion.
* In patch #3 we introduceconstruct_rmrr_e820_maps()to check if we can insert RMRR maps and then we will sort all e820 entries.
* Clean patch #4
* In patch #5 we reusecheck_mmio_hole()to check if current mmio range is fine to RMRR maps. If not, we just issue error to notify the user since mostly mmio should be configured again.While we work for supporting RMRR mapping for Windows GFX driver in case shared table,
http://osdir.com/ml/general/2014-07/msg55347.html
http://osdir.com/ml/general/2014-07/msg55348.htmlwe realize we should reserve RMRR range to avoid any potential MMIO/RAM overlap with our discussion so here these preliminary patches are intended to cover this.
----------------------------------------------------------------
Jan Beulich (1):
introduceXENMEM_reserved_device_memory_mapTiejun Chen (6):
tools/libxc: introduce hypercall forxc_reserved_device_memory_maptools/libxc: check if mmio BAR is out of reserved device memory maps libxc/hvm_info_table: introduce a new fieldnr_reserved_device_memory_map
hvmloader: introduce hypercall forxc_reserved_device_memory_map
hvmloader: check to reserved device memory maps in e820 xen/vtd: make USB RMRR mapping safetools/firmware/hvmloader/e820.c | 100 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++++++++++++++++++++ tools/firmware/hvmloader/util.c | 22 ++++++++++++++++++++++ tools/firmware/hvmloader/util.h | 3 +++ tools/libxc/xc_domain.c | 29 +++++++++++++++++++++++++++++ tools/libxc/xc_hvm_build_x86.c | 82 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++-- tools/libxc/xenctrl.h | 4 ++++ xen/common/compat/memory.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++++ xen/common/memory.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++++ xen/drivers/passthrough/iommu.c | 10 ++++++++++ xen/drivers/passthrough/vtd/dmar.c | 17 +++++++++++++++++ xen/drivers/passthrough/vtd/extern.h | 1 + xen/drivers/passthrough/vtd/iommu.c | 9 +-------- xen/include/public/hvm/hvm_info_table.h | 3 +++ xen/include/public/memory.h | 24 +++++++++++++++++++++++- xen/include/xen/iommu.h | 4 ++++ xen/include/xlat.lst | 3 ++- 16 files changed, 400 insertions(+), 12 deletions(-)Thanks
Tiejun
Quoted 6 lines · Kevin
Quoted 3 lines
From: Chen, Tiejun
Sent: Tuesday, September 09, 2014 10:50 PMcurrently the confliction is detected absolutely. Do we need a way to allow the confliction if there is no device assigned at all?
How to handle a hot-plug case when guest already boot? I think it may not be worth distinguishing such fine gain, things will be becoming complicated.
Thanks
Tiejun
Quoted 102 lines
Quoted 101 lines
v6:
* Jan cleanup to regenerate one patch to replace the two original patches then rebase the others.
* Refine all patches short/long logs.
* Replace those info with 'reserved device memory maps' and s/RMRR/RDM.
* Fix some code styles.
* Fix one interation error when we grab e820 info
This is not valid when j == nr - 1 (last iteration).
* Cleanup some codes.
* One comment to introduce a new field,nr_reserved_device_memory_maplibxc/hvm_info_table, is still pending to wait the tools maintainer's further comment.v5:
* Add patch #2 to introduce a global count,
acpi_rmrr_unit_entries.
* Refine hypercall return value to make sure the caller can distinguish clearly between "xen filled in N entries" and "xen said you need N entries for all information".
* Refine some structures
* Then Rebasev4:
* Drop the original patch #1. Instead, we use
acpi_rmrr_unitsto get rmrr info directly.
* Refine the hypercall definition to make sure we can use it safely.
* Introduce introducenr_reserved_device_memory_mapinhvm_info_table, then we can avoid issue unnecessary hypercall, even we can know current RMRR entries to issue hypercall one time.
* Cleanup and rebasev3:
* Use
XENMEM_reserved_device_memory_mapto replaceXENMEM_RMRR_memory_map
* Then rebase all patchesv2:
* Don't use e820map to define RMRR maps directly to avoid any confusion.
* In patch #3 we introduceconstruct_rmrr_e820_maps()to check if we can insert RMRR maps and then we will sort all e820 entries.
* Clean patch #4
* In patch #5 we reusecheck_mmio_hole()to check if current mmio range is fine to RMRR maps. If not, we just issue error to notify the user since mostly mmio should be configured again.While we work for supporting RMRR mapping for Windows GFX driver in case shared table,
http://osdir.com/ml/general/2014-07/msg55347.html
http://osdir.com/ml/general/2014-07/msg55348.htmlwe realize we should reserve RMRR range to avoid any potential MMIO/RAM overlap with our discussion so here these preliminary patches are intended to cover this.
----------------------------------------------------------------
Jan Beulich (1):
introduceXENMEM_reserved_device_memory_mapTiejun Chen (6):
tools/libxc: introduce hypercall forxc_reserved_device_memory_maptools/libxc: check if mmio BAR is out of reserved device memory maps libxc/hvm_info_table: introduce a new fieldnr_reserved_device_memory_map
hvmloader: introduce hypercall forxc_reserved_device_memory_map
hvmloader: check to reserved device memory maps in e820 xen/vtd: make USB RMRR mapping safetools/firmware/hvmloader/e820.c | 100 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++++++++++++++++++++ tools/firmware/hvmloader/util.c | 22 ++++++++++++++++++++++ tools/firmware/hvmloader/util.h | 3 +++ tools/libxc/xc_domain.c | 29 +++++++++++++++++++++++++++++ tools/libxc/xc_hvm_build_x86.c | 82 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++-- tools/libxc/xenctrl.h | 4 ++++ xen/common/compat/memory.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++++ xen/common/memory.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++++ xen/drivers/passthrough/iommu.c | 10 ++++++++++ xen/drivers/passthrough/vtd/dmar.c | 17 +++++++++++++++++ xen/drivers/passthrough/vtd/extern.h | 1 + xen/drivers/passthrough/vtd/iommu.c | 9 +-------- xen/include/public/hvm/hvm_info_table.h | 3 +++ xen/include/public/memory.h | 24 +++++++++++++++++++++++- xen/include/xen/iommu.h | 4 ++++ xen/include/xlat.lst | 3 ++- 16 files changed, 400 insertions(+), 12 deletions(-)Thanks
Tiejun
Quoted 6 lines
Quoted 1 line
Quoted 1 line · On 11.09.14 at 03:38, <[email protected]> wrote:
Quoted 4 lines · Kevin
Quoted 3 lines
From: Chen, Tiejun
Sent: Tuesday, September 09, 2014 10:50 PM
Quoted 6 lines
Quoted 2 lines
currently the confliction is detected absolutely. Do we need a way to allow the confliction if there is no device assigned at all?
How to handle a hot-plug case when guest already boot? I think it may not be worth distinguishing such fine gain, things will be becoming complicated.
In that case hotplug should fail. I'm very much in agreement with Kevin that an override should be possible if there's any risk of the detection done now could cause problems on certain systems (as pointed out before, I'm mainly concerned about RMRRs being defined in regions that overlap where the guest's BIOS wants to be placed).
Jan
Quoted 13 lines · Jan Beulich
Quoted 6 lines
Quoted 1 line
Quoted 1 line · On 11.09.14 at 03:38, <[email protected]> wrote:
Quoted 4 lines · Kevin
Quoted 3 lines
From: Chen, Tiejun
Sent: Tuesday, September 09, 2014 10:50 PMQuoted 6 lines
Quoted 2 lines
currently the confliction is detected absolutely. Do we need a way to allow the confliction if there is no device assigned at all?
How to handle a hot-plug case when guest already boot? I think it may not be worth distinguishing such fine gain, things will be becoming complicated.
Quoted 7 lines
In that case hotplug should fail. I'm very much in agreement with Kevin that an override should be possible if there's any risk of the detection done now could cause problems on certain systems (as pointed out before, I'm mainly concerned about RMRRs being defined in regions that overlap where the guest's BIOS wants to be placed).
Looks I have to refactor most codes again :)
I just take a quick look at Xen codes to try figuring out this possible way. As I see, either we do assign a device when create VM, or we do attach a device to a running VM, eventually we always call libxl__device_pci_add() to finish this operation. Furthermore, xc_test_assign_device() call such a hypercall, XEN_DOMCTL_test_assign_device. Right?
If yes, I guess I can do check if RMRR is overlapping something in two cases:
#1: In case of creating a VM, we may use global flag to make sure our original codes work indeed. It should be easy and especially I guess we can't check any conflict directly here since those memory info, RAM/MMIO/others, don't be filled completely, right?
#2: In case of a running VM, we can check if any overlapping exist then determine what's next. Often we should drop to attach such a conflicting device.
I'm not sure if I understand this path in Xen, so if I'm wrong please correct me.
Thanks
Tiejun
Quoted 5 lines
Quoted 1 line
Quoted 1 line · On 11.09.14 at 11:39, <[email protected]> wrote:
#1: In case of creating a VM, we may use global flag to make sure our original codes work indeed. It should be easy and especially I guess we can't check any conflict directly here since those memory info, RAM/MMIO/others, don't be filled completely, right?
This part will probably need fully dealing with in the tools, with the hypervisor only refusing to set up conflicting P2M entries (as iirc you're already doing).
Quoted 3 lines
#2: In case of a running VM, we can check if any overlapping exist then determine what's next. Often we should drop to attach such a conflicting device.
This one I would guess would benefit from the tools doing the checking if they can, but perhaps the hypervisor is in a better position to do this. If I'm right with this, at the tools level it would need to be made certain that this doesn't end up in a cryptic and/or well hidden error messages, leaving the operator to guess what went wrong.
Jan