Xen Q35 Bringup patches + support for PCIe Extended Capabilities for passed through devices
Re: [Qemu-devel] [RFC PATCH 16/30] q35/xen: Add Xen platform device support for Q35
In reply to Alexey GerasimenkoQuoted 10 lines · Alexey Gerasimenko
Current Xen/QEMU method to control Xen Platform device on i440 is a bit odd -- enabling/disabling Xen platform device actually modifies the QEMU emulated machine type, namely xenfv <--> pc.
In order to avoid multiplying machine types, use a new way to control Xen Platform device for QEMU -- "xen-platform-dev" machine property (bool). To maintain backward compatibility with existing Xen/QEMU setups, this is only applicable to q35 machine currently. i440 emulation still uses the old method (i.e. xenfv/pc machine selection) to control Xen Platform device, this may be changed later to xen-platform-dev property as well.
The change you made to q35 is pretty tiny, so I imagine the equiv change to pc machine is equally small. IOW, I think you should just convert them both straight away rather than providing an inconsistent configuration approach for q35 vs pc.
Quoted 110 lines
This way we can use a single machine type (q35) and change just xen-platform-dev value to on/off to control Xen platform device.
Signed-off-by Alexey Gerasimenko <[email protected]>--- hw/core/machine.c | 21 +++++++++++++++++++++ hw/i386/pc_q35.c | 14 ++++++++++++++ include/hw/boards.h | 1 + qemu-options.hx | 1 + 4 files changed, 37 insertions(+)hw/core/machine.c +21
-0@@ -290,6 +290,20 @@ static void machine_set_igd_gfx_passthru(Object *obj, bool value, Error **errp) ms->igd_gfx_passthru = value; } +static bool machine_get_xen_platform_dev (Object *obj,Error **errp) +{ +MachineState *ms =MACHINE (obj); + +return ms->xen_platform_dev; +} + +static void machine_set_xen_platform_dev (Object *obj,bool value,Error **errp) +{ +MachineState *ms =MACHINE (obj); + + ms->xen_platform_dev = value; +} +static char *machine_get_firmware (Object *obj,Error **errp) {MachineState *ms =MACHINE (obj);@@ -595,6 +609,13 @@ static void machine_class_init(ObjectClass *oc, void *data)object_class_property_set_description (oc,"igd-passthru" ,"Set on/off to enable/disable igd passthrou" , &error_abort); +object_class_property_add_bool (oc,"xen-platform-dev" , + machine_get_xen_platform_dev, + machine_set_xen_platform_dev, &error_abort); +object_class_property_set_description (oc,"xen-platform-dev" , +"Set on/off to enable/disable Xen Platform device" , + &error_abort); +object_class_property_add_str (oc,"firmware" , machine_get_firmware, machine_set_firmware, &error_abort);hw/i386/pc_q35.c +14
-0@@ -56,6 +56,18 @@/* ICH9 AHCI has 6 ports */ #define MAX_SATA_PORTS 6 +static void q35_xen_hvm_init (MachineState *machine) +{ +PCMachineState *pcms =PC_MACHINE (machine); + +if (xen_enabled ()) { +/* check if Xen Platform device is enabled */ +if (machine->xen_platform_dev) { +pci_create_simple (pcms->bus,-1 ,"xen-platform" ); + } + } +} +/* PC hardware initialisation */ static void pc_q35_init (MachineState *machine) {@@ -207,6 +219,8 @@ static void pc_q35_init(MachineState *machine)if (xen_enabled ()) {pci_bus_irqs (host_bus, xen_cmn_set_irq, xen_cmn_pci_slot_get_pirq, ich9_lpc,ICH9_XEN_NUM_IRQ_SOURCES ); + +q35_xen_hvm_init (machine); }else {pci_bus_irqs (host_bus, ich9_lpc_set_irq, ich9_lpc_map_irq, ich9_lpc,ICH9_LPC_NB_PIRQS );include/hw/boards.h +1
-0@@ -238,6 +238,7 @@ struct MachineState {bool usb;bool usb_disabled;bool igd_gfx_passthru; +bool xen_platform_dev;char *firmware;bool iommu;bool suppress_vmdesc;qemu-options.hx +1
-0@@ -38,6 +38,7 @@ DEF("machine", HAS_ARG, QEMU_OPTION_machine, \ " dump-guest-core=on|off include guest memory in a core dump (default=on)\n" " mem-merge=on|off controls memory merge support (default: on)\n" " igd-passthru=on|off controls IGD GFX passthrough support (default=off)\n" + " xen-platform-dev=on|off controls Xen Platform device (default=off)\n" " aes-key-wrap=on|off controls support for AES key wrapping (default=on)\n" " dea-key-wrap=on|off controls support for DEA key wrapping (default=on)\n" " suppress-vmdesc=on|off disables self-describing migration (default=off)\n"Signature · 1 line
2.11.0
Regards,
Daniel
Signature · 8 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
RFC PATCH 17/30 q35: Fix incorrect values for PCIEXBAR masks
In reply to Alexey GerasimenkoThere are two small issues in PCIEXBAR address mask handling:
- wrong bit positions for address mask bits (see PCIEXBAR description in Q35 datasheet)
- incorrect usage of 64ADR_MASK
Due to this, attempting to write a valid PCIEXBAR address may cause it to shift to another address, causing memory layout corruption where emulated MMIO regions may overlap real (passed through) MMIO ranges. Fix this by providing correct values.
--- hw/pci-host/q35.c | 6 +++--- include/hw/pci-host/q35.h | 4 ++-- 2 files changed, 5 insertions(+), 5 deletions(-)
hw/pci-host/q35.c +3 -3
@@ -322,12 +322,12 @@ static void mch_update_pciexbar(MCHPCIState *mch)break ;case MCH_HOST_BRIDGE_PCIEXBAR_LENGTH_128M : length =128 *1024 *1024 ; - addr_mask |=MCH_HOST_BRIDGE_PCIEXBAR_128ADMSK | -MCH_HOST_BRIDGE_PCIEXBAR_64ADMSK ; + addr_mask |=MCH_HOST_BRIDGE_PCIEXBAR_128ADMSK ;break ;case MCH_HOST_BRIDGE_PCIEXBAR_LENGTH_64M : length =64 *1024 *1024 ; - addr_mask |=MCH_HOST_BRIDGE_PCIEXBAR_64ADMSK ; + addr_mask |=MCH_HOST_BRIDGE_PCIEXBAR_64ADMSK | +MCH_HOST_BRIDGE_PCIEXBAR_128ADMSK ;break ;case MCH_HOST_BRIDGE_PCIEXBAR_LENGTH_RVD :default :
include/hw/pci-host/q35.h +2 -2
@@ -103,8 +103,8 @@ typedef struct Q35PCIHost {#define MCH_HOST_BRIDGE_PCIEXBAR_DEFAULT 0xb0000000#define MCH_HOST_BRIDGE_PCIEXBAR_MAX (0x10000000)/* 256M */ #define MCH_HOST_BRIDGE_PCIEXBAR_ADMSK Q35_MASK(64, 35, 28) -#define MCH_HOST_BRIDGE_PCIEXBAR_128ADMSK ((uint64_t)(1 << 26)) -#define MCH_HOST_BRIDGE_PCIEXBAR_64ADMSK ((uint64_t)(1 << 25)) +#define MCH_HOST_BRIDGE_PCIEXBAR_128ADMSK ((uint64_t)(1 << 27)) +#define MCH_HOST_BRIDGE_PCIEXBAR_64ADMSK ((uint64_t)(1 << 26))#define MCH_HOST_BRIDGE_PCIEXBAR_LENGTH_MASK ((uint64_t)(0x3 << 1))#define MCH_HOST_BRIDGE_PCIEXBAR_LENGTH_256M ((uint64_t)(0x0 << 1))#define MCH_HOST_BRIDGE_PCIEXBAR_LENGTH_128M ((uint64_t)(0x1 << 1))
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
RFC PATCH 18/30 xen/pt: XenHostPCIDevice: provide functions for PCI Capabilities and PCIe Extended Capabilities enumeration
In reply to Alexey GerasimenkoThis patch introduces 2 new functions,
- xen_host_pci_find_next_ext_cap (actually a reworked xen_host_pci_find_ext_cap_offset function which is unused)
- xen_host_pci_find_next_cap
These functions allow to search for PCI/PCIe capabilities in a uniform way. Both functions allow to search either a specific capability or any encountered next (by specifying CAP_ID_ANY as a capability ID) -- this may be useful when we merely need to traverse the capability list one-by-one. In both functions the 'pos' argument allows to continue searching from last position (0 means to start from beginning).
In order not to probe PCIe Extended Capabilities existence every time, xen_host_pci_find_next_ext_cap makes use of the new 'has_pcie_ext_caps' field in XenHostPCIDevice structure which is filled only once (in xen_host_pci_device_get).
--- hw/xen/xen-host-pci-device.c | 95 +++++++++++++++++++++++++++++++++++++------- hw/xen/xen-host-pci-device.h | 5 ++- 2 files changed, 85 insertions(+), 15 deletions(-)
hw/xen/xen-host-pci-device.c +81 -14
@@ -14,6 +14,7 @@#define XEN_HOST_PCI_MAX_EXT_CAP \ ((PCIE_CONFIG_SPACE_SIZE - PCI_CONFIG_SPACE_SIZE) / (PCI_CAP_SIZEOF + 4)) +#define XEN_HOST_PCI_CAP_MAX 48#ifdef XEN_HOST_PCI_DEVICE_DEBUG # define XEN_HOST_PCI_LOG(f, a...) fprintf(stderr, "%s: " f, __func__, ##a)
@@ -199,6 +200,19 @@ static bool xen_host_pci_dev_is_virtfn(XenHostPCIDevice *d)return !stat (path, &buf); } +static bool xen_host_pci_dev_has_pcie_ext_caps (XenHostPCIDevice *d) +{ +uint32_t header; + +if (xen_host_pci_get_long (d,PCI_CONFIG_SPACE_SIZE , &header)) +return false; + +if (header ==0 || header == ~0U ) +return false; + +return true; +} +static void xen_host_pci_config_open (XenHostPCIDevice *d,Error **errp) {char path[PATH_MAX ];
@@ -297,37 +311,89 @@ int xen_host_pci_set_block(XenHostPCIDevice *d, int pos, uint8_t *buf, int len)return xen_host_pci_config_write (d, pos, buf, len); } -int xen_host_pci_find_ext_cap_offset (XenHostPCIDevice *d,uint32_t cap) +int xen_host_pci_find_next_ext_cap (XenHostPCIDevice *d,int pos,uint32_t cap) {uint32_t header =0 ;int max_cap =XEN_HOST_PCI_MAX_EXT_CAP ; -int pos =PCI_CONFIG_SPACE_SIZE ; + +if (!d->has_pcie_ext_caps) +return 0 ; + +if (!pos) { + pos =PCI_CONFIG_SPACE_SIZE ; + }else { +if (xen_host_pci_get_long (d, pos, &header)) +return 0 ; + + pos =PCI_EXT_CAP_NEXT (header); + }do { -if (xen_host_pci_get_long (d, pos, &header)) { +if (!pos || pos <PCI_CONFIG_SPACE_SIZE ) +break ; + +if (xen_host_pci_get_long (d, pos, &header))break ; - }/* * If we have no capabilities, this is indicated by cap ID, * cap version and next pointer all being 0. +* Also check for all F's returned (which means PCIe ext conf space +* is unreadable for some reason) */ -if (header ==0 ) { +if (header ==0 || header == ~0U )break ; - } -if (PCI_EXT_CAP_ID (header) == cap) { +if (cap ==CAP_ID_ANY ) +return pos; +else if (PCI_EXT_CAP_ID (header) == cap)return pos; - } pos =PCI_EXT_CAP_NEXT (header); -if (pos <PCI_CONFIG_SPACE_SIZE ) { + }while (--max_cap); + +return 0 ; +} + +int xen_host_pci_find_next_cap (XenHostPCIDevice *d,int pos,uint32_t cap) +{ +uint8_t id; +unsigned max_cap =XEN_HOST_PCI_CAP_MAX ; +uint8_t status =0 ; +uint8_t curpos; + +if (xen_host_pci_get_byte (d,PCI_STATUS , &status)) +return 0 ; + +if ((status &PCI_STATUS_CAP_LIST ) ==0 ) +return 0 ; + +if (pos <PCI_CAPABILITY_LIST ) { + curpos =PCI_CAPABILITY_LIST ; + }else { + curpos = (uint8_t ) pos; + } + +while (max_cap--) { +if (xen_host_pci_get_byte (d, curpos, &curpos)) +break ; +if (!curpos)break ; - } - max_cap--; - }while (max_cap >0 ); +if (cap ==CAP_ID_ANY ) +return curpos; -return -1 ; +if (xen_host_pci_get_byte (d, curpos +PCI_CAP_LIST_ID , &id)) +break ; + +if (id ==0xff ) +break ; +else if (id == cap) +return curpos; + + curpos +=PCI_CAP_LIST_NEXT ; + } + +return 0 ; } void xen_host_pci_device_get(XenHostPCIDevice *d, uint16_t domain,
@@ -377,7 +443,8 @@ void xen_host_pci_device_get(XenHostPCIDevice *d, uint16_t domain, } d->class_code = v; - d->is_virtfn =xen_host_pci_dev_is_virtfn (d); + d->is_virtfn =xen_host_pci_dev_is_virtfn (d); + d->has_pcie_ext_caps =xen_host_pci_dev_has_pcie_ext_caps (d);return ;
hw/xen/xen-host-pci-device.h +4 -1
@@ -32,6 +32,7 @@ typedef struct XenHostPCIDevice {XenHostPCIIORegion rom;bool is_virtfn; +bool has_pcie_ext_caps;int config_fd; } XenHostPCIDevice;
@@ -53,6 +54,8 @@ int xen_host_pci_set_long(XenHostPCIDevice *d, int pos, uint32_t data);int xen_host_pci_set_block (XenHostPCIDevice *d,int pos,uint8_t *buf,int len); -int xen_host_pci_find_ext_cap_offset (XenHostPCIDevice *s,uint32_t cap); +#define CAP_ID_ANY (~0U) +int xen_host_pci_find_next_cap (XenHostPCIDevice *s,int pos,uint32_t cap); +int xen_host_pci_find_next_ext_cap (XenHostPCIDevice *d,int pos,uint32_t cap);#endif /* XEN_HOST_PCI_DEVICE_H */
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
RFC PATCH 19/30 xen/pt: avoid reading PCIe device type and cap version multiple times
In reply to Alexey Gerasimenkoxen_pt_config_init.c reads Device/Port Type and Capability version fields in many places. Two functions are used for this purpose:get_capability_version and get_device_type. These functions perform PCI conf space reading every time they're called. Another bad thing is that these functions know nothing about where PCI Expess Capability is located, so its offset must be provided explicitly in function arguments. Their typical usage is like this:
uint8_t cap_ver = get_capability_version(s, real_offset - reg->offset);
uint8_t dev_type = get_device_type(s, real_offset - reg->offset);To avoid this, the PCI Express Capability register now being read only once and stored in XenHostPCIDevice structure (pcie_flags field). The capabiliy offset parameter is no longer needed, simplifying functions usage. Also, get_device_type and get_capability_version were renamed to more descriptive get_pcie_device_type and get_pcie_capability_version.
--- hw/xen/xen-host-pci-device.c | 15 +++++++++++++++ hw/xen/xen-host-pci-device.h | 1 + hw/xen/xen_pt_config_init.c | 34 ++++++++++++++-------------------- 3 files changed, 30 insertions(+), 20 deletions(-)
hw/xen/xen-host-pci-device.c +15 -0
@@ -402,6 +402,7 @@ void xen_host_pci_device_get(XenHostPCIDevice *d, uint16_t domain, {unsigned int v;Error *err =NULL ; +int pcie_cap_pos; d->config_fd =-1 ; d->domain = domain;
@@ -446,6 +447,20 @@ void xen_host_pci_device_get(XenHostPCIDevice *d, uint16_t domain, d->is_virtfn =xen_host_pci_dev_is_virtfn (d); d->has_pcie_ext_caps =xen_host_pci_dev_has_pcie_ext_caps (d); +/* read and store PCIe Capabilities field for later use */ + pcie_cap_pos =xen_host_pci_find_next_cap (d,0 ,PCI_CAP_ID_EXP ); + +if (pcie_cap_pos) { +if (xen_host_pci_get_word (d, pcie_cap_pos +PCI_EXP_FLAGS , + &d->pcie_flags)) { +error_setg (&err,"Unable to read from PCI Express capability " +"structure at 0x%x" , pcie_cap_pos); + goto error; + } + }else { + d->pcie_flags =0xFFFF ; + } +return ; error:
hw/xen/xen-host-pci-device.h +1 -0
@@ -27,6 +27,7 @@ typedef struct XenHostPCIDevice {uint16_t device_id;uint32_t class_code;int irq; +uint16_t pcie_flags;XenHostPCIIORegion io_regions[PCI_NUM_REGIONS -1 ];XenHostPCIIORegion rom;
hw/xen/xen_pt_config_init.c +14 -20
@@ -828,24 +828,18 @@ static XenPTRegInfo xen_pt_emu_reg_vendor[] = {* PCI Express Capability */ -static inline uint8_t get_capability_version (XenPCIPassthroughState *s, -uint32_t offset) +static inline uint8_t get_pcie_capability_version (XenPCIPassthroughState *s) { -uint8_t flag; -if (xen_host_pci_get_byte (&s->real_device, offset +PCI_EXP_FLAGS , &flag)) { -return 0 ; - } -return flag &PCI_EXP_FLAGS_VERS ; +assert (s->real_device.pcie_flags !=0xFFFF ); + +return (uint8_t ) (s->real_device.pcie_flags &PCI_EXP_FLAGS_VERS ); } -static inline uint8_t get_device_type (XenPCIPassthroughState *s, -uint32_t offset) +static inline uint8_t get_pcie_device_type (XenPCIPassthroughState *s) { -uint8_t flag; -if (xen_host_pci_get_byte (&s->real_device, offset +PCI_EXP_FLAGS , &flag)) { -return 0 ; - } -return (flag &PCI_EXP_FLAGS_TYPE ) >>4 ; +assert (s->real_device.pcie_flags !=0xFFFF ); + +return (uint8_t ) ((s->real_device.pcie_flags &PCI_EXP_FLAGS_TYPE ) >>4 ); }/* initialize Link Control register */
@@ -853,8 +847,8 @@ static int xen_pt_linkctrl_reg_init(XenPCIPassthroughState *s, XenPTRegInfo *reg, uint32_t real_offset, uint32_t *data) { - uint8_t cap_ver = get_capability_version(s, real_offset - reg->offset); -uint8_t dev_type =get_device_type (s, real_offset - reg->offset); + uint8_t cap_ver = get_pcie_capability_version(s); +uint8_t dev_type =get_pcie_device_type (s);/* no need to initialize in case of Root Complex Integrated Endpoint * with cap_ver 1.x
@@ -871,7 +865,7 @@ static int xen_pt_devctrl2_reg_init(XenPCIPassthroughState *s, XenPTRegInfo *reg, uint32_t real_offset, uint32_t *data) { - uint8_t cap_ver = get_capability_version(s, real_offset - reg->offset); + uint8_t cap_ver = get_pcie_capability_version(s);/* no need to initialize in case of cap_ver 1.x */ if (cap_ver ==1 ) {
@@ -886,7 +880,7 @@ static int xen_pt_linkctrl2_reg_init(XenPCIPassthroughState *s, XenPTRegInfo *reg, uint32_t real_offset, uint32_t *data) { - uint8_t cap_ver = get_capability_version(s, real_offset - reg->offset); + uint8_t cap_ver = get_pcie_capability_version(s);uint32_t reg_field =0 ;/* no need to initialize in case of cap_ver 1.x */
@@ -1586,8 +1580,8 @@ static int xen_pt_pcie_size_init(XenPCIPassthroughState *s, uint32_t base_offset, uint8_t *size) { PCIDevice *d = &s->dev; -uint8_t version =get_capability_version (s, base_offset); -uint8_t type =get_device_type (s, base_offset); +uint8_t version =get_pcie_capability_version (s); +uint8_t type =get_pcie_device_type (s);uint8_t pcie_size =0 ;
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
RFC PATCH 20/30 xen/pt: determine the legacy/PCIe mode for a passed through device
In reply to Alexey GerasimenkoEven if we have some real PCIe device being passed through to a guest, there are situations when we cannot use its PCIe features, primarily allowing to access extended (>256) config space.
Basically, we can allow reading PCIe extended config space only if both the device and emulated system are PCIe-capable. So it's a combination of checks:
- PCI Express capability presence
- pci_is_express(device)
- pci_bus_is_express(device bus)
The AND-product of these checks is stored to pcie_enabled_dev flag in XenPCIPassthroughState for later use in functions like xen_pt_pci_config_access_check.
This way we get consistent behavior when the same PCIe device being passed through to either i440 domain or Q35 one.
--- hw/xen/xen_pt.c | 28 ++++++++++++++++++++++++++-- hw/xen/xen_pt.h | 1 + 2 files changed, 27 insertions(+), 2 deletions(-)
hw/xen/xen_pt.c +26 -2
@@ -687,6 +687,21 @@ static const MemoryListener xen_pt_io_listener = { .priority = 10, }; +static inline bool xen_pt_dev_is_pcie_mode (PCIDevice *d) +{ +XenPCIPassthroughState *s =XEN_PT_DEVICE (d); +PCIBus *bus =pci_get_bus (d); + +if (bus !=NULL ) { +if (pci_is_express (d) &&pci_bus_is_express (bus) && +xen_host_pci_find_next_cap (&s->real_device,0 ,PCI_CAP_ID_EXP )) { +return true; + } + } + +return false; +} +static void xen_igd_passthrough_isa_bridge_create (XenPCIPassthroughState *s,XenHostPCIDevice *dev)
@@ -794,8 +809,17 @@ static void xen_pt_realize(PCIDevice *d, Error **errp) s->real_device.dev, s->real_device.func); } -/* Initialize virtualized PCI configuration (Extended 256 Bytes) */ -memset (d->config,0 ,PCI_CONFIG_SPACE_SIZE ); + s->pcie_enabled_dev =xen_pt_dev_is_pcie_mode (d); +if (s->pcie_enabled_dev) { +XEN_PT_LOG (d,"Host device %04x:%02x:%02x.%d passed thru " +"in PCIe mode\n" , s->real_device.domain, + s->real_device.bus, s->real_device.dev, + s->real_device.func); + } + +/* Initialize virtualized PCI configuration space (256/4K bytes) */ +memset (d->config,0 ,pci_is_express (d) ?PCIE_CONFIG_SPACE_SIZE + :PCI_CONFIG_SPACE_SIZE ); s->memory_listener = xen_pt_memory_listener; s->io_listener = xen_pt_io_listener;
hw/xen/xen_pt.h +1 -0
@@ -212,6 +212,7 @@ struct XenPCIPassthroughState {PCIHostDeviceAddress hostaddr;bool is_virtfn; +bool pcie_enabled_dev;bool permissive;bool permissive_warned;XenHostPCIDevice real_device;
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
RFC PATCH 21/30 xen/pt: Xen PCIe passthrough support for Q35: bypass PCIe topology check
In reply to Alexey GerasimenkoCompared to legacy i440 system, there are certain difficulties while passing through PCIe devices to guest OSes like Windows 7 and above on platforms with native support of PCIe bus (in our case Q35). This problem is not applicable to older OSes like Windows XP -- PCIe passthrough on such OSes can be used normally as these OSes have no support for PCIe-specific features and treat all PCIe devices as legacy PCI ones.
The problem manifests itself as "Code 10" error for a passed thru PCIe device in Windows Device Manager (along with exclamation mark on it). The device with such error do not function no matter the fact that Windows successfully booted while actually using this device, ex. as a primary VGA card with VBE features, LFB, etc. working properly during boot time. It doesn't matter which PCI class the device have -- the problem is common to GPUs, NIC cards, USB controllers, etc. In the same time, all these devices can be passed thru successfully using i440 emulation on same Windows 7+ OSes.
The actual root cause of the problem lies in the fact that Windows kernel (PnP manager particularly) while processing StartDevice IRP refuses to continue to start the device and control flow actually doesn't even reach the IRP handler in the device driver at all. The real reason for this typically does not appear at the time PnP manager tries to start the device, but happens much earlier -- during the Windows boot stage, while enumerating devices on a PCI/PCIe bus in the Windows pci.sys driver. There is a set of checks for every discovered device on the PCIe bus. Failing some of them leads to marking the discovered PCIe device as 'invalid' by setting the flag. Later on, StartDevice attempt will fail due to this flag, finally resulting in Code 10 error.
The actual check in pci.sys which results in the PCIe device being marked as 'invalid' in our case is a validation of upstream PCIe bus hierarchy to which passed through device belongs. Basically, pci.sys checks if the PCIe device has parent devices, such as PCIe Root Port or upstream PCIe switch. In our case the PCIe device has no parents and resides on bus 0 without eg. corresponding Root Port.
Therefore, in order to resolve this problem in a architecturally correct way, we need to introduce to Xen some support of at least trivial non-flat PCI bus hierarchy. In very simplest case - just one virtual Root Port, on secondary bus of which all physical functions of the real passed thru device will reside, eg. GPU and its HDAudio function.
This solution is not hard to implement technically, but there are multiple affecting limitations present in Xen (many related to each other) currently:
- in many places the code is limited to use bus 0 only. This applicable to both hypervisor and supplemental modules like hvmloader. This limitation is enforced on API level -- many functions and interfaces allow to specify only devfn argument while bus 0 being implied.
- lot of code assumes Type0 PCI config space layout only, while we need to handle Type1 PCI devices as well
- currently there no way to assign to a guest domain even a simplest linked hierarchy of passed thru PCI devices. In some cases we might need to passthrough a real PCIe Switch/Root Port with his downstream child devices.
- in a similar way Xen/hvmloader lacks the concept of IO/MMIO space nesting. Both code which does MMIO hole sizing and code which allocates BARs to MMIO hole have no idea of MMIO ranges nesting and their relations. In case of virtual Root Port we have basically an emulated PCI-PCI bridge with some parts of its MMIO range used for real MMIO ranges of passed through device(s).
So, adding to Xen multiple PCI buses support will require a bit of effort and discussions regarding the actual design of the feature. Nevertheless, this task is crucial for PCI/GPU passthrough features of Xen to work properly.
To summarize, we need to implement following things in the future:
1) Get rid of PCI bus 0 limitation everywhere. This could've been a simplest of subtasks but in reality this will require to change interfaces as well - AFAIR even adding a PCI device via QMP only allows to specify a device slot while we need to have some way to place the device on an arbitrary bus.
2) Fully or partially emulated PCI-PCI bridge which will provide a secondary bus for PCIe device placement - there might be a possibility to reuse some existing emulation QEMU provides. This also includes Type1 devices support.
The task will become more complicated if there arise necessity, for example, to control the PCIe link for a passed through PCIe device. As PT device reset is mandatory in most cases, there might be a chance to encounter a situation when we need to retrain the PCIe link to restore PCIe link speed after the reset. In this case there will be a need to selectively translate accesses to certain registers of emulated PCIe Switch/Root Port to the corresponding physical upstream PCIe Switch/RootPort. This will require some interaction with Dom0, hopefully extending xen-pciback will be enough.
3) The concept of I/O and MMIO ranges nesting, for tasks like sizing MMIO hole or PCI BAR allocation. This one should be pretty simple.
The actual implementation still is a matter to discuss of course.
In the meantime there can be used a very simple workaround which allows to bypass pci.sys limitation for PCIe topology check - there exist one good exception to "must have upstream PCIe parent" rule of pci.sys. It's chipset-integrated devices. How pci.sys can tell if it deals with a chipset built-in device? It checks one of PCI Express Capability fields in the device PCI conf space. For chipset built-in devices this field will state "root complex integrated device" while in our case for a normal passed thru PCIe device there will be a "PCIe endpoint" type. So that's what the workaround does - it intercepts reading of this particular field for passed through devices and returns the "root complex integrated device" value for PCIe endpoints. This makes pci.sys happy and allows Windows 7 and above to use PT device on PCIe-capable system normally. So far no negative side effects were encountered while using this approach, so it's a good temporary solution until multiple PCI bus support will be added to Xen.
--- hw/xen/xen_pt_config_init.c | 60 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+)
hw/xen/xen_pt_config_init.c +60 -0
@@ -902,6 +902,55 @@ static int xen_pt_linkctrl2_reg_init(XenPCIPassthroughState *s, *data = reg_field;return 0 ; } +/* initialize PCI Express Capabilities register */ +static int xen_pt_pcie_capabilities_reg_init (XenPCIPassthroughState *s, +XenPTRegInfo *reg, +uint32_t real_offset, +uint32_t *data) +{ +uint8_t dev_type =get_pcie_device_type (s); +uint16_t reg_field; + +if (xen_host_pci_get_word (&s->real_device, + real_offset - reg->offset +PCI_EXP_FLAGS , + ®_field)) { +XEN_PT_ERR (&s->dev,"Error reading PCIe Capabilities reg\n" ); + *data =0 ; +return 0 ; + } + +/* +* Q35 workaround for Win7+ pci.sys PCIe topology check. +* As our PT device currently located on a bus 0, fake the +* device/port type field to the "Root Complex integrated device" +* value to bypass the check +*/ +switch (dev_type) { +case PCI_EXP_TYPE_ENDPOINT : +case PCI_EXP_TYPE_LEG_END : +XEN_PT_LOG (&s->dev,"Original PCIe Capabilities reg is 0x%04X\n" , + reg_field); + reg_field &= ~PCI_EXP_FLAGS_TYPE ; + reg_field |= ((PCI_EXP_TYPE_RC_END /*9*/ <<4 ) &PCI_EXP_FLAGS_TYPE ); +XEN_PT_LOG (&s->dev,"Q35 PCIe topology check workaround: " +"faking Capabilities reg to 0x%04X\n" , reg_field); +break ; + +case PCI_EXP_TYPE_ROOT_PORT : +case PCI_EXP_TYPE_UPSTREAM : +case PCI_EXP_TYPE_DOWNSTREAM : +case PCI_EXP_TYPE_PCI_BRIDGE : +case PCI_EXP_TYPE_PCIE_BRIDGE : +case PCI_EXP_TYPE_RC_END : +case PCI_EXP_TYPE_RC_EC : +default : +/* do nothing, return as is */ +break ; + } + + *data = reg_field; +return 0 ; +}/* PCI Express Capability Structure reg static information table */ static XenPTRegInfo xen_pt_emu_reg_pcie[] = {
@@ -916,6 +965,17 @@ static XenPTRegInfo xen_pt_emu_reg_pcie[] = { .u.b.read = xen_pt_byte_reg_read, .u.b.write = xen_pt_byte_reg_write, }, +/* PCI Express Capabilities Register */ + { + .offset = PCI_EXP_FLAGS, + .size = 2, + .init_val = 0x0000, + .ro_mask = 0xFFFF, + .emu_mask = 0xFFFF, + .init = xen_pt_pcie_capabilities_reg_init, + .u.w.read = xen_pt_word_reg_read, + .u.w.write = xen_pt_word_reg_write, + }, /* Device Capabilities reg */ { .offset = PCI_EXP_DEVCAP,
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
RFC PATCH 22/30 xen/pt: add support for PCIe Extended Capabilities and larger config space
In reply to Alexey GerasimenkoThis patch provides basic facilities for PCIe Extended Capabilities and support for controlled (via s->pcie_enabled_dev flag) access to PCIe config space (>256).
PCIe Extended Capabilities make use of 16-bit capability ID. Also, a capability size might exceed 8-bit width. So as the very first step we need to increase type size for grp_id, grp_size, etc -- they were limited to 8-bit.
The only troublesome issue with PCIe Extended Capability IDs is that their value range is actually same as for basic PCI capabilities. Eg. capability ID 3 means VPD Capability for PCI and at the same time Device Serial Number Capability for PCIe Extended caps. This adds a bit of inconvenience.
In order to distinguish between two sets of same capability IDs, the patch introduces a set of macros to mark a capability ID as PCIe Extended one (or check if it is basic/extended + get a raw ID value):
- PCIE_EXT_CAP_ID(cap_id)
- IS_PCIE_EXT_CAP_ID(grp_id)
- GET_PCIE_EXT_CAP_ID(grp_id)
Here is how it's used:
/* Intel IGD Opregion group */
{
.grp_id = XEN_PCI_INTEL_OPREGION, /* no change */
.grp_type = XEN_PT_GRP_TYPE_EMU,
.grp_size = 0x4,
.size_init = xen_pt_reg_grp_size_init,
.emu_regs = xen_pt_emu_reg_igd_opregion,
},
/* Vendor-specific Extended Capability reg group */
{
.grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_VNDR),
.grp_type = XEN_PT_GRP_TYPE_EMU,
.grp_size = 0xFF,
.size_init = xen_pt_ext_cap_vendor_size_init,
.emu_regs = xen_pt_ext_cap_emu_reg_vendor,
},
By using the PCIE_EXT_CAP_ID() macro it is possible to reuse existing header files with already defined PCIe Extended Capability ID values.
find_cap_offset() receive capabily ID and checks if it's an Extended one by using IS_PCIE_EXT_CAP_ID(cap) macro, passing the real capabiliy ID value to either xen_host_pci_find_next_ext_cap
or xen_host_pci_find_next_cap.
--- hw/xen/xen_pt.c | 14 +++++- hw/xen/xen_pt.h | 13 +++-- hw/xen/xen_pt_config_init.c | 113 +++++++++++++++++++++----------------------- 3 files changed, 74 insertions(+), 66 deletions(-)
hw/xen/xen_pt.c +12 -2
@@ -82,10 +82,20 @@ void xen_pt_log(const PCIDevice *d, const char *f, ...)/* Config Space */ -static int xen_pt_pci_config_access_check (PCIDevice *d,uint32_t addr,int len) +static int xen_pt_pci_config_access_check (PCIDevice *d, +uint32_t addr,int len) { +XenPCIPassthroughState *s =XEN_PT_DEVICE (d); +/* check offset range */ -if (addr >0xFF ) { +if (s->pcie_enabled_dev) { +if (addr >=PCIE_CONFIG_SPACE_SIZE ) { +XEN_PT_ERR (d,"Failed to access register with offset " +"exceeding 0xFFF. (addr: 0x%02x, len: %d)\n" , + addr, len); +return -1 ; + } + }else if (addr >=PCI_CONFIG_SPACE_SIZE ) {XEN_PT_ERR (d,"Failed to access register with offset exceeding 0xFF. " "(addr: 0x%02x, len: %d)\n" , addr, len);return -1 ;
hw/xen/xen_pt.h +9 -4
@@ -31,6 +31,11 @@ void xen_pt_log(const PCIDevice *d, const char *f, ...) GCC_FMT_ATTR(2, 3);/* Helper */ #define XEN_PFN (x) ((x) >> XC_PAGE_SHIFT) +/* Macro's for PCIe Extended Capabilities */ +#define PCIE_EXT_CAP_ID (cap_id) ((cap_id) | (1U << 16)) +#define IS_PCIE_EXT_CAP_ID (grp_id) ((grp_id) & (1U << 16)) +#define GET_PCIE_EXT_CAP_ID (grp_id) ((grp_id) & 0xFFFF) +typedef const struct XenPTRegInfo XenPTRegInfo ;typedef struct XenPTReg XenPTReg ;
@@ -152,13 +157,13 @@ typedef const struct XenPTRegGroupInfo XenPTRegGroupInfo;/* emul reg group size initialize method */ typedef int (*xen_pt_reg_size_init_fn ) (XenPCIPassthroughState *,XenPTRegGroupInfo *, -uint32_t base_offset,uint8_t *size); +uint32_t base_offset,uint32_t *size);/* emulated register group information */ struct XenPTRegGroupInfo { -uint8_t grp_id; +uint32_t grp_id;XenPTRegisterGroupType grp_type; -uint8_t grp_size; +uint32_t grp_size;xen_pt_reg_size_init_fn size_init;XenPTRegInfo *emu_regs; };
@@ -168,7 +173,7 @@ typedef struct XenPTRegGroup {QLIST_ENTRY (XenPTRegGroup) entries;XenPTRegGroupInfo *reg_grp;uint32_t base_offset; -uint8_t size; +uint32_t size; QLIST_HEAD(, XenPTReg) reg_tbl_list; } XenPTRegGroup;
hw/xen/xen_pt_config_init.c +53 -60
161 lines not on this page · raw message
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
RFC PATCH 23/30 xen/pt: handle PCIe Extended Capabilities Next register
In reply to Alexey GerasimenkoThe patch adds new xen_pt_ext_cap_ptr_reg_init function which is used to initialize the emulated next pcie extended capability pointer.
Primary mission of this function is to have a method to selectively hide some extended capabilities from the capability linked list, skipping them by altering the Next capability pointer value.
--- hw/xen/xen_pt_config_init.c | 73 +++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 71 insertions(+), 2 deletions(-)
hw/xen/xen_pt_config_init.c +71 -2
@@ -23,11 +23,14 @@#define XEN_PT_INVALID_REG 0xFFFFFFFF/* invalid register value */ -/* prototype */ +/* prototypes */ static int xen_pt_ptr_reg_init (XenPCIPassthroughState *s,XenPTRegInfo *reg,uint32_t real_offset,uint32_t *data); - +static int xen_pt_ext_cap_ptr_reg_init (XenPCIPassthroughState *s, +XenPTRegInfo *reg, +uint32_t real_offset, +uint32_t *data);/* helper */
@@ -1932,6 +1935,72 @@ out:return 0 ; } +#define PCIE_EXT_CAP_NEXT_SHIFT 4 +#define PCIE_EXT_CAP_VER_MASK 0xF + +static int xen_pt_ext_cap_ptr_reg_init (XenPCIPassthroughState *s, +XenPTRegInfo *reg, +uint32_t real_offset, +uint32_t *data) +{ +int i, rc; +XenHostPCIDevice *d = &s->real_device; +uint16_t reg_field; +uint16_t cur_offset, version, cap_id; +uint32_t header; + +if (real_offset <PCI_CONFIG_SPACE_SIZE ) { +XEN_PT_ERR (&s->dev,"Incorrect PCIe extended capability offset" +"encountered: 0x%04x\n" , real_offset); +return -EINVAL ; + } + + rc =xen_host_pci_get_word (d, real_offset, ®_field); +if (rc) +return rc; + +/* preserve version field */ + version = reg_field &PCIE_EXT_CAP_VER_MASK ; + cur_offset = reg_field >>PCIE_EXT_CAP_NEXT_SHIFT ; + +while (cur_offset && cur_offset !=0xFFF ) { + rc =xen_host_pci_get_long (d, cur_offset, &header); +if (rc) { +XEN_PT_ERR (&s->dev,"Failed to read PCIe extended capability " +"@0x%x (rc:%d)\n" , cur_offset, rc); +return rc; + } + + cap_id =PCI_EXT_CAP_ID (header); + +for (i =0 ; xen_pt_emu_reg_grps[i].grp_size !=0 ; i++) { +uint32_t cur_grp_id = xen_pt_emu_reg_grps[i].grp_id; + +if (!IS_PCIE_EXT_CAP_ID (cur_grp_id)) +continue ; + +if (xen_pt_hide_dev_cap (d, cur_grp_id)) +continue ; + +if (GET_PCIE_EXT_CAP_ID (cur_grp_id) == cap_id) { +if (xen_pt_emu_reg_grps[i].grp_type ==XEN_PT_GRP_TYPE_EMU ) + goto out; + +/* skip TYPE_HARDWIRED capability, move the ptr to next one */ +break ; + } + } + +/* next capability */ + cur_offset =PCI_EXT_CAP_NEXT (header); + } + +out: + *data = (cur_offset <<PCIE_EXT_CAP_NEXT_SHIFT ) | version; +return 0 ; +} + +/************* * Main
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
RFC PATCH 24/30 xen/pt: allow to hide PCIe Extended Capabilities
In reply to Alexey GerasimenkoWe need to hide some unwanted PCI/PCIe capabilities for passed through devices.
Normally we do this by marking the capability register group as XEN_PT_GRP_TYPE_HARDWIRED which exclude this capability from the capability list and returns zeroes on attempts to read capability body. Skipping the capability in the linked list of capabilities can be done by changing Next Capability register to skip one or many unwanted capabilities.
One difference between PCI and PCIe Extended capabilities is that we don't have the list head field anymore. PCIe Extended capabilities always start at offset 0x100 if they're present. Unfortunately, there are typically only few PCIe extended capabilities present which means there is a chance that some capability we want to hide will reside at offset 0x100 in PCIe config space.
The simplest way to hide such capabilities from guest OS or drivers is faking their capability ID value.
This patch adds the Capability ID register handler which checks
- if the capability to which this register belong starts at offset 0x100 in PCIe config space
- if this capability is marked as XEN_PT_GRP_TYPE_HARDWIRED
If it is the case, then a fake Capability ID value is returned.
--- hw/xen/xen_pt.c | 11 +++++++- hw/xen/xen_pt.h | 5 ++++ hw/xen/xen_pt_config_init.c | 62 ++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 76 insertions(+), 2 deletions(-)
hw/xen/xen_pt.c +10 -1
@@ -154,7 +154,16 @@ static uint32_t xen_pt_pci_read_config(PCIDevice *d, uint32_t addr, int len) reg_grp_entry =xen_pt_find_reg_grp (s, addr);if (reg_grp_entry) {/* check 0-Hardwired register group */ -if (reg_grp_entry->reg_grp->grp_type ==XEN_PT_GRP_TYPE_HARDWIRED ) { +if (reg_grp_entry->reg_grp->grp_type ==XEN_PT_GRP_TYPE_HARDWIRED && +/* +* For PCIe Extended Capabilities we need to emulate +* CapabilityID and NextCapability/Version registers for a +* hardwired reg group located at the offset 0x100 in PCIe +* config space. This allows us to hide the first extended +* capability as well. +*/ + !(reg_grp_entry->base_offset ==PCI_CONFIG_SPACE_SIZE && +ranges_overlap (addr, len,0x100 ,4 ))) {/* no need to emulate, just return 0 */ val =0 ; goto exit;
hw/xen/xen_pt.h +5 -0
@@ -78,6 +78,11 @@ typedef int (*xen_pt_conf_byte_read)#define XEN_PCI_INTEL_OPREGION 0xfc +#define XEN_PCIE_CAP_ID 0 +#define XEN_PCIE_CAP_LIST_NEXT 2 + +#define XEN_PCIE_FAKE_CAP_ID_BASE 0xFE00 + typedef enum { XEN_PT_GRP_TYPE_HARDWIRED = 0, /* 0 Hardwired reg group */ XEN_PT_GRP_TYPE_EMU, /* emul reg group */
hw/xen/xen_pt_config_init.c +61 -1
80 lines not on this page · raw message
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
RFC PATCH 25/30 xen/pt: add Vendor-specific PCIe Extended Capability descriptor and sizing
In reply to Alexey GerasimenkoThe patch provides Vendor-specific PCIe Extended Capability description structure and corresponding sizing function. In this particular case the size of the Vendor capability is available in the VSEC Length field.
--- hw/xen/xen_pt_config_init.c | 77 +++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 75 insertions(+), 2 deletions(-)
hw/xen/xen_pt_config_init.c +75 -2
@@ -129,6 +129,18 @@ static uint32_t get_throughable_mask(const XenPCIPassthroughState *s,return throughable_mask & valid_mask; } +static void log_pcie_extended_cap (XenPCIPassthroughState *s, +const char *cap_name, +uint32_t base_offset,uint32_t size) +{ +if (size) { +XEN_PT_LOG (&s->dev,"Found PCIe Extended Capability: %s at 0x%04x, " +"size 0x%x bytes\n" , cap_name, + (uint16_t ) base_offset, size); + } +} + +/**************** * general register functions */
@@ -1684,6 +1696,44 @@ static int xen_pt_ext_cap_capid_reg_init(XenPCIPassthroughState *s, } +/* Vendor-specific Ext Capability Structure reg static information table */ +static XenPTRegInfo xen_pt_ext_cap_emu_reg_vendor[] = { + { + .offset =XEN_PCIE_CAP_ID , + .size =2 , + .init_val =0x0000 , + .ro_mask =0xFFFF , + .emu_mask =0xFFFF , + .init = xen_pt_ext_cap_capid_reg_init, + .u.w.read = xen_pt_word_reg_read, + .u.w.write = xen_pt_word_reg_write, + }, + { + .offset =XEN_PCIE_CAP_LIST_NEXT , + .size =2 , + .init_val =0x0000 , + .ro_mask =0xFFFF , + .emu_mask =0xFFFF , + .init = xen_pt_ext_cap_ptr_reg_init, + .u.w.read = xen_pt_word_reg_read, + .u.w.write = xen_pt_word_reg_write, + }, + { + .offset =PCI_VNDR_HEADER , + .size =4 , + .init_val =0x00000000 , + .ro_mask =0xFFFFFFFF , + .emu_mask =0x00000000 , + .init = xen_pt_common_reg_init, + .u.dw.read = xen_pt_long_reg_read, + .u.dw.write = xen_pt_long_reg_write, + }, + { + .size =0 , + }, +}; + +/**************************** * Capabilities */
@@ -1708,6 +1758,23 @@ static int xen_pt_vendor_size_init(XenPCIPassthroughState *s, *size = sz;return ret; } + +static int xen_pt_ext_cap_vendor_size_init (XenPCIPassthroughState *s, +const XenPTRegGroupInfo *grp_reg, +uint32_t base_offset, +uint32_t *size) +{ +uint32_t vsec_hdr =0 ; +int ret =xen_host_pci_get_long (&s->real_device, + base_offset +PCI_VNDR_HEADER , + &vsec_hdr); + + *size =PCI_VNDR_HEADER_LEN (vsec_hdr); + +log_pcie_extended_cap (s,"Vendor-specific" , base_offset, *size); + +return ret; +}/* get PCI Express Capability Structure register group size */ static int xen_pt_pcie_size_init(XenPCIPassthroughState *s, const XenPTRegGroupInfo *grp_reg,
@@ -1934,6 +2001,14 @@ static const XenPTRegGroupInfo xen_pt_emu_reg_grps[] = { .size_init = xen_pt_reg_grp_size_init, .emu_regs = xen_pt_emu_reg_igd_opregion, }, +/* Vendor-specific Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_VNDR), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0xFF, + .size_init = xen_pt_ext_cap_vendor_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_vendor, + }, { .grp_size = 0, },
@@ -2054,8 +2129,6 @@ out:return 0 ; } - -/************* * Main */
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
RFC PATCH 26/30 xen/pt: add fixed-size PCIe Extended Capabilities descriptors
In reply to Alexey GerasimenkoThis adds description structures for all fixed-size PCIe Extended Capabilities.
For every capability register group, only 2 registers are emulated
currently: Capability ID (16 bit) and Next Capability Offset/Version (16 bit). Both needed to implement selective capability hiding. All other registers are passed through at the moment (unless they belong to a "hardwired" capability which is hidden)
--- hw/xen/xen_pt_config_init.c | 183 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 183 insertions(+)
hw/xen/xen_pt_config_init.c +183 -0
@@ -1734,6 +1734,37 @@ static XenPTRegInfo xen_pt_ext_cap_emu_reg_vendor[] = { }; +/* Common reg static information table for all passthru-type +* PCIe Extended Capabilities. Only Extended Cap ID and +* Next pointer are handled (to support capability hiding). +*/ +static XenPTRegInfo xen_pt_ext_cap_emu_reg_dummy[] = { + { + .offset =XEN_PCIE_CAP_ID , + .size =2 , + .init_val =0x0000 , + .ro_mask =0xFFFF , + .emu_mask =0xFFFF , + .init = xen_pt_ext_cap_capid_reg_init, + .u.w.read = xen_pt_word_reg_read, + .u.w.write = xen_pt_word_reg_write, + }, + { + .offset =XEN_PCIE_CAP_LIST_NEXT , + .size =2 , + .init_val =0x0000 , + .ro_mask =0xFFFF , + .emu_mask =0xFFFF , + .init = xen_pt_ext_cap_ptr_reg_init, + .u.w.read = xen_pt_word_reg_read, + .u.w.write = xen_pt_word_reg_write, + }, + { + .size =0 , + }, +}; + +/**************************** * Capabilities */
@@ -2009,6 +2040,158 @@ static const XenPTRegGroupInfo xen_pt_emu_reg_grps[] = { .size_init = xen_pt_ext_cap_vendor_size_init, .emu_regs = xen_pt_ext_cap_emu_reg_vendor, }, + /* Device Serial Number Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_DSN), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = PCI_EXT_CAP_DSN_SIZEOF, /*0x0C*/ + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Power Budgeting Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_PWR), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = PCI_EXT_CAP_PWR_SIZEOF, /*0x10*/ + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Root Complex Internal Link Control Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_RCILC), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0x0C, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Root Complex Event Collector Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_RCEC), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0x08, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Root Complex Register Block Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_RCRB), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0x14, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Configuration Access Correlation Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_CAC), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0x08, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Alternate Routing ID Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_ARI), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = PCI_EXT_CAP_ARI_SIZEOF, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Address Translation Services Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_ATS), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = PCI_EXT_CAP_ATS_SIZEOF, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Single Root I/O Virtualization Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_SRIOV), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = PCI_EXT_CAP_SRIOV_SIZEOF, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Page Request Interface Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_PRI), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = PCI_EXT_CAP_PRI_SIZEOF, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Latency Tolerance Reporting Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_LTR), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = PCI_EXT_CAP_LTR_SIZEOF, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Secondary PCIe Capability Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_SECPCI), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0x10, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Process Address Space ID Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_PASID), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = PCI_EXT_CAP_PASID_SIZEOF, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* L1 PM Substates Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_L1SS), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0x10, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Precision Time Measurement Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_PTM), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0x0C, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* M-PCIe Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(0x20), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0x1C, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* LN Requester (LNR) Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(0x1C), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0x08, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Function Readiness Status (FRS) Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(0x21), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0x10, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Readiness Time Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(0x22), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0x0C, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, { .grp_size = 0, },
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
RFC PATCH 27/30 xen/pt: add AER PCIe Extended Capability descriptor and sizing
In reply to Alexey GerasimenkoThe patch provides Advanced Error Reporting PCIe Extended Capability description structure and corresponding capability sizing function.
--- hw/xen/xen_pt_config_init.c | 72 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 72 insertions(+)
hw/xen/xen_pt_config_init.c +72 -0
@@ -1924,6 +1924,70 @@ static int xen_pt_msix_size_init(XenPCIPassthroughState *s,return 0 ; } +/* get Advanced Error Reporting Extended Capability register group size */ +#define PCI_ERR_CAP_TLP_PREFIX_LOG (1U << 11) +#define PCI_DEVCAP2_END_END_TLP_PREFIX (1U << 21) +static int xen_pt_ext_cap_aer_size_init (XenPCIPassthroughState *s, +const XenPTRegGroupInfo *grp_reg, +uint32_t base_offset, +uint32_t *size) +{ +uint8_t dev_type =get_pcie_device_type (s); +uint32_t aer_caps =0 ; +uint32_t sz =0 ; +int pcie_cap_pos; +uint32_t devcaps2; +int ret =0 ; + + pcie_cap_pos =xen_host_pci_find_next_cap (&s->real_device,0 , +PCI_CAP_ID_EXP ); +if (!pcie_cap_pos) { +XEN_PT_ERR (&s->dev, +"Cannot find a required PCI Express Capability\n" ); +return -1 ; + } + +if (get_pcie_capability_version (s) >1 ) { + ret =xen_host_pci_get_long (&s->real_device, + pcie_cap_pos +PCI_EXP_DEVCAP2 , + &devcaps2); +if (ret) { +XEN_PT_ERR (&s->dev,"Error while reading Device " +"Capabilities 2 Register \n" ); +return -1 ; + } + } + +if (devcaps2 &PCI_DEVCAP2_END_END_TLP_PREFIX ) { + ret =xen_host_pci_get_long (&s->real_device, + base_offset +PCI_ERR_CAP , + &aer_caps); +if (ret) { +XEN_PT_ERR (&s->dev, +"Error while reading AER Extended Capability\n" ); +return -1 ; + } + +if (aer_caps &PCI_ERR_CAP_TLP_PREFIX_LOG ) { + sz =0x48 ; + } + } + +if (!sz) { +if (dev_type ==PCI_EXP_TYPE_ROOT_PORT || + dev_type ==PCI_EXP_TYPE_RC_EC ) { + sz =0x38 ; + }else { + sz =0x2C ; + } + } + + *size = sz; + +log_pcie_extended_cap (s,"AER" , base_offset, *size); +return ret; +} +static const XenPTRegGroupInfo xen_pt_emu_reg_grps[] = {/* Header Type0 reg group */
@@ -2192,6 +2256,14 @@ static const XenPTRegGroupInfo xen_pt_emu_reg_grps[] = { .size_init = xen_pt_reg_grp_size_init, .emu_regs = xen_pt_ext_cap_emu_reg_dummy, }, +/* Advanced Error Reporting Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_ERR), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0xFF, + .size_init = xen_pt_ext_cap_aer_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, { .grp_size = 0, },
Signature · 1 line
2.11.0
RFC PATCH 28/30 xen/pt: add descriptors and size calculation for RCLD/ACS/PMUX/DPA/MCAST/TPH/DPC PCIe Extended Capabilities
In reply to Alexey GerasimenkoAdd few more PCIe Extended Capabilities entries to the xen_pt_emu_reg_grps[] array along with their corresponding *_size_init() functions.
All these capabilities have non-fixed size but their size calculation is very simple, hence adding them in a single batch.
For every capability register group, only 2 registers are emulated
currently: Capability ID (16 bit) and Next Capability Offset/Version (16 bit). Both needed to implement the selective capability hiding. All other registers are passed through at the moment (unless they belong to a capability marked as "hardwired" which is hidden)
--- hw/xen/xen_pt_config_init.c | 224 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 224 insertions(+)
hw/xen/xen_pt_config_init.c +224 -0
@@ -1988,6 +1988,174 @@ static int xen_pt_ext_cap_aer_size_init(XenPCIPassthroughState *s,return ret; } +/* get Root Complex Link Declaration Extended Capability register group size */ +#define RCLD_GET_NUM_ENTRIES (x) (((x) >> 8) & 0xFF) +static int xen_pt_ext_cap_rcld_size_init (XenPCIPassthroughState *s, +const XenPTRegGroupInfo *grp_reg, +uint32_t base_offset, +uint32_t *size) +{ +uint32_t elem_self_descr =0 ; + +int ret =xen_host_pci_get_long (&s->real_device, + base_offset +4 , + &elem_self_descr); + + *size =0x10 +RCLD_GET_NUM_ENTRIES (elem_self_descr) *0x10 ; + +log_pcie_extended_cap (s,"Root Complex Link Declaration" , + base_offset, *size); +return ret; +} + +/* get Access Control Services Extended Capability register group size */ +#define ACS_VECTOR_SIZE_BITS (x) ((((x) >> 8) & 0xFF) ?: 256) +static int xen_pt_ext_cap_acs_size_init (XenPCIPassthroughState *s, +const XenPTRegGroupInfo *grp_reg, +uint32_t base_offset, +uint32_t *size) +{ +uint16_t acs_caps =0 ; + +int ret =xen_host_pci_get_word (&s->real_device, + base_offset +PCI_ACS_CAP , + &acs_caps); + +if (acs_caps &PCI_ACS_EC ) { +uint32_t vector_sz =ACS_VECTOR_SIZE_BITS (acs_caps); + + *size =PCI_ACS_EGRESS_CTL_V + ((vector_sz +7 ) & ~7 ) /8 ; + }else { + *size =PCI_ACS_EGRESS_CTL_V ; + } + +log_pcie_extended_cap (s,"ACS" , base_offset, *size); +return ret; +} + +/* get Multicast Extended Capability register group size */ +static int xen_pt_ext_cap_multicast_size_init (XenPCIPassthroughState *s, +const XenPTRegGroupInfo *grp_reg, +uint32_t base_offset, +uint32_t *size) +{ +uint8_t dev_type =get_pcie_device_type (s); + +switch (dev_type) { +case PCI_EXP_TYPE_ENDPOINT : +case PCI_EXP_TYPE_LEG_END : +case PCI_EXP_TYPE_RC_END : +case PCI_EXP_TYPE_RC_EC : +default : + *size =PCI_EXT_CAP_MCAST_ENDPOINT_SIZEOF ; +break ; + +case PCI_EXP_TYPE_ROOT_PORT : +case PCI_EXP_TYPE_UPSTREAM : +case PCI_EXP_TYPE_DOWNSTREAM : + *size =0x30 ; +break ; + } + +log_pcie_extended_cap (s,"Multicast" , base_offset, *size); +return 0 ; +} + +/* get Dynamic Power Allocation Extended Capability register group size */ +static int xen_pt_ext_cap_dpa_size_init (XenPCIPassthroughState *s, +const XenPTRegGroupInfo *grp_reg, +uint32_t base_offset, +uint32_t *size) +{ +uint32_t dpa_caps =0 ; +uint32_t num_entries; + +int ret =xen_host_pci_get_long (&s->real_device, + base_offset +PCI_DPA_CAP , + &dpa_caps); + + num_entries = (dpa_caps &PCI_DPA_CAP_SUBSTATE_MASK ) +1 ; + + *size =PCI_DPA_BASE_SIZEOF + num_entries/*byte-size registers*/ ; + +log_pcie_extended_cap (s,"Dynamic Power Allocation" , base_offset, *size); +return ret; +} + +/* get TPH Requester Extended Capability register group size */ +static int xen_pt_ext_cap_tph_size_init (XenPCIPassthroughState *s, +const XenPTRegGroupInfo *grp_reg, +uint32_t base_offset, +uint32_t *size) +{ +uint32_t tph_caps =0 ; +uint32_t num_entries; + +int ret =xen_host_pci_get_long (&s->real_device, + base_offset +PCI_TPH_CAP , + &tph_caps); + +switch (tph_caps &PCI_TPH_CAP_LOC_MASK ) { +case PCI_TPH_LOC_CAP : + num_entries = (tph_caps &PCI_TPH_CAP_ST_MASK ) >>PCI_TPH_CAP_ST_SHIFT ; + num_entries++; +break ; + +case PCI_TPH_LOC_NONE : +case PCI_TPH_LOC_MSIX : +default : +/* not in the capability */ + num_entries =0 ; + } + + *size =PCI_TPH_BASE_SIZEOF + num_entries *2 ; + +log_pcie_extended_cap (s,"TPH Requester" , base_offset, *size); +return ret; +} + +/* get Downstream Port Containment Extended Capability register group size */ +static int xen_pt_ext_cap_dpc_size_init (XenPCIPassthroughState *s, +const XenPTRegGroupInfo *grp_reg, +uint32_t base_offset, +uint32_t *size) +{ +uint16_t dpc_caps =0 ; + +int ret =xen_host_pci_get_word (&s->real_device, + base_offset +PCI_EXP_DPC_CAP , + &dpc_caps); + +if (dpc_caps &PCI_EXP_DPC_CAP_RP_EXT ) { + *size =0x20 + ((dpc_caps &PCI_EXP_DPC_RP_PIO_LOG_SIZE ) >>8 ) *4 ; + }else { + *size =0xC ; + } + +log_pcie_extended_cap (s,"Downstream Port Containment" , + base_offset, *size); +return ret; +} + +/* get Protocol Multiplexing Extended Capability register group size */ +#define PMUX_GET_NUM_ENTRIES (x) ((x) & 0x3F) +static int xen_pt_ext_cap_pmux_size_init (XenPCIPassthroughState *s, +const XenPTRegGroupInfo *grp_reg, +uint32_t base_offset, +uint32_t *size) +{ +uint32_t pmux_caps =0 ; + +int ret =xen_host_pci_get_long (&s->real_device, + base_offset +4 , + &pmux_caps); + + *size =0x10 +PMUX_GET_NUM_ENTRIES (pmux_caps) *4 ; + +log_pcie_extended_cap (s,"PMUX" , base_offset, *size); +return ret; +} +static const XenPTRegGroupInfo xen_pt_emu_reg_grps[] = {/* Header Type0 reg group */
@@ -2264,6 +2432,62 @@ static const XenPTRegGroupInfo xen_pt_emu_reg_grps[] = { .size_init = xen_pt_ext_cap_aer_size_init, .emu_regs = xen_pt_ext_cap_emu_reg_dummy, }, +/* Root Complex Link Declaration Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_RCLD), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0xFF, + .size_init = xen_pt_ext_cap_rcld_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Access Control Services Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_ACS), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0xFF, + .size_init = xen_pt_ext_cap_acs_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Multicast Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_MCAST), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0xFF, + .size_init = xen_pt_ext_cap_multicast_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Dynamic Power Allocation Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_DPA), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0xFF, + .size_init = xen_pt_ext_cap_dpa_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* TPH Requester Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_TPH), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0xFF, + .size_init = xen_pt_ext_cap_tph_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Protocol Multiplexing Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_PMUX), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0xFF, + .size_init = xen_pt_ext_cap_pmux_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Downstream Port Containment Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_DPC), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0xFF, + .size_init = xen_pt_ext_cap_dpc_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, { .grp_size = 0, },
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
RFC PATCH 29/30 xen/pt: add Resizable BAR PCIe Extended Capability descriptor and sizing
In reply to Alexey GerasimenkoUnlike other PCIe Extended Capabilities, we currently cannot allow attempts to use Resizable BAR Capability. Without specifically handling BAR resizing we're likely end up with corrupted MMIO hole layout if guest OS will attempt to use this feature. Actually, recent Windows versions started to understand and use the Resizable BAR Capability (see [1]).
For now, we need to hide the Resizable BAR Capability from guest OS until BAR resizing emulation support will be implemented in Xen. This support is a pretty much mandatory todo-feature as the effect of writing to Resizable BAR control registers can be considered similar to reprogramming normal BAR registers -- i.e. this needs to be handled explicitly, resulting in corresponding MMIO BAR range(s) remapping. Until then, we mark the Resizable BAR Capability as XEN_PT_GRP_TYPE_HARDWIRED.
[1]: https://docs.microsoft.com/en-us/windows-hardware/drivers/display/resizable-bar-support
--- hw/xen/xen_pt_config_init.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+)
hw/xen/xen_pt_config_init.c +27 -0
@@ -2156,6 +2156,26 @@ static int xen_pt_ext_cap_pmux_size_init(XenPCIPassthroughState *s,return ret; } +/* get Resizable BAR Extended Capability register group size */ +static int xen_pt_ext_cap_rebar_size_init (XenPCIPassthroughState *s, +const XenPTRegGroupInfo *grp_reg, +uint32_t base_offset, +uint32_t *size) +{ +uint32_t rebar_ctl =0 ; +uint32_t num_entries; + +int ret =xen_host_pci_get_long (&s->real_device, + base_offset +PCI_REBAR_CTRL , + &rebar_ctl); + num_entries = + (rebar_ctl &PCI_REBAR_CTRL_NBAR_MASK ) >>PCI_REBAR_CTRL_NBAR_SHIFT ; + + *size = num_entries*8 +4 ; + +log_pcie_extended_cap (s,"Resizable BAR" , base_offset, *size); +return ret; +}static const XenPTRegGroupInfo xen_pt_emu_reg_grps[] = {/* Header Type0 reg group */
@@ -2488,6 +2508,13 @@ static const XenPTRegGroupInfo xen_pt_emu_reg_grps[] = { .size_init = xen_pt_ext_cap_dpc_size_init, .emu_regs = xen_pt_ext_cap_emu_reg_dummy, }, +/* Resizable BAR Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_REBAR), + .grp_type = XEN_PT_GRP_TYPE_HARDWIRED, + .grp_size = 0xFF, + .size_init = xen_pt_ext_cap_rebar_size_init, + }, { .grp_size = 0, },
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
RFC PATCH 30/30 xen/pt: add VC/VC9/MFVC PCIe Extended Capabilities descriptors and sizing
In reply to Alexey GerasimenkoVirtual Channel/MFVC capabilities are relatively useless for emulation (passing through accesses to them should be enough in most cases) yet they have hardest format of all PCIe Extended Capabilities, mostly because VC capability format allows the sparse config space layout with gaps between the parts which make up the VC capability.
We have the main capability body followed by variable number of entries where each entry may additionally reference the arbitration table outside main capability body. There are no constrains on these arbitration table offsets -- in theory, they may reside outside the VC capability range anywhere in PCIe extended config space. Also, every arbitration table size is not fixed - it depends on current VC/Port Arbitration Select field value.
To simplify things, this patch assume that changing VC/Port Arbitration Select value (i.e. resizing arbitration tables) do not cause arbitration table offsets to change. Normally the device must place arbitration tables considering their maximum size, not current one. Maximum arbitration table size depends on VC/Port Arbitration Capability bitmask -- this is what actually used to calculate the arbitration table size.
--- hw/xen/xen_pt_config_init.c | 192 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 192 insertions(+)
hw/xen/xen_pt_config_init.c +192 -0
@@ -2177,6 +2177,174 @@ static int xen_pt_ext_cap_rebar_size_init(XenPCIPassthroughState *s,return ret; } +/* get VC/VC9/MFVC Extended Capability register group size */ +static uint32_t get_arb_table_len_max (XenPCIPassthroughState *s, +uint32_t max_bit_supported, +uint32_t arb_cap) +{ +int n_bit; +uint32_t table_max_size =0 ; + +if (!arb_cap) { +return 0 ; + } + +for (n_bit =7 ; n_bit >=0 && !(arb_cap & (1 << n_bit)); n_bit--); + +if (n_bit > max_bit_supported) { +XEN_PT_ERR (&s->dev,"Warning: encountered unknown VC arbitration " +"capability supported: 0x%02x\n" , (uint8_t ) arb_cap); + } + +switch (n_bit) { +case 0 :break ; +case 1 :return 32 ; +case 2 :return 64 ; +case 3 :/*128 too*/ +case 4 :return 128 ; +default : + table_max_size =8 << n_bit; + } + +return table_max_size; +} + +#define GET_ARB_TABLE_OFFSET (x) (((x) >> 24) * 0x10) +#define GET_VC_ARB_CAPABILITY (x) ((x) & 0xFF) +#define ARB_TABLE_ENTRY_SIZE_BITS (x) (1 << (((x) & PCI_VC_CAP1_ARB_SIZE)\ + >> 10)) +static int xen_pt_ext_cap_vchan_size_init (XenPCIPassthroughState *s, +const XenPTRegGroupInfo *grp_reg, +uint32_t base_offset, +uint32_t *size) +{ +uint32_t header; +uint32_t vc_cap_max_size =PCIE_CONFIG_SPACE_SIZE - base_offset; +uint32_t next_ptr; +uint32_t arb_table_start_max =0 , arb_table_end_max =0 ; +uint32_t port_vc_cap1, port_vc_cap2, vc_rsrc_cap; +uint32_t ext_vc_count =0 ; +uint32_t arb_table_entry_size;/* in bits */ +const char *cap_name; +int ret; +int i; + + ret =xen_host_pci_get_long (&s->real_device, base_offset, &header); +if (ret) { + goto err_read; + } + + next_ptr =PCI_EXT_CAP_NEXT (header); + +switch (PCI_EXT_CAP_ID (header)) { +case PCI_EXT_CAP_ID_VC : +case PCI_EXT_CAP_ID_VC9 : + cap_name ="Virtual Channel" ; +break ; +case PCI_EXT_CAP_ID_MFVC : + cap_name ="Multi-Function VC" ; +break ; +default : +XEN_PT_ERR (&s->dev,"Unknown VC Extended Capability ID " +"encountered: 0x%04x\n" ,PCI_EXT_CAP_ID (header)); +return -1 ; + } + +if (next_ptr && next_ptr > base_offset) { + vc_cap_max_size = next_ptr - base_offset; + } + + ret =xen_host_pci_get_long (&s->real_device, + base_offset +PCI_VC_PORT_CAP1 , + &port_vc_cap1); +if (ret) { + goto err_read; + } + + ret =xen_host_pci_get_long (&s->real_device, + base_offset +PCI_VC_PORT_CAP2 , + &port_vc_cap2); +if (ret) { + goto err_read; + } + + ext_vc_count = port_vc_cap1 &PCI_VC_CAP1_EVCC ; + + arb_table_start_max =GET_ARB_TABLE_OFFSET (port_vc_cap2); + +/* check arbitration table offset for validity */ +if (arb_table_start_max >= vc_cap_max_size) { +XEN_PT_ERR (&s->dev,"Warning: VC arbitration table offset points " +"outside the expected range: %#04x\n" , + (uint16_t ) arb_table_start_max); +/* skip this arbitration table */ + arb_table_start_max =0 ; + } + +if (arb_table_start_max) { +uint32_t vc_arb_cap =GET_VC_ARB_CAPABILITY (port_vc_cap2); +uint32_t num_phases =get_arb_table_len_max (s,3 , vc_arb_cap); +uint32_t arb_tbl_sz =QEMU_ALIGN_UP (num_phases *4 ,32 ) /8 ; + + arb_table_end_max = base_offset + arb_table_start_max + arb_tbl_sz; + } + +/* get Function/Port Arbitration Table Entry size */ + arb_table_entry_size =ARB_TABLE_ENTRY_SIZE_BITS (port_vc_cap1); + +/* process all VC Resource entries */ +for (i =0 ; i < ext_vc_count; i++) { +uint32_t arb_table_offset; + +/* read VC Resource Capability */ + ret =xen_host_pci_get_long (&s->real_device, + base_offset +PCI_VC_RES_CAP + i *PCI_CAP_VC_PER_VC_SIZEOF , + &vc_rsrc_cap); +if (ret) { + goto err_read; + } + + arb_table_offset =GET_ARB_TABLE_OFFSET (vc_rsrc_cap); + +if (arb_table_offset > arb_table_start_max) { +/* check arbitration table offset for validity */ +if (arb_table_offset >= vc_cap_max_size) { +XEN_PT_ERR (&s->dev,"Warning: Port/Function arbitration table " +"offset points outside the expected range: %#04x\n" , + (uint16_t ) arb_table_offset); +/* skip this arbitration table */ + arb_table_offset =0 ; + }else { + arb_table_start_max = arb_table_offset; + } + +if (arb_table_offset) { +uint32_t vc_arb_cap =GET_VC_ARB_CAPABILITY (vc_rsrc_cap); +uint32_t num_phases =get_arb_table_len_max (s,5 , vc_arb_cap); +uint32_t arb_tbl_sz = +QEMU_ALIGN_UP (num_phases * arb_table_entry_size,32 ) /8 ; + + arb_table_end_max = base_offset + arb_table_offset + arb_tbl_sz; + } + } + } + +if (arb_table_end_max) { + *size = arb_table_end_max - base_offset; + }else { + *size =PCI_CAP_VC_BASE_SIZEOF + + ext_vc_count *PCI_CAP_VC_PER_VC_SIZEOF ; + } + +log_pcie_extended_cap (s, cap_name, base_offset, *size); +return 0 ; + +err_read: +XEN_PT_ERR (&s->dev,"Error while reading VC Extended Capability\n" ); +return ret; +} + +static const XenPTRegGroupInfo xen_pt_emu_reg_grps[] = {/* Header Type0 reg group */ {
@@ -2515,6 +2683,30 @@ static const XenPTRegGroupInfo xen_pt_emu_reg_grps[] = { .grp_size = 0xFF, .size_init = xen_pt_ext_cap_rebar_size_init, }, +/* Virtual Channel Extended Capability reg group (2) */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_VC), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0xFF, + .size_init = xen_pt_ext_cap_vchan_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Virtual Channel Extended Capability reg group (9) */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_VC9), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0xFF, + .size_init = xen_pt_ext_cap_vchan_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Multi-Function Virtual Channel Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_MFVC), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0xFF, + .size_init = xen_pt_ext_cap_vchan_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, { .grp_size = 0, },
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
The subject line says to expect 30 patches, but you've only sent 18 to the list here. I eventually figured out that the first 12 patches were in Xen code and so not sent to qemu-devel.
For future if you have changes that affect multiple completely separate projects, send them as separate series. ie just send PATCH 00/18 to QEMU devel so it doesn't look like a bunch of patches have gone missing.
Quoted 7 lines · Alexey Gerasimenko
How to use the Q35 feature:
A new domain config option was implemented:
device_model_machine. It's a string which has following possible values:
- "i440" -- i440 emulation (default)
- "q35" -- emulate a Q35 machine. By default, the storage interface is AHCI.
Presumably this is mapping to the QEMU -machine arg, so it feels desirable to keep the same naming scheme. ie allow any of the versioned machine names that QEMU uses. eg any of "pc-q35-2.x" versioned types, or 'q35' as an alias for latest, and use "pc-i440fx-2.x" versioned types of 'pc' as an alias for latest, rather than 'i440' which is needlessly divering from the QEMU machine type.
Regards,
Daniel
Signature · 8 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 8 lines · On Tue, 13 Mar 2018 09:21:54 +0000 Daniel P. Berrangé <[email protected]>…
The subject line says to expect 30 patches, but you've only sent 18 to the list here. I eventually figured out that the first 12 patches were in Xen code and so not sent to qemu-devel.
For future if you have changes that affect multiple completely separate projects, send them as separate series. ie just send PATCH 00/18 to QEMU devel so it doesn't look like a bunch of patches have gone missing.
OK, we'll do for next versions.
Quoted 6 lines
Quoted 5 lines
A new domain config option was implemented:
device_model_machine. It's a string which has following possible values:
- "i440" -- i440 emulation (default)
- "q35" -- emulate a Q35 machine. By default, the storage interface is AHCI.
Quoted 6 lines
Presumably this is mapping to the QEMU -machine arg, so it feels desirable to keep the same naming scheme. ie allow any of the versioned machine names that QEMU uses. eg any of "pc-q35-2.x" versioned types, or 'q35' as an alias for latest, and use "pc-i440fx-2.x" versioned types of 'pc' as an alias for latest, rather than 'i440' which is needlessly divering from the QEMU machine type.
Yes, it is translated into the '-machine' argument.
A direct mapping between the Xen device_model_machine option and QEMU '-machine' argument won't be accepted by Xen maintainers I guess.
The main problem with this approach is a requirement to have a match between Xen/libxl and QEMU versions. If, for example, device_model_machine tells something like "pc-q35-2.11" and later we downgrade QEMU to some older version we'll likely have a problem without changing anything in the domain config. So I guess the "use the latest available" approach for machine selection (pc, q35, etc) is the only possible option. Perhaps having the way to specify the exact QEMU machine name and version in a separate domain config parameter (for advanced use) might be feasible.
Also, parameter names do not speak for themselves I'm afraid. This way we'll have, for example, device_model_machine="pc" vs device_model_machine="q35"... a bit unclear I think. This may be obvious for a QEMU user, but many Xen users didn't get used to QEMU machines and there might be some wondering why "q35" is not "pc" and why "pc" is an i440 system precisely.
Another obstacle here is xen_platform_device option which indirectly selects QEMU machine type for i440 at the moment (pc/xenfv), but this may be addressed by controlling the Xen platform device independently via a separate machine property or '-device xen-platform' like Eduardo Habkost suggested.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 32 lines · Alexey G
Quoted 8 lines · On Tue, 13 Mar 2018 09:21:54 +0000 Daniel P. Berrangé <[email protected]>…
The subject line says to expect 30 patches, but you've only sent 18 to the list here. I eventually figured out that the first 12 patches were in Xen code and so not sent to qemu-devel.
For future if you have changes that affect multiple completely separate projects, send them as separate series. ie just send PATCH 00/18 to QEMU devel so it doesn't look like a bunch of patches have gone missing.
OK, we'll do for next versions.
Quoted 6 lines
Quoted 5 lines
A new domain config option was implemented:
device_model_machine. It's a string which has following possible values:
- "i440" -- i440 emulation (default)
- "q35" -- emulate a Q35 machine. By default, the storage interface is AHCI.Quoted 6 lines
Presumably this is mapping to the QEMU -machine arg, so it feels desirable to keep the same naming scheme. ie allow any of the versioned machine names that QEMU uses. eg any of "pc-q35-2.x" versioned types, or 'q35' as an alias for latest, and use "pc-i440fx-2.x" versioned types of 'pc' as an alias for latest, rather than 'i440' which is needlessly divering from the QEMU machine type.
Yes, it is translated into the '-machine' argument.
A direct mapping between the Xen
device_model_machineoption and QEMU '-machine' argument won't be accepted by Xen maintainers I guess.
Quoted 9 lines
The main problem with this approach is a requirement to have a match between Xen/libxl and QEMU versions. If, for example,
device_model_machinetells something like "pc-q35-2.11" and later we downgrade QEMU to some older version we'll likely have a problem without changing anything in the domain config. So I guess the "use the latest available" approach for machine selection (pc, q35, etc) is the only possible option. Perhaps having the way to specify the exact QEMU machine name and version in a separate domain config parameter (for advanced use) might be feasible.
At least with plain QEMU or KVM, using the versioned machine type names is important as that is what guarantees you a stable guest machine ABI, independant of QEMU version. If your deployment has a mixture of QEMU versions on different hosts, then you very much want to pick a versioned machine type to ensure compatibility for live migration. With libvirt we accept the short "pc" or "q35" names on input, but expand them to the fully versioned name when saving the config file, so no matter which QEMU version is used each time the guest is launched, the ABI is always the same.
Quoted 8 lines
Also, parameter names do not speak for themselves I'm afraid. This way we'll have, for example, device_model_machine="pc" vs device_model_machine="q35"... a bit unclear I think. This may be obvious for a QEMU user, but many Xen users didn't get used to QEMU machines and there might be some wondering why "q35" is not "pc" and why "pc" is an i440 system precisely.
Quoted 5 lines
Another obstacle here is
xen_platform_deviceoption which indirectly selects QEMU machine type for i440 at the moment (pc/xenfv), but this may be addressed by controlling the Xen platform device independently via a separate machine property or '-device xen-platform' like Eduardo Habkost suggested.
Regards,
Daniel
Signature · 8 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 32 lines · Alexey G
Quoted 8 lines · On Tue, 13 Mar 2018 09:21:54 +0000 Daniel P. Berrangé <[email protected]>…
The subject line says to expect 30 patches, but you've only sent 18 to the list here. I eventually figured out that the first 12 patches were in Xen code and so not sent to qemu-devel.
For future if you have changes that affect multiple completely separate projects, send them as separate series. ie just send PATCH 00/18 to QEMU devel so it doesn't look like a bunch of patches have gone missing.
OK, we'll do for next versions.
Quoted 6 lines
Quoted 5 lines
A new domain config option was implemented:
device_model_machine. It's a string which has following possible values:
- "i440" -- i440 emulation (default)
- "q35" -- emulate a Q35 machine. By default, the storage interface is AHCI.Quoted 6 lines
Presumably this is mapping to the QEMU -machine arg, so it feels desirable to keep the same naming scheme. ie allow any of the versioned machine names that QEMU uses. eg any of "pc-q35-2.x" versioned types, or 'q35' as an alias for latest, and use "pc-i440fx-2.x" versioned types of 'pc' as an alias for latest, rather than 'i440' which is needlessly divering from the QEMU machine type.
Yes, it is translated into the '-machine' argument.
A direct mapping between the Xen
device_model_machineoption and QEMU '-machine' argument won't be accepted by Xen maintainers I guess.
Quoted 9 lines
The main problem with this approach is a requirement to have a match between Xen/libxl and QEMU versions. If, for example,
device_model_machinetells something like "pc-q35-2.11" and later we downgrade QEMU to some older version we'll likely have a problem without changing anything in the domain config. So I guess the "use the latest available" approach for machine selection (pc, q35, etc) is the only possible option. Perhaps having the way to specify the exact QEMU machine name and version in a separate domain config parameter (for advanced use) might be feasible.
At least with plain QEMU or KVM, using the versioned machine type names is important as that is what guarantees you a stable guest machine ABI, independant of QEMU version. If your deployment has a mixture of QEMU versions on different hosts, then you very much want to pick a versioned machine type to ensure compatibility for live migration. With libvirt we accept the short "pc" or "q35" names on input, but expand them to the fully versioned name when saving the config file, so no matter which QEMU version is used each time the guest is launched, the ABI is always the same.
Quoted 8 lines
Also, parameter names do not speak for themselves I'm afraid. This way we'll have, for example, device_model_machine="pc" vs device_model_machine="q35"... a bit unclear I think. This may be obvious for a QEMU user, but many Xen users didn't get used to QEMU machines and there might be some wondering why "q35" is not "pc" and why "pc" is an i440 system precisely.
Quoted 5 lines
Another obstacle here is
xen_platform_deviceoption which indirectly selects QEMU machine type for i440 at the moment (pc/xenfv), but this may be addressed by controlling the Xen platform device independently via a separate machine property or '-device xen-platform' like Eduardo Habkost suggested.
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
A gentle RFC-ping.
Any thoughts on this? Regarding the feature as a whole. So far there were responses mostly targeting individual patches, while I'd like to hear about chosen approaches in general, whether the overall direction is correct (or not), etc. It's just RFC after all, not v11. :)
I can split it into two series if that would be preferable, one for general Q35 bring up and basic access to PCIe extended config space via ECAM (this is what the feature was used for initially) and the second part is providing support for PCIe Extended Capabilities emulation infrastructure (hw/xen/xen-pt*.c in QEMU).
Quoted 4 lines · On Tue, 13 Mar 2018 04:33:45 +1000 Alexey Gerasimenko <[email protected]> wrote:
This patch series introduces support of Q35 emulation for Xen HVM guests (via QEMU). This feature is present in other virtualization products and Xen can greatly benefit from this feature as well.
Quoted 14 lines
The main goal for implementing Q35 emulation for Xen was extending PCI/GPU passthrough capabilities. It's the main advantage of Q35 emulation
- availability of extra features for PCIe device passthrough. The most important PCIe-specific passthrough feature Q35 provides is a support for PCIe config space ECAM (aka MMCONFIG) to allow accesses to extended PCIe config space (>256), which is MMIO-based. Lots of PCIe devices and their drivers make use of PCIe Extended Capabilities, whose can be accessed only using ECAM and offsets above0x100in PCI config space. Supporting ECAM is a mandatory feature for PCIe passthrough. Not only this allows passthrough PCIe devices to function properly, but opens a road to extend Xen PCIe passthrough features further -- eg. providing support for AER. One of possible directions is providing support for PCIe Resizable BARs -- a feature which likely
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Hi Alexey, thanks for the ping. I think this is a good feature to have and I would like to check it in when it is ready. I spoke with Anthony and agreed that he will be reviewing it. Please be patient but we'll get there :-)
Quoted 7 lines · Alexey G
A gentle RFC-ping.
Any thoughts on this? Regarding the feature as a whole. So far there were responses mostly targeting individual patches, while I'd like to hear about chosen approaches in general, whether the overall direction is correct (or not), etc. It's just RFC after all, not v11. :)
Quoted 28 lines
I can split it into two series if that would be preferable, one for general Q35 bring up and basic access to PCIe extended config space via ECAM (this is what the feature was used for initially) and the second part is providing support for PCIe Extended Capabilities emulation infrastructure (
hw/xen/xen-pt*.cin QEMU).Quoted 18 lines · On Tue, 13 Mar 2018 04:33:45 +1000 Alexey Gerasimenko <[email protected]> wrote:
This patch series introduces support of Q35 emulation for Xen HVM guests (via QEMU). This feature is present in other virtualization products and Xen can greatly benefit from this feature as well.
The main goal for implementing Q35 emulation for Xen was extending PCI/GPU passthrough capabilities. It's the main advantage of Q35 emulation
- availability of extra features for PCIe device passthrough. The most important PCIe-specific passthrough feature Q35 provides is a support for PCIe config space ECAM (aka MMCONFIG) to allow accesses to extended PCIe config space (>256), which is MMIO-based. Lots of PCIe devices and their drivers make use of PCIe Extended Capabilities, whose can be accessed only using ECAM and offsets above0x100in PCI config space. Supporting ECAM is a mandatory feature for PCIe passthrough. Not only this allows passthrough PCIe devices to function properly, but opens a road to extend Xen PCIe passthrough features further -- eg. providing support for AER. One of possible directions is providing support for PCIe Resizable BARs -- a feature which likely
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 6 lines · Alexey G
A gentle RFC-ping.
Any thoughts on this? Regarding the feature as a whole. So far there were responses mostly targeting individual patches, while I'd like to hear about chosen approaches in general, whether the overall direction is correct (or not), etc. It's just RFC after all, not v11. :)
I plan to look at the series, but in general you should wait at least 7 days (one week) before pinging.
Quoted 5 lines
I can split it into two series if that would be preferable, one for general Q35 bring up and basic access to PCIe extended config space via ECAM (this is what the feature was used for initially) and the second part is providing support for PCIe Extended Capabilities emulation infrastructure (
hw/xen/xen-pt*.cin QEMU).
I would wait a bit before doing more work, as said I plan to look at the series, and other probably are too, it's just that you gave us too little time ;).
Keep in mind we are approaching feature freeze, and there are some more mature series that will likely get more review attention than yours ATM, in order to try to get them in before the freeze.
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel