RFC PV Passthrough PCI Device Hotplug Support

Hi all,

This patch set adds passthrough PCI device hotplug support for PV driver domains.

I began working on this some little time ago and developed independently of the HVM PCI device hotplug code. Now that HVM PCI device hotplug was checked in, some modifications are needed to apply this patch to current xen-unstable.

Please see below for detailed descriptions.
Comments are welcome.

[Usage]
To hot add a new PCI device '0000:00:1d.0',
xm pci-attach <PV domain> 0000:00:1d.0
To hot remove a PCI device '0000:00:1d.0',
xm pci-detach <PV domain> 0000:00:1d.0

[Implementation details]

- Interface changes
New xenbus states "Reconfiguring" and "Reconfigured" are introduced.

- Xenstore changes
Substates(state-#) and virtual pci slots(vdev-#) entries are added for pciback driver. For example, when PCI devices 00:1d.0 and 00:1d.1 are connected, xenstore-ls will show information like this.

    pci = ""
     3 = ""
      0 = ""
       ...
       state = "4"
       dev-0 = "0000:00:1d.00"
       vdev-0 = "0000:00:00.00"
       state-0 = "3"
       dev-1 = "0000:00:1d.1"
       vdev-1 = "0000:00:00.01"
       state-1 = "3"
       ...

- Attach sequence
1) User executes xm pci-attach.
2) If there's no pcidev, Xend create new one.
Otherwise, Xend does following things:
2a) write new dev-# entry on xenstore
2b) write new state-# entry and set it as "Initialising" 2c) enable io resources
2d) switch pciback state from "Connected" to "Reconfiguring"
3) pcifront detects backend change and switch its state from "Connected" to "Reconfiguring".
4) pciback detects frontend change and:
4a) scan xenstore and find the device being attached 4b) export the device, write vdev-# entry on xenstore and change
state-# to "Initialized"
4c) switch its state from "Reconfiguring" to "Reconfigured"
5) pcifront detects backend change and:
5a) rescan the pcibus for the attached device and enable it. 5b) switch its state from "Reconfiguring" to "Connected"
6) pciback detects pcifront change and switch its state from "Reconfigured" to "Connected".

- Detach sequence
1) User executes xm pci-detach.
2) If the specified device exists, Xend does following things:
2a) change the device's substate(state-#) to "Closing" 2b) switch pciback state from "Connected" to "Reconfiguring"
3) pcifront detects backend change and:
3a) scan xenstore and find the device being detached
(the virtual pci slot can be identified by vdev-# entry)
3b) remove the device
3c) switch its state from "Connected" to "Reconfiguring"
4) pciback detects frontend change and:
4a) scan xenstore and find the device being detached 4b) remove the device
4c) switch its state from "Reconfiguring" to "Reconfigured"
5) pcifront detects backend change and switch its state from "Reconfiguring" to "Connected".
6) pciback detects pcifront change and switch its state from "Reconfigured" to "Connected".
7) Xend, who has been watching on the xenbus state, detects backend change and:
7a) cleanup xenstore entries
7b) disable io resources
7c) if there's no device left, destroy pcidev

- Limitations
Hotplug currently only works when pciback is compiled with CONFIG_XEN_PCIDEV_BACKEND_VPCI or CONFIG_XEN_PCIDEV_BACKEND_SLOT.

Thanks,
-------------------
Yosuke Iwamatsu
NEC Corporation