Skip to contentSkip to conversation

Fix HVM PCI hot removal failure

PATCH In review Zhai, Edwin 6 participants 15 messages Apr 28, 2009 – May 7, 2009

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Fix HVM PCI hot removal failure

Fix a bug of HVM PCI hot removal failure introduced in r19573. xend's deassign_device should not be called for HVM, as qemu already do so.

Signature · 2 lines
best rgds,
edwin
tools/python/xen/xend/server/pciif.py +8 -6
Index: hv/tools/python/xen/xend/server/pciif.py
===================================================================
@@ -493,12 +493,14 @@ class PciController(DevController):
         # DMA transaction, etc
         dev.do_FLR()
 
-        pci_str = "0x%x, 0x%x, 0x%x, 0x%x" % (domain, bus, slot, func)
-        bdf = xc.deassign_device(fe_domid, pci_str)
-        if bdf > 0:
-            raise VmError("Failed to deassign device from IOMMU (%x:%x.%x)"
-                          % (bus, slot, func))
-        log.debug("pci: Deassign device %x:%x.%x" % (bus, slot, func))
+        # qemu would do following in case of HVM
+        if not self.vm.info.is_hvm():
+            pci_str = "0x%x, 0x%x, 0x%x, 0x%x" % (domain, bus, slot, func)
+            bdf = xc.deassign_device(fe_domid, pci_str)
+            if bdf > 0:
+                raise VmError("Failed to deassign device from IOMMU (%x:%x.%x)"
+                              % (bus, slot, func))
+            log.debug("pci: Deassign device %x:%x.%x" % (bus, slot, func))
 
         for (start, size) in dev.ioports:
             log.debug('pci: disabling ioport 0x%x/0x%x'%(start,size))

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

#1 lore raw

I think Yuji has a patch pending for qemu, not yet applied by Ian. That should fix this issue for -rc3.

-- Keir

Quoted 2 lines · Edwin

Fix a bug of HVM PCI hot removal failure introduced in r19573. xend's deassign_device should not be called for HVM, as qemu already do so.

#2 lore raw

Sorry, I neglected this patch:(

Do we have plan to improve the coexistence of qemu and xen?
You know, sometimes 2 coupled repos make debug difficult given that qemu has more and more features(hotplug, power). How about adding a bridge between qemu's git and xen's hg, so that all the changes can be tracked easily.

Thanks,

Quoted 14 lines · Keir Fraser

I think Yuji has a patch pending for qemu, not yet applied by Ian. That should fix this issue for -rc3.

-- Keir

Quoted 3 lines · Edwin

Fix a bug of HVM PCI hot removal failure introduced in r19573. xend's deassign_device should not be called for HVM, as qemu already do so.

Signature · 2 lines
best rgds,
edwin
#3 lore raw
Quoted 7 lines · Edwin

Sorry, I neglected this patch:(

Do we have plan to improve the coexistence of qemu and xen? You know, sometimes 2 coupled repos make debug difficult given that qemu has more and more features(hotplug, power). How about adding a bridge between qemu's git and xen's hg, so that all the changes can be tracked easily.

How might that work?

-- Keir

#4 lore raw
Quoted 11 lines · Keir Fraser
Quoted 7 lines · Edwin

Sorry, I neglected this patch:(

Do we have plan to improve the coexistence of qemu and xen? You know, sometimes 2 coupled repos make debug difficult given that qemu has more and more features(hotplug, power). How about adding a bridge between qemu's git and xen's hg, so that all the changes can be tracked easily.

How might that work?

Just my thoughts, not sure if it's workable:)
* Ian maintains his git repo background, other developers just see a hg repo including qemu
* Keep sync between git qemu and hg qemu:
1. Any patches including qemu changes go into git repo
automatically. Can scripts do this?
2. When rebase qemu, Ian generate a big patch against previous git
repo, then push it back to hg repo.

I have no idea of current qemu rebase process, not sure if any extra work load needed. Ian can comment.

Thanks,

Quoted 4 lines

-- Keir

Signature · 2 lines
best rgds,
edwin
#5 lore raw
Quoted 22 lines · Edwin
Quoted 10 lines · Keir Fraser
Quoted 7 lines · Edwin

Sorry, I neglected this patch:(

Do we have plan to improve the coexistence of qemu and xen? You know, sometimes 2 coupled repos make debug difficult given that qemu has more and more features(hotplug, power). How about adding a bridge between qemu's git and xen's hg, so that all the changes can be tracked easily.

How might that work?

Just my thoughts, not sure if it's workable:)
* Ian maintains his git repo background, other developers just see a hg repo including qemu
* Keep sync between git qemu and hg qemu:
1. Any patches including qemu changes go into git repo
automatically. Can scripts do this?
2. When rebase qemu, Ian generate a big patch against previous git
repo, then push it back to hg repo.

Quoted 3 lines

I have no idea of current qemu rebase process, not sure if any extra work load needed. Ian can comment.

A two way synchronization between the qemu hg mirror and the qemu git repo would mean trouble.

Maybe we could find a way to mirror the qemu git repo inside the xen-unstable hg repo, but it would still just a mirror.

#6 lore raw

Repositories and build system

Stefano Stabellini writes ("Re: [Xen-devel] [PATCH] [Xend] Fix HVM PCI hot removal failure"):

Quoted 5 lines

A two way synchronization between the qemu hg mirror and the qemu git repo would mean trouble.

Maybe we could find a way to mirror the qemu git repo inside the xen-unstable hg repo, but it would still just a mirror.

I agree that the current situation isn't ideal. I had hoped that we could avoid constantly making changes that introduce cross-repository incompatibilities but it seems to be too difficult to do that. We keep introducing new lockstep changes.

Stefano's idea of mirroring the qemu git repo inside the xen-unstable hg repo would be one approach. It could probably be done, although it would be a novel arrangement.

I think a better thing to try would be to make it easier for people to pull and update all of the sub-repos at once. It's not just qemu - we have linux (which is in at least two major branches: 2.6.18 and pvops) and now some of the Ocaml work too. Perhaps it's time to create a meta-component which permits make pull', make world', etc. ?

Ian.

#7 lore raw

RE: Repositories and build system

A few thoughts:

I fear that the real issue is that knowledge of the locksteps are hidden in people's brains, or at best in cryptic comments in some sub-repos. For any solution to work, this knowledge needs to be made available, at least in human-readable format and preferably (or perhaps additionally) in machine-parseable format.

The problem is that each developer is working on a different component and has little interest in the other components other than to ensure they don't break "the component I care about". (And sometimes a developer cares about two or more components... and may become the "lockstep culprit".) Furthermore, there are sometimes "partial locksteps", where say, a new qemu is required beyond this changeset in Xen, but only if you are using stub domains.

Generally, I think the idea of a meta-pull and meta-make are good (and maybe a meta-install?), but in order to be useful, they need to work not only on "latest" but older branches of components as well. For example:

"Pull and build all latest known stable components that work with xen-unstable changeset 19314"

Without the "lockstep database", this is impossible. And without this functionality, binary searching for a problem is also impossible.

Quoted 39 lines

-----Original Message-----
From: Ian Jackson [mailto:[email protected]]
Sent: Wednesday, April 29, 2009 9:42 AM
To: Stefano Stabellini
Cc: Yuji Shimada; Xen Developers; Keir Fraser; Zhai, Edwin
Subject: [Xen-devel] Repositories and build system

Stefano Stabellini writes ("Re: [Xen-devel] [PATCH] [Xend] Fix HVM PCI hot removal failure"):

Quoted 1 line

A two way synchronization between the qemu hg mirror and

the qemu git

Quoted 4 lines

repo would mean trouble.

Maybe we could find a way to mirror the qemu git repo inside the xen-unstable hg repo, but it would still just a mirror.

I agree that the current situation isn't ideal. I had hoped that we could avoid constantly making changes that introduce cross-repository incompatibilities but it seems to be too difficult to do that. We keep introducing new lockstep changes.

Stefano's idea of mirroring the qemu git repo inside the xen-unstable hg repo would be one approach. It could probably be done, although it would be a novel arrangement.

I think a better thing to try would be to make it easier for people to pull and update all of the sub-repos at once. It's not just qemu - we have linux (which is in at least two major branches: 2.6.18 and pvops) and now some of the Ocaml work too. Perhaps it's time to create a meta-component which permits make pull', make world', etc. ?

Ian.

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

#8 lore raw

RE: Repositories and build system

Dan Magenheimer writes ("RE: [Xen-devel] Repositories and build system"):

Quoted 5 lines

Generally, I think the idea of a meta-pull and meta-make are good (and maybe a meta-install?), but in order to be useful, they need to work not only on "latest" but older branches of components as well. For example:

I would expect that make pull' would pull from whatever branch you checked out. So if you had checked out xen-unstable, make pull' would pull Xen and dom0 and qemu and ocaml etc.

The result would be that if you pulled you'd necessarily get a new set of compatible stuff, just as if it had been in a single tree and you'd pulled that.

Quoted 2 lines

"Pull and build all latest known stable components that work with xen-unstable changeset 19314"

That would require formal, machine-readable tracking of the inter-version dependencies. Since at the moment we can't reliably accidentally avoid introducing cross-version breakage I don't think we can reliably avoid buggy metadata either.

Ian.

#9 lore raw

RE: Repositories and build system

Quoted 7 lines
Quoted 2 lines

"Pull and build all latest known stable components that work with xen-unstable changeset 19314"

That would require formal, machine-readable tracking of the inter-version dependencies. Since at the moment we can't reliably accidentally avoid introducing cross-version breakage I don't think we can reliably avoid buggy metadata either.

Understood. But it is a big loss to be unable to do a bisect search for a bug. With the proliferation of repositories and components, it may become necessary to impose some discipline to avoid the introduction of (undocumented) locksteps.

#10 lore raw

Re: Repositories and build system

Quoted 9 lines · Dan Magenheimer
Quoted 4 lines

That would require formal, machine-readable tracking of the inter-version dependencies. Since at the moment we can't reliably accidentally avoid introducing cross-version breakage I don't think we can reliably avoid buggy metadata either.

Understood. But it is a big loss to be unable to do a bisect search for a bug. With the proliferation of repositories and components, it may become necessary to impose some discipline to avoid the introduction of (undocumented) locksteps.

Fixing the interface between qemu and xen repos would be nice. Perhaps we could do that for all but device passthru (which I think was all that broke in this case). Passthru is still the main thing that is churning at a furious rate (and sensibly so imo since I think some bits of logic are still in the wrong places, and some interfaces are probably still not great).

-- Keir

#11 lore raw

Re: Repositories and build system

In reply to Dan Magenheimer
Quoted 7 lines · Dan Magenheimer
Quoted 6 lines
Quoted 2 lines

"Pull and build all latest known stable components that work with xen-unstable changeset 19314"

That would require formal, machine-readable tracking of the inter-version dependencies. Since at the moment we can't reliably accidentally avoid introducing cross-version breakage I don't think we can reliably avoid buggy metadata either.

Quoted 5 lines

Understood. But it is a big loss to be unable to do a bisect search for a bug. With the proliferation of repositories and components, it may become necessary to impose some discipline to avoid the introduction of (undocumented) locksteps.

I agree.
In fact I think we should start to write in clear letters when a commit breaks xen\qemu compatibility, specifying the required changeset in the other repository.
We could formally introduce this practice with the 3.5 development cycle.

#12 lore raw

Re: Repositories and build system

Quoted 16 lines · Stefano Stabellini
Quoted 7 lines · Dan Magenheimer
Quoted 6 lines
Quoted 2 lines

"Pull and build all latest known stable components that work with xen-unstable changeset 19314"

That would require formal, machine-readable tracking of the inter-version dependencies. Since at the moment we can't reliably accidentally avoid introducing cross-version breakage I don't think we can reliably avoid buggy metadata either.

Quoted 5 lines

Understood. But it is a big loss to be unable to do a bisect search for a bug. With the proliferation of repositories and components, it may become necessary to impose some discipline to avoid the introduction of (undocumented) locksteps.

Quoted 5 lines

I agree.
In fact I think we should start to write in clear letters when a commit breaks xen\qemu compatibility, specifying the required changeset in the other repository.
We could formally introduce this practice with the 3.5 development cycle.

This would certainly be an improvement, but it depends on the developer always knowing and recording exactly what synchronization is needed. This doesn't quite jibe with bisect, whose raison d'etre is to help find problems that went unnoticed for potentially long periods of time. Even when the hints work, bisection is still quite painful across two repositories in two different version control systems.

#13 lore raw

Re: Repositories and build system

Quoted 6 lines · Brendan Cully
Quoted 5 lines

I agree.
In fact I think we should start to write in clear letters when a commit breaks xen\qemu compatibility, specifying the required changeset in the other repository.
We could formally introduce this practice with the 3.5 development cycle.

Quoted 6 lines

This would certainly be an improvement, but it depends on the developer always knowing and recording exactly what synchronization is needed. This doesn't quite jibe with bisect, whose raison d'etre is to help find problems that went unnoticed for potentially long periods of time. Even when the hints work, bisection is still quite painful across two repositories in two different version control systems.

We plan to turn on the QEMU_TAG mechanism always after 3.4 is branched, so that by default xen-unstable.hg will always clone a specific version of the qemu git repository. I'm sure it should be then be possible to make bisect update the foreign qemu repo as it goes quite straightforwardly.

-- Keir

#14 lore raw
In reply to Zhai, Edwin
Quoted 11 lines · Edwin

Just my thoughts, not sure if it's workable:)
* Ian maintains his git repo background, other developers just see a hg repo including qemu
* Keep sync between git qemu and hg qemu:
1. Any patches including qemu changes go into git repo
automatically. Can scripts do this?
2. When rebase qemu, Ian generate a big patch against previous git
repo, then push it back to hg repo.

I have no idea of current qemu rebase process, not sure if any extra work load needed. Ian can comment.

It'd be more convenient for everyone but Ian. :-D It does seem that consistency between Xen and qemu is going to be an ongoing issue though, since I don't think refactoring of all pci passthru stuff is done yet.

-- Keir