Skip to contentSkip to conversation

Make xend reject duplicates and rename zombies

PATCH In review Dan Smith 4 participants 9 messages Sep 15, 2005 – Sep 16, 2005

lore
Share & tools
lore

Dan Smith

author
#0 lore raw

PATCH 1/1 Make xend reject duplicates and rename zombies

This patch is an update of my previous anti-duplicate-domain patch. Now, we check an existing same-name domain to see if it's in the "terminated" state, renaming it to "zombie-domid-name" if so.

This basically makes the problem go away for me, as it gives the dying domain time to clean itself up.

Test 10_create_fastdestroy from the next release of xm-test validates that this fixes the problem.

tools/python/xen/xend/XendDomain.py +14 -0
diff -r c27431cf81f9 tools/python/xen/xend/XendDomain.py
@@ -297,6 +297,20 @@
         @param config: configuration
         @return: domain
         """
+
+        existing = self.domains.get_by_name(sxp.child_value(config, "name"))
+        if existing:
+            if existing.is_terminated():
+                newname = "zombie-%i-%s" % (existing.domid, existing.name)
+                log.debug("Renaming zombie domain %s -> %s" %
+                          (existing.name, newname))
+                existing.setName(newname)
+            else:
+                log.debug("Attempt to create duplicate domain %s" %
+                          existing.name)
+                raise XendError("Domain %s already exists as %i!" %
+                                (existing.name, existing.id))
+        
         dominfo = XendDomainInfo.create(self.dbmap, config)
         return dominfo
 
Signature · 5 lines
-- 
Dan Smith
IBM Linux Technology Center
Open Hypervisor Team
email: [email protected]

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

#1 lore raw

I think this is not the right solution to the problem at hand. The problem stems from the fact that xm destroy is just a call to xc_domain_destroy which is really just a request to the hypervisor to destroy the domain.

Therefore, there is a race condition if you assume that the domain is dead after xm destroy returns. This patch renames the domain name which prevents a name class but does not solve the general problem. Consider, for instance, if a domain is using a block device and you do an xm destroy. It is not safe to create a new domain with that same block device until you know that the previously mentioned domain is gone.

This patch would allow:

xm destroy xmexample1 && xm create /etc/xen/xmexample1

Even though it might really be conflicting. This could lead to *very* subtle device corruption down the road.

I think the right solution is to make xm destroy not return until the domain has actually gone away and add a flag to xm destroy to return immediately if that behavior is ever desired.

I'll work up a patch tonight after class.

Regards,

Anthony Liguori

Quoted 55 lines · Dan Smith

This patch is an update of my previous anti-duplicate-domain patch. Now, we check an existing same-name domain to see if it's in the "terminated" state, renaming it to "zombie-domid-name" if so.

This basically makes the problem go away for me, as it gives the dying domain time to clean itself up.

Test 10_create_fastdestroy from the next release of xm-test validates that this fixes the problem.

------------------------------------------------------------------------

diff -r c27431cf81f9 tools/python/xen/xend/XendDomain.py

tools/python/xen/xend/XendDomain.py +14 -2
@@ -297,6 +297,20 @@
        @param config: configuration
        @return: domain
        """
+
+        existing = self.domains.get_by_name(sxp.child_value(config, "name"))
+        if existing:
+            if existing.is_terminated():
+                newname = "zombie-%i-%s" % (existing.domid, existing.name)
+                log.debug("Renaming zombie domain %s -> %s" %
+                          (existing.name, newname))
+                existing.setName(newname)
+            else:
+                log.debug("Attempt to create duplicate domain %s" %
+                          existing.name)
+                raise XendError("Domain %s already exists as %i!" %
+                                (existing.name, existing.id))
+        
        dominfo = XendDomainInfo.create(self.dbmap, config)
        return dominfo
 
 
 
------------------------------------------------------------------------
 
 
 
 
------------------------------------------------------------------------

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

author
#2 lore raw

AL> This patch renames the domain name which prevents a name class but AL> does not solve the general problem.

I agree. This is why in my original patch, I simply had the create routine reject attempts to create domains with duplicate names.

Further, I will point out that what I meant by my comment attached to this patch was not that it solved the general problem, but that it seemed to make the problem not get in the way of some of my existing tests :)

AL> I think the right solution is to make xm destroy not return until AL> the domain has actually gone away and add a flag to xm destroy to AL> return immediately if that behavior is ever desired.

Yep, I think that's a much better solution. So, I think my original patch should still be applied: xend should do its own checking for duplicates, so that we don't rely on whatever tool is asking us to do so.

Signature · 4 lines
Dan Smith
IBM Linux Technology Center
Open Hypervisor Team
email: [email protected]
#3 lore raw
Quoted 11 lines · Dan Smith

AL> This patch renames the domain name which prevents a name class but AL> does not solve the general problem.

I agree. This is why in my original patch, I simply had the create routine reject attempts to create domains with duplicate names.

Further, I will point out that what I meant by my comment attached to this patch was not that it solved the general problem, but that it seemed to make the problem not get in the way of some of my existing tests :)

Quoted 9 lines

AL> I think the right solution is to make xm destroy not return until AL> the domain has actually gone away and add a flag to xm destroy to AL> return immediately if that behavior is ever desired.

Yep, I think that's a much better solution. So, I think my original patch should still be applied: xend should do its own checking for duplicates, so that we don't rely on whatever tool is asking us to do so.

Has this been applied yet?

I still have this exact issue with change set 6884.

Regards,
Ted

#4 lore raw
In reply to Anthony Liguori
Quoted 4 lines · Anthony Liguori

I think this is not the right solution to the problem at hand. The problem stems from the fact that xm destroy is just a call to xc_domain_destroy which is really just a request to the hypervisor to destroy the domain.

Indeed.

Quoted 6 lines

Therefore, there is a race condition if you assume that the domain is dead after xm destroy returns. This patch renames the domain name which prevents a name class but does not solve the general problem. Consider, for instance, if a domain is using a block device and you do an xm destroy. It is not safe to create a new domain with that same block device until you know that the previously mentioned domain is gone.

I think renaming would make sense if a domain is crashed. You might want to keep the domain around to attach a debugger to it but at the same time restart a fresh copy as soon as possible.

Quoted 6 lines

This patch would allow:
xm destroy xmexample1 && xm create /etc/xen/xmexample1

I think the right solution is to make xm destroy not return until the domain has actually gone away and add a flag to xm destroy to return immediately if that behavior is ever desired.

Sounds good.

Quoted 1 line

I'll work up a patch tonight after class.

Thanks!

christian

#5 lore raw
Quoted 23 lines · Christian Limpach
Quoted 6 lines · Anthony Liguori

I think this is not the right solution to the problem at hand. The problem stems from the fact that xm destroy is just a call to xc_domain_destroy which is really just a request to the hypervisor to destroy the domain.

Indeed.

Quoted 8 lines

Therefore, there is a race condition if you assume that the domain is dead after xm destroy returns. This patch renames the domain name which prevents a name class but does not solve the general problem. Consider, for instance, if a domain is using a block device and you do an xm destroy. It is not safe to create a new domain with that same block device until you know that the previously mentioned domain is gone.

Quoted 5 lines

I think renaming would make sense if a domain is crashed. You might want to keep the domain around to attach a debugger to it but at the same time restart a fresh copy as soon as possible.

As long as you can make sure to rename *after* all of the devices have been properly torn down.

Otherwise, we need to make sure to make it well known that restarting a domain after a crash can result in very bad things :-)

Regards,

Anthony Liguori

Quoted 23 lines
Quoted 8 lines

This patch would allow:
xm destroy xmexample1 && xm create /etc/xen/xmexample1

I think the right solution is to make xm destroy not return until the domain has actually gone away and add a flag to xm destroy to return immediately if that behavior is ever desired.

Sounds good.

Quoted 3 lines

I'll work up a patch tonight after class.

Thanks!

christian

#6 lore raw
Quoted 9 lines · Anthony Liguori
Quoted 3 lines

I think renaming would make sense if a domain is crashed. You might want to keep the domain around to attach a debugger to it but at the same time restart a fresh copy as soon as possible.

As long as you can make sure to rename *after* all of the devices have been properly torn down.

Otherwise, we need to make sure to make it well known that restarting a domain after a crash can result in very bad things :-)

How so? The crashed domain is certainly not going to do anything with those devices anymore... Actually, I'm not even sure I buy the "unsafe to reuse devices" argument if the domain is only hanging around in an almost dead state -- it's almost dead, it's not going to do any requests on those devices anymore...

So regarding the check for termination in check_name, if we only switch the state to terminated once the domain is almost dead, reusing the name should be fine...

christian

#7 lore raw
Quoted 16 lines · Christian Limpach
Quoted 5 lines · Anthony Liguori
Quoted 5 lines

I think renaming would make sense if a domain is crashed. You might want to keep the domain around to attach a debugger to it but at the same time restart a fresh copy as soon as possible.

Quoted 7 lines

As long as you can make sure to rename *after* all of the devices have been properly torn down.

Otherwise, we need to make sure to make it well known that restarting a domain after a crash can result in very bad things :-)

Quoted 7 lines

How so? The crashed domain is certainly not going to do anything with those devices anymore... Actually, I'm not even sure I buy the "unsafe to reuse devices" argument if the domain is only hanging around in an almost dead state -- it's almost dead, it's not going to do any requests on those devices anymore...

I'm thinking specifically about block devices. While the domain may have died, there may be requests in the block devices queues that have not been flushed to disk yet. This is what would lead to problems..

Regards,

Anthony Liguori

Quoted 8 lines

So regarding the check for termination in check_name, if we only switch the state to terminated once the domain is almost dead, reusing the name should be fine...

christian

#8 lore raw
Quoted 3 lines · Anthony Liguori

I'm thinking specifically about block devices. While the domain may have died, there may be requests in the block devices queues that have not been flushed to disk yet. This is what would lead to problems..

sure, but the logic which prevents you from using the same block device multiple times should kick in at this point -- granted, that code's a bit broken right now...

christian