Skip to contentSkip to conversation

xend: disallow multiple destroy() call

PATCH In review Joe Jin 4 participants 10 messages Feb 4, 2013 – Mar 19, 2013

lore
Share & tools
lore

Joe Jin

author
#0 lore raw

PATCH 1/1 xend: disallow multiple destroy() call

Current Xend allowing multiple call destroy() for same domain, this lead multiple hard resets(FLR) for pci pass-through, and some controller might failed.

In our test, we pass through 2 LSI HAB controllers to the PVHVM guest, after guest brought up, call xm-destroy twice, the adapters's BIOS will hung, and we had to reboot the server to recovery it.

Cc Konrad Rzeszutek Wilk <[email protected]>
Cc Ian Campbell <[email protected]>
---
 tools/python/xen/xend/XendDomainInfo.py | 11 +++++++++++
 1 file changed, 11 insertions(+)
tools/python/xen/xend/XendDomainInfo.py +11 -0
index e9d3e7e..658c3b6 100644
@@ -367,6 +367,8 @@ class XendDomainInfo:
     @type refresh_shutdown_lock: threading.Condition
     @ivar _deviceControllers: device controller cache for this domain
     @type _deviceControllers: dict 'string' to DevControllers
+    @ivar destroying: Is this domain destroying
+    @type destroying: bool
     """
     
     def __init__(self, info, domid = None, dompath = None, augment = False,
@@ -455,6 +457,8 @@ class XendDomainInfo:
         self._checkName(self.info['name_label'])
 
         self.metrics = XendVMMetrics(uuid.createString(), self)
+
+        self.destroying = False
             
 
     #
@@ -3073,6 +3077,12 @@ class XendDomainInfo:
 
         if self.domid is None:
             return
+
+        if self.destroying == False:
+            self.destroying = True
+        else:
+            raise VmError("Domain (domid=%s) is destroying, please wait!", str(self.domid))
+
         from xen.xend import XendDomain
         log.debug("XendDomainInfo.destroy: domid=%s", str(self.domid))
 
@@ -3088,6 +3098,7 @@ class XendDomainInfo:
                     self.info[state] = 0
                 self._stateSet(DOM_STATE_HALTED)
             except:
+                self.destroying = False
                 log.exception("XendDomainInfo.destroy: domain destruction failed.")
 
             XendDomain.instance().remove_domain(self)
Signature · 1 line
1.8.1
#1 lore raw
Quoted 7 lines · Joe Jin

Current Xend allowing multiple call destroy() for same domain, this lead multiple hard resets(FLR) for pci pass-through, and some controller might failed.

In our test, we pass through 2 LSI HAB controllers to the PVHVM guest, after guest brought up, call xm-destroy twice, the adapters's BIOS will hung, and we had to reboot the server to recovery it.

Does the same problem happen with libxl/xl?

author
#2 lore raw
Quoted 10 lines · Roger Pau Monné
Quoted 7 lines · Joe Jin

Current Xend allowing multiple call destroy() for same domain, this lead multiple hard resets(FLR) for pci pass-through, and some controller might failed.

In our test, we pass through 2 LSI HAB controllers to the PVHVM guest, after guest brought up, call xm-destroy twice, the adapters's BIOS will hung, and we had to reboot the server to recovery it.

Does the same problem happen with libxl/xl?

execute xl-destroy twice crashed my server!

Thanks,
Joe

#3 lore raw
Quoted 1 line · Joe Jin

execute xl-destroy twice crashed my server!

Can you give more details please.

Ian.

author
#4 lore raw
Quoted 4 lines · Ian Campbell
Quoted 1 line · Joe Jin
Quoted 1 line

execute xl-destroy twice crashed my server!

Can you give more details please.

Hi Ian,

Sorry the server not available for test now.

Would you please help to review my patch for xm-destroy?

Thanks,
Joe

author
#6 lore raw
In reply to Ian Campbell

I got serial console output now and mostly like panic happened by "xl create":

(XEN) HVM1: Detected Xen v4.1.2-OVM
(XEN) HVM1: CPU speed is 2926 MHz
(XEN) HVM1: Xenbus rings @0xfeffc000, event channel 13 (XEN) irq.c:264: Dom1 PCI link 0 changed 0 -> 5
(XEN) HVM1: PCI-ISA link 0 routed to IRQ5
(XEN) irq.c:264: Dom1 PCI link 1 changed 0 -> 10
(XEN) HVM1: PCI-ISA link 1 routed to IRQ10
(XEN) irq.c:264: Dom1 PCI link 2 changed 0 -> 11
(XEN) HVM1: PCI-ISA link 2 routed to IRQ11
(XEN) irq.c:264: Dom1 PCI link 3 changed 0 -> 5
(XEN) HVM1: PCI-ISA link 3 routed to IRQ5
(XEN) HVM1: pci dev 01:3 INTA->IRQ10
(XEN) HVM1: pci dev 03:0 INTA->IRQ5
(XEN) HVM1: pci dev 04:0 INTA->IRQ5
(XEN) HVM1: pci dev 05:0 INTA->IRQ10
(XEN) HVM1: pci dev 02:0 bar 10 size 02000000: f0000008 (XEN) HVM1: pci dev 03:0 bar 14 size 01000000: f2000008 (XEN) HVM1: pci dev 04:0 bar 30 size 00080000: f3000000 (XEN) HVM1: pci dev 05:0 bar 30 size 00080000: f3080000 (XEN) domctl.c:985:d0 memory_map:add: gfn=f3100 mfn=df040 nr_mfns=40 (XEN) HVM1: pci dev 04:0 bar 1c size 00040000: f3100004 (XEN) domctl.c:985:d0 memory_map:add: gfn=f3140 mfn=df640 nr_mfns=40 (XEN) HVM1: pci dev 05:0 bar 1c size 00040000: f3140004 (XEN) domctl.c:985:d0 memory_map:add: gfn=f3180 mfn=df03c nr_mfns=4 (XEN) domctl.c:995:d0 memory_map:remove: gfn=f3182 mfn=df03e nr_mfns=1 (XEN) HVM1: pci dev 04:0 bar 14 size 00004000: f3180004 (XEN) domctl.c:985:d0 memory_map:add: gfn=f3184 mfn=df63c nr_mfns=4 (XEN) domctl.c:995:d0 memory_map:remove: gfn=f3186 mfn=df63e nr_mfns=1 (XEN) HVM1: pci dev 05:0 bar 14 size 00004000: f3184004 (XEN) HVM1: pci dev 02:0 bar 14 size 00001000: f3188000 (XEN) HVM1: pci dev 03:0 bar 10 size 00000100: 0000c001 (XEN) HVM1: pci dev 04:0 bar 10 size 00000100: 0000c101 (XEN) domctl.c:1041:d0 ioport_map:add f_gport=c100 f_mport=9000 np=100 (XEN) HVM1: pci dev 05:0 bar 10 size 00000100: 0000c201 (XEN) domctl.c:1041:d0 ioport_map:add f_gport=c200 f_mport=d000 np=100 (XEN) HVM1: pci dev 01:1 bar 20 size 00000010: 0000c301 (XEN) HVM1: Multiprocessor initialisation:

(XEN) HVM1:  - CPU0 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU1 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU2 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU3 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU4 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU5 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU6 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU7 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU8 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU9 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU10 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU11 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.

(XEN) HVM1: Writing SMBIOS tables ...
(XEN) HVM1: Loading ROMBIOS ...
(XEN) HVM1: 10364 bytes of ROMBIOS high-memory extensions:
(XEN) HVM1: Relocating to 0xfc000000-0xfc00287c ... done
(XEN) HVM1: Creating MP tables ...
(XEN) HVM1: Loading Cirrus VGABIOS ...
(XEN) domctl.c:985:d0 memory_map:add: gfn=f3000 mfn=df080 nr_mfns=80 (XEN) HVM1: Loading PCI Option ROM ...
(XEN) HVM1: - Manufacturer: LSI Corporation
(XEN) HVM1: - Product name: LSI MPI Boot Support
(XEN) domctl.c:995:d0 memory_map:remove: gfn=f3000 mfn=df080 nr_mfns=80 (XEN) domctl.c:985:d0 memory_map:add: gfn=f3080 mfn=df680 nr_mfns=80 (XEN) domctl.c:995:d0 memory_map:remove: gfn=f3080 mfn=df680 nr_mfns=80 (XEN) HVM1: Loading ACPI ...
(XEN) HVM1: - Lo data: 000ea020-000ea04f
(XEN) HVM1: - Hi data: fc002c00-fc00eb4f
(XEN) HVM1: vm86 TSS at fc00ec00
(XEN) HVM1: BIOS map:

(XEN) HVM1:  c0000-c8fff: VGA BIOS
(XEN) HVM1:  c9000-d4fff: PCI Option ROMs
(XEN) HVM1:  eb000-eb302: SMBIOS tables
(XEN) HVM1:  f0000-fffff: Main BIOS
(XEN) HVM1: E820 table:
(XEN) HVM1:  [00]: 00000000:00000000 - 00000000:0009e000: RAM
(XEN) HVM1:  [01]: 00000000:0009e000 - 00000000:0009fc00: RESERVED
(XEN) HVM1:  [02]: 00000000:0009fc00 - 00000000:000a0000: RESERVED
(XEN) HVM1:  HOLE: 00000000:000a0000 - 00000000:000e0000
(XEN) HVM1:  [03]: 00000000:000e0000 - 00000000:00100000: RESERVED
(XEN) HVM1:  [04]: 00000000:00100000 - 00000000:f0000000: RAM
(XEN) HVM1:  HOLE: 00000000:f0000000 - 00000000:fc000000
(XEN) HVM1:  [05]: 00000000:fc000000 - 00000001:00000000: RESERVED
(XEN) HVM1:  [06]: 00000001:00000000 - 00000004:0f800000: RAM

(XEN) HVM1: Invoking ROMBIOS ...
(XEN) HVM1: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $ (XEN) stdvga.c:147:d1 entering stdvga and caching modes
(XEN) HVM1: VGABios $Id: vgabios.c,v 1.67 2008/01/27 09:44:12 vruppert Exp $ (XEN) HVM1: Bochs BIOS - build: 06/23/99
(XEN) HVM1: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $ (XEN) HVM1: Options: apmbios pcibios eltorito PMM
(XEN) HVM1:
(XEN) HVM1: ata0-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 (XEN) HVM1: ata0 master: QEMU HARDDISK ATA-7 Hard-Disk (60000 MBytes) (XEN) HVM1: ata0-1: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 (XEN) HVM1: ata0 slave: QEMU HARDDISK ATA-7 Hard-Disk ( 97 GBytes) (XEN) HVM1: ata1-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 (XEN) HVM1: ata1 master: QEMU HARDDISK ATA-7 Hard-Disk (20000 MBytes) (XEN) HVM1: IDE time out
(XEN) HVM1:
(XEN) HVM1: PCI device 1000:0070 not found at index 0
(XEN) HVM1: PCI device 1000:0072 not found at index 2
(XEN) HVM1: PCI device 1000:0074 not found at index 0
(XEN) HVM1: PCI device 1000:0076 not found at index 0
(XEN) HVM1: PCI device 1000:0077 not found at index 0
(XEN) HVM1: PCI device 1000:0064 not found at index 0
(XEN) HVM1: PCI device 1000:0065 not found at index 0
(XEN) HVM1: PCI device 1000:0080 not found at index 0
(XEN) HVM1: PCI device 1000:0081 not found at index 0
(XEN) HVM1: PCI device 1000:0082 not found at index 0
(XEN) HVM1: PCI device 1000:0083 not found at index 0
(XEN) HVM1: PCI device 1000:0084 not found at index 0
(XEN) HVM1: PCI device 1000:0085 not found at index 0
(XEN) HVM1: PCI device 1000:0086 not found at index 0
(XEN) HVM1: PCI device 1000:0087 not found at index 0
(XEN) HVM1: PCI device 1000:0070 not found at index 0
(XEN) HVM1: PCI device 1000:0072 not found at index 2
(XEN) HVM1: PCI device 1000:0074 not found at index 0
(XEN) HVM1: PCI device 1000:0076 not found at index 0
(XEN) HVM1: PCI device 1000:0077 not found at index 0
(XEN) HVM1: PCI device 1000:0064 not found at index 0
(XEN) HVM1: PCI device 1000:0065 not found at index 0
(XEN) HVM1: PCI device 1000:0080 not found at index 0
(XEN) HVM1: PCI device 1000:0081 not found at index 0
(XEN) HVM1: PCI device 1000:0082 not found at index 0
(XEN) HVM1: PCI device 1000:0083 not found at index 0
(XEN) HVM1: PCI device 1000:0084 not found at index 0
(XEN) HVM1: PCI device 1000:0085 not found at index 0
(XEN) HVM1: PCI device 1000:0086 not found at index 0
(XEN) HVM1: PCI device 1000:0087 not found at index 0
(XEN) HVM1:
(XEN) HVM1:
(XEN) HVM1: Press F12 for boot menu.
(XEN) HVM1:
(XEN) HVM1: Booting from Hard Disk...
(XEN) HVM1: Booting from 0000:7c00
(XEN) HVM1: *** int 15h function AX=00c0, BX=0000 not yet supported! (XEN) HVM1: *** int 15h function AX=ec00, BX=0002 not yet supported! (XEN) HVM1: KBD: unsupported int 16h function 03
(XEN) HVM1: *** int 15h function AX=e980, BX=0000 not yet supported! (XEN) irq.c:330: Dom1 callback via changed to Direct Vector 0xe9 (XEN) domctl.c:1065:d0 ioport_map:remove f_gport=c100 f_mport=9000 np=100 (XEN) domctl.c:1041:d0 ioport_map:add f_gport=c100 f_mport=9000 np=100 (XEN) domctl.c:1065:d0 ioport_map:remove f_gport=c200 f_mport=d000 np=100 (XEN) domctl.c:1041:d0 ioport_map:add f_gport=c200 f_mport=d000 np=100 (XEN) irq.c:264: Dom1 PCI link 0 changed 5 -> 0
(XEN) irq.c:264: Dom1 PCI link 1 changed 10 -> 0
(XEN) irq.c:264: Dom1 PCI link 2 changed 11 -> 0
(XEN) irq.c:264: Dom1 PCI link 3 changed 5 -> 0

------------[ cut here ]------------
WARNING: at fs/proc/generic.c:850 remove_proc_entry+0x22d/0x240()
Hardware name: SUN FIRE X4370 M2 SERVER       
remove_proc_entry: removing non-empty directory 'irq/536', leaking at least 'eth7'
Modules linked in: xt_physdev iptable_filter ip_tables xen_blkback xen_netback xen_pciback xen_gntdev xen_evtchn ipmi_devintf ipmi_si lockd sunrpc bridge stp llc bonding dm_round_robin dm_multipath be2iscsi iscsi_boot_sysfs ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp bnx2i cnic uio ipv6 cxgb3i libcxgbi cxgb3 libiscsi_tcp libiscsi scsi_transport_iscsi xenfs xen_privcmd dm_mirror video sbs sbshc acpi_memhotplug acpi_ipmi ipmi_msghandler parport_pc lp parport ses enclosure ixgbe igb e1000e mdio snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd soundcore snd_page_alloc i2c_i801 i2c_core i7core_edac iTCO_wdt pcspkr ioatdma iTCO_vendor_support edac_core dca ghes hed dm_region_hash dm_log dm_mod usb_storage s
 hpchp sg ahci libahci sd_mod crc_t10dif raid1 ext3 jbd mbcache [last unloaded: raid_class]
Pid: 19329, comm: qemu-dm Not tainted 2.6.39-200.1.15.el5uek #1
Call Trace:
 [<ffffffff811d10dd>] ? remove_proc_entry+0x22d/0x240
 [<ffffffff8106de10>] warn_slowpath_common+0x90/0xc0
 [<ffffffff8106df3e>] warn_slowpath_fmt+0x6e/0x70
 [<ffffffff8100a7cf>] ? xen_restore_fl_direct_reloc+0x4/0x4
 [<ffffffff8125cf98>] ? sprintf+0x68/0x70
 [<ffffffff811cfef1>] ? __xlate_proc_name+0x41/0xc0
 [<ffffffff81501b9e>] ? _raw_spin_lock+0xe/0x20
 [<ffffffff811d10dd>] remove_proc_entry+0x22d/0x240
 [<ffffffff8113232c>] ? zap_pte_range+0x1ec/0x400
 [<ffffffff810d9430>] unregister_irq_proc+0xd0/0xf0
 [<ffffffff810d571a>] free_desc+0x2a/0x70
 [<ffffffff810d579e>] irq_free_descs+0x3e/0x80
 [<ffffffff812f7822>] xen_free_irq+0x52/0x70
 [<ffffffff812f82cb>] unbind_from_irq+0xeb/0x180
 [<ffffffff81009f0d>] ? xen_force_evtchn_callback+0xd/0x10
 [<ffffffff812f837f>] evtchn_put+0x1f/0x40
 [<ffffffffa055b4e9>] gntdev_put_map+0x39/0x110 [xen_gntdev]
 [<ffffffff812640e1>] ? list_del+0x11/0x40
 [<ffffffffa055c38c>] gntdev_ioctl_unmap_grant_ref+0xac/0xd0 [xen_gntdev]
 [<ffffffffa055c708>] gntdev_ioctl+0x98/0xa0 [xen_gntdev]
 [<ffffffff8117eb8d>] vfs_ioctl+0x1d/0x50
 [<ffffffff8117f623>] do_vfs_ioctl+0x63/0x1b0
 [<ffffffff8113a740>] ? do_munmap+0x240/0x280
 [<ffffffff8117f804>] sys_ioctl+0x94/0xa0
 [<ffffffff8150a242>] system_call_fastpath+0x16/0x1b
---[ end trace fa61ba514bac93c7 ]---
BUG: unable to handle kernel NULL pointer dereference at 000000000000001c
IP: [<ffffffff812f6a76>] evtchn_from_irq+0x16/0x40
PGD 6a2cb067 PUD 71d8c067 PMD 0 
Oops: 0000 [#1] SMP 
CPU 0 
Modules linked in: xt_physdev iptable_filter ip_tables xen_blkback xen_netback xen_pciback xen_gntdev xen_evtchn ipmi_devintf ipmi_si lockd sunrpc bridge stp llc bonding dm_round_robin dm_multipath be2iscsi iscsi_boot_sysfs ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp bnx2i cnic uio ipv6 cxgb3i libcxgbi cxgb3 libiscsi_tcp libiscsi scsi_transport_iscsi xenfs xen_privcmd dm_mirror video sbs sbshc acpi_memhotplug acpi_ipmi ipmi_msghandler parport_pc lp parport ses enclosure ixgbe igb e1000e mdio snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd soundcore snd_page_alloc i2c_i801 i2c_core i7core_edac iTCO_wdt pcspkr ioatdma iTCO_vendor_support edac_core dca ghes hed dm_region_hash dm_log dm_mod usb_storage s
 hpchp sg ahci libahci sd_mod crc_t10dif raid1 ext3 jbd mbcache [last unloaded: raid_class]

Pid: 19329, comm: qemu-dm Tainted: G W 2.6.39-200.1.15.el5uek #1 ORACLE CORPORATION SUN FIRE X4370 M2 SERVER /ASSY,SC_BD,T4

RIP: e030:[<ffffffff812f6a76>]  [<ffffffff812f6a76>] evtchn_from_irq+0x16/0x40
RSP: e02b:ffff88006cfc5dd8  EFLAGS: 00010246
RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffff88007e38be48
RDX: 0000000000000000 RSI: 0000000000000218 RDI: ffff88007e38bd70
RBP: ffff88006cfc5dd8 R08: 0000000000000001 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000246 R12: 0000000000000218
R13: 0000000000000000 R14: 0000000000000017 R15: 0000000000104701
FS:  00007ff36f4046e0(0000) GS:ffff88007f49a000(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 000000000000001c CR3: 000000005f445000 CR4: 0000000000002660
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process qemu-dm (pid: 19329, threadinfo ffff88006cfc4000, task ffff880067084200)
Stack:
 ffff88006cfc5e38 ffffffff812f81fc 00007ff36e620000 0000003e38e10000
 ffff88006cfc5e38 000000000000014f 0000000000000206 ffff88006b2e8138
 ffffffff81009f0d ffff88006b951cc0 ffff88006b951cc0 0000000000000000
Call Trace:
 [<ffffffff812f81fc>] unbind_from_irq+0x1c/0x180
 [<ffffffff81009f0d>] ? xen_force_evtchn_callback+0xd/0x10
 [<ffffffff812f837f>] evtchn_put+0x1f/0x40
 [<ffffffffa055b4e9>] gntdev_put_map+0x39/0x110 [xen_gntdev]
 [<ffffffff812640e1>] ? list_del+0x11/0x40
 [<ffffffffa055c38c>] gntdev_ioctl_unmap_grant_ref+0xac/0xd0 [xen_gntdev]
 [<ffffffffa055c708>] gntdev_ioctl+0x98/0xa0 [xen_gntdev]
 [<ffffffff8117eb8d>] vfs_ioctl+0x1d/0x50
 [<ffffffff8117f623>] do_vfs_ioctl+0x63/0x1b0
 [<ffffffff8113a740>] ? do_munmap+0x240/0x280
 [<ffffffff8117f804>] sys_ioctl+0x94/0xa0
 [<ffffffff8150a242>] system_call_fastpath+0x16/0x1b
Code: 8b 64 24 08 c9 c3 0f 1f 80 00 00 00 00 0f 1f 84 00 00 00 00 00 55 48 89 e5 66 66 66 66 90 39 3d f1 16 4c 00 76 0b e8 ea fa ff ff <0f> b7 40 1c c9 c3 89 f9 31 c0 48 c7 c2 2f 72 70 81 be d1 00 00 
RIP  [<ffffffff812f6a76>] evtchn_from_irq+0x16/0x40
 RSP <ffff88006cfc5dd8>

CR2: 000000000000001c

Thanks,
Joe

Quoted 9 lines · Ian Campbell
Quoted 1 line · Joe Jin

execute xl-destroy twice crashed my server!

Can you give more details please.

Ian.

author
#7 lore raw

Execute "xl-destroy" twice lead the dom0 panic:
config file as below:

vncunused = 1
kernel = '/usr/lib/xen/boot/hvmloader'
vnc = 1
name = 'test'
memory = '16384'
#memory = '1024'
timer_mode = 0
device_model = '/usr/lib64/xen/bin/qemu-dm'
builder = 'hvm'
vnclisten = '0.0.0.0'
cpus = '0,2,4,6,8,11,1,3,5,7,9,10'
#on_crash = 'coredump-restart'
#on_crash = 'preserve'
on_crash = 'destroy'
on_reboot = 'restart'
vcpus = 12
pci = ['0d:00.0', '1f:00.0']
#pci = ['0d:00.0']
pae = 1
apic = 1
vif = ['type=netfront,bridge=priv1', 'type=netfront,bridge=net1', 'type=netfront,bridge=net2', 'type=netfront,bridge=net3', 'type=netfront,bridge=net4'] serial = 'pty'
disk = ['file:/path/System.img,xvda,w', 'file:/path/u01.img,xvdb,w', 'file:/path/swap.img,xvdc,w']
acpi = 1

Crash info from serial console:

(XEN) domctl.c:1065:d0 ioport_map:remove f_gport=c100 f_mport=9000 np=100 (XEN) domctl.c:995:d0 memory_map:remove: gfn=f3180 mfn=df03c nr_mfns=4 (XEN) p2m.c:2797:d0 clear_mmio_p2m_entry: gfn_to_mfn failed! gfn=000f3182 (XEN) domctl.c:995:d0 memory_map:remove: gfn=f3100 mfn=df040 nr_mfns=40

------------[ cut here ]------------
kernel BUG at drivers/pci/access.c:423!
invalid opcode: 0000 [#1] SMP 
CPU 18 
Modules linked in: xt_physdev iptable_filter ip_tables xen_blkback xen_netback xen_pciback xen_gntdev xen_evtchn ipmi_devintf ipmi_si lockd sunrpc bridge stp llc bonding dm_round_robin dm_multipath be2iscsi iscsi_boot_sysfs ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp bnx2i cnic uio ipv6 cxgb3i libcxgbi cxgb3 libiscsi_tcp libiscsi scsi_transport_iscsi xenfs xen_privcmd dm_mirror video sbs sbshc acpi_memhotplug acpi_ipmi ipmi_msghandler parport_pc lp parport igb ses enclosure ixgbe mdio e1000e snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd soundcore i2c_i801 snd_page_alloc ghes iTCO_wdt i2c_core pcspkr iTCO_vendor_support i7core_edac ioatdma edac_core hed dca dm_region_hash dm_log dm_mod usb_storage s
 hpchp sg ahci libahci sd_mod crc_t10dif raid1 ext3 jbd mbcache [last unloaded: raid_class]

Pid: 20399, comm: xl Not tainted 2.6.39-200.1.15.el5uek #1 ORACLE CORPORATION SUN FIRE X4370 M2 SERVER /ASSY,SC_BD,T4

RIP: e030:[<ffffffff812704bf>]  [<ffffffff812704bf>] pci_block_user_cfg_access+0x4f/0x60
RSP: e02b:ffff88006e609dd8  EFLAGS: 00010202
RAX: 0000000000000024 RBX: 0000000000000001 RCX: 00000000800d0004
RDX: 0000000000000400 RSI: 0000000000000200 RDI: 0000000000000200
RBP: ffff88006e609de8 R08: 0000000000000002 R09: 0000000000000400
R10: 0000000000000000 R11: 0000000000000202 R12: ffff8800731e4000
R13: ffff8800731e4000 R14: ffffffff81565e20 R15: ffff8800731e40a0
FS:  00007fb7b5dfa730(0000) GS:ffff88007f66e000(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000003340613000 CR3: 0000000068440000 CR4: 0000000000002660
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process xl (pid: 20399, threadinfo ffff88006e608000, task ffff880071fe0500)
Stack:
 0000000000000000 0000000000000000 ffff88006e609e18 ffffffff812749dc
 ffff8800731e4000 0000000000000000 ffff8800731e4000 00000000ffffffed
 ffff88006e609e38 ffffffff812764c4 ffff8800731e4090 0000000000000001
Call Trace:
 [<ffffffff812749dc>] pci_dev_reset+0x6c/0xd0
 [<ffffffff812764c4>] pci_reset_function+0x54/0x70
 [<ffffffff8127a6ba>] reset_store+0x5a/0x70
 [<ffffffff81342070>] dev_attr_store+0x20/0x30
 [<ffffffff811db48f>] flush_write_buffer+0x5f/0x90
 [<ffffffff811db856>] sysfs_write_file+0x66/0xa0
 [<ffffffff8116dd8e>] vfs_write+0xce/0x190
 [<ffffffff8116e3e5>] sys_write+0x55/0x90
 [<ffffffff8150a242>] system_call_fastpath+0x16/0x1b
Code: 07 00 00 48 c7 c7 e0 73 d7 81 89 c3 83 c8 20 c0 eb 05 41 88 84 24 80 07 00 00 83 e3 01 e8 6a 17 29 00 85 db 75 05 5b 41 5c c9 c3 <0f> 0b eb fe 66 0f 1f 44 00 00 0f 1f 80 00 00 00 00 55 48 89 e5 
RIP  [<ffffffff812704bf>] pci_block_user_cfg_access+0x4f/0x60
 RSP <ffff88006e609dd8>

pciback 0000:0d:

BTW: xm-create work fine but xl-create trigger the panic as below:

Quoted 148 lines · Joe Jin

I got serial console output now and mostly like panic happened by "xl create":

(XEN) HVM1: Detected Xen v4.1.2-OVM
(XEN) HVM1: CPU speed is 2926 MHz
(XEN) HVM1: Xenbus rings @0xfeffc000, event channel 13 (XEN) irq.c:264: Dom1 PCI link 0 changed 0 -> 5
(XEN) HVM1: PCI-ISA link 0 routed to IRQ5
(XEN) irq.c:264: Dom1 PCI link 1 changed 0 -> 10
(XEN) HVM1: PCI-ISA link 1 routed to IRQ10
(XEN) irq.c:264: Dom1 PCI link 2 changed 0 -> 11
(XEN) HVM1: PCI-ISA link 2 routed to IRQ11
(XEN) irq.c:264: Dom1 PCI link 3 changed 0 -> 5
(XEN) HVM1: PCI-ISA link 3 routed to IRQ5
(XEN) HVM1: pci dev 01:3 INTA->IRQ10
(XEN) HVM1: pci dev 03:0 INTA->IRQ5
(XEN) HVM1: pci dev 04:0 INTA->IRQ5
(XEN) HVM1: pci dev 05:0 INTA->IRQ10
(XEN) HVM1: pci dev 02:0 bar 10 size 02000000: f0000008 (XEN) HVM1: pci dev 03:0 bar 14 size 01000000: f2000008 (XEN) HVM1: pci dev 04:0 bar 30 size 00080000: f3000000 (XEN) HVM1: pci dev 05:0 bar 30 size 00080000: f3080000 (XEN) domctl.c:985:d0 memory_map:add: gfn=f3100 mfn=df040 nr_mfns=40 (XEN) HVM1: pci dev 04:0 bar 1c size 00040000: f3100004 (XEN) domctl.c:985:d0 memory_map:add: gfn=f3140 mfn=df640 nr_mfns=40 (XEN) HVM1: pci dev 05:0 bar 1c size 00040000: f3140004 (XEN) domctl.c:985:d0 memory_map:add: gfn=f3180 mfn=df03c nr_mfns=4 (XEN) domctl.c:995:d0 memory_map:remove: gfn=f3182 mfn=df03e nr_mfns=1 (XEN) HVM1: pci dev 04:0 bar 14 size 00004000: f3180004 (XEN) domctl.c:985:d0 memory_map:add: gfn=f3184 mfn=df63c nr_mfns=4 (XEN) domctl.c:995:d0 memory_map:remove: gfn=f3186 mfn=df63e nr_mfns=1 (XEN) HVM1: pci dev 05:0 bar 14 size 00004000: f3184004 (XEN) HVM1: pci dev 02:0 bar 14 size 00001000: f3188000 (XEN) HVM1: pci dev 03:0 bar 10 size 00000100: 0000c001 (XEN) HVM1: pci dev 04:0 bar 10 size 00000100: 0000c101 (XEN) domctl.c:1041:d0 ioport_map:add f_gport=c100 f_mport=9000 np=100 (XEN) HVM1: pci dev 05:0 bar 10 size 00000100: 0000c201 (XEN) domctl.c:1041:d0 ioport_map:add f_gport=c200 f_mport=d000 np=100 (XEN) HVM1: pci dev 01:1 bar 20 size 00000010: 0000c301 (XEN) HVM1: Multiprocessor initialisation:

(XEN) HVM1:  - CPU0 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU1 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU2 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU3 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU4 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU5 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU6 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU7 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU8 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU9 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU10 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU11 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.

(XEN) HVM1: Writing SMBIOS tables ...
(XEN) HVM1: Loading ROMBIOS ...
(XEN) HVM1: 10364 bytes of ROMBIOS high-memory extensions:
(XEN) HVM1: Relocating to 0xfc000000-0xfc00287c ... done
(XEN) HVM1: Creating MP tables ...
(XEN) HVM1: Loading Cirrus VGABIOS ...
(XEN) domctl.c:985:d0 memory_map:add: gfn=f3000 mfn=df080 nr_mfns=80 (XEN) HVM1: Loading PCI Option ROM ...
(XEN) HVM1: - Manufacturer: LSI Corporation
(XEN) HVM1: - Product name: LSI MPI Boot Support
(XEN) domctl.c:995:d0 memory_map:remove: gfn=f3000 mfn=df080 nr_mfns=80 (XEN) domctl.c:985:d0 memory_map:add: gfn=f3080 mfn=df680 nr_mfns=80 (XEN) domctl.c:995:d0 memory_map:remove: gfn=f3080 mfn=df680 nr_mfns=80 (XEN) HVM1: Loading ACPI ...
(XEN) HVM1: - Lo data: 000ea020-000ea04f
(XEN) HVM1: - Hi data: fc002c00-fc00eb4f
(XEN) HVM1: vm86 TSS at fc00ec00
(XEN) HVM1: BIOS map:

(XEN) HVM1:  c0000-c8fff: VGA BIOS
(XEN) HVM1:  c9000-d4fff: PCI Option ROMs
(XEN) HVM1:  eb000-eb302: SMBIOS tables
(XEN) HVM1:  f0000-fffff: Main BIOS
(XEN) HVM1: E820 table:
(XEN) HVM1:  [00]: 00000000:00000000 - 00000000:0009e000: RAM
(XEN) HVM1:  [01]: 00000000:0009e000 - 00000000:0009fc00: RESERVED
(XEN) HVM1:  [02]: 00000000:0009fc00 - 00000000:000a0000: RESERVED
(XEN) HVM1:  HOLE: 00000000:000a0000 - 00000000:000e0000
(XEN) HVM1:  [03]: 00000000:000e0000 - 00000000:00100000: RESERVED
(XEN) HVM1:  [04]: 00000000:00100000 - 00000000:f0000000: RAM
(XEN) HVM1:  HOLE: 00000000:f0000000 - 00000000:fc000000
(XEN) HVM1:  [05]: 00000000:fc000000 - 00000001:00000000: RESERVED
(XEN) HVM1:  [06]: 00000001:00000000 - 00000004:0f800000: RAM

(XEN) HVM1: Invoking ROMBIOS ...
(XEN) HVM1: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $ (XEN) stdvga.c:147:d1 entering stdvga and caching modes
(XEN) HVM1: VGABios $Id: vgabios.c,v 1.67 2008/01/27 09:44:12 vruppert Exp $ (XEN) HVM1: Bochs BIOS - build: 06/23/99
(XEN) HVM1: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $ (XEN) HVM1: Options: apmbios pcibios eltorito PMM
(XEN) HVM1:
(XEN) HVM1: ata0-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 (XEN) HVM1: ata0 master: QEMU HARDDISK ATA-7 Hard-Disk (60000 MBytes) (XEN) HVM1: ata0-1: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 (XEN) HVM1: ata0 slave: QEMU HARDDISK ATA-7 Hard-Disk ( 97 GBytes) (XEN) HVM1: ata1-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 (XEN) HVM1: ata1 master: QEMU HARDDISK ATA-7 Hard-Disk (20000 MBytes) (XEN) HVM1: IDE time out
(XEN) HVM1:
(XEN) HVM1: PCI device 1000:0070 not found at index 0
(XEN) HVM1: PCI device 1000:0072 not found at index 2
(XEN) HVM1: PCI device 1000:0074 not found at index 0
(XEN) HVM1: PCI device 1000:0076 not found at index 0
(XEN) HVM1: PCI device 1000:0077 not found at index 0
(XEN) HVM1: PCI device 1000:0064 not found at index 0
(XEN) HVM1: PCI device 1000:0065 not found at index 0
(XEN) HVM1: PCI device 1000:0080 not found at index 0
(XEN) HVM1: PCI device 1000:0081 not found at index 0
(XEN) HVM1: PCI device 1000:0082 not found at index 0
(XEN) HVM1: PCI device 1000:0083 not found at index 0
(XEN) HVM1: PCI device 1000:0084 not found at index 0
(XEN) HVM1: PCI device 1000:0085 not found at index 0
(XEN) HVM1: PCI device 1000:0086 not found at index 0
(XEN) HVM1: PCI device 1000:0087 not found at index 0
(XEN) HVM1: PCI device 1000:0070 not found at index 0
(XEN) HVM1: PCI device 1000:0072 not found at index 2
(XEN) HVM1: PCI device 1000:0074 not found at index 0
(XEN) HVM1: PCI device 1000:0076 not found at index 0
(XEN) HVM1: PCI device 1000:0077 not found at index 0
(XEN) HVM1: PCI device 1000:0064 not found at index 0
(XEN) HVM1: PCI device 1000:0065 not found at index 0
(XEN) HVM1: PCI device 1000:0080 not found at index 0
(XEN) HVM1: PCI device 1000:0081 not found at index 0
(XEN) HVM1: PCI device 1000:0082 not found at index 0
(XEN) HVM1: PCI device 1000:0083 not found at index 0
(XEN) HVM1: PCI device 1000:0084 not found at index 0
(XEN) HVM1: PCI device 1000:0085 not found at index 0
(XEN) HVM1: PCI device 1000:0086 not found at index 0
(XEN) HVM1: PCI device 1000:0087 not found at index 0
(XEN) HVM1:
(XEN) HVM1:
(XEN) HVM1: Press F12 for boot menu.
(XEN) HVM1:
(XEN) HVM1: Booting from Hard Disk...
(XEN) HVM1: Booting from 0000:7c00
(XEN) HVM1: *** int 15h function AX=00c0, BX=0000 not yet supported! (XEN) HVM1: *** int 15h function AX=ec00, BX=0002 not yet supported! (XEN) HVM1: KBD: unsupported int 16h function 03
(XEN) HVM1: *** int 15h function AX=e980, BX=0000 not yet supported! (XEN) irq.c:330: Dom1 callback via changed to Direct Vector 0xe9 (XEN) domctl.c:1065:d0 ioport_map:remove f_gport=c100 f_mport=9000 np=100 (XEN) domctl.c:1041:d0 ioport_map:add f_gport=c100 f_mport=9000 np=100 (XEN) domctl.c:1065:d0 ioport_map:remove f_gport=c200 f_mport=d000 np=100 (XEN) domctl.c:1041:d0 ioport_map:add f_gport=c200 f_mport=d000 np=100 (XEN) irq.c:264: Dom1 PCI link 0 changed 5 -> 0
(XEN) irq.c:264: Dom1 PCI link 1 changed 10 -> 0
(XEN) irq.c:264: Dom1 PCI link 2 changed 11 -> 0
(XEN) irq.c:264: Dom1 PCI link 3 changed 5 -> 0

Quoted 5 lines
------------[ cut here ]------------
WARNING: at fs/proc/generic.c:850 remove_proc_entry+0x22d/0x240()
Hardware name: SUN FIRE X4370 M2 SERVER       
remove_proc_entry: removing non-empty directory 'irq/536', leaking at least 'eth7'
Modules linked in: xt_physdev iptable_filter ip_tables xen_blkback xen_netback xen_pciback xen_gntdev xen_evtchn ipmi_devintf ipmi_si lockd sunrpc bridge stp llc bonding dm_round_robin dm_multipath be2iscsi iscsi_boot_sysfs ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp bnx2i cnic uio ipv6 cxgb3i libcxgbi cxgb3 libiscsi_tcp libiscsi scsi_transport_iscsi xenfs xen_privcmd dm_mirror video sbs sbshc acpi_memhotplug acpi_ipmi ipmi_msghandler parport_pc lp parport ses enclosure ixgbe igb e1000e mdio snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd soundcore snd_page_alloc i2c_i801 i2c_core i7core_edac iTCO_wdt pcspkr ioatdma iTCO_vendor_support edac_core dca ghes hed dm_region_hash dm_log dm_mod usb_storage

shpchp sg ahci libahci sd_mod crc_t10dif raid1 ext3 jbd mbcache [last unloaded: raid_class]

Quoted 26 lines

Pid: 19329, comm: qemu-dm Not tainted 2.6.39-200.1.15.el5uek #1

Call Trace:
 [<ffffffff811d10dd>] ? remove_proc_entry+0x22d/0x240
 [<ffffffff8106de10>] warn_slowpath_common+0x90/0xc0
 [<ffffffff8106df3e>] warn_slowpath_fmt+0x6e/0x70
 [<ffffffff8100a7cf>] ? xen_restore_fl_direct_reloc+0x4/0x4
 [<ffffffff8125cf98>] ? sprintf+0x68/0x70
 [<ffffffff811cfef1>] ? __xlate_proc_name+0x41/0xc0
 [<ffffffff81501b9e>] ? _raw_spin_lock+0xe/0x20
 [<ffffffff811d10dd>] remove_proc_entry+0x22d/0x240
 [<ffffffff8113232c>] ? zap_pte_range+0x1ec/0x400
 [<ffffffff810d9430>] unregister_irq_proc+0xd0/0xf0
 [<ffffffff810d571a>] free_desc+0x2a/0x70
 [<ffffffff810d579e>] irq_free_descs+0x3e/0x80
 [<ffffffff812f7822>] xen_free_irq+0x52/0x70
 [<ffffffff812f82cb>] unbind_from_irq+0xeb/0x180
 [<ffffffff81009f0d>] ? xen_force_evtchn_callback+0xd/0x10
 [<ffffffff812f837f>] evtchn_put+0x1f/0x40
 [<ffffffffa055b4e9>] gntdev_put_map+0x39/0x110 [xen_gntdev]
 [<ffffffff812640e1>] ? list_del+0x11/0x40
 [<ffffffffa055c38c>] gntdev_ioctl_unmap_grant_ref+0xac/0xd0 [xen_gntdev]
 [<ffffffffa055c708>] gntdev_ioctl+0x98/0xa0 [xen_gntdev]
 [<ffffffff8117eb8d>] vfs_ioctl+0x1d/0x50
 [<ffffffff8117f623>] do_vfs_ioctl+0x63/0x1b0
 [<ffffffff8113a740>] ? do_munmap+0x240/0x280
 [<ffffffff8117f804>] sys_ioctl+0x94/0xa0
Quoted 8 lines
 [<ffffffff8150a242>] system_call_fastpath+0x16/0x1b
---[ end trace fa61ba514bac93c7 ]---
BUG: unable to handle kernel NULL pointer dereference at 000000000000001c
IP: [<ffffffff812f6a76>] evtchn_from_irq+0x16/0x40
PGD 6a2cb067 PUD 71d8c067 PMD 0 
Oops: 0000 [#1] SMP 
CPU 0 
Modules linked in: xt_physdev iptable_filter ip_tables xen_blkback xen_netback xen_pciback xen_gntdev xen_evtchn ipmi_devintf ipmi_si lockd sunrpc bridge stp llc bonding dm_round_robin dm_multipath be2iscsi iscsi_boot_sysfs ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp bnx2i cnic uio ipv6 cxgb3i libcxgbi cxgb3 libiscsi_tcp libiscsi scsi_transport_iscsi xenfs xen_privcmd dm_mirror video sbs sbshc acpi_memhotplug acpi_ipmi ipmi_msghandler parport_pc lp parport ses enclosure ixgbe igb e1000e mdio snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd soundcore snd_page_alloc i2c_i801 i2c_core i7core_edac iTCO_wdt pcspkr ioatdma iTCO_vendor_support edac_core dca ghes hed dm_region_hash dm_log dm_mod usb_storage

shpchp sg ahci libahci sd_mod crc_t10dif raid1 ext3 jbd mbcache [last unloaded: raid_class]

Quoted 49 lines

Pid: 19329, comm: qemu-dm Tainted: G W 2.6.39-200.1.15.el5uek #1 ORACLE CORPORATION SUN FIRE X4370 M2 SERVER /ASSY,SC_BD,T4

RIP: e030:[<ffffffff812f6a76>]  [<ffffffff812f6a76>] evtchn_from_irq+0x16/0x40
RSP: e02b:ffff88006cfc5dd8  EFLAGS: 00010246
RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffff88007e38be48
RDX: 0000000000000000 RSI: 0000000000000218 RDI: ffff88007e38bd70
RBP: ffff88006cfc5dd8 R08: 0000000000000001 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000246 R12: 0000000000000218
R13: 0000000000000000 R14: 0000000000000017 R15: 0000000000104701
FS:  00007ff36f4046e0(0000) GS:ffff88007f49a000(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 000000000000001c CR3: 000000005f445000 CR4: 0000000000002660
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process qemu-dm (pid: 19329, threadinfo ffff88006cfc4000, task ffff880067084200)
Stack:
 ffff88006cfc5e38 ffffffff812f81fc 00007ff36e620000 0000003e38e10000
 ffff88006cfc5e38 000000000000014f 0000000000000206 ffff88006b2e8138
 ffffffff81009f0d ffff88006b951cc0 ffff88006b951cc0 0000000000000000
Call Trace:
 [<ffffffff812f81fc>] unbind_from_irq+0x1c/0x180
 [<ffffffff81009f0d>] ? xen_force_evtchn_callback+0xd/0x10
 [<ffffffff812f837f>] evtchn_put+0x1f/0x40
 [<ffffffffa055b4e9>] gntdev_put_map+0x39/0x110 [xen_gntdev]
 [<ffffffff812640e1>] ? list_del+0x11/0x40
 [<ffffffffa055c38c>] gntdev_ioctl_unmap_grant_ref+0xac/0xd0 [xen_gntdev]
 [<ffffffffa055c708>] gntdev_ioctl+0x98/0xa0 [xen_gntdev]
 [<ffffffff8117eb8d>] vfs_ioctl+0x1d/0x50
 [<ffffffff8117f623>] do_vfs_ioctl+0x63/0x1b0
 [<ffffffff8113a740>] ? do_munmap+0x240/0x280
 [<ffffffff8117f804>] sys_ioctl+0x94/0xa0
 [<ffffffff8150a242>] system_call_fastpath+0x16/0x1b
Code: 8b 64 24 08 c9 c3 0f 1f 80 00 00 00 00 0f 1f 84 00 00 00 00 00 55 48 89 e5 66 66 66 66 90 39 3d f1 16 4c 00 76 0b e8 ea fa ff ff <0f> b7 40 1c c9 c3 89 f9 31 c0 48 c7 c2 2f 72 70 81 be d1 00 00 
RIP  [<ffffffff812f6a76>] evtchn_from_irq+0x16/0x40
 RSP <ffff88006cfc5dd8>

CR2: 000000000000001c

Thanks,
Joe

Quoted 9 lines · Ian Campbell
Quoted 1 line · Joe Jin

execute xl-destroy twice crashed my server!

Can you give more details please.

Ian.

#5 lore raw
In reply to Joe Jin
Quoted 8 lines · Joe Jin
Quoted 4 lines · Ian Campbell
Quoted 1 line · Joe Jin
Quoted 1 line

execute xl-destroy twice crashed my server!

Can you give more details please.

Hi Ian,

Sorry the server not available for test now.

OK, please let us know if/when you can gather details. Oh I see you already have, thanks!

Quoted 1 line

Would you please help to review my patch for xm-destroy?

xend is deprecated and I'm afraid I don't have many spare cycles to spend on it. I think really it needs to be down to those who want to keep xend alive to review each others patches.

Ian.

#8 lore raw
In reply to Joe Jin
Quoted 2 lines · Joe Jin

Execute "xl-destroy" twice lead the dom0 panic:
config file as below:

Thanks.

Quoted 1 line

kernel BUG at drivers/pci/access.c:423!

[...]

Quoted 1 line

BTW: xm-create work fine but xl-create trigger the panic as below:

Both the panic above and the warn+panic below seem like kernel bugs which are simply exposed by xl rather than xl bugs as such. Presumably xl just tickles things different to xm.

The dom0 kernel you are using is a bit dated and appears (from the revision) to be rather heavily patched.

Quoted 3 lines
Quoted 3 lines
WARNING: at fs/proc/generic.c:850 remove_proc_entry+0x22d/0x240()
Hardware name: SUN FIRE X4370 M2 SERVER

remove_proc_entry: removing non-empty directory 'irq/536', leaking at least 'eth7'

[..]

Quoted 2 lines
Quoted 2 lines
BUG: unable to handle kernel NULL pointer dereference at 000000000000001c
IP: [<ffffffff812f6a76>] evtchn_from_irq+0x16/0x40

Wei recently posted a bug fix for an issue which looked a bit like this last one. That was something do with error handling on allocation failure though so I doubt it represents the actual root cause though.

Ian.

Reviewed-by on v1 1/1
#9 lore raw
In reply to Joe Jin
Quoted 10 lines · Joe Jin

Current Xend allowing multiple call destroy() for same domain, this lead multiple hard resets(FLR) for pci pass-through, and some controller might failed.

In our test, we pass through 2 LSI HAB controllers to the PVHVM guest, after guest brought up, call xm-destroy twice, the adapters's BIOS will hung, and we had to reboot the server to recovery it.

Cc Konrad Rzeszutek Wilk <[email protected]>

The other bugs (the kernel ones) are a nice find and should also be fixed - but I think with the upstream kernel they have been mostly fixed.

Quoted 57 lines
Cc Ian Campbell <[email protected]>
---
 tools/python/xen/xend/XendDomainInfo.py | 11 +++++++++++
 1 file changed, 11 insertions(+)
tools/python/xen/xend/XendDomainInfo.py +11 -0
index e9d3e7e..658c3b6 100644
@@ -367,6 +367,8 @@ class XendDomainInfo:
     @type refresh_shutdown_lock: threading.Condition
     @ivar _deviceControllers: device controller cache for this domain
     @type _deviceControllers: dict 'string' to DevControllers
+    @ivar destroying: Is this domain destroying
+    @type destroying: bool
     """
 
     def __init__(self, info, domid = None, dompath = None, augment = False,
@@ -455,6 +457,8 @@ class XendDomainInfo:
         self._checkName(self.info['name_label'])
 
         self.metrics = XendVMMetrics(uuid.createString(), self)
+
+        self.destroying = False
 
 
     #
@@ -3073,6 +3077,12 @@ class XendDomainInfo:
 
         if self.domid is None:
             return
+
+        if self.destroying == False:
+            self.destroying = True
+        else:
+            raise VmError("Domain (domid=%s) is destroying, please wait!", str(self.domid))
+
         from xen.xend import XendDomain
         log.debug("XendDomainInfo.destroy: domid=%s", str(self.domid))
 
@@ -3088,6 +3098,7 @@ class XendDomainInfo:
                     self.info[state] = 0
                 self._stateSet(DOM_STATE_HALTED)
             except:
+                self.destroying = False
                 log.exception("XendDomainInfo.destroy: domain destruction failed.")
 
             XendDomain.instance().remove_domain(self)
Signature · 7 lines
1.8.1


_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel