fix iommu interrupt setup: setup_vector() may fail
PATCH 1/1 fix iommu interrupt setup: setup_vector() may fail
Since changeset 19175 (Cleanup IOMMU interrupt setup) I am experiencing a boot failure on my HP dc7800 running in x86_64 mode. The most recent version that I have tested is 19210 (x86: Clean up vpt-align patch.), which is after 19207 (iommu: Fix interrupt setup.) which Shimura-san created to fix some of the problems caused by 19175.
After investigating this problem I found that the cause is that for some vectors setup_vector() fails (returning -EBUSY).
setup_vector() is called by request_irq_vector(). And request_irq_vector() is called by iommu_set_interrupt(). Which is in turn called by init_vtd_hw().
And 19175 essentailly adds error checking code which now checks for errors returned by iommu_set_interrupt() and thus indirectly the error returned by setup_vector().
Unfortunately as things stands, the way that init_vtd_hw() handles such errors causes my dc7800 not to boot.
I am not entirely sure of the correctness of my proposed change, but by calling iommu_set_interrupt() before iommu_set_root_entry() in init_vtd_hw() my system boots correctly.
In particularl I'm not sure:
a) if it is always valid to call iommu_set_interrupt() before iommu_set_root_entry()
b) if this would adversely affect a situation where iommu_set_root_entry() fails.
- This seems to be ok, I tested replacing the body of
iommu_set_root_entry() with return -EINVAL;
---
(XEN) Xen version 3.4-unstable ([email protected]) (gcc version 4.0.2) Mon Feb 16 14:00:20 EST 2009 (XEN) Latest ChangeSet: Fri Feb 13 09:48:56 2009 +0000 19210:09a6fa059b37
(XEN) Command line: com1=115200,8n1,0x1248,0 console=com1 loglvl=all guest_loglvl=all vtd=1 iommu=1 (XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: none; EDID transfer time: 0 seconds (XEN) EDID info not retrieved because no DDC retrieval method detected
(XEN) Disc information:
(XEN) Found 1 MBR signatures
(XEN) Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 000000000009fc00 (usable) (XEN) 000000000009fc00 - 00000000000a0000 (reserved) (XEN) 00000000000e8000 - 0000000000100000 (reserved) (XEN) 0000000000100000 - 00000000defafe00 (usable) (XEN) 00000000defafe00 - 00000000defb1ea0 (ACPI NVS) (XEN) 00000000defb1ea0 - 00000000e0000000 (reserved) (XEN) 00000000f4000000 - 00000000f8000000 (reserved) (XEN) 00000000fec00000 - 00000000fed40000 (reserved) (XEN) 00000000fed45000 - 0000000100000000 (reserved) (XEN) 0000000100000000 - 000000011c000000 (usable)
(XEN) System RAM: 4015MB (4111672kB)
(XEN) ACPI: RSDP 000E5C10, 0014 (r0 COMPAQ)
(XEN) ACPI: RSDT DEFC1E40, 0044 (r1 HPQOEM SLIC-BPC 20080318 0)
(XEN) ACPI: FACP DEFC1EE8, 0074 (r1 COMPAQ BEARLAKE 1 0)
(XEN) ACPI: DSDT DEFC2427, A370 (r1 COMPAQ DSDT_PRJ 1 MSFT 100000E)
(XEN) ACPI: FACS DEFC1E00, 0040
(XEN) ACPI: APIC DEFC1F5C, 0084 (r1 COMPAQ BEARLAKE 1 0)
(XEN) ACPI: ASF! DEFC1FE0, 0063 (r32 COMPAQ BEARLAKE 1 0)
(XEN) ACPI: MCFG DEFC2043, 003C (r1 COMPAQ BEARLAKE 1 0) (XEN) ACPI: TCPA DEFC207F, 0032 (r1 COMPAQ BEARLAKE 1 0) (XEN) ACPI: SLIC DEFC20B1, 0176 (r1 HPQOEM SLIC-BPC 1 0) (XEN) ACPI: HPET DEFC2227, 0038 (r1 COMPAQ BEARLAKE 1 0) (XEN) ACPI: DMAR DEFC225F, 0198 (r1 COMPAQ BEARLAKE 1 0)
(XEN) NUMA turned off
(XEN) Faking a node at 0000000000000000-000000011c000000 (XEN) Domain heap initialised
(XEN) found SMP MP-table at 000f9bf0
(XEN) DMI 2.5 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0xf808
(XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[f804,460], pm1x_evt[f800,0]
(XEN) ACPI: wakeup_vec[defc1e0c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled) (XEN) Processor #0 7:7 APIC version 20
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled) (XEN) Processor #1 7:7 APIC version 20
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x00] disabled) (XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x00] disabled) (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) (XEN) ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1]) (XEN) ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1]) (XEN) ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1]) (XEN) ACPI: IOAPIC (id[0x01] address[0xfec00000] gsi_base[0]) (XEN) IOAPIC[0]: apic_id 1, version 32, address 0xfec00000, GSI 0-23 (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) (XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
(XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000
(XEN) [VT-D]dmar.c:460: Host address width 36
(XEN) [VT-D]dmar.c:469: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:337: dmaru->address = fed90000
(XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:1b.0 (XEN) [VT-D]dmar.c:469: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:337: dmaru->address = fed91000
(XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:2.0
(XEN) [VT-D]dmar.c:469: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:337: dmaru->address = fed92000
(XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:3.0
(XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:3.2
(XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:3.3
(XEN) [VT-D]dmar.c:469: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:337: dmaru->address = fed93000
(XEN) [VT-D]dmar.c:346: found INCLUDE_ALL
(XEN) [VT-D]dmar.c:473: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:2.0
(XEN) [VT-D]dmar.c:473: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:1d.7 (XEN) [VT-D]dmar.c:473: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:1a.7 (XEN) [VT-D]dmar.c:473: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:1d.0 (XEN) [VT-D]dmar.c:473: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:1d.1 (XEN) [VT-D]dmar.c:473: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:1d.2 (XEN) [VT-D]dmar.c:473: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:1a.0 (XEN) [VT-D]dmar.c:473: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:1a.1 (XEN) Intel VT-d has been enabled
(XEN) Using ACPI (MADT) for SMP configuration information (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Initializing CPU#0
(XEN) Detected 2826.324 MHz processor.
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 6144K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 0
(XEN) VMX: Supported advanced features:
(XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap
(XEN) HVM: VMX enabled
(XEN) MCE: Intel newly family MC Init
(XEN) Intel machine check reporting enabled on CPU#0.
(XEN) CPU0: Thermal monitoring enabled (TM2)
(XEN) CMCI: CPU0 has no CMCI support
(XEN) CPU0: Intel(R) Core(TM)2 Duo CPU E8300 @ 2.83GHz stepping 06
(XEN) Booting processor 1/1 eip 8c000
(XEN) Initializing CPU#1
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 6144K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 1
(XEN) MCE: Intel newly family MC Init
(XEN) Intel machine check reporting enabled on CPU#1.
(XEN) CPU1: Thermal monitoring enabled (TM2)
(XEN) CMCI: CPU1 has no CMCI support
(XEN) CPU1: Intel(R) Core(TM)2 Duo CPU E8300 @ 2.83GHz stepping 06
(XEN) Total of 2 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using new ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) checking TSC synchronization across 2 CPUs: passed.
(XEN) Platform timer is 14.318MHz HPET
(XEN) Brought up 2 CPUs
(XEN) microcode.c:73:d32767 microcode: CPU1 resumed
(XEN) [VT-D]iommu.c:1707: Queued Invalidation hardware not found
(XEN) [VT-D]iommu.c:1707: Queued Invalidation hardware not found
(XEN) [VT-D]iommu.c:1707: Queued Invalidation hardware not found
(XEN) [VT-D]iommu.c:1707: Queued Invalidation hardware not found
(XEN) [VT-D]iommu.c:1715: Interrupt Remapping hardware not found
(XEN) [VT-D]iommu.c:1715: Interrupt Remapping hardware not found
(XEN) [VT-D]iommu.c:1715: Interrupt Remapping hardware not found
(XEN) [VT-D]iommu.c:1715: Interrupt Remapping hardware not found
(XEN) Intel VT-d snoop control disabled
(XEN) I/O virtualisation enabled
(XEN) I/O virtualisation for PV guests disabled
(XEN) ACPI sleep modes: S3
(XEN) mcheck_poll: Init_mcheck_timer
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:0.0
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:2.0
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:3.0
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:3.2
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:3.3
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:19.0
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1a.0
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1a.1
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1a.7
(XEN) [VT-D]iommu.c:1246:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1d.0
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1d.1
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1d.2
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1d.7
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1f.0
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1f.2
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1f.5
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1a.1
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1a.0
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1d.2
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1d.1
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1d.0
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1a.7
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1d.7
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:2.0
(XEN) [VT-D]iommu.c:618: iommu_enable_translation: iommu->reg = ffff828bfff57000
(XEN) [VT-D]iommu.c:618: iommu_enable_translation: iommu->reg = ffff828bfff56000
(XEN) [VT-D]iommu.c:618: iommu_enable_translation: iommu->reg = ffff828bfff55000
(XEN) [VT-D]iommu.c:618: iommu_enable_translation: iommu->reg = ffff828bfff54000
(XEN) *** LOADING DOMAIN 0 ***
(XEN) elf_init: not an ELF binary
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) Could not set up DOM0 guest OS
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...
(XEN) [VT-D]iommu.c:721: iommu_page_fault: iommu->reg = ffff828bfff54000
(XEN) [VT-D]iommu.c:690: iommu_fault_status: Fault Overflow
(XEN) [VT-D]iommu.c:693: iommu_fault_status: Primary Pending Fault
(XEN) [VT-D]iommu.c:675: iommu_fault:DMA Read: 0:1d.2 addr defce000 REASON 6 iommu->reg = ffff828bfff54000 (XEN) print_vtd_entries: iommu = ffff83011bfd4310 bdf = 0:1d:2 gmfn = defce
(XEN) root_entry = ffff83011bfcb000 (XEN) root_entry[0] = 118453001 (XEN) context = ffff830118453000 (XEN) context[ea] = 101_11bfb1001 (XEN) l3 = ffff83011bfb1000 (XEN) l3_index = 3 (XEN) l3[3] = 11862d003 (XEN) l2 = ffff83011862d000 (XEN) l2_index = f7 (XEN) l2[f7] = 118535003 (XEN) l1 = ffff830118535000 (XEN) l1_index = 1ce (XEN) l1[1ce] = 0 (XEN) l1[1ce] not present
xen/drivers/passthrough/vtd/iommu.c +6 -6
@@ -1674,12 +1674,6 @@ static int init_vtd_hw(void)for_each_drhd_unit ( drhd ) { iommu = drhd->iommu; - ret =iommu_set_root_entry (iommu); -if ( ret ) - { - gdprintk(XENLOG_ERR VTDPREFIX, "IOMMU: set root entry failed\n"); -return -EIO ; - } vector =iommu_set_interrupt (iommu);if ( vector <0 )
@@ -1687,6 +1681,12 @@ static int init_vtd_hw(void) gdprintk(XENLOG_ERR VTDPREFIX, "IOMMU: interrupt setup failed\n");return vector; } + ret =iommu_set_root_entry (iommu); +if ( ret ) + { + gdprintk(XENLOG_ERR VTDPREFIX, "IOMMU: set root entry failed\n"); +return -EIO ; + }dma_msi_data_init (iommu, vector);dma_msi_addr_init (iommu,cpu_physical_id (first_cpu (cpu_online_map))); iommu->vector = vector;
Quoted 5 lines · Simon Horman
Since changeset 19175 (Cleanup IOMMU interrupt setup) I am experiencing a boot failure on my HP dc7800 running in
x86_64mode. The most recent version that I have tested is 19210 (x86: Clean up vpt-align patch.), which is after 19207 (iommu: Fix interrupt setup.) which Shimura-san created to fix some of the problems caused by 19175.
Despite my long-winded explanation, there seems to be an error in my testing and this patch not work. I hope to have a revised version available shortly.
Signature · 3 lines
Simon Horman VA Linux Systems Japan K.K., Sydney, Australia Satellite Office H: www.vergenet.net/~horms/ W: www.valinux.co.jp/en