Refine IR initialization flow and fixes bugs related to X2APIC
cover Refine IR initialization flow and fixes bugs related to X2APIC
When converting x86 to new hierarchy irqdoamin framework, Thomas noticed that the interrupt remapping initialization flow is a little complex and has troubles in memory allocation. Then there is a joint force to simplify IR initialization flow, please refer to related threads at:
https://lkml.org/lkml/2014/12/5/114
https://lkml.org/lkml/2014/12/10/20
https://www.mail-archive.com/[email protected]/msg788792.html
This patch set is based on latest mainstream kernel and I will rebased it onto v3.19-rc1 once v3.19-rc1 is out. And you may access it at:
https://github.com/jiangliu/linux.git ir_init_v1
This patch set combimes above three patches to simplify IR initalization flow and solves memory allocation issues. While at it, this patch set also refines CPU X2APIC initialization code for maintenance. It also fixes two bugs related to X2APIC support.
1) System hangs or panics if BIOS enables CPU X2APIC mode but kernel doesn't support X2APIC.
2) System livelocks if BIOS enables CPU X2APIC but opt-outs IR X2APIC.
This patch set has been tested with on an Intel 4-socket system with following configuration:
---------------------------------------------------------------------------
[CPU X2APIC] [IR X2APIC] [Linux IR] [Linux X2APIC] [Result]
1 D / D D OK 2 D / E D OK 3 D / E E OK 4 P / E E OK 5 P / D D Panic(expected) 6 P / E D Panic(expected) 7 P H E E OK
----------------------------------------------------------------------------
CPU X2APIC: whether CPU X2APIC is enabled by hardware and BIOS IR X2APIC: whether interrupt remapping hardware supports X2APIC mode Linux IR: whether interrupt remapping is enabled by Linux kernel Linux X2APIC: whether X2APIC is supported by Linux kernel
D: disabled
E: enabled
/: Not care
P: CPU X2APIC pre-enabled by BIOS
H: Hard-coded to opt-out X2APIC support in interrupt remapping hardware
The patch set changes the behevior of the last three rows:
1) Row 5 and 6 panics with clear messages instead of random hang or panic.
2) Row 7 boots successfully instead of livelocking.
The patch set also passes Fengguang's 0day test suites.
Due to lack of hardware platforms for tests, tests on AMD platform are welcomed!
Jiang Liu (11):
x86/apic: Panic if kernel doesn't support x2apic but BIOS has enabled
x2apic
x86/apic: Kill useless variable x2apic_enabled in function
enable_IR_x2apic()
x86/apic: Correctly detect X2APIC status in function enable_IR() x86/apic: Refine enable_IR_x2apic() and related functions iommu/vt-d: Prepare for killing function irq_remapping_supported() iommu/vt-d: Allow IR works in XAPIC mode though CPU works in X2APIC
mode
x86/apic: Only disable CPU x2apic mode when necessary iommu/irq_remapping: Kill function irq_remapping_supported() and
related code
iommu/irq_remapping: Refine function irq_remapping_prepare() for
maintenance
iommu/irq_remapping: Change variable disable_irq_remap to be static iommu/irq_remapping: Normailize the way to detect whether IR is
enabled
Joerg Roedel (2):
iommu/vt-d: Allocate IRQ remapping data structures only for all
IOMMUs
iommu/amd: Check for irq-remap support amd_iommu_prepare()
Thomas Gleixner (4):
x86, smpboot: Remove pointless preempt_disable() in
native_smp_prepare_cpus()
iommu, x86: Restructure setup of the irq remapping feature iommu/vt-d: Move iommu preparatory allocations to
irq_remap_ops.prepare
iommu/vt-d: Convert allocations to GFP_KERNEL
arch/x86/include/asm/irq_remapping.h | 4 -- arch/x86/kernel/apic/apic.c | 104 ++++++++++++++++------------------ arch/x86/kernel/smpboot.c | 8 +-- drivers/iommu/amd_iommu.c | 1 - drivers/iommu/amd_iommu_init.c | 10 +--- drivers/iommu/amd_iommu_proto.h | 1 - drivers/iommu/intel_irq_remapping.c | 96 ++++++++++++++++--------------- drivers/iommu/irq_remapping.c | 74 ++++++++---------------- drivers/iommu/irq_remapping.h | 5 -- 9 files changed, 129 insertions(+), 174 deletions(-)
Signature · 1 line
1.7.10.4
PATCH 1/17 x86, smpboot: Remove pointless preempt_disable() in native_smp_prepare_cpus()
From: Thomas Gleixner <[email protected]>
There is no reason to keep preemption disabled in this function.
We only have two other threads live: kthreadd and idle. Neither of them is going to preempt. But that preempt_disable forces all the code inside to do GFP_ATOMIC allocations which is just insane.
Remove it.
--- arch/x86/kernel/smpboot.c | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-)
arch/x86/kernel/smpboot.c +1 -7
@@ -1084,7 +1084,6 @@ void __init native_smp_prepare_cpus(unsigned int max_cpus) {unsigned int i; -preempt_disable ();smp_cpu_index_default ();/*
@@ -1102,22 +1101,19 @@ void __init native_smp_prepare_cpus(unsigned int max_cpus) }set_cpu_sibling_map (0 ); -if (smp_sanity_check (max_cpus) <0 ) {pr_info ("SMP disabled\n" );disable_smp (); - goto out; +return ; }default_setup_apic_routing (); -preempt_disable ();if (read_apic_id () != boot_cpu_physical_apicid) {panic ("Boot APIC ID in local APIC unexpected (%d vs %d)" ,read_apic_id (), boot_cpu_physical_apicid);/* Or can we switch back to PIC here? */ } -preempt_enable ();connect_bsp_APIC ();
@@ -1151,8 +1147,6 @@ void __init native_smp_prepare_cpus(unsigned int max_cpus)uv_system_init ();set_mtrr_aps_delayed_init (); -out: -preempt_enable (); }void arch_enable_nonboot_cpus_begin (void )
Signature · 1 line
1.7.10.4
PATCH 2/17 iommu, x86: Restructure setup of the irq remapping feature
In reply to Jiang LiuFrom: Thomas Gleixner <[email protected]>
enable_IR_x2apic() calls setup_irq_remapping_ops() which by default installs the intel dmar remapping ops and then calls the amd iommu irq remapping prepare callback to figure out whether we are running on an AMD machine with irq remapping hardware.
Right after that it calls irq_remapping_prepare() which pointlessly checks:
if (!remap_ops || !remap_ops->prepare)
return -ENODEV;and then calls
remap_ops->prepare()
which is silly in the AMD case as it got called from setup_irq_remapping_ops() already a few microseconds ago.
Simplify this and just collapse everything intoirq_remapping_prepare().
The irq_remapping_prepare() remains still silly as it assigns blindly the intel ops, but that's not scope of this patch.
The scope here is to move the preperatory work, i.e. memory allocations out of the atomic section which is required to enable irq remapping.
--- arch/x86/include/asm/irq_remapping.h | 2 -- arch/x86/kernel/apic/apic.c | 3 --- drivers/iommu/irq_remapping.c | 19 +++++++------------ 3 files changed, 7 insertions(+), 17 deletions(-)
arch/x86/include/asm/irq_remapping.h +0 -2
@@ -33,7 +33,6 @@ struct irq_cfg;#ifdef CONFIG_IRQ_REMAP -extern void setup_irq_remapping_ops (void );extern int irq_remapping_supported (void );extern void set_irq_remapping_broken (void );extern int irq_remapping_prepare (void );
@@ -60,7 +59,6 @@ void irq_remap_modify_chip_defaults(struct irq_chip *chip);#else /* CONFIG_IRQ_REMAP */ -static inline void setup_irq_remapping_ops (void ) { }static inline int irq_remapping_supported (void ) {return 0 ; }static inline void set_irq_remapping_broken (void ) { }static inline int irq_remapping_prepare (void ) {return -ENODEV ; }
arch/x86/kernel/apic/apic.c +0 -3
@@ -1597,9 +1597,6 @@ void __init enable_IR_x2apic(void)int ret, x2apic_enabled =0 ;int hardware_init_ret; -/* Make sure irq_remap_ops are initialized */ -setup_irq_remapping_ops (); - hardware_init_ret =irq_remapping_prepare ();if (hardware_init_ret && !x2apic_supported ())return ;
drivers/iommu/irq_remapping.c +7 -12
@@ -194,16 +194,6 @@ static __init int setup_irqremap(char *str) }early_param ("intremap" , setup_irqremap); -void __init setup_irq_remapping_ops (void ) -{ - remap_ops = &intel_irq_remap_ops; - -#ifdef CONFIG_AMD_IOMMU -if (amd_iommu_irq_ops.prepare () ==0 ) - remap_ops = &amd_iommu_irq_ops; -#endif -} -void set_irq_remapping_broken (void ) { irq_remap_broken =1 ;
@@ -222,9 +212,14 @@ int irq_remapping_supported(void)int __init irq_remapping_prepare (void ) { -if (!remap_ops || !remap_ops->prepare) -return -ENODEV ; + remap_ops = &intel_irq_remap_ops; +#ifdef CONFIG_AMD_IOMMU +if (amd_iommu_irq_ops.prepare () ==0 ) { + remap_ops = &amd_iommu_irq_ops; +return 0 ; + } +#endif return remap_ops->prepare (); }
Signature · 1 line
1.7.10.4
PATCH 3/17 iommu/vt-d: Move iommu preparatory allocations to irq_remap_ops.prepare
In reply to Jiang LiuFrom: Thomas Gleixner <[email protected]>
The whole iommu setup for irq remapping is a convoluted mess. The iommu detect function gets called from mem_init() and the prepare callback gets called from enable_IR_x2apic() for unknown reasons.
Of course AMD and Intel setup differs in nonsensical ways. Intels prepare callback is explicit while AMDs prepare callback is implicit in setup_irq_remapping_ops() just to be called in the prepare call again.
Because all of this gets called from enable_IR_x2apic() and the dmar prepare function merily parses the ACPI tables, but does not allocate memory we end up with memory allocation from irq disabled context later on.
AMDs iommu code at least allocates the required memory from the prepare function. That has issues as well, but thats not scope of this patch.
The goal of this change is to distangle the allocation from the actual enablement. There is no point to allocate memory from irq disabled regions with GFP_ATOMIC just because it does not matter at that point in the boot stage. It matters with physical hotplug later on.
There is another issue with the current setup. Due to the conversion to stacked irqdomains we end up with a call into the irqdomain allocation code from irq disabled context, but that code does GFP_KERNEL allocations rightfully as there is no reason to do preperatory allocations with GFP_ATOMIC.
That change caused the allocator code to complain about GFP_KERNEL allocations invoked in atomic context. Boris provided a temporary hackaround which changed the GFP flags if irq_domain_add() got called from atomic context. Not pretty and we really dont want to get this into a mainline release for obvious reasons.
Move the ACPI table parsing and the resulting memory allocations from the enable to the prepare function. That allows to get rid of the horrible hackaround in irq_domain_add() later.
[Jiang] Rebased onto v3.19
--- drivers/iommu/intel_irq_remapping.c | 64 ++++++++++++++++++++++++----------- 1 file changed, 44 insertions(+), 20 deletions(-)
drivers/iommu/intel_irq_remapping.c +44 -20
@@ -595,22 +595,57 @@ static int __init intel_irq_remapping_supported(void)return 1 ; } -static int __init intel_enable_irq_remapping (void ) +static void __init intel_cleanup_irq_remapping (void ) +{ +struct dmar_drhd_unit *drhd; +struct intel_iommu *iommu; + +for_each_iommu (iommu, drhd) { +if (ecap_ir_support (iommu->ecap)) { +iommu_disable_irq_remapping (iommu); +intel_teardown_irq_remapping (iommu); + } + } + +if (x2apic_supported ()) +pr_warn ("Failed to enable irq remapping. You are vulnerable to irq-injection attacks.\n" ); +} + +static int __init intel_prepare_irq_remapping (void ) {struct dmar_drhd_unit *drhd;struct intel_iommu *iommu; -bool x2apic_present; -int setup =0 ; -int eim =0 ; - x2apic_present =x2apic_supported (); +if (dmar_table_init () <0 ) +return -1 ;if (parse_ioapics_under_ir () !=1 ) { -printk (KERN_INFO "Not enable interrupt remapping\n" ); +printk (KERN_INFO "Not enabling interrupt remapping\n" ); goto error; } -if (x2apic_present) { +for_each_iommu (iommu, drhd) { +if (!ecap_ir_support (iommu->ecap)) +continue ; + +/* Do the allocations early */ +if (intel_setup_irq_remapping (iommu)) + goto error; + } +return 0 ; +error: +intel_cleanup_irq_remapping (); +return -1 ; +} + +static int __init intel_enable_irq_remapping (void ) +{ +struct dmar_drhd_unit *drhd; +struct intel_iommu *iommu; +int setup =0 ; +int eim =0 ; + +if (x2apic_supported ()) {pr_info ("Queued invalidation will be enabled to support x2apic and Intr-remapping.\n" ); eim = !dmar_x2apic_optout ();
@@ -678,9 +713,6 @@ static int __init intel_enable_irq_remapping(void)if (!ecap_ir_support (iommu->ecap))continue ; -if (intel_setup_irq_remapping (iommu)) - goto error; -iommu_set_irq_remapping (iommu, eim); setup =1 ; }
@@ -702,15 +734,7 @@ static int __init intel_enable_irq_remapping(void)return eim ?IRQ_REMAP_X2APIC_MODE :IRQ_REMAP_XAPIC_MODE ; error: -for_each_iommu (iommu, drhd) -if (ecap_ir_support (iommu->ecap)) { -iommu_disable_irq_remapping (iommu); -intel_teardown_irq_remapping (iommu); - } - -if (x2apic_present) -pr_warn ("Failed to enable irq remapping. You are vulnerable to irq-injection attacks.\n" ); - +intel_cleanup_irq_remapping ();return -1 ; }
@@ -1200,7 +1224,7 @@ static int intel_alloc_hpet_msi(unsigned int irq, unsigned int id)struct irq_remap_ops intel_irq_remap_ops = { .supported = intel_irq_remapping_supported, - .prepare = dmar_table_init, + .prepare = intel_prepare_irq_remapping, .enable = intel_enable_irq_remapping, .disable = disable_irq_remapping, .reenable = reenable_irq_remapping,
Signature · 1 line
1.7.10.4
PATCH 4/17 iommu/vt-d: Convert allocations to GFP_KERNEL
In reply to Jiang LiuFrom: Thomas Gleixner <[email protected]>
No reason anymore to do GFP_ATOMIC allocations which are not harmful in the normal bootup case, but matter in the physical hotplug scenario.
--- drivers/iommu/intel_irq_remapping.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/iommu/intel_irq_remapping.c +2 -2
@@ -481,11 +481,11 @@ static int intel_setup_irq_remapping(struct intel_iommu *iommu)if (iommu->ir_table)return 0 ; - ir_table =kzalloc (sizeof (struct ir_table ),GFP_ATOMIC ); + ir_table =kzalloc (sizeof (struct ir_table ),GFP_KERNEL );if (!ir_table)return -ENOMEM ; - pages =alloc_pages_node (iommu->node,GFP_ATOMIC | __GFP_ZERO, + pages =alloc_pages_node (iommu->node,GFP_KERNEL | __GFP_ZERO,INTR_REMAP_PAGE_ORDER );if (!pages) {
Signature · 1 line
1.7.10.4
PATCH 5/17 x86/apic: Panic if kernel doesn't support x2apic but BIOS has enabled x2apic
In reply to Jiang LiuWhen kernel doesn't support X2APIC but BIOS has enabled X2APIC, system may panic or hang without useful messages. On the other hand, it's hard to dynamically disable X2APIC when CONFIG_X86_X2APIC is disabled. So panic with a clear message in such a case.
System panics as below when X2APIC is disabled and interrupt remapping is enabled:
[ 0.316118] LAPIC pending interrupts after 512 EOI [ 0.322126] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.368655] Kernel panic - not syncing: timer doesn't work through Interrupt-remapped IO-APIC [ 0.378300] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.18.0+ #340 [ 0.385300] Hardware name: Intel Corporation BRICKLAND/BRICKLAND, BIOS BRIVTIN1.86B.0051.L05.1406240953 06/24/2014 [ 0.396997] ffff88046dc03000 ffff88046c307dd8 ffffffff8179dada 00000000000043f2 [ 0.405629] ffffffff81a92158 ffff88046c307e58 ffffffff8179b757 0000000000000002 [ 0.414261] 0000000000000008 ffff88046c307e68 ffff88046c307e08 ffffffff813ad82b [ 0.422890] Call Trace: [ 0.425711] [<ffffffff8179dada>] dump_stack+0x45/0x57 [ 0.431533] [<ffffffff8179b757>] panic+0xc1/0x1f5 [ 0.436978] [<ffffffff813ad82b>] ? delay_tsc+0x3b/0x70 [ 0.442910] [<ffffffff8166fa2c>] panic_if_irq_remap+0x1c/0x20 [ 0.449524] [<ffffffff81d73645>] setup_IO_APIC+0x405/0x82e [ 0.464979] [<ffffffff81d6fcc2>] native_smp_prepare_cpus+0x2d9/0x31c [ 0.472274] [<ffffffff81d5d0ac>] kernel_init_freeable+0xd6/0x223 [ 0.479170] [<ffffffff81792ad0>] ? rest_init+0x80/0x80 [ 0.485099] [<ffffffff81792ade>] kernel_init+0xe/0xf0 [ 0.490932] [<ffffffff817a537c>] ret_from_fork+0x7c/0xb0 [ 0.497054] [<ffffffff81792ad0>] ? rest_init+0x80/0x80 [ 0.502983] ---[ end Kernel panic - not syncing: timer doesn't work through Interrupt-remapped IO-APIC
System hangs as below when X2APIC and interrupt remapping are both disabled:
[ 1.102782] pci 0000:00:02.0: System wakeup disabled by ACPI [ 1.109351] pci 0000:00:03.0: System wakeup disabled by ACPI [ 1.115915] pci 0000:00:03.2: System wakeup disabled by ACPI [ 1.122479] pci 0000:00:03.3: System wakeup disabled by ACPI [ 1.132274] pci 0000:00:1c.0: Enabling MPC IRBNCE [ 1.137620] pci 0000:00:1c.0: Intel PCH root port ACS workaround enabled [ 1.145239] pci 0000:00:1c.0: System wakeup disabled by ACPI [ 1.151790] pci 0000:00:1c.7: Enabling MPC IRBNCE [ 1.157128] pci 0000:00:1c.7: Intel PCH root port ACS workaround enabled [ 1.164748] pci 0000:00:1c.7: System wakeup disabled by ACPI [ 1.171447] pci 0000:00:1e.0: System wakeup disabled by ACPI [ 1.178612] acpiphp: Slot [8] registered [ 1.183095] pci 0000:00:02.0: PCI bridge to [bus 01] [ 1.188867] acpiphp: Slot [2] registered
--- arch/x86/kernel/apic/apic.c | 8 ++++++++ 1 file changed, 8 insertions(+)
arch/x86/kernel/apic/apic.c +8 -0
@@ -1597,6 +1597,14 @@ void __init enable_IR_x2apic(void)int ret, x2apic_enabled =0 ;int hardware_init_ret; +if (!IS_ENABLED (CONFIG_X86_X2APIC )) { +u64 msr; + +rdmsrl (MSR_IA32_APICBASE , msr); +if (msr &X2APIC_ENABLE ) +panic ("BIOS has enabled x2apic but kernel doesn't support x2apic, please disable x2apic in BIOS.\n" ); + } + hardware_init_ret =irq_remapping_prepare ();if (hardware_init_ret && !x2apic_supported ())return ;
Signature · 1 line
1.7.10.4
PATCH 6/17 x86/apic: Kill useless variable x2apic_enabled in function enable_IR_x2apic()
In reply to Jiang LiuLocal variable x2apic_enabled has been assigned to but never referred, so kill it.
--- arch/x86/kernel/apic/apic.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
arch/x86/kernel/apic/apic.c +1 -3
@@ -1594,7 +1594,7 @@ int __init enable_IR(void)void __init enable_IR_x2apic (void ) {unsigned long flags; -int ret, x2apic_enabled =0 ; +int ret;int hardware_init_ret;if (!IS_ENABLED (CONFIG_X86_X2APIC )) {
@@ -1652,8 +1652,6 @@ void __init enable_IR_x2apic(void) goto skip_x2apic; } - x2apic_enabled =1 ; -if (x2apic_supported () && !x2apic_mode) { x2apic_mode =1 ;enable_x2apic ();
Signature · 1 line
1.7.10.4
PATCH 7/17 x86/apic: Correctly detect X2APIC status in function enable_IR()
In reply to Jiang LiuX2APIC will be disabled if user specifies "nox2apic" on kernel command line, even when x2apic_preenabled is true. So correctly detect X2APIC status by using x2apic_enabled() instead of x2apic_preenabled.
--- arch/x86/kernel/apic/apic.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kernel/apic/apic.c +1 -1
@@ -1580,7 +1580,7 @@ int __init enable_IR(void)return -1 ; } -if (!x2apic_preenabled && skip_ioapic_setup) { +if (!x2apic_enabled () && skip_ioapic_setup) {pr_info ("Skipped enabling intr-remap because of skipping " "io-apic setup\n" );return -1 ;
Signature · 1 line
1.7.10.4
PATCH 8/17 x86/apic: Refine enable_IR_x2apic() and related functions
In reply to Jiang LiuRefine enable_IR_x2apic() and related functions for better readability.
It also changes the way to handle IR in XAPIC mode when enabling X2APIC. Previously it just skips X2APIC initialization without checking max CPU APIC ID in system, which may cause problem if system has CPU with APIC ID bigger than 255. So treat IR in XAPIC mode as same as IR is disabled when enabling CPU X2APIC.
--- arch/x86/kernel/apic/apic.c | 85 +++++++++++++++++++++---------------------- 1 file changed, 41 insertions(+), 44 deletions(-)
arch/x86/kernel/apic/apic.c +41 -44
@@ -1572,7 +1572,7 @@ void enable_x2apic(void) }#endif /* CONFIG_X86_X2APIC */ -int __init enable_IR (void ) +static int __init try_to_enable_IR (void ) {#ifdef CONFIG_IRQ_REMAP if (!irq_remapping_supported ()) {
@@ -1585,17 +1585,48 @@ int __init enable_IR(void) "io-apic setup\n");return -1 ; } +#endif return irq_remapping_enable (); +} + +static __init void try_to_enable_x2apic (int ir_stat) +{ +#ifdef CONFIG_X86_X2APIC +if (!x2apic_supported ()) +return ; + +if (ir_stat !=IRQ_REMAP_X2APIC_MODE ) { +/* IR is required if there is APIC ID > 255 even when running +* under KVM +*/ +if (max_physical_apicid >255 || + !hypervisor_x2apic_available ()) { +pr_info ("IRQ remapping doesn't support X2APIC mode, disable x2apic.\n" ); +if (x2apic_preenabled) +disable_x2apic (); +return ; + } + +/* +* without IR all CPUs can be addressed by IOAPIC/MSI +* only in physical mode +*/ +x2apic_force_phys (); + } + +if (!x2apic_mode) { + x2apic_mode =1 ; +enable_x2apic (); +pr_info ("Enabled x2apic\n" ); + }#endif -return -1 ; }void __init enable_IR_x2apic (void ) {unsigned long flags; -int ret; -int hardware_init_ret; +int ret, ir_stat;if (!IS_ENABLED (CONFIG_X86_X2APIC )) {u64 msr;
@@ -1605,8 +1636,8 @@ void __init enable_IR_x2apic(void)panic ("BIOS has enabled x2apic but kernel doesn't support x2apic, please disable x2apic in BIOS.\n" ); } - hardware_init_ret =irq_remapping_prepare (); -if (hardware_init_ret && !x2apic_supported ()) + ir_stat =irq_remapping_prepare (); +if (ir_stat <0 && !x2apic_supported ())return ; ret =save_ioapic_entries ();
@@ -1621,45 +1652,11 @@ void __init enable_IR_x2apic(void)if (x2apic_preenabled && nox2apic)disable_x2apic (); +if (ir_stat >=0 ) + ir_stat =try_to_enable_IR (); +try_to_enable_x2apic (ir_stat); -if (hardware_init_ret) - ret =-1 ; -else - ret =enable_IR (); - -if (!x2apic_supported ()) - goto skip_x2apic; - -if (ret <0 ) { -/* IR is required if there is APIC ID > 255 even when running -* under KVM -*/ -if (max_physical_apicid >255 || - !hypervisor_x2apic_available ()) { -if (x2apic_preenabled) -disable_x2apic (); - goto skip_x2apic; - } -/* -* without IR all CPUs can be addressed by IOAPIC/MSI -* only in physical mode -*/ -x2apic_force_phys (); - } - -if (ret ==IRQ_REMAP_XAPIC_MODE ) { -pr_info ("x2apic not enabled, IRQ remapping is in xapic mode\n" ); - goto skip_x2apic; - } - -if (x2apic_supported () && !x2apic_mode) { - x2apic_mode =1 ; -enable_x2apic (); -pr_info ("Enabled x2apic\n" ); - } - -skip_x2apic: -if (ret <0 )/* IR enabling failed */ +if (ir_stat <0 )/* IR enabling failed */ restore_ioapic_entries (); legacy_pic->restore_mask ();local_irq_restore (flags);
Signature · 1 line
1.7.10.4
PATCH 9/17 iommu/vt-d: Prepare for killing function irq_remapping_supported()
In reply to Jiang LiuPrepare for killing function irq_remapping_supported() by moving code from intel_irq_remapping_supported() into intel_prepare_irq_remapping().
Combined with patch from Joerg at https://lkml.org/lkml/2014/12/15/487,
so assume an signed-off from Joerg.
--- drivers/iommu/intel_irq_remapping.c | 58 +++++++++++++++-------------------- 1 file changed, 25 insertions(+), 33 deletions(-)
drivers/iommu/intel_irq_remapping.c +25 -33
@@ -568,30 +568,6 @@ static int __init dmar_x2apic_optout(void)static int __init intel_irq_remapping_supported (void ) { -struct dmar_drhd_unit *drhd; -struct intel_iommu *iommu; - -if (disable_irq_remap) -return 0 ; -if (irq_remap_broken) { -printk (KERN_WARNING -"This system BIOS has enabled interrupt remapping\n" -"on a chipset that contains an erratum making that\n" -"feature unstable. To maintain system stability\n" -"interrupt remapping is being disabled. Please\n" -"contact your BIOS vendor for an update\n" ); -add_taint (TAINT_FIRMWARE_WORKAROUND ,LOCKDEP_STILL_OK ); - disable_irq_remap =1 ; -return 0 ; - } - -if (!dmar_ir_support ()) -return 0 ; - -for_each_iommu (iommu, drhd) -if (!ecap_ir_support (iommu->ecap)) -return 0 ; -return 1 ; }
@@ -616,26 +592,42 @@ static int __init intel_prepare_irq_remapping(void)struct dmar_drhd_unit *drhd;struct intel_iommu *iommu; +/* First check whether IRQ remapping should be enabled */ +if (disable_irq_remap) +return -ENODEV ; + +if (irq_remap_broken) { +printk (KERN_WARNING +"This system BIOS has enabled interrupt remapping\n" +"on a chipset that contains an erratum making that\n" +"feature unstable. To maintain system stability\n" +"interrupt remapping is being disabled. Please\n" +"contact your BIOS vendor for an update\n" ); +add_taint (TAINT_FIRMWARE_WORKAROUND ,LOCKDEP_STILL_OK ); + disable_irq_remap =1 ; +return -ENODEV ; + } +if (dmar_table_init () <0 ) -return -1 ; +return -ENODEV ; + +if (!dmar_ir_support ()) +return -ENODEV ;if (parse_ioapics_under_ir () !=1 ) {printk (KERN_INFO "Not enabling interrupt remapping\n" ); goto error; } -for_each_iommu (iommu, drhd) { -if (!ecap_ir_support (iommu->ecap)) -continue ; - -/* Do the allocations early */ -if (intel_setup_irq_remapping (iommu)) +for_each_iommu (iommu, drhd) +if (!ecap_ir_support (iommu->ecap) || +intel_setup_irq_remapping (iommu)) goto error; - }return 0 ; + error:intel_cleanup_irq_remapping (); -return -1 ; +return -ENODEV ; }static int __init intel_enable_irq_remapping (void )
Signature · 1 line
1.7.10.4
PATCH 10/17 iommu/vt-d: Allocate IRQ remapping data structures only for all IOMMUs
In reply to Jiang LiuFrom: Joerg Roedel <[email protected]>
IRQ remapping is only supported when all IOMMUs in the system support it. So check if all IOMMUs in the system support IRQ remapping before doing the allocations.
[Jiang]
1) Rebased onto v3.19.
2) Remove redundant check of ecap_ir_support(iommu->ecap) in function intel_enable_irq_remapping().
--- drivers/iommu/intel_irq_remapping.c | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-)
drivers/iommu/intel_irq_remapping.c +9 -10
@@ -619,10 +619,16 @@ static int __init intel_prepare_irq_remapping(void) goto error; } +/* First make sure all IOMMUs support IRQ remapping */ for_each_iommu (iommu, drhd) -if (!ecap_ir_support (iommu->ecap) || -intel_setup_irq_remapping (iommu)) +if (!ecap_ir_support (iommu->ecap)) + goto error; + +/* Do the allocations early */ +for_each_iommu (iommu, drhd) +if (intel_setup_irq_remapping (iommu)) goto error; +return 0 ; error:
@@ -673,16 +679,12 @@ static int __init intel_enable_irq_remapping(void)/* * check for the Interrupt-remapping support */ -for_each_iommu (iommu, drhd) { -if (!ecap_ir_support (iommu->ecap)) -continue ; - +for_each_iommu (iommu, drhd)if (eim && !ecap_eim_support (iommu->ecap)) {printk (KERN_INFO "DRHD %Lx: EIM not supported by DRHD, " " ecap %Lx\n" , drhd->reg_base_addr, iommu->ecap); goto error; } - }/* * Enable queued invalidation for all the DRHD's.
@@ -702,9 +704,6 @@ static int __init intel_enable_irq_remapping(void)* Setup Interrupt-remapping for all the DRHD's now. */ for_each_iommu (iommu, drhd) { -if (!ecap_ir_support (iommu->ecap)) -continue ; -iommu_set_irq_remapping (iommu, eim); setup =1 ; }
Signature · 1 line
1.7.10.4
PATCH 11/17 iommu/vt-d: Allow IR works in XAPIC mode though CPU works in X2APIC mode
In reply to Jiang LiuCurrently if CPU supports X2APIC, IR hardware must work in X2APIC mode or disabled. Change the code to support IR working in XAPIC mode when CPU supports X2APIC. Then the CPU APIC driver will decide how to handle such as configuration by:
1) Disabling X2APIC mode
2) Forcing X2APIC physical mode
This change also fixes a live locking when
1) BIOS enables CPU X2APIC
2) DMAR table disables X2APIC mode or IR hardware doesn't support X2APIC with following messages:
[ 37.863463] dmar: INTR-REMAP: Request device [[f0:1f.7] fault index 2 [ 37.863463] INTR-REMAP:[fault reason 36] Detected reserved fields in the IRTE entry [ 37.879372] dmar: INTR-REMAP: Request device [[f0:1f.7] fault index 2 [ 37.879372] INTR-REMAP:[fault reason 36] Detected reserved fields in the IRTE entry [ 37.895282] dmar: INTR-REMAP: Request device [[f0:1f.7] fault index 2 [ 37.895282] INTR-REMAP:[fault reason 36] Detected reserved fields in the IRTE entry [ 37.911192] dmar: INTR-REMAP: Request device [[f0:1f.7] fault index 2
--- drivers/iommu/intel_irq_remapping.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
drivers/iommu/intel_irq_remapping.c +6 -4
@@ -32,8 +32,9 @@ struct hpet_scope { };#define IR_X2APIC_MODE (mode) (mode ? (1 << 11) : 0) -#define IRTE_DEST (dest) ((x2apic_mode) ? dest : dest << 8) +#define IRTE_DEST (dest) ((eim_mode) ? dest : dest << 8) +static int __read_mostly eim_mode;static struct ioapic_scope ir_ioapic[MAX_IO_APICS ];static struct hpet_scope ir_hpet[MAX_HPET_TBS ];
@@ -644,8 +645,6 @@ static int __init intel_enable_irq_remapping(void)int eim =0 ;if (x2apic_supported ()) { -pr_info ("Queued invalidation will be enabled to support x2apic and Intr-remapping.\n" ); - eim = !dmar_x2apic_optout ();if (!eim) printk(KERN_WARNING
@@ -683,8 +682,11 @@ static int __init intel_enable_irq_remapping(void)if (eim && !ecap_eim_support (iommu->ecap)) {printk (KERN_INFO "DRHD %Lx: EIM not supported by DRHD, " " ecap %Lx\n" , drhd->reg_base_addr, iommu->ecap); - goto error; + eim =0 ; } + eim_mode = eim; +if (eim) +pr_info ("Queued invalidation will be enabled to support x2apic and Intr-remapping.\n" );/* * Enable queued invalidation for all the DRHD's.
Signature · 1 line
1.7.10.4
PATCH 12/17 x86/apic: Only disable CPU x2apic mode when necessary
In reply to Jiang LiuWhen interrupt remapping hardware is not in X2APIC, CPU X2APIC mode will be disabled if:
1) Maximum CPU APIC ID is bigger than 255
2) hypervisior doesn't support x2apic mode.
But we should only check whether hypervisor supports X2APIC mode when hypervisor(CONFIG_HYPERVISOR_GUEST) is enabled, otherwise X2APIC will always be disabled when CONFIG_HYPERVISOR_GUEST is disabled and IR doesn't work in X2APIC mode.
--- arch/x86/kernel/apic/apic.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kernel/apic/apic.c +2 -1
@@ -1601,7 +1601,8 @@ static __init void try_to_enable_x2apic(int ir_stat)* under KVM */ if (max_physical_apicid >255 || - !hypervisor_x2apic_available ()) { + (IS_ENABLED (CONFIG_HYPERVISOR_GUEST ) && + !hypervisor_x2apic_available ())) {pr_info ("IRQ remapping doesn't support X2APIC mode, disable x2apic.\n" );if (x2apic_preenabled)disable_x2apic ();
Signature · 1 line
1.7.10.4
PATCH 13/17 iommu/amd: Check for irq-remap support amd_iommu_prepare()
In reply to Jiang LiuFrom: Joerg Roedel <[email protected]>
This allows to get rid of the irq_remapping_supported() function and all its call-backs into the Intel and AMD IOMMU drivers.
--- drivers/iommu/amd_iommu_init.c | 3 +++ 1 file changed, 3 insertions(+)
drivers/iommu/amd_iommu_init.c +3 -0
@@ -2123,6 +2123,9 @@ static int __init iommu_go_to_state(enum iommu_init_state state)#ifdef CONFIG_IRQ_REMAP int __init amd_iommu_prepare (void ) { +if (!amd_iommu_irq_remap) +return -1 ; +return iommu_go_to_state (IOMMU_ACPI_FINISHED ); }
Signature · 1 line
1.7.10.4
PATCH 14/17 iommu/irq_remapping: Kill function irq_remapping_supported() and related code
In reply to Jiang LiuSimplify irq_remapping code by killing irq_remapping_supported() and related interfaces.
Joerg posted a similar patch at https://lkml.org/lkml/2014/12/15/490,
so assume an signed-off from Joerg.
--- arch/x86/include/asm/irq_remapping.h | 2 -- arch/x86/kernel/apic/apic.c | 7 +------ drivers/iommu/amd_iommu.c | 1 - drivers/iommu/amd_iommu_init.c | 5 ----- drivers/iommu/amd_iommu_proto.h | 1 - drivers/iommu/intel_irq_remapping.c | 6 ------ drivers/iommu/irq_remapping.c | 12 ++---------- drivers/iommu/irq_remapping.h | 3 --- 8 files changed, 3 insertions(+), 34 deletions(-)
arch/x86/include/asm/irq_remapping.h +0 -2
@@ -33,7 +33,6 @@ struct irq_cfg;#ifdef CONFIG_IRQ_REMAP -extern int irq_remapping_supported (void );extern void set_irq_remapping_broken (void );extern int irq_remapping_prepare (void );extern int irq_remapping_enable (void );
@@ -59,7 +58,6 @@ void irq_remap_modify_chip_defaults(struct irq_chip *chip);#else /* CONFIG_IRQ_REMAP */ -static inline int irq_remapping_supported (void ) {return 0 ; }static inline void set_irq_remapping_broken (void ) { }static inline int irq_remapping_prepare (void ) {return -ENODEV ; }static inline int irq_remapping_enable (void ) {return -ENODEV ; }
arch/x86/kernel/apic/apic.c +1 -6
@@ -1574,12 +1574,7 @@ void enable_x2apic(void)static int __init try_to_enable_IR (void ) { -#ifdef CONFIG_IRQ_REMAP -if (!irq_remapping_supported ()) { -pr_debug ("intr-remapping not supported\n" ); -return -1 ; - } - +#ifdef CONFIG_X86_IO_APIC if (!x2apic_enabled () && skip_ioapic_setup) {pr_info ("Skipped enabling intr-remap because of skipping " "io-apic setup\n" );
drivers/iommu/amd_iommu.c +0 -1
@@ -4284,7 +4284,6 @@ static int alloc_hpet_msi(unsigned int irq, unsigned int id) }struct irq_remap_ops amd_iommu_irq_ops = { - .supported = amd_iommu_supported, .prepare = amd_iommu_prepare, .enable = amd_iommu_enable, .disable = amd_iommu_disable,
drivers/iommu/amd_iommu_init.c +0 -5
@@ -2129,11 +2129,6 @@ int __init amd_iommu_prepare(void)return iommu_go_to_state (IOMMU_ACPI_FINISHED ); } -int __init amd_iommu_supported (void ) -{ -return amd_iommu_irq_remap ?1 :0 ; -} -int __init amd_iommu_enable (void ) {int ret;
drivers/iommu/amd_iommu_proto.h +0 -1
@@ -33,7 +33,6 @@ extern void amd_iommu_init_notifier(void);extern void amd_iommu_init_api (void );/* Needed for interrupt remapping */ -extern int amd_iommu_supported (void );extern int amd_iommu_prepare (void );extern int amd_iommu_enable (void );extern void amd_iommu_disable (void );
drivers/iommu/intel_irq_remapping.c +0 -6
@@ -567,11 +567,6 @@ static int __init dmar_x2apic_optout(void)return dmar->flags &DMAR_X2APIC_OPT_OUT ; } -static int __init intel_irq_remapping_supported (void ) -{ -return 1 ; -} -static void __init intel_cleanup_irq_remapping (void ) {struct dmar_drhd_unit *drhd;
@@ -1216,7 +1211,6 @@ static int intel_alloc_hpet_msi(unsigned int irq, unsigned int id) }struct irq_remap_ops intel_irq_remap_ops = { - .supported = intel_irq_remapping_supported, .prepare = intel_prepare_irq_remapping, .enable = intel_enable_irq_remapping, .disable = disable_irq_remapping,
drivers/iommu/irq_remapping.c +2 -10
@@ -199,19 +199,11 @@ void set_irq_remapping_broken(void) irq_remap_broken =1 ; } -int irq_remapping_supported (void ) +int __init irq_remapping_prepare (void ) {if (disable_irq_remap) -return 0 ; - -if (!remap_ops || !remap_ops->supported) -return 0 ; +return -ENOSYS ; -return remap_ops->supported (); -} - -int __init irq_remapping_prepare (void ) -{ remap_ops = &intel_irq_remap_ops;#ifdef CONFIG_AMD_IOMMU
drivers/iommu/irq_remapping.h +0 -3
@@ -38,9 +38,6 @@ extern int no_x2apic_optout;extern int irq_remapping_enabled;struct irq_remap_ops { -/* Check whether Interrupt Remapping is supported */ -int (*supported)(void ); -/* Initializes hardware and makes it ready for remapping interrupts */ int (*prepare)(void );
Signature · 1 line
1.7.10.4
PATCH 15/17 iommu/irq_remapping: Refine function irq_remapping_prepare() for maintenance
In reply to Jiang LiuAssign intel_irq_remap_ops to remap_ops only ifintel_irq_remap_ops.prepare() succeeds.
--- drivers/iommu/irq_remapping.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
drivers/iommu/irq_remapping.c +8 -8
@@ -204,15 +204,15 @@ int __init irq_remapping_prepare(void)if (disable_irq_remap)return -ENOSYS ; - remap_ops = &intel_irq_remap_ops; - -#ifdef CONFIG_AMD_IOMMU -if (amd_iommu_irq_ops.prepare () ==0 ) { +if (intel_irq_remap_ops.prepare () ==0 ) + remap_ops = &intel_irq_remap_ops; +else if (IS_ENABLED (CONFIG_AMD_IOMMU ) && + amd_iommu_irq_ops.prepare () ==0 ) remap_ops = &amd_iommu_irq_ops; -return 0 ; - } -#endif -return remap_ops->prepare (); +else +return -ENOSYS ; + +return 0 ; }int __init irq_remapping_enable (void )
Signature · 1 line
1.7.10.4
PATCH 16/17 iommu/irq_remapping: Change variable disable_irq_remap to be static
In reply to Jiang LiuChange variable disable_irq_remap to be static and simplify the code.
--- drivers/iommu/amd_iommu_init.c | 6 +----- drivers/iommu/intel_irq_remapping.c | 5 ----- drivers/iommu/irq_remapping.c | 3 +-- drivers/iommu/irq_remapping.h | 2 -- 4 files changed, 2 insertions(+), 14 deletions(-)
drivers/iommu/amd_iommu_init.c +1 -5
@@ -2014,9 +2014,6 @@ static bool detect_ivrs(void)/* Make sure ACS will be enabled during PCI probe */ pci_request_acs (); -if (!disable_irq_remap) - amd_iommu_irq_remap = true; -return true; }
@@ -2123,8 +2120,7 @@ static int __init iommu_go_to_state(enum iommu_init_state state)#ifdef CONFIG_IRQ_REMAP int __init amd_iommu_prepare (void ) { -if (!amd_iommu_irq_remap) -return -1 ; + amd_iommu_irq_remap = true;return iommu_go_to_state (IOMMU_ACPI_FINISHED ); }
drivers/iommu/intel_irq_remapping.c +0 -5
@@ -588,10 +588,6 @@ static int __init intel_prepare_irq_remapping(void)struct dmar_drhd_unit *drhd;struct intel_iommu *iommu; -/* First check whether IRQ remapping should be enabled */ -if (disable_irq_remap) -return -ENODEV ; -if (irq_remap_broken) {printk (KERN_WARNING "This system BIOS has enabled interrupt remapping\n"
@@ -600,7 +596,6 @@ static int __init intel_prepare_irq_remapping(void)"interrupt remapping is being disabled. Please\n" "contact your BIOS vendor for an update\n");add_taint (TAINT_FIRMWARE_WORKAROUND ,LOCKDEP_STILL_OK ); - disable_irq_remap =1 ;return -ENODEV ; }
drivers/iommu/irq_remapping.c +1 -2
@@ -17,12 +17,11 @@#include "irq_remapping.h" int irq_remapping_enabled; - -int disable_irq_remap;int irq_remap_broken;int disable_sourceid_checking;int no_x2apic_optout; +static int disable_irq_remap;static struct irq_remap_ops *remap_ops;static int msi_alloc_remapped_irq (struct pci_dev *pdev,int irq,int nvec);
drivers/iommu/irq_remapping.h +0 -2
@@ -31,7 +31,6 @@ struct cpumask;struct pci_dev ;struct msi_msg ; -extern int disable_irq_remap;extern int irq_remap_broken;extern int disable_sourceid_checking;extern int no_x2apic_optout;
@@ -86,7 +85,6 @@ extern struct irq_remap_ops amd_iommu_irq_ops;#else /* CONFIG_IRQ_REMAP */ #define irq_remapping_enabled 0 -#define disable_irq_remap 1#define irq_remap_broken 0#endif /* CONFIG_IRQ_REMAP */
Signature · 1 line
1.7.10.4
PATCH 17/17 iommu/irq_remapping: Normailize the way to detect whether IR is enabled
In reply to Jiang LiuRefine code by normailizing the way to detect whether IR is enabled.
--- drivers/iommu/irq_remapping.c | 38 ++++++++++++++------------------------ 1 file changed, 14 insertions(+), 24 deletions(-)
drivers/iommu/irq_remapping.c +14 -24
@@ -218,7 +218,7 @@ int __init irq_remapping_enable(void) {int ret; -if (!remap_ops || !remap_ops->enable) +if (!remap_ops->enable)return -ENODEV ; ret = remap_ops->enable ();
@@ -231,22 +231,16 @@ int __init irq_remapping_enable(void)void irq_remapping_disable (void ) { -if (!irq_remapping_enabled || - !remap_ops || - !remap_ops->disable) -return ; - - remap_ops->disable (); +if (irq_remapping_enabled && remap_ops->disable) + remap_ops->disable (); }int irq_remapping_reenable (int mode) { -if (!irq_remapping_enabled || - !remap_ops || - !remap_ops->reenable) -return 0 ; +if (irq_remapping_enabled && remap_ops->reenable) +return remap_ops->reenable (mode); -return remap_ops->reenable (mode); +return 0 ; }int __init irq_remap_enable_fault_handling (void )
@@ -254,7 +248,7 @@ int __init irq_remap_enable_fault_handling(void)if (!irq_remapping_enabled)return 0 ; -if (!remap_ops || !remap_ops->enable_faulting) +if (!remap_ops->enable_faulting)return -ENODEV ;return remap_ops->enable_faulting ();
@@ -265,7 +259,7 @@ int setup_ioapic_remapped_entry(int irq, unsigned int destination, int vector, struct io_apic_irq_attr *attr) { - if (!remap_ops || !remap_ops->setup_ioapic_entry) + if (!remap_ops->setup_ioapic_entry) return -ENODEV; return remap_ops->setup_ioapic_entry(irq, entry, destination,
@@ -275,8 +269,7 @@ int setup_ioapic_remapped_entry(int irq,static int set_remapped_irq_affinity (struct irq_data *data,const struct cpumask *mask,bool force) { -if (!config_enabled (CONFIG_SMP ) || !remap_ops || - !remap_ops->set_affinity) +if (!config_enabled (CONFIG_SMP ) || !remap_ops->set_affinity)return 0 ;return remap_ops->set_affinity (data, mask, force);
@@ -286,10 +279,7 @@ void free_remapped_irq(int irq) {struct irq_cfg *cfg =irq_get_chip_data (irq); -if (!remap_ops || !remap_ops->free_irq) -return ; - -if (irq_remapped (cfg)) +if (irq_remapped (cfg) && remap_ops->free_irq) remap_ops->free_irq (irq); }
@@ -301,13 +291,13 @@ void compose_remapped_msi_msg(struct pci_dev *pdev,if (!irq_remapped (cfg))native_compose_msi_msg (pdev, irq, dest, msg, hpet_id); -else if (remap_ops && remap_ops->compose_msi_msg) +else if (remap_ops->compose_msi_msg) remap_ops->compose_msi_msg (pdev, irq, dest, msg, hpet_id); }static int msi_alloc_remapped_irq (struct pci_dev *pdev,int irq,int nvec) { -if (!remap_ops || !remap_ops->msi_alloc_irq) +if (!remap_ops->msi_alloc_irq)return -ENODEV ;return remap_ops->msi_alloc_irq (pdev, irq, nvec);
@@ -316,7 +306,7 @@ static int msi_alloc_remapped_irq(struct pci_dev *pdev, int irq, int nvec)static int msi_setup_remapped_irq (struct pci_dev *pdev,unsigned int irq,int index,int sub_handle) { -if (!remap_ops || !remap_ops->msi_setup_irq) +if (!remap_ops->msi_setup_irq)return -ENODEV ;return remap_ops->msi_setup_irq (pdev, irq, index, sub_handle);
@@ -326,7 +316,7 @@ int setup_hpet_msi_remapped(unsigned int irq, unsigned int id) {int ret; -if (!remap_ops || !remap_ops->alloc_hpet_msi) +if (!remap_ops->alloc_hpet_msi)return -ENODEV ; ret = remap_ops->alloc_hpet_msi (irq, id);
Signature · 1 line
1.7.10.4