x86: fix probe_nr_irqs for xen
PATCH 1/1 x86: fix probe_nr_irqs for xen
otherwise Xen is _completely_ unusable with 5 or more VCPUs. ( when !CONFIG_HAVE_SPARSE_IRQ)
based on Alex's patch
also add +1 offset after redir_entries
arch/x86/kernel/io_apic.c +8 -3
@@ -3625,16 +3625,21 @@ int __init probe_nr_irqs(void) {int idx;int nr =0 ; +#ifndef CONFIG_XEN +int nr_min =32 ; +#else +int nr_min =NR_IRQS ; +#endif for (idx =0 ; idx < nr_ioapics; idx++) - nr +=io_apic_get_redir_entries (idx); + nr +=io_apic_get_redir_entries (idx) +1 ;/* double it for hotplug and msi and nmi */ nr <<=1 ;/* something wrong ? */ -if (nr <32 ) - nr =32 ; +if (nr < nr_min) + nr = nr_min;return nr; }
Quoted 10 lines · Yinghai Lu
otherwise Xen is _completely_ unusable with 5 or more VCPUs. ( when !CONFIG_HAVE_SPARSE_IRQ)
based on Alex's patch
also add +1 offset after
redir_entriesSigned-off-by Yinghai Lu <[email protected]>Cc Alex Nixon <[email protected]>
Looks good to me. We can fix this in a better way later.
Quoted 35 lines
Index:
linux-2.6/arch/x86/kernel/io_apic.c
===================================================================arch/x86/kernel/io_apic.c +8
-4@@ -3625,16 +3625,21 @@ int __init probe_nr_irqs(void) {int idx;int nr =0 ; +#ifndef CONFIG_XEN +int nr_min =32 ; +#else +int nr_min =NR_IRQS ; +#endif for (idx =0 ; idx < nr_ioapics; idx++) - nr +=io_apic_get_redir_entries (idx); + nr +=io_apic_get_redir_entries (idx) +1 ;/* double it for hotplug and msi and nmi */ nr <<=1 ;/* something wrong ? */ -if (nr <32 ) - nr =32 ; +if (nr < nr_min) + nr = nr_min;return nr; } --To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Quoted 6 lines · Jeremy Fitzhardinge
Quoted 5 lines · Yinghai Lu
otherwise Xen is _completely_ unusable with 5 or more VCPUs. ( when !CONFIG_HAVE_SPARSE_IRQ)
based on Alex's patch
Quoted 7 lines
Quoted 5 lines
also add +1 offset after
redir_entriesSigned-off-by Yinghai Lu <[email protected]>Cc Alex Nixon <[email protected]>Looks good to me. We can fix this in a better way later.
later could set CONFIG_HAVE_SPARSE_IRQ to true by default.
YH
Quoted 11 lines · Yinghai Lu
Quoted 9 lines · Jeremy Fitzhardinge
Quoted 7 lines · Yinghai Lu
otherwise Xen is _completely_ unusable with 5 or more VCPUs. ( when !CONFIG_HAVE_SPARSE_IRQ)
based on Alex's patch
also add +1 offset after
redir_entries
Quoted 9 lines
Quoted 6 lines
Quoted 4 lines
Signed-off-by Yinghai Lu <[email protected]>Cc Alex Nixon <[email protected]>Looks good to me. We can fix this in a better way later.
later could set
CONFIG_HAVE_SPARSE_IRQto true by default.
Well, perhaps. But we can use vectors rather than IRQs for Xen event channel mappings, which will save us from having to allocate lots of per-cpu irqs.
Also, implementing probe_nr_irqs() in a more generic (rather than io_apic-specific) way. Using some interface that queries each irq chip for how many irqs it supports, or something.
J
Quoted 19 lines · Jeremy Fitzhardinge
Quoted 18 lines · Yinghai Lu
Quoted 15 lines · Jeremy Fitzhardinge
Quoted 11 lines · Yinghai Lu
otherwise Xen is _completely_ unusable with 5 or more VCPUs. ( when !CONFIG_HAVE_SPARSE_IRQ)
based on Alex's patch
also add +1 offset after
redir_entriesSigned-off-by Yinghai Lu <[email protected]>Cc Alex Nixon <[email protected]>Looks good to me. We can fix this in a better way later.
Quoted 6 lines
Quoted 2 lines
later could set
CONFIG_HAVE_SPARSE_IRQto true by default.Well, perhaps. But we can use vectors rather than IRQs for Xen event channel mappings, which will save us from having to allocate lots of per-cpu irqs.
like system vectors with smp-ipi etc?
Quoted 4 lines
Also, implementing
probe_nr_irqs()in a more generic (rather than io_apic-specific) way. Using some interface that queries each irq chip for how many irqs it supports, or something.
hope we can kill nr_irqs/NR_IRQS
YH
Quoted 2 lines · Yinghai Lu
like system vectors with smp-ipi etc?
More or less. Xen currently uses 6 per-cpu event channels, and it maps each to its own irq. My plan is to allocate just 6 irqs, map 6 vectors to them, and bind each event channel to a vector/cpu pair. If nothing else, it will make /proc/interrupts somewhat sane again.
Quoted 7 lines
Quoted 4 lines
Also, implementing
probe_nr_irqs()in a more generic (rather than io_apic-specific) way. Using some interface that queries each irq chip for how many irqs it supports, or something.hope we can kill nr_irqs/NR_IRQS
When irqs are truely dynamically allocated?
J
Quoted 15 lines · Jeremy Fitzhardinge
Quoted 2 lines · Yinghai Lu
like system vectors with smp-ipi etc?
More or less. Xen currently uses 6 per-cpu event channels, and it maps each to its own irq. My plan is to allocate just 6 irqs, map 6 vectors to them, and bind each event channel to a vector/cpu pair. If nothing else, it will make
/proc/interruptssomewhat sane again.Quoted 5 lines
Quoted 4 lines
Also, implementing
probe_nr_irqs()in a more generic (rather than io_apic-specific) way. Using some interface that queries each irq chip for how many irqs it supports, or something.
Quoted 5 lines
Quoted 2 lines
hope we can kill nr_irqs/NR_IRQS
When irqs are truely dynamically allocated?
need more test with sparse_irq
YH
Quoted 3 lines · Jeremy Fitzhardinge
Quoted 1 line
hope we can kill nr_irqs/NR_IRQS
When irqs are truely dynamically allocated?
they in essence already are truly dynamic in tip/master. Not yet unconditionally so on x86, we chose to also have a !SPARSEIRQ build model to make sure dense IRQ arrays still work fine as well. (that's far easier to test and validate than to keep a farm of cross compilers to other arches alive.)
But by all means SPARSEIRQ is the primary and only model on x86.
We could even plug in the dynamic vector allocator from Alan Mayer now, both on 32-bit and on 64-bit x86: Yinghai, any complications expected there? I guess we need to unify the local APIC code first.
Ingo
Quoted 13 lines · Ingo Molnar
Quoted 3 lines · Jeremy Fitzhardinge
Quoted 1 line
hope we can kill nr_irqs/NR_IRQS
When irqs are truely dynamically allocated?
they in essence already are truly dynamic in tip/master. Not yet unconditionally so on x86, we chose to also have a !SPARSEIRQ build model to make sure dense IRQ arrays still work fine as well. (that's far easier to test and validate than to keep a farm of cross compilers to other arches alive.)
Quoted 5 lines
But by all means SPARSEIRQ is the primary and only model on x86.
We could even plug in the dynamic vector allocator from Alan Mayer now, both on 32-bit and on 64-bit x86: Yinghai, any complications expected there? I guess we need to unify the local APIC code first.
apic_32.c and apic_64.cirqinit_32.c and irqinit_64.cirq_32.c and irq_64.c
Cyrill, is still working on merging apic_32.c apic_64.c.?
YH
Quoted 11 lines · Yinghai Lu
Quoted 5 lines
But by all means SPARSEIRQ is the primary and only model on x86.
We could even plug in the dynamic vector allocator from Alan Mayer now, both on 32-bit and on 64-bit x86: Yinghai, any complications expected there? I guess we need to unify the local APIC code first.
apic_32.candapic_64.cirqinit_32.candirqinit_64.cirq_32.candirq_64.cCyrill, is still working on merging
apic_32.capic_64.c.?
there's a few weeks of inactivity expected in that area i think - so if you'd like to pick it up from there, feel free ...
but apic_32/64.c is looking already pretty good in tip/master:
1 file changed, 391 insertions(+), 358 deletions(-)
the most significant bit of that delta seems to be around x2apic, calibration differences and some older 32-bit legacies which are not on the 64-bit side (but which we can add safely).
Ingo
[Ingo Molnar - Fri, Aug 22, 2008 at 07:16:56AM +0200]
| | * Yinghai Lu <[email protected]> wrote: | | > > But by all means SPARSEIRQ is the primary and only model on x86. | > > | > > We could even plug in the dynamic vector allocator from Alan Mayer | > > now, both on 32-bit and on 64-bit x86: Yinghai, any complications | > > expected there? I guess we need to unify the local APIC code first. | > | > apic_32.c and apic_64.c | > irqinit_32.c and irqinit_64.c | > irq_32.c and irq_64.c | > | > Cyrill, is still working on merging apic_32.c apic_64.c.? | | there's a few weeks of inactivity expected in that area i think - so if | you'd like to pick it up from there, feel free ... | | but apic_32/64.c is looking already pretty good in tip/master: | | 1 file changed, 391 insertions(+), 358 deletions(-) | | the most significant bit of that delta seems to be around x2apic, | calibration differences and some older 32-bit legacies which are not on | the 64-bit side (but which we can add safely). | | Ingo |
Yes - I will be busy for a few weeks with other duties, sorry for that. There not that much diff rest:
- apic clock calibration
- x2apic operations
- interrupt handlers (different declarations)
- command line handling
ugh... maybe not that small heap of todo's :)
Anyway - before sending patches I'll fetch tip/x86/apic branch so no problem if someone bring any changes into the branch.
- Cyrill -
Ingo Molnar
AppliedQuoted 8 lines · Jeremy Fitzhardinge
Quoted 7 lines · Yinghai Lu
otherwise Xen is _completely_ unusable with 5 or more VCPUs. ( when !CONFIG_HAVE_SPARSE_IRQ)
based on Alex's patch
also add +1 offset after
redir_entries
Quoted 7 lines
Quoted 3 lines
Signed-off-by Yinghai Lu <[email protected]>Cc Alex Nixon <[email protected]>Looks good to me. We can fix this in a better way later.
applied to tip/irq/sparseirq.
Alex, does this approach work for you too?
Ingo
Quoted 15 lines · Ingo Molnar
Quoted 13 lines · Jeremy Fitzhardinge
Quoted 10 lines · Yinghai Lu
otherwise Xen is _completely_ unusable with 5 or more VCPUs. ( when !CONFIG_HAVE_SPARSE_IRQ)
based on Alex's patch
also add +1 offset after
redir_entriesSigned-off-by Yinghai Lu <[email protected]>Cc Alex Nixon <[email protected]>Looks good to me. We can fix this in a better way later.
Quoted 7 lines
Quoted 1 line
applied to tip/irq/sparseirq.
Alex, does this approach work for you too?
Ingo
Yep.
- Alex