Multiple vector domain support - cpu and domain

RFC PATCH In review Kenji Kaneshige 4 participants 5 messages Jul 14, 2005 – Jul 16, 2005

Share & tools

Kenji Kaneshige

author

2005-07-14 09:20 UTC #0 lore raw

RFC PATCH 2/10 Multiple vector domain support - cpu and domain

This patch add the code to handle the relationship between cpu and domains. We need more consideration about how to separate vector domains.

Signed-off-by Kenji Kaneshige <[email protected]>

---

 linux-2.6.13-rc1-kanesige/arch/ia64/kernel/irq_ia64.c |   24 ++++++++++++++++++
 linux-2.6.13-rc1-kanesige/arch/ia64/kernel/setup.c    |    2 +
 linux-2.6.13-rc1-kanesige/arch/ia64/kernel/smpboot.c  |    2 +
 linux-2.6.13-rc1-kanesige/include/asm-ia64/hw_irq.h   |   13 +++++++++
 4 files changed, 41 insertions(+)

arch/ia64/kernel/irq_ia64.c renamed from arch/ia64/kernel/irq_ia64.c~vector-domain-ia64-attach-cpu +24 -0

diff -puN arch/ia64/kernel/irq_ia64.c~vector-domain-ia64-attach-cpu arch/ia64/kernel/irq_ia64.c
@@ -286,3 +286,27 @@ ia64_send_ipi (int cpu, int vector, int 
 
 	writeq(ipi_data, ipi_addr);
 }
+
+void __init
+ia64_vector_domain_init (void)
+{
+	/* Attach BSP to domain #0 */
+	ia64_attach_cpu_to_domain(0);
+}
+
+#ifdef CONFIG_VECTOR_DOMAIN
+int ia64_cpu_domain_map[NR_CPUS];
+cpumask_t ia64_domain_cpumask[NR_VECTOR_DOMAINS];
+
+void __devinit
+ia64_attach_cpu_to_domain (int cpu)
+{
+	static int domain = -1;
+
+	/* Simple Round Robin for now */
+	if (++domain >= NR_VECTOR_DOMAINS)
+		domain = 0;
+	ia64_cpu_domain_map[cpu] = domain;
+	cpu_set(cpu, ia64_domain_cpumask[domain]);
+}
+#endif /* CONFIG_VECTOR_DOMAIN */

arch/ia64/kernel/setup.c renamed from arch/ia64/kernel/setup.c~vector-domain-ia64-attach-cpu +2 -0

diff -puN arch/ia64/kernel/setup.c~vector-domain-ia64-attach-cpu arch/ia64/kernel/setup.c
@@ -406,6 +406,8 @@ setup_arch (char **cmdline_p)
 
 	cpu_init();	/* initialize the bootstrap CPU */
 
+	ia64_vector_domain_init();
+
 #ifdef CONFIG_ACPI_BOOT
 	acpi_boot_init();
 #endif

arch/ia64/kernel/smpboot.c renamed from arch/ia64/kernel/smpboot.c~vector-domain-ia64-attach-cpu +2 -0

diff -puN arch/ia64/kernel/smpboot.c~vector-domain-ia64-attach-cpu arch/ia64/kernel/smpboot.c
@@ -766,6 +766,8 @@ __cpu_up (unsigned int cpu)
 	int ret;
 	int sapicid;
 
+	ia64_attach_cpu_to_domain(cpu);
+
 	sapicid = ia64_cpu_to_sapicid[cpu];
 	if (sapicid = -1)
 		return -EINVAL;

include/asm-ia64/hw_irq.h renamed from include/asm-ia64/hw_irq.h~vector-domain-ia64-attach-cpu +13 -0

diff -puN include/asm-ia64/hw_irq.h~vector-domain-ia64-attach-cpu include/asm-ia64/hw_irq.h
@@ -79,6 +79,18 @@ enum {
 extern __u8 isa_irq_to_vector_map[16];
 #define isa_irq_to_vector(x)	isa_irq_to_vector_map[(x)]
 
+#ifdef CONFIG_VECTOR_DOMAIN
+extern void ia64_attach_cpu_to_domain(int cpu);
+extern int ia64_cpu_domain_map[NR_CPUS];
+extern cpumask_t ia64_domain_cpumask[NR_VECTOR_DOMAINS];
+#define ia64_cpu_to_domain(cpu)			(ia64_cpu_domain_map[cpu])
+#define ia64_domain_to_cpumask(domain)		(ia64_domain_cpumask[domain])
+#else
+#define ia64_attach_cpu_to_domain(cpu)		do {} while (0)
+#define ia64_cpu_to_domain(cpu)			(cpu^cpu)
+#define ia64_domain_to_cpumask(domain)		(cpu_online_map)
+#endif /* CONFIG_VECTOR_DOMAIN */
+
 extern struct hw_interrupt_type irq_type_ia64_lsapic;	/* CPU-internal interrupt controller */
 
 extern int assign_irq_vector_nopanic (int irq); /* allocate a free vector without panic */

@@ -86,6 +98,7 @@ extern int assign_irq_vector (int irq);	
 extern void free_irq_vector (int vector);
 extern void ia64_send_ipi (int cpu, int vector, int delivery_mode, int redirect);
 extern void register_percpu_irq (ia64_vector vec, struct irqaction *action);
+extern void __init ia64_vector_domain_init(void);
 
 static inline void
 hw_resend_irq (struct hw_interrupt_type *h, unsigned int vector)

Ashok Raj

2005-07-14 18:03 UTC #1 lore raw

Re: [RFC][patch 2/10] Multiple vector domain support - cpu and domain management

Quoted 9 lines · Kenji Kaneshige

This patch add the code to handle the relationship between cpu and domains. We need more consideration about how to separate vector domains.
Signed-off-by Kenji Kaneshige <[email protected]>
---

Should we consider this based on some mach-* like schemes, so for e.g in NUMA case we could use node as a VECTOR_DOMAIN? ... or maybe we could somehow tie this into CPUSETS?..

Quoted 17 lines

arch/ia64/kernel/irq_ia64.c renamed from arch/ia64/kernel/irq_ia64.c~vector-domain-ia64-attach-cpu +17 -0

+
+#ifdef CONFIG_VECTOR_DOMAIN
+int ia64_cpu_domain_map[NR_CPUS];
+cpumask_t ia64_domain_cpumask[NR_VECTOR_DOMAINS];
+
+void __devinit
+ia64_attach_cpu_to_domain (int cpu)
+{
+	static int domain = -1;
+
+	/* Simple Round Robin for now */
+	if (++domain >= NR_VECTOR_DOMAINS)
+		domain = 0;
+	ia64_cpu_domain_map[cpu] = domain;
+	cpu_set(cpu, ia64_domain_cpumask[domain]);
+}
+#endif /* CONFIG_VECTOR_DOMAIN */

Signature · 3 lines

Cheers,
Ashok Raj
- Open Source Technology Center

david mosberger

2005-07-14 18:09 UTC #2 lore raw

Re: [RFC][patch 2/10] Multiple vector domain support - cpu and domain management

In reply to Kenji Kaneshige

I didn't have time to look at these patches yet, but what I was
wondering: SGI Altix already does per-CPU interrupts. Do we really need another abstraction or could you use the same approach as SGI?

--david

Quoted 42 lines · Ashok Raj

Quoted 9 lines · Kenji Kaneshige
This patch add the code to handle the relationship between cpu and domains. We need more consideration about how to separate vector domains.
Signed-off-by Kenji Kaneshige <[email protected]>
---
Should we consider this based on some mach-* like schemes, so for e.g in NUMA case we could use node as a VECTOR_DOMAIN? ... or maybe we could somehow tie this into CPUSETS?..
Quoted 7 lines
arch/ia64/kernel/irq_ia64.c renamed from arch/ia64/kernel/irq_ia64.c~vector-domain-ia64-attach-cpu +7 -0
+
+#ifdef CONFIG_VECTOR_DOMAIN
+int ia64_cpu_domain_map[NR_CPUS];
+cpumask_t ia64_domain_cpumask[NR_VECTOR_DOMAINS];
+
+void __devinit
+ia64_attach_cpu_to_domain (int cpu)
arch/ia64/kernel/irq_ia64.c ia64_send_ipi (int cpu, int vector, int · line 312
Quoted 10 lines
arch/ia64/kernel/irq_ia64.c renamed from arch/ia64/kernel/irq_ia64.c~vector-domain-ia64-attach-cpu +10 -0
+{
+     static int domain = -1;
+
+     /* Simple Round Robin for now */
+     if (++domain >= NR_VECTOR_DOMAINS)
+             domain = 0;
+     ia64_cpu_domain_map[cpu] = domain;
+     cpu_set(cpu, ia64_domain_cpumask[domain]);
+}
+#endif /* CONFIG_VECTOR_DOMAIN */
--
Cheers,
Ashok Raj
- Open Source Technology Center
-
To unsubscribe from this list: send the line "unsubscribe linux-ia64" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Zwane Mwaikambo

2005-07-14 18:23 UTC #3 lore raw

In reply to Kenji Kaneshige

Quoted 6 lines · Ashok Raj

Quoted 5 lines · Kenji Kaneshige
This patch add the code to handle the relationship between cpu and domains. We need more consideration about how to separate vector domains.

Quoted 8 lines

Quoted 4 lines
Signed-off-by Kenji Kaneshige <[email protected]>
---
Should we consider this based on some mach-* like schemes, so for e.g in NUMA case we could use node as a VECTOR_DOMAIN? ... or maybe we could somehow tie this into CPUSETS?..

There is, on some hardware, a requirement that only specific processors may service specific devices due to interrupt controller dispatching, these are usually tied together as nodes and it'd be a lot easier determining interrupt domain participants if we didn't have to deal with arbitrary cpu masks but had a fixed group of processors and hence could figure out which node they all belonged to.

Zwane

Kenji Kaneshige

author

2005-07-16 01:41 UTC #4 lore raw

In reply to Kenji Kaneshige

Quoted 8 lines · Ashok Raj

Quoted 6 lines · Kenji Kaneshige
This patch add the code to handle the relationship between cpu and domains. We need more consideration about how to separate vector domains.
Signed-off-by Kenji Kaneshige <[email protected]>

Quoted 9 lines

Quoted 2 lines
---
Should we consider this based on some mach-* like schemes, so for e.g in NUMA case we could use node as a VECTOR_DOMAIN? ... or maybe we could somehow tie this into CPUSETS?..

I think some hardware implements hardware interrupt redirection on each node. For such system, I think it is good idea to use NUMA node as a VECTOR_DOMAIN to make it possible to use hardware interrupt redirection. Currently I don't have other reasons to use the set of processors as a vector domain instead of per-CPU approach.

Does anyone have other reasons to use other approach than per-CPU approach?

Thanks,
Kenji Kaneshige