Skip to contentSkip to conversation

irq: allow percpu_devid interrupts to be requested with target mask

RFC PATCH In review Will Deacon 2 participants 4 messages Nov 27, 2015 – Dec 7, 2015 IRQ

lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 irq: allow percpu_devid interrupts to be requested with target mask

On multi-cluster platforms, a per-cpu interrupt may actually be wired on a per-cluster basis, meaning that different devices may have the same interrupt number depending on the CPU.

This is problematic for drivers using the percpu_device interface as there is currently no way to request an interrupt of this type for a subset of CPUs. Furthermore, interrupt sharing is not permitted.

This patch adds the ability to provide a CPU mask to the percpu interrupt request/free functions such that the interrupt is only enabled on those CPUs set in the mask. Each irqaction has a copy of the mask to allow the percpu_devid flow handler to dispatch to the correct driver when an interrupt occurs. Whilst interrupt sharing is still forbidden in the usual sense, multiple actions are permitted providing that their target CPU masks do not intersect.

Cc Thomas Gleixner <[email protected]>
Cc Marc Zyngier <[email protected]>
---
 include/linux/interrupt.h | 37 +++++++++++++++++---
 kernel/irq/chip.c         | 56 ++++++++++++++++++++++++++----
 kernel/irq/internals.h    |  4 +--
 kernel/irq/manage.c       | 88 ++++++++++++++++++++++++++++++++---------------
 4 files changed, 144 insertions(+), 41 deletions(-)
include/linux/interrupt.h +33 -4
index ad16809c8596..0953a4ee19d1 100644
@@ -97,6 +97,7 @@ typedef irqreturn_t (*irq_handler_t)(int, void *);
  * @name:	name of the device
  * @dev_id:	cookie to identify the device
  * @percpu_dev_id:	cookie to identify the device
+ * @active_cpus: mask of CPUs for which this percpu interrupt can fire
  * @next:	pointer to the next irqaction for shared interrupts
  * @irq:	interrupt number
  * @flags:	flags (see IRQF_* above)
@@ -111,6 +112,7 @@ struct irqaction {
 	irq_handler_t		handler;
 	void			*dev_id;
 	void __percpu		*percpu_dev_id;
+	cpumask_var_t		active_cpus;
 	struct irqaction	*next;
 	irq_handler_t		thread_fn;
 	struct task_struct	*thread;
@@ -142,11 +144,26 @@ request_any_context_irq(unsigned int irq, irq_handler_t handler,
 			unsigned long flags, const char *name, void *dev_id);
 
 extern int __must_check
+request_percpu_irq_mask(unsigned int irq, irq_handler_t handler,
+			const char *devname, void __percpu *percpu_dev_id,
+			const cpumask_t *mask);
+
+static inline int __must_check
 request_percpu_irq(unsigned int irq, irq_handler_t handler,
-		   const char *devname, void __percpu *percpu_dev_id);
+		   const char *devname, void __percpu *percpu_dev_id)
+{
+	return request_percpu_irq_mask(irq, handler, devname, percpu_dev_id,
+				       cpu_possible_mask);
+}
 
 extern void free_irq(unsigned int, void *);
-extern void free_percpu_irq(unsigned int, void __percpu *);
+extern void free_percpu_irq_mask(unsigned int, void __percpu *,
+				 const cpumask_t *);
+
+static inline void free_percpu_irq(unsigned int irq, void __percpu *dev_id)
+{
+	free_percpu_irq_mask(irq, dev_id, cpu_possible_mask);
+}
 
 struct device;
 
@@ -192,9 +209,21 @@ extern void devm_free_irq(struct device *dev, unsigned int irq, void *dev_id);
 extern void disable_irq_nosync(unsigned int irq);
 extern bool disable_hardirq(unsigned int irq);
 extern void disable_irq(unsigned int irq);
-extern void disable_percpu_irq(unsigned int irq);
+extern void disable_percpu_irq_mask(unsigned int irq, const cpumask_t *mask);
 extern void enable_irq(unsigned int irq);
-extern void enable_percpu_irq(unsigned int irq, unsigned int type);
+extern void enable_percpu_irq_mask(unsigned int irq, unsigned int type,
+				   const cpumask_t *mask);
+
+static inline void disable_percpu_irq(unsigned int irq)
+{
+	disable_percpu_irq_mask(irq, cpumask_of(smp_processor_id()));
+}
+
+static inline void enable_percpu_irq(unsigned int irq, unsigned int type)
+{
+	enable_percpu_irq_mask(irq, type, cpumask_of(smp_processor_id()));
+}
+
 extern void irq_wake_thread(unsigned int irq, void *dev_id);
 
 /* The following three functions are for the core kernel use only. */
kernel/irq/chip.c +49 -7

86 lines not on this page · raw message

kernel/irq/internals.h +2 -2
index fcab63c66905..85d5d673bb84 100644
@@ -69,8 +69,8 @@ extern int irq_startup(struct irq_desc *desc, bool resend);
 extern void irq_shutdown(struct irq_desc *desc);
 extern void irq_enable(struct irq_desc *desc);
 extern void irq_disable(struct irq_desc *desc);
-extern void irq_percpu_enable(struct irq_desc *desc, unsigned int cpu);
-extern void irq_percpu_disable(struct irq_desc *desc, unsigned int cpu);
+extern void irq_percpu_enable(struct irq_desc *desc, const cpumask_t *mask);
+extern void irq_percpu_disable(struct irq_desc *desc, const cpumask_t *mask);
 extern void mask_irq(struct irq_desc *desc);
 extern void unmask_irq(struct irq_desc *desc);
 extern void unmask_threaded_irq(struct irq_desc *desc);
kernel/irq/manage.c +60 -28

216 lines not on this page · raw message

Signature · 1 line
2.1.4
#1 lore raw

Will,

Quoted 8 lines · Will Deacon

On multi-cluster platforms, a per-cpu interrupt may actually be wired on a per-cluster basis, meaning that different devices may have the same interrupt number depending on the CPU.

This is problematic for drivers using the percpu_device interface as there is currently no way to request an interrupt of this type for a subset of CPUs. Furthermore, interrupt sharing is not permitted.

Quoted 7 lines

This patch adds the ability to provide a CPU mask to the percpu interrupt request/free functions such that the interrupt is only enabled on those CPUs set in the mask. Each irqaction has a copy of the mask to allow the percpu_devid flow handler to dispatch to the correct driver when an interrupt occurs. Whilst interrupt sharing is still forbidden in the usual sense, multiple actions are permitted providing that their target CPU masks do not intersect.

I'm not too excited about this approach. That looks pretty much bolted on the existing percpu_irq machinery and you add the overhead of the action loop plus the mask check to each per cpu interrupt. Not really efficient.

I assume that

Quoted 2 lines

meaning that different devices may have the same interrupt number depending on the CPU.

means: They have the same HW interrupt number and therefor the same linux irq number due to the way percpu_irq works today.

So the scenario is:

|----------------------|   |--------------------|
|Cluster 0             |   |Cluster1            |
|----------------------|   |--------------------|
|PerCPU irqs 	       |   |PerCPU irqs         |
|----------------------|   |--------------------|
|PerClusterPerCPU irqs |   |PerCluster irqs     |
|----------------------|   |--------------------|
|Global shared irqs    |   |Global shared irqs  |
|----------------------|   |--------------------|

1) PerCPU irqs:

Same HWirq number on all CPUs. Interrupt is raised locally on the CPU from a CPU local facility (local timer, IPI receive, perf ...)

Covered by the existing percpu interface.

2) Shared irqs:

Unique HWirq number for a single device interrupt. Used by globally available devices. irq can be routed to any CPU.

3) PerCluster irqs:

Similar to PerCPU irqs, but the same HWirq number is only shared on a per cluster basis.

So we might end up with hwirqX on cluster 0 wired to the local timer while it is wired to perf on cluster 1 and vice versa.

Now you want to use the same Linux irq number for the local timer or whatever on all CPUs independent of the cluster, right?

So the per cpu timer interrupt is requested once as a per cpu interrupt with the linux irq number of cluster 0. But when you call enable_per_cpu_irq() on a cpu from cluster 1 you want to use the same linux irq number. But with the current scheme this ends up at the wrong hw irq.

That's why you introduced that cpumask stuff.

We can do better than that. Hierarchical irq domains to the rescue.

|------------|
| GIC domain |<--- Global PerCPU irqs
|	     |
|	     |<--- Global shared irqs
|	     |
|	     |<--- [ITS domain] <--- [MSI domain]
|	     |
|	     |<--- [PerCluster PerCPU domain]
|------------|

Now that PerCluster PerCPU domain does the following:

When the irq is allocated and mapped it requests N consective Linux irq numbers, where N is the number of clusters. The return value is the first irq number, i.e. the one which maps to cluster 0.

The allocater/mapping sets up irq + 0 for cluster 0, irq + 1 for cluster 1,.... Each irq has a different irq to hwirq mapping.

The domain also needs a cluster aware mapping mechanism. See further below.

All handling functions setup/request/free/remove_percpu_irq() now do:

    struct irq_desc *desc = irq_to_desc(irq);
    struct irqdomain *domain = irq_desc_get_domain(desc);
    int i;

    __setup_irq(desc, handler);

    if (!domain->is_clustered)
       return;

    for (i = 1; i < domain->nr_clusters: i++) {
    	desc = irq_to_desc(irq + i);
	desc->handler = handler;
    }

    This might also require some special handling in your domains
    activate() callback, which is invoked from __setup_irq() but that's
    a detail.

enable/disable_percpu_irq() does:

    struct irq_desc *desc = irq_to_desc(irq);
    struct irqdomain *domain = irq_desc_get_domain(desc);

    if (domain && domain->nr_clusters > 1)
	  desc = irq_to_desc(irq + domain->get_cluster(smp_processor_id());

    enable/disable(desc);

So lets look at an example

setup_local_timer()
irq = map_irq();
Allocates irq10 and irq11, returns irq 10 Maps irq10 to hwirq 20 and irq11 to hwirq 21
...

  request_percpu_irq(10, timer_handler)
  	irq10->handler = timer_handler;
  	irq11->handler = timer_handler;

enable_percpu_irq(10) <- On cluster 0
enable_irq(irq10) -> hwirq20

enable_percpu_irq(10) <- On cluster 1
enable_irq(irq11) -> hwirq21

setup_perf()
irq = map_irq();
Allocates irq12 and irq13, returns irq 12 Maps irq12 to hwirq 21 and irq13 to hwirq 20
...

  request_percpu_irq(12, perf_handler)
  	irq12->handler = perf_handler;
  	irq12->handler = perf_handler;

enable_percpu_irq(12) <- On cluster 0
enable_irq(irq12) -> hwirq21

enable_percpu_irq(12) <- On cluster 1
enable_irq(irq13) -> hwirq20

We can do that, because all operations have to be invoked on the target cpu.

So far so good. Now we have the interrupt entry code which wants to translate hwirqX to a linux irq number. So we need some modification to irq_find_mapping() as well. We could do a domain specific find_mapping() callback, or add something to the core code.

	if (domain->is_clustered_mapped) {
	      map = domain->map[domain->get_cluster(cpuid)];
	      return map[hwirq];
	}			  

works for a linear domain. A domain specific find_mapping() callback might be better suited, but thats a detail.

That might be nice counterpart to the work Qais is doing:

http://marc.info/?l=linux-kernel&m=144845325611158&w=2

The way MIPS sets up IPIs and per cpu irqs is not clustered as in your case. It needs one hwirq per CPU, but we still want to operate from a single base irq. It's a bit different, as it does not need that extra reverse mapping function because he operates on a globally unique hwirq number, but still needs to setup N irq descriptors/handlers per PERCPU irq as in your cluster scenario. So we might be able to share that part and just handle the IPI allocation/send mechanism in Qais new code.

Thoughts?

Thanks,

tglx

#2 lore raw
Quoted 1 line · Thomas Gleixner

Will,

Hi Thomas,

Thanks for the comprehensive review. Comments inline.

Quoted 18 lines · Will Deacon
Quoted 9 lines

On multi-cluster platforms, a per-cpu interrupt may actually be wired on a per-cluster basis, meaning that different devices may have the same interrupt number depending on the CPU.

This is problematic for drivers using the percpu_device interface as there is currently no way to request an interrupt of this type for a subset of CPUs. Furthermore, interrupt sharing is not permitted.

Quoted 7 lines

This patch adds the ability to provide a CPU mask to the percpu interrupt request/free functions such that the interrupt is only enabled on those CPUs set in the mask. Each irqaction has a copy of the mask to allow the percpu_devid flow handler to dispatch to the correct driver when an interrupt occurs. Whilst interrupt sharing is still forbidden in the usual sense, multiple actions are permitted providing that their target CPU masks do not intersect.

Quoted 4 lines

I'm not too excited about this approach. That looks pretty much bolted on the existing percpu_irq machinery and you add the overhead of the action loop plus the mask check to each per cpu interrupt. Not really efficient.

Sure -- I largely just lifted the IRQF_SHARED code to get discussion started.

Quoted 7 lines

I assume that

Quoted 2 lines

meaning that different devices may have the same interrupt number depending on the CPU.

means: They have the same HW interrupt number and therefor the same linux irq number due to the way percpu_irq works today.

Right.

Quoted 35 lines

So the scenario is:

|----------------------|   |--------------------|
|Cluster 0             |   |Cluster1            |
|----------------------|   |--------------------|
|PerCPU irqs 	       |   |PerCPU irqs         |
|----------------------|   |--------------------|
|PerClusterPerCPU irqs |   |PerCluster irqs     |
|----------------------|   |--------------------|
|Global shared irqs    |   |Global shared irqs  |
|----------------------|   |--------------------|

1) PerCPU irqs:

Same HWirq number on all CPUs. Interrupt is raised locally on the CPU from a CPU local facility (local timer, IPI receive, perf ...)

Covered by the existing percpu interface.

2) Shared irqs:

Unique HWirq number for a single device interrupt. Used by globally available devices. irq can be routed to any CPU.

3) PerCluster irqs:

Similar to PerCPU irqs, but the same HWirq number is only shared on a per cluster basis.

So we might end up with hwirqX on cluster 0 wired to the local timer while it is wired to perf on cluster 1 and vice versa.

Now you want to use the same Linux irq number for the local timer or whatever on all CPUs independent of the cluster, right?

Quoted 7 lines

So the per cpu timer interrupt is requested once as a per cpu interrupt with the linux irq number of cluster 0. But when you call enable_per_cpu_irq() on a cpu from cluster 1 you want to use the same linux irq number. But with the current scheme this ends up at the wrong hw irq.

That's why you introduced that cpumask stuff.

There's a slight niggle here, which I didn't communicate in the commit log (because my use-case is indeed per-cluster). Interrupts can also be wired on a per-cpu basis, but where you have multiple device/driver instances and therefore can't use a single call to request_percpu_irq. An example of this is Freescale's watchdog:

http://www.spinics.net/lists/arm-kernel/msg464000.html

which is why I added Bhupesh to Cc. They seem to have 1 watchdog per core, but signalling the same PPI number...

Quoted 1 line

We can do better than that. Hierarchical irq domains to the rescue.

... so would we want a per-cpu domain instead?

Quoted 81 lines
|------------|
| GIC domain |<--- Global PerCPU irqs
|	     |
|	     |<--- Global shared irqs
|	     |
|	     |<--- [ITS domain] <--- [MSI domain]
|	     |
|	     |<--- [PerCluster PerCPU domain]
|------------|

Now that PerCluster PerCPU domain does the following:

When the irq is allocated and mapped it requests N consective Linux irq numbers, where N is the number of clusters. The return value is the first irq number, i.e. the one which maps to cluster 0.

The allocater/mapping sets up irq + 0 for cluster 0, irq + 1 for cluster 1,.... Each irq has a different irq to hwirq mapping.

The domain also needs a cluster aware mapping mechanism. See further below.

All handling functions setup/request/free/remove_percpu_irq() now do:

    struct irq_desc *desc = irq_to_desc(irq);
    struct irqdomain *domain = irq_desc_get_domain(desc);
    int i;

    __setup_irq(desc, handler);

    if (!domain->is_clustered)
       return;

    for (i = 1; i < domain->nr_clusters: i++) {
    	desc = irq_to_desc(irq + i);
	desc->handler = handler;
    }

    This might also require some special handling in your domains
    activate() callback, which is invoked from __setup_irq() but that's
    a detail.

enable/disable_percpu_irq() does:

    struct irq_desc *desc = irq_to_desc(irq);
    struct irqdomain *domain = irq_desc_get_domain(desc);

    if (domain && domain->nr_clusters > 1)
	  desc = irq_to_desc(irq + domain->get_cluster(smp_processor_id());

    enable/disable(desc);

So lets look at an example

setup_local_timer()
irq = map_irq();
Allocates irq10 and irq11, returns irq 10 Maps irq10 to hwirq 20 and irq11 to hwirq 21
...

  request_percpu_irq(10, timer_handler)
  	irq10->handler = timer_handler;
  	irq11->handler = timer_handler;

enable_percpu_irq(10) <- On cluster 0
enable_irq(irq10) -> hwirq20

enable_percpu_irq(10) <- On cluster 1
enable_irq(irq11) -> hwirq21

setup_perf()
irq = map_irq();
Allocates irq12 and irq13, returns irq 12 Maps irq12 to hwirq 21 and irq13 to hwirq 20
...

  request_percpu_irq(12, perf_handler)
  	irq12->handler = perf_handler;
  	irq12->handler = perf_handler;
Quoted 8 lines

enable_percpu_irq(12) <- On cluster 0
enable_irq(irq12) -> hwirq21

enable_percpu_irq(12) <- On cluster 1
enable_irq(irq13) -> hwirq20

We can do that, because all operations have to be invoked on the target cpu.

Ok, but what about existing callers? I think we either need a new function or flag to indicate that the irq is per-cluster, otherwise we run the risk of breaking things where the hwirq is actually the same.

At the moment, the ARM perf binding in device-tree has an optional "interrupt-affinity" field that describes the set of CPUs which use the PPI number described elsewhere in the node. Perhaps we could parse this when we create the new domain and selectively map irqs there.

Dunno -- needs prototyping!

Quoted 5 lines

So far so good. Now we have the interrupt entry code which wants to translate hwirqX to a linux irq number. So we need some modification to irq_find_mapping() as well. We could do a domain specific find_mapping() callback, or add something to the core code.

Quoted 7 lines
	if (domain->is_clustered_mapped) {
	      map = domain->map[domain->get_cluster(cpuid)];
	      return map[hwirq];
	}			  

works for a linear domain. A domain specific find_mapping() callback might be better suited, but thats a detail.

It's a little scary to tie the device-tree topology description (which is usually used for scheduling and power management afaiu) directly to the IRQ routing, but then if this does end up being per-cpu, there's no real issue.

Quoted 4 lines

That might be nice counterpart to the work Qais is doing:

   http://marc.info/?l=linux-kernel&m=144845325611158&w=2
Quoted 8 lines

The way MIPS sets up IPIs and per cpu irqs is not clustered as in your case. It needs one hwirq per CPU, but we still want to operate from a single base irq. It's a bit different, as it does not need that extra reverse mapping function because he operates on a globally unique hwirq number, but still needs to setup N irq descriptors/handlers per PERCPU irq as in your cluster scenario. So we might be able to share that part and just handle the IPI allocation/send mechanism in Qais new code.

Thanks, I'll take a look.

Will

#3 lore raw

Will,

Quoted 18 lines · Will Deacon
Quoted 10 lines · Thomas Gleixner

Now you want to use the same Linux irq number for the local timer or whatever on all CPUs independent of the cluster, right?

So the per cpu timer interrupt is requested once as a per cpu interrupt with the linux irq number of cluster 0. But when you call enable_per_cpu_irq() on a cpu from cluster 1 you want to use the same linux irq number. But with the current scheme this ends up at the wrong hw irq.

That's why you introduced that cpumask stuff.

There's a slight niggle here, which I didn't communicate in the commit log (because my use-case is indeed per-cluster). Interrupts can also be wired on a per-cpu basis, but where you have multiple device/driver instances and therefore can't use a single call to request_percpu_irq. An example of this is Freescale's watchdog:

Quoted 4 lines

http://www.spinics.net/lists/arm-kernel/msg464000.html

which is why I added Bhupesh to Cc. They seem to have 1 watchdog per core, but signalling the same PPI number...

You can create both a PerCluster and a PerCPU domain.

But in that particular case it's the same as the local timer which signals the same PPI on all CPUs, right? So the existing percpu irq stuff should just work for that watchdog thingy.

Quoted 9 lines
Quoted 5 lines

enable_percpu_irq(12) <- On cluster 1
enable_irq(irq13) -> hwirq20

We can do that, because all operations have to be invoked on the target cpu.

Ok, but what about existing callers? I think we either need a new function or flag to indicate that the irq is per-cluster, otherwise we run the risk of breaking things where the hwirq is actually the same.

That's what I wrote a few lines up:

Quoted 11 lines
Quoted 9 lines

enable/disable_percpu_irq() does:

    struct irq_desc *desc = irq_to_desc(irq);
    struct irqdomain *domain = irq_desc_get_domain(desc);

    if (domain && domain->nr_clusters > 1)
        desc = irq_to_desc(irq + domain->get_cluster(smp_processor_id());

    enable/disable(desc);
Quoted 6 lines

At the moment, the ARM perf binding in device-tree has an optional "interrupt-affinity" field that describes the set of CPUs which use the PPI number described elsewhere in the node. Perhaps we could parse this when we create the new domain and selectively map irqs there.

Dunno -- needs prototyping!

My device tree foo is close to zero, so I let the DT wizards come up with a solution. But you need some decription of this in the DT anyway, right?

Quoted 10 lines
Quoted 10 lines

So far so good. Now we have the interrupt entry code which wants to translate hwirqX to a linux irq number. So we need some modification to irq_find_mapping() as well. We could do a domain specific find_mapping() callback, or add something to the core code.

	if (domain->is_clustered_mapped) {
	      map = domain->map[domain->get_cluster(cpuid)];
	      return map[hwirq];
	}			  
Quoted 7 lines
Quoted 2 lines

works for a linear domain. A domain specific find_mapping() callback might be better suited, but thats a detail.

It's a little scary to tie the device-tree topology description (which is usually used for scheduling and power management afaiu) directly to the IRQ routing, but then if this does end up being per-cpu, there's no real issue.

Well, if your hardware is wired in scary ways, i.e. PPIs have a different meaning on different clusters/cpus, then you need some description in DT anyway. Translating that into a mapping is pretty much straight forward I guess.

Thanks,

tglx