genirq/irqdomain: Add reference counting to IRQs
Subsystems (11)
v4 cover genirq/irqdomain: Add reference counting to IRQs
This is another attempt to add reference counting to IRQ descriptors; or - more to the point - reuse already existing kobj from irq_desc. This allows the same IRQ to be used several times (such as legacy PCI INTx) and when disposing those, only the last reference drop clears the hardware mappings. Domains do not add references to irq_desc as the whole point of this exercise is to move actual cleanup in hardware to the last reference drop. This only changes sparse interrupts (no idea about the other case yet).
No changelog as it is all completely rewritten. I am still running tests but I hope this demonstrates the idea.
Some context from Cedric:
The background context for such a need is that the POWER9 and POWER10 processors have a new XIVE interrupt controller which uses MMIO pages for interrupt management. Each interrupt has a pair of pages which are required to be unmapped in some environment, like PHB removal. And so, all interrupts need to be unmmaped.
1/8 .. 3/8 are removing confusing "realloc" which not strictly required but I was touching this anyway and legacy interrupts should probably use the new counting anyway;
4/8 .. 6/8 is reordering irq_desc disposal;
7/8 adds extra references (probably missed other places);
8/8 is the fix for the original XIVE bug; it is here for demonstration.
I am cc'ing ppc list so people can pull the patches from that patchworks.
This is based on sha1418baf2c28f3 Linus Torvalds "Linux 5.10-rc5".
and pushed out to
https://github.com/aik/linux/commits/irqs
sha1 3955f97c448242f6a
Please comment. Thanks.
Alexey Kardashevskiy (7):
genirq/ipi: Simplify irq_reserve_ipi
genirq/irqdomain: Clean legacy IRQ allocation genirq/irqdomain: Drop unused realloc parameter from
__irq_domain_alloc_irqs
genirq: Free IRQ descriptor via embedded kobject
genirq: Add free_irq hook for IRQ descriptor and use for mapping
disposal
genirq/irqdomain: Move hierarchical IRQ cleanup to kobject_release genirq/irqdomain: Reference irq_desc for already mapped irqs
Oliver O'Halloran (1):
powerpc/pci: Remove LSI mappings on device teardown
include/linux/irqdesc.h | 1 + include/linux/irqdomain.h | 9 +- include/linux/irqhandler.h | 1 + arch/powerpc/kernel/pci-common.c | 21 ++++ arch/x86/kernel/apic/io_apic.c | 13 ++- drivers/gpio/gpiolib.c | 1 - drivers/irqchip/irq-armada-370-xp.c | 2 +- drivers/irqchip/irq-bcm2836.c | 3 +- drivers/irqchip/irq-gic-v3.c | 3 +- drivers/irqchip/irq-gic-v4.c | 6 +- drivers/irqchip/irq-gic.c | 3 +- drivers/irqchip/irq-ixp4xx.c | 1 - kernel/irq/ipi.c | 16 +-- kernel/irq/irqdesc.c | 45 +++----- kernel/irq/irqdomain.c | 160 +++++++++++++++++----------- kernel/irq/msi.c | 2 +- 16 files changed, 158 insertions(+), 129 deletions(-)
Signature · 1 line
2.17.1
PATCH v4 1/8 genirq/ipi: Simplify irq_reserve_ipi
__irq_domain_alloc_irqs() can already handle virq==-1 and free descriptors if it failed allocating hardware interrupts so let's skip this extra step.
--- kernel/irq/ipi.c | 16 +++------------- 1 file changed, 3 insertions(+), 13 deletions(-)
kernel/irq/ipi.c +3 -13
@@ -75,18 +75,12 @@ int irq_reserve_ipi(struct irq_domain *domain, } } - virq =irq_domain_alloc_descs (-1 , nr_irqs,0 ,NUMA_NO_NODE ,NULL ); -if (virq <=0 ) { -pr_warn ("Can't reserve IPI, failed to alloc descs\n" ); -return -ENOMEM ; - } - - virq =__irq_domain_alloc_irqs (domain, virq, nr_irqs,NUMA_NO_NODE , - (void *) dest, true,NULL ); + virq =__irq_domain_alloc_irqs (domain,-1 , nr_irqs,NUMA_NO_NODE , + (void *) dest, false,NULL );if (virq <=0 ) {pr_warn ("Can't reserve IPI, failed to alloc hw irqs\n" ); - goto free_descs; +return -EBUSY ; }for (i =0 ; i < nr_irqs; i++) {
@@ -96,10 +90,6 @@ int irq_reserve_ipi(struct irq_domain *domain,irq_set_status_flags (virq + i,IRQ_NO_BALANCING ); }return virq; - -free_descs: -irq_free_descs (virq, nr_irqs); -return -EBUSY ; }/**
Signature · 1 line
2.17.1
Re: [PATCH kernel v4 1/8] genirq/ipi: Simplify irq_reserve_ipi
Quoted 5 lines · Alexey Kardashevskiy
__irq_domain_alloc_irqs()can already handle virq==-1 and free descriptors if it failed allocating hardware interrupts so let's skip this extra step.Signed-off-by Alexey Kardashevskiy <[email protected]>
LGTM,
Copying the MIPS folks since the IPI interface is only used under arch/mips.
C.
Quoted 32 lines
--- kernel/irq/ipi.c | 16 +++------------- 1 file changed, 3 insertions(+), 13 deletions(-)kernel/irq/ipi.c +3
-9@@ -75,18 +75,12 @@ int irq_reserve_ipi(struct irq_domain *domain, } } - virq =irq_domain_alloc_descs (-1 , nr_irqs,0 ,NUMA_NO_NODE ,NULL ); -if (virq <=0 ) { -pr_warn ("Can't reserve IPI, failed to alloc descs\n" ); -return -ENOMEM ; - } - - virq =__irq_domain_alloc_irqs (domain, virq, nr_irqs,NUMA_NO_NODE , - (void *) dest, true,NULL ); + virq =__irq_domain_alloc_irqs (domain,-1 , nr_irqs,NUMA_NO_NODE , + (void *) dest, false,NULL );if (virq <=0 ) {pr_warn ("Can't reserve IPI, failed to alloc hw irqs\n" ); - goto free_descs; +return -EBUSY ; }for (i =0 ; i < nr_irqs; i++) {
kernel/irq/ipi.c int irq_reserve_ipi(struct irq_domain *domain, · line 95
Quoted 10 lines
kernel/irq/ipi.c +0
-4@@ -96,10 +90,6 @@ int irq_reserve_ipi(struct irq_domain *domain,irq_set_status_flags (virq + i,IRQ_NO_BALANCING ); }return virq; - -free_descs: -irq_free_descs (virq, nr_irqs); -return -EBUSY ; }/**
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v4 2/8 genirq/irqdomain: Clean legacy IRQ allocation
In reply to Alexey KardashevskiyThere are 10 users of __irq_domain_alloc_irqs() and only one - IOAPIC - passes realloc==true. There is no obvious reason for handling this specific case in the generic code.
This splits out __irq_domain_alloc_irqs_data() to make it clear what IOAPIC does and makes __irq_domain_alloc_irqs() cleaner.
This should cause no behavioral change.
--- include/linux/irqdomain.h | 3 ++ arch/x86/kernel/apic/io_apic.c | 13 +++-- kernel/irq/irqdomain.c | 89 ++++++++++++++++++++-------------- 3 files changed, 65 insertions(+), 40 deletions(-)
include/linux/irqdomain.h +3 -0
@@ -470,6 +470,9 @@ static inline struct irq_domain *irq_domain_add_hierarchy(struct irq_domain *par ops, host_data); } +extern int __irq_domain_alloc_irqs_data (struct irq_domain *domain,int virq, +unsigned int nr_irqs,int node,void *arg, +const struct irq_affinity_desc *affinity); extern int __irq_domain_alloc_irqs(struct irq_domain *domain, int irq_base, unsigned int nr_irqs, int node, void *arg, bool realloc,
arch/x86/kernel/apic/io_apic.c +9 -4
@@ -970,9 +970,14 @@ static int alloc_irq_from_domain(struct irq_domain *domain, int ioapic, u32 gsi,return -1 ; } -return __irq_domain_alloc_irqs (domain, irq,1 , -ioapic_alloc_attr_node (info), - info, legacy,NULL ); +if (irq ==-1 || !legacy) +return __irq_domain_alloc_irqs (domain, irq,1 , +ioapic_alloc_attr_node (info), + info, false,NULL ); + +return __irq_domain_alloc_irqs_data (domain, irq,1 , +ioapic_alloc_attr_node (info), + info,NULL ); }/*
@@ -1006,7 +1011,7 @@ static int alloc_isa_irq_from_domain(struct irq_domain *domain,return -ENOMEM ; }else { info->flags |=X86_IRQ_ALLOC_LEGACY ; - irq =__irq_domain_alloc_irqs (domain, irq,1 , node, info, true, + irq =__irq_domain_alloc_irqs_data (domain, irq,1 , node, info,NULL );if (irq >=0 ) { irq_data =irq_domain_get_irq_data (domain, irq);
kernel/irq/irqdomain.c +53 -36
113 lines not on this page · raw message
Signature · 1 line
2.17.1
Re: [PATCH kernel v4 2/8] genirq/irqdomain: Clean legacy IRQ allocation
kernel/irq/irqdomain.c int __irq_domain_alloc_irqs(struct irq_domain *domain, int irq_base, · line 1477
Quoted 13 lines · Alexey Kardashevskiy
There are 10 users of
__irq_domain_alloc_irqs()and only one - IOAPIC - passes realloc==true. There is no obvious reason for handling this specific case in the generic code.This splits out
__irq_domain_alloc_irqs_data()to make it clear what IOAPIC does and makes__irq_domain_alloc_irqs()cleaner.This should cause no behavioral change.
kernel/irq/irqdomain.c +2
-0+ ret =__irq_domain_alloc_irqs_data (domain, virq, nr_irqs, node, arg, affinity); +if (ret <=0 ) goto out_free_desc;
Was or wasn't 0 considered as error code previously?
Quoted 5 lines
return virq;
out_free_desc:irq_free_descs(virq, nr_irqs); return ret;
Signature · 2 lines
With Best Regards, Andy Shevchenko
Re: [PATCH kernel v4 2/8] genirq/irqdomain: Clean legacy IRQ allocation
Quoted 9 lines · Andy Shevchenko
Quoted 8 lines · Alexey Kardashevskiy
There are 10 users of
__irq_domain_alloc_irqs()and only one - IOAPIC - passes realloc==true. There is no obvious reason for handling this specific case in the generic code.This splits out
__irq_domain_alloc_irqs_data()to make it clear what IOAPIC does and makes__irq_domain_alloc_irqs()cleaner.
Quoted 7 lines
kernel/irq/irqdomain.c int __irq_domain_alloc_irqs(struct irq_domain *domain, int irq_base, · line 1477
Quoted 5 lines
This should cause no behavioral change.
kernel/irq/irqdomain.c +2
-0+ ret =__irq_domain_alloc_irqs_data (domain, virq, nr_irqs, node, arg, affinity); +if (ret <=0 ) goto out_free_desc;Was or wasn't 0 considered as error code previously?
Oh. I need to clean this up, the idea is since this does not allocate IRQs, this should return error code and not an irq, I'll make this explicit.
Quoted 7 lines
Quoted 6 lines
return virq;out_free_desc: irq_free_descs(virq, nr_irqs); return ret;
Signature · 1 line
Alexey
Re: [PATCH kernel v4 2/8] genirq/irqdomain: Clean legacy IRQ allocation
In reply to Alexey KardashevskiyAlexey,
Quoted 3 lines · Alexey Kardashevskiy
There are 10 users of
__irq_domain_alloc_irqs()and only one - IOAPIC - passes realloc==true. There is no obvious reason for handling this specific case in the generic code.
There is also no obvious reason for _NOT_ handling it at the core code.
Quoted 2 lines
This splits out
__irq_domain_alloc_irqs_data()to make it clear what IOAPIC does and makes__irq_domain_alloc_irqs()cleaner.
That's your interpretation of cleaner.
You need to expose __irq_domain_alloc_irqs_data() for that which is a core only functionality, so it's not cleaner. It's exposing internals which are not to be exposed.
The right thing to do is to get rid of the legacy allocation of x86 during early_irq_init() which is possible with the recent restructuring of the interrupt initialization code in x86. That's a cleanup which will actually remove code and not expose internals just because.
Quoted 1 line
This should cause no behavioral change.
Should not cause is a pretty weak statement.
You're missing a nasty detail here. Contrary to the normal irqdomain rules virq 0 _IS_ valid on x86 for historical reasons and that's not trivial to change.
Thanks,
tglx
PATCH v4 3/8 genirq/irqdomain: Drop unused realloc parameter from __irq_domain_alloc_irqs
In reply to Alexey KardashevskiyThe two previous patches made @realloc obsolete. This finishes removing it.
--- include/linux/irqdomain.h | 4 +--- arch/x86/kernel/apic/io_apic.c | 2 +- drivers/gpio/gpiolib.c | 1 - drivers/irqchip/irq-armada-370-xp.c | 2 +- drivers/irqchip/irq-bcm2836.c | 3 +-- drivers/irqchip/irq-gic-v3.c | 3 +-- drivers/irqchip/irq-gic-v4.c | 6 ++---- drivers/irqchip/irq-gic.c | 3 +-- drivers/irqchip/irq-ixp4xx.c | 1 - kernel/irq/ipi.c | 2 +- kernel/irq/irqdomain.c | 4 +--- kernel/irq/msi.c | 2 +- 12 files changed, 11 insertions(+), 22 deletions(-)
include/linux/irqdomain.h +1 -3
@@ -475,7 +475,6 @@ extern int __irq_domain_alloc_irqs_data(struct irq_domain *domain, int virq, const struct irq_affinity_desc *affinity);extern int __irq_domain_alloc_irqs (struct irq_domain *domain,int irq_base,unsigned int nr_irqs,int node,void *arg, -bool realloc,const struct irq_affinity_desc *affinity);extern void irq_domain_free_irqs (unsigned int virq,unsigned int nr_irqs);extern int irq_domain_activate_irq (struct irq_data *irq_data,bool early);
@@ -484,8 +483,7 @@ extern void irq_domain_deactivate_irq(struct irq_data *irq_data);static inline int irq_domain_alloc_irqs (struct irq_domain *domain,unsigned int nr_irqs,int node,void *arg) { -return __irq_domain_alloc_irqs (domain,-1 , nr_irqs, node, arg, false, -NULL ); +return __irq_domain_alloc_irqs (domain,-1 , nr_irqs, node, arg,NULL ); } extern int irq_domain_alloc_irqs_hierarchy(struct irq_domain *domain,
arch/x86/kernel/apic/io_apic.c +1 -1
@@ -973,7 +973,7 @@ static int alloc_irq_from_domain(struct irq_domain *domain, int ioapic, u32 gsi,if (irq ==-1 || !legacy)return __irq_domain_alloc_irqs (domain, irq,1 ,ioapic_alloc_attr_node (info), - info, false,NULL ); + info,NULL ); return __irq_domain_alloc_irqs_data(domain, irq, 1, ioapic_alloc_attr_node(info),
drivers/gpio/gpiolib.c +0 -1
@@ -1059,7 +1059,6 @@ static void gpiochip_set_hierarchical_irqchip(struct gpio_chip *gc,1 ,NUMA_NO_NODE , &fwspec, - false, NULL);if (ret <0 ) { chip_err(gc,
drivers/irqchip/irq-armada-370-xp.c +1 -1
@@ -431,7 +431,7 @@ static __init void armada_xp_ipi_init(struct device_node *node)irq_domain_update_bus_token (ipi_domain,DOMAIN_BUS_IPI ); base_ipi =__irq_domain_alloc_irqs (ipi_domain,-1 ,IPI_DOORBELL_END , -NUMA_NO_NODE ,NULL , false,NULL ); +NUMA_NO_NODE ,NULL ,NULL );if (WARN_ON (!base_ipi))return ;
drivers/irqchip/irq-bcm2836.c +1 -2
@@ -269,8 +269,7 @@ static void __init bcm2836_arm_irqchip_smp_init(void)irq_domain_update_bus_token (ipi_domain,DOMAIN_BUS_IPI ); base_ipi =__irq_domain_alloc_irqs (ipi_domain,-1 ,BITS_PER_MBOX , -NUMA_NO_NODE ,NULL , - false,NULL ); +NUMA_NO_NODE ,NULL ,NULL );if (WARN_ON (!base_ipi))return ;
drivers/irqchip/irq-gic-v3.c +1 -2
@@ -1163,8 +1163,7 @@ static void __init gic_smp_init(void)/* Register all 8 non-secure SGIs */ base_sgi =__irq_domain_alloc_irqs (gic_data.domain,-1 ,8 , -NUMA_NO_NODE , &sgi_fwspec, - false,NULL ); +NUMA_NO_NODE , &sgi_fwspec,NULL );if (WARN_ON (base_sgi <=0 ))return ;
drivers/irqchip/irq-gic-v4.c +2 -4
@@ -117,8 +117,7 @@ static int its_alloc_vcpu_sgis(struct its_vpe *vpe, int idx) goto err; sgi_base =__irq_domain_alloc_irqs (vpe->sgi_domain,-1 ,16 , -NUMA_NO_NODE , vpe, - false,NULL ); +NUMA_NO_NODE , vpe,NULL );if (sgi_base <=0 ) goto err;
@@ -154,8 +153,7 @@ int its_alloc_vcpu_irqs(struct its_vm *vm) } vpe_base_irq =__irq_domain_alloc_irqs (vm->domain,-1 , vm->nr_vpes, -NUMA_NO_NODE , vm, - false,NULL ); +NUMA_NO_NODE , vm,NULL );if (vpe_base_irq <=0 ) goto err;
drivers/irqchip/irq-gic.c +1 -2
@@ -846,8 +846,7 @@ static __init void gic_smp_init(void) gic_starting_cpu, NULL); base_sgi =__irq_domain_alloc_irqs (gic_data[0 ].domain,-1 ,8 , -NUMA_NO_NODE , &sgi_fwspec, - false,NULL ); +NUMA_NO_NODE , &sgi_fwspec,NULL );if (WARN_ON (base_sgi <=0 ))return ;
drivers/irqchip/irq-ixp4xx.c +0 -1
@@ -353,7 +353,6 @@ void __init ixp4xx_irq_init(resource_size_t irqbase, chunk->nr_irqs,NUMA_NO_NODE , &fwspec, - false, NULL);if (ret <0 ) { pr_crit("IXP4XX: can not allocate irqs in hierarchy %d\n",
kernel/irq/ipi.c +1 -1
@@ -76,7 +76,7 @@ int irq_reserve_ipi(struct irq_domain *domain, } virq =__irq_domain_alloc_irqs (domain,-1 , nr_irqs,NUMA_NO_NODE , - (void *) dest, false,NULL ); + (void *) dest,NULL );if (virq <=0 ) {pr_warn ("Can't reserve IPI, failed to alloc hw irqs\n" );
kernel/irq/irqdomain.c +1 -3
@@ -1438,12 +1438,10 @@ int __irq_domain_alloc_irqs_data(struct irq_domain *domain, int virq,* @nr_irqs: number of IRQs to allocate * @node: NUMA node id for memory allocation * @arg: domain specific argument -* @realloc: IRQ descriptors have already been allocated if true * @affinity: Optional irq affinity mask for multiqueue devices * * Allocate IRQ numbers and initialized all data structures to support * hierarchy IRQ domains. -* Parameter @realloc is mainly to support legacy IRQs. * Returns error code or allocated IRQ number * * The whole process to setup an IRQ has been split into two steps.
@@ -1455,7 +1453,7 @@ int __irq_domain_alloc_irqs_data(struct irq_domain *domain, int virq,*/ int __irq_domain_alloc_irqs (struct irq_domain *domain,int irq_base,unsigned int nr_irqs,int node,void *arg, -bool realloc,const struct irq_affinity_desc *affinity) +const struct irq_affinity_desc *affinity) {int ret, virq;
kernel/irq/msi.c +1 -1
@@ -414,7 +414,7 @@ int __msi_domain_alloc_irqs(struct irq_domain *domain, struct device *dev, ops->set_desc (&arg, desc); virq =__irq_domain_alloc_irqs (domain,-1 , desc->nvec_used, -dev_to_node (dev), &arg, false, +dev_to_node (dev), &arg, desc->affinity);if (virq <0 ) { ret = -ENOSPC ;
Signature · 1 line
2.17.1
PATCH v4 4/8 genirq: Free IRQ descriptor via embedded kobject
In reply to Alexey KardashevskiyAt the moment the IRQ descriptor is freed via the free_desc() helper. We want to add reference counting to IRQ descriptors and there is already kobj embedded into irq_desc which we want to reuse.
This shuffles free_desc()/etc to make it simply call kobject_put() and moves all the cleanup into the kobject_release hook.
As a bonus, we do not need irq_sysfs_del() as kobj removes itself from sysfs if it knows that it was added.
This should cause no behavioral change.
--- kernel/irq/irqdesc.c | 42 ++++++++++++------------------------------ 1 file changed, 12 insertions(+), 30 deletions(-)
kernel/irq/irqdesc.c +12 -30
@@ -295,18 +295,6 @@ static void irq_sysfs_add(int irq, struct irq_desc *desc) } } -static void irq_sysfs_del (struct irq_desc *desc) -{ -/* -* If irq_sysfs_init() has not yet been invoked (early boot), then -* irq_kobj_base is NULL and the descriptor was never added. -* kobject_del() complains about a object with no parent, so make -* it conditional. -*/ -if (irq_kobj_base) -kobject_del (&desc->kobj); -} -static int __init irq_sysfs_init (void ) {struct irq_desc *desc;
@@ -337,7 +325,6 @@ static struct kobj_type irq_kobj_type = { };static void irq_sysfs_add (int irq,struct irq_desc *desc) {} -static void irq_sysfs_del (struct irq_desc *desc) {}#endif /* CONFIG_SYSFS */
@@ -419,39 +406,34 @@ static struct irq_desc *alloc_desc(int irq, int node, unsigned int flags,return NULL ; } -static void irq_kobj_release (struct kobject *kobj) -{ - struct irq_desc *desc = container_of(kobj, struct irq_desc, kobj); - -free_masks (desc); -free_percpu (desc->kstat_irqs); -kfree (desc); -} -static void delayed_free_desc (struct rcu_head *rhp) { struct irq_desc *desc = container_of(rhp, struct irq_desc, rcu); +free_masks (desc); +free_percpu (desc->kstat_irqs); +kfree (desc); +} + +static void free_desc (unsigned int irq) +{ +struct irq_desc *desc =irq_to_desc (irq); +kobject_put (&desc->kobj); } -static void free_desc (unsigned int irq) +static void irq_kobj_release (struct kobject *kobj) { -struct irq_desc *desc =irq_to_desc (irq); + struct irq_desc *desc = container_of(kobj, struct irq_desc, kobj); +unsigned int irq = desc->irq_data.irq;irq_remove_debugfs_entry (desc);unregister_irq_proc (irq, desc);/* -* sparse_irq_lock protects also show_interrupts() and -* kstat_irq_usr(). Once we deleted the descriptor from the -* sparse tree we can free it. Access in proc will fail to -* lookup the descriptor. -* * The sysfs entry must be serialized against a concurrent * irq_sysfs_init() as well. */ -irq_sysfs_del (desc);delete_irq_desc (irq);/*
Signature · 1 line
2.17.1
PATCH v4 5/8 genirq: Add free_irq hook for IRQ descriptor and use for mapping disposal
In reply to Alexey KardashevskiyWe want to make the irq_desc.kobj's release hook free associated resources but we do not want to pollute the irqdesc code with domains.
This adds a free_irq hook which is called when the last reference to the descriptor is dropped.
The first user is mapped irqs. This potentially can break the existing users; however they seem to do the right thing and call dispose once per mapping.
--- include/linux/irqdesc.h | 1 + include/linux/irqdomain.h | 2 -- include/linux/irqhandler.h | 1 + kernel/irq/irqdesc.c | 3 +++ kernel/irq/irqdomain.c | 14 ++++++++++++-- 5 files changed, 17 insertions(+), 4 deletions(-)
include/linux/irqdesc.h +1 -0
@@ -57,6 +57,7 @@ struct irq_desc {struct irq_data irq_data;unsigned int __percpu *kstat_irqs;irq_flow_handler_t handle_irq; +irq_free_handler_t free_irq;struct irqaction *action;/* IRQ action list */ unsigned int status_use_accessors;unsigned int core_internal_state__do_not_mess_with_it;
include/linux/irqdomain.h +0 -2
@@ -381,8 +381,6 @@ extern int irq_domain_associate(struct irq_domain *domain, unsigned int irq,extern void irq_domain_associate_many (struct irq_domain *domain,unsigned int irq_base,irq_hw_number_t hwirq_base,int count); -extern void irq_domain_disassociate (struct irq_domain *domain, -unsigned int irq);extern unsigned int irq_create_mapping (struct irq_domain *host,irq_hw_number_t hwirq);
include/linux/irqhandler.h +1 -0
@@ -10,5 +10,6 @@struct irq_desc ;struct irq_data ;typedef void (*irq_flow_handler_t )(struct irq_desc *desc); +typedef void (*irq_free_handler_t )(struct irq_desc *desc);#endif
kernel/irq/irqdesc.c +3 -0
@@ -427,6 +427,9 @@ static void irq_kobj_release(struct kobject *kobj) struct irq_desc *desc = container_of(kobj, struct irq_desc, kobj);unsigned int irq = desc->irq_data.irq; +if (desc->free_irq) + desc->free_irq (desc); +irq_remove_debugfs_entry (desc);unregister_irq_proc (irq, desc);
kernel/irq/irqdomain.c +12 -2
@@ -485,7 +485,7 @@ static void irq_domain_set_mapping(struct irq_domain *domain, } } -void irq_domain_disassociate (struct irq_domain *domain,unsigned int irq) +static void irq_domain_disassociate (struct irq_domain *domain,unsigned int irq) {struct irq_data *irq_data =irq_get_irq_data (irq);irq_hw_number_t hwirq;
@@ -582,6 +582,13 @@ void irq_domain_associate_many(struct irq_domain *domain, unsigned int irq_base, }EXPORT_SYMBOL_GPL (irq_domain_associate_many); +static void irq_mapped_free_desc (struct irq_desc *desc) +{ +unsigned int virq = desc->irq_data.irq; + +irq_domain_disassociate (desc->irq_data.domain, virq); +} +/** * irq_create_direct_mapping() - Allocate an irq for direct mapping * @domain: domain to allocate the irq for or NULL for default domain
@@ -638,6 +645,7 @@ unsigned int irq_create_mapping(struct irq_domain *domain, {struct device_node *of_node;int virq; +struct irq_desc *desc;pr_debug ("irq_create_mapping(0x%p, 0x%lx)\n" , domain, hwirq);
@@ -674,6 +682,9 @@ unsigned int irq_create_mapping(struct irq_domain *domain,pr_debug ("irq %lu on domain %s mapped to virtual irq %u\n" , hwirq,of_node_full_name (of_node), virq); + desc =irq_to_desc (virq); + desc->free_irq = irq_mapped_free_desc; +return virq; }EXPORT_SYMBOL_GPL (irq_create_mapping);
@@ -865,7 +876,6 @@ void irq_dispose_mapping(unsigned int virq)if (irq_domain_is_hierarchy (domain)) {irq_domain_free_irqs (virq,1 ); }else { -irq_domain_disassociate (domain, virq);irq_free_desc (virq); } }
Signature · 1 line
2.17.1
Re: [PATCH kernel v4 5/8] genirq: Add free_irq hook for IRQ descriptor and use for mapping disposal
Alexey,
Quoted 2 lines · Alexey Kardashevskiy
We want to make the irq_desc.kobj's release hook free associated resources but we do not want to pollute the irqdesc code with domains.
Can you please describe your changelog in factual ways without 'we and I and want'? See Documentation/process/
Quoted 6 lines
This adds a
free_irqhook which is called when the last reference to the descriptor is dropped.The first user is mapped irqs. This potentially can break the existing users; however they seem to do the right thing and call dispose once per mapping.
Q: How is this supposed to work with CONFIG_SPARSE_IRQ=n?
A: Not at all.
Also 'seem to do the right thing' is from the same quality as 'should not break stuff'. Either you have done a proper analysis or you did not. Aside of that how is anyone supposed to decode the subtle wreckage which is this going to cause if 'seem to do the right thing' turns out to be wrong?
kernel/irq/irqdomain.c void irq_domain_associate_many(struct irq_domain *domain, unsigned int irq_base, · line 585
Quoted 10 lines
kernel/irq/irqdomain.c +2
-1-void irq_domain_disassociate (struct irq_domain *domain,unsigned int irq) +static void irq_domain_disassociate (struct irq_domain *domain,unsigned int irq) {struct irq_data *irq_data =irq_get_irq_data (irq);irq_hw_number_t hwirq;@@ -582,6 +582,13 @@ void irq_domain_associate_many(struct irq_domain *domain, unsigned int irq_base, }EXPORT_SYMBOL_GPL (irq_domain_associate_many); +static void irq_mapped_free_desc (struct irq_desc *desc)
That function name is really misleading and badly chosen. The function is not about freeing the irq descriptor as the name suggests. It's called from that code in order to clean up the mapping.
Quoted 10 lines
kernel/irq/irqdomain.c +6
-0+{ +unsigned int virq = desc->irq_data.irq; + +irq_domain_disassociate (desc->irq_data.domain, virq); +} +/** * irq_create_direct_mapping() - Allocate an irq for direct mapping * @domain: domain to allocate the irq for or NULL for default domain
kernel/irq/irqdomain.c unsigned int irq_create_mapping(struct irq_domain *domain, · line 648
Quoted 4 lines
kernel/irq/irqdomain.c +1
-0@@ -638,6 +645,7 @@ unsigned int irq_create_mapping(struct irq_domain *domain, {struct device_node *of_node;int virq; +struct irq_desc *desc;
This code uses reverse fir tree ordering of variables
struct device_node *of_node; struct irq_desc *desc;
int virq;
Why? Because it's simpler to read than the vertical camel case above.
Thanks,
tglx
Re: [PATCH kernel v4 5/8] genirq: Add free_irq hook for IRQ descriptor and use for mapping disposal
Quoted 6 lines · Thomas Gleixner
Quoted 2 lines · Alexey Kardashevskiy
We want to make the irq_desc.kobj's release hook free associated resources but we do not want to pollute the irqdesc code with domains.
Can you please describe your changelog in factual ways without 'we and I and want'? See
Documentation/process/
And while we are at process. MAINTAINERS clearly says:
IRQ DOMAINS (IRQ NUMBER MAPPING LIBRARY)
T: git git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/core
IRQ SUBSYSTEM
T: git git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/core
So why are these patches not applying against that git branch?
Thanks,
tglx
PATCH v4 6/8 genirq/irqdomain: Move hierarchical IRQ cleanup to kobject_release
In reply to Alexey KardashevskiyThis moves hierarchical domain's irqs cleanup into the kobject release hook to make irq_domain_free_irqs() as simple as kobject_put.
--- kernel/irq/irqdomain.c | 43 +++++++++++++++++++++--------------------- 1 file changed, 22 insertions(+), 21 deletions(-)
kernel/irq/irqdomain.c +22 -21
@@ -863,21 +863,9 @@ EXPORT_SYMBOL_GPL(irq_create_of_mapping);*/ void irq_dispose_mapping (unsigned int virq) { -struct irq_data *irq_data =irq_get_irq_data (virq); -struct irq_domain *domain; +struct irq_desc *desc =irq_to_desc (virq); -if (!virq || !irq_data) -return ; - - domain = irq_data->domain; -if (WARN_ON (domain ==NULL )) -return ; - -if (irq_domain_is_hierarchy (domain)) { -irq_domain_free_irqs (virq,1 ); - }else { -irq_free_desc (virq); - } +kobject_put (&desc->kobj); }EXPORT_SYMBOL_GPL (irq_dispose_mapping);
@@ -1396,6 +1384,19 @@ int irq_domain_alloc_irqs_hierarchy(struct irq_domain *domain,return domain->ops->alloc (domain, irq_base, nr_irqs, arg); } +static void irq_domain_hierarchy_free_desc (struct irq_desc *desc) +{ +unsigned int virq = desc->irq_data.irq; +struct irq_data *data =irq_get_irq_data (virq); + +mutex_lock (&irq_domain_mutex); +irq_domain_remove_irq (virq); +irq_domain_free_irqs_hierarchy (data->domain, virq,1 ); +mutex_unlock (&irq_domain_mutex); + +irq_domain_free_irq_data (virq,1 ); +} +int __irq_domain_alloc_irqs_data (struct irq_domain *domain,int virq,unsigned int nr_irqs,int node,void *arg,const struct irq_affinity_desc *affinity)
@@ -1430,7 +1431,10 @@ int __irq_domain_alloc_irqs_data(struct irq_domain *domain, int virq, }for (i =0 ; i < nr_irqs; i++) { +struct irq_desc *desc =irq_to_desc (virq + i); +irq_domain_insert_irq (virq + i); + desc->free_irq = irq_domain_hierarchy_free_desc; }mutex_unlock (&irq_domain_mutex);
@@ -1675,14 +1679,11 @@ void irq_domain_free_irqs(unsigned int virq, unsigned int nr_irqs) "NULL pointer, cannot free irq\n"))return ; -mutex_lock (&irq_domain_mutex); -for (i =0 ; i < nr_irqs; i++) -irq_domain_remove_irq (virq + i); -irq_domain_free_irqs_hierarchy (data->domain, virq, nr_irqs); -mutex_unlock (&irq_domain_mutex); +for (i =0 ; i < nr_irqs; i++) { +struct irq_desc *desc =irq_to_desc (virq + i); -irq_domain_free_irq_data (virq, nr_irqs); -irq_free_descs (virq, nr_irqs); +kobject_put (&desc->kobj); + } }/**
Signature · 1 line
2.17.1
kernel test robot
build botRe: [PATCH kernel v4 6/8] genirq/irqdomain: Move hierarchical IRQ cleanup to kobject_release
Thank you for the patch! Yet something to improve:
Hi Alexey,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on linux/master]
[also build test ERROR on linus/master v5.10-rc5 next-20201123] [cannot apply to tip/irq/core tip/x86/core]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Alexey-Kardashevskiy/genirq-irqdomain-Add-reference-counting-to-IRQs/20201124-142727 base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 09162bc32c880a791c6c0668ce0745cf7958f576 config: alpha-randconfig-r013-20201124 (attached as .config) compiler: alpha-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/50199be6fbdf9f1f27ff037a6bd6c602e57f7a5f
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Alexey-Kardashevskiy/genirq-irqdomain-Add-reference-counting-to-IRQs/20201124-142727 git checkout 50199be6fbdf9f1f27ff037a6bd6c602e57f7a5f
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=alpha
If you fix the issue, kindly add following tag as appropriate
All errors (new ones prefixed by >>):
kernel/irq/irqdomain.c: In function 'irq_dispose_mapping':
>> kernel/irq/irqdomain.c:868:19: error: 'struct irq_desc' has no member named 'kobj'
868 | kobject_put(&desc->kobj);
| ^~
kernel/irq/irqdomain.c: In function 'irq_domain_free_irqs':
kernel/irq/irqdomain.c:1685:20: error: 'struct irq_desc' has no member named 'kobj'
1685 | kobject_put(&desc->kobj);
| ^~ kernel/irq/irqdomain.c: At top level:
kernel/irq/irqdomain.c:1907:13: warning: no previous prototype for 'irq_domain_debugfs_init' [-Wmissing-prototypes]
1907 | void __init irq_domain_debugfs_init(struct dentry *root)
| ^~~~~~~~~~~~~~~~~~~~~~~
vim +868 kernel/irq/irqdomain.c
859
860 /**
861 * irq_dispose_mapping() - Unmap an interrupt
862 * @virq: linux irq number of the interrupt to unmap
863 */
864 void irq_dispose_mapping(unsigned int virq)
865 {
866 struct irq_desc *desc = irq_to_desc(virq);
867
> 868 kobject_put(&desc->kobj);
869 }
870 EXPORT_SYMBOL_GPL(irq_dispose_mapping);
871
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
Re: [PATCH kernel v4 6/8] genirq/irqdomain: Move hierarchical IRQ cleanup to kobject_release
In reply to Alexey KardashevskiyAlexey,
Quoted 2 lines · Alexey Kardashevskiy
This moves hierarchical domain's irqs cleanup into the kobject release hook to make
irq_domain_free_irqs()as simple askobject_put.
Truly simple: Simply broken in multiple ways.
CONFIG_SPARSE_IRQ=n is now completely buggered. It does not even compile anymore. Running core code changes through a larger set of cross compilers is neither rocket science nor optional.
For CONFIG_SPARSE_IRQ=y, see below.
Quoted 5 lines
kernel/irq/irqdomain.c +0
-1@@ -1675,14 +1679,11 @@ void irq_domain_free_irqs(unsigned int virq, unsigned int nr_irqs) "NULL pointer, cannot free irq\n"))return ; -mutex_lock (&irq_domain_mutex);
kernel/irq/irqdomain.c void irq_domain_free_irqs(unsigned int virq, unsigned int nr_irqs) · line 1685
Quoted 10 lines
kernel/irq/irqdomain.c +3
-6-for (i =0 ; i < nr_irqs; i++) -irq_domain_remove_irq (virq + i); -irq_domain_free_irqs_hierarchy (data->domain, virq, nr_irqs); -mutex_unlock (&irq_domain_mutex); +for (i =0 ; i < nr_irqs; i++) { +struct irq_desc *desc =irq_to_desc (virq + i); -irq_domain_free_irq_data (virq, nr_irqs); -irq_free_descs (virq, nr_irqs); +kobject_put (&desc->kobj);
So up to this point both irq_dispose_mapping() _and_ irq_domain_free_irqs() invoked irq_free_descs().
Let's look at the call chains:
irq_domain_free_irqs()
irq_free_descs()
mutex_lock(&sparse_irq_lock);
for (i...)
free_desc(from + i)
irq_remove_debugfs_entry();
unregister_irq_proc();
irq_sysfs_del();
delete_irq_desc();
call_rcu();
bitmap_clear(allocated_irqs, ...);
mutex_unlock(&sparse_irq_lock);with your modifications it does:
irq_domain_free_irqs()
for (i...)
kobject_put(&desc->kobj)
irq_kobj_release()
if (desc->free_irq)
desc->free_irq(desc);
irq_remove_debugfs_entry();
unregister_irq_proc();
delete_irq_desc();
call_rcu();Can you spot the wreckage? It's not even subtle, it's more than obvious.
1) None of the operations in irq_kobj_release() is protected by sparse_irq_lock anymore. There was a comment in free_desc() which explained what is protected. You removed parts of that comment and just left the sysfs portion of it above delete_irq_desc() which is completely bogus because you removed the irq_sysfs_del() call.
2) Nothing removes the freed interrupts from the allocation bitmap. Run this often enough and you exhausted the interrupt space.
And no, you cannot just go and invoke irq_free_descs() instead of kobject_put(), simply because you'd create lock order inversion vs. the free_irq() callback.
So no, it's not that simple and I'm not at all interested in another respin of this with some more duct tape applied.
It can be done, but that needs way more thought, a proper design which preserves the existing semantics completely and wants to be a fine grained series where each patch does exactly ONE small thing which is reviewable and testable on _ALL_ users of this code, i.e. _ALL_ architectures and irq chip implementations.
Thanks,
tglx
PATCH v4 7/8 genirq/irqdomain: Reference irq_desc for already mapped irqs
In reply to Alexey KardashevskiyThis references an irq_desc if already mapped interrupt requested to map again. This happends for PCI legacy interrupts where 4 interrupts are shared among all devices on the same PCI host bus adapter.
From now on, the users shall call irq_dispose_mapping() for every irq_create_fwspec_mapping(). Most (all?) users do not bother with disposing though so it is not very likely to break many things.
--- kernel/irq/irqdomain.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
kernel/irq/irqdomain.c +9 -1
@@ -663,7 +663,9 @@ unsigned int irq_create_mapping(struct irq_domain *domain,/* Check if mapping already exists */ virq =irq_find_mapping (domain, hwirq);if (virq) { + desc =irq_to_desc (virq);pr_debug ("-> existing mapping on virq %d\n" , virq); +kobject_get (&desc->kobj);return virq; }
@@ -762,6 +764,7 @@ unsigned int irq_create_fwspec_mapping(struct irq_fwspec *fwspec)irq_hw_number_t hwirq;unsigned int type =IRQ_TYPE_NONE ;int virq; +struct irq_desc *desc;if (fwspec->fwnode) { domain =irq_find_matching_fwspec (fwspec,DOMAIN_BUS_WIRED );
@@ -798,8 +801,11 @@ unsigned int irq_create_fwspec_mapping(struct irq_fwspec *fwspec)* current trigger type then we are done so return the * interrupt number. */ -if (type ==IRQ_TYPE_NONE || type ==irq_get_trigger_type (virq)) +if (type ==IRQ_TYPE_NONE || type ==irq_get_trigger_type (virq)) { + desc =irq_to_desc (virq); +kobject_get (&desc->kobj);return virq; + }/* * If the trigger type has not been set yet, then set
@@ -811,6 +817,8 @@ unsigned int irq_create_fwspec_mapping(struct irq_fwspec *fwspec)return 0 ;irqd_set_trigger_type (irq_data, type); + desc =irq_to_desc (virq); +kobject_get (&desc->kobj);return virq; }
Signature · 1 line
2.17.1
PATCH v4 8/8 powerpc/pci: Remove LSI mappings on device teardown
In reply to Alexey KardashevskiyFrom: Oliver O'Halloran <[email protected]>
When a passthrough IO adapter is removed from a pseries machine using hash MMU and the XIVE interrupt mode, the POWER hypervisor expects the guest OS to clear all page table entries related to the adapter. If some are still present, the RTAS call which isolates the PCI slot returns error 9001 "valid outstanding translations" and the removal of the IO adapter fails. This is because when the PHBs are scanned, Linux maps automatically the INTx interrupts in the Linux interrupt number space but these are never removed.
This problem can be fixed by adding the corresponding unmap operation when the device is removed. There's no pcibios_* hook for the remove case, but the same effect can be achieved using a bus notifier.
--- arch/powerpc/kernel/pci-common.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+)
arch/powerpc/kernel/pci-common.c +21 -0
@@ -404,6 +404,27 @@ static int pci_read_irq_line(struct pci_dev *pci_dev)return 0 ; } +static int ppc_pci_unmap_irq_line (struct notifier_block *nb, +unsigned long action,void *data) +{ +struct pci_dev *pdev =to_pci_dev (data); + +if (action ==BUS_NOTIFY_DEL_DEVICE ) +irq_dispose_mapping (pdev->irq); + +return NOTIFY_DONE ; +} + +static struct notifier_block ppc_pci_unmap_irq_notifier = { + .notifier_call = ppc_pci_unmap_irq_line, +}; + +static int ppc_pci_register_irq_notifier (void ) +{ +return bus_register_notifier (&pci_bus_type, &ppc_pci_unmap_irq_notifier); +} +arch_initcall (ppc_pci_register_irq_notifier); +/* * Platform support for /proc/bus/pci/X/Y mmap()s. * -- paulus.
Signature · 7 lines
2.17.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel