powerpc/xive: misc cleanups
Subsystems (2)
v2 cover powerpc/xive: misc cleanups
Hello,
The most important change is the removal of support of OPAL flags required for P9 DD1. It provides a good cleanup of some complex routines.
Thanks,
C.
Changes since v1:
- dropped the change on the allocation of the pages donated to the XIVE IC. Needs a retest on a specific system.
- Took into account Greg's comments on flag removal.
Cédric Le Goater (13):
KVM: PPC: Book3S HV: XIVE: Show detailed configuration in debug output powerpc/xive: Rename XIVE_IRQ_NO_EOI to show its a flag powerpc/xive: Introduce XIVE_IPI_HW_IRQ
powerpc/xive: Use cpu_to_node() instead of ibm,chip-id property powerpc/xive: Add a name to the IRQ domain
powerpc/xive: Add a debug_show handler to the XIVE irq_domain
powerpc: Increase NR_IRQS range to support more KVM guests powerpc/xive: Remove P9 DD1 flag XIVE_IRQ_FLAG_SHIFT_BUG powerpc/xive: Remove P9 DD1 flag XIVE_IRQ_FLAG_MASK_FW powerpc/xive: Remove P9 DD1 flag XIVE_IRQ_FLAG_EOI_FW powerpc/xive: Simplify xive_do_source_eoi()
powerpc/xive: Improve error reporting of OPAL calls
KVM: PPC: Book3S HV: XIVE: Add a comment regarding VP numbering
arch/powerpc/include/asm/opal-api.h | 6 +- arch/powerpc/include/asm/xive.h | 8 +- arch/powerpc/kvm/book3s_xive.h | 13 ++ arch/powerpc/sysdev/xive/xive-internal.h | 7 +- arch/powerpc/kvm/book3s_xive.c | 134 +++++++------- arch/powerpc/kvm/book3s_xive_native.c | 24 ++- arch/powerpc/kvm/book3s_xive_template.c | 5 - arch/powerpc/sysdev/xive/common.c | 214 +++++++++++------------ arch/powerpc/sysdev/xive/native.c | 46 ++--- arch/powerpc/sysdev/xive/spapr.c | 8 +- arch/powerpc/Kconfig | 2 +- 11 files changed, 234 insertions(+), 233 deletions(-)
Signature · 1 line
2.26.2
PATCH v2 1/13 KVM: PPC: Book3S HV: XIVE: Show detailed configuration in debug output
This is useful to track allocation of the HW resources on per guest basis. Making sure IPIs are local to the chip of the vCPUs reduces rerouting between interrupt controllers and gives better performance in case of pinning. Checking the distribution of VP structures on the chips also helps in reducing PowerBUS traffic.
--- arch/powerpc/kvm/book3s_xive.h | 2 + arch/powerpc/kvm/book3s_xive.c | 76 ++++++++++++++++++++++----- arch/powerpc/kvm/book3s_xive_native.c | 21 ++++++-- 3 files changed, 82 insertions(+), 17 deletions(-)
arch/powerpc/kvm/book3s_xive.h +2 -0
@@ -290,6 +290,8 @@ extern int (*__xive_vm_h_eoi)(struct kvm_vcpu *vcpu, unsigned long xirr);*/ void kvmppc_xive_disable_vcpu_interrupts (struct kvm_vcpu *vcpu);int kvmppc_xive_debug_show_queues (struct seq_file *m,struct kvm_vcpu *vcpu); +void kvmppc_xive_debug_show_sources (struct seq_file *m, +struct kvmppc_xive_src_block *sb);struct kvmppc_xive_src_block *kvmppc_xive_create_src_block (struct kvmppc_xive *xive,int irq);void kvmppc_xive_free_sources (struct kvmppc_xive_src_block *sb);
arch/powerpc/kvm/book3s_xive.c +63 -13
119 lines not on this page · raw message
arch/powerpc/kvm/book3s_xive_native.c +17 -4
@@ -1257,18 +1257,31 @@ static int xive_native_debug_show(struct seq_file *m, void *private)if (!xc)continue ; -seq_printf (m,"cpu server %#x VP=%#x NSR=%02x CPPR=%02x IBP=%02x PIPR=%02x w01=%016llx w2=%08x\n" , - xc->server_num, xc->vp_id, +seq_printf (m,"VCPU %d: VP=%#x/%02x\n" +" NSR=%02x CPPR=%02x IBP=%02x PIPR=%02x w01=%016llx w2=%08x\n" , + xc->server_num, xc->vp_id, xc->vp_chip_id, vcpu->arch.xive_saved_state.nsr, vcpu->arch.xive_saved_state.cppr, vcpu->arch.xive_saved_state.ipb, vcpu->arch.xive_saved_state.pipr, - vcpu->arch.xive_saved_state.w01, - (u32 ) vcpu->arch.xive_cam_word); +be64_to_cpu (vcpu->arch.xive_saved_state.w01), +be32_to_cpu (vcpu->arch.xive_cam_word));kvmppc_xive_debug_show_queues (m, vcpu); } +seq_puts (m,"=========\nSources\n=========\n" ); + +for (i =0 ; i <= xive->max_sbid; i++) { +struct kvmppc_xive_src_block *sb = xive->src_blocks[i]; + +if (sb) { +arch_spin_lock (&sb->lock); +kvmppc_xive_debug_show_sources (m, sb); +arch_spin_unlock (&sb->lock); + } + } +return 0 ; }
Signature · 1 line
2.26.2
PATCH v2 2/13 powerpc/xive: Rename XIVE_IRQ_NO_EOI to show its a flag
In reply to Cédric Le GoaterThis is a simple cleanup to identify easily all flags of the XIVE interrupt structure. The interrupts flagged with XIVE_IRQ_FLAG_NO_EOI are the escalations used to wake up vCPUs in KVM. They are handled very differently from the rest.
--- arch/powerpc/include/asm/xive.h | 2 +- arch/powerpc/kvm/book3s_xive.c | 4 ++-- arch/powerpc/sysdev/xive/common.c | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-)
arch/powerpc/include/asm/xive.h +1 -1
@@ -66,7 +66,7 @@ struct xive_irq_data {#define XIVE_IRQ_FLAG_H_INT_ESB 0x20/* Special flag set by KVM for excalation interrupts */ -#define XIVE_IRQ_NO_EOI 0x80 +#define XIVE_IRQ_FLAG_NO_EOI 0x80#define XIVE_INVALID_CHIP_ID -1
arch/powerpc/kvm/book3s_xive.c +2 -2
@@ -219,7 +219,7 @@ int kvmppc_xive_attach_escalation(struct kvm_vcpu *vcpu, u8 prio,/* In single escalation mode, we grab the ESB MMIO of the * interrupt and mask it. Also populate the VCPU v/raddr * of the ESB page for use by asm entry/exit code. Finally -* set the XIVE_IRQ_NO_EOI flag which will prevent the +* set the XIVE_IRQ_FLAG_NO_EOI flag which will prevent the * core code from performing an EOI on the escalation * interrupt, thus leaving it effectively masked after * it fires once.
@@ -231,7 +231,7 @@ int kvmppc_xive_attach_escalation(struct kvm_vcpu *vcpu, u8 prio,xive_vm_esb_load (xd,XIVE_ESB_SET_PQ_01 ); vcpu->arch.xive_esc_raddr = xd->eoi_page; vcpu->arch.xive_esc_vaddr = (__force u64 )xd->eoi_mmio; - xd->flags |=XIVE_IRQ_NO_EOI ; + xd->flags |=XIVE_IRQ_FLAG_NO_EOI ; }return 0 ;
arch/powerpc/sysdev/xive/common.c +1 -1
@@ -416,7 +416,7 @@ static void xive_irq_eoi(struct irq_data *d)* been passed-through to a KVM guest */ if (!irqd_irq_disabled (d) && !irqd_is_forwarded_to_vcpu (d) && - !(xd->flags &XIVE_IRQ_NO_EOI )) + !(xd->flags &XIVE_IRQ_FLAG_NO_EOI ))xive_do_source_eoi (irqd_to_hwirq (d), xd);else xd->stale_p = true;
Signature · 1 line
2.26.2
PATCH v2 3/13 powerpc/xive: Introduce XIVE_IPI_HW_IRQ
In reply to Cédric Le GoaterThe XIVE driver deals with CPU IPIs in a peculiar way. Each CPU has its own XIVE IPI interrupt allocated at the HW level, for PowerNV, or at the hypervisor level for pSeries. In practice, these interrupts are not always used. pSeries/PowerVM prefers local doorbells for local threads since they are faster. On PowerNV, global doorbells are also preferred for the same reason.
The mapping in the Linux is reduced to a single interrupt using HW interrupt number 0 and a custom irq_chip to handle EOI. This can cause performance issues in some benchmark (ipistorm) on multichip systems.
Clarify the use of the 0 value, it will help in improving multichip support.
--- arch/powerpc/sysdev/xive/xive-internal.h | 2 ++ arch/powerpc/sysdev/xive/common.c | 10 +++++----- 2 files changed, 7 insertions(+), 5 deletions(-)
arch/powerpc/sysdev/xive/xive-internal.h +2 -0
@@ -5,6 +5,8 @@#ifndef __XIVE_INTERNAL_H#define __XIVE_INTERNAL_H +#define XIVE_IPI_HW_IRQ 0/* interrupt source # for IPIs */ +/* * A "disabled" interrupt should never fire, to catch problems * we set its logical number to this
arch/powerpc/sysdev/xive/common.c +5 -5
@@ -1142,7 +1142,7 @@ static void __init xive_request_ipi(void)return ;/* Initialize it */ - virq =irq_create_mapping (xive_irq_domain,0 ); + virq =irq_create_mapping (xive_irq_domain,XIVE_IPI_HW_IRQ ); xive_ipi_irq = virq; WARN_ON(request_irq(virq, xive_muxed_ipi_action,
@@ -1242,7 +1242,7 @@ static int xive_irq_domain_map(struct irq_domain *h, unsigned int virq,#ifdef CONFIG_SMP /* IPIs are special and come up with HW number 0 */ -if (hw ==0 ) { +if (hw ==XIVE_IPI_HW_IRQ ) {/* * IPIs are marked per-cpu. We use separate HW interrupts under * the hood but associated with the same "linux" interrupt
@@ -1271,7 +1271,7 @@ static void xive_irq_domain_unmap(struct irq_domain *d, unsigned int virq)if (!data)return ; hw_irq = (unsigned int )irqd_to_hwirq (data); -if (hw_irq) +if (hw_irq !=XIVE_IPI_HW_IRQ )xive_irq_free_data (virq); }
@@ -1421,7 +1421,7 @@ static void xive_flush_cpu_queue(unsigned int cpu, struct xive_cpu *xc)* Ignore anything that isn't a XIVE irq and ignore * IPIs, so can just be dropped. */ -if (d->domain != xive_irq_domain || hw_irq ==0 ) +if (d->domain != xive_irq_domain || hw_irq ==XIVE_IPI_HW_IRQ )continue ;/*
@@ -1655,7 +1655,7 @@ static int xive_core_debug_show(struct seq_file *m, void *private) hw_irq = (unsigned int )irqd_to_hwirq (d);/* IPIs are special (HW number 0) */ -if (hw_irq) +if (hw_irq !=XIVE_IPI_HW_IRQ )xive_debug_show_irq (m, hw_irq, d); }return 0 ;
Signature · 1 line
2.26.2
PATCH v2 4/13 powerpc/xive: Use cpu_to_node() instead of ibm, chip-id property
In reply to Cédric Le GoaterThe 'chip_id' field of the XIVE CPU structure is used to choose a target for a source located on the same chip when possible. This field is assigned on the PowerNV platform using the "ibm,chip-id" property on pSeries under KVM when NUMA nodes are defined but it is undefined under PowerVM. The XIVE source structure has a similar field 'src_chip' which is only assigned on the PowerNV platform.
cpu_to_node() returns a compatible value on all platforms, 0 being the default node. It will also give us the opportunity to set the affinity of a source on pSeries when we can localize them.
--- arch/powerpc/sysdev/xive/common.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-)
arch/powerpc/sysdev/xive/common.c +1 -6
@@ -1342,16 +1342,11 @@ static int xive_prepare_cpu(unsigned int cpu) xc =per_cpu (xive_cpu, cpu);if (!xc) { -struct device_node *np; - xc =kzalloc_node (sizeof (struct xive_cpu ),GFP_KERNEL ,cpu_to_node (cpu));if (!xc)return -ENOMEM ; - np =of_get_cpu_node (cpu,NULL ); -if (np) - xc->chip_id =of_get_ibm_chip_id (np); -of_node_put (np); + xc->chip_id =cpu_to_node (cpu); xc->hw_ipi =XIVE_BAD_IRQ ;per_cpu (xive_cpu, cpu) = xc;
Signature · 1 line
2.26.2
PATCH v2 5/13 powerpc/xive: Add a name to the IRQ domain
In reply to Cédric Le GoaterWe hope one day to handle multiple irq_domain in the XIVE driver. Start simple by setting the name using the DT node.
--- arch/powerpc/sysdev/xive/xive-internal.h | 4 ++-- arch/powerpc/sysdev/xive/common.c | 10 +++++----- arch/powerpc/sysdev/xive/native.c | 2 +- arch/powerpc/sysdev/xive/spapr.c | 2 +- 4 files changed, 9 insertions(+), 9 deletions(-)
arch/powerpc/sysdev/xive/xive-internal.h +2 -2
@@ -63,8 +63,8 @@ struct xive_ops {const char *name; }; -bool xive_core_init (const struct xive_ops *ops,void __iomem *area,u32 offset, -u8 max_prio); +bool xive_core_init (struct device_node *np,const struct xive_ops *ops, +void __iomem *area,u32 offset,u8 max_prio);__be32 *xive_queue_page_alloc (unsigned int cpu,u32 queue_shift);int xive_core_debug_init (void );
arch/powerpc/sysdev/xive/common.c +5 -5
@@ -1310,9 +1310,9 @@ static const struct irq_domain_ops xive_irq_domain_ops = { .xlate = xive_irq_domain_xlate, }; -static void __init xive_init_host (void ) +static void __init xive_init_host (struct device_node *np) { - xive_irq_domain =irq_domain_add_nomap (NULL ,XIVE_MAX_IRQ , + xive_irq_domain =irq_domain_add_nomap (np,XIVE_MAX_IRQ , &xive_irq_domain_ops,NULL );if (WARN_ON (xive_irq_domain ==NULL ))return ;
@@ -1508,8 +1508,8 @@ void xive_shutdown(void) xive_ops->shutdown (); } -bool __init xive_core_init (const struct xive_ops *ops,void __iomem *area,u32 offset, -u8 max_prio) +bool __init xive_core_init (struct device_node *np,const struct xive_ops *ops, +void __iomem *area,u32 offset,u8 max_prio) { xive_tima = area; xive_tima_offset = offset;
@@ -1520,7 +1520,7 @@ bool __init xive_core_init(const struct xive_ops *ops, void __iomem *area, u32 o __xive_enabled = true;pr_devel ("Initializing host..\n" ); -xive_init_host (); +xive_init_host (np);pr_devel ("Initializing boot CPU..\n" );
arch/powerpc/sysdev/xive/native.c +1 -1
@@ -622,7 +622,7 @@ bool __init xive_native_init(void)xive_native_setup_pools ();/* Initialize XIVE core with our backend */ -if (!xive_core_init (&xive_native_ops, tima,TM_QW3_HV_PHYS , +if (!xive_core_init (np, &xive_native_ops, tima,TM_QW3_HV_PHYS , max_prio)) {opal_xive_reset (OPAL_XIVE_MODE_EMU );return false;
arch/powerpc/sysdev/xive/spapr.c +1 -1
@@ -857,7 +857,7 @@ bool __init xive_spapr_init(void) }/* Initialize XIVE core with our backend */ -if (!xive_core_init (&xive_spapr_ops, tima,TM_QW1_OS , max_prio)) +if (!xive_core_init (np, &xive_spapr_ops, tima,TM_QW1_OS , max_prio))return false;pr_info ("Using %dkB queues\n" ,1 << (xive_queue_shift -10 ));
Signature · 1 line
2.26.2
PATCH v2 6/13 powerpc/xive: Add a debug_show handler to the XIVE irq_domain
In reply to Cédric Le GoaterFull state of the Linux interrupt descriptors can be dumped under debugfs when compiled with CONFIG_GENERIC_IRQ_DEBUGFS. Add support for the XIVE interrupt controller.
--- arch/powerpc/sysdev/xive/common.c | 58 +++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+)
arch/powerpc/sysdev/xive/common.c +58 -0
@@ -1303,11 +1303,69 @@ static int xive_irq_domain_match(struct irq_domain *h, struct device_node *node,return xive_ops->match (node); } +#ifdef CONFIG_GENERIC_IRQ_DEBUGFS +static const char *const esb_names[] = {"RESET" ,"OFF" ,"PENDING" ,"QUEUED" }; + +static const struct { +u64 mask; +char *name; +} xive_irq_flags[] = { + {XIVE_IRQ_FLAG_STORE_EOI ,"STORE_EOI" }, + {XIVE_IRQ_FLAG_LSI ,"LSI" }, + {XIVE_IRQ_FLAG_SHIFT_BUG ,"SHIFT_BUG" }, + {XIVE_IRQ_FLAG_MASK_FW ,"MASK_FW" }, + {XIVE_IRQ_FLAG_EOI_FW ,"EOI_FW" }, + {XIVE_IRQ_FLAG_H_INT_ESB ,"H_INT_ESB" }, + {XIVE_IRQ_FLAG_NO_EOI ,"NO_EOI" }, +}; + +static void xive_irq_domain_debug_show (struct seq_file *m,struct irq_domain *d, +struct irq_data *irqd,int ind) +{ +struct xive_irq_data *xd; +u64 val; +int i; + +/* No IRQ domain level information. To be done */ +if (!irqd) +return ; + +if (!is_xive_irq (irq_data_get_irq_chip (irqd))) +return ; + +seq_printf (m,"%*sXIVE:\n" , ind,"" ); + ind++; + + xd =irq_data_get_irq_handler_data (irqd); +if (!xd) { +seq_printf (m,"%*snot assigned\n" , ind,"" ); +return ; + } + + val =xive_esb_read (xd,XIVE_ESB_GET ); +seq_printf (m,"%*sESB: %s\n" , ind,"" , esb_names[val &0x3 ]); +seq_printf (m,"%*sPstate: %s %s\n" , ind,"" , xd->stale_p ?"stale" :"" , + xd->saved_p ?"saved" :"" ); +seq_printf (m,"%*sTarget: %d\n" , ind,"" , xd->target); +seq_printf (m,"%*sChip: %d\n" , ind,"" , xd->src_chip); +seq_printf (m,"%*sTrigger: 0x%016llx\n" , ind,"" , xd->trig_page); +seq_printf (m,"%*sEOI: 0x%016llx\n" , ind,"" , xd->eoi_page); +seq_printf (m,"%*sFlags: 0x%llx\n" , ind,"" , xd->flags); +for (i =0 ; i <ARRAY_SIZE (xive_irq_flags); i++) { +if (xd->flags & xive_irq_flags[i].mask) +seq_printf (m,"%*s%s\n" , ind +12 ,"" , xive_irq_flags[i].name); + } +} +#endif +static const struct irq_domain_ops xive_irq_domain_ops = { .match = xive_irq_domain_match, .map = xive_irq_domain_map, .unmap = xive_irq_domain_unmap, .xlate = xive_irq_domain_xlate, +#ifdef CONFIG_GENERIC_IRQ_DEBUGFS + .debug_show = xive_irq_domain_debug_show, +#endif };static void __init xive_init_host (struct device_node *np)
Signature · 1 line
2.26.2
PATCH v2 7/13 powerpc: Increase NR_IRQS range to support more KVM guests
In reply to Cédric Le GoaterPowerNV systems can handle up to 4K guests and 1M interrupt numbers per chip. Increase the range of allowed interrupts to support a larger number of guests.
--- arch/powerpc/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/powerpc/Kconfig +1 -1
@@ -66,7 +66,7 @@ config NEED_PER_CPU_PAGE_FIRST_CHUNK config NR_IRQS int "Number of virtual interrupt numbers" - range 32 32768 + range 32 1048576 default "512" help This defines the number of virtual interrupt numbers the kernel
Signature · 1 line
2.26.2
Michael Ellerman
reviewerRe: [PATCH v2 07/13] powerpc: Increase NR_IRQS range to support more KVM guests
Quoted 15 lines · Cédric Le Goater
PowerNV systems can handle up to 4K guests and 1M interrupt numbers per chip. Increase the range of allowed interrupts to support a larger number of guests.
Reviewed-by Greg Kurz <[email protected]>Signed-off-by Cédric Le Goater <[email protected]>--- arch/powerpc/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/powerpc/Kconfig +0
-0
arch/powerpc/Kconfig config NEED_PER_CPU_PAGE_FIRST_CHUNK · line 72
Quoted 8 lines
arch/powerpc/Kconfig +1
-1@@ -66,7 +66,7 @@ config NEED_PER_CPU_PAGE_FIRST_CHUNK config NR_IRQS int "Number of virtual interrupt numbers" - range 32 32768 + range 32 1048576 default "512" help This defines the number of virtual interrupt numbers the kernel
We should really do what other arches do, and size this appropriately based on the config, rather than asking users to guess what size they need.
But I guess I'll take this for now, and we can do something fancier later.
cheers
Re: [PATCH v2 07/13] powerpc: Increase NR_IRQS range to support more KVM guests
Quoted 29 lines · Michael Ellerman
Quoted 15 lines · Cédric Le Goater
PowerNV systems can handle up to 4K guests and 1M interrupt numbers per chip. Increase the range of allowed interrupts to support a larger number of guests.
Reviewed-by Greg Kurz <[email protected]>Signed-off-by Cédric Le Goater <[email protected]>--- arch/powerpc/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/powerpc/Kconfig +0
-0arch/powerpc/Kconfig config NEED_PER_CPU_PAGE_FIRST_CHUNK · line 72
Quoted 8 lines
arch/powerpc/Kconfig +1
-1@@ -66,7 +66,7 @@ config NEED_PER_CPU_PAGE_FIRST_CHUNK config NR_IRQS int "Number of virtual interrupt numbers" - range 32 32768 + range 32 1048576 default "512" help This defines the number of virtual interrupt numbers the kernelWe should really do what other arches do, and size this appropriately based on the config, rather than asking users to guess what size they need.
Quoted 2 lines
But I guess I'll take this for now, and we can do something fancier later.
I was thinking on adding a property to OPAL to size the HW interrupt number space. Is that it ? That would be good because it's increasing from 20bits on P9 to 24bits on P10.
I am checking other arches.
Thanks,
C.
Michael Ellerman
reviewerRe: [PATCH v2 07/13] powerpc: Increase NR_IRQS range to support more KVM guests
Quoted 30 lines · Cédric Le Goater
Quoted 29 lines · Michael Ellerman
Quoted 15 lines · Cédric Le Goater
PowerNV systems can handle up to 4K guests and 1M interrupt numbers per chip. Increase the range of allowed interrupts to support a larger number of guests.
Reviewed-by Greg Kurz <[email protected]>Signed-off-by Cédric Le Goater <[email protected]>--- arch/powerpc/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/powerpc/Kconfig +0
-0arch/powerpc/Kconfig config NEED_PER_CPU_PAGE_FIRST_CHUNK · line 72
Quoted 8 lines
arch/powerpc/Kconfig +1
-1@@ -66,7 +66,7 @@ config NEED_PER_CPU_PAGE_FIRST_CHUNK config NR_IRQS int "Number of virtual interrupt numbers" - range 32 32768 + range 32 1048576 default "512" help This defines the number of virtual interrupt numbers the kernelWe should really do what other arches do, and size this appropriately based on the config, rather than asking users to guess what size they need.
Quoted 5 lines
Quoted 2 lines
But I guess I'll take this for now, and we can do something fancier later.
I was thinking on adding a property to OPAL to size the HW interrupt number space. Is that it ?
That's a separate issue. NR_IRQS is about the maximum number of Linux interrupts, and it's a compile time limit.
In the old days there was an array of irq_desc[NR_IRQS] in .data, so you didn't want NR_IRQS to be too big. These days we don't do that, because of the sparse IRQ support, but I don't know if it's completely free to make NR_IRQS arbitrarily large at build time.
Quoted 2 lines
That would be good because it's increasing from 20bits on P9 to 24bits on P10.
That's probably still helpful, it might mean we can shrink some structures at runtime.
Quoted 1 line
I am checking other arches.
Thanks.
cheers
PATCH v2 8/13 powerpc/xive: Remove P9 DD1 flag XIVE_IRQ_FLAG_SHIFT_BUG
In reply to Cédric Le GoaterThis flag was used to support the PHB4 LSIs on P9 DD1 and we have stopped supporting this CPU when DD2 came out. See skiboot commit:
https://github.com/open-power/skiboot/commit/0b0d15e3c170
--- arch/powerpc/include/asm/opal-api.h | 2 +- arch/powerpc/include/asm/xive.h | 2 +- arch/powerpc/kvm/book3s_xive_native.c | 3 --- arch/powerpc/kvm/book3s_xive_template.c | 3 --- arch/powerpc/sysdev/xive/common.c | 9 --------- arch/powerpc/sysdev/xive/native.c | 2 -- 6 files changed, 2 insertions(+), 19 deletions(-)
arch/powerpc/include/asm/opal-api.h +1 -1
@@ -1091,7 +1091,7 @@ enum {OPAL_XIVE_IRQ_TRIGGER_PAGE =0x00000001 ,OPAL_XIVE_IRQ_STORE_EOI =0x00000002 ,OPAL_XIVE_IRQ_LSI =0x00000004 , -OPAL_XIVE_IRQ_SHIFT_BUG =0x00000008 , +OPAL_XIVE_IRQ_SHIFT_BUG =0x00000008 ,/* P9 DD1.0 workaround */ OPAL_XIVE_IRQ_MASK_VIA_FW =0x00000010 , OPAL_XIVE_IRQ_EOI_VIA_FW = 0x00000020, };
arch/powerpc/include/asm/xive.h +1 -1
@@ -60,7 +60,7 @@ struct xive_irq_data { };#define XIVE_IRQ_FLAG_STORE_EOI 0x01#define XIVE_IRQ_FLAG_LSI 0x02 -#define XIVE_IRQ_FLAG_SHIFT_BUG 0x04 +/* #define XIVE_IRQ_FLAG_SHIFT_BUG 0x04 */ /* P9 DD1.0 workaround */ #define XIVE_IRQ_FLAG_MASK_FW 0x08#define XIVE_IRQ_FLAG_EOI_FW 0x10#define XIVE_IRQ_FLAG_H_INT_ESB 0x20
arch/powerpc/kvm/book3s_xive_native.c +0 -3
@@ -37,9 +37,6 @@ static u8 xive_vm_esb_load(struct xive_irq_data *xd, u32 offset)* ordering. */ -if (xd->flags &XIVE_IRQ_FLAG_SHIFT_BUG ) - offset |= offset <<4 ; - val =in_be64 (xd->eoi_mmio + offset);return (u8 )val; }
arch/powerpc/kvm/book3s_xive_template.c +0 -3
@@ -61,9 +61,6 @@ static u8 GLUE(X_PFX,esb_load)(struct xive_irq_data *xd, u32 offset)if (offset ==XIVE_ESB_SET_PQ_10 && xd->flags &XIVE_IRQ_FLAG_STORE_EOI ) offset |=XIVE_ESB_LD_ST_MO ; -if (xd->flags &XIVE_IRQ_FLAG_SHIFT_BUG ) - offset |= offset <<4 ; - val =__x_readq (__x_eoi_page (xd) + offset);#ifdef __LITTLE_ENDIAN__ val >>=64 -8 ;
arch/powerpc/sysdev/xive/common.c +0 -9
@@ -200,10 +200,6 @@ static notrace u8 xive_esb_read(struct xive_irq_data *xd, u32 offset)if (offset ==XIVE_ESB_SET_PQ_10 && xd->flags &XIVE_IRQ_FLAG_STORE_EOI ) offset |=XIVE_ESB_LD_ST_MO ; -/* Handle HW errata */ -if (xd->flags &XIVE_IRQ_FLAG_SHIFT_BUG ) - offset |= offset <<4 ; -if ((xd->flags &XIVE_IRQ_FLAG_H_INT_ESB ) && xive_ops->esb_rw) val = xive_ops->esb_rw (xd->hw_irq, offset,0 ,0 );else
@@ -214,10 +210,6 @@ static notrace u8 xive_esb_read(struct xive_irq_data *xd, u32 offset)static void xive_esb_write (struct xive_irq_data *xd,u32 offset,u64 data) { -/* Handle HW errata */ -if (xd->flags &XIVE_IRQ_FLAG_SHIFT_BUG ) - offset |= offset <<4 ; -if ((xd->flags &XIVE_IRQ_FLAG_H_INT_ESB ) && xive_ops->esb_rw) xive_ops->esb_rw (xd->hw_irq, offset, data,1 );else
@@ -1312,7 +1304,6 @@ static const struct { } xive_irq_flags[] = { { XIVE_IRQ_FLAG_STORE_EOI, "STORE_EOI" }, { XIVE_IRQ_FLAG_LSI, "LSI" }, - { XIVE_IRQ_FLAG_SHIFT_BUG, "SHIFT_BUG" }, { XIVE_IRQ_FLAG_MASK_FW, "MASK_FW" }, { XIVE_IRQ_FLAG_EOI_FW, "EOI_FW" }, { XIVE_IRQ_FLAG_H_INT_ESB, "H_INT_ESB" },
arch/powerpc/sysdev/xive/native.c +0 -2
@@ -64,8 +64,6 @@ int xive_native_populate_irq_data(u32 hw_irq, struct xive_irq_data *data) data->flags |=XIVE_IRQ_FLAG_STORE_EOI ;if (opal_flags &OPAL_XIVE_IRQ_LSI ) data->flags |=XIVE_IRQ_FLAG_LSI ; -if (opal_flags &OPAL_XIVE_IRQ_SHIFT_BUG ) - data->flags |=XIVE_IRQ_FLAG_SHIFT_BUG ;if (opal_flags &OPAL_XIVE_IRQ_MASK_VIA_FW ) data->flags |=XIVE_IRQ_FLAG_MASK_FW ;if (opal_flags &OPAL_XIVE_IRQ_EOI_VIA_FW )
Signature · 1 line
2.26.2
PATCH v2 9/13 powerpc/xive: Remove P9 DD1 flag XIVE_IRQ_FLAG_MASK_FW
In reply to Cédric Le GoaterThis flag was used to support the PHB4 LSIs on P9 DD1 and we have stopped supporting this CPU when DD2 came out. See skiboot commit:
https://github.com/open-power/skiboot/commit/0b0d15e3c170
--- arch/powerpc/include/asm/opal-api.h | 2 +- arch/powerpc/include/asm/xive.h | 2 +- arch/powerpc/kvm/book3s_xive.c | 54 +++++------------------------ arch/powerpc/sysdev/xive/common.c | 40 +-------------------- arch/powerpc/sysdev/xive/native.c | 2 -- 5 files changed, 11 insertions(+), 89 deletions(-)
arch/powerpc/include/asm/opal-api.h +1 -1
@@ -1092,7 +1092,7 @@ enum {OPAL_XIVE_IRQ_STORE_EOI =0x00000002 ,OPAL_XIVE_IRQ_LSI =0x00000004 ,OPAL_XIVE_IRQ_SHIFT_BUG =0x00000008 ,/* P9 DD1.0 workaround */ -OPAL_XIVE_IRQ_MASK_VIA_FW =0x00000010 , +OPAL_XIVE_IRQ_MASK_VIA_FW =0x00000010 ,/* P9 DD1.0 workaround */ OPAL_XIVE_IRQ_EOI_VIA_FW = 0x00000020, };
arch/powerpc/include/asm/xive.h +1 -1
@@ -61,7 +61,7 @@ struct xive_irq_data {#define XIVE_IRQ_FLAG_STORE_EOI 0x01#define XIVE_IRQ_FLAG_LSI 0x02/* #define XIVE_IRQ_FLAG_SHIFT_BUG 0x04 */ /* P9 DD1.0 workaround */ -#define XIVE_IRQ_FLAG_MASK_FW 0x08 +/* #define XIVE_IRQ_FLAG_MASK_FW 0x08 */ /* P9 DD1.0 workaround */ #define XIVE_IRQ_FLAG_EOI_FW 0x10#define XIVE_IRQ_FLAG_H_INT_ESB 0x20
arch/powerpc/kvm/book3s_xive.c +8 -46
68 lines not on this page · raw message
arch/powerpc/sysdev/xive/common.c +1 -39
@@ -424,9 +424,7 @@ static void xive_irq_eoi(struct irq_data *d) }/* -* Helper used to mask and unmask an interrupt source. This -* is only called for normal interrupts that do not require -* masking/unmasking via firmware. +* Helper used to mask and unmask an interrupt source. */ static void xive_do_source_set_mask (struct xive_irq_data *xd,bool mask)
@@ -673,20 +671,6 @@ static void xive_irq_unmask(struct irq_data *d)pr_devel ("xive_irq_unmask: irq %d data @%p\n" , d->irq, xd); -/* -* This is a workaround for PCI LSI problems on P9, for -* these, we call FW to set the mask. The problems might -* be fixed by P9 DD2.0, if that is the case, firmware -* will no longer set that flag. -*/ -if (xd->flags &XIVE_IRQ_FLAG_MASK_FW ) { -unsigned int hw_irq = (unsigned int )irqd_to_hwirq (d); - xive_ops->configure_irq (hw_irq, -get_hard_smp_processor_id (xd->target), - xive_irq_priority, d->irq); -return ; - } -xive_do_source_set_mask (xd, false); }
@@ -696,20 +680,6 @@ static void xive_irq_mask(struct irq_data *d)pr_devel ("xive_irq_mask: irq %d data @%p\n" , d->irq, xd); -/* -* This is a workaround for PCI LSI problems on P9, for -* these, we call OPAL to set the mask. The problems might -* be fixed by P9 DD2.0, if that is the case, firmware -* will no longer set that flag. -*/ -if (xd->flags &XIVE_IRQ_FLAG_MASK_FW ) { -unsigned int hw_irq = (unsigned int )irqd_to_hwirq (d); - xive_ops->configure_irq (hw_irq, -get_hard_smp_processor_id (xd->target), -0xff , d->irq); -return ; - } -xive_do_source_set_mask (xd, true); }
@@ -852,13 +822,6 @@ static int xive_irq_set_vcpu_affinity(struct irq_data *d, void *state)int rc;u8 pq; -/* -* We only support this on interrupts that do not require -* firmware calls for masking and unmasking -*/ -if (xd->flags &XIVE_IRQ_FLAG_MASK_FW ) -return -EIO ; -/* * This is called by KVM with state non-NULL for enabling * pass-through or NULL for disabling it
@@ -1304,7 +1267,6 @@ static const struct { } xive_irq_flags[] = { { XIVE_IRQ_FLAG_STORE_EOI, "STORE_EOI" }, { XIVE_IRQ_FLAG_LSI, "LSI" }, - { XIVE_IRQ_FLAG_MASK_FW, "MASK_FW" }, { XIVE_IRQ_FLAG_EOI_FW, "EOI_FW" }, { XIVE_IRQ_FLAG_H_INT_ESB, "H_INT_ESB" }, { XIVE_IRQ_FLAG_NO_EOI, "NO_EOI" },
arch/powerpc/sysdev/xive/native.c +0 -2
@@ -64,8 +64,6 @@ int xive_native_populate_irq_data(u32 hw_irq, struct xive_irq_data *data) data->flags |=XIVE_IRQ_FLAG_STORE_EOI ;if (opal_flags &OPAL_XIVE_IRQ_LSI ) data->flags |=XIVE_IRQ_FLAG_LSI ; -if (opal_flags &OPAL_XIVE_IRQ_MASK_VIA_FW ) - data->flags |=XIVE_IRQ_FLAG_MASK_FW ;if (opal_flags &OPAL_XIVE_IRQ_EOI_VIA_FW ) data->flags |=XIVE_IRQ_FLAG_EOI_FW ; data->eoi_page =be64_to_cpu (eoi_page);
Signature · 1 line
2.26.2
PATCH v2 10/13 powerpc/xive: Remove P9 DD1 flag XIVE_IRQ_FLAG_EOI_FW
In reply to Cédric Le GoaterThis flag was used to support the P9 DD1 and we have stopped supporting this CPU when DD2 came out. See skiboot commit:
https://github.com/open-power/skiboot/commit/0b0d15e3c170
Also, remove eoi handler which is now unused.
--- arch/powerpc/include/asm/opal-api.h | 2 +- arch/powerpc/include/asm/xive.h | 2 +- arch/powerpc/sysdev/xive/xive-internal.h | 1 - arch/powerpc/kvm/book3s_xive_template.c | 2 -- arch/powerpc/sysdev/xive/common.c | 14 +------------- arch/powerpc/sysdev/xive/native.c | 12 ------------ arch/powerpc/sysdev/xive/spapr.c | 6 ------ 7 files changed, 3 insertions(+), 36 deletions(-)
arch/powerpc/include/asm/opal-api.h +1 -1
@@ -1093,7 +1093,7 @@ enum {OPAL_XIVE_IRQ_LSI =0x00000004 ,OPAL_XIVE_IRQ_SHIFT_BUG =0x00000008 ,/* P9 DD1.0 workaround */ OPAL_XIVE_IRQ_MASK_VIA_FW =0x00000010 ,/* P9 DD1.0 workaround */ - OPAL_XIVE_IRQ_EOI_VIA_FW = 0x00000020, + OPAL_XIVE_IRQ_EOI_VIA_FW = 0x00000020, /* P9 DD1.0 workaround */ };/* Flags for OPAL_XIVE_GET/SET_QUEUE_INFO */
arch/powerpc/include/asm/xive.h +1 -1
@@ -62,7 +62,7 @@ struct xive_irq_data {#define XIVE_IRQ_FLAG_LSI 0x02/* #define XIVE_IRQ_FLAG_SHIFT_BUG 0x04 */ /* P9 DD1.0 workaround */ /* #define XIVE_IRQ_FLAG_MASK_FW 0x08 */ /* P9 DD1.0 workaround */ -#define XIVE_IRQ_FLAG_EOI_FW 0x10 +/* #define XIVE_IRQ_FLAG_EOI_FW 0x10 */ /* P9 DD1.0 workaround */ #define XIVE_IRQ_FLAG_H_INT_ESB 0x20/* Special flag set by KVM for excalation interrupts */
arch/powerpc/sysdev/xive/xive-internal.h +0 -1
@@ -52,7 +52,6 @@ struct xive_ops {void (*shutdown)(void );void (*update_pending)(struct xive_cpu *xc); -void (*eoi)(u32 hw_irq);void (*sync_source)(u32 hw_irq);u64 (*esb_rw)(u32 hw_irq,u32 offset,u64 data,bool write);#ifdef CONFIG_SMP
arch/powerpc/kvm/book3s_xive_template.c +0 -2
@@ -74,8 +74,6 @@ static void GLUE(X_PFX,source_eoi)(u32 hw_irq, struct xive_irq_data *xd)/* If the XIVE supports the new "store EOI facility, use it */ if (xd->flags &XIVE_IRQ_FLAG_STORE_EOI )__x_writeq (0 ,__x_eoi_page (xd) +XIVE_ESB_STORE_EOI ); -else if (hw_irq && xd->flags &XIVE_IRQ_FLAG_EOI_FW ) -opal_int_eoi (hw_irq);else if (xd->flags &XIVE_IRQ_FLAG_LSI ) {/* * For LSIs the HW EOI cycle is used rather than PQ bits,
arch/powerpc/sysdev/xive/common.c +1 -13
@@ -354,18 +354,7 @@ static void xive_do_source_eoi(u32 hw_irq, struct xive_irq_data *xd)/* If the XIVE supports the new "store EOI facility, use it */ if (xd->flags &XIVE_IRQ_FLAG_STORE_EOI )xive_esb_write (xd,XIVE_ESB_STORE_EOI ,0 ); -else if (hw_irq && xd->flags &XIVE_IRQ_FLAG_EOI_FW ) { -/* -* The FW told us to call it. This happens for some -* interrupt sources that need additional HW whacking -* beyond the ESB manipulation. For example LPC interrupts -* on P9 DD1.0 needed a latch to be clared in the LPC bridge -* itself. The Firmware will take care of it. -*/ -if (WARN_ON_ONCE (!xive_ops->eoi)) -return ; - xive_ops->eoi (hw_irq); - }else { +else {u8 eoi_val;/*
@@ -1267,7 +1256,6 @@ static const struct { } xive_irq_flags[] = { { XIVE_IRQ_FLAG_STORE_EOI, "STORE_EOI" }, { XIVE_IRQ_FLAG_LSI, "LSI" }, - { XIVE_IRQ_FLAG_EOI_FW, "EOI_FW" }, { XIVE_IRQ_FLAG_H_INT_ESB, "H_INT_ESB" }, { XIVE_IRQ_FLAG_NO_EOI, "NO_EOI" }, };
arch/powerpc/sysdev/xive/native.c +0 -12
@@ -64,8 +64,6 @@ int xive_native_populate_irq_data(u32 hw_irq, struct xive_irq_data *data) data->flags |=XIVE_IRQ_FLAG_STORE_EOI ;if (opal_flags &OPAL_XIVE_IRQ_LSI ) data->flags |=XIVE_IRQ_FLAG_LSI ; -if (opal_flags &OPAL_XIVE_IRQ_EOI_VIA_FW ) - data->flags |=XIVE_IRQ_FLAG_EOI_FW ; data->eoi_page =be64_to_cpu (eoi_page); data->trig_page =be64_to_cpu (trig_page); data->esb_shift =be32_to_cpu (esb_shift);
@@ -380,15 +378,6 @@ static void xive_native_update_pending(struct xive_cpu *xc) } } -static void xive_native_eoi (u32 hw_irq) -{ -/* -* Not normally used except if specific interrupts need -* a workaround on EOI. -*/ -opal_int_eoi (hw_irq); -} -static void xive_native_setup_cpu (unsigned int cpu,struct xive_cpu *xc) {s64 rc;
@@ -471,7 +460,6 @@ static const struct xive_ops xive_native_ops = { .match = xive_native_match, .shutdown = xive_native_shutdown, .update_pending = xive_native_update_pending, - .eoi = xive_native_eoi, .setup_cpu = xive_native_setup_cpu, .teardown_cpu = xive_native_teardown_cpu, .sync_source = xive_native_sync_source,
arch/powerpc/sysdev/xive/spapr.c +0 -6
@@ -628,11 +628,6 @@ static void xive_spapr_update_pending(struct xive_cpu *xc) } } -static void xive_spapr_eoi (u32 hw_irq) -{ -/* Not used */ ; -} -static void xive_spapr_setup_cpu (unsigned int cpu,struct xive_cpu *xc) {/* Only some debug on the TIMA settings */
@@ -677,7 +672,6 @@ static const struct xive_ops xive_spapr_ops = { .match = xive_spapr_match, .shutdown = xive_spapr_shutdown, .update_pending = xive_spapr_update_pending, - .eoi = xive_spapr_eoi, .setup_cpu = xive_spapr_setup_cpu, .teardown_cpu = xive_spapr_teardown_cpu, .sync_source = xive_spapr_sync_source,
Signature · 1 line
2.26.2
PATCH v2 11/13 powerpc/xive: Simplify xive_do_source_eoi()
In reply to Cédric Le GoaterPrevious patches removed the need of the first argument which was a hack for Firwmware EOI. Remove it and flatten the routine which has became simpler.
--- arch/powerpc/sysdev/xive/common.c | 72 ++++++++++++++----------------- 1 file changed, 33 insertions(+), 39 deletions(-)
arch/powerpc/sysdev/xive/common.c +33 -39
@@ -348,39 +348,40 @@ static void xive_do_queue_eoi(struct xive_cpu *xc)* EOI an interrupt at the source. There are several methods * to do this depending on the HW version and source type */ -static void xive_do_source_eoi (u32 hw_irq,struct xive_irq_data *xd) +static void xive_do_source_eoi (struct xive_irq_data *xd) { +u8 eoi_val; + xd->stale_p = false; +/* If the XIVE supports the new "store EOI facility, use it */ -if (xd->flags &XIVE_IRQ_FLAG_STORE_EOI ) +if (xd->flags &XIVE_IRQ_FLAG_STORE_EOI ) {xive_esb_write (xd,XIVE_ESB_STORE_EOI ,0 ); -else { -u8 eoi_val; +return ; + } -/* -* Otherwise for EOI, we use the special MMIO that does -* a clear of both P and Q and returns the old Q, -* except for LSIs where we use the "EOI cycle" special -* load. -* -* This allows us to then do a re-trigger if Q was set -* rather than synthesizing an interrupt in software -* -* For LSIs the HW EOI cycle is used rather than PQ bits, -* as they are automatically re-triggred in HW when still -* pending. -*/ -if (xd->flags &XIVE_IRQ_FLAG_LSI ) -xive_esb_read (xd,XIVE_ESB_LOAD_EOI ); -else { - eoi_val =xive_esb_read (xd,XIVE_ESB_SET_PQ_00 ); -DBG_VERBOSE ("eoi_val=%x\n" , eoi_val); - -/* Re-trigger if needed */ -if ((eoi_val &XIVE_ESB_VAL_Q ) && xd->trig_mmio) -out_be64 (xd->trig_mmio,0 ); - } +/* +* For LSIs, we use the "EOI cycle" special load rather than +* PQ bits, as they are automatically re-triggered in HW when +* still pending. +*/ +if (xd->flags &XIVE_IRQ_FLAG_LSI ) { +xive_esb_read (xd,XIVE_ESB_LOAD_EOI ); +return ; } + +/* +* Otherwise, we use the special MMIO that does a clear of +* both P and Q and returns the old Q. This allows us to then +* do a re-trigger if Q was set rather than synthesizing an +* interrupt in software +*/ + eoi_val =xive_esb_read (xd,XIVE_ESB_SET_PQ_00 ); +DBG_VERBOSE ("eoi_val=%x\n" , eoi_val); + +/* Re-trigger if needed */ +if ((eoi_val &XIVE_ESB_VAL_Q ) && xd->trig_mmio) +out_be64 (xd->trig_mmio,0 ); }/* irq_chip eoi callback, called with irq descriptor lock held */
@@ -398,7 +399,7 @@ static void xive_irq_eoi(struct irq_data *d)*/ if (!irqd_irq_disabled (d) && !irqd_is_forwarded_to_vcpu (d) && !(xd->flags &XIVE_IRQ_FLAG_NO_EOI )) -xive_do_source_eoi (irqd_to_hwirq (d), xd); +xive_do_source_eoi (xd);else xd->stale_p = true;
@@ -788,14 +789,7 @@ static int xive_irq_retrigger(struct irq_data *d)* 11, then perform an EOI. */ xive_esb_read (xd,XIVE_ESB_SET_PQ_11 ); - -/* -* Note: We pass "0" to the hw_irq argument in order to -* avoid calling into the backend EOI code which we don't -* want to do in the case of a re-trigger. Backends typically -* only do EOI for LSIs anyway. -*/ -xive_do_source_eoi (0 , xd); +xive_do_source_eoi (xd);return 1 ; }
@@ -910,7 +904,7 @@ static int xive_irq_set_vcpu_affinity(struct irq_data *d, void *state)* while masked, the generic code will re-mask it anyway. */ if (!xd->saved_p) -xive_do_source_eoi (hw_irq, xd); +xive_do_source_eoi (xd); }return 0 ;
@@ -1054,7 +1048,7 @@ static void xive_ipi_eoi(struct irq_data *d)DBG_VERBOSE ("IPI eoi: irq=%d [0x%lx] (HW IRQ 0x%x) pending=%02x\n" , d->irq,irqd_to_hwirq (d), xc->hw_ipi, xc->pending_prio); -xive_do_source_eoi (xc->hw_ipi, &xc->ipi_data); +xive_do_source_eoi (&xc->ipi_data);xive_do_queue_eoi (xc); }
@@ -1440,7 +1434,7 @@ static void xive_flush_cpu_queue(unsigned int cpu, struct xive_cpu *xc)* still asserted. Otherwise do an MSI retrigger. */ if (xd->flags &XIVE_IRQ_FLAG_LSI ) -xive_do_source_eoi (irqd_to_hwirq (d), xd); +xive_do_source_eoi (xd);else xive_irq_retrigger (d);
Signature · 1 line
2.26.2
PATCH v2 12/13 powerpc/xive: Improve error reporting of OPAL calls
In reply to Cédric Le GoaterIntroduce a vp_err() macro to standardize error reporting.
--- arch/powerpc/sysdev/xive/native.c | 28 ++++++++++++++++------------ 1 file changed, 16 insertions(+), 12 deletions(-)
arch/powerpc/sysdev/xive/native.c +16 -12
@@ -122,6 +122,8 @@ static int xive_native_get_irq_config(u32 hw_irq, u32 *target, u8 *prio,return rc ==0 ?0 : -ENXIO ; } +#define vp_err (vp, fmt, ...) pr_err("VP[0x%x]: " fmt, vp, ##__VA_ARGS__) +/* This can be called multiple time to change a queue configuration */ int xive_native_configure_queue (u32 vp_id,struct xive_q *q,u8 prio,__be32 *qpage,u32 order,bool can_escalate)
@@ -149,7 +151,7 @@ int xive_native_configure_queue(u32 vp_id, struct xive_q *q, u8 prio, &esc_irq_be, NULL);if (rc) { -pr_err ("Error %lld getting queue info prio %d\n" , rc, prio); +vp_err (vp_id,"Failed to get queue %d info : %lld\n" , prio, rc); rc = -EIO ; goto fail; }
@@ -172,7 +174,7 @@ int xive_native_configure_queue(u32 vp_id, struct xive_q *q, u8 prio,msleep (OPAL_BUSY_DELAY_MS ); }if (rc) { -pr_err ("Error %lld setting queue for prio %d\n" , rc, prio); +vp_err (vp_id,"Failed to set queue %d info: %lld\n" , prio, rc); rc = -EIO ; }else {/*
@@ -199,7 +201,7 @@ static void __xive_native_disable_queue(u32 vp_id, struct xive_q *q, u8 prio)msleep (OPAL_BUSY_DELAY_MS ); }if (rc) -pr_err ("Error %lld disabling queue for prio %d\n" , rc, prio); +vp_err (vp_id,"Failed to disable queue %d : %lld\n" , prio, rc); }void xive_native_disable_queue (u32 vp_id,struct xive_q *q,u8 prio)
@@ -698,6 +700,8 @@ int xive_native_enable_vp(u32 vp_id, bool single_escalation)break ;msleep (OPAL_BUSY_DELAY_MS ); } +if (rc) +vp_err (vp_id,"Failed to enable VP : %lld\n" , rc);return rc ? -EIO :0 ; }EXPORT_SYMBOL_GPL (xive_native_enable_vp);
@@ -712,6 +716,8 @@ int xive_native_disable_vp(u32 vp_id)break ;msleep (OPAL_BUSY_DELAY_MS ); } +if (rc) +vp_err (vp_id,"Failed to disable VP : %lld\n" , rc);return rc ? -EIO :0 ; }EXPORT_SYMBOL_GPL (xive_native_disable_vp);
@@ -723,8 +729,10 @@ int xive_native_get_vp_info(u32 vp_id, u32 *out_cam_id, u32 *out_chip_id)s64 rc; rc =opal_xive_get_vp_info (vp_id,NULL , &vp_cam_be,NULL , &vp_chip_id_be); -if (rc) +if (rc) { +vp_err (vp_id,"Failed to get VP info : %lld\n" , rc);return -EIO ; + } *out_cam_id =be64_to_cpu (vp_cam_be) &0xffffffffu ; *out_chip_id =be32_to_cpu (vp_chip_id_be);
@@ -755,8 +763,7 @@ int xive_native_get_queue_info(u32 vp_id, u32 prio, rc =opal_xive_get_queue_info (vp_id, prio, &qpage, &qsize, &qeoi_page, &escalate_irq, &qflags);if (rc) { -pr_err ("OPAL failed to get queue info for VCPU %d/%d : %lld\n" , - vp_id, prio, rc); +vp_err (vp_id,"failed to get queue %d info : %lld\n" , prio, rc);return -EIO ; }
@@ -784,8 +791,7 @@ int xive_native_get_queue_state(u32 vp_id, u32 prio, u32 *qtoggle, u32 *qindex) rc =opal_xive_get_queue_state (vp_id, prio, &opal_qtoggle, &opal_qindex);if (rc) { -pr_err ("OPAL failed to get queue state for VCPU %d/%d : %lld\n" , - vp_id, prio, rc); +vp_err (vp_id,"failed to get queue %d state : %lld\n" , prio, rc);return -EIO ; }
@@ -804,8 +810,7 @@ int xive_native_set_queue_state(u32 vp_id, u32 prio, u32 qtoggle, u32 qindex) rc =opal_xive_set_queue_state (vp_id, prio, qtoggle, qindex);if (rc) { -pr_err ("OPAL failed to set queue state for VCPU %d/%d : %lld\n" , - vp_id, prio, rc); +vp_err (vp_id,"failed to set queue %d state : %lld\n" , prio, rc);return -EIO ; }
@@ -827,8 +832,7 @@ int xive_native_get_vp_state(u32 vp_id, u64 *out_state) rc =opal_xive_get_vp_state (vp_id, &state);if (rc) { -pr_err ("OPAL failed to get vp state for VCPU %d : %lld\n" , - vp_id, rc); +vp_err (vp_id,"failed to get vp state : %lld\n" , rc);return -EIO ; }
Signature · 1 line
2.26.2
PATCH v2 13/13 KVM: PPC: Book3S HV: XIVE: Add a comment regarding VP numbering
In reply to Cédric Le GoaterWhen the XIVE resources are allocated at the HW level, the VP structures describing the vCPUs of a guest are distributed among the chips to optimize the PowerBUS usage. For best performance, the guest vCPUs can be pinned to match the VP structure distribution.
Currently, the VP identifiers are deduced from the vCPU id using the kvmppc_pack_vcpu_id() routine which is not incorrect but not optimal either. It VSMT is used, the result is not continuous and the constraints on HW resources described above can not be met.
--- arch/powerpc/kvm/book3s_xive.h | 11 +++++++++++ 1 file changed, 11 insertions(+)
arch/powerpc/kvm/book3s_xive.h +11 -0
@@ -218,6 +218,17 @@ static inline struct kvmppc_xive_src_block *kvmppc_xive_find_source(struct kvmppreturn xive->src_blocks[bid]; } +/* +* When the XIVE resources are allocated at the HW level, the VP +* structures describing the vCPUs of a guest are distributed among +* the chips to optimize the PowerBUS usage. For best performance, the +* guest vCPUs can be pinned to match the VP structure distribution. +* +* Currently, the VP identifiers are deduced from the vCPU id using +* the kvmppc_pack_vcpu_id() routine which is not incorrect but not +* optimal either. It VSMT is used, the result is not continuous and +* the constraints on HW resources described above can not be met. +*/ static inline u32 kvmppc_xive_vp (struct kvmppc_xive *xive,u32 server) {return xive->vp_base +kvmppc_pack_vcpu_id (xive->kvm, server);
Signature · 1 line
2.26.2
Quoted 9 lines · Cédric Le Goater
The most important change is the removal of support of OPAL flags required for P9 DD1. It provides a good cleanup of some complex routines.
Thanks,
C.
[...]
Patches 1-3 and 5-13 applied to powerpc/next.
[01/13] KVM: PPC: Book3S HV: XIVE: Show detailed configuration in debug output
https://git.kernel.org/powerpc/c/98983675008ab3ae9b37fc7a4bfa083998079215 [02/13] powerpc/xive: Rename XIVE_IRQ_NO_EOI to show its a flag https://git.kernel.org/powerpc/c/4f1c3f7b08187e6b97701c7fb2dc6f3749566c62 [03/13] powerpc/xive: Introduce XIVE_IPI_HW_IRQ https://git.kernel.org/powerpc/c/e2cf43d59525477cfd030378c3c808187952c531 [05/13] powerpc/xive: Add a name to the IRQ domain https://git.kernel.org/powerpc/c/9dfe4b14df93532da3dbf11952a17389ae3cdc67 [06/13] powerpc/xive: Add a debug_show handler to the XIVE irq_domain https://git.kernel.org/powerpc/c/a5021abc48a0f44083a15a37b3e61378519cb00d [07/13] powerpc: Increase NR_IRQS range to support more KVM guests https://git.kernel.org/powerpc/c/7b3b3de3b04ecb7393cdfaa30a3468dd47b750cf [08/13] powerpc/xive: Remove P9 DD1 flag XIVE_IRQ_FLAG_SHIFT_BUG https://git.kernel.org/powerpc/c/4cc0e36df2c0a41fd38645ddde08d2bfba699b7a [09/13] powerpc/xive: Remove P9 DD1 flag XIVE_IRQ_FLAG_MASK_FW https://git.kernel.org/powerpc/c/b5277d18c65e31ce51f6733ebdca3985a962fab5 [10/13] powerpc/xive: Remove P9 DD1 flag XIVE_IRQ_FLAG_EOI_FW https://git.kernel.org/powerpc/c/cf58b746665d0177b86d42d18e60985fa1fdb909 [11/13] powerpc/xive: Simplify xive_do_source_eoi() https://git.kernel.org/powerpc/c/614546d56296380b59e94484813eeef62a7d2b6b [12/13] powerpc/xive: Improve error reporting of OPAL calls https://git.kernel.org/powerpc/c/07efbca11c1a985efa4d15bd76a637c6bffc253b [13/13] KVM: PPC: Book3S HV: XIVE: Add a comment regarding VP numbering https://git.kernel.org/powerpc/c/dddc4ef92d1ce92987da1d6926cdfa99e8acb622
cheers