ARM: VGIC/GIC separation cleanups
v4 cover ARM: VGIC/GIC separation cleanups
Hi,
an updated version, focussing on the complaints Julien raised about the locking on assigned hardware IRQs. Patch 5/8 has been changed to address his concerns.
I dropped the tag on this patch, also on 2/8 since I moved the per-CPU variable declaration into gic.h, as requested. Not sure if that is what people wanted, so please have a look.
The rest of that patch is unchanged, and I checked that the code is still correct even after the rebase to staging.
Please test and apply.
Cheers,
Andre
Changelov v3->v4
01: added tags
02: remove extra header files, move per-CPU variable into gic.h 03, 04: unchanged
05: rework locking, rework dropping hardware IRQ connection 06: add ASSERT
07: unchanged
08: removed read_mostly attribute
Changelog v2->v3:
01: reworked
02: rebased to match staging, added tag
03, 07: added tag
08: new, as suggested by Julien
================
By the original VGIC design, Xen differentiates between the actual VGIC emulation on one hand and the GIC hardware accesses on the other. It seems there were some deviations from that scheme (over time?), so at the moment we end up happily accessing VGIC specific data structures like struct pending_irq and struct vgic_irq_rank from pure GIC files like gic.c or even irq.c (try: git grep -l struct\ pending_irq xen/arch/arm). But any future VGIC rework will depend on a clean separation, so this series tries to clean this up.
After this series there are no more references to VGIC structures from GIC files, at least for non-ITS code. The ITS is a beast own its own (blame the author) and will be addressed later.
Andre Przywara (8):
ARM: VGIC: drop unneeded gic_restore_pending_irqs()
ARM: VGIC: split gic.c to observe hardware/virtual GIC separation
ARM: VGIC: split up gic_dump_info() to cover virtual part separately
ARM: VGIC: rework events_need_delivery()
ARM: VGIC: factor out vgic_connect_hw_irq()
ARM: VGIC: factor out vgic_get_hw_irq_desc()
ARM: VGIC: rework gicv[23]_update_lr to not use pending_irq
ARM: make nr_irqs a constant
xen/arch/arm/Makefile | 1 + xen/arch/arm/domain.c | 1 + xen/arch/arm/gic-v2.c | 14 +- xen/arch/arm/gic-v3.c | 12 +- xen/arch/arm/gic-vgic.c | 461 ++++++++++++++++++++++++++++++++++++++++++++ xen/arch/arm/gic.c | 423 ++-------------------------------------- xen/arch/arm/irq.c | 9 +- xen/arch/arm/vgic.c | 11 ++ xen/include/asm-arm/event.h | 13 +- xen/include/asm-arm/gic.h | 8 +- xen/include/asm-arm/irq.h | 5 +- xen/include/asm-arm/vgic.h | 6 + 12 files changed, 521 insertions(+), 443 deletions(-) create mode 100644 xen/arch/arm/gic-vgic.c
Signature · 7 lines
2.14.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v4 1/8 ARM: VGIC: drop unneeded gic_restore_pending_irqs()
In gic_restore_pending_irqs() we push our pending virtual IRQs into the list registers. This function is called once from gic_inject(), just before we return to the guest, but also in gic_restore_state(), when we context-switch a VCPU. Having a closer look it turns out that the later call is not needed, since we will always call gic_inject() anyway. So remove that call (and the forward declaration) to streamline this interface and make separating the GIC from the VGIC world later.
--- xen/arch/arm/gic.c | 4 ---- 1 file changed, 4 deletions(-)
xen/arch/arm/gic.c +0 -4
@@ -36,8 +36,6 @@#include <asm/vgic.h> #include <asm/acpi.h> -static void gic_restore_pending_irqs (struct vcpu *v); - static DEFINE_PER_CPU(uint64_t, lr_mask);#define lr_all_full () (this_cpu(lr_mask) == ((1 << gic_hw_ops->info->nr_lrs) - 1))
@@ -91,8 +89,6 @@ void gic_restore_state(struct vcpu *v) gic_hw_ops->restore_state (v);isb (); - -gic_restore_pending_irqs (v); }/* desc->irq needs to be disabled before calling this function */
Signature · 7 lines
2.14.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v4 2/8 ARM: VGIC: split gic.c to observe hardware/virtual GIC separation
In reply to Andre PrzywaraCurrently gic.c holds code to handle hardware IRQs as well as code to bridge VGIC requests to the GIC virtualization hardware. Despite being named gic.c, this file reaches into the VGIC and uses data structures describing virtual IRQs.
To improve abstraction, move the VGIC functions into a separate file, so that gic.c does what it says on the tin.
--- xen/arch/arm/Makefile | 1 + xen/arch/arm/gic-vgic.c | 396 ++++++++++++++++++++++++++++++++++++++++++++++ xen/arch/arm/gic.c | 363 +----------------------------------------- xen/include/asm-arm/gic.h | 3 + 4 files changed, 402 insertions(+), 361 deletions(-) create mode 100644 xen/arch/arm/gic-vgic.c
xen/arch/arm/Makefile +1 -0
@@ -16,6 +16,7 @@ obj-y += domain_build.o obj-y += domctl.o obj-$(EARLY_PRINTK) += early_printk.o obj-y += gic.o +obj-y += gic-vgic.o obj-y += gic-v2.o obj-$(CONFIG_HAS_GICV3) += gic-v3.o obj-$(CONFIG_HAS_ITS) += gic-v3-its.o
xen/arch/arm/gic-vgic.c new file +396 -0
396 lines not on this page · raw message
xen/arch/arm/gic.c +2 -361
378 lines not on this page · raw message
xen/include/asm-arm/gic.h +3 -0
@@ -220,6 +220,8 @@ enum gic_version { GIC_V3, }; +DECLARE_PER_CPU (uint64_t, lr_mask); +extern enum gic_version gic_hw_version (void );/* Program the IRQ type into the GIC */
@@ -372,6 +374,7 @@ struct gic_hw_operations {void (*do_LPI)(unsigned int lpi); }; +extern const struct gic_hw_operations *gic_hw_ops;void register_gic_ops (const struct gic_hw_operations *ops); int gic_make_hwdom_dt_node(const struct domain *d, const struct dt_device_node *gic,
Signature · 7 lines
2.14.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v4 2/8] ARM: VGIC: split gic.c to observe hardware/virtual GIC separation
Hi,
Quoted 19 lines · Andre Przywara
xen/arch/arm/gic-vgic.c new file +13
-0@@ -0,0 +1,396 @@ +/* +* xen/arch/arm/gic-vgic.c +* +* ARM Generic Interrupt Controller virtualization support +* +* Tim Deegan <[email protected]> +* Copyright (c) 2011 Citrix Systems. +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. +*
xen/arch/arm/gic-vgic.c line 23
Quoted 10 lines
xen/arch/arm/gic-vgic.c new file +10
-0+* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +*/ + +#include <xen/lib.h> +#include <xen/irq.h> +#include <xen/sched.h> +#include <xen/errno.h>
Please order them alphabetically. With that:
Cheers,
Signature · 6 lines
Julien Grall _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v4 3/8 ARM: VGIC: split up gic_dump_info() to cover virtual part separately
In reply to Andre PrzywaraCurrently gic_dump_info() not only dumps the hardware state of the GIC, but also the VGIC internal virtual IRQ lists.
Split the latter off and move it into gic-vgic.c to observe the abstraction.
--- xen/arch/arm/domain.c | 1 + xen/arch/arm/gic-vgic.c | 11 +++++++++++ xen/arch/arm/gic.c | 12 ------------ xen/include/asm-arm/gic.h | 1 + 4 files changed, 13 insertions(+), 12 deletions(-)
xen/arch/arm/domain.c +1 -0
@@ -941,6 +941,7 @@ long arch_do_vcpu_op(int cmd, struct vcpu *v, XEN_GUEST_HANDLE_PARAM(void) arg)void arch_dump_vcpu_info (struct vcpu *v) {gic_dump_info (v); +gic_dump_vgic_info (v); }void vcpu_mark_events_pending (struct vcpu *v)
xen/arch/arm/gic-vgic.c +11 -0
@@ -386,6 +386,17 @@ void gic_inject(void) gic_hw_ops->update_hcr_status (GICH_HCR_UIE , true); } +void gic_dump_vgic_info (struct vcpu *v) +{ +struct pending_irq *p; + + list_for_each_entry ( p, &v->arch.vgic.inflight_irqs, inflight ) +printk ("Inflight irq=%u lr=%u\n" , p->irq, p->lr); + +list_for_each_entry ( p, &v->arch.vgic.lr_pending, lr_queue ) +printk ("Pending irq=%d\n" , p->irq); +} +/* * Local variables: * mode: C
xen/arch/arm/gic.c +0 -12
@@ -443,20 +443,8 @@ static void maintenance_interrupt(int irq, void *dev_id, struct cpu_user_regs *rvoid gic_dump_info (struct vcpu *v) { -struct pending_irq *p; -printk ("GICH_LRs (vcpu %d) mask=%" PRIx64"\n" , v->vcpu_id, v->arch.lr_mask); gic_hw_ops->dump_state (v); - - list_for_each_entry ( p, &v->arch.vgic.inflight_irqs, inflight ) - { -printk ("Inflight irq=%u lr=%u\n" , p->irq, p->lr); - } - -list_for_each_entry ( p, &v->arch.vgic.lr_pending, lr_queue ) - { -printk ("Pending irq=%d\n" , p->irq); - } }void init_maintenance_interrupt (void )
xen/include/asm-arm/gic.h +1 -0
@@ -287,6 +287,7 @@ extern void send_SGI_allbutself(enum gic_sgi sgi);/* print useful debug info */ extern void gic_dump_info (struct vcpu *v); +extern void gic_dump_vgic_info (struct vcpu *v);/* Number of interrupt lines */ extern unsigned int gic_number_lines (void );
Signature · 7 lines
2.14.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v4 4/8 ARM: VGIC: rework events_need_delivery()
In reply to Andre PrzywaraIn event.h we very deeply dive into the VGIC to learn if an event for a guest is pending.
Rework that function to abstract the VGIC specific part out. Also reorder the queries there, as we only actually need to check for the event channel if there are no other pending IRQs.
--- xen/arch/arm/vgic.c | 11 +++++++++++ xen/include/asm-arm/event.h | 13 +++---------- xen/include/asm-arm/vgic.h | 2 ++ 3 files changed, 16 insertions(+), 10 deletions(-)
xen/arch/arm/vgic.c +11 -0
@@ -593,6 +593,17 @@ void arch_evtchn_inject(struct vcpu *v)vgic_vcpu_inject_irq (v, v->domain->arch.evtchn_irq); } +bool vgic_evtchn_irq_pending (struct vcpu *v) +{ +struct pending_irq *p; + + p =irq_to_pending (v, v->domain->arch.evtchn_irq); +/* Does not work for LPIs. */ +ASSERT (!is_lpi (v->domain->arch.evtchn_irq)); + +return list_empty (&p->inflight); +} +bool vgic_emulate (struct cpu_user_regs *regs,union hsr hsr) {struct vcpu *v = current;
xen/include/asm-arm/event.h +3 -10
@@ -16,12 +16,6 @@ static inline int vcpu_event_delivery_is_enabled(struct vcpu *v)static inline int local_events_need_delivery_nomask (void ) { -struct pending_irq *p =irq_to_pending (current, - current->domain->arch.evtchn_irq); - -/* Does not work for LPIs. */ -ASSERT (!is_lpi (current->domain->arch.evtchn_irq)); -/* XXX: if the first interrupt has already been delivered, we should * check whether any other interrupts with priority higher than the * one in GICV_IAR are in the lr_pending queue or in the LR
@@ -33,11 +27,10 @@ static inline int local_events_need_delivery_nomask(void)if (gic_events_need_delivery () )return 1 ; -if (vcpu_info (current, evtchn_upcall_pending) && -list_empty (&p->inflight) ) -return 1 ; +if ( !vcpu_info (current, evtchn_upcall_pending) ) +return 0 ; -return 0 ; +return vgic_evtchn_irq_pending (current); }static inline int local_events_need_delivery (void )
xen/include/asm-arm/vgic.h +2 -0
@@ -218,6 +218,8 @@ extern void register_vgic_ops(struct domain *d, const struct vgic_ops *ops);int vgic_v2_init (struct domain *d,int *mmio_count);int vgic_v3_init (struct domain *d,int *mmio_count); +bool vgic_evtchn_irq_pending (struct vcpu *v); +extern int domain_vgic_register (struct domain *d,int *mmio_count);extern int vcpu_vgic_free (struct vcpu *v); extern bool vgic_to_sgi(struct vcpu *v, register_t sgir,
Signature · 7 lines
2.14.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v4 5/8 ARM: VGIC: factor out vgic_connect_hw_irq()
In reply to Andre PrzywaraAt the moment we happily access VGIC internal data structures like the rank and struct pending_irq in gic.c, which should be VGIC agnostic.
Factor out a new function vgic_connect_hw_irq(), which allows a virtual IRQ to be connected to a hardware IRQ (using the hw bit in the LR).
This removes said accesses to VGIC data structures and improves abstraction.
--- xen/arch/arm/gic-vgic.c | 36 ++++++++++++++++++++++++++++++++++++ xen/arch/arm/gic.c | 44 ++++++++++---------------------------------- xen/include/asm-arm/vgic.h | 2 ++ 3 files changed, 48 insertions(+), 34 deletions(-)
xen/arch/arm/gic-vgic.c +36 -0
@@ -397,6 +397,42 @@ void gic_dump_vgic_info(struct vcpu *v)printk ("Pending irq=%d\n" , p->irq); } +int vgic_connect_hw_irq (struct domain *d,struct vcpu *v,unsigned int virq, +struct irq_desc *desc,bool connect) +{ +unsigned long flags; +/* Use vcpu0 to retrieve the pending_irq struct. Given that we only +* route SPIs to guests, it doesn't make any difference. */ +struct vcpu *v_target =vgic_get_target_vcpu (d->vcpu[0 ], virq); +struct vgic_irq_rank *rank =vgic_rank_irq (v_target, virq); +struct pending_irq *p =irq_to_pending (v_target, virq); +int ret =0 ; + +ASSERT (connect && desc); + +/* We are taking to rank lock to prevent parallel connections. */ +vgic_lock_rank (v_target, rank, flags); + +if ( connect ) + { +/* The VIRQ should not be already enabled by the guest */ +if ( !p->desc && + !test_bit (GIC_IRQ_GUEST_ENABLED , &p->status) ) + p->desc = desc; +else + ret = -EBUSY ; + } +else + { +if ( !desc || p->desc == desc ) + p->desc =NULL ; + } + +vgic_unlock_rank (v_target, rank, flags); + +return ret; +} +/* * Local variables: * mode: C
xen/arch/arm/gic.c +10 -34
85 lines not on this page · raw message
xen/include/asm-arm/vgic.h +2 -0
@@ -219,6 +219,8 @@ int vgic_v2_init(struct domain *d, int *mmio_count);int vgic_v3_init (struct domain *d,int *mmio_count);bool vgic_evtchn_irq_pending (struct vcpu *v); +int vgic_connect_hw_irq (struct domain *d,struct vcpu *v,unsigned int virq, +struct irq_desc *desc,bool connect);extern int domain_vgic_register (struct domain *d,int *mmio_count);extern int vcpu_vgic_free (struct vcpu *v);
Signature · 7 lines
2.14.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v4 5/8] ARM: VGIC: factor out vgic_connect_hw_irq()
Hi Andre,
Quoted 7 lines · Andre Przywara
At the moment we happily access VGIC internal data structures like the rank and struct
pending_irqingic.c, which should be VGIC agnostic.Factor out a new function
vgic_connect_hw_irq(), which allows a virtual IRQ to be connected to a hardware IRQ (using the hw bit in the LR).This removes said accesses to VGIC data structures and improves abstraction.
I was expecting some explanation regarding the new locking order in the commit message.
Quoted 13 lines
Signed-off-by Andre Przywara <[email protected]>--- xen/arch/arm/gic-vgic.c | 36 ++++++++++++++++++++++++++++++++++++ xen/arch/arm/gic.c | 44 ++++++++++---------------------------------- xen/include/asm-arm/vgic.h | 2 ++ 3 files changed, 48 insertions(+), 34 deletions(-)xen/arch/arm/gic-vgic.c +0
-0
xen/arch/arm/gic-vgic.c void gic_dump_vgic_info(struct vcpu *v) · line 405
Quoted 9 lines
xen/arch/arm/gic-vgic.c +6
-0@@ -397,6 +397,42 @@ void gic_dump_vgic_info(struct vcpu *v)printk ("Pending irq=%d\n" , p->irq); } +int vgic_connect_hw_irq (struct domain *d,struct vcpu *v,unsigned int virq, +struct irq_desc *desc,bool connect) +{ +unsigned long flags; +/* Use vcpu0 to retrieve the pending_irq struct. Given that we only +* route SPIs to guests, it doesn't make any difference. */
Please fix the coding style as requested in v3.
xen/arch/arm/gic-vgic.c void gic_dump_vgic_info(struct vcpu *v) · line 411
Quoted 6 lines
xen/arch/arm/gic-vgic.c +6
-0+struct vcpu *v_target =vgic_get_target_vcpu (d->vcpu[0 ], virq); +struct vgic_irq_rank *rank =vgic_rank_irq (v_target, virq); +struct pending_irq *p =irq_to_pending (v_target, virq); +int ret =0 ; + +ASSERT (connect && desc);
I am not sure why you allow desc to be non-NULL when disconnecting it (see more below).
Quoted 6 lines
xen/arch/arm/gic-vgic.c +6
-0+ +/* We are taking to rank lock to prevent parallel connections. */ +vgic_lock_rank (v_target, rank, flags); + +if ( connect ) + {
xen/arch/arm/gic-vgic.c void gic_dump_vgic_info(struct vcpu *v) · line 427
Quoted 10 lines
xen/arch/arm/gic-vgic.c +10
-0+/* The VIRQ should not be already enabled by the guest */ +if ( !p->desc && + !test_bit (GIC_IRQ_GUEST_ENABLED , &p->status) ) + p->desc = desc; +else + ret = -EBUSY ; + } +else + { +if ( !desc || p->desc == desc )
From a quick glance, no caller will have desc is NULL. Even if it was, it will not harm because p->desc will be set to NULL.
xen/arch/arm/gic-vgic.c void gic_dump_vgic_info(struct vcpu *v) · line 429
Quoted 2 lines
xen/arch/arm/gic-vgic.c +2
-0+ p->desc =NULL ; + }
But likely you want to return an error if p->desc != desc as this is a user input error. Ignoring it is a pretty bad.
xen/arch/arm/gic-vgic.c void gic_dump_vgic_info(struct vcpu *v) · line 438
Quoted 9 lines
xen/arch/arm/gic-vgic.c +6
-0+ +vgic_unlock_rank (v_target, rank, flags); + +return ret; +} +/* * Local variables: * mode: C
Cheers,
Signature · 6 lines
Julien Grall _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v4 5/8] ARM: VGIC: factor out vgic_connect_hw_irq()
Quoted 6 lines · Julien Grall
Hi Andre,
Quoted 3 lines · Andre Przywara
At the moment we happily access VGIC internal data structures like the rank and struct
pending_irqingic.c, which should be VGIC agnostic.
Quoted 6 lines
Quoted 4 lines
Factor out a new function
vgic_connect_hw_irq(), which allows a virtual IRQ to be connected to a hardware IRQ (using the hw bit in the LR).This removes said accesses to VGIC data structures and improves abstraction.
I was expecting some explanation regarding the new locking order in the commit message.
From a quick chat with Andre on IRC, it looks like figuring out the current locking might be tricky. So I would be happy if we only point out the change of locking (+ a word on the new locking) in the commit message.
This would help in case we find a regression later on.
Cheers,
Signature · 6 lines
Julien Grall _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v4 5/8] ARM: VGIC: factor out vgic_connect_hw_irq()
In reply to Julien GrallHi,
Quoted 6 lines · Julien Grall
Hi Andre,
Quoted 3 lines · Andre Przywara
At the moment we happily access VGIC internal data structures like the rank and struct
pending_irqingic.c, which should be VGIC agnostic.
Quoted 8 lines
Quoted 5 lines
Factor out a new function
vgic_connect_hw_irq(), which allows a virtual IRQ to be connected to a hardware IRQ (using the hw bit in the LR).This removes said accesses to VGIC data structures and improves abstraction.
I was expecting some explanation regarding the new locking order in the commit message.
Well, there is no real change in the new locking order. We enter both gic_route_irq_to_guest() and gic_remove_irq_from_guest() with the desc lock held, then take the rank lock at some point in time and drop it again. The only change is how long we hold the rank lock, but this should have no effect, since we don't manipulate any virtual IRQ properties (rank or not) in the code lines that are now no longer under the lock.
I can try to summarise this in the commit message.
Quoted 8 lines
Quoted 8 lines
Signed-off-by Andre Przywara <[email protected]>---
xen/arch/arm/gic-vgic.c | 36 ++++++++++++++++++++++++++++++++++++ xen/arch/arm/gic.c | 44
++++++++++----------------------------------
xen/include/asm-arm/vgic.h| 2 ++3 files changed, 48 insertions(+), 34 deletions(-)
Quoted 16 lines
Quoted 7 lines
xen/arch/arm/gic-vgic.c +0
-0@@ -397,6 +397,42 @@ void gic_dump_vgic_info(struct vcpu *v) printk("Pending irq=%d\n", p->irq); }Quoted 7 lines
+int vgic_connect_hw_irq(struct domain *d, struct vcpu *v, unsigned int virq,
+ structirq_desc*desc, bool connect) +{
+ unsigned long flags;
+ /* Use vcpu0 to retrieve thepending_irqstruct. Given that we only + * route SPIs to guests, it doesn't make any difference. */Please fix the coding style as requested in v3.
Ah, sorry, I forgot that over the rewrite.
Quoted 9 lines
Quoted 6 lines
+ struct vcpu
*v_target=vgic_get_target_vcpu(d->vcpu[0], virq); + structvgic_irq_rank*rank =vgic_rank_irq(v_target, virq); + structpending_irq*p =irq_to_pending(v_target, virq); + int ret = 0;
+
+ASSERT(connect && desc);I am not sure why you allow desc to be non-NULL when disconnecting it (see more below).
I consider passing the desc pointer in addition to the vIRQ number redundant in case we want to drop the association. The only reason we do this is to work around the somewhat opposite locking order. So this is a property of the existing code, which I consider at least somewhat dodgy.
In order to allow changing this in the future (for instance with the new VGIC), I introduced the "connect" parameter and decided to make "desc" optional in case "connect" is false.
If we give it anyway, we check for a match, which is what we do with the current code. So we keep this molly guard.
Quoted 10 lines
Quoted 10 lines
xen/arch/arm/gic-vgic.c +10
-0+ + /* We are taking to rank lock to prevent parallel connections. */ + vgic_lock_rank(v_target, rank, flags); + + if ( connect ) + { + /* The VIRQ should not be already enabled by the guest */ + if ( !p->desc && + !test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) ) + p->desc = desc;
Quoted 8 lines
Quoted 6 lines
+ else
+ ret =-EBUSY;
+ }
+ else
+ {
+ if ( !desc ||p->desc== desc )From a quick glance, no caller will have desc is NULL.
Yes, for now.
Quoted 7 lines
Even if it was,
it will not harm becausep->descwill be set to NULL.Quoted 2 lines
+
p->desc= NULL;
+ }But likely you want to return an error if
p->desc!= desc as this is a user input error. Ignoring it is a pretty bad.
Right, good point. Actually this is what I had in mind, but somehow managed to derail it. Will fix it.
Thanks for the review!
Andre.
Quoted 12 lines
Quoted 9 lines
xen/arch/arm/gic-vgic.c +6
-0+ + vgic_unlock_rank(v_target, rank, flags); + + return ret; +} +/*
* Local variables:
* mode: CCheers,
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v4 5/8] ARM: VGIC: factor out vgic_connect_hw_irq()
Quoted 18 lines · Andre Przywara
Hi,
Quoted 6 lines · Julien Grall
Hi Andre,
Quoted 3 lines · Andre Przywara
At the moment we happily access VGIC internal data structures like the rank and struct
pending_irqingic.c, which should be VGIC agnostic.Quoted 8 lines
Quoted 5 lines
Factor out a new function
vgic_connect_hw_irq(), which allows a virtual IRQ to be connected to a hardware IRQ (using the hw bit in the LR).This removes said accesses to VGIC data structures and improves abstraction.
I was expecting some explanation regarding the new locking order in the commit message.
Quoted 8 lines
Well, there is no real change in the new locking order. We enter both
gic_route_irq_to_guest()andgic_remove_irq_from_guest()with the desc lock held, then take the rank lock at some point in time and drop it again. The only change is how long we hold the rank lock, but this should have no effect, since we don't manipulate any virtual IRQ properties (rank or not) in the code lines that are now no longer under the lock.
I can try to summarise this in the commit message.
Sorry, somehow I wrote locking order when I was meant to say about how long you take the lock.
Something along what you say above looks good to me. What I want to avoid is running into a problem and wondering why the locking was modified.
Cheers,
Quoted 96 lines
Quoted 9 lines
Quoted 9 lines
Signed-off-by Andre Przywara <[email protected]>--- xen/arch/arm/gic-vgic.c | 36 ++++++++++++++++++++++++++++++++++++ xen/arch/arm/gic.c | 44 ++++++++++---------------------------------- xen/include/asm-arm/vgic.h | 2 ++ 3 files changed, 48 insertions(+), 34 deletions(-)Quoted 16 lines
Quoted 5 lines
xen/arch/arm/gic-vgic.c +0
-0xen/arch/arm/gic-vgic.c void gic_dump_vgic_info(struct vcpu *v) · line 405
Quoted 9 lines
xen/arch/arm/gic-vgic.c +5
-0@@ -397,6 +397,42 @@ void gic_dump_vgic_info(struct vcpu *v) printk("Pending irq=%d\n", p->irq); } +int vgic_connect_hw_irq(struct domain *d, struct vcpu *v, unsigned int virq, + struct irq_desc *desc, bool connect) +{ + unsigned long flags; + /* Use vcpu0 to retrieve the pending_irq struct. Given that we only +* route SPIs to guests, it doesn't make any difference. */ Please fix the coding style as requested in v3.
Ah, sorry, I forgot that over the rewrite.
Quoted 9 lines
xen/arch/arm/gic-vgic.c void gic_dump_vgic_info(struct vcpu *v) · line 411
Quoted 6 lines
xen/arch/arm/gic-vgic.c +6
-0+ struct vcpu *v_target = vgic_get_target_vcpu(d->vcpu[0], virq); + struct vgic_irq_rank *rank = vgic_rank_irq(v_target, virq); + struct pending_irq *p = irq_to_pending(v_target, virq); + int ret = 0; + + ASSERT(connect && desc);I am not sure why you allow desc to be non-NULL when disconnecting it (see more below).
I consider passing the desc pointer in addition to the vIRQ number redundant in case we want to drop the association. The only reason we do this is to work around the somewhat opposite locking order. So this is a property of the existing code, which I consider at least somewhat dodgy.
In order to allow changing this in the future (for instance with the new VGIC), I introduced the "connect" parameter and decided to make "desc" optional in case "connect" is false.
If we give it anyway, we check for a match, which is what we do with the current code. So we keep this molly guard.Quoted 10 lines
Quoted 10 lines
xen/arch/arm/gic-vgic.c +10
-0+ + /* We are taking to rank lock to prevent parallel connections. */ + vgic_lock_rank(v_target, rank, flags); + + if ( connect ) + { + /* The VIRQ should not be already enabled by the guest */ + if ( !p->desc && + !test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) ) + p->desc = desc;Quoted 8 lines
xen/arch/arm/gic-vgic.c void gic_dump_vgic_info(struct vcpu *v) · line 427
Quoted 6 lines
xen/arch/arm/gic-vgic.c +6
-0+ else + ret = -EBUSY; + } + else + { + if ( !desc || p->desc == desc )From a quick glance, no caller will have desc is NULL.
Yes, for now.
Quoted 7 lines
Even if it was,
it will not harm becausep->descwill be set to NULL.xen/arch/arm/gic-vgic.c void gic_dump_vgic_info(struct vcpu *v) · line 429
Quoted 2 lines
xen/arch/arm/gic-vgic.c +2
-0+ p->desc = NULL; + }But likely you want to return an error if
p->desc!= desc as this is a user input error. Ignoring it is a pretty bad.Right, good point. Actually this is what I had in mind, but somehow managed to derail it. Will fix it.
Thanks for the review!
Andre.Quoted 12 lines
xen/arch/arm/gic-vgic.c void gic_dump_vgic_info(struct vcpu *v) · line 438
Quoted 9 lines
xen/arch/arm/gic-vgic.c +6
-0+ + vgic_unlock_rank(v_target, rank, flags); + + return ret; +} + /** Local variables: * mode: C Cheers,
Signature · 6 lines
Julien Grall _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v4 6/8 ARM: VGIC: factor out vgic_get_hw_irq_desc()
In reply to Andre PrzywaraAt the moment we happily access the VGIC internal struct pending_irq (which describes a virtual IRQ) in irq.c.
Factor out the actually needed functionality to learn the associated hardware IRQ and move that into gic-vgic.c to improve abstraction.
--- xen/arch/arm/gic-vgic.c | 17 +++++++++++++++++ xen/arch/arm/irq.c | 7 ++----- xen/include/asm-arm/vgic.h | 2 ++ 3 files changed, 21 insertions(+), 5 deletions(-)
xen/arch/arm/gic-vgic.c +17 -0
@@ -397,6 +397,23 @@ void gic_dump_vgic_info(struct vcpu *v)printk ("Pending irq=%d\n" , p->irq); } +struct irq_desc *vgic_get_hw_irq_desc (struct domain *d,struct vcpu *v, +unsigned int virq) +{ +struct pending_irq *p; + +ASSERT (!v && virq >=32 ); + +if ( !v ) + v = d->vcpu[0 ]; + + p =irq_to_pending (v, virq); +if ( !p ) +return NULL ; + +return p->desc; +} +int vgic_connect_hw_irq (struct domain *d,struct vcpu *v,unsigned int virq,struct irq_desc *desc,bool connect) {
xen/arch/arm/irq.c +2 -5
@@ -534,19 +534,16 @@ int release_guest_irq(struct domain *d, unsigned int virq)struct irq_desc *desc;struct irq_guest *info;unsigned long flags; -struct pending_irq *p;int ret;/* Only SPIs are supported */ if ( virq <NR_LOCAL_IRQS || virq >=vgic_num_irqs (d) )return -EINVAL ; - p =spi_to_pending (d, virq); -if ( !p->desc ) + desc =vgic_get_hw_irq_desc (d,NULL , virq); +if ( !desc )return -EINVAL ; - desc = p->desc; -spin_lock_irqsave (&desc->lock, flags); ret = -EINVAL ;
xen/include/asm-arm/vgic.h +2 -0
@@ -219,6 +219,8 @@ int vgic_v2_init(struct domain *d, int *mmio_count);int vgic_v3_init (struct domain *d,int *mmio_count);bool vgic_evtchn_irq_pending (struct vcpu *v); +struct irq_desc *vgic_get_hw_irq_desc (struct domain *d,struct vcpu *v, +unsigned int virq);int vgic_connect_hw_irq (struct domain *d,struct vcpu *v,unsigned int virq,struct irq_desc *desc,bool connect);
Signature · 7 lines
2.14.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v4 6/8] ARM: VGIC: factor out vgic_get_hw_irq_desc()
Hi Andre,
Quoted 7 lines · Andre Przywara
At the moment we happily access the VGIC internal struct
pending_irq(which describes a virtual IRQ) inirq.c.
Factor out the actually needed functionality to learn the associated hardware IRQ and move that intogic-vgic.cto improve abstraction.Signed-off-by Andre Przywara <[email protected]>
Cheers,
Quoted 74 lines
--- xen/arch/arm/gic-vgic.c | 17 +++++++++++++++++ xen/arch/arm/irq.c | 7 ++----- xen/include/asm-arm/vgic.h | 2 ++ 3 files changed, 21 insertions(+), 5 deletions(-)xen/arch/arm/gic-vgic.c +17
-0@@ -397,6 +397,23 @@ void gic_dump_vgic_info(struct vcpu *v)printk ("Pending irq=%d\n" , p->irq); } +struct irq_desc *vgic_get_hw_irq_desc (struct domain *d,struct vcpu *v, +unsigned int virq) +{ +struct pending_irq *p; + +ASSERT (!v && virq >=32 ); + +if ( !v ) + v = d->vcpu[0 ]; + + p =irq_to_pending (v, virq); +if ( !p ) +return NULL ; + +return p->desc; +} +int vgic_connect_hw_irq (struct domain *d,struct vcpu *v,unsigned int virq,struct irq_desc *desc,bool connect) {xen/arch/arm/irq.c +2
-5@@ -534,19 +534,16 @@ int release_guest_irq(struct domain *d, unsigned int virq)struct irq_desc *desc;struct irq_guest *info;unsigned long flags; -struct pending_irq *p;int ret;/* Only SPIs are supported */ if ( virq <NR_LOCAL_IRQS || virq >=vgic_num_irqs (d) )return -EINVAL ; - p =spi_to_pending (d, virq); -if ( !p->desc ) + desc =vgic_get_hw_irq_desc (d,NULL , virq); +if ( !desc )return -EINVAL ; - desc = p->desc; -spin_lock_irqsave (&desc->lock, flags); ret = -EINVAL ;xen/include/asm-arm/vgic.h +2
-0@@ -219,6 +219,8 @@ int vgic_v2_init(struct domain *d, int *mmio_count);int vgic_v3_init (struct domain *d,int *mmio_count);bool vgic_evtchn_irq_pending (struct vcpu *v); +struct irq_desc *vgic_get_hw_irq_desc (struct domain *d,struct vcpu *v, +unsigned int virq);int vgic_connect_hw_irq (struct domain *d,struct vcpu *v,unsigned int virq,struct irq_desc *desc,bool connect);
Signature · 6 lines
Julien Grall _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v4 7/8 ARM: VGIC: rework gicv[23]_update_lr to not use pending_irq
In reply to Andre PrzywaraThe functions to actually populate a list register were accessing the VGIC internal pending_irq struct, although they should be abstracting from that.
Break the needed information down to remove the reference to pending_irq from gic-v[23].c.
--- xen/arch/arm/gic-v2.c | 14 +++++++------- xen/arch/arm/gic-v3.c | 12 ++++++------ xen/arch/arm/gic-vgic.c | 3 ++- xen/include/asm-arm/gic.h | 4 ++-- xen/include/asm-arm/irq.h | 3 +++ 5 files changed, 20 insertions(+), 16 deletions(-)
xen/arch/arm/gic-v2.c +7 -7
@@ -428,8 +428,8 @@ static void gicv2_disable_interface(void)spin_unlock (&gicv2.lock); } -static void gicv2_update_lr (int lr,const struct pending_irq *p, -unsigned int state) +static void gicv2_update_lr (int lr,unsigned int virq,uint8_t priority, +unsigned int hw_irq,unsigned int state) {uint32_t lr_reg;
@@ -437,12 +437,12 @@ static void gicv2_update_lr(int lr, const struct pending_irq *p,BUG_ON (lr <0 ); lr_reg = (((state &GICH_V2_LR_STATE_MASK ) <<GICH_V2_LR_STATE_SHIFT ) | - ((GIC_PRI_TO_GUEST (p->priority) &GICH_V2_LR_PRIORITY_MASK ) - <<GICH_V2_LR_PRIORITY_SHIFT ) | - ((p->irq &GICH_V2_LR_VIRTUAL_MASK ) <<GICH_V2_LR_VIRTUAL_SHIFT )); + ((GIC_PRI_TO_GUEST (priority) &GICH_V2_LR_PRIORITY_MASK ) + <<GICH_V2_LR_PRIORITY_SHIFT ) | + ((virq &GICH_V2_LR_VIRTUAL_MASK ) <<GICH_V2_LR_VIRTUAL_SHIFT )); -if ( p->desc !=NULL ) - lr_reg |=GICH_V2_LR_HW | ((p->desc->irq &GICH_V2_LR_PHYSICAL_MASK ) +if ( hw_irq !=INVALID_IRQ ) + lr_reg |=GICH_V2_LR_HW | ((hw_irq &GICH_V2_LR_PHYSICAL_MASK ) <<GICH_V2_LR_PHYSICAL_SHIFT );writel_gich (lr_reg,GICH_LR + lr *4 );
xen/arch/arm/gic-v3.c +6 -6
@@ -966,8 +966,8 @@ static void gicv3_disable_interface(void)spin_unlock (&gicv3.lock); } -static void gicv3_update_lr (int lr,const struct pending_irq *p, -unsigned int state) +static void gicv3_update_lr (int lr,unsigned int virq,uint8_t priority, +unsigned int hw_irq,unsigned int state) {uint64_t val =0 ;
@@ -983,11 +983,11 @@ static void gicv3_update_lr(int lr, const struct pending_irq *p,if ( current->domain->arch.vgic.version ==GIC_V3 ) val |=GICH_LR_GRP1 ; - val |= ((uint64_t )p->priority &0xff ) <<GICH_LR_PRIORITY_SHIFT ; - val |= ((uint64_t )p->irq &GICH_LR_VIRTUAL_MASK ) <<GICH_LR_VIRTUAL_SHIFT ; + val |= (uint64_t )priority <<GICH_LR_PRIORITY_SHIFT ; + val |= ((uint64_t )virq &GICH_LR_VIRTUAL_MASK ) <<GICH_LR_VIRTUAL_SHIFT ; -if ( p->desc !=NULL ) - val |=GICH_LR_HW | (((uint64_t )p->desc->irq &GICH_LR_PHYSICAL_MASK ) +if ( hw_irq !=INVALID_IRQ ) + val |=GICH_LR_HW | (((uint64_t )hw_irq &GICH_LR_PHYSICAL_MASK ) <<GICH_LR_PHYSICAL_SHIFT );gicv3_ich_write_lr (lr, val);
xen/arch/arm/gic-vgic.c +2 -1
@@ -38,7 +38,8 @@ static inline void gic_set_lr(int lr, struct pending_irq *p,clear_bit (GIC_IRQ_GUEST_PRISTINE_LPI , &p->status); - gic_hw_ops->update_lr (lr, p, state); + gic_hw_ops->update_lr (lr, p->irq, p->priority, + p->desc ? p->desc->irq :INVALID_IRQ , state);set_bit (GIC_IRQ_GUEST_VISIBLE , &p->status);clear_bit (GIC_IRQ_GUEST_QUEUED , &p->status);
xen/include/asm-arm/gic.h +2 -2
@@ -344,8 +344,8 @@ struct gic_hw_operations {/* Disable CPU physical and virtual interfaces */ void (*disable_interface)(void );/* Update LR register with state and priority */ -void (*update_lr)(int lr,const struct pending_irq *pending_irq, -unsigned int state); +void (*update_lr)(int lr,unsigned int virq,uint8_t priority, +unsigned int hw_irq,unsigned int state);/* Update HCR status register */ void (*update_hcr_status)(uint32_t flag,bool set);/* Clear LR register */
xen/include/asm-arm/irq.h +3 -0
@@ -31,6 +31,9 @@ struct arch_irq_desc {/* LPIs are always numbered starting at 8192, so 0 is a good invalid case. */ #define INVALID_LPI 0 +/* This is a spurious interrupt ID which never makes it into the GIC code. */ +#define INVALID_IRQ 1023 +extern unsigned int nr_irqs;#define nr_static_irqs NR_IRQS#define arch_hwdom_irqs (domid) NR_IRQS
Signature · 7 lines
2.14.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v4 8/8 ARM: make nr_irqs a constant
In reply to Andre PrzywaraOn ARM the maximum number of IRQs is a constant, but we share it being a variable to match x86. Since we are not supposed to alter it, let's mark it as "const" to avoid accidental change.
--- xen/arch/arm/irq.c | 2 +- xen/include/asm-arm/irq.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
xen/arch/arm/irq.c +1 -1
@@ -27,7 +27,7 @@#include <asm/gic.h> #include <asm/vgic.h> -unsigned int __read_mostly nr_irqs =NR_IRQS ; +const unsigned int nr_irqs =NR_IRQS ;static unsigned int local_irqs_type[NR_LOCAL_IRQS ];static DEFINE_SPINLOCK (local_irqs_type_lock);
xen/include/asm-arm/irq.h +1 -1
@@ -34,7 +34,7 @@ struct arch_irq_desc {/* This is a spurious interrupt ID which never makes it into the GIC code. */ #define INVALID_IRQ 1023 -extern unsigned int nr_irqs; +extern const unsigned int nr_irqs;#define nr_static_irqs NR_IRQS#define arch_hwdom_irqs (domid) NR_IRQS
Signature · 7 lines
2.14.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel