genirq: Treewide hunt for irq descriptor abuse and assorted fixes
Subsystems (27)
cover genirq: Treewide hunt for irq descriptor abuse and assorted fixes
A recent request to export kstat_irqs() pointed to a copy of the same in the i915 code, which made me look for further usage of irq descriptors in drivers.
The usage in drivers ranges from creative to broken in all colours.
irqdesc.h clearly says that this is core functionality and the fact C does not allow full encapsulation is not a justification to fiddle with it just because. It took us a lot of effort to make the core functionality provide what drivers need.
If there is a shortcoming, it's not asked too much to talk to the relevant maintainers instead of going off and fiddling with the guts of interrupt descriptors and often enough without understanding lifetime and locking rules.
As people insist on not respecting boundaries, this series cleans up the (ab)use and at the end removes the export of irq_to_desc() to make it at least harder. All legitimate users of this are built in.
While at it I stumbled over some other oddities related to interrupt counting and cleaned them up as well.
The series applies on top of
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/core
and is also available from git:
git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git genirq
Thanks,
tglx
--- arch/alpha/kernel/sys_jensen.c | 2 arch/arm/kernel/smp.c | 2 arch/parisc/kernel/irq.c | 7 arch/s390/kernel/irq.c | 2 arch/x86/kernel/topology.c | 1 arch/arm64/kernel/smp.c | 2 drivers/gpu/drm/i915/display/intel_lpe_audio.c | 4 drivers/gpu/drm/i915/i915_irq.c | 34 +++ drivers/gpu/drm/i915/i915_pmu.c | 18 - drivers/gpu/drm/i915/i915_pmu.h | 8 drivers/mfd/ab8500-debugfs.c | 16 - drivers/net/ethernet/mellanox/mlx4/en_cq.c | 8 drivers/net/ethernet/mellanox/mlx4/en_rx.c | 6 drivers/net/ethernet/mellanox/mlx4/mlx4_en.h | 3 drivers/net/ethernet/mellanox/mlx5/core/en.h | 2 drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 2 drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c | 6 drivers/ntb/msi.c | 4 drivers/pci/controller/mobiveil/pcie-mobiveil-host.c | 8 drivers/pci/controller/pcie-xilinx-nwl.c | 8 drivers/pinctrl/nomadik/pinctrl-nomadik.c | 3 drivers/xen/events/events_base.c | 172 +++++++++++-------- drivers/xen/evtchn.c | 34 --- include/linux/interrupt.h | 1 include/linux/irq.h | 7 include/linux/irqdesc.h | 40 +--- include/linux/kernel_stat.h | 1 kernel/irq/irqdesc.c | 42 ++-- kernel/irq/manage.c | 37 ++++ kernel/irq/proc.c | 5 30 files changed, 263 insertions(+), 222 deletions(-)
PATCH 1/30 genirq: Move irq_has_action() into core code
This function uses irq_to_desc() and is going to be used by modules to replace the open coded irq_to_desc() (ab)usage. The final goal is to remove the export of irq_to_desc() so driver cannot fiddle with it anymore.
Move it into the core code and fixup the usage sites to include the proper header.
--- arch/alpha/kernel/sys_jensen.c | 2 +- arch/x86/kernel/topology.c | 1 + include/linux/interrupt.h | 1 + include/linux/irqdesc.h | 7 +------ kernel/irq/manage.c | 17 +++++++++++++++++ 5 files changed, 21 insertions(+), 7 deletions(-)
arch/alpha/kernel/sys_jensen.c +1 -1
@@ -7,7 +7,7 @@* * Code supporting the Jensen. */ - +#include <linux/interrupt.h> #include <linux/kernel.h> #include <linux/types.h> #include <linux/mm.h>
arch/x86/kernel/topology.c +1 -0
@@ -25,6 +25,7 @@* * Send feedback to <[email protected]> */ +#include <linux/interrupt.h> #include <linux/nodemask.h> #include <linux/export.h> #include <linux/mmzone.h>
include/linux/interrupt.h +1 -0
@@ -232,6 +232,7 @@ extern void devm_free_irq(struct device# define local_irq_enable_in_hardirq () local_irq_enable()#endif +bool irq_has_action (unsigned int irq);extern void disable_irq_nosync (unsigned int irq);extern bool disable_hardirq (unsigned int irq);extern void disable_irq (unsigned int irq);
include/linux/irqdesc.h +1 -6
@@ -179,12 +179,7 @@ int handle_domain_nmi(struct irq_domain/* Test to see if a driver has successfully requested an irq */ static inline int irq_desc_has_action (struct irq_desc *desc) { -return desc->action !=NULL ; -} - -static inline int irq_has_action (unsigned int irq) -{ -return irq_desc_has_action (irq_to_desc (irq)); +return desc && desc->action !=NULL ; }/**
kernel/irq/manage.c +17 -0
@@ -2752,3 +2752,20 @@ int irq_set_irqchip_state(unsigned int ireturn err; }EXPORT_SYMBOL_GPL (irq_set_irqchip_state); + +/** +* irq_has_action - Check whether an interrupt is requested +* @irq: The linux irq number +* +* Returns: A snapshot of the current state +*/ +bool irq_has_action (unsigned int irq) +{ +bool res; + +rcu_read_lock (); + res =irq_desc_has_action (irq_to_desc (irq)); +rcu_read_unlock (); +return res; +} +EXPORT_SYMBOL_GPL (irq_has_action);
[tip: irq/core] genirq: Move irq_has_action() into core code
The following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: db123744af5d237048f23ede05cc7ef0dc48db01 Gitweb: https://git.kernel.org/tip/db123744af5d237048f23ede05cc7ef0dc48db01 Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:25:37 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:02 +01:00
genirq: Move irq_has_action() into core code
This function uses irq_to_desc() and is going to be used by modules to replace the open coded irq_to_desc() (ab)usage. The final goal is to remove the export of irq_to_desc() so driver cannot fiddle with it anymore.
Move it into the core code and fixup the usage sites to include the proper header.
--- arch/alpha/kernel/sys_jensen.c | 2 +- arch/x86/kernel/topology.c | 1 + include/linux/interrupt.h | 1 + include/linux/irqdesc.h | 7 +------ kernel/irq/manage.c | 17 +++++++++++++++++ 5 files changed, 21 insertions(+), 7 deletions(-)
arch/alpha/kernel/sys_jensen.c +1 -1
@@ -7,7 +7,7 @@* * Code supporting the Jensen. */ - +#include <linux/interrupt.h> #include <linux/kernel.h> #include <linux/types.h> #include <linux/mm.h>
arch/x86/kernel/topology.c +1 -0
@@ -25,6 +25,7 @@* * Send feedback to <[email protected]> */ +#include <linux/interrupt.h> #include <linux/nodemask.h> #include <linux/export.h> #include <linux/mmzone.h>
include/linux/interrupt.h +1 -0
@@ -232,6 +232,7 @@ extern void devm_free_irq(struct device *dev, unsigned int irq, void *dev_id);# define local_irq_enable_in_hardirq () local_irq_enable()#endif +bool irq_has_action (unsigned int irq);extern void disable_irq_nosync (unsigned int irq);extern bool disable_hardirq (unsigned int irq);extern void disable_irq (unsigned int irq);
include/linux/irqdesc.h +1 -6
@@ -179,12 +179,7 @@ int handle_domain_nmi(struct irq_domain *domain, unsigned int hwirq,/* Test to see if a driver has successfully requested an irq */ static inline int irq_desc_has_action (struct irq_desc *desc) { -return desc->action !=NULL ; -} - -static inline int irq_has_action (unsigned int irq) -{ -return irq_desc_has_action (irq_to_desc (irq)); +return desc && desc->action !=NULL ; }/**
kernel/irq/manage.c +17 -0
@@ -2752,3 +2752,20 @@ out_unlock:return err; }EXPORT_SYMBOL_GPL (irq_set_irqchip_state); + +/** +* irq_has_action - Check whether an interrupt is requested +* @irq: The linux irq number +* +* Returns: A snapshot of the current state +*/ +bool irq_has_action (unsigned int irq) +{ +bool res; + +rcu_read_lock (); + res =irq_desc_has_action (irq_to_desc (irq)); +rcu_read_unlock (); +return res; +} +EXPORT_SYMBOL_GPL (irq_has_action);
PATCH 2/30 genirq: Move status flag checks to core
In reply to Thomas GleixnerThese checks are used by modules and prevent the removal of the export of irq_to_desc(). Move the accessor into the core.
--- include/linux/irqdesc.h | 17 +++++------------ kernel/irq/manage.c | 17 +++++++++++++++++ 2 files changed, 22 insertions(+), 12 deletions(-)
include/linux/irqdesc.h +5 -12
@@ -223,28 +223,21 @@ irq_set_chip_handler_name_locked(struct data->chip = chip; } +bool irq_check_status_bit (unsigned int irq,unsigned int bitmask); +static inline bool irq_balancing_disabled (unsigned int irq) { -struct irq_desc *desc; - - desc =irq_to_desc (irq); -return desc->status_use_accessors &IRQ_NO_BALANCING_MASK ; +return irq_check_status_bit (irq,IRQ_NO_BALANCING_MASK ); }static inline bool irq_is_percpu (unsigned int irq) { -struct irq_desc *desc; - - desc =irq_to_desc (irq); -return desc->status_use_accessors &IRQ_PER_CPU ; +return irq_check_status_bit (irq,IRQ_PER_CPU ); }static inline bool irq_is_percpu_devid (unsigned int irq) { -struct irq_desc *desc; - - desc =irq_to_desc (irq); -return desc->status_use_accessors &IRQ_PER_CPU_DEVID ; +return irq_check_status_bit (irq,IRQ_PER_CPU_DEVID ); } static inline void
kernel/irq/manage.c +20 -0
@@ -2769,3 +2769,23 @@ bool irq_has_action(unsigned int irq)return res; }EXPORT_SYMBOL_GPL (irq_has_action); + +/** +* irq_check_status_bit - Check whether bits in the irq descriptor status are set +* @irq: The linux irq number +* @bitmask: The bitmask to evaluate +* +* Returns: True if one of the bits in @bitmask is set +*/ +bool irq_check_status_bit (unsigned int irq,unsigned int bitmask) +{ +struct irq_desc *desc; +bool res = false; + +rcu_read_lock (); + desc =irq_to_desc (irq); +if (desc) + res = !!(desc->status_use_accessors & bitmask); +rcu_read_unlock (); +return res; +}
[tip: irq/core] genirq: Move status flag checks to core
The following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: a578bb32cb1a46676caae72dc0c668630ab40c1f Gitweb: https://git.kernel.org/tip/a578bb32cb1a46676caae72dc0c668630ab40c1f Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:25:38 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:02 +01:00
genirq: Move status flag checks to core
These checks are used by modules and prevent the removal of the export of irq_to_desc(). Move the accessor into the core.
--- include/linux/irqdesc.h | 17 +++++------------ kernel/irq/manage.c | 20 ++++++++++++++++++++ 2 files changed, 25 insertions(+), 12 deletions(-)
include/linux/irqdesc.h +5 -12
@@ -223,28 +223,21 @@ irq_set_chip_handler_name_locked(struct irq_data *data, struct irq_chip *chip, data->chip = chip; } +bool irq_check_status_bit (unsigned int irq,unsigned int bitmask); +static inline bool irq_balancing_disabled (unsigned int irq) { -struct irq_desc *desc; - - desc =irq_to_desc (irq); -return desc->status_use_accessors &IRQ_NO_BALANCING_MASK ; +return irq_check_status_bit (irq,IRQ_NO_BALANCING_MASK ); }static inline bool irq_is_percpu (unsigned int irq) { -struct irq_desc *desc; - - desc =irq_to_desc (irq); -return desc->status_use_accessors &IRQ_PER_CPU ; +return irq_check_status_bit (irq,IRQ_PER_CPU ); }static inline bool irq_is_percpu_devid (unsigned int irq) { -struct irq_desc *desc; - - desc =irq_to_desc (irq); -return desc->status_use_accessors &IRQ_PER_CPU_DEVID ; +return irq_check_status_bit (irq,IRQ_PER_CPU_DEVID ); } static inline void
kernel/irq/manage.c +20 -0
@@ -2769,3 +2769,23 @@ bool irq_has_action(unsigned int irq)return res; }EXPORT_SYMBOL_GPL (irq_has_action); + +/** +* irq_check_status_bit - Check whether bits in the irq descriptor status are set +* @irq: The linux irq number +* @bitmask: The bitmask to evaluate +* +* Returns: True if one of the bits in @bitmask is set +*/ +bool irq_check_status_bit (unsigned int irq,unsigned int bitmask) +{ +struct irq_desc *desc; +bool res = false; + +rcu_read_lock (); + desc =irq_to_desc (irq); +if (desc) + res = !!(desc->status_use_accessors & bitmask); +rcu_read_unlock (); +return res; +}
Re: [patch 02/30] genirq: Move status flag checks to core
In reply to Thomas GleixnerQuoted 4 lines · Thomas Gleixner
These checks are used by modules and prevent the removal of the export of
irq_to_desc(). Move the accessor into the core.Signed-off-by Thomas Gleixner <[email protected]>
Yes, but that means that irq_check_status_bit() may be called from modules, but it is not exported, resulting in build errors such as the following.
arm64:allmodconfig:
ERROR: modpost: "irq_check_status_bit" [drivers/perf/arm_spe_pmu.ko] undefined!
Guenter
Quoted 67 lines
--- include/linux/irqdesc.h | 17 +++++------------ kernel/irq/manage.c | 17 +++++++++++++++++ 2 files changed, 22 insertions(+), 12 deletions(-)include/linux/irqdesc.h +5
-12@@ -223,28 +223,21 @@ irq_set_chip_handler_name_locked(struct data->chip = chip; } +bool irq_check_status_bit (unsigned int irq,unsigned int bitmask); +static inline bool irq_balancing_disabled (unsigned int irq) { -struct irq_desc *desc; - - desc =irq_to_desc (irq); -return desc->status_use_accessors &IRQ_NO_BALANCING_MASK ; +return irq_check_status_bit (irq,IRQ_NO_BALANCING_MASK ); }static inline bool irq_is_percpu (unsigned int irq) { -struct irq_desc *desc; - - desc =irq_to_desc (irq); -return desc->status_use_accessors &IRQ_PER_CPU ; +return irq_check_status_bit (irq,IRQ_PER_CPU ); }static inline bool irq_is_percpu_devid (unsigned int irq) { -struct irq_desc *desc; - - desc =irq_to_desc (irq); -return desc->status_use_accessors &IRQ_PER_CPU_DEVID ; +return irq_check_status_bit (irq,IRQ_PER_CPU_DEVID ); } static inline voidkernel/irq/manage.c +20
-0@@ -2769,3 +2769,23 @@ bool irq_has_action(unsigned int irq)return res; }EXPORT_SYMBOL_GPL (irq_has_action); + +/** +* irq_check_status_bit - Check whether bits in the irq descriptor status are set +* @irq: The linux irq number +* @bitmask: The bitmask to evaluate +* +* Returns: True if one of the bits in @bitmask is set +*/ +bool irq_check_status_bit (unsigned int irq,unsigned int bitmask) +{ +struct irq_desc *desc; +bool res = false; + +rcu_read_lock (); + desc =irq_to_desc (irq); +if (desc) + res = !!(desc->status_use_accessors & bitmask); +rcu_read_unlock (); +return res; +}
Re: [patch 02/30] genirq: Move status flag checks to core
Quoted 7 lines · Guenter Roeck
On Thu, Dec 10, 2020 at 08:25:38PM +0100, Thomas Gleixner wrote:
Yes, but that means thatirq_check_status_bit()may be called from modules, but it is not exported, resulting in build errors such as the following.arm64:allmodconfig:
ERROR: modpost: "
irq_check_status_bit" [drivers/perf/arm_spe_pmu.ko] undefined!
Duh. Yes, that lacks an export obviously.
Thanks,
tglx
[tip: irq/urgent] genirq: Export irq_check_status_bit()
In reply to Guenter RoeckThe following commit has been merged into the irq/urgent branch of tip:
The following commit has been merged into the irq/urgent branch of tip:
Commit-ID: ce09ccc50208c04a1b03abfd530b5d6314258fd0 Gitweb: https://git.kernel.org/tip/ce09ccc50208c04a1b03abfd530b5d6314258fd0 Author: Thomas Gleixner <[email protected]> AuthorDate: Wed, 13 Jan 2021 15:43:18 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Wed, 13 Jan 2021 15:48:05 +01:00
genirq: Export irq_check_status_bit()
One of the users can be built modular:
ERROR: modpost: "irq_check_status_bit" [drivers/perf/arm_spe_pmu.ko] undefined!
--- kernel/irq/manage.c | 1 + 1 file changed, 1 insertion(+)
kernel/irq/manage.c +1 -0
@@ -2859,3 +2859,4 @@ bool irq_check_status_bit(unsigned int irq, unsigned int bitmask)rcu_read_unlock ();return res; } +EXPORT_SYMBOL_GPL (irq_check_status_bit);
PATCH 3/30 genirq: Move irq_set_lockdep_class() to core
In reply to Thomas Gleixnerirq_set_lockdep_class() is used from modules and requires irq_to_desc() to be exported. Move it into the core code which lifts another requirement for the export.
--- include/linux/irqdesc.h | 10 ++++------ kernel/irq/irqdesc.c | 14 ++++++++++++++ 2 files changed, 18 insertions(+), 6 deletions(-)
include/linux/irqdesc.h +4 -6
@@ -240,16 +240,14 @@ static inline bool irq_is_percpu_devid(u return irq_check_status_bit(irq, IRQ_PER_CPU_DEVID); } +void __irq_set_lockdep_class (unsigned int irq,struct lock_class_key *lock_class, +struct lock_class_key *request_class);static inline void irq_set_lockdep_class (unsigned int irq,struct lock_class_key *lock_class,struct lock_class_key *request_class) { -struct irq_desc *desc =irq_to_desc (irq); - -if (desc) { -lockdep_set_class (&desc->lock, lock_class); -lockdep_set_class (&desc->request_mutex, request_class); - } +if (IS_ENABLED (CONFIG_LOCKDEP )) +__irq_set_lockdep_class (irq, lock_class, request_class); }#endif
kernel/irq/irqdesc.c +14 -0
@@ -968,3 +968,17 @@ unsigned int kstat_irqs_usr(unsigned intrcu_read_unlock ();return sum; } + +#ifdef CONFIG_LOCKDEP +void __irq_set_lockdep_class (unsigned int irq,struct lock_class_key *lock_class, +struct lock_class_key *request_class) +{ +struct irq_desc *desc =irq_to_desc (irq); + +if (desc) { +lockdep_set_class (&desc->lock, lock_class); +lockdep_set_class (&desc->request_mutex, request_class); + } +} +EXPORT_SYMBOL_GPL (irq_set_lockdep_class); +#endif
Re: [patch 03/30] genirq: Move irq_set_lockdep_class() to core
Quoted 4 lines · Thomas Gleixner
irq_set_lockdep_class()is used from modules and requiresirq_to_desc()to be exported. Move it into the core code which lifts another requirement for the export.
...
include/linux/irqdesc.h static inline bool irq_is_percpu_devid(u · line 250
Quoted 2 lines
include/linux/irqdesc.h +2
-0+if (IS_ENABLED (CONFIG_LOCKDEP )) +__irq_set_lockdep_class (irq, lock_class, request_class);
Maybe I missed something, but even if the compiler does not warn the use of if IS_ENABLED() with complimentary #ifdef seems inconsistent.
Quoted 1 line
+#ifdef
CONFIG_LOCKDEP
...
Quoted 2 lines
kernel/irq/irqdesc.c +2
-0+EXPORT_SYMBOL_GPL (irq_set_lockdep_class); +#endif
Signature · 2 lines
With Best Regards, Andy Shevchenko
Re: [patch 03/30] genirq: Move irq_set_lockdep_class() to core
Quoted 10 lines · Andy Shevchenko
Quoted 4 lines · Thomas Gleixner
irq_set_lockdep_class()is used from modules and requiresirq_to_desc()to be exported. Move it into the core code which lifts another requirement for the export....
include/linux/irqdesc.h static inline bool irq_is_percpu_devid(u · line 250
Quoted 2 lines
include/linux/irqdesc.h +2
-0+if (IS_ENABLED (CONFIG_LOCKDEP )) +__irq_set_lockdep_class (irq, lock_class, request_class);
You are right. Let me fix that.
Re: [patch 03/30] genirq: Move irq_set_lockdep_class() to core
Quoted 8 lines · Thomas Gleixner
Quoted 6 lines · Andy Shevchenko
Quoted 4 lines · Thomas Gleixner
irq_set_lockdep_class()is used from modules and requiresirq_to_desc()to be exported. Move it into the core code which lifts another requirement for the export.
Quoted 6 lines
Quoted 4 lines
...
include/linux/irqdesc.h static inline bool irq_is_percpu_devid(u · line 250
Quoted 2 lines
include/linux/irqdesc.h +2
-0+if (IS_ENABLED (CONFIG_LOCKDEP )) +__irq_set_lockdep_class (irq, lock_class, request_class);You are right. Let me fix that.
No. I have to correct myself. You're wrong.
The inline is evaluated in the compilation units which include that header and because the function declaration is unconditional it is happy.
Now the optimizer stage makes the whole thing a NOOP if CONFIG_LOCKDEP=n and thereby drops the reference to the function which makes it not required for linking.
So in the file where the function is implemented:
#ifdef CONFIG_LOCKDEP
void __irq_set_lockdep_class(....)
{
}
#endif
The whole block is either discarded because CONFIG_LOCKDEP is not defined or compile if it is defined which makes it available for the linker.
And in the latter case the optimizer keeps the call in the inline (it optimizes the condition away because it's always true).
So in both cases the compiler and the linker are happy and everything works as expected.
It would fail if the header file had the following:
#ifdef CONFIG_LOCKDEP
void __irq_set_lockdep_class(....);
#endif
Because then it would complain about the missing function prototype when it evaluates the inline.
Thanks,
tglx
Re: [patch 03/30] genirq: Move irq_set_lockdep_class() to core
Quoted 46 lines · Thomas Gleixner
Quoted 8 lines · Thomas Gleixner
Quoted 6 lines · Andy Shevchenko
Quoted 4 lines · Thomas Gleixner
irq_set_lockdep_class()is used from modules and requiresirq_to_desc()to be exported. Move it into the core code which lifts another requirement for the export.Quoted 6 lines
Quoted 4 lines
...
include/linux/irqdesc.h static inline bool irq_is_percpu_devid(u · line 250
Quoted 2 lines
include/linux/irqdesc.h +2
-0+if (IS_ENABLED (CONFIG_LOCKDEP )) +__irq_set_lockdep_class (irq, lock_class, request_class);You are right. Let me fix that.
No. I have to correct myself. You're wrong.
The inline is evaluated in the compilation units which include that header and because the function declaration is unconditional it is happy.
Now the optimizer stage makes the whole thing a NOOP if
CONFIG_LOCKDEP=nand thereby drops the reference to the function which makes it not required for linking.So in the file where the function is implemented:
#ifdef
CONFIG_LOCKDEP
void__irq_set_lockdep_class(....)
{
}
#endifThe whole block is either discarded because
CONFIG_LOCKDEPis not defined or compile if it is defined which makes it available for the linker.And in the latter case the optimizer keeps the call in the inline (it optimizes the condition away because it's always true).
So in both cases the compiler and the linker are happy and everything works as expected.
Quoted 8 lines
It would fail if the header file had the following:
#ifdef
CONFIG_LOCKDEP
void__irq_set_lockdep_class(....);
#endifBecause then it would complain about the missing function prototype when it evaluates the inline.
I understand that (that's why I put "if even no warning") and what I'm talking about is the purpose of IS_ENABLED(). It's usually good for compile testing !CONFIG_FOO cases. But here it seems inconsistent.
The pattern I usually see in the cases like this is
#ifdef CONFIG_LOCKDEP
void __irq_set_lockdep_class(....);
#else
static inline void ... {}
#endif
and call it directly in the caller.
It's not a big deal, so up to you.
Signature · 2 lines
With Best Regards, Andy Shevchenko
[tip: irq/core] genirq: Move irq_set_lockdep_class() to core
In reply to Thomas GleixnerThe following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: a07d244f00de679c66643e89eba6e1baa62d3517 Gitweb: https://git.kernel.org/tip/a07d244f00de679c66643e89eba6e1baa62d3517 Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:25:39 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:02 +01:00
genirq: Move irq_set_lockdep_class() to core
irq_set_lockdep_class() is used from modules and requires irq_to_desc() to be exported. Move it into the core code which lifts another requirement for the export.
--- include/linux/irqdesc.h | 10 ++++------ kernel/irq/irqdesc.c | 14 ++++++++++++++ 2 files changed, 18 insertions(+), 6 deletions(-)
include/linux/irqdesc.h +4 -6
@@ -240,16 +240,14 @@ static inline bool irq_is_percpu_devid(unsigned int irq) return irq_check_status_bit(irq, IRQ_PER_CPU_DEVID); } +void __irq_set_lockdep_class (unsigned int irq,struct lock_class_key *lock_class, +struct lock_class_key *request_class);static inline void irq_set_lockdep_class (unsigned int irq,struct lock_class_key *lock_class,struct lock_class_key *request_class) { -struct irq_desc *desc =irq_to_desc (irq); - -if (desc) { -lockdep_set_class (&desc->lock, lock_class); -lockdep_set_class (&desc->request_mutex, request_class); - } +if (IS_ENABLED (CONFIG_LOCKDEP )) +__irq_set_lockdep_class (irq, lock_class, request_class); }#endif
kernel/irq/irqdesc.c +14 -0
@@ -968,3 +968,17 @@ unsigned int kstat_irqs_usr(unsigned int irq)rcu_read_unlock ();return sum; } + +#ifdef CONFIG_LOCKDEP +void __irq_set_lockdep_class (unsigned int irq,struct lock_class_key *lock_class, +struct lock_class_key *request_class) +{ +struct irq_desc *desc =irq_to_desc (irq); + +if (desc) { +lockdep_set_class (&desc->lock, lock_class); +lockdep_set_class (&desc->request_mutex, request_class); + } +} +EXPORT_SYMBOL_GPL (irq_set_lockdep_class); +#endif
PATCH 4/30 genirq: Provide irq_get_effective_affinity()
In reply to Thomas GleixnerProvide an accessor to the effective interrupt affinity mask. Going to be used to replace open coded fiddling with the irq descriptor.
--- include/linux/irq.h | 7 +++++++ 1 file changed, 7 insertions(+)
include/linux/irq.h +7 -0
@@ -907,6 +907,13 @@ struct cpumask *irq_data_get_effective_a }#endif +static inline struct cpumask *irq_get_effective_affinity_mask (unsigned int irq) +{ +struct irq_data *d =irq_get_irq_data (irq); + +return d ?irq_data_get_effective_affinity_mask (d) :NULL ; +} +unsigned int arch_dynirq_lower_bound (unsigned int from); int __irq_alloc_descs(int irq, unsigned int from, unsigned int cnt, int node,
[tip: irq/core] genirq: Provide irq_get_effective_affinity()
The following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 276b1e6e6fecc2f138c72519705dabc8bd01ffa9 Gitweb: https://git.kernel.org/tip/276b1e6e6fecc2f138c72519705dabc8bd01ffa9 Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:25:40 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:02 +01:00
genirq: Provide irq_get_effective_affinity()
Provide an accessor to the effective interrupt affinity mask. Going to be used to replace open coded fiddling with the irq descriptor.
--- include/linux/irq.h | 7 +++++++ 1 file changed, 7 insertions(+)
include/linux/irq.h +7 -0
@@ -907,6 +907,13 @@ struct cpumask *irq_data_get_effective_affinity_mask(struct irq_data *d) }#endif +static inline struct cpumask *irq_get_effective_affinity_mask (unsigned int irq) +{ +struct irq_data *d =irq_get_irq_data (irq); + +return d ?irq_data_get_effective_affinity_mask (d) :NULL ; +} +unsigned int arch_dynirq_lower_bound (unsigned int from); int __irq_alloc_descs(int irq, unsigned int from, unsigned int cnt, int node,
PATCH 5/30 genirq: Annotate irq stats data races
In reply to Thomas GleixnerBoth the per cpu stats and the accumulated count are accessed lockless and can be concurrently modified. That's intentional and the stats are a rough estimate anyway. Annotate them with data_race().
--- kernel/irq/irqdesc.c | 4 ++-- kernel/irq/proc.c | 5 +++-- 2 files changed, 5 insertions(+), 4 deletions(-)
kernel/irq/irqdesc.c +2 -2
@@ -943,10 +943,10 @@ unsigned int kstat_irqs(unsigned int irqif (!irq_settings_is_per_cpu_devid (desc) && !irq_settings_is_per_cpu (desc) && !irq_is_nmi (desc)) -return desc->tot_count; +return data_race (desc->tot_count);for_each_possible_cpu (cpu) - sum += *per_cpu_ptr (desc->kstat_irqs, cpu); + sum +=data_race (*per_cpu_ptr (desc->kstat_irqs, cpu));return sum; }
kernel/irq/proc.c +3 -2
@@ -488,9 +488,10 @@ int show_interrupts(struct seq_file *p,if (!desc ||irq_settings_is_hidden (desc)) goto outsparse; -if (desc->kstat_irqs) +if (desc->kstat_irqs) {for_each_online_cpu (j) - any_count |= *per_cpu_ptr (desc->kstat_irqs, j); + any_count |=data_race (*per_cpu_ptr (desc->kstat_irqs, j)); + }if ((!desc->action ||irq_desc_is_chained (desc)) && !any_count) goto outsparse;
[tip: irq/core] genirq: Annotate irq stats data races
The following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: fb4f676fc901cb547226efb3e69ffeaeefa124be Gitweb: https://git.kernel.org/tip/fb4f676fc901cb547226efb3e69ffeaeefa124be Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:25:41 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:03 +01:00
genirq: Annotate irq stats data races
Both the per cpu stats and the accumulated count are accessed lockless and can be concurrently modified. That's intentional and the stats are a rough estimate anyway. Annotate them with data_race().
--- kernel/irq/irqdesc.c | 4 ++-- kernel/irq/proc.c | 5 +++-- 2 files changed, 5 insertions(+), 4 deletions(-)
kernel/irq/irqdesc.c +2 -2
@@ -943,10 +943,10 @@ unsigned int kstat_irqs(unsigned int irq)if (!irq_settings_is_per_cpu_devid (desc) && !irq_settings_is_per_cpu (desc) && !irq_is_nmi (desc)) -return desc->tot_count; +return data_race (desc->tot_count);for_each_possible_cpu (cpu) - sum += *per_cpu_ptr (desc->kstat_irqs, cpu); + sum +=data_race (*per_cpu_ptr (desc->kstat_irqs, cpu));return sum; }
kernel/irq/proc.c +3 -2
@@ -488,9 +488,10 @@ int show_interrupts(struct seq_file *p, void *v)if (!desc ||irq_settings_is_hidden (desc)) goto outsparse; -if (desc->kstat_irqs) +if (desc->kstat_irqs) {for_each_online_cpu (j) - any_count |= *per_cpu_ptr (desc->kstat_irqs, j); + any_count |=data_race (*per_cpu_ptr (desc->kstat_irqs, j)); + }if ((!desc->action ||irq_desc_is_chained (desc)) && !any_count) goto outsparse;
PATCH 6/30 parisc/irq: Simplify irq count output for /proc/interrupts
In reply to Thomas GleixnerThe SMP variant works perfectly fine on UP as well.
--- arch/parisc/kernel/irq.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)
arch/parisc/kernel/irq.c +1 -4
@@ -216,12 +216,9 @@ int show_interrupts(struct seq_file *p,if (!action) goto skip;seq_printf (p,"%3d: " , i); -#ifdef CONFIG_SMP +for_each_online_cpu (j)seq_printf (p,"%10u " ,kstat_irqs_cpu (i, j)); -#else -seq_printf (p,"%10u " ,kstat_irqs (i)); -#endif seq_printf (p," %14s" ,irq_desc_get_chip (desc)->name);#ifndef PARISC_IRQ_CR16_COUNTS
[tip: irq/core] parisc/irq: Simplify irq count output for /proc/interrupts
The following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 89c1ca3058ced5cc8176de7b668b9d0ff08f8f4b Gitweb: https://git.kernel.org/tip/89c1ca3058ced5cc8176de7b668b9d0ff08f8f4b Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:25:42 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:03 +01:00
parisc/irq: Simplify irq count output for /proc/interrupts
The SMP variant works perfectly fine on UP as well.
--- arch/parisc/kernel/irq.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)
arch/parisc/kernel/irq.c +1 -4
@@ -216,12 +216,9 @@ int show_interrupts(struct seq_file *p, void *v)if (!action) goto skip;seq_printf (p,"%3d: " , i); -#ifdef CONFIG_SMP +for_each_online_cpu (j)seq_printf (p,"%10u " ,kstat_irqs_cpu (i, j)); -#else -seq_printf (p,"%10u " ,kstat_irqs (i)); -#endif seq_printf (p," %14s" ,irq_desc_get_chip (desc)->name);#ifndef PARISC_IRQ_CR16_COUNTS
PATCH 7/30 genirq: Make kstat_irqs() static
In reply to Thomas GleixnerNo more users outside the core code.
--- include/linux/kernel_stat.h | 1 - kernel/irq/irqdesc.c | 19 ++++++------------- 2 files changed, 6 insertions(+), 14 deletions(-)
include/linux/kernel_stat.h +0 -1
@@ -67,7 +67,6 @@ static inline unsigned int kstat_softirq/* * Number of interrupts per specific IRQ source, since bootup */ -extern unsigned int kstat_irqs (unsigned int irq);extern unsigned int kstat_irqs_usr (unsigned int irq);/*
kernel/irq/irqdesc.c +6 -13
@@ -924,15 +924,7 @@ static bool irq_is_nmi(struct irq_desc *return desc->istate &IRQS_NMI ; } -/** -* kstat_irqs - Get the statistics for an interrupt -* @irq: The interrupt number -* -* Returns the sum of interrupt counts on all cpus since boot for -* @irq. The caller must ensure that the interrupt is not removed -* concurrently. -*/ -unsigned int kstat_irqs (unsigned int irq) +static unsigned int kstat_irqs (unsigned int irq) {struct irq_desc *desc =irq_to_desc (irq);unsigned int sum =0 ;
@@ -951,13 +943,14 @@ unsigned int kstat_irqs(unsigned int irq }/** -* kstat_irqs_usr - Get the statistics for an interrupt +* kstat_irqs_usr - Get the statistics for an interrupt from thread context * @irq: The interrupt number * * Returns the sum of interrupt counts on all cpus since boot for @irq. -* Contrary to kstat_irqs() this can be called from any context. -* It uses rcu since a concurrent removal of an interrupt descriptor is -* observing an rcu grace period before delayed_free_desc()/irq_kobj_release(). +* +* It uses rcu to protect the access since a concurrent removal of an +* interrupt descriptor is observing an rcu grace period before +* delayed_free_desc()/irq_kobj_release(). */ unsigned int kstat_irqs_usr (unsigned int irq) {
[tip: irq/core] genirq: Make kstat_irqs() static
The following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 63ca4e066a62afce8ebaf75ebd4c6ef67f7bb5ee Gitweb: https://git.kernel.org/tip/63ca4e066a62afce8ebaf75ebd4c6ef67f7bb5ee Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:25:43 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:03 +01:00
genirq: Make kstat_irqs() static
No more users outside the core code.
--- include/linux/kernel_stat.h | 1 - kernel/irq/irqdesc.c | 19 ++++++------------- 2 files changed, 6 insertions(+), 14 deletions(-)
include/linux/kernel_stat.h +0 -1
@@ -67,7 +67,6 @@ static inline unsigned int kstat_softirqs_cpu(unsigned int irq, int cpu)/* * Number of interrupts per specific IRQ source, since bootup */ -extern unsigned int kstat_irqs (unsigned int irq);extern unsigned int kstat_irqs_usr (unsigned int irq);/*
kernel/irq/irqdesc.c +6 -13
@@ -924,15 +924,7 @@ static bool irq_is_nmi(struct irq_desc *desc)return desc->istate &IRQS_NMI ; } -/** -* kstat_irqs - Get the statistics for an interrupt -* @irq: The interrupt number -* -* Returns the sum of interrupt counts on all cpus since boot for -* @irq. The caller must ensure that the interrupt is not removed -* concurrently. -*/ -unsigned int kstat_irqs (unsigned int irq) +static unsigned int kstat_irqs (unsigned int irq) {struct irq_desc *desc =irq_to_desc (irq);unsigned int sum =0 ;
@@ -951,13 +943,14 @@ unsigned int kstat_irqs(unsigned int irq) }/** -* kstat_irqs_usr - Get the statistics for an interrupt +* kstat_irqs_usr - Get the statistics for an interrupt from thread context * @irq: The interrupt number * * Returns the sum of interrupt counts on all cpus since boot for @irq. -* Contrary to kstat_irqs() this can be called from any context. -* It uses rcu since a concurrent removal of an interrupt descriptor is -* observing an rcu grace period before delayed_free_desc()/irq_kobj_release(). +* +* It uses rcu to protect the access since a concurrent removal of an +* interrupt descriptor is observing an rcu grace period before +* delayed_free_desc()/irq_kobj_release(). */ unsigned int kstat_irqs_usr (unsigned int irq) {
PATCH 8/30 genirq: Provide kstat_irqdesc_cpu()
In reply to Thomas GleixnerMost users of kstat_irqs_cpu() have the irq descriptor already. No point in calling into the core code and looking it up once more.
Use it in per_cpu_count_show() to start with.
--- include/linux/irqdesc.h | 6 ++++++ kernel/irq/irqdesc.c | 4 ++-- 2 files changed, 8 insertions(+), 2 deletions(-)
include/linux/irqdesc.h +6 -0
@@ -113,6 +113,12 @@ static inline void irq_unlock_sparse(voiextern struct irq_desc irq_desc[NR_IRQS ];#endif +static inline unsigned int irq_desc_kstat_cpu (struct irq_desc *desc, +unsigned int cpu) +{ +return desc->kstat_irqs ? *per_cpu_ptr (desc->kstat_irqs, cpu) :0 ; +} +static inline struct irq_desc *irq_data_to_desc (struct irq_data *data) { return container_of(data->common, struct irq_desc, irq_common_data);
kernel/irq/irqdesc.c +2 -2
@@ -147,12 +147,12 @@ static ssize_t per_cpu_count_show(struct struct kobj_attribute *attr, char *buf) { struct irq_desc *desc = container_of(kobj, struct irq_desc, kobj); -int cpu, irq = desc->irq_data.irq;ssize_t ret =0 ;char *p ="" ; +int cpu;for_each_possible_cpu (cpu) { -unsigned int c =kstat_irqs_cpu (irq, cpu); +unsigned int c =irq_desc_kstat_cpu (desc, cpu); ret +=scnprintf (buf + ret,PAGE_SIZE - ret,"%s%u" , p, c); p ="," ;
[tip: irq/core] genirq: Provide kstat_irqdesc_cpu()
The following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: ea5f36556b8bde14234ec223ebc59968ade4735a Gitweb: https://git.kernel.org/tip/ea5f36556b8bde14234ec223ebc59968ade4735a Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:25:44 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:03 +01:00
genirq: Provide kstat_irqdesc_cpu()
Most users of kstat_irqs_cpu() have the irq descriptor already. No point in calling into the core code and looking it up once more.
Use it in per_cpu_count_show() to start with.
--- include/linux/irqdesc.h | 6 ++++++ kernel/irq/irqdesc.c | 4 ++-- 2 files changed, 8 insertions(+), 2 deletions(-)
include/linux/irqdesc.h +6 -0
@@ -113,6 +113,12 @@ static inline void irq_unlock_sparse(void) { }extern struct irq_desc irq_desc[NR_IRQS ];#endif +static inline unsigned int irq_desc_kstat_cpu (struct irq_desc *desc, +unsigned int cpu) +{ +return desc->kstat_irqs ? *per_cpu_ptr (desc->kstat_irqs, cpu) :0 ; +} +static inline struct irq_desc *irq_data_to_desc (struct irq_data *data) { return container_of(data->common, struct irq_desc, irq_common_data);
kernel/irq/irqdesc.c +2 -2
@@ -147,12 +147,12 @@ static ssize_t per_cpu_count_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) { struct irq_desc *desc = container_of(kobj, struct irq_desc, kobj); -int cpu, irq = desc->irq_data.irq;ssize_t ret =0 ;char *p ="" ; +int cpu;for_each_possible_cpu (cpu) { -unsigned int c =kstat_irqs_cpu (irq, cpu); +unsigned int c =irq_desc_kstat_cpu (desc, cpu); ret +=scnprintf (buf + ret,PAGE_SIZE - ret,"%s%u" , p, c); p ="," ;
PATCH 9/30 ARM: smp: Use irq_desc_kstat_cpu() in show_ipi_list()
In reply to Thomas GleixnerThe irq descriptor is already there, no need to look it up again.
--- arch/arm/kernel/smp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm/kernel/smp.c +1 -1
@@ -550,7 +550,7 @@ void show_ipi_list(struct seq_file *p, iseq_printf (p,"%*s%u: " , prec -1 ,"IPI" , i);for_each_online_cpu (cpu) -seq_printf (p,"%10u " ,kstat_irqs_cpu (irq, cpu)); +seq_printf (p,"%10u " ,irq_desc_kstat_cpu (ipi_desc[i], cpu));seq_printf (p," %s\n" , ipi_types[i]); }
Re: [patch 09/30] ARM: smp: Use irq_desc_kstat_cpu() in show_ipi_list()
Quoted 16 lines · Thomas Gleixner
The irq descriptor is already there, no need to look it up again.
Signed-off-by Thomas Gleixner <[email protected]>Cc Marc Zyngier <[email protected]>Cc Russell King <[email protected]>--- arch/arm/kernel/smp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm/kernel/smp.c void show_ipi_list(struct seq_file *p, i · line 556
Quoted 8 lines
arch/arm/kernel/smp.c +1
-1@@ -550,7 +550,7 @@ void show_ipi_list(struct seq_file *p, iseq_printf (p,"%*s%u: " , prec -1 ,"IPI" , i);for_each_online_cpu (cpu) -seq_printf (p,"%10u " ,kstat_irqs_cpu (irq, cpu)); +seq_printf (p,"%10u " ,irq_desc_kstat_cpu (ipi_desc[i], cpu));seq_printf (p," %s\n" , ipi_types[i]); }
Signature · 1 line
Without deviation from the norm, progress is not possible.
[tip: irq/core] ARM: smp: Use irq_desc_kstat_cpu() in show_ipi_list()
In reply to Thomas GleixnerThe following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: fe80f1d579dfd4edf78a66c3bd3befa985b4b02b Gitweb: https://git.kernel.org/tip/fe80f1d579dfd4edf78a66c3bd3befa985b4b02b Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:25:45 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:03 +01:00
ARM: smp: Use irq_desc_kstat_cpu() in show_ipi_list()
The irq descriptor is already there, no need to look it up again.
--- arch/arm/kernel/smp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm/kernel/smp.c +1 -1
@@ -550,7 +550,7 @@ void show_ipi_list(struct seq_file *p, int prec)seq_printf (p,"%*s%u: " , prec -1 ,"IPI" , i);for_each_online_cpu (cpu) -seq_printf (p,"%10u " ,kstat_irqs_cpu (irq, cpu)); +seq_printf (p,"%10u " ,irq_desc_kstat_cpu (ipi_desc[i], cpu));seq_printf (p," %s\n" , ipi_types[i]); }
PATCH 10/30 arm64/smp: Use irq_desc_kstat_cpu() in arch_show_interrupts()
In reply to Thomas GleixnerThe irq descriptor is already there, no need to look it up again.
--- arch/arm64/kernel/smp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm64/kernel/smp.c +1 -1
@@ -809,7 +809,7 @@ int arch_show_interrupts(struct seq_fileseq_printf (p,"%*s%u:%s" , prec -1 ,"IPI" , i, prec >=4 ?" " :"" );for_each_online_cpu (cpu) -seq_printf (p,"%10u " ,kstat_irqs_cpu (irq, cpu)); +seq_printf (p,"%10u " ,irq_desc_kstat_cpu (ipi_desc[i], cpu));seq_printf (p," %s\n" , ipi_types[i]); }
Re: [patch 10/30] arm64/smp: Use irq_desc_kstat_cpu() in arch_show_interrupts()
Quoted 18 lines · Thomas Gleixner
The irq descriptor is already there, no need to look it up again.
Signed-off-by Thomas Gleixner <[email protected]>Cc Mark Rutland <[email protected]>Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>Cc Marc Zyngier <[email protected]>--- arch/arm64/kernel/smp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm64/kernel/smp.c int arch_show_interrupts(struct seq_file · line 815
Quoted 8 lines
arch/arm64/kernel/smp.c +1
-1@@ -809,7 +809,7 @@ int arch_show_interrupts(struct seq_fileseq_printf (p,"%*s%u:%s" , prec -1 ,"IPI" , i, prec >=4 ?" " :"" );for_each_online_cpu (cpu) -seq_printf (p,"%10u " ,kstat_irqs_cpu (irq, cpu)); +seq_printf (p,"%10u " ,irq_desc_kstat_cpu (ipi_desc[i], cpu));seq_printf (p," %s\n" , ipi_types[i]); }
Signature · 1 line
Without deviation from the norm, progress is not possible.
[tip: irq/core] arm64/smp: Use irq_desc_kstat_cpu() in arch_show_interrupts()
In reply to Thomas GleixnerThe following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 2f516e34383d0e97ced094d18424838181ce51b7 Gitweb: https://git.kernel.org/tip/2f516e34383d0e97ced094d18424838181ce51b7 Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:25:46 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:04 +01:00
arm64/smp: Use irq_desc_kstat_cpu() in arch_show_interrupts()
The irq descriptor is already there, no need to look it up again.
--- arch/arm64/kernel/smp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm64/kernel/smp.c +1 -1
@@ -809,7 +809,7 @@ int arch_show_interrupts(struct seq_file *p, int prec)seq_printf (p,"%*s%u:%s" , prec -1 ,"IPI" , i, prec >=4 ?" " :"" );for_each_online_cpu (cpu) -seq_printf (p,"%10u " ,kstat_irqs_cpu (irq, cpu)); +seq_printf (p,"%10u " ,irq_desc_kstat_cpu (ipi_desc[i], cpu));seq_printf (p," %s\n" , ipi_types[i]); }
PATCH 11/30 parisc/irq: Use irq_desc_kstat_cpu() in show_interrupts()
In reply to Thomas GleixnerThe irq descriptor is already there, no need to look it up again.
--- arch/parisc/kernel/irq.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/parisc/kernel/irq.c +1 -1
@@ -218,7 +218,7 @@ int show_interrupts(struct seq_file *p,seq_printf (p,"%3d: " , i);for_each_online_cpu (j) -seq_printf (p,"%10u " ,kstat_irqs_cpu (i, j)); +seq_printf (p,"%10u " ,irq_desc_kstat_cpu (desc, j));seq_printf (p," %14s" ,irq_desc_get_chip (desc)->name);#ifndef PARISC_IRQ_CR16_COUNTS
[tip: irq/core] parisc/irq: Use irq_desc_kstat_cpu() in show_interrupts()
The following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 27fb43cb32d016366d1d87986970d5be7d6607e1 Gitweb: https://git.kernel.org/tip/27fb43cb32d016366d1d87986970d5be7d6607e1 Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:25:47 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:04 +01:00
parisc/irq: Use irq_desc_kstat_cpu() in show_interrupts()
The irq descriptor is already there, no need to look it up again.
--- arch/parisc/kernel/irq.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/parisc/kernel/irq.c +1 -1
@@ -218,7 +218,7 @@ int show_interrupts(struct seq_file *p, void *v)seq_printf (p,"%3d: " , i);for_each_online_cpu (j) -seq_printf (p,"%10u " ,kstat_irqs_cpu (i, j)); +seq_printf (p,"%10u " ,irq_desc_kstat_cpu (desc, j));seq_printf (p," %14s" ,irq_desc_get_chip (desc)->name);#ifndef PARISC_IRQ_CR16_COUNTS
PATCH 12/30 s390/irq: Use irq_desc_kstat_cpu() in show_msi_interrupt()
In reply to Thomas GleixnerThe irq descriptor is already there, no need to look it up again.
--- arch/s390/kernel/irq.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/s390/kernel/irq.c +1 -1
@@ -124,7 +124,7 @@ static void show_msi_interrupt(struct seraw_spin_lock_irqsave (&desc->lock, flags);seq_printf (p,"%3d: " , irq);for_each_online_cpu (cpu) -seq_printf (p,"%10u " ,kstat_irqs_cpu (irq, cpu)); +seq_printf (p,"%10u " ,irq_desc_kstat_irq (desc, cpu));if (desc->irq_data.chip)seq_printf (p," %8s" , desc->irq_data.chip->name);
Re: [patch 12/30] s390/irq: Use irq_desc_kstat_cpu() in show_msi_interrupt()
Quoted 9 lines · Thomas Gleixner
The irq descriptor is already there, no need to look it up again.
Signed-off-by Thomas Gleixner <[email protected]>Cc Christian Borntraeger <[email protected]>Cc Heiko Carstens <[email protected]>--- arch/s390/kernel/irq.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[tip: irq/core] s390/irq: Use irq_desc_kstat_cpu() in show_msi_interrupt()
In reply to Thomas GleixnerThe following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 7a0a6b22431b36549151df8f30cec4f667a018fc Gitweb: https://git.kernel.org/tip/7a0a6b22431b36549151df8f30cec4f667a018fc Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:25:48 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:04 +01:00
s390/irq: Use irq_desc_kstat_cpu() in show_msi_interrupt()
The irq descriptor is already there, no need to look it up again.
--- arch/s390/kernel/irq.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/s390/kernel/irq.c +1 -1
@@ -124,7 +124,7 @@ static void show_msi_interrupt(struct seq_file *p, int irq)raw_spin_lock_irqsave (&desc->lock, flags);seq_printf (p,"%3d: " , irq);for_each_online_cpu (cpu) -seq_printf (p,"%10u " ,kstat_irqs_cpu (irq, cpu)); +seq_printf (p,"%10u " ,irq_desc_kstat_irq (desc, cpu));if (desc->irq_data.chip)seq_printf (p," %8s" , desc->irq_data.chip->name);
PATCH 13/30 drm/i915/lpe_audio: Remove pointless irq_to_desc() usage
In reply to Thomas GleixnerNothing uses the result and nothing should ever use it in driver code.
--- drivers/gpu/drm/i915/display/intel_lpe_audio.c | 4 ---- 1 file changed, 4 deletions(-)
drivers/gpu/drm/i915/display/intel_lpe_audio.c +0 -4
@@ -297,13 +297,9 @@ int intel_lpe_audio_init(struct drm_i915*/ void intel_lpe_audio_teardown (struct drm_i915_private *dev_priv) { -struct irq_desc *desc; -if (!HAS_LPE_AUDIO (dev_priv))return ; - desc =irq_to_desc (dev_priv->lpe_audio.irq); -lpe_audio_platdev_destroy (dev_priv);irq_free_desc (dev_priv->lpe_audio.irq);
Re: [Intel-gfx] [patch 13/30] drm/i915/lpe_audio: Remove pointless irq_to_desc() usage
Quoted 13 lines · Thomas Gleixner
Nothing uses the result and nothing should ever use it in driver code.
Signed-off-by Thomas Gleixner <[email protected]>Cc Jani Nikula <[email protected]>Cc Joonas Lahtinen <[email protected]>Cc Rodrigo Vivi <[email protected]>Cc David Airlie <[email protected]>Cc Daniel Vetter <[email protected]>Cc Pankaj Bharadiya <[email protected]>Cc Chris Wilson <[email protected]>Cc Wambui Karuga <[email protected]>
Quoted 25 lines
--- drivers/gpu/drm/i915/display/intel_lpe_audio.c | 4 ---- 1 file changed, 4 deletions(-)drivers/gpu/drm/i915/display/intel_lpe_audio.c +0
-4@@ -297,13 +297,9 @@ int intel_lpe_audio_init(struct drm_i915*/ void intel_lpe_audio_teardown (struct drm_i915_private *dev_priv) { -struct irq_desc *desc; -if (!HAS_LPE_AUDIO (dev_priv))return ; - desc =irq_to_desc (dev_priv->lpe_audio.irq); -lpe_audio_platdev_destroy (dev_priv);irq_free_desc (dev_priv->lpe_audio.irq);_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Signature · 2 lines
Ville Syrj�l� Intel
Re: [Intel-gfx] [patch 13/30] drm/i915/lpe_audio: Remove pointless irq_to_desc() usage
Quoted 15 lines · Ville Syrjälä
Quoted 13 lines · Thomas Gleixner
Nothing uses the result and nothing should ever use it in driver code.
Signed-off-by Thomas Gleixner <[email protected]>Cc Jani Nikula <[email protected]>Cc Joonas Lahtinen <[email protected]>Cc Rodrigo Vivi <[email protected]>Cc David Airlie <[email protected]>Cc Daniel Vetter <[email protected]>Cc Pankaj Bharadiya <[email protected]>Cc Chris Wilson <[email protected]>Cc Wambui Karuga <[email protected]>
Quoted 1 line
Reviewed-by Ville Syrjälä <[email protected]>
Thomas, I presume you want to merge this series as a whole.
for merging via whichever tree makes most sense. Please let us know if you want us to pick this up via drm-intel instead.
Quoted 26 lines
Quoted 26 lines
--- drivers/gpu/drm/i915/display/intel_lpe_audio.c | 4 ---- 1 file changed, 4 deletions(-)drivers/gpu/drm/i915/display/intel_lpe_audio.c +0
-4@@ -297,13 +297,9 @@ int intel_lpe_audio_init(struct drm_i915*/ void intel_lpe_audio_teardown (struct drm_i915_private *dev_priv) { -struct irq_desc *desc; -if (!HAS_LPE_AUDIO (dev_priv))return ; - desc =irq_to_desc (dev_priv->lpe_audio.irq); -lpe_audio_platdev_destroy (dev_priv);irq_free_desc (dev_priv->lpe_audio.irq);_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Signature · 1 line
Jani Nikula, Intel Open Source Graphics Center
[tip: irq/core] drm/i915/lpe_audio: Remove pointless irq_to_desc() usage
In reply to Thomas GleixnerThe following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 766ba1f937be5ec418bd759e40fccc0c8c42e271 Gitweb: https://git.kernel.org/tip/766ba1f937be5ec418bd759e40fccc0c8c42e271 Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:25:49 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:04 +01:00
drm/i915/lpe_audio: Remove pointless irq_to_desc() usage
Nothing uses the result and nothing should ever use it in driver code.
--- drivers/gpu/drm/i915/display/intel_lpe_audio.c | 4 ---- 1 file changed, 4 deletions(-)
drivers/gpu/drm/i915/display/intel_lpe_audio.c +0 -4
@@ -297,13 +297,9 @@ int intel_lpe_audio_init(struct drm_i915_private *dev_priv)*/ void intel_lpe_audio_teardown (struct drm_i915_private *dev_priv) { -struct irq_desc *desc; -if (!HAS_LPE_AUDIO (dev_priv))return ; - desc =irq_to_desc (dev_priv->lpe_audio.irq); -lpe_audio_platdev_destroy (dev_priv);irq_free_desc (dev_priv->lpe_audio.irq);
PATCH 14/30 drm/i915/pmu: Replace open coded kstat_irqs() copy
In reply to Thomas GleixnerDriver code has no business with the internals of the irq descriptor.
Aside of that the count is per interrupt line and therefore takes interrupts from other devices into account which share the interrupt line and are not handled by the graphics driver.
Replace it with a pmu private count which only counts interrupts which originate from the graphics card.
To avoid atomics or heuristics of some sort make the counter field 'unsigned long'. That limits the count to 4e9 on 32bit which is a lot and postprocessing can easily deal with the occasional wraparound.
--- drivers/gpu/drm/i915/i915_irq.c | 34 ++++++++++++++++++++++++++++++++++ drivers/gpu/drm/i915/i915_pmu.c | 18 +----------------- drivers/gpu/drm/i915/i915_pmu.h | 8 ++++++++ 3 files changed, 43 insertions(+), 17 deletions(-)
drivers/gpu/drm/i915/i915_irq.c +34 -0
@@ -60,6 +60,24 @@* and related files, but that will be described in separate chapters. */ +/* +* Interrupt statistic for PMU. Increments the counter only if the +* interrupt originated from the the GPU so interrupts from a device which +* shares the interrupt line are not accounted. +*/ +static inline void pmu_irq_stats (struct drm_i915_private *priv, +irqreturn_t res) +{ +if (unlikely (res !=IRQ_HANDLED )) +return ; + +/* +* A clever compiler translates that into INC. A not so clever one +* should at least prevent store tearing. +*/ +WRITE_ONCE (priv->pmu.irq_count, priv->pmu.irq_count +1 ); +} +typedef bool (*long_pulse_detect_func )(enum hpd_pin pin,u32 val);static const u32 hpd_ilk[HPD_NUM_PINS ] = {
@@ -1599,6 +1617,8 @@ static irqreturn_t valleyview_irq_handlevalleyview_pipestat_irq_handler (dev_priv, pipe_stats); }while (0 ); +pmu_irq_stats (dev_priv, ret); +enable_rpm_wakeref_asserts (&dev_priv->runtime_pm);return ret;
@@ -1676,6 +1696,8 @@ static irqreturn_t cherryview_irq_handlevalleyview_pipestat_irq_handler (dev_priv, pipe_stats); }while (0 ); +pmu_irq_stats (dev_priv, ret); +enable_rpm_wakeref_asserts (&dev_priv->runtime_pm);return ret;
@@ -2103,6 +2125,8 @@ static irqreturn_t ilk_irq_handler(int iif (sde_ier)raw_reg_write (regs,SDEIER , sde_ier); +pmu_irq_stats (i915, ret); +/* IRQs are synced during runtime_suspend, we don't require a wakeref */ enable_rpm_wakeref_asserts (&i915->runtime_pm);
@@ -2419,6 +2443,8 @@ static irqreturn_t gen8_irq_handler(intgen8_master_intr_enable (regs); +pmu_irq_stats (dev_priv,IRQ_HANDLED ); +return IRQ_HANDLED ; }
@@ -2514,6 +2540,8 @@ static __always_inline irqreturn_tgen11_gu_misc_irq_handler (gt, gu_misc_iir); +pmu_irq_stats (i915,IRQ_HANDLED ); +return IRQ_HANDLED ; }
@@ -3688,6 +3716,8 @@ static irqreturn_t i8xx_irq_handler(inti8xx_pipestat_irq_handler (dev_priv, iir, pipe_stats); }while (0 ); +pmu_irq_stats (dev_priv, ret); +enable_rpm_wakeref_asserts (&dev_priv->runtime_pm);return ret;
@@ -3796,6 +3826,8 @@ static irqreturn_t i915_irq_handler(inti915_pipestat_irq_handler (dev_priv, iir, pipe_stats); }while (0 ); +pmu_irq_stats (dev_priv, ret); +enable_rpm_wakeref_asserts (&dev_priv->runtime_pm);return ret;
@@ -3941,6 +3973,8 @@ static irqreturn_t i965_irq_handler(inti965_pipestat_irq_handler (dev_priv, iir, pipe_stats); }while (0 ); +pmu_irq_stats (dev_priv,IRQ_HANDLED ); +enable_rpm_wakeref_asserts (&dev_priv->runtime_pm);return ret;
drivers/gpu/drm/i915/i915_pmu.c +1 -17
@@ -423,22 +423,6 @@ static enum hrtimer_restart i915_sample(return HRTIMER_RESTART ; } -static u64 count_interrupts (struct drm_i915_private *i915) -{ -/* open-coded kstat_irqs() */ -struct irq_desc *desc =irq_to_desc (i915->drm.pdev->irq); -u64 sum =0 ; -int cpu; - -if (!desc || !desc->kstat_irqs) -return 0 ; - -for_each_possible_cpu (cpu) - sum += *per_cpu_ptr (desc->kstat_irqs, cpu); - -return sum; -} -static void i915_pmu_event_destroy (struct perf_event *event) { struct drm_i915_private *i915 =
@@ -581,7 +565,7 @@ static u64 __i915_pmu_event_read(struct USEC_PER_SEC /* to MHz */);break ;case I915_PMU_INTERRUPTS : - val =count_interrupts (i915); + val =READ_ONCE (pmu->irq_count);break ;case I915_PMU_RC6_RESIDENCY : val =get_rc6 (&i915->gt);
drivers/gpu/drm/i915/i915_pmu.h +8 -0
@@ -108,6 +108,14 @@ struct i915_pmu {*/ ktime_t sleep_last;/** +* @irq_count: Number of interrupts +* +* Intentionally unsigned long to avoid atomics or heuristics on 32bit. +* 4e9 interrupts are a lot and postprocessing can really deal with an +* occasional wraparound easily. It's 32bit after all. +*/ +unsigned long irq_count; +/** * @events_attr_group: Device events attribute group. */ struct attribute_group events_attr_group;
Re: [patch 14/30] drm/i915/pmu: Replace open coded kstat_irqs() copy
Quoted 6 lines · Thomas Gleixner
Driver code has no business with the internals of the irq descriptor.
Aside of that the count is per interrupt line and therefore takes interrupts from other devices into account which share the interrupt line and are not handled by the graphics driver.
Quoted 6 lines
Replace it with a pmu private count which only counts interrupts which originate from the graphics card.
To avoid atomics or heuristics of some sort make the counter field 'unsigned long'. That limits the count to 4e9 on 32bit which is a lot and postprocessing can easily deal with the occasional wraparound.
I'll let Tvrtko and Chris review the substance here, but assuming they don't object,
for merging via whichever tree makes most sense.
Quoted 167 lines
Signed-off-by Thomas Gleixner <[email protected]>Cc Tvrtko Ursulin <[email protected]>Cc Jani Nikula <[email protected]>Cc Joonas Lahtinen <[email protected]>Cc Rodrigo Vivi <[email protected]>Cc David Airlie <[email protected]>Cc Daniel Vetter <[email protected]>--- drivers/gpu/drm/i915/i915_irq.c | 34 ++++++++++++++++++++++++++++++++++ drivers/gpu/drm/i915/i915_pmu.c | 18 +----------------- drivers/gpu/drm/i915/i915_pmu.h | 8 ++++++++ 3 files changed, 43 insertions(+), 17 deletions(-)drivers/gpu/drm/i915/i915_irq.c +34
-0@@ -60,6 +60,24 @@* and related files, but that will be described in separate chapters. */ +/* +* Interrupt statistic for PMU. Increments the counter only if the +* interrupt originated from the the GPU so interrupts from a device which +* shares the interrupt line are not accounted. +*/ +static inline void pmu_irq_stats (struct drm_i915_private *priv, +irqreturn_t res) +{ +if (unlikely (res !=IRQ_HANDLED )) +return ; + +/* +* A clever compiler translates that into INC. A not so clever one +* should at least prevent store tearing. +*/ +WRITE_ONCE (priv->pmu.irq_count, priv->pmu.irq_count +1 ); +} +typedef bool (*long_pulse_detect_func )(enum hpd_pin pin,u32 val);static const u32 hpd_ilk[HPD_NUM_PINS ] = {@@ -1599,6 +1617,8 @@ static irqreturn_t valleyview_irq_handlevalleyview_pipestat_irq_handler (dev_priv, pipe_stats); }while (0 ); +pmu_irq_stats (dev_priv, ret); +enable_rpm_wakeref_asserts (&dev_priv->runtime_pm);return ret;@@ -1676,6 +1696,8 @@ static irqreturn_t cherryview_irq_handlevalleyview_pipestat_irq_handler (dev_priv, pipe_stats); }while (0 ); +pmu_irq_stats (dev_priv, ret); +enable_rpm_wakeref_asserts (&dev_priv->runtime_pm);return ret;@@ -2103,6 +2125,8 @@ static irqreturn_t ilk_irq_handler(int iif (sde_ier)raw_reg_write (regs,SDEIER , sde_ier); +pmu_irq_stats (i915, ret); +/* IRQs are synced during runtime_suspend, we don't require a wakeref */ enable_rpm_wakeref_asserts (&i915->runtime_pm);@@ -2419,6 +2443,8 @@ static irqreturn_t gen8_irq_handler(intgen8_master_intr_enable (regs); +pmu_irq_stats (dev_priv,IRQ_HANDLED ); +return IRQ_HANDLED ; }@@ -2514,6 +2540,8 @@ static __always_inline irqreturn_tgen11_gu_misc_irq_handler (gt, gu_misc_iir); +pmu_irq_stats (i915,IRQ_HANDLED ); +return IRQ_HANDLED ; }@@ -3688,6 +3716,8 @@ static irqreturn_t i8xx_irq_handler(inti8xx_pipestat_irq_handler (dev_priv, iir, pipe_stats); }while (0 ); +pmu_irq_stats (dev_priv, ret); +enable_rpm_wakeref_asserts (&dev_priv->runtime_pm);return ret;@@ -3796,6 +3826,8 @@ static irqreturn_t i915_irq_handler(inti915_pipestat_irq_handler (dev_priv, iir, pipe_stats); }while (0 ); +pmu_irq_stats (dev_priv, ret); +enable_rpm_wakeref_asserts (&dev_priv->runtime_pm);return ret;@@ -3941,6 +3973,8 @@ static irqreturn_t i965_irq_handler(inti965_pipestat_irq_handler (dev_priv, iir, pipe_stats); }while (0 ); +pmu_irq_stats (dev_priv,IRQ_HANDLED ); +enable_rpm_wakeref_asserts (&dev_priv->runtime_pm);return ret;drivers/gpu/drm/i915/i915_pmu.c +1
-17@@ -423,22 +423,6 @@ static enum hrtimer_restart i915_sample(return HRTIMER_RESTART ; } -static u64 count_interrupts (struct drm_i915_private *i915) -{ -/* open-coded kstat_irqs() */ -struct irq_desc *desc =irq_to_desc (i915->drm.pdev->irq); -u64 sum =0 ; -int cpu; - -if (!desc || !desc->kstat_irqs) -return 0 ; - -for_each_possible_cpu (cpu) - sum += *per_cpu_ptr (desc->kstat_irqs, cpu); - -return sum; -} -static void i915_pmu_event_destroy (struct perf_event *event) { struct drm_i915_private *i915 =@@ -581,7 +565,7 @@ static u64 __i915_pmu_event_read(struct USEC_PER_SEC /* to MHz */);break ;case I915_PMU_INTERRUPTS : - val =count_interrupts (i915); + val =READ_ONCE (pmu->irq_count);break ;case I915_PMU_RC6_RESIDENCY : val =get_rc6 (&i915->gt);drivers/gpu/drm/i915/i915_pmu.h +8
-0@@ -108,6 +108,14 @@ struct i915_pmu {*/ ktime_t sleep_last;/** +* @irq_count: Number of interrupts +* +* Intentionally unsigned long to avoid atomics or heuristics on 32bit. +* 4e9 interrupts are a lot and postprocessing can really deal with an +* occasional wraparound easily. It's 32bit after all. +*/ +unsigned long irq_count; +/** * @events_attr_group: Device events attribute group. */ struct attribute_group events_attr_group;
Signature · 1 line
Jani Nikula, Intel Open Source Graphics Center
Re: [patch 14/30] drm/i915/pmu: Replace open coded kstat_irqs() copy
In reply to Thomas GleixnerQuoted 6 lines · Thomas Gleixner
Driver code has no business with the internals of the irq descriptor.
Aside of that the count is per interrupt line and therefore takes interrupts from other devices into account which share the interrupt line and are not handled by the graphics driver.
Quoted 6 lines
Replace it with a pmu private count which only counts interrupts which originate from the graphics card.
To avoid atomics or heuristics of some sort make the counter field 'unsigned long'. That limits the count to 4e9 on 32bit which is a lot and postprocessing can easily deal with the occasional wraparound.
After my failed hasty sketch from last night I had a different one which was kind of heuristics based (re-reading the upper dword and retrying if it changed on 32-bit). But you are right - it is okay to at least start like this today and if later there is a need we can either do that or deal with wrap at PMU read time.
So thanks for dealing with it, some small comments below but overall it is fine.
Quoted 18 lines
Signed-off-by Thomas Gleixner <[email protected]>Cc Tvrtko Ursulin <[email protected]>Cc Jani Nikula <[email protected]>Cc Joonas Lahtinen <[email protected]>Cc Rodrigo Vivi <[email protected]>Cc David Airlie <[email protected]>Cc Daniel Vetter <[email protected]>--- drivers/gpu/drm/i915/i915_irq.c | 34 ++++++++++++++++++++++++++++++++++ drivers/gpu/drm/i915/i915_pmu.c | 18 +----------------- drivers/gpu/drm/i915/i915_pmu.h | 8 ++++++++ 3 files changed, 43 insertions(+), 17 deletions(-)
drivers/gpu/drm/i915/i915_irq.c line 68
Quoted 9 lines
drivers/gpu/drm/i915/i915_irq.c +6
-0@@ -60,6 +60,24 @@* and related files, but that will be described in separate chapters. */ +/* +* Interrupt statistic for PMU. Increments the counter only if the +* interrupt originated from the the GPU so interrupts from a device which +* shares the interrupt line are not accounted. +*/ +static inline void pmu_irq_stats(struct drm_i915_private *priv,
We never use priv as a local name, it should be either i915 or dev_priv.
drivers/gpu/drm/i915/i915_irq.c line 78
Quoted 10 lines
drivers/gpu/drm/i915/i915_irq.c +10
-0+ irqreturn_t res) +{ +if (unlikely (res !=IRQ_HANDLED )) +return ; + +/* +* A clever compiler translates that into INC. A not so clever one +* should at least prevent store tearing. +*/ +WRITE_ONCE (priv->pmu.irq_count, priv->pmu.irq_count +1 );
Curious, probably more educational for me - given x86_32 and x86_64, and the context of it getting called, what is the difference from just doing irq_count++?
Quoted 80 lines
drivers/gpu/drm/i915/i915_irq.c +18
-0+} +typedef bool (*long_pulse_detect_func )(enum hpd_pin pin,u32 val);static const u32 hpd_ilk[HPD_NUM_PINS ] = {@@ -1599,6 +1617,8 @@ static irqreturn_t valleyview_irq_handlevalleyview_pipestat_irq_handler (dev_priv, pipe_stats); }while (0 ); +pmu_irq_stats (dev_priv, ret); +enable_rpm_wakeref_asserts (&dev_priv->runtime_pm);return ret;@@ -1676,6 +1696,8 @@ static irqreturn_t cherryview_irq_handlevalleyview_pipestat_irq_handler (dev_priv, pipe_stats); }while (0 ); +pmu_irq_stats (dev_priv, ret); +enable_rpm_wakeref_asserts (&dev_priv->runtime_pm);return ret;@@ -2103,6 +2125,8 @@ static irqreturn_t ilk_irq_handler(int iif (sde_ier)raw_reg_write (regs,SDEIER , sde_ier); +pmu_irq_stats (i915, ret); +/* IRQs are synced during runtime_suspend, we don't require a wakeref */ enable_rpm_wakeref_asserts (&i915->runtime_pm);@@ -2419,6 +2443,8 @@ static irqreturn_t gen8_irq_handler(intgen8_master_intr_enable (regs); +pmu_irq_stats (dev_priv,IRQ_HANDLED ); +return IRQ_HANDLED ; }@@ -2514,6 +2540,8 @@ static __always_inline irqreturn_tgen11_gu_misc_irq_handler (gt, gu_misc_iir); +pmu_irq_stats (i915,IRQ_HANDLED ); +return IRQ_HANDLED ; }@@ -3688,6 +3716,8 @@ static irqreturn_t i8xx_irq_handler(inti8xx_pipestat_irq_handler (dev_priv, iir, pipe_stats); }while (0 ); +pmu_irq_stats (dev_priv, ret); +enable_rpm_wakeref_asserts (&dev_priv->runtime_pm);return ret;@@ -3796,6 +3826,8 @@ static irqreturn_t i915_irq_handler(inti915_pipestat_irq_handler (dev_priv, iir, pipe_stats); }while (0 ); +pmu_irq_stats (dev_priv, ret); +enable_rpm_wakeref_asserts (&dev_priv->runtime_pm);return ret;@@ -3941,6 +3973,8 @@ static irqreturn_t i965_irq_handler(inti965_pipestat_irq_handler (dev_priv, iir, pipe_stats); }while (0 ); +pmu_irq_stats (dev_priv,IRQ_HANDLED ); +enable_rpm_wakeref_asserts (&dev_priv->runtime_pm);return ret;
drivers/gpu/drm/i915/i915_pmu.c static enum hrtimer_restart i915_sample( · line 424
Quoted 2 lines
drivers/gpu/drm/i915/i915_pmu.c +0
-0@@ -423,22 +423,6 @@ static enum hrtimer_restart i915_sample(return HRTIMER_RESTART ; }
In this file you can also drop the #include <linux/irq.h> line.
Quoted 21 lines
drivers/gpu/drm/i915/i915_pmu.c +0
-16-static u64 count_interrupts (struct drm_i915_private *i915) -{ -/* open-coded kstat_irqs() */ -struct irq_desc *desc =irq_to_desc (i915->drm.pdev->irq); -u64 sum =0 ; -int cpu; - -if (!desc || !desc->kstat_irqs) -return 0 ; - -for_each_possible_cpu (cpu) - sum += *per_cpu_ptr (desc->kstat_irqs, cpu); - -return sum; -} -static void i915_pmu_event_destroy (struct perf_event *event) { struct drm_i915_private *i915 =
drivers/gpu/drm/i915/i915_pmu.c static u64 __i915_pmu_event_read(struct · line 568
Quoted 5 lines
drivers/gpu/drm/i915/i915_pmu.c +1
-1@@ -581,7 +565,7 @@ static u64 __i915_pmu_event_read(struct USEC_PER_SEC /* to MHz */);break ;case I915_PMU_INTERRUPTS : - val =count_interrupts (i915); + val =READ_ONCE (pmu->irq_count);
I guess same curiosity about READ_ONCE like in the increment site.
Quoted 11 lines
drivers/gpu/drm/i915/i915_pmu.c +0
-0break ;case I915_PMU_RC6_RESIDENCY : val =get_rc6 (&i915->gt);drivers/gpu/drm/i915/i915_pmu.h +1
-0@@ -108,6 +108,14 @@ struct i915_pmu {*/ ktime_t sleep_last;/** +* @irq_count: Number of interrupts
drivers/gpu/drm/i915/i915_pmu.h struct i915_pmu { · line 121
Quoted 10 lines
drivers/gpu/drm/i915/i915_pmu.h +7
-0+* +* Intentionally unsigned long to avoid atomics or heuristics on 32bit. +* 4e9 interrupts are a lot and postprocessing can really deal with an +* occasional wraparound easily. It's 32bit after all. +*/ +unsigned long irq_count; +/** * @events_attr_group: Device events attribute group. */ struct attribute_group events_attr_group;
Regards,
Tvrtko
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [patch 14/30] drm/i915/pmu: Replace open coded kstat_irqs() copy
Quoted 10 lines · Tvrtko Ursulin
Quoted 8 lines · Thomas Gleixner
Aside of that the count is per interrupt line and therefore takes interrupts from other devices into account which share the interrupt line and are not handled by the graphics driver.
Replace it with a pmu private count which only counts interrupts which originate from the graphics card.
Quoted 7 lines
Quoted 3 lines
To avoid atomics or heuristics of some sort make the counter field 'unsigned long'. That limits the count to 4e9 on 32bit which is a lot and postprocessing can easily deal with the occasional wraparound.
After my failed hasty sketch from last night I had a different one which was kind of heuristics based (re-reading the upper dword and retrying if it changed on 32-bit).
The problem is that there will be two seperate modifications for the low and high word. Several ways how the compiler can translate this, but the problem is the same for all of them:
CPU 0 CPU 1
load low
load high
add low, 1
addc high, 0
store low load high
--> NMI load low
load high and compare
store high
You can't catch that. If this really becomes an issue you need a sequence counter around it.
Quoted 3 lines
But you are right - it is okay to at least start
like this today and if later there is a need we can either do that or deal with wrap at PMU read time.
Right.
Quoted 9 lines
drivers/gpu/drm/i915/i915_irq.c line 68
Quoted 6 lines
drivers/gpu/drm/i915/i915_irq.c +6
-0+/* +* Interrupt statistic for PMU. Increments the counter only if the +* interrupt originated from the the GPU so interrupts from a device which +* shares the interrupt line are not accounted. +*/ +static inline void pmu_irq_stats(struct drm_i915_private *priv,We never use priv as a local name, it should be either i915 or
dev_priv.
Sure, will fix.
Quoted 9 lines
drivers/gpu/drm/i915/i915_irq.c line 78
Quoted 5 lines
drivers/gpu/drm/i915/i915_irq.c +5
-0+/* +* A clever compiler translates that into INC. A not so clever one +* should at least prevent store tearing. +*/ +WRITE_ONCE (priv->pmu.irq_count, priv->pmu.irq_count +1 );Curious, probably more educational for me - given
x86_32andx86_64, and the context of it getting called, what is the difference from just doing irq_count++?
Several reasons:
1) The compiler can pretty much do what it wants with cnt++ including tearing and whatever. https://lwn.net/Articles/816850/
for the full set of insanities.
Not really a problem here, but
2) It's annotating the reader and the writer side and documenting that this is subject to concurrency
3) It will prevent KCSAN to complain about the data race, i.e. concurrent modification while reading.
Thanks,
tglx
Quoted 7 lines
drivers/gpu/drm/i915/i915_pmu.c static enum hrtimer_restart i915_sample( · line 424
Quoted 5 lines
drivers/gpu/drm/i915/i915_pmu.c +0
-0@@ -423,22 +423,6 @@ static enum hrtimer_restart i915_sample(return HRTIMER_RESTART ; }In this file you can also drop the #include <
linux/irq.h> line.
Indeed.
Thanks,
tglx
RE: [patch 14/30] drm/i915/pmu: Replace open coded kstat_irqs() copy
From: Thomas Gleixner
Quoted 1 line
Sent: 11 December 2020 12:58
..
Quoted 18 lines
Quoted 3 lines
After my failed hasty sketch from last night I had a different one which was kind of heuristics based (re-reading the upper dword and retrying if it changed on 32-bit).
The problem is that there will be two seperate modifications for the low and high word. Several ways how the compiler can translate this, but the problem is the same for all of them:
CPU 0 CPU 1
load low
load high
add low, 1
addc high, 0
store low load high
--> NMI load low
load high and compare
store high
Quoted 2 lines
You can't catch that. If this really becomes an issue you need a sequence counter around it.
Or just two copies of the high word.
Provided the accesses are sequenced:
writer:
load high:low
add small_value,high:low
store high
store low
store high_copy
reader:
load high_copy load low load high if (high != high_copy) low = 0;
The read value is always stale, so it probably doesn't matter that the value you have is one that is between the value when you started and that when you finished.
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
RE: [patch 14/30] drm/i915/pmu: Replace open coded kstat_irqs() copy
Quoted 13 lines · David Laight
From: Thomas Gleixner
Quoted 2 lines
You can't catch that. If this really becomes an issue you need a sequence counter around it.
Or just two copies of the high word.
Provided the accesses are sequenced:
writer:
load high:low
add small_value,high:low
store high
store low
storehigh_copy
reader:
Quoted 5 lines
load high_copy load low load high if (high != high_copy) low = 0;
And low = 0 is solving what? You need to loop back and retry until it's consistent and then it's nothing else than an open coded sequence count.
Thanks,
tglx
RE: [patch 14/30] drm/i915/pmu: Replace open coded kstat_irqs() copy
From: Thomas Gleixner
Quoted 22 lines
Sent: 11 December 2020 21:11
Quoted 13 lines · David Laight
From: Thomas Gleixner
Quoted 2 lines
You can't catch that. If this really becomes an issue you need a sequence counter around it.
Or just two copies of the high word.
Provided the accesses are sequenced:
writer:
load high:low
add small_value,high:low
store high
store low
storehigh_copy
reader:Quoted 5 lines
load high_copy load low load high if (high != high_copy) low = 0;
Quoted 2 lines
And low = 0 is solving what? You need to loop back and retry until it's consistent and then it's nothing else than an open coded sequence count.
If it is a counter or timestamp then the high:0 value happened some time between when you started trying to read the value and when you finished trying to read it.
As such it is a perfectly reasonable return value.
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
[tip: irq/core] drm/i915/pmu: Replace open coded kstat_irqs() copy
In reply to Thomas GleixnerThe following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: b64217174467e1c4c3da04a7f7835393fa234818 Gitweb: https://git.kernel.org/tip/b64217174467e1c4c3da04a7f7835393fa234818 Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:25:50 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:04 +01:00
drm/i915/pmu: Replace open coded kstat_irqs() copy
Driver code has no business with the internals of the irq descriptor.
Aside of that the count is per interrupt line and therefore takes interrupts from other devices into account which share the interrupt line and are not handled by the graphics driver.
Replace it with a pmu private count which only counts interrupts which originate from the graphics card.
To avoid atomics or heuristics of some sort make the counter field 'unsigned long'. That limits the count to 4e9 on 32bit which is a lot and postprocessing can easily deal with the occasional wraparound.
--- drivers/gpu/drm/i915/i915_irq.c | 34 ++++++++++++++++++++++++++++++++- drivers/gpu/drm/i915/i915_pmu.c | 19 +------------------ drivers/gpu/drm/i915/i915_pmu.h | 8 ++++++++- 3 files changed, 43 insertions(+), 18 deletions(-)
drivers/gpu/drm/i915/i915_irq.c +34 -0
@@ -60,6 +60,24 @@* and related files, but that will be described in separate chapters. */ +/* +* Interrupt statistic for PMU. Increments the counter only if the +* interrupt originated from the the GPU so interrupts from a device which +* shares the interrupt line are not accounted. +*/ +static inline void pmu_irq_stats (struct drm_i915_private *i915, +irqreturn_t res) +{ +if (unlikely (res !=IRQ_HANDLED )) +return ; + +/* +* A clever compiler translates that into INC. A not so clever one +* should at least prevent store tearing. +*/ +WRITE_ONCE (i915->pmu.irq_count, i915->pmu.irq_count +1 ); +} +typedef bool (*long_pulse_detect_func )(enum hpd_pin pin,u32 val);static const u32 hpd_ilk[HPD_NUM_PINS ] = {
@@ -1599,6 +1617,8 @@ static irqreturn_t valleyview_irq_handler(int irq, void *arg)valleyview_pipestat_irq_handler (dev_priv, pipe_stats); }while (0 ); +pmu_irq_stats (dev_priv, ret); +enable_rpm_wakeref_asserts (&dev_priv->runtime_pm);return ret;
@@ -1676,6 +1696,8 @@ static irqreturn_t cherryview_irq_handler(int irq, void *arg)valleyview_pipestat_irq_handler (dev_priv, pipe_stats); }while (0 ); +pmu_irq_stats (dev_priv, ret); +enable_rpm_wakeref_asserts (&dev_priv->runtime_pm);return ret;
@@ -2103,6 +2125,8 @@ static irqreturn_t ilk_irq_handler(int irq, void *arg)if (sde_ier)raw_reg_write (regs,SDEIER , sde_ier); +pmu_irq_stats (i915, ret); +/* IRQs are synced during runtime_suspend, we don't require a wakeref */ enable_rpm_wakeref_asserts (&i915->runtime_pm);
@@ -2419,6 +2443,8 @@ static irqreturn_t gen8_irq_handler(int irq, void *arg)gen8_master_intr_enable (regs); +pmu_irq_stats (dev_priv,IRQ_HANDLED ); +return IRQ_HANDLED ; }
@@ -2514,6 +2540,8 @@ __gen11_irq_handler(struct drm_i915_private * const i915,gen11_gu_misc_irq_handler (gt, gu_misc_iir); +pmu_irq_stats (i915,IRQ_HANDLED ); +return IRQ_HANDLED ; }
@@ -3688,6 +3716,8 @@ static irqreturn_t i8xx_irq_handler(int irq, void *arg)i8xx_pipestat_irq_handler (dev_priv, iir, pipe_stats); }while (0 ); +pmu_irq_stats (dev_priv, ret); +enable_rpm_wakeref_asserts (&dev_priv->runtime_pm);return ret;
@@ -3796,6 +3826,8 @@ static irqreturn_t i915_irq_handler(int irq, void *arg)i915_pipestat_irq_handler (dev_priv, iir, pipe_stats); }while (0 ); +pmu_irq_stats (dev_priv, ret); +enable_rpm_wakeref_asserts (&dev_priv->runtime_pm);return ret;
@@ -3941,6 +3973,8 @@ static irqreturn_t i965_irq_handler(int irq, void *arg)i965_pipestat_irq_handler (dev_priv, iir, pipe_stats); }while (0 ); +pmu_irq_stats (dev_priv,IRQ_HANDLED ); +enable_rpm_wakeref_asserts (&dev_priv->runtime_pm);return ret;
drivers/gpu/drm/i915/i915_pmu.c +1 -18
@@ -4,7 +4,6 @@* Copyright © 2017-2018 Intel Corporation */ -#include <linux/irq.h> #include <linux/pm_runtime.h> #include "gt/intel_engine.h"
@@ -423,22 +422,6 @@ static enum hrtimer_restart i915_sample(struct hrtimer *hrtimer)return HRTIMER_RESTART ; } -static u64 count_interrupts (struct drm_i915_private *i915) -{ -/* open-coded kstat_irqs() */ -struct irq_desc *desc =irq_to_desc (i915->drm.pdev->irq); -u64 sum =0 ; -int cpu; - -if (!desc || !desc->kstat_irqs) -return 0 ; - -for_each_possible_cpu (cpu) - sum += *per_cpu_ptr (desc->kstat_irqs, cpu); - -return sum; -} -static void i915_pmu_event_destroy (struct perf_event *event) { struct drm_i915_private *i915 =
@@ -581,7 +564,7 @@ static u64 __i915_pmu_event_read(struct perf_event *event) USEC_PER_SEC /* to MHz */);break ;case I915_PMU_INTERRUPTS : - val =count_interrupts (i915); + val =READ_ONCE (pmu->irq_count);break ;case I915_PMU_RC6_RESIDENCY : val =get_rc6 (&i915->gt);
drivers/gpu/drm/i915/i915_pmu.h +8 -0
@@ -108,6 +108,14 @@ struct i915_pmu {*/ ktime_t sleep_last;/** +* @irq_count: Number of interrupts +* +* Intentionally unsigned long to avoid atomics or heuristics on 32bit. +* 4e9 interrupts are a lot and postprocessing can really deal with an +* occasional wraparound easily. It's 32bit after all. +*/ +unsigned long irq_count; +/** * @events_attr_group: Device events attribute group. */ struct attribute_group events_attr_group;
PATCH 15/30 pinctrl: nomadik: Use irq_has_action()
In reply to Thomas GleixnerLet the core code do the fiddling with irq_desc.
--- drivers/pinctrl/nomadik/pinctrl-nomadik.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
drivers/pinctrl/nomadik/pinctrl-nomadik.c +1 -2
@@ -948,7 +948,6 @@ static void nmk_gpio_dbg_show_one(struct (mode < 0) ? "unknown" : modes[mode]); }else {int irq = chip->to_irq (chip, offset); -struct irq_desc *desc =irq_to_desc (irq);const int pullidx = pull ?1 :0 ;int val; static const char * const pulls[] = {
@@ -969,7 +968,7 @@ static void nmk_gpio_dbg_show_one(struct* This races with request_irq(), set_irq_type(), * and set_irq_wake() ... but those are "rare". */ -if (irq >0 && desc && desc->action) { +if (irq >0 &&irq_has_action (irq)) {char *trigger;if (nmk_chip->edge_rising &BIT (offset))
Re: [patch 15/30] pinctrl: nomadik: Use irq_has_action()
Quoted 6 lines · Thomas Gleixner
Let the core code do the fiddling with
irq_desc.Signed-off-by Thomas Gleixner <[email protected]>Cc Linus Walleij <[email protected]>
I suppose you will funnel this directly to Torvalds, else tell me and I'll apply it to my tree.
Yours,
Linus Walleij
[tip: irq/core] pinctrl: nomadik: Use irq_has_action()
In reply to Thomas GleixnerThe following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 52ae486b230b6d9508068d49da850fb17d25a174 Gitweb: https://git.kernel.org/tip/52ae486b230b6d9508068d49da850fb17d25a174 Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:25:51 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:04 +01:00
pinctrl: nomadik: Use irq_has_action()
Let the core code do the fiddling with irq_desc.
--- drivers/pinctrl/nomadik/pinctrl-nomadik.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
drivers/pinctrl/nomadik/pinctrl-nomadik.c +1 -2
@@ -948,7 +948,6 @@ static void nmk_gpio_dbg_show_one(struct seq_file *s, (mode < 0) ? "unknown" : modes[mode]); }else {int irq = chip->to_irq (chip, offset); -struct irq_desc *desc =irq_to_desc (irq);const int pullidx = pull ?1 :0 ;int val; static const char * const pulls[] = {
@@ -969,7 +968,7 @@ static void nmk_gpio_dbg_show_one(struct seq_file *s,* This races with request_irq(), set_irq_type(), * and set_irq_wake() ... but those are "rare". */ -if (irq >0 && desc && desc->action) { +if (irq >0 &&irq_has_action (irq)) {char *trigger;if (nmk_chip->edge_rising &BIT (offset))
PATCH 16/30 mfd: ab8500-debugfs: Remove the racy fiddling with irq_desc
In reply to Thomas GleixnerFirst of all drivers have absolutely no business to dig into the internals of an irq descriptor. That's core code and subject to change. All of this information is readily available to /proc/interrupts in a safe and race free way.
Remove the inspection code which is a blatant violation of subsystem boundaries and racy against concurrent modifications of the interrupt descriptor.
Print the irq line instead so the information can be looked up in a sane way in /proc/interrupts.
--- drivers/mfd/ab8500-debugfs.c | 16 +++------------- 1 file changed, 3 insertions(+), 13 deletions(-)
drivers/mfd/ab8500-debugfs.c +3 -13
@@ -1513,24 +1513,14 @@ static int ab8500_interrupts_show(struct {int line; -seq_puts (s,"name: number: number of: wake:\n" ); +seq_puts (s,"name: number: irq: number of: wake:\n" );for (line =0 ; line < num_interrupt_lines; line++) { -struct irq_desc *desc =irq_to_desc (line + irq_first); - -seq_printf (s,"%3i: %6i %4i" , +seq_printf (s,"%3i: %6i %4i %4i\n" , line, + line + irq_first, num_interrupts[line], num_wake_interrupts[line]); - -if (desc && desc->name) -seq_printf (s,"-%-8s" , desc->name); -if (desc && desc->action) { -struct irqaction *action = desc->action; - -seq_printf (s," %s" , action->name); -while ((action = action->next) !=NULL ) -seq_printf (s,", %s" , action->name); }seq_putc (s,'\n' ); }
Re: [patch 16/30] mfd: ab8500-debugfs: Remove the racy fiddling with irq_desc
Quoted 9 lines · Thomas Gleixner
First of all drivers have absolutely no business to dig into the internals of an irq descriptor. That's core code and subject to change. All of this information is readily available to
/proc/interruptsin a safe and race free way.Remove the inspection code which is a blatant violation of subsystem boundaries and racy against concurrent modifications of the interrupt descriptor.
Quoted 7 lines
Print the irq line instead so the information can be looked up in a sane way in
/proc/interrupts.Signed-off-by Thomas Gleixner <[email protected]>Cc Linus Walleij <[email protected]>Cc Lee Jones <[email protected]>
Yours,
Linus Walleij
Re: [patch 16/30] mfd: ab8500-debugfs: Remove the racy fiddling with irq_desc
In reply to Thomas GleixnerQuoted 12 lines · Thomas Gleixner
First of all drivers have absolutely no business to dig into the internals of an irq descriptor. That's core code and subject to change. All of this information is readily available to
/proc/interruptsin a safe and race free way.Remove the inspection code which is a blatant violation of subsystem boundaries and racy against concurrent modifications of the interrupt descriptor.
Print the irq line instead so the information can be looked up in a sane way in
/proc/interrupts.
Quoted 7 lines
Signed-off-by Thomas Gleixner <[email protected]>Cc Linus Walleij <[email protected]>Cc Lee Jones <[email protected]>--- drivers/mfd/ab8500-debugfs.c | 16 +++------------- 1 file changed, 3 insertions(+), 13 deletions(-)
Signature · 4 lines
Lee Jones [李琼斯] Senior Technical Lead - Developer Services Linaro.org │ Open source software for Arm SoCs Follow Linaro: Facebook | Twitter | Blog
Re: [patch 16/30] mfd: ab8500-debugfs: Remove the racy fiddling with irq_desc
In reply to Thomas GleixnerQuoted 12 lines · Thomas Gleixner
First of all drivers have absolutely no business to dig into the internals of an irq descriptor. That's core code and subject to change. All of this information is readily available to
/proc/interruptsin a safe and race free way.Remove the inspection code which is a blatant violation of subsystem boundaries and racy against concurrent modifications of the interrupt descriptor.
Print the irq line instead so the information can be looked up in a sane way in
/proc/interrupts.
...
drivers/mfd/ab8500-debugfs.c static int ab8500_interrupts_show(struct · line 1519
Quoted 2 lines
drivers/mfd/ab8500-debugfs.c +1
-1- seq_printf(s, "%3i: %6i %4i", + seq_printf(s, "%3i: %6i %4i %4i\n",
Seems different specifiers, I think the intention was something like
seq_printf(s, "%3i: %4i %6i %4i\n",
Quoted 4 lines
line,
+ line +irq_first,num_interrupts[line], num_wake_interrupts[line]);
Signature · 2 lines
With Best Regards, Andy Shevchenko
[tip: irq/core] mfd: ab8500-debugfs: Remove the racy fiddling with irq_desc
In reply to Thomas GleixnerThe following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 4fa1cf7cde28ad4d7e4388cccfe682dded6a7aca Gitweb: https://git.kernel.org/tip/4fa1cf7cde28ad4d7e4388cccfe682dded6a7aca Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:25:52 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:05 +01:00
mfd: ab8500-debugfs: Remove the racy fiddling with irq_desc
First of all drivers have absolutely no business to dig into the internals of an irq descriptor. That's core code and subject to change. All of this information is readily available to /proc/interrupts in a safe and race free way.
Remove the inspection code which is a blatant violation of subsystem boundaries and racy against concurrent modifications of the interrupt descriptor.
Print the irq line instead so the information can be looked up in a sane way in /proc/interrupts.
--- drivers/mfd/ab8500-debugfs.c | 16 +++------------- 1 file changed, 3 insertions(+), 13 deletions(-)
drivers/mfd/ab8500-debugfs.c +3 -13
@@ -1513,24 +1513,14 @@ static int ab8500_interrupts_show(struct seq_file *s, void *p) {int line; -seq_puts (s,"name: number: number of: wake:\n" ); +seq_puts (s,"name: number: irq: number of: wake:\n" );for (line =0 ; line < num_interrupt_lines; line++) { -struct irq_desc *desc =irq_to_desc (line + irq_first); - -seq_printf (s,"%3i: %6i %4i" , +seq_printf (s,"%3i: %4i %6i %4i\n" , line, + line + irq_first, num_interrupts[line], num_wake_interrupts[line]); - -if (desc && desc->name) -seq_printf (s,"-%-8s" , desc->name); -if (desc && desc->action) { -struct irqaction *action = desc->action; - -seq_printf (s," %s" , action->name); -while ((action = action->next) !=NULL ) -seq_printf (s,", %s" , action->name); }seq_putc (s,'\n' ); }
PATCH 17/30 NTB/msi: Use irq_has_action()
In reply to Thomas GleixnerUse the proper core function.
--- drivers/ntb/msi.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
drivers/ntb/msi.c +1 -3
@@ -282,15 +282,13 @@ int ntbm_msi_request_threaded_irq(struct struct ntb_msi_desc *msi_desc) {struct msi_desc *entry; -struct irq_desc *desc;int ret;if (!ntb->msi)return -EINVAL ;for_each_pci_msi_entry (entry, ntb->pdev) { - desc =irq_to_desc (entry->irq); -if (desc->action) +if (irq_has_action (entry->irq))continue ; ret = devm_request_threaded_irq(&ntb->dev, entry->irq, handler,
Re: [patch 17/30] NTB/msi: Use irq_has_action()
Quoted 7 lines · Thomas Gleixner
Use the proper core function.
Signed-off-by Thomas Gleixner <[email protected]>Cc Jon Mason <[email protected]>Cc Dave Jiang <[email protected]>Cc Allen Hubbe <[email protected]>
Looks good to me.
Quoted 24 lines
--- drivers/ntb/msi.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)drivers/ntb/msi.c +1
-3@@ -282,15 +282,13 @@ int ntbm_msi_request_threaded_irq(struct struct ntb_msi_desc *msi_desc) {struct msi_desc *entry; -struct irq_desc *desc;int ret;if (!ntb->msi)return -EINVAL ;for_each_pci_msi_entry (entry, ntb->pdev) { - desc =irq_to_desc (entry->irq); -if (desc->action) +if (irq_has_action (entry->irq))continue ; ret = devm_request_threaded_irq(&ntb->dev, entry->irq, handler,
[tip: irq/core] NTB/msi: Use irq_has_action()
In reply to Thomas GleixnerThe following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: d53c576bd3d4787b3d7d0d814b28ae67099d84a1 Gitweb: https://git.kernel.org/tip/d53c576bd3d4787b3d7d0d814b28ae67099d84a1 Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:25:53 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:05 +01:00
NTB/msi: Use irq_has_action()
Use the proper core function.
--- drivers/ntb/msi.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
drivers/ntb/msi.c +1 -3
@@ -282,15 +282,13 @@ int ntbm_msi_request_threaded_irq(struct ntb_dev *ntb, irq_handler_t handler, struct ntb_msi_desc *msi_desc) {struct msi_desc *entry; -struct irq_desc *desc;int ret;if (!ntb->msi)return -EINVAL ;for_each_pci_msi_entry (entry, ntb->pdev) { - desc =irq_to_desc (entry->irq); -if (desc->action) +if (irq_has_action (entry->irq))continue ; ret = devm_request_threaded_irq(&ntb->dev, entry->irq, handler,
PATCH 18/30 PCI: xilinx-nwl: Use irq_data_get_irq_chip_data()
In reply to Thomas GleixnerGoing through a full irq descriptor lookup instead of just using the proper helper function which provides direct access is suboptimal.
In fact it _is_ wrong because the chip callback needs to get the chip data which is relevant for the chip while using the irq descriptor variant returns the irq chip data of the top level chip of a hierarchy. It does not matter in this case because the chip is the top level chip, but that doesn't make it more correct.
--- drivers/pci/controller/pcie-xilinx-nwl.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-)
drivers/pci/controller/pcie-xilinx-nwl.c +2 -6
@@ -379,13 +379,11 @@ static void nwl_pcie_msi_handler_low(strstatic void nwl_mask_leg_irq (struct irq_data *data) { -struct irq_desc *desc =irq_to_desc (data->irq); -struct nwl_pcie *pcie; +struct nwl_pcie *pcie =irq_data_get_irq_chip_data (data);unsigned long flags;u32 mask;u32 val; - pcie =irq_desc_get_chip_data (desc); mask =1 << (data->hwirq -1 );raw_spin_lock_irqsave (&pcie->leg_mask_lock, flags); val =nwl_bridge_readl (pcie,MSGF_LEG_MASK );
@@ -395,13 +393,11 @@ static void nwl_mask_leg_irq(struct irq_static void nwl_unmask_leg_irq (struct irq_data *data) { -struct irq_desc *desc =irq_to_desc (data->irq); -struct nwl_pcie *pcie; +struct nwl_pcie *pcie =irq_data_get_irq_chip_data (data);unsigned long flags;u32 mask;u32 val; - pcie =irq_desc_get_chip_data (desc); mask =1 << (data->hwirq -1 );raw_spin_lock_irqsave (&pcie->leg_mask_lock, flags); val =nwl_bridge_readl (pcie,MSGF_LEG_MASK );
Re: [patch 18/30] PCI: xilinx-nwl: Use irq_data_get_irq_chip_data()
Quoted 17 lines · Thomas Gleixner
Going through a full irq descriptor lookup instead of just using the proper helper function which provides direct access is suboptimal.
In fact it _is_ wrong because the chip callback needs to get the chip data which is relevant for the chip while using the irq descriptor variant returns the irq chip data of the top level chip of a hierarchy. It does not matter in this case because the chip is the top level chip, but that doesn't make it more correct.
Signed-off-by Thomas Gleixner <[email protected]>Cc Lorenzo Pieralisi <[email protected]>Cc Rob Herring <[email protected]>Cc Bjorn Helgaas <[email protected]>Cc Michal Simek <[email protected]>
Quoted 3 lines
--- drivers/pci/controller/pcie-xilinx-nwl.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-)
[tip: irq/core] PCI: xilinx-nwl: Use irq_data_get_irq_chip_data()
In reply to Thomas GleixnerThe following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: f44d2329d5c179d24708ada87f5e77b781e336ca Gitweb: https://git.kernel.org/tip/f44d2329d5c179d24708ada87f5e77b781e336ca Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:25:54 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:05 +01:00
PCI: xilinx-nwl: Use irq_data_get_irq_chip_data()
Going through a full irq descriptor lookup instead of just using the proper helper function which provides direct access is suboptimal.
In fact it _is_ wrong because the chip callback needs to get the chip data which is relevant for the chip while using the irq descriptor variant returns the irq chip data of the top level chip of a hierarchy. It does not matter in this case because the chip is the top level chip, but that doesn't make it more correct.
--- drivers/pci/controller/pcie-xilinx-nwl.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-)
drivers/pci/controller/pcie-xilinx-nwl.c +2 -6
@@ -379,13 +379,11 @@ static void nwl_pcie_msi_handler_low(struct irq_desc *desc)static void nwl_mask_leg_irq (struct irq_data *data) { -struct irq_desc *desc =irq_to_desc (data->irq); -struct nwl_pcie *pcie; +struct nwl_pcie *pcie =irq_data_get_irq_chip_data (data);unsigned long flags;u32 mask;u32 val; - pcie =irq_desc_get_chip_data (desc); mask =1 << (data->hwirq -1 );raw_spin_lock_irqsave (&pcie->leg_mask_lock, flags); val =nwl_bridge_readl (pcie,MSGF_LEG_MASK );
@@ -395,13 +393,11 @@ static void nwl_mask_leg_irq(struct irq_data *data)static void nwl_unmask_leg_irq (struct irq_data *data) { -struct irq_desc *desc =irq_to_desc (data->irq); -struct nwl_pcie *pcie; +struct nwl_pcie *pcie =irq_data_get_irq_chip_data (data);unsigned long flags;u32 mask;u32 val; - pcie =irq_desc_get_chip_data (desc); mask =1 << (data->hwirq -1 );raw_spin_lock_irqsave (&pcie->leg_mask_lock, flags); val =nwl_bridge_readl (pcie,MSGF_LEG_MASK );
PATCH 19/30 PCI: mobiveil: Use irq_data_get_irq_chip_data()
In reply to Thomas GleixnerGoing through a full irq descriptor lookup instead of just using the proper helper function which provides direct access is suboptimal.
In fact it _is_ wrong because the chip callback needs to get the chip data which is relevant for the chip while using the irq descriptor variant returns the irq chip data of the top level chip of a hierarchy. It does not matter in this case because the chip is the top level chip, but that doesn't make it more correct.
--- drivers/pci/controller/mobiveil/pcie-mobiveil-host.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-)
drivers/pci/controller/mobiveil/pcie-mobiveil-host.c +2 -6
@@ -306,13 +306,11 @@ int mobiveil_host_init(struct mobiveil_pstatic void mobiveil_mask_intx_irq (struct irq_data *data) { -struct irq_desc *desc =irq_to_desc (data->irq); -struct mobiveil_pcie *pcie; +struct mobiveil_pcie *pcie =irq_data_get_irq_chip_data (data);struct mobiveil_root_port *rp;unsigned long flags;u32 mask, shifted_val; - pcie =irq_desc_get_chip_data (desc); rp = &pcie->rp; mask =1 << ((data->hwirq +PAB_INTX_START ) -1 );raw_spin_lock_irqsave (&rp->intx_mask_lock, flags);
@@ -324,13 +322,11 @@ static void mobiveil_mask_intx_irq(strucstatic void mobiveil_unmask_intx_irq (struct irq_data *data) { -struct irq_desc *desc =irq_to_desc (data->irq); -struct mobiveil_pcie *pcie; +struct mobiveil_pcie *pcie =irq_data_get_irq_chip_data (data);struct mobiveil_root_port *rp;unsigned long flags;u32 shifted_val, mask; - pcie =irq_desc_get_chip_data (desc); rp = &pcie->rp; mask =1 << ((data->hwirq +PAB_INTX_START ) -1 );raw_spin_lock_irqsave (&rp->intx_mask_lock, flags);
Re: [patch 19/30] PCI: mobiveil: Use irq_data_get_irq_chip_data()
Quoted 17 lines · Thomas Gleixner
Going through a full irq descriptor lookup instead of just using the proper helper function which provides direct access is suboptimal.
In fact it _is_ wrong because the chip callback needs to get the chip data which is relevant for the chip while using the irq descriptor variant returns the irq chip data of the top level chip of a hierarchy. It does not matter in this case because the chip is the top level chip, but that doesn't make it more correct.
Signed-off-by Thomas Gleixner <[email protected]>Cc Karthikeyan Mitran <[email protected]>Cc Hou Zhiqiang <[email protected]>Cc Lorenzo Pieralisi <[email protected]>Cc Rob Herring <[email protected]>Cc Bjorn Helgaas <[email protected]>
Quoted 3 lines
--- drivers/pci/controller/mobiveil/pcie-mobiveil-host.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-)
[tip: irq/core] PCI: mobiveil: Use irq_data_get_irq_chip_data()
In reply to Thomas GleixnerThe following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: c9bfac576ac620b71f42fb695493d6eb6e382637 Gitweb: https://git.kernel.org/tip/c9bfac576ac620b71f42fb695493d6eb6e382637 Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:25:55 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:05 +01:00
PCI: mobiveil: Use irq_data_get_irq_chip_data()
Going through a full irq descriptor lookup instead of just using the proper helper function which provides direct access is suboptimal.
In fact it _is_ wrong because the chip callback needs to get the chip data which is relevant for the chip while using the irq descriptor variant returns the irq chip data of the top level chip of a hierarchy. It does not matter in this case because the chip is the top level chip, but that doesn't make it more correct.
--- drivers/pci/controller/mobiveil/pcie-mobiveil-host.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-)
drivers/pci/controller/mobiveil/pcie-mobiveil-host.c +2 -6
@@ -306,13 +306,11 @@ int mobiveil_host_init(struct mobiveil_pcie *pcie, bool reinit)static void mobiveil_mask_intx_irq (struct irq_data *data) { -struct irq_desc *desc =irq_to_desc (data->irq); -struct mobiveil_pcie *pcie; +struct mobiveil_pcie *pcie =irq_data_get_irq_chip_data (data);struct mobiveil_root_port *rp;unsigned long flags;u32 mask, shifted_val; - pcie =irq_desc_get_chip_data (desc); rp = &pcie->rp; mask =1 << ((data->hwirq +PAB_INTX_START ) -1 );raw_spin_lock_irqsave (&rp->intx_mask_lock, flags);
@@ -324,13 +322,11 @@ static void mobiveil_mask_intx_irq(struct irq_data *data)static void mobiveil_unmask_intx_irq (struct irq_data *data) { -struct irq_desc *desc =irq_to_desc (data->irq); -struct mobiveil_pcie *pcie; +struct mobiveil_pcie *pcie =irq_data_get_irq_chip_data (data);struct mobiveil_root_port *rp;unsigned long flags;u32 shifted_val, mask; - pcie =irq_desc_get_chip_data (desc); rp = &pcie->rp; mask =1 << ((data->hwirq +PAB_INTX_START ) -1 );raw_spin_lock_irqsave (&rp->intx_mask_lock, flags);
PATCH 20/30 net/mlx4: Replace irq_to_desc() abuse
In reply to Thomas GleixnerNo driver has any business with the internals of an interrupt descriptor. Storing a pointer to it just to use yet another helper at the actual usage site to retrieve the affinity mask is creative at best. Just because C does not allow encapsulation does not mean that the kernel has no limits.
Retrieve a pointer to the affinity mask itself and use that. It's still using an interface which is usually not for random drivers, but definitely less hideous than the previous hack.
--- drivers/net/ethernet/mellanox/mlx4/en_cq.c | 8 +++----- drivers/net/ethernet/mellanox/mlx4/en_rx.c | 6 +----- drivers/net/ethernet/mellanox/mlx4/mlx4_en.h | 3 ++- 3 files changed, 6 insertions(+), 11 deletions(-)
drivers/net/ethernet/mellanox/mlx4/en_cq.c +3 -5
@@ -90,7 +90,7 @@ int mlx4_en_activate_cq(struct mlx4_en_p int cq_idx) {struct mlx4_en_dev *mdev = priv->mdev; -int err =0 ; +int irq, err =0 ;int timestamp_en =0 ;bool assigned_eq = false;
@@ -116,10 +116,8 @@ int mlx4_en_activate_cq(struct mlx4_en_p assigned_eq = true; } - - cq->irq_desc = -irq_to_desc (mlx4_eq_get_irq (mdev->dev, - cq->vector)); + irq =mlx4_eq_get_irq (mdev->dev, cq->vector); + cq->aff_mask =irq_get_affinity_mask (irq); }else {/* For TX we use the same irq per ring we assigned for the RX */
drivers/net/ethernet/mellanox/mlx4/en_rx.c +1 -5
@@ -959,8 +959,6 @@ int mlx4_en_poll_rx_cq(struct napi_struc/* If we used up all the quota - we're probably not done yet... */ if (done == budget || !clean_complete) { -const struct cpumask *aff; -struct irq_data *idata;int cpu_curr;/* in case we got here because of !clean_complete */
@@ -969,10 +967,8 @@ int mlx4_en_poll_rx_cq(struct napi_strucINC_PERF_COUNTER (priv->pstats.napi_quota); cpu_curr =smp_processor_id (); - idata =irq_desc_get_irq_data (cq->irq_desc); - aff =irq_data_get_affinity_mask (idata); -if (likely (cpumask_test_cpu (cpu_curr, aff))) +if (likely (cpumask_test_cpu (cpu_curr, cq->aff_mask)))return budget;/* Current cpu is not according to smp_irq_affinity -
drivers/net/ethernet/mellanox/mlx4/mlx4_en.h +2 -1
@@ -46,6 +46,7 @@#endif #include <linux/cpu_rmap.h> #include <linux/ptp_clock_kernel.h> +#include <linux/irq.h> #include <net/xdp.h> #include <linux/mlx4/device.h>
@@ -380,7 +381,7 @@ struct mlx4_en_cq {struct mlx4_cqe *buf;#define MLX4_EN_OPCODE_ERROR 0x1e -struct irq_desc *irq_desc; +const struct cpumask *aff_mask; };struct mlx4_en_port_profile {
[tip: irq/core] net/mlx4: Replace irq_to_desc() abuse
The following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 5ea444c8f27b50f14ccfece192a6199d3ada9253 Gitweb: https://git.kernel.org/tip/5ea444c8f27b50f14ccfece192a6199d3ada9253 Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:25:56 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:05 +01:00
net/mlx4: Replace irq_to_desc() abuse
No driver has any business with the internals of an interrupt descriptor. Storing a pointer to it just to use yet another helper at the actual usage site to retrieve the affinity mask is creative at best. Just because C does not allow encapsulation does not mean that the kernel has no limits.
Retrieve a pointer to the affinity mask itself and use that. It's still using an interface which is usually not for random drivers, but definitely less hideous than the previous hack.
--- drivers/net/ethernet/mellanox/mlx4/en_cq.c | 8 +++----- drivers/net/ethernet/mellanox/mlx4/en_rx.c | 6 +----- drivers/net/ethernet/mellanox/mlx4/mlx4_en.h | 3 ++- 3 files changed, 6 insertions(+), 11 deletions(-)
drivers/net/ethernet/mellanox/mlx4/en_cq.c +3 -5
@@ -90,7 +90,7 @@ int mlx4_en_activate_cq(struct mlx4_en_priv *priv, struct mlx4_en_cq *cq, int cq_idx) {struct mlx4_en_dev *mdev = priv->mdev; -int err =0 ; +int irq, err =0 ;int timestamp_en =0 ;bool assigned_eq = false;
@@ -116,10 +116,8 @@ int mlx4_en_activate_cq(struct mlx4_en_priv *priv, struct mlx4_en_cq *cq, assigned_eq = true; } - - cq->irq_desc = -irq_to_desc (mlx4_eq_get_irq (mdev->dev, - cq->vector)); + irq =mlx4_eq_get_irq (mdev->dev, cq->vector); + cq->aff_mask =irq_get_affinity_mask (irq); }else {/* For TX we use the same irq per ring we assigned for the RX */
drivers/net/ethernet/mellanox/mlx4/en_rx.c +1 -5
@@ -959,8 +959,6 @@ int mlx4_en_poll_rx_cq(struct napi_struct *napi, int budget)/* If we used up all the quota - we're probably not done yet... */ if (done == budget || !clean_complete) { -const struct cpumask *aff; -struct irq_data *idata;int cpu_curr;/* in case we got here because of !clean_complete */
@@ -969,10 +967,8 @@ int mlx4_en_poll_rx_cq(struct napi_struct *napi, int budget)INC_PERF_COUNTER (priv->pstats.napi_quota); cpu_curr =smp_processor_id (); - idata =irq_desc_get_irq_data (cq->irq_desc); - aff =irq_data_get_affinity_mask (idata); -if (likely (cpumask_test_cpu (cpu_curr, aff))) +if (likely (cpumask_test_cpu (cpu_curr, cq->aff_mask)))return budget;/* Current cpu is not according to smp_irq_affinity -
drivers/net/ethernet/mellanox/mlx4/mlx4_en.h +2 -1
@@ -46,6 +46,7 @@#endif #include <linux/cpu_rmap.h> #include <linux/ptp_clock_kernel.h> +#include <linux/irq.h> #include <net/xdp.h> #include <linux/mlx4/device.h>
@@ -380,7 +381,7 @@ struct mlx4_en_cq {struct mlx4_cqe *buf;#define MLX4_EN_OPCODE_ERROR 0x1e -struct irq_desc *irq_desc; +const struct cpumask *aff_mask; };struct mlx4_en_port_profile {
Re: [patch 20/30] net/mlx4: Replace irq_to_desc() abuse
In reply to Thomas GleixnerQuoted 16 lines · Thomas Gleixner
No driver has any business with the internals of an interrupt descriptor. Storing a pointer to it just to use yet another helper at the actual usage site to retrieve the affinity mask is creative at best. Just because C does not allow encapsulation does not mean that the kernel has no limits.
Retrieve a pointer to the affinity mask itself and use that. It's still using an interface which is usually not for random drivers, but definitely less hideous than the previous hack.
Signed-off-by Thomas Gleixner <[email protected]>Cc Tariq Toukan <[email protected]>Cc "David S. Miller" <[email protected]>Cc Jakub Kicinski <[email protected]>
Quoted 6 lines
--- drivers/net/ethernet/mellanox/mlx4/en_cq.c | 8 +++----- drivers/net/ethernet/mellanox/mlx4/en_rx.c | 6 +----- drivers/net/ethernet/mellanox/mlx4/mlx4_en.h | 3 ++- 3 files changed, 6 insertions(+), 11 deletions(-)
Thanks for your patch.
PATCH 21/30 net/mlx4: Use effective interrupt affinity
In reply to Thomas GleixnerUsing the interrupt affinity mask for checking locality is not really working well on architectures which support effective affinity masks.
The affinity mask is either the system wide default or set by user space, but the architecture can or even must reduce the mask to the effective set, which means that checking the affinity mask itself does not really tell about the actual target CPUs.
--- drivers/net/ethernet/mellanox/mlx4/en_cq.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx4/en_cq.c +1 -1
@@ -117,7 +117,7 @@ int mlx4_en_activate_cq(struct mlx4_en_p assigned_eq = true; } irq =mlx4_eq_get_irq (mdev->dev, cq->vector); - cq->aff_mask =irq_get_affinity_mask (irq); + cq->aff_mask =irq_get_effective_affinity_mask (irq); }else {/* For TX we use the same irq per ring we assigned for the RX */
[tip: irq/core] net/mlx4: Use effective interrupt affinity
The following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: f23130b1af7bccb23dff1e9223d826869c2ad45d Gitweb: https://git.kernel.org/tip/f23130b1af7bccb23dff1e9223d826869c2ad45d Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:25:57 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:05 +01:00
net/mlx4: Use effective interrupt affinity
Using the interrupt affinity mask for checking locality is not really working well on architectures which support effective affinity masks.
The affinity mask is either the system wide default or set by user space, but the architecture can or even must reduce the mask to the effective set, which means that checking the affinity mask itself does not really tell about the actual target CPUs.
--- drivers/net/ethernet/mellanox/mlx4/en_cq.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx4/en_cq.c +1 -1
@@ -117,7 +117,7 @@ int mlx4_en_activate_cq(struct mlx4_en_priv *priv, struct mlx4_en_cq *cq, assigned_eq = true; } irq =mlx4_eq_get_irq (mdev->dev, cq->vector); - cq->aff_mask =irq_get_affinity_mask (irq); + cq->aff_mask =irq_get_effective_affinity_mask (irq); }else {/* For TX we use the same irq per ring we assigned for the RX */
Re: [patch 21/30] net/mlx4: Use effective interrupt affinity
In reply to Thomas GleixnerQuoted 22 lines · Thomas Gleixner
Using the interrupt affinity mask for checking locality is not really working well on architectures which support effective affinity masks.
The affinity mask is either the system wide default or set by user space, but the architecture can or even must reduce the mask to the effective set, which means that checking the affinity mask itself does not really tell about the actual target CPUs.
Signed-off-by Thomas Gleixner <[email protected]>Cc Tariq Toukan <[email protected]>Cc "David S. Miller" <[email protected]>Cc Jakub Kicinski <[email protected]>--- drivers/net/ethernet/mellanox/mlx4/en_cq.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx4/en_cq.c int mlx4_en_activate_cq(struct mlx4_en_p · line 123
Quoted 8 lines
drivers/net/ethernet/mellanox/mlx4/en_cq.c +1
-1@@ -117,7 +117,7 @@ int mlx4_en_activate_cq(struct mlx4_en_p assigned_eq = true; } irq =mlx4_eq_get_irq (mdev->dev, cq->vector); - cq->aff_mask =irq_get_affinity_mask (irq); + cq->aff_mask =irq_get_effective_affinity_mask (irq); }else {/* For TX we use the same irq per ring we assigned for the RX */
Thanks.
PATCH 22/30 net/mlx5: Replace irq_to_desc() abuse
In reply to Thomas GleixnerNo driver has any business with the internals of an interrupt descriptor. Storing a pointer to it just to use yet another helper at the actual usage site to retrieve the affinity mask is creative at best. Just because C does not allow encapsulation does not mean that the kernel has no limits.
Retrieve a pointer to the affinity mask itself and use that. It's still using an interface which is usually not for random drivers, but definitely less hideous than the previous hack.
--- drivers/net/ethernet/mellanox/mlx5/core/en.h | 2 +- drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 2 +- drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c | 6 +----- 3 files changed, 3 insertions(+), 7 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en.h +1 -1
@@ -669,7 +669,7 @@ struct mlx5e_channel {spinlock_t async_icosq_lock;/* data path - accessed per napi poll */ -struct irq_desc *irq_desc; +const struct cpumask *aff_mask;struct mlx5e_ch_stats *stats;/* control */
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +1 -1
@@ -1998,7 +1998,7 @@ static int mlx5e_open_channel(struct mlx c->num_tc = params->num_tc; c->xdp = !!params->xdp_prog; c->stats = &priv->channel_stats[ix].ch; - c->irq_desc =irq_to_desc (irq); + c->aff_mask =irq_get_affinity_mask (irq); c->lag_port =mlx5e_enumerate_lag_port (priv->mdev, ix);netif_napi_add (netdev, &c->napi, mlx5e_napi_poll,64 );
drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c +1 -5
@@ -40,12 +40,8 @@static inline bool mlx5e_channel_no_affinity_change (struct mlx5e_channel *c) {int current_cpu =smp_processor_id (); -const struct cpumask *aff; -struct irq_data *idata; - idata =irq_desc_get_irq_data (c->irq_desc); - aff =irq_data_get_affinity_mask (idata); -return cpumask_test_cpu (current_cpu, aff); +return cpumask_test_cpu (current_cpu, c->aff_mask); }static void mlx5e_handle_tx_dim (struct mlx5e_txqsq *sq)
[tip: irq/core] net/mlx5: Replace irq_to_desc() abuse
The following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 3d18847680102182fed60b90ae65cbbb3d929a3c Gitweb: https://git.kernel.org/tip/3d18847680102182fed60b90ae65cbbb3d929a3c Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:25:58 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:06 +01:00
net/mlx5: Replace irq_to_desc() abuse
No driver has any business with the internals of an interrupt descriptor. Storing a pointer to it just to use yet another helper at the actual usage site to retrieve the affinity mask is creative at best. Just because C does not allow encapsulation does not mean that the kernel has no limits.
Retrieve a pointer to the affinity mask itself and use that. It's still using an interface which is usually not for random drivers, but definitely less hideous than the previous hack.
--- drivers/net/ethernet/mellanox/mlx5/core/en.h | 2 +- drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 2 +- drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c | 6 +----- 3 files changed, 3 insertions(+), 7 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en.h +1 -1
@@ -669,7 +669,7 @@ struct mlx5e_channel {spinlock_t async_icosq_lock;/* data path - accessed per napi poll */ -struct irq_desc *irq_desc; +const struct cpumask *aff_mask;struct mlx5e_ch_stats *stats;/* control */
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +1 -1
@@ -1998,7 +1998,7 @@ static int mlx5e_open_channel(struct mlx5e_priv *priv, int ix, c->num_tc = params->num_tc; c->xdp = !!params->xdp_prog; c->stats = &priv->channel_stats[ix].ch; - c->irq_desc =irq_to_desc (irq); + c->aff_mask =irq_get_affinity_mask (irq); c->lag_port =mlx5e_enumerate_lag_port (priv->mdev, ix);netif_napi_add (netdev, &c->napi, mlx5e_napi_poll,64 );
drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c +1 -5
@@ -40,12 +40,8 @@static inline bool mlx5e_channel_no_affinity_change (struct mlx5e_channel *c) {int current_cpu =smp_processor_id (); -const struct cpumask *aff; -struct irq_data *idata; - idata =irq_desc_get_irq_data (c->irq_desc); - aff =irq_data_get_affinity_mask (idata); -return cpumask_test_cpu (current_cpu, aff); +return cpumask_test_cpu (current_cpu, c->aff_mask); }static void mlx5e_handle_tx_dim (struct mlx5e_txqsq *sq)
Re: [patch 22/30] net/mlx5: Replace irq_to_desc() abuse
In reply to Thomas GleixnerQuoted 10 lines · Thomas Gleixner
No driver has any business with the internals of an interrupt descriptor. Storing a pointer to it just to use yet another helper at the actual usage site to retrieve the affinity mask is creative at best. Just because C does not allow encapsulation does not mean that the kernel has no limits.
Retrieve a pointer to the affinity mask itself and use that. It's still using an interface which is usually not for random drivers, but definitely less hideous than the previous hack.
Quoted 7 lines
Signed-off-by Thomas Gleixner <[email protected]>--- drivers/net/ethernet/mellanox/mlx5/core/en.h | 2 +- drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 2 +- drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c | 6 +----- 3 files changed, 3 insertions(+), 7 deletions(-)
Thanks.
Re: [patch 22/30] net/mlx5: Replace irq_to_desc() abuse
In reply to Thomas GleixnerQuoted 9 lines · Thomas Gleixner
No driver has any business with the internals of an interrupt descriptor. Storing a pointer to it just to use yet another helper at the
actual usage site to retrieve the affinity mask is creative at best. Just
because C does not allow encapsulation does not mean that the kernel has no
limits.
you can't blame the developers for using stuff from include/linux/ Not all developers are the same, and sometime we don't read in between the lines, you can't assume all driver developers to be expert on irq APIs disciplines.
your rules must be programmatically expressed, for instance, you can just hide struct irq_desc and irq_to_desc() in kernel/irq/ and remove them from include/linux/ header files, if you want privacy in your subsystem, don't put all your header files on display under include/linux.
Quoted 27 lines
Retrieve a pointer to the affinity mask itself and use that. It's still
using an interface which is usually not for random drivers, but definitely
less hideous than the previous hack.Signed-off-by Thomas Gleixner <[email protected]>--- drivers/net/ethernet/mellanox/mlx5/core/en.h | 2 +- drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 2 +- drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c | 6 +----- 3 files changed, 3 insertions(+), 7 deletions(-)drivers/net/ethernet/mellanox/mlx5/core/en.h +1
-1@@ -669,7 +669,7 @@ struct mlx5e_channel {spinlock_t async_icosq_lock;/* data path - accessed per napi poll */ -struct irq_desc *irq_desc; +const struct cpumask *aff_mask;struct mlx5e_ch_stats *stats;/* control */
drivers/net/ethernet/mellanox/mlx5/core/en_main.c static int mlx5e_open_channel(struct mlx · line 2001
Quoted 5 lines
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +1
-1@@ -1998,7 +1998,7 @@ static int mlx5e_open_channel(struct mlx c->num_tc = params->num_tc; c->xdp = !!params->xdp_prog; c->stats = &priv->channel_stats[ix].ch; - c->irq_desc =irq_to_desc (irq); + c->aff_mask =irq_get_affinity_mask (irq);
as long as the affinity mask pointer stays the same for the lifetime of the irq vector.
Assuming that:
PATCH 23/30 net/mlx5: Use effective interrupt affinity
In reply to Thomas GleixnerUsing the interrupt affinity mask for checking locality is not really working well on architectures which support effective affinity masks.
The affinity mask is either the system wide default or set by user space, but the architecture can or even must reduce the mask to the effective set, which means that checking the affinity mask itself does not really tell about the actual target CPUs.
--- drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +1 -1
@@ -1998,7 +1998,7 @@ static int mlx5e_open_channel(struct mlx c->num_tc = params->num_tc; c->xdp = !!params->xdp_prog; c->stats = &priv->channel_stats[ix].ch; - c->aff_mask =irq_get_affinity_mask (irq); + c->aff_mask =irq_get_effective_affinity_mask (irq); c->lag_port =mlx5e_enumerate_lag_port (priv->mdev, ix);netif_napi_add (netdev, &c->napi, mlx5e_napi_poll,64 );
[tip: irq/core] net/mlx5: Use effective interrupt affinity
The following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: ca00c76aa8acf5a7563dd9f5020f9c6dc9a0577d Gitweb: https://git.kernel.org/tip/ca00c76aa8acf5a7563dd9f5020f9c6dc9a0577d Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:25:59 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:06 +01:00
net/mlx5: Use effective interrupt affinity
Using the interrupt affinity mask for checking locality is not really working well on architectures which support effective affinity masks.
The affinity mask is either the system wide default or set by user space, but the architecture can or even must reduce the mask to the effective set, which means that checking the affinity mask itself does not really tell about the actual target CPUs.
--- drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +1 -1
@@ -1998,7 +1998,7 @@ static int mlx5e_open_channel(struct mlx5e_priv *priv, int ix, c->num_tc = params->num_tc; c->xdp = !!params->xdp_prog; c->stats = &priv->channel_stats[ix].ch; - c->aff_mask =irq_get_affinity_mask (irq); + c->aff_mask =irq_get_effective_affinity_mask (irq); c->lag_port =mlx5e_enumerate_lag_port (priv->mdev, ix);netif_napi_add (netdev, &c->napi, mlx5e_napi_poll,64 );
Re: [patch 23/30] net/mlx5: Use effective interrupt affinity
In reply to Thomas GleixnerQuoted 23 lines · Thomas Gleixner
Using the interrupt affinity mask for checking locality is not really working well on architectures which support effective affinity masks.
The affinity mask is either the system wide default or set by user space, but the architecture can or even must reduce the mask to the effective set, which means that checking the affinity mask itself does not really tell about the actual target CPUs.
Signed-off-by Thomas Gleixner <[email protected]>Cc Saeed Mahameed <[email protected]>Cc Leon Romanovsky <[email protected]>Cc "David S. Miller" <[email protected]>Cc Jakub Kicinski <[email protected]>--- drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/en_main.c static int mlx5e_open_channel(struct mlx · line 2004
Quoted 8 lines
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +1
-1@@ -1998,7 +1998,7 @@ static int mlx5e_open_channel(struct mlx c->num_tc = params->num_tc; c->xdp = !!params->xdp_prog; c->stats = &priv->channel_stats[ix].ch; - c->aff_mask =irq_get_affinity_mask (irq); + c->aff_mask =irq_get_effective_affinity_mask (irq); c->lag_port =mlx5e_enumerate_lag_port (priv->mdev, ix);netif_napi_add (netdev, &c->napi, mlx5e_napi_poll,64 );
Thanks.
Re: [patch 23/30] net/mlx5: Use effective interrupt affinity
In reply to Thomas GleixnerQuoted 11 lines · Thomas Gleixner
Using the interrupt affinity mask for checking locality is not really working well on architectures which support effective affinity masks.
The affinity mask is either the system wide default or set by user space,
but the architecture can or even must reduce the mask to the effective set,
which means that checking the affinity mask itself does not really tell
about the actual target CPUs.
Quoted 8 lines
Signed-off-by Thomas Gleixner <[email protected]>Cc Saeed Mahameed <[email protected]>Cc Leon Romanovsky <[email protected]>Cc "David S. Miller" <[email protected]>Cc Jakub Kicinski <[email protected]>
PATCH 24/30 xen/events: Remove unused bind_evtchn_to_irq_lateeoi()
In reply to Thomas Gleixner--- drivers/xen/events/events_base.c | 6 ------ 1 file changed, 6 deletions(-)
drivers/xen/events/events_base.c +0 -6
@@ -1132,12 +1132,6 @@ int bind_evtchn_to_irq(evtchn_port_t evt }EXPORT_SYMBOL_GPL (bind_evtchn_to_irq); -int bind_evtchn_to_irq_lateeoi (evtchn_port_t evtchn) -{ -return bind_evtchn_to_irq_chip (evtchn, &xen_lateeoi_chip); -} -EXPORT_SYMBOL_GPL (bind_evtchn_to_irq_lateeoi); -static int bind_ipi_to_irq (unsigned int ipi,unsigned int cpu) {struct evtchn_bind_ipi bind_ipi;
Re: [patch 24/30] xen/events: Remove unused bind_evtchn_to_irq_lateeoi()
Quoted 12 lines · Thomas Gleixner
Signed-off-by Thomas Gleixner <[email protected]>Cc Boris Ostrovsky <[email protected]>Cc Juergen Gross <[email protected]>Cc Stefano Stabellini <[email protected]>--- drivers/xen/events/events_base.c | 6 ------ 1 file changed, 6 deletions(-)
drivers/xen/events/events_base.c int bind_evtchn_to_irq(evtchn_port_t evt · line 1139
Quoted 8 lines
drivers/xen/events/events_base.c +0
-5@@ -1132,12 +1132,6 @@ int bind_evtchn_to_irq(evtchn_port_t evt }EXPORT_SYMBOL_GPL (bind_evtchn_to_irq); -int bind_evtchn_to_irq_lateeoi (evtchn_port_t evtchn) -{ -return bind_evtchn_to_irq_chip (evtchn, &xen_lateeoi_chip); -} -EXPORT_SYMBOL_GPL (bind_evtchn_to_irq_lateeoi);
include/xen/events.h also needs to be updated (and in the next patch for xen_set_affinity_evtchn() as well).
-boris
Re: [patch 24/30] xen/events: Remove unused bind_evtchn_to_irq_lateeoi()
Quoted 4 lines · boris ostrovsky
Quoted 1 line · Thomas Gleixner
-EXPORT_SYMBOL_GPL(bind_evtchn_to_irq_lateeoi);
include/xen/events.halso needs to be updated (and in the next patch forxen_set_affinity_evtchn()as well).
Darn, I lost that.
[tip: irq/core] xen/events: Remove unused bind_evtchn_to_irq_lateeoi()
In reply to Thomas GleixnerThe following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 16b2efd9e4c60f9010f27ada31b0c7d73f23e0d3 Gitweb: https://git.kernel.org/tip/16b2efd9e4c60f9010f27ada31b0c7d73f23e0d3 Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:26:00 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:06 +01:00
xen/events: Remove unused bind_evtchn_to_irq_lateeoi()
--- drivers/xen/events/events_base.c | 6 ------ 1 file changed, 6 deletions(-)
drivers/xen/events/events_base.c +0 -6
@@ -1132,12 +1132,6 @@ int bind_evtchn_to_irq(evtchn_port_t evtchn) }EXPORT_SYMBOL_GPL (bind_evtchn_to_irq); -int bind_evtchn_to_irq_lateeoi (evtchn_port_t evtchn) -{ -return bind_evtchn_to_irq_chip (evtchn, &xen_lateeoi_chip); -} -EXPORT_SYMBOL_GPL (bind_evtchn_to_irq_lateeoi); -static int bind_ipi_to_irq (unsigned int ipi,unsigned int cpu) {struct evtchn_bind_ipi bind_ipi;
PATCH 25/30 xen/events: Remove disfunct affinity spreading
In reply to Thomas GleixnerThis function can only ever work when the event channels:
- are already established
- interrupts assigned to them
- the affinity has been set by user space already
because any newly set up event channel is forced to be bound to CPU0 and the affinity mask of the interrupt is forced to contain cpumask_of(0).
As the CPU0 enforcement was in place _before_ this was implemented it's entirely unclear how that can ever have worked at all.
Remove it as preparation for doing it proper.
--- drivers/xen/events/events_base.c | 9 --------- drivers/xen/evtchn.c | 34 +--------------------------------- 2 files changed, 1 insertion(+), 42 deletions(-)
drivers/xen/events/events_base.c +0 -9
@@ -1696,15 +1696,6 @@ static int set_affinity_irq(struct irq_dreturn ret; } -/* To be called with desc->lock held. */ -int xen_set_affinity_evtchn (struct irq_desc *desc,unsigned int tcpu) -{ -struct irq_data *d =irq_desc_get_irq_data (desc); - -return set_affinity_irq (d,cpumask_of (tcpu), false); -} -EXPORT_SYMBOL_GPL (xen_set_affinity_evtchn); -static void enable_dynirq (struct irq_data *data) {evtchn_port_t evtchn =evtchn_from_irq (data->irq);
drivers/xen/evtchn.c +1 -33
@@ -421,36 +421,6 @@ static void evtchn_unbind_from_user(strudel_evtchn (u, evtchn); } -static DEFINE_PER_CPU(int, bind_last_selected_cpu); - -static void evtchn_bind_interdom_next_vcpu (evtchn_port_t evtchn) -{ -unsigned int selected_cpu, irq; -struct irq_desc *desc; -unsigned long flags; - - irq =irq_from_evtchn (evtchn); - desc =irq_to_desc (irq); - -if (!desc) -return ; - -raw_spin_lock_irqsave (&desc->lock, flags); - selected_cpu =this_cpu_read (bind_last_selected_cpu); - selected_cpu =cpumask_next_and (selected_cpu, - desc->irq_common_data.affinity, cpu_online_mask); - -if (unlikely (selected_cpu >= nr_cpu_ids)) - selected_cpu =cpumask_first_and (desc->irq_common_data.affinity, - cpu_online_mask); - -this_cpu_write (bind_last_selected_cpu, selected_cpu); - -/* unmask expects irqs to be disabled */ -xen_set_affinity_evtchn (desc, selected_cpu); -raw_spin_unlock_irqrestore (&desc->lock, flags); -} -static long evtchn_ioctl (struct file *file,unsigned int cmd,unsigned long arg) {
@@ -508,10 +478,8 @@ static long evtchn_ioctl(struct file *fibreak ; rc =evtchn_bind_to_user (u, bind_interdomain.local_port); -if (rc ==0 ) { +if (rc ==0 ) rc = bind_interdomain.local_port; -evtchn_bind_interdom_next_vcpu (rc); - }break ; }
[tip: irq/core] xen/events: Remove disfunct affinity spreading
The following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 26599cd34323b73894ebcb5197693cdd668b4aa8 Gitweb: https://git.kernel.org/tip/26599cd34323b73894ebcb5197693cdd668b4aa8 Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:26:01 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:06 +01:00
xen/events: Remove disfunct affinity spreading
This function can only ever work when the event channels:
- are already established
- interrupts assigned to them
- the affinity has been set by user space already
because any newly set up event channel is forced to be bound to CPU0 and the affinity mask of the interrupt is forced to contain cpumask_of(0).
As the CPU0 enforcement was in place _before_ this was implemented it's entirely unclear how that can ever have worked at all.
Remove it as preparation for doing it proper.
--- drivers/xen/events/events_base.c | 9 +-------- drivers/xen/evtchn.c | 34 +------------------------------- 2 files changed, 1 insertion(+), 42 deletions(-)
drivers/xen/events/events_base.c +0 -9
@@ -1696,15 +1696,6 @@ static int set_affinity_irq(struct irq_data *data, const struct cpumask *dest,return ret; } -/* To be called with desc->lock held. */ -int xen_set_affinity_evtchn (struct irq_desc *desc,unsigned int tcpu) -{ -struct irq_data *d =irq_desc_get_irq_data (desc); - -return set_affinity_irq (d,cpumask_of (tcpu), false); -} -EXPORT_SYMBOL_GPL (xen_set_affinity_evtchn); -static void enable_dynirq (struct irq_data *data) {evtchn_port_t evtchn =evtchn_from_irq (data->irq);
drivers/xen/evtchn.c +1 -33
@@ -421,36 +421,6 @@ static void evtchn_unbind_from_user(struct per_user_data *u,del_evtchn (u, evtchn); } -static DEFINE_PER_CPU(int, bind_last_selected_cpu); - -static void evtchn_bind_interdom_next_vcpu (evtchn_port_t evtchn) -{ -unsigned int selected_cpu, irq; -struct irq_desc *desc; -unsigned long flags; - - irq =irq_from_evtchn (evtchn); - desc =irq_to_desc (irq); - -if (!desc) -return ; - -raw_spin_lock_irqsave (&desc->lock, flags); - selected_cpu =this_cpu_read (bind_last_selected_cpu); - selected_cpu =cpumask_next_and (selected_cpu, - desc->irq_common_data.affinity, cpu_online_mask); - -if (unlikely (selected_cpu >= nr_cpu_ids)) - selected_cpu =cpumask_first_and (desc->irq_common_data.affinity, - cpu_online_mask); - -this_cpu_write (bind_last_selected_cpu, selected_cpu); - -/* unmask expects irqs to be disabled */ -xen_set_affinity_evtchn (desc, selected_cpu); -raw_spin_unlock_irqrestore (&desc->lock, flags); -} -static long evtchn_ioctl (struct file *file,unsigned int cmd,unsigned long arg) {
@@ -508,10 +478,8 @@ static long evtchn_ioctl(struct file *file,break ; rc =evtchn_bind_to_user (u, bind_interdomain.local_port); -if (rc ==0 ) { +if (rc ==0 ) rc = bind_interdomain.local_port; -evtchn_bind_interdom_next_vcpu (rc); - }break ; }
PATCH 26/30 xen/events: Use immediate affinity setting
In reply to Thomas GleixnerThere is absolutely no reason to mimic the x86 deferred affinity setting. This mechanism is required to handle the hardware induced issues of IO/APIC and MSI and is not in use when the interrupts are remapped.
XEN does not need this and can simply change the affinity from the calling context. The core code invokes this with the interrupt descriptor lock held so it is fully serialized against any other operation.
Mark the interrupts with IRQ_MOVE_PCNTXT to disable the deferred affinity setting. The conditional mask/unmask operation is already handled in xen_rebind_evtchn_to_cpu().
This makes XEN on x86 use the same mechanics as on e.g. ARM64 where deferred affinity setting is not required and not implemented and the code path in the ack functions is compiled out.
--- drivers/xen/events/events_base.c | 35 +++++++++-------------------------- 1 file changed, 9 insertions(+), 26 deletions(-)
drivers/xen/events/events_base.c +9 -26
@@ -628,6 +628,11 @@ static void xen_irq_init(unsigned irq) info->refcnt =-1 ;set_info_for_irq (irq, info); +/* +* Interrupt affinity setting can be immediate. No point +* in delaying it until an interrupt is handled. +*/ +irq_set_status_flags (irq,IRQ_MOVE_PCNTXT );INIT_LIST_HEAD (&info->eoi_list);list_add_tail (&info->list, &xen_irq_list_head);
@@ -739,18 +744,7 @@ static void eoi_pirq(struct irq_data *daif (!VALID_EVTCHN (evtchn))return ; -if (unlikely (irqd_is_setaffinity_pending (data)) && -likely (!irqd_irq_disabled (data))) { -int masked =test_and_set_mask (evtchn); - -clear_evtchn (evtchn); - -irq_move_masked_irq (data); - -if (!masked) -unmask_evtchn (evtchn); - }else -clear_evtchn (evtchn); +clear_evtchn (evtchn);if (pirq_needs_eoi (data->irq)) { rc =HYPERVISOR_physdev_op (PHYSDEVOP_eoi, &eoi);
@@ -1641,7 +1635,6 @@ void rebind_evtchn_irq(evtchn_port_t evtmutex_unlock (&irq_mapping_update_lock);bind_evtchn_to_cpu (evtchn, info->cpu); -/* This will be deferred until interrupt is processed */ irq_set_affinity (irq,cpumask_of (info->cpu));/* Unmask the event channel. */
@@ -1688,8 +1681,9 @@ static int set_affinity_irq(struct irq_d bool force) {unsigned tcpu =cpumask_first_and (dest, cpu_online_mask); -int ret =xen_rebind_evtchn_to_cpu (evtchn_from_irq (data->irq), tcpu); +int ret; + ret =xen_rebind_evtchn_to_cpu (evtchn_from_irq (data->irq), tcpu);if (!ret)irq_data_update_effective_affinity (data,cpumask_of (tcpu));
@@ -1719,18 +1713,7 @@ static void ack_dynirq(struct irq_data *if (!VALID_EVTCHN (evtchn))return ; -if (unlikely (irqd_is_setaffinity_pending (data)) && -likely (!irqd_irq_disabled (data))) { -int masked =test_and_set_mask (evtchn); - -clear_evtchn (evtchn); - -irq_move_masked_irq (data); - -if (!masked) -unmask_evtchn (evtchn); - }else -clear_evtchn (evtchn); +clear_evtchn (evtchn); }static void mask_ack_dynirq (struct irq_data *data)
[tip: irq/core] xen/events: Use immediate affinity setting
The following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 9b23ffcbfa15f2d4959eaa42faae11e3e39d027a Gitweb: https://git.kernel.org/tip/9b23ffcbfa15f2d4959eaa42faae11e3e39d027a Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:26:02 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:06 +01:00
xen/events: Use immediate affinity setting
There is absolutely no reason to mimic the x86 deferred affinity setting. This mechanism is required to handle the hardware induced issues of IO/APIC and MSI and is not in use when the interrupts are remapped.
XEN does not need this and can simply change the affinity from the calling context. The core code invokes this with the interrupt descriptor lock held so it is fully serialized against any other operation.
Mark the interrupts with IRQ_MOVE_PCNTXT to disable the deferred affinity setting. The conditional mask/unmask operation is already handled in xen_rebind_evtchn_to_cpu().
This makes XEN on x86 use the same mechanics as on e.g. ARM64 where deferred affinity setting is not required and not implemented and the code path in the ack functions is compiled out.
--- drivers/xen/events/events_base.c | 35 +++++++------------------------ 1 file changed, 9 insertions(+), 26 deletions(-)
drivers/xen/events/events_base.c +9 -26
@@ -628,6 +628,11 @@ static void xen_irq_init(unsigned irq) info->refcnt =-1 ;set_info_for_irq (irq, info); +/* +* Interrupt affinity setting can be immediate. No point +* in delaying it until an interrupt is handled. +*/ +irq_set_status_flags (irq,IRQ_MOVE_PCNTXT );INIT_LIST_HEAD (&info->eoi_list);list_add_tail (&info->list, &xen_irq_list_head);
@@ -739,18 +744,7 @@ static void eoi_pirq(struct irq_data *data)if (!VALID_EVTCHN (evtchn))return ; -if (unlikely (irqd_is_setaffinity_pending (data)) && -likely (!irqd_irq_disabled (data))) { -int masked =test_and_set_mask (evtchn); - -clear_evtchn (evtchn); - -irq_move_masked_irq (data); - -if (!masked) -unmask_evtchn (evtchn); - }else -clear_evtchn (evtchn); +clear_evtchn (evtchn);if (pirq_needs_eoi (data->irq)) { rc =HYPERVISOR_physdev_op (PHYSDEVOP_eoi, &eoi);
@@ -1641,7 +1635,6 @@ void rebind_evtchn_irq(evtchn_port_t evtchn, int irq)mutex_unlock (&irq_mapping_update_lock);bind_evtchn_to_cpu (evtchn, info->cpu); -/* This will be deferred until interrupt is processed */ irq_set_affinity (irq,cpumask_of (info->cpu));/* Unmask the event channel. */
@@ -1688,8 +1681,9 @@ static int set_affinity_irq(struct irq_data *data, const struct cpumask *dest, bool force) {unsigned tcpu =cpumask_first_and (dest, cpu_online_mask); -int ret =xen_rebind_evtchn_to_cpu (evtchn_from_irq (data->irq), tcpu); +int ret; + ret =xen_rebind_evtchn_to_cpu (evtchn_from_irq (data->irq), tcpu);if (!ret)irq_data_update_effective_affinity (data,cpumask_of (tcpu));
@@ -1719,18 +1713,7 @@ static void ack_dynirq(struct irq_data *data)if (!VALID_EVTCHN (evtchn))return ; -if (unlikely (irqd_is_setaffinity_pending (data)) && -likely (!irqd_irq_disabled (data))) { -int masked =test_and_set_mask (evtchn); - -clear_evtchn (evtchn); - -irq_move_masked_irq (data); - -if (!masked) -unmask_evtchn (evtchn); - }else -clear_evtchn (evtchn); +clear_evtchn (evtchn); }static void mask_ack_dynirq (struct irq_data *data)
PATCH 27/30 xen/events: Only force affinity mask for percpu interrupts
In reply to Thomas GleixnerAll event channel setups bind the interrupt on CPU0 or the target CPU for percpu interrupts and overwrite the affinity mask with the corresponding cpumask. That does not make sense.
The XEN implementation of irqchip::irq_set_affinity() already picks a single target CPU out of the affinity mask and the actual target is stored in the effective CPU mask, so destroying the user chosen affinity mask which might contain more than one CPU is wrong.
Change the implementation so that the channel is bound to CPU0 at the XEN level and leave the affinity mask alone. At startup of the interrupt affinity will be assigned out of the affinity mask and the XEN binding will be updated. Only keep the enforcement for real percpu interrupts.
On resume the overwrite is not required either because info->cpu and the affinity mask are still the same as at the time of suspend. Same for rebind_evtchn_irq().
This also prepares for proper interrupt spreading.
--- drivers/xen/events/events_base.c | 42 ++++++++++++++++++++++++++------------- 1 file changed, 28 insertions(+), 14 deletions(-)
drivers/xen/events/events_base.c +28 -14
@@ -433,15 +433,20 @@ static bool pirq_needs_eoi_flag(unsignedreturn info->u.pirq.flags &PIRQ_NEEDS_EOI ; } -static void bind_evtchn_to_cpu (evtchn_port_t evtchn,unsigned int cpu) +static void bind_evtchn_to_cpu (evtchn_port_t evtchn,unsigned int cpu, +bool force_affinity) {int irq =get_evtchn_to_irq (evtchn);struct irq_info *info =info_for_irq (irq);BUG_ON (irq ==-1 ); -#ifdef CONFIG_SMP -cpumask_copy (irq_get_affinity_mask (irq),cpumask_of (cpu)); -#endif + +if (IS_ENABLED (CONFIG_SMP ) && force_affinity) { +cpumask_copy (irq_get_affinity_mask (irq),cpumask_of (cpu)); +cpumask_copy (irq_get_effective_affinity_mask (irq), +cpumask_of (cpu)); + } +xen_evtchn_port_bind_to_cpu (evtchn, cpu, info->cpu); info->cpu = cpu;
@@ -788,7 +793,7 @@ static unsigned int __startup_pirq(unsig goto err; info->evtchn = evtchn; -bind_evtchn_to_cpu (evtchn,0 ); +bind_evtchn_to_cpu (evtchn,0 , false); rc =xen_evtchn_port_setup (evtchn);if (rc)
@@ -1107,8 +1112,8 @@ static int bind_evtchn_to_irq_chip(evtch irq = ret; goto out; } -/* New interdomain events are bound to VCPU 0. */ -bind_evtchn_to_cpu (evtchn,0 ); +/* New interdomain events are initially bound to VCPU 0. */ +bind_evtchn_to_cpu (evtchn,0 , false); }else {struct irq_info *info =info_for_irq (irq);WARN_ON (info ==NULL || info->type !=IRQT_EVTCHN );
@@ -1156,7 +1161,11 @@ static int bind_ipi_to_irq(unsigned int irq = ret; goto out; } -bind_evtchn_to_cpu (evtchn, cpu); +/* +* Force the affinity mask to the target CPU so proc shows +* the correct target. +*/ +bind_evtchn_to_cpu (evtchn, cpu, true); }else {struct irq_info *info =info_for_irq (irq);WARN_ON (info ==NULL || info->type !=IRQT_IPI );
@@ -1269,7 +1278,11 @@ int bind_virq_to_irq(unsigned int virq, goto out; } -bind_evtchn_to_cpu (evtchn, cpu); +/* +* Force the affinity mask for percpu interrupts so proc +* shows the correct target. +*/ +bind_evtchn_to_cpu (evtchn, cpu, percpu); }else {struct irq_info *info =info_for_irq (irq);WARN_ON (info ==NULL || info->type !=IRQT_VIRQ );
@@ -1634,8 +1647,7 @@ void rebind_evtchn_irq(evtchn_port_t evtmutex_unlock (&irq_mapping_update_lock); -bind_evtchn_to_cpu (evtchn, info->cpu); -irq_set_affinity (irq,cpumask_of (info->cpu)); +bind_evtchn_to_cpu (evtchn, info->cpu, false);/* Unmask the event channel. */ enable_irq (irq);
@@ -1669,7 +1681,7 @@ static int xen_rebind_evtchn_to_cpu(evtc* it, but don't do the xenlinux-level rebind in that case. */ if (HYPERVISOR_event_channel_op (EVTCHNOP_bind_vcpu, &bind_vcpu) >=0 ) -bind_evtchn_to_cpu (evtchn, tcpu); +bind_evtchn_to_cpu (evtchn, tcpu, false);if (!masked)unmask_evtchn (evtchn);
@@ -1798,7 +1810,8 @@ static void restore_cpu_virqs(unsigned i/* Record the new mapping. */ (void )xen_irq_info_virq_setup (cpu, irq, evtchn, virq); -bind_evtchn_to_cpu (evtchn, cpu); +/* The affinity mask is still valid */ +bind_evtchn_to_cpu (evtchn, cpu, false); } }
@@ -1823,7 +1836,8 @@ static void restore_cpu_ipis(unsigned in/* Record the new mapping. */ (void )xen_irq_info_ipi_setup (cpu, irq, evtchn, ipi); -bind_evtchn_to_cpu (evtchn, cpu); +/* The affinity mask is still valid */ +bind_evtchn_to_cpu (evtchn, cpu, false); } }
Re: [patch 27/30] xen/events: Only force affinity mask for percpu interrupts
Quoted 9 lines · Thomas Gleixner
All event channel setups bind the interrupt on CPU0 or the target CPU for percpu interrupts and overwrite the affinity mask with the corresponding cpumask. That does not make sense.
The XEN implementation of
irqchip::irq_set_affinity()already picks a single target CPU out of the affinity mask and the actual target is stored in the effective CPU mask, so destroying the user chosen affinity mask which might contain more than one CPU is wrong.
Quoted 4 lines
Change the implementation so that the channel is bound to CPU0 at the XEN level and leave the affinity mask alone. At startup of the interrupt affinity will be assigned out of the affinity mask and the XEN binding will be updated.
If that's the case then I wonder whether we need this call at all and instead bind at startup time.
-boris
Quoted 1 line
Only keep the enforcement for real percpu interrupts.
Re: [patch 27/30] xen/events: Only force affinity mask for percpu interrupts
Quoted 10 lines · boris ostrovsky
Quoted 9 lines · Thomas Gleixner
All event channel setups bind the interrupt on CPU0 or the target CPU for percpu interrupts and overwrite the affinity mask with the corresponding cpumask. That does not make sense.
The XEN implementation of
irqchip::irq_set_affinity()already picks a single target CPU out of the affinity mask and the actual target is stored in the effective CPU mask, so destroying the user chosen affinity mask which might contain more than one CPU is wrong.
Quoted 7 lines
Quoted 4 lines
Change the implementation so that the channel is bound to CPU0 at the XEN level and leave the affinity mask alone. At startup of the interrupt affinity will be assigned out of the affinity mask and the XEN binding will be updated.
If that's the case then I wonder whether we need this call at all and instead bind at startup time.
I was wondering about that, but my knowledge about the Xen internal requirements is pretty limited. The current set at least survived basic testing by Jürgen.
Thanks,
tglx
Jürgen Groß
maintainerRe: [patch 27/30] xen/events: Only force affinity mask for percpu interrupts
In reply to [email protected]Quoted 11 lines · On 11.12.20 00:20, [email protected] wrote:
Quoted 9 lines · Thomas Gleixner
All event channel setups bind the interrupt on CPU0 or the target CPU for percpu interrupts and overwrite the affinity mask with the corresponding cpumask. That does not make sense.
The XEN implementation of
irqchip::irq_set_affinity()already picks a single target CPU out of the affinity mask and the actual target is stored in the effective CPU mask, so destroying the user chosen affinity mask which might contain more than one CPU is wrong.
Quoted 7 lines
Quoted 4 lines
Change the implementation so that the channel is bound to CPU0 at the XEN level and leave the affinity mask alone. At startup of the interrupt affinity will be assigned out of the affinity mask and the XEN binding will be updated.
If that's the case then I wonder whether we need this call at all and instead bind at startup time.
This binding to cpu0 was introduced with commit 97253eeeb792d61ed2 and I have no reason to believe the underlying problem has been eliminated.
Juergen
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [patch 27/30] xen/events: Only force affinity mask for percpu interrupts
Quoted 18 lines · Jürgen Groß
Quoted 17 lines · On 11.12.20 00:20, [email protected] wrote:
Quoted 13 lines · Thomas Gleixner
All event channel setups bind the interrupt on CPU0 or the target CPU for percpu interrupts and overwrite the affinity mask with the corresponding cpumask. That does not make sense.
The XEN implementation of
irqchip::irq_set_affinity()already picks a single target CPU out of the affinity mask and the actual target is stored in the effective CPU mask, so destroying the user chosen affinity mask which might contain more than one CPU is wrong.Change the implementation so that the channel is bound to CPU0 at the XEN level and leave the affinity mask alone. At startup of the interrupt affinity will be assigned out of the affinity mask and the XEN binding will be updated.
Quoted 5 lines
Quoted 1 line
If that's the case then I wonder whether we need this call at all and instead bind at startup time.
This binding to cpu0 was introduced with commit
97253eeeb792d61ed2and I have no reason to believe the underlying problem has been eliminated.
"The kernel-side VCPU binding was not being correctly set for newly
allocated or bound interdomain events. In ARM guests where 2-level events were used, this would result in no interdomain events being handled because the kernel-side VCPU masks would all be clear.
x86 guests would work because the irq affinity was set during irq setup and this would set the correct kernel-side VCPU binding."
I'm not convinced that this is really correctly analyzed because affinity setting is done at irq startup.
switch (__irq_startup_managed(desc, aff, force)) {
case IRQ_STARTUP_NORMAL:
ret = __irq_startup(desc);
irq_setup_affinity(desc);
break;which is completely architecture agnostic. So why should this magically work on x86 and not on ARM if both are using the same XEN irqchip with the same irqchip callbacks.
Thanks,
tglx
Jürgen Groß
maintainerRe: [patch 27/30] xen/events: Only force affinity mask for percpu interrupts
Quoted 42 lines · Thomas Gleixner
Quoted 18 lines · Jürgen Groß
Quoted 17 lines · On 11.12.20 00:20, [email protected] wrote:
Quoted 13 lines · Thomas Gleixner
All event channel setups bind the interrupt on CPU0 or the target CPU for percpu interrupts and overwrite the affinity mask with the corresponding cpumask. That does not make sense.
The XEN implementation of
irqchip::irq_set_affinity()already picks a single target CPU out of the affinity mask and the actual target is stored in the effective CPU mask, so destroying the user chosen affinity mask which might contain more than one CPU is wrong.Change the implementation so that the channel is bound to CPU0 at the XEN level and leave the affinity mask alone. At startup of the interrupt affinity will be assigned out of the affinity mask and the XEN binding will be updated.
Quoted 5 lines
Quoted 1 line
If that's the case then I wonder whether we need this call at all and instead bind at startup time.
This binding to cpu0 was introduced with commit
97253eeeb792d61ed2and I have no reason to believe the underlying problem has been eliminated."The kernel-side VCPU binding was not being correctly set for newly
allocated or bound interdomain events. In ARM guests where 2-level events were used, this would result in no interdomain events being handled because the kernel-side VCPU masks would all be clear.x86 guests would work because the irq affinity was set during irq setup and this would set the correct kernel-side VCPU binding."
I'm not convinced that this is really correctly analyzed because affinity setting is done at irq startup.
switch (__irq_startup_managed(desc, aff, force)) { case IRQ_STARTUP_NORMAL: ret = __irq_startup(desc); irq_setup_affinity(desc); break;
Quoted 3 lines
which is completely architecture agnostic. So why should this magically work on x86 and not on ARM if both are using the same XEN irqchip with the same irqchip callbacks.
I think this might be related to _initial_ cpu binding of events and changing the binding later. This might be handled differently in the hypervisor.
Juergen
Jürgen Groß
maintainerRe: [patch 27/30] xen/events: Only force affinity mask for percpu interrupts
In reply to [email protected]Quoted 11 lines · On 11.12.20 00:20, [email protected] wrote:
Quoted 9 lines · Thomas Gleixner
All event channel setups bind the interrupt on CPU0 or the target CPU for percpu interrupts and overwrite the affinity mask with the corresponding cpumask. That does not make sense.
The XEN implementation of
irqchip::irq_set_affinity()already picks a single target CPU out of the affinity mask and the actual target is stored in the effective CPU mask, so destroying the user chosen affinity mask which might contain more than one CPU is wrong.
Quoted 7 lines
Quoted 4 lines
Change the implementation so that the channel is bound to CPU0 at the XEN level and leave the affinity mask alone. At startup of the interrupt affinity will be assigned out of the affinity mask and the XEN binding will be updated.
If that's the case then I wonder whether we need this call at all and instead bind at startup time.
After some discussion with Thomas on IRC and xen-devel archaeology the result is: this will be needed especially for systems running on a single vcpu (e.g. small guests), as the .irq_set_affinity() callback won't be called in this case when starting the irq.
Juergen
Re: [patch 27/30] xen/events: Only force affinity mask for percpu interrupts
Quoted 18 lines · Jürgen Groß
Quoted 17 lines · On 11.12.20 00:20, [email protected] wrote:
Quoted 13 lines · Thomas Gleixner
All event channel setups bind the interrupt on CPU0 or the target CPU for percpu interrupts and overwrite the affinity mask with the corresponding cpumask. That does not make sense.
The XEN implementation of
irqchip::irq_set_affinity()already picks a single target CPU out of the affinity mask and the actual target is stored in the effective CPU mask, so destroying the user chosen affinity mask which might contain more than one CPU is wrong.Change the implementation so that the channel is bound to CPU0 at the XEN level and leave the affinity mask alone. At startup of the interrupt affinity will be assigned out of the affinity mask and the XEN binding will be updated.
Quoted 6 lines
Quoted 1 line
If that's the case then I wonder whether we need this call at all and instead bind at startup time.
After some discussion with Thomas on IRC and xen-devel archaeology the result is: this will be needed especially for systems running on a single vcpu (e.g. small guests), as the .irq_set_affinity() callback won't be called in this case when starting the irq.
That's right, but not limited to ARM. The same problem exists on x86 UP. So yes, the call makes sense, but the changelog is not really useful. Let me add a comment to this.
Thanks,
tglx
Re: [patch 27/30] xen/events: Only force affinity mask for percpu interrupts
Quoted 17 lines · Thomas Gleixner
Quoted 16 lines · Jürgen Groß
Quoted 15 lines · On 11.12.20 00:20, [email protected] wrote:
Quoted 13 lines · Thomas Gleixner
All event channel setups bind the interrupt on CPU0 or the target CPU for percpu interrupts and overwrite the affinity mask with the corresponding cpumask. That does not make sense.
The XEN implementation of
irqchip::irq_set_affinity()already picks a single target CPU out of the affinity mask and the actual target is stored in the effective CPU mask, so destroying the user chosen affinity mask which might contain more than one CPU is wrong.Change the implementation so that the channel is bound to CPU0 at the XEN level and leave the affinity mask alone. At startup of the interrupt affinity will be assigned out of the affinity mask and the XEN binding will be updated.
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
If that's the case then I wonder whether we need this call at all and instead bind at startup time.
After some discussion with Thomas on IRC and xen-devel archaeology the result is: this will be needed especially for systems running on a single vcpu (e.g. small guests), as the .irq_set_affinity() callback won't be called in this case when starting the irq.
On UP are we not then going to end up with an empty affinity mask? Or are we guaranteed to have it set to 1 by interrupt generic code?
This is actually why I brought this up in the first place --- a potential mismatch between the affinity mask and Xen-specific data (e.g. info->cpu and then protocol-specific data in event channel code). Even if they are re-synchronized later, at startup time (for SMP).
I don't see anything that would cause a problem right now but I worry that this inconsistency may come up at some point.
-boris
Quoted 8 lines
That's right, but not limited to ARM. The same problem exists on x86 UP. So yes, the call makes sense, but the changelog is not really useful. Let me add a comment to this.
Thanks,
tglx
Re: [patch 27/30] xen/events: Only force affinity mask for percpu interrupts
Quoted 9 lines · boris ostrovsky
Quoted 8 lines · Thomas Gleixner
Quoted 7 lines · Jürgen Groß
Quoted 6 lines · On 11.12.20 00:20, [email protected] wrote:
Quoted 4 lines · Thomas Gleixner
Change the implementation so that the channel is bound to CPU0 at the XEN level and leave the affinity mask alone. At startup of the interrupt affinity will be assigned out of the affinity mask and the XEN binding will be updated.
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
If that's the case then I wonder whether we need this call at all and instead bind at startup time.
After some discussion with Thomas on IRC and xen-devel archaeology the result is: this will be needed especially for systems running on a single vcpu (e.g. small guests), as the .irq_set_affinity() callback won't be called in this case when starting the irq.
On UP are we not then going to end up with an empty affinity mask? Or are we guaranteed to have it set to 1 by interrupt generic code?
An UP kernel does not ever look on the affinity mask. The chip::irq_set_affinity() callback is not invoked so the mask is irrelevant.
A SMP kernel on a UP machine sets CPU0 in the mask so all is good.
Quoted 5 lines
This is actually why I brought this up in the first place --- a potential mismatch between the affinity mask and Xen-specific data (e.g.
info->cpuand then protocol-specific data in event channel code). Even if they are re-synchronized later, at startup time (for SMP).
Which is not a problem either. The affinity mask is only relevant for setting the affinity, but it's not relevant for delivery and never can be.
Quoted 2 lines
I don't see anything that would cause a problem right now but I worry that this inconsistency may come up at some point.
As long as the affinity mask becomes not part of the event channel magic this should never matter.
Look at it from hardware:
interrupt is affine to CPU0
CPU0 runs:
set_affinity(CPU0 -> CPU1)
local_irq_disable()--> interrupt is raised in hardware and pending on CPU0
irq hardware is reconfigured to be affine to CPU1
local_irq_enable()
--> interrupt is handled on CPU0
the next interrupt will be raised on CPU1
So info->cpu which is registered via the hypercall binds the 'hardware delivery' and whenever the new affinity is written it is rebound to some other CPU and the next interrupt is then raised on this other CPU.
It's not any different from the hardware example at least not as far as I understood the code.
Thanks,
tglx
Re: [patch 27/30] xen/events: Only force affinity mask for percpu interrupts
Quoted 54 lines · Thomas Gleixner
Quoted 8 lines · boris ostrovsky
Quoted 7 lines · Thomas Gleixner
Quoted 6 lines · Jürgen Groß
Quoted 5 lines · On 11.12.20 00:20, [email protected] wrote:
Quoted 4 lines · Thomas Gleixner
Change the implementation so that the channel is bound to CPU0 at the XEN level and leave the affinity mask alone. At startup of the interrupt affinity will be assigned out of the affinity mask and the XEN binding will be updated.
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
If that's the case then I wonder whether we need this call at all and instead bind at startup time.
After some discussion with Thomas on IRC and xen-devel archaeology the result is: this will be needed especially for systems running on a single vcpu (e.g. small guests), as the .irq_set_affinity() callback won't be called in this case when starting the irq.
On UP are we not then going to end up with an empty affinity mask? Or are we guaranteed to have it set to 1 by interrupt generic code?
An UP kernel does not ever look on the affinity mask. The
chip::irq_set_affinity()callback is not invoked so the mask is irrelevant.A SMP kernel on a UP machine sets CPU0 in the mask so all is good.
Quoted 5 lines
This is actually why I brought this up in the first place --- a potential mismatch between the affinity mask and Xen-specific data (e.g.
info->cpuand then protocol-specific data in event channel code). Even if they are re-synchronized later, at startup time (for SMP).Which is not a problem either. The affinity mask is only relevant for setting the affinity, but it's not relevant for delivery and never can be.
Quoted 2 lines
I don't see anything that would cause a problem right now but I worry that this inconsistency may come up at some point.
As long as the affinity mask becomes not part of the event channel magic this should never matter.
Look at it from hardware:
interrupt is affine to CPU0
CPU0 runs: set_affinity(CPU0 -> CPU1) local_irq_disable()--> interrupt is raised in hardware and pending on CPU0
irq hardware is reconfigured to be affine to CPU1
local_irq_enable()--> interrupt is handled on CPU0
Quoted 8 lines
the next interrupt will be raised on CPU1
So
info->cpuwhich is registered via the hypercall binds the 'hardware delivery' and whenever the new affinity is written it is rebound to some other CPU and the next interrupt is then raised on this other CPU.It's not any different from the hardware example at least not as far as I understood the code.
Xen's event channels do have a couple of quirks.
Binding an event channel always results in one spurious event being delivered. This is to cover notifications which can get lost during the bidirectional setup, or re-setups in certain configurations.
Binding an interdomain or pirq event channel always defaults to vCPU0. There is no way to atomically set the affinity while binding. I believe the API predates SMP guest support in Xen, and noone has fixed it up since.
As a consequence, the guest will observe the event raised on vCPU0 as part of setting up the event, even if it attempts to set a different affinity immediately afterwards. A little bit of care needs to be taken when binding an event channel on vCPUs other than 0, to ensure that the callback is safe with respect to any remaining state needing initialisation.
Beyond this, there is nothing magic I'm aware of.
We have seen soft lockups before in certain scenarios, simply due to the quantity of events hitting vCPU0 before irqbalance gets around to spreading the load. This is why there is an attempt to round-robin the userspace event channel affinities by default, but I still don't see why this would need custom affinity logic itself.
Thanks,
~Andrew
Re: [patch 27/30] xen/events: Only force affinity mask for percpu interrupts
Andrew,
Quoted 5 lines · Andrew Cooper
Quoted 2 lines · Thomas Gleixner
It's not any different from the hardware example at least not as far as I understood the code.
Xen's event channels do have a couple of quirks.
Why am I not surprised?
Quoted 8 lines
Binding an event channel always results in one spurious event being delivered. This is to cover notifications which can get lost during the bidirectional setup, or re-setups in certain configurations.
Binding an interdomain or pirq event channel always defaults to vCPU0. There is no way to atomically set the affinity while binding. I believe the API predates SMP guest support in Xen, and noone has fixed it up since.
That's fine. I'm not changing that.
What I'm changing is the unwanted and unnecessary overwriting of the actual affinity mask.
We have a similar issue on real hardware where we can only target _one_ CPU and not all CPUs in the affinity mask. So we still can preserve the (user) requested mask and just affine it to one CPU which is reflected in the effective affinity mask. This the right thing to do for two reasons:
1) It allows proper interrupt distribution
2) It does not break (user) requested affinity when the effective target CPU goes offline and the affinity mask still contains online CPUs. If you overwrite it you lost track of the requested broader mask.
Quoted 6 lines
As a consequence, the guest will observe the event raised on vCPU0 as part of setting up the event, even if it attempts to set a different affinity immediately afterwards. A little bit of care needs to be taken when binding an event channel on vCPUs other than 0, to ensure that the callback is safe with respect to any remaining state needing initialisation.
That's preserved for all non percpu interrupts. The percpu variant of VIRQ and IPIs did binding to vCPU != 0 already before this change.
Quoted 7 lines
Beyond this, there is nothing magic I'm aware of.
We have seen soft lockups before in certain scenarios, simply due to the quantity of events hitting vCPU0 before irqbalance gets around to spreading the load. This is why there is an attempt to round-robin the userspace event channel affinities by default, but I still don't see why this would need custom affinity logic itself.
Just the previous attempt makes no sense for the reasons I outlined in the changelog. So now with this new spreading mechanics you get the distribution for all cases:
1) Post setup using and respecting the default affinity mask which can be set as a kernel commandline parameter.
2) Runtime (user) requested affinity change with a mask which contains more than one vCPU. The previous logic always chose the first one in the mask.
So assume userspace affines 4 irqs to a CPU 0-3 and 4 irqs to CPU 4-7 then 4 irqs end up on CPU0 and 4 on CPU4
The new algorithm which is similar to what we have on x86 (minus the vector space limitation) picks the CPU which has the least number of channels affine to it at that moment. If e.g. all 8 CPUs have the same number of vectors before that change then in the example above the first 4 are spread to CPU0-3 and the second 4 to CPU4-7
Thanks,
tglx
[tip: irq/core] xen/events: Only force affinity mask for percpu interrupts
In reply to Thomas GleixnerThe following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: f61b8351facda691f1564097074976b30e987a81 Gitweb: https://git.kernel.org/tip/f61b8351facda691f1564097074976b30e987a81 Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:26:03 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:06 +01:00
xen/events: Only force affinity mask for percpu interrupts
All event channel setups bind the interrupt on CPU0 or the target CPU for percpu interrupts and overwrite the affinity mask with the corresponding cpumask. That does not make sense.
The XEN implementation of irqchip::irq_set_affinity() already picks a single target CPU out of the affinity mask and the actual target is stored in the effective CPU mask, so destroying the user chosen affinity mask which might contain more than one CPU is wrong.
Change the implementation so that the channel is bound to CPU0 at the XEN level and leave the affinity mask alone. At startup of the interrupt affinity will be assigned out of the affinity mask and the XEN binding will be updated. Only keep the enforcement for real percpu interrupts.
On resume the overwrite is not required either because info->cpu and the affinity mask are still the same as at the time of suspend. Same for rebind_evtchn_irq().
This also prepares for proper interrupt spreading.
--- drivers/xen/events/events_base.c | 48 +++++++++++++++++++++---------- 1 file changed, 34 insertions(+), 14 deletions(-)
drivers/xen/events/events_base.c +34 -14
@@ -433,15 +433,20 @@ static bool pirq_needs_eoi_flag(unsigned irq)return info->u.pirq.flags &PIRQ_NEEDS_EOI ; } -static void bind_evtchn_to_cpu (evtchn_port_t evtchn,unsigned int cpu) +static void bind_evtchn_to_cpu (evtchn_port_t evtchn,unsigned int cpu, +bool force_affinity) {int irq =get_evtchn_to_irq (evtchn);struct irq_info *info =info_for_irq (irq);BUG_ON (irq ==-1 ); -#ifdef CONFIG_SMP -cpumask_copy (irq_get_affinity_mask (irq),cpumask_of (cpu)); -#endif + +if (IS_ENABLED (CONFIG_SMP ) && force_affinity) { +cpumask_copy (irq_get_affinity_mask (irq),cpumask_of (cpu)); +cpumask_copy (irq_get_effective_affinity_mask (irq), +cpumask_of (cpu)); + } +xen_evtchn_port_bind_to_cpu (evtchn, cpu, info->cpu); info->cpu = cpu;
@@ -788,7 +793,7 @@ static unsigned int __startup_pirq(unsigned int irq) goto err; info->evtchn = evtchn; -bind_evtchn_to_cpu (evtchn,0 ); +bind_evtchn_to_cpu (evtchn,0 , false); rc =xen_evtchn_port_setup (evtchn);if (rc)
@@ -1107,8 +1112,14 @@ static int bind_evtchn_to_irq_chip(evtchn_port_t evtchn, struct irq_chip *chip) irq = ret; goto out; } -/* New interdomain events are bound to VCPU 0. */ -bind_evtchn_to_cpu (evtchn,0 ); +/* +* New interdomain events are initially bound to vCPU0 This +* is required to setup the event channel in the first +* place and also important for UP guests because the +* affinity setting is not invoked on them so nothing would +* bind the channel. +*/ +bind_evtchn_to_cpu (evtchn,0 , false); }else {struct irq_info *info =info_for_irq (irq);WARN_ON (info ==NULL || info->type !=IRQT_EVTCHN );
@@ -1156,7 +1167,11 @@ static int bind_ipi_to_irq(unsigned int ipi, unsigned int cpu) irq = ret; goto out; } -bind_evtchn_to_cpu (evtchn, cpu); +/* +* Force the affinity mask to the target CPU so proc shows +* the correct target. +*/ +bind_evtchn_to_cpu (evtchn, cpu, true); }else {struct irq_info *info =info_for_irq (irq);WARN_ON (info ==NULL || info->type !=IRQT_IPI );
@@ -1269,7 +1284,11 @@ int bind_virq_to_irq(unsigned int virq, unsigned int cpu, bool percpu) goto out; } -bind_evtchn_to_cpu (evtchn, cpu); +/* +* Force the affinity mask for percpu interrupts so proc +* shows the correct target. +*/ +bind_evtchn_to_cpu (evtchn, cpu, percpu); }else {struct irq_info *info =info_for_irq (irq);WARN_ON (info ==NULL || info->type !=IRQT_VIRQ );
@@ -1634,8 +1653,7 @@ void rebind_evtchn_irq(evtchn_port_t evtchn, int irq)mutex_unlock (&irq_mapping_update_lock); -bind_evtchn_to_cpu (evtchn, info->cpu); -irq_set_affinity (irq,cpumask_of (info->cpu)); +bind_evtchn_to_cpu (evtchn, info->cpu, false);/* Unmask the event channel. */ enable_irq (irq);
@@ -1669,7 +1687,7 @@ static int xen_rebind_evtchn_to_cpu(evtchn_port_t evtchn, unsigned int tcpu)* it, but don't do the xenlinux-level rebind in that case. */ if (HYPERVISOR_event_channel_op (EVTCHNOP_bind_vcpu, &bind_vcpu) >=0 ) -bind_evtchn_to_cpu (evtchn, tcpu); +bind_evtchn_to_cpu (evtchn, tcpu, false);if (!masked)unmask_evtchn (evtchn);
@@ -1798,7 +1816,8 @@ static void restore_cpu_virqs(unsigned int cpu)/* Record the new mapping. */ (void )xen_irq_info_virq_setup (cpu, irq, evtchn, virq); -bind_evtchn_to_cpu (evtchn, cpu); +/* The affinity mask is still valid */ +bind_evtchn_to_cpu (evtchn, cpu, false); } }
@@ -1823,7 +1842,8 @@ static void restore_cpu_ipis(unsigned int cpu)/* Record the new mapping. */ (void )xen_irq_info_ipi_setup (cpu, irq, evtchn, ipi); -bind_evtchn_to_cpu (evtchn, cpu); +/* The affinity mask is still valid */ +bind_evtchn_to_cpu (evtchn, cpu, false); } }
PATCH 28/30 xen/events: Reduce irq_info::spurious_cnt storage size
In reply to Thomas GleixnerTo prepare for interrupt spreading reduce the storage size of irq_info::spurious_cnt to u8 so the required flag for the spreading logic will not increase the storage size.
Protect the usage site against overruns.
--- drivers/xen/events/events_base.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
drivers/xen/events/events_base.c +5 -3
@@ -95,7 +95,7 @@ struct irq_info {struct list_head list;struct list_head eoi_list;short refcnt; -short spurious_cnt; +u8 spurious_cnt;enum xen_irq_type type;/* type */ unsigned irq;evtchn_port_t evtchn;/* event channel */
@@ -528,8 +528,10 @@ static void xen_irq_lateeoi_locked(strucreturn ;if (spurious) { -if ((1 << info->spurious_cnt) < (HZ <<2 )) - info->spurious_cnt++; +if ((1 << info->spurious_cnt) < (HZ <<2 )) { +if (info->spurious_cnt !=0xFF ) + info->spurious_cnt++; + }if (info->spurious_cnt >1 ) { delay =1 << (info->spurious_cnt -2 );if (delay >HZ )
[tip: irq/core] xen/events: Reduce irq_info:: Spurious_cnt storage size
The following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 6e8e8d49554722ddc076a6b87dc7048ddceb2375 Gitweb: https://git.kernel.org/tip/6e8e8d49554722ddc076a6b87dc7048ddceb2375 Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:26:04 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:07 +01:00
xen/events: Reduce irq_info:: Spurious_cnt storage size
To prepare for interrupt spreading reduce the storage size of irq_info::spurious_cnt to u8 so the required flag for the spreading logic will not increase the storage size.
Protect the usage site against overruns.
--- drivers/xen/events/events_base.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
drivers/xen/events/events_base.c +5 -3
@@ -95,7 +95,7 @@ struct irq_info {struct list_head list;struct list_head eoi_list;short refcnt; -short spurious_cnt; +u8 spurious_cnt;enum xen_irq_type type;/* type */ unsigned irq;evtchn_port_t evtchn;/* event channel */
@@ -528,8 +528,10 @@ static void xen_irq_lateeoi_locked(struct irq_info *info, bool spurious)return ;if (spurious) { -if ((1 << info->spurious_cnt) < (HZ <<2 )) - info->spurious_cnt++; +if ((1 << info->spurious_cnt) < (HZ <<2 )) { +if (info->spurious_cnt !=0xFF ) + info->spurious_cnt++; + }if (info->spurious_cnt >1 ) { delay =1 << (info->spurious_cnt -2 );if (delay >HZ )
PATCH 29/30 xen/events: Implement irq distribution
In reply to Thomas GleixnerKeep track of the assignments of event channels to CPUs and select the online CPU with the least assigned channels in the affinity mask which is handed to irq_chip::irq_set_affinity() from the core code.
--- drivers/xen/events/events_base.c | 72 ++++++++++++++++++++++++++++++++++----- 1 file changed, 64 insertions(+), 8 deletions(-)
drivers/xen/events/events_base.c +64 -8
@@ -96,6 +96,7 @@ struct irq_info {struct list_head eoi_list;short refcnt;u8 spurious_cnt; +u8 is_accounted;enum xen_irq_type type;/* type */ unsigned irq;evtchn_port_t evtchn;/* event channel */
@@ -161,6 +162,9 @@ static DEFINE_PER_CPU(int [NR_VIRQS], vi/* IRQ <-> IPI mapping */ static DEFINE_PER_CPU(int [XEN_NR_IPIS], ipi_to_irq) = {[0 ... XEN_NR_IPIS-1] = -1}; +/* Event channel distribution data */ +static atomic_t channels_on_cpu[NR_CPUS ]; +static int **evtchn_to_irq;#ifdef CONFIG_X86 static unsigned long *pirq_eoi_map;
@@ -257,6 +261,32 @@ static void set_info_for_irq(unsigned inirq_set_chip_data (irq, info); } +/* Per CPU channel accounting */ +static void channels_on_cpu_dec (struct irq_info *info) +{ +if (!info->is_accounted) +return ; + + info->is_accounted =0 ; + +if (WARN_ON_ONCE (info->cpu >= nr_cpu_ids)) +return ; + +WARN_ON_ONCE (!atomic_add_unless (&channels_on_cpu[info->cpu],-1 ,0 )); +} + +static void channels_on_cpu_inc (struct irq_info *info) +{ +if (WARN_ON_ONCE (info->cpu >= nr_cpu_ids)) +return ; + +if (WARN_ON_ONCE (!atomic_add_unless (&channels_on_cpu[info->cpu],1 , +INT_MAX ))) +return ; + + info->is_accounted =1 ; +} +/* Constructors for packed IRQ information. */ static int xen_irq_info_common_setup(struct irq_info *info, unsigned irq,
@@ -339,6 +369,7 @@ static void xen_irq_info_cleanup(struct {set_evtchn_to_irq (info->evtchn,-1 ); info->evtchn =0 ; +channels_on_cpu_dec (info); }/*
@@ -449,7 +480,9 @@ static void bind_evtchn_to_cpu(evtchn_poxen_evtchn_port_bind_to_cpu (evtchn, cpu, info->cpu); +channels_on_cpu_dec (info); info->cpu = cpu; +channels_on_cpu_inc (info); }/**
@@ -622,11 +655,6 @@ static void xen_irq_init(unsigned irq) {struct irq_info *info; -#ifdef CONFIG_SMP -/* By default all event channels notify CPU#0. */ -cpumask_copy (irq_get_affinity_mask (irq),cpumask_of (0 )); -#endif - info =kzalloc (sizeof (*info),GFP_KERNEL );if (info ==NULL )panic ("Unable to allocate metadata for IRQ%d\n" , irq);
@@ -1691,10 +1719,34 @@ static int xen_rebind_evtchn_to_cpu(evtcreturn 0 ; } +/* +* Find the CPU within @dest mask which has the least number of channels +* assigned. This is not precise as the per cpu counts can be modified +* concurrently. +*/ +static unsigned int select_target_cpu (const struct cpumask *dest) +{ +unsigned int cpu, best_cpu =UINT_MAX , minch =UINT_MAX ; + +for_each_cpu_and (cpu, dest, cpu_online_mask) { +unsigned int curch =atomic_read (&channels_on_cpu[cpu]); + +if (curch < minch) { + minch = curch; + best_cpu = cpu; + } + } + +/* If this happens accounting is screwed up */ +if (WARN_ON_ONCE (best_cpu ==UINT_MAX )) + best_cpu =cpumask_first (dest); +return best_cpu; +} +static int set_affinity_irq (struct irq_data *data,const struct cpumask *dest,bool force) { -unsigned tcpu =cpumask_first_and (dest, cpu_online_mask); +unsigned int tcpu =select_target_cpu (dest);int ret; ret =xen_rebind_evtchn_to_cpu (evtchn_from_irq (data->irq), tcpu);
@@ -1922,8 +1974,12 @@ void xen_irq_resume(void)xen_evtchn_resume ();/* No IRQ <-> event-channel mappings. */ -list_for_each_entry (info, &xen_irq_list_head, list) - info->evtchn =0 ;/* zap event-channel binding */ +list_for_each_entry (info, &xen_irq_list_head, list) { +/* Zap event-channel binding */ + info->evtchn =0 ; +/* Adjust accounting */ +channels_on_cpu_dec (info); + }clear_evtchn_to_irq_all ();
[tip: irq/core] xen/events: Implement irq distribution
The following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 93b6adbf0420a7bd84167a4feb289cc2c1b4efca Gitweb: https://git.kernel.org/tip/93b6adbf0420a7bd84167a4feb289cc2c1b4efca Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:26:05 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:07 +01:00
xen/events: Implement irq distribution
Keep track of the assignments of event channels to CPUs and select the online CPU with the least assigned channels in the affinity mask which is handed to irq_chip::irq_set_affinity() from the core code.
--- drivers/xen/events/events_base.c | 76 +++++++++++++++++++++++++++---- 1 file changed, 68 insertions(+), 8 deletions(-)
drivers/xen/events/events_base.c +68 -8
@@ -96,6 +96,7 @@ struct irq_info {struct list_head eoi_list;short refcnt;u8 spurious_cnt; +u8 is_accounted;enum xen_irq_type type;/* type */ unsigned irq;evtchn_port_t evtchn;/* event channel */
@@ -161,6 +162,9 @@ static DEFINE_PER_CPU(int [NR_VIRQS], virq_to_irq) = {[0 ... NR_VIRQS-1] = -1};/* IRQ <-> IPI mapping */ static DEFINE_PER_CPU(int [XEN_NR_IPIS], ipi_to_irq) = {[0 ... XEN_NR_IPIS-1] = -1}; +/* Event channel distribution data */ +static atomic_t channels_on_cpu[NR_CPUS ]; +static int **evtchn_to_irq;#ifdef CONFIG_X86 static unsigned long *pirq_eoi_map;
@@ -257,6 +261,32 @@ static void set_info_for_irq(unsigned int irq, struct irq_info *info)irq_set_chip_data (irq, info); } +/* Per CPU channel accounting */ +static void channels_on_cpu_dec (struct irq_info *info) +{ +if (!info->is_accounted) +return ; + + info->is_accounted =0 ; + +if (WARN_ON_ONCE (info->cpu >= nr_cpu_ids)) +return ; + +WARN_ON_ONCE (!atomic_add_unless (&channels_on_cpu[info->cpu],-1 ,0 )); +} + +static void channels_on_cpu_inc (struct irq_info *info) +{ +if (WARN_ON_ONCE (info->cpu >= nr_cpu_ids)) +return ; + +if (WARN_ON_ONCE (!atomic_add_unless (&channels_on_cpu[info->cpu],1 , +INT_MAX ))) +return ; + + info->is_accounted =1 ; +} +/* Constructors for packed IRQ information. */ static int xen_irq_info_common_setup(struct irq_info *info, unsigned irq,
@@ -339,6 +369,7 @@ static void xen_irq_info_cleanup(struct irq_info *info) {set_evtchn_to_irq (info->evtchn,-1 ); info->evtchn =0 ; +channels_on_cpu_dec (info); }/*
@@ -449,7 +480,9 @@ static void bind_evtchn_to_cpu(evtchn_port_t evtchn, unsigned int cpu,xen_evtchn_port_bind_to_cpu (evtchn, cpu, info->cpu); +channels_on_cpu_dec (info); info->cpu = cpu; +channels_on_cpu_inc (info); }/**
@@ -622,11 +655,6 @@ static void xen_irq_init(unsigned irq) {struct irq_info *info; -#ifdef CONFIG_SMP -/* By default all event channels notify CPU#0. */ -cpumask_copy (irq_get_affinity_mask (irq),cpumask_of (0 )); -#endif - info =kzalloc (sizeof (*info),GFP_KERNEL );if (info ==NULL )panic ("Unable to allocate metadata for IRQ%d\n" , irq);
@@ -1697,10 +1725,38 @@ static int xen_rebind_evtchn_to_cpu(evtchn_port_t evtchn, unsigned int tcpu)return 0 ; } +/* +* Find the CPU within @dest mask which has the least number of channels +* assigned. This is not precise as the per cpu counts can be modified +* concurrently. +*/ +static unsigned int select_target_cpu (const struct cpumask *dest) +{ +unsigned int cpu, best_cpu =UINT_MAX , minch =UINT_MAX ; + +for_each_cpu_and (cpu, dest, cpu_online_mask) { +unsigned int curch =atomic_read (&channels_on_cpu[cpu]); + +if (curch < minch) { + minch = curch; + best_cpu = cpu; + } + } + +/* +* Catch the unlikely case that dest contains no online CPUs. Can't +* recurse. +*/ +if (best_cpu ==UINT_MAX ) +return select_target_cpu (cpu_online_mask); + +return best_cpu; +} +static int set_affinity_irq (struct irq_data *data,const struct cpumask *dest,bool force) { -unsigned tcpu =cpumask_first_and (dest, cpu_online_mask); +unsigned int tcpu =select_target_cpu (dest);int ret; ret =xen_rebind_evtchn_to_cpu (evtchn_from_irq (data->irq), tcpu);
@@ -1928,8 +1984,12 @@ void xen_irq_resume(void)xen_evtchn_resume ();/* No IRQ <-> event-channel mappings. */ -list_for_each_entry (info, &xen_irq_list_head, list) - info->evtchn =0 ;/* zap event-channel binding */ +list_for_each_entry (info, &xen_irq_list_head, list) { +/* Zap event-channel binding */ + info->evtchn =0 ; +/* Adjust accounting */ +channels_on_cpu_dec (info); + }clear_evtchn_to_irq_all ();
PATCH 30/30 genirq: Remove export of irq_to_desc()
In reply to Thomas GleixnerNo more (ab)use in modules finally. Remove the export so there won't come new ones.
--- kernel/irq/irqdesc.c | 1 - 1 file changed, 1 deletion(-)
kernel/irq/irqdesc.c +0 -1
@@ -352,7 +352,6 @@ struct irq_desc *irq_to_desc(unsigned in {return radix_tree_lookup (&irq_desc_tree, irq); } -EXPORT_SYMBOL (irq_to_desc);static void delete_irq_desc (unsigned int irq) {
[tip: irq/core] genirq: Remove export of irq_to_desc()
The following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: f07147b162a1cec348d8a9a7e7e73fdf7ce5f88b Gitweb: https://git.kernel.org/tip/f07147b162a1cec348d8a9a7e7e73fdf7ce5f88b Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 10 Dec 2020 20:26:06 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sat, 12 Dec 2020 12:59:07 +01:00
genirq: Remove export of irq_to_desc()
No more (ab)use in modules finally. Remove the export so there won't come new ones.
--- kernel/irq/irqdesc.c | 1 - 1 file changed, 1 deletion(-)
kernel/irq/irqdesc.c +0 -1
@@ -352,7 +352,6 @@ struct irq_desc *irq_to_desc(unsigned int irq) {return radix_tree_lookup (&irq_desc_tree, irq); } -EXPORT_SYMBOL (irq_to_desc);static void delete_irq_desc (unsigned int irq) {
Patchwork
CI[Intel-gfx] ✗ Fi.CI.BUILD: failure for genirq: Treewide hunt for irq descriptor abuse and assorted fixes
In reply to Thomas Gleixner== Series Details ==
== Series Details ==
Series: genirq: Treewide hunt for irq descriptor abuse and assorted fixes
URL : https://patchwork.freedesktop.org/series/84805/
State : failure
== Summary ==
Applying: genirq: Move irq_has_action() into core code
Applying: genirq: Move status flag checks to core
Applying: genirq: Move irq_set_lockdep_class() to core
Applying: genirq: Provide irq_get_effective_affinity()
Applying: genirq: Annotate irq stats data races
Applying: parisc/irq: Simplify irq count output for /proc/interrupts
Applying: genirq: Make kstat_irqs() static
Applying: genirq: Provide kstat_irqdesc_cpu()
Applying: ARM: smp: Use irq_desc_kstat_cpu() in show_ipi_list()
Applying: arm64/smp: Use irq_desc_kstat_cpu() in arch_show_interrupts()
Applying: parisc/irq: Use irq_desc_kstat_cpu() in show_interrupts()
Applying: s390/irq: Use irq_desc_kstat_cpu() in show_msi_interrupt()
Applying: drm/i915/lpe_audio: Remove pointless irq_to_desc() usage
Applying: drm/i915/pmu: Replace open coded kstat_irqs() copy
error: sha1 information is lacking or useless (drivers/gpu/drm/i915/i915_irq.c).
error: could not build fake ancestor
hint: Use 'git am --show-current-patch=diff' to see the failed patch Patch failed at 0014 drm/i915/pmu: Replace open coded kstat_irqs() copy When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx