irq_bcm2836: Send event when onlining sleeping cores
Subsystems (3)
PATCH 1/1 irq_bcm2836: Send event when onlining sleeping cores
In order to reduce power consumption and bus traffic, it is sensible for secondary cores to enter a low-power idle state when waiting to be started. The wfe instruction causes a core to wait until an event or interrupt arrives before continuing to the next instruction. The sev instruction sends a wakeup event to the other cores, so call it from bcm2836_smp_boot_secondary, the function that wakes up the waiting cores during booting.
It is harmless to use this patch without the corresponding change adding wfe to the ARMv7/ARMv8-32 stubs, but if the stubs are updated and this patch is not applied then the other cores will sleep forever.
See: https://github.com/raspberrypi/linux/issues/1989
--- drivers/irqchip/irq-bcm2836.c | 3 +++ 1 file changed, 3 insertions(+)
drivers/irqchip/irq-bcm2836.c +3 -0
@@ -248,6 +248,9 @@ static int __init bcm2836_smp_boot_secondary(unsigned int cpu,writel (secondary_startup_phys, intc.base +LOCAL_MAILBOX3_SET0 +16 * cpu); +dsb (sy);/* Ensure write has completed before waking the other CPUs */ +sev (); +return 0 ; }
Signature · 1 line
1.9.1
Quoted 24 lines · Phil Elwell
In order to reduce power consumption and bus traffic, it is sensible for secondary cores to enter a low-power idle state when waiting to be started. The wfe instruction causes a core to wait until an event or interrupt arrives before continuing to the next instruction. The sev instruction sends a wakeup event to the other cores, so call it from
bcm2836_smp_boot_secondary, the function that wakes up the waiting cores during booting.It is harmless to use this patch without the corresponding change adding wfe to the ARMv7/ARMv8-32 stubs, but if the stubs are updated and this patch is not applied then the other cores will sleep forever.
See: https://github.com/raspberrypi/linux/issues/1989
Signed-off-by Phil Elwell <[email protected]>--- drivers/irqchip/irq-bcm2836.c | 3 +++ 1 file changed, 3 insertions(+)drivers/irqchip/irq-bcm2836.c +0
-0
drivers/irqchip/irq-bcm2836.c static int __init bcm2836_smp_boot_secondary(unsigned int cpu, · line 255
Quoted 8 lines
drivers/irqchip/irq-bcm2836.c +3
-0@@ -248,6 +248,9 @@ static int __init bcm2836_smp_boot_secondary(unsigned int cpu,writel (secondary_startup_phys, intc.base +LOCAL_MAILBOX3_SET0 +16 * cpu); +dsb (sy);/* Ensure write has completed before waking the other CPUs */ +sev (); +return 0 ; }
This is also the behavior that the standard arm64 spin-table method has, which we unfortunately can't quite use.
Quoted 29 lines · Eric Anholt
Quoted 27 lines · Phil Elwell
In order to reduce power consumption and bus traffic, it is sensible for secondary cores to enter a low-power idle state when waiting to be started. The wfe instruction causes a core to wait until an event or interrupt arrives before continuing to the next instruction. The sev instruction sends a wakeup event to the other cores, so call it from
bcm2836_smp_boot_secondary, the function that wakes up the waiting cores during booting.It is harmless to use this patch without the corresponding change adding wfe to the ARMv7/ARMv8-32 stubs, but if the stubs are updated and this patch is not applied then the other cores will sleep forever.
See: https://github.com/raspberrypi/linux/issues/1989
Signed-off-by Phil Elwell <[email protected]>--- drivers/irqchip/irq-bcm2836.c | 3 +++ 1 file changed, 3 insertions(+)drivers/irqchip/irq-bcm2836.c +0
-0@@ -248,6 +248,9 @@ static int __init bcm2836_smp_boot_secondary(unsigned int cpu,writel (secondary_startup_phys, intc.base +LOCAL_MAILBOX3_SET0 +16 * cpu);
Quoted 8 lines
drivers/irqchip/irq-bcm2836.c static int __init bcm2836_smp_boot_secondary(unsigned int cpu, · line 255
Quoted 5 lines
drivers/irqchip/irq-bcm2836.c +3
-0+dsb (sy);/* Ensure write has completed before waking the other CPUs */ +sev (); +return 0 ; }This is also the behavior that the standard arm64 spin-table method has, which we unfortunately can't quite use.
And why is that so? Why do you have to reinvent the wheel (and hide the cloned wheel in an interrupt controller driver)?
That doesn't seem right to me.
M.
Signature · 1 line
Jazz is not dead. It just smells funny...
Quoted 39 lines · Marc Zyngier
Quoted 29 lines · Eric Anholt
Quoted 27 lines · Phil Elwell
In order to reduce power consumption and bus traffic, it is sensible for secondary cores to enter a low-power idle state when waiting to be started. The wfe instruction causes a core to wait until an event or interrupt arrives before continuing to the next instruction. The sev instruction sends a wakeup event to the other cores, so call it from
bcm2836_smp_boot_secondary, the function that wakes up the waiting cores during booting.It is harmless to use this patch without the corresponding change adding wfe to the ARMv7/ARMv8-32 stubs, but if the stubs are updated and this patch is not applied then the other cores will sleep forever.
See: https://github.com/raspberrypi/linux/issues/1989
Signed-off-by Phil Elwell <[email protected]>--- drivers/irqchip/irq-bcm2836.c | 3 +++ 1 file changed, 3 insertions(+)drivers/irqchip/irq-bcm2836.c +0
-0@@ -248,6 +248,9 @@ static int __init bcm2836_smp_boot_secondary(unsigned int cpu,writel (secondary_startup_phys, intc.base +LOCAL_MAILBOX3_SET0 +16 * cpu);Quoted 8 lines
drivers/irqchip/irq-bcm2836.c static int __init bcm2836_smp_boot_secondary(unsigned int cpu, · line 255
Quoted 5 lines
drivers/irqchip/irq-bcm2836.c +3
-0+dsb (sy);/* Ensure write has completed before waking the other CPUs */ +sev (); +return 0 ; }This is also the behavior that the standard arm64 spin-table method has, which we unfortunately can't quite use.
Quoted 4 lines
And why is that so? Why do you have to reinvent the wheel (and hide the cloned wheel in an interrupt controller driver)?
That doesn't seem right to me.
The armv8 stubs (firmware-supplied code in the low page that do the spinning) do actually implement arm64's spin-table method. It's the armv7 stubs that use these registers in the irqchip instead of plain addresses in system memory.
Quoted 44 lines · Eric Anholt
Quoted 42 lines · Marc Zyngier
Quoted 29 lines · Eric Anholt
Quoted 27 lines · Phil Elwell
In order to reduce power consumption and bus traffic, it is sensible for secondary cores to enter a low-power idle state when waiting to be started. The wfe instruction causes a core to wait until an event or interrupt arrives before continuing to the next instruction. The sev instruction sends a wakeup event to the other cores, so call it from
bcm2836_smp_boot_secondary, the function that wakes up the waiting cores during booting.It is harmless to use this patch without the corresponding change adding wfe to the ARMv7/ARMv8-32 stubs, but if the stubs are updated and this patch is not applied then the other cores will sleep forever.
See: https://github.com/raspberrypi/linux/issues/1989
Signed-off-by Phil Elwell <[email protected]>--- drivers/irqchip/irq-bcm2836.c | 3 +++ 1 file changed, 3 insertions(+)drivers/irqchip/irq-bcm2836.c +0
-0@@ -248,6 +248,9 @@ static int __init bcm2836_smp_boot_secondary(unsigned int cpu,writel (secondary_startup_phys, intc.base +LOCAL_MAILBOX3_SET0 +16 * cpu);Quoted 8 lines
drivers/irqchip/irq-bcm2836.c static int __init bcm2836_smp_boot_secondary(unsigned int cpu, · line 255
Quoted 5 lines
drivers/irqchip/irq-bcm2836.c +3
-0+dsb (sy);/* Ensure write has completed before waking the other CPUs */ +sev (); +return 0 ; }This is also the behavior that the standard arm64 spin-table method has, which we unfortunately can't quite use.
And why is that so? Why do you have to reinvent the wheel (and hide the cloned wheel in an interrupt controller driver)?
Quoted 6 lines
Quoted 1 line
That doesn't seem right to me.
The armv8 stubs (firmware-supplied code in the low page that do the spinning) do actually implement arm64's spin-table method. It's the armv7 stubs that use these registers in the irqchip instead of plain addresses in system memory.
Let's put ARMv7 aside for the time being. If your firmware already implements spin-tables, why don't you simply use that at least on arm64?
Thanks,
M.
Signature · 1 line
Jazz is not dead. It just smells funny...
Quoted 47 lines · Marc Zyngier
Quoted 46 lines · Eric Anholt
Quoted 43 lines · Marc Zyngier
Quoted 29 lines · Eric Anholt
Quoted 27 lines · Phil Elwell
In order to reduce power consumption and bus traffic, it is sensible for secondary cores to enter a low-power idle state when waiting to be started. The wfe instruction causes a core to wait until an event or interrupt arrives before continuing to the next instruction. The sev instruction sends a wakeup event to the other cores, so call it from
bcm2836_smp_boot_secondary, the function that wakes up the waiting cores during booting.It is harmless to use this patch without the corresponding change adding wfe to the ARMv7/ARMv8-32 stubs, but if the stubs are updated and this patch is not applied then the other cores will sleep forever.
See: https://github.com/raspberrypi/linux/issues/1989
Signed-off-by Phil Elwell <[email protected]>--- drivers/irqchip/irq-bcm2836.c | 3 +++ 1 file changed, 3 insertions(+)drivers/irqchip/irq-bcm2836.c +0
-0@@ -248,6 +248,9 @@ static int __init bcm2836_smp_boot_secondary(unsigned int cpu,writel (secondary_startup_phys, intc.base +LOCAL_MAILBOX3_SET0 +16 * cpu);Quoted 8 lines
drivers/irqchip/irq-bcm2836.c static int __init bcm2836_smp_boot_secondary(unsigned int cpu, · line 255
Quoted 5 lines
drivers/irqchip/irq-bcm2836.c +3
-0+dsb (sy);/* Ensure write has completed before waking the other CPUs */ +sev (); +return 0 ; }This is also the behavior that the standard arm64 spin-table method has, which we unfortunately can't quite use.
And why is that so? Why do you have to reinvent the wheel (and hide the cloned wheel in an interrupt controller driver)?
That doesn't seem right to me.
Quoted 7 lines
Quoted 4 lines
The armv8 stubs (firmware-supplied code in the low page that do the spinning) do actually implement arm64's spin-table method. It's the armv7 stubs that use these registers in the irqchip instead of plain addresses in system memory.
Let's put ARMv7 aside for the time being. If your firmware already implements spin-tables, why don't you simply use that at least on arm64?
We do.
Quoted 53 lines · Phil Elwell
Quoted 52 lines · Marc Zyngier
Quoted 50 lines · Eric Anholt
Quoted 39 lines · Marc Zyngier
Quoted 29 lines · Eric Anholt
Quoted 27 lines · Phil Elwell
In order to reduce power consumption and bus traffic, it is sensible for secondary cores to enter a low-power idle state when waiting to be started. The wfe instruction causes a core to wait until an event or interrupt arrives before continuing to the next instruction. The sev instruction sends a wakeup event to the other cores, so call it from
bcm2836_smp_boot_secondary, the function that wakes up the waiting cores during booting.It is harmless to use this patch without the corresponding change adding wfe to the ARMv7/ARMv8-32 stubs, but if the stubs are updated and this patch is not applied then the other cores will sleep forever.
See: https://github.com/raspberrypi/linux/issues/1989
Signed-off-by Phil Elwell <[email protected]>--- drivers/irqchip/irq-bcm2836.c | 3 +++ 1 file changed, 3 insertions(+)drivers/irqchip/irq-bcm2836.c +0
-0@@ -248,6 +248,9 @@ static int __init bcm2836_smp_boot_secondary(unsigned int cpu,writel (secondary_startup_phys, intc.base +LOCAL_MAILBOX3_SET0 +16 * cpu);Quoted 8 lines
drivers/irqchip/irq-bcm2836.c static int __init bcm2836_smp_boot_secondary(unsigned int cpu, · line 255
Quoted 5 lines
drivers/irqchip/irq-bcm2836.c +3
-0+dsb (sy);/* Ensure write has completed before waking the other CPUs */ +sev (); +return 0 ; }This is also the behavior that the standard arm64 spin-table method has, which we unfortunately can't quite use.
Quoted 4 lines
And why is that so? Why do you have to reinvent the wheel (and hide the cloned wheel in an interrupt controller driver)?
That doesn't seem right to me.
The armv8 stubs (firmware-supplied code in the low page that do the spinning) do actually implement arm64's spin-table method. It's the armv7 stubs that use these registers in the irqchip instead of plain addresses in system memory.
Quoted 4 lines
Quoted 2 lines
Let's put ARMv7 aside for the time being. If your firmware already implements spin-tables, why don't you simply use that at least on arm64?
We do.
Obviously not the way it is intended if you have to duplicate the core architectural code in the interrupt controller driver, which couldn't care less.
M.
Signature · 1 line
Jazz is not dead. It just smells funny...
Quoted 54 lines · Marc Zyngier
Quoted 53 lines · Phil Elwell
Quoted 52 lines · Marc Zyngier
Quoted 50 lines · Eric Anholt
Quoted 39 lines · Marc Zyngier
Quoted 29 lines · Eric Anholt
Quoted 27 lines · Phil Elwell
In order to reduce power consumption and bus traffic, it is sensible for secondary cores to enter a low-power idle state when waiting to be started. The wfe instruction causes a core to wait until an event or interrupt arrives before continuing to the next instruction. The sev instruction sends a wakeup event to the other cores, so call it from
bcm2836_smp_boot_secondary, the function that wakes up the waiting cores during booting.It is harmless to use this patch without the corresponding change adding wfe to the ARMv7/ARMv8-32 stubs, but if the stubs are updated and this patch is not applied then the other cores will sleep forever.
See: https://github.com/raspberrypi/linux/issues/1989
Signed-off-by Phil Elwell <[email protected]>--- drivers/irqchip/irq-bcm2836.c | 3 +++ 1 file changed, 3 insertions(+)drivers/irqchip/irq-bcm2836.c +0
-0@@ -248,6 +248,9 @@ static int __init bcm2836_smp_boot_secondary(unsigned int cpu,writel (secondary_startup_phys, intc.base +LOCAL_MAILBOX3_SET0 +16 * cpu);Quoted 8 lines
drivers/irqchip/irq-bcm2836.c static int __init bcm2836_smp_boot_secondary(unsigned int cpu, · line 255
Quoted 5 lines
drivers/irqchip/irq-bcm2836.c +3
-0+dsb (sy);/* Ensure write has completed before waking the other CPUs */ +sev (); +return 0 ; }This is also the behavior that the standard arm64 spin-table method has, which we unfortunately can't quite use.
Quoted 4 lines
And why is that so? Why do you have to reinvent the wheel (and hide the cloned wheel in an interrupt controller driver)?
That doesn't seem right to me.
The armv8 stubs (firmware-supplied code in the low page that do the spinning) do actually implement arm64's spin-table method. It's the armv7 stubs that use these registers in the irqchip instead of plain addresses in system memory.
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
Let's put ARMv7 aside for the time being. If your firmware already implements spin-tables, why don't you simply use that at least on arm64?
We do.
Obviously not the way it is intended if you have to duplicate the core architectural code in the interrupt controller driver, which couldn't care less.
If we were using this method on arm64 then the other cores would not start up because armstub8.S has always included a wfe. Nothing in the commit mentions arm64 - this is an ARCH=arm fix.
Phil
Quoted 60 lines · Phil Elwell
Quoted 59 lines · Marc Zyngier
Quoted 57 lines · Phil Elwell
Quoted 47 lines · Marc Zyngier
Quoted 46 lines · Eric Anholt
Quoted 43 lines · Marc Zyngier
Quoted 29 lines · Eric Anholt
Quoted 27 lines · Phil Elwell
In order to reduce power consumption and bus traffic, it is sensible for secondary cores to enter a low-power idle state when waiting to be started. The wfe instruction causes a core to wait until an event or interrupt arrives before continuing to the next instruction. The sev instruction sends a wakeup event to the other cores, so call it from
bcm2836_smp_boot_secondary, the function that wakes up the waiting cores during booting.It is harmless to use this patch without the corresponding change adding wfe to the ARMv7/ARMv8-32 stubs, but if the stubs are updated and this patch is not applied then the other cores will sleep forever.
See: https://github.com/raspberrypi/linux/issues/1989
Signed-off-by Phil Elwell <[email protected]>--- drivers/irqchip/irq-bcm2836.c | 3 +++ 1 file changed, 3 insertions(+)drivers/irqchip/irq-bcm2836.c +0
-0@@ -248,6 +248,9 @@ static int __init bcm2836_smp_boot_secondary(unsigned int cpu,writel (secondary_startup_phys, intc.base +LOCAL_MAILBOX3_SET0 +16 * cpu);Quoted 8 lines
drivers/irqchip/irq-bcm2836.c static int __init bcm2836_smp_boot_secondary(unsigned int cpu, · line 255
Quoted 5 lines
drivers/irqchip/irq-bcm2836.c +3
-0+dsb (sy);/* Ensure write has completed before waking the other CPUs */ +sev (); +return 0 ; }This is also the behavior that the standard arm64 spin-table method has, which we unfortunately can't quite use.
And why is that so? Why do you have to reinvent the wheel (and hide the cloned wheel in an interrupt controller driver)?
That doesn't seem right to me.
Quoted 7 lines
Quoted 4 lines
The armv8 stubs (firmware-supplied code in the low page that do the spinning) do actually implement arm64's spin-table method. It's the armv7 stubs that use these registers in the irqchip instead of plain addresses in system memory.
Let's put ARMv7 aside for the time being. If your firmware already implements spin-tables, why don't you simply use that at least on arm64?
We do.
Quoted 7 lines
Quoted 3 lines
Obviously not the way it is intended if you have to duplicate the core architectural code in the interrupt controller driver, which couldn't care less.
If we were using this method on arm64 then the other cores would not start up because
armstub8.Shas always included a wfe. Nothing in the commit mentions arm64 - this is an ARCH=arm fix.
Thanks for the clarification, which you could have added to the commit message.
The question still remains: why do we have CPU bring-up code in an interrupt controller, instead of having it in the architecture code?
The RPi-2 is the *only* platform to have its SMP bringup code outside of arch/arm, so the first course of action would be to move that code where it belongs.
Thanks,
M.
Signature · 1 line
Jazz is not dead. It just smells funny...
Quoted 72 lines · Marc Zyngier
Quoted 60 lines · Phil Elwell
Quoted 59 lines · Marc Zyngier
Quoted 57 lines · Phil Elwell
Quoted 47 lines · Marc Zyngier
Quoted 46 lines · Eric Anholt
Quoted 43 lines · Marc Zyngier
Quoted 29 lines · Eric Anholt
Quoted 27 lines · Phil Elwell
In order to reduce power consumption and bus traffic, it is sensible for secondary cores to enter a low-power idle state when waiting to be started. The wfe instruction causes a core to wait until an event or interrupt arrives before continuing to the next instruction. The sev instruction sends a wakeup event to the other cores, so call it from
bcm2836_smp_boot_secondary, the function that wakes up the waiting cores during booting.It is harmless to use this patch without the corresponding change adding wfe to the ARMv7/ARMv8-32 stubs, but if the stubs are updated and this patch is not applied then the other cores will sleep forever.
See: https://github.com/raspberrypi/linux/issues/1989
Signed-off-by Phil Elwell <[email protected]>--- drivers/irqchip/irq-bcm2836.c | 3 +++ 1 file changed, 3 insertions(+)drivers/irqchip/irq-bcm2836.c +0
-0@@ -248,6 +248,9 @@ static int __init bcm2836_smp_boot_secondary(unsigned int cpu,writel (secondary_startup_phys, intc.base +LOCAL_MAILBOX3_SET0 +16 * cpu);Quoted 8 lines
drivers/irqchip/irq-bcm2836.c static int __init bcm2836_smp_boot_secondary(unsigned int cpu, · line 255
Quoted 5 lines
drivers/irqchip/irq-bcm2836.c +3
-0+dsb (sy);/* Ensure write has completed before waking the other CPUs */ +sev (); +return 0 ; }This is also the behavior that the standard arm64 spin-table method has, which we unfortunately can't quite use.
And why is that so? Why do you have to reinvent the wheel (and hide the cloned wheel in an interrupt controller driver)?
That doesn't seem right to me.
Quoted 7 lines
Quoted 4 lines
The armv8 stubs (firmware-supplied code in the low page that do the spinning) do actually implement arm64's spin-table method. It's the armv7 stubs that use these registers in the irqchip instead of plain addresses in system memory.
Let's put ARMv7 aside for the time being. If your firmware already implements spin-tables, why don't you simply use that at least on arm64?
We do.
Quoted 7 lines
Quoted 3 lines
Obviously not the way it is intended if you have to duplicate the core architectural code in the interrupt controller driver, which couldn't care less.
If we were using this method on arm64 then the other cores would not start up because
armstub8.Shas always included a wfe. Nothing in the commit mentions arm64 - this is an ARCH=arm fix.Thanks for the clarification, which you could have added to the commit message.
Quoted 6 lines
The question still remains: why do we have CPU bring-up code in an interrupt controller, instead of having it in the architecture code?
The RPi-2 is the *only* platform to have its SMP bringup code outside of
arch/arm, so the first course of action would be to move that code where it belongs.
You were CC'd on the commit (41f4988cc287e5f836d3f6620c9f900bc9b560e9) that introduced bcm2836_smp_boot_secondary - it seems strange to start objecting now. Yes, I think it is odd that it didn't go into arch/arm/mach-bcm, but in the interests of making changes in small, independent steps, do you have a problem with this commit?
Phil
Quoted 77 lines · Phil Elwell
Quoted 76 lines · Marc Zyngier
Quoted 61 lines · Phil Elwell
Quoted 60 lines · Marc Zyngier
Quoted 58 lines · Phil Elwell
Quoted 48 lines · Marc Zyngier
Quoted 47 lines · Eric Anholt
Quoted 44 lines · Marc Zyngier
Quoted 29 lines · Eric Anholt
Quoted 27 lines · Phil Elwell
In order to reduce power consumption and bus traffic, it is sensible for secondary cores to enter a low-power idle state when waiting to be started. The wfe instruction causes a core to wait until an event or interrupt arrives before continuing to the next instruction. The sev instruction sends a wakeup event to the other cores, so call it from
bcm2836_smp_boot_secondary, the function that wakes up the waiting cores during booting.It is harmless to use this patch without the corresponding change adding wfe to the ARMv7/ARMv8-32 stubs, but if the stubs are updated and this patch is not applied then the other cores will sleep forever.
See: https://github.com/raspberrypi/linux/issues/1989
Signed-off-by Phil Elwell <[email protected]>--- drivers/irqchip/irq-bcm2836.c | 3 +++ 1 file changed, 3 insertions(+)drivers/irqchip/irq-bcm2836.c +0
-0@@ -248,6 +248,9 @@ static int __init bcm2836_smp_boot_secondary(unsigned int cpu,writel (secondary_startup_phys, intc.base +LOCAL_MAILBOX3_SET0 +16 * cpu);Quoted 9 lines
drivers/irqchip/irq-bcm2836.c static int __init bcm2836_smp_boot_secondary(unsigned int cpu, · line 255
Quoted 5 lines
drivers/irqchip/irq-bcm2836.c +3
-0+dsb (sy);/* Ensure write has completed before waking the other CPUs */ +sev (); +return 0 ; }This is also the behavior that the standard arm64 spin-table method has,
which we unfortunately can't quite use.And why is that so? Why do you have to reinvent the wheel (and hide the cloned wheel in an interrupt controller driver)?
That doesn't seem right to me.
Quoted 7 lines
Quoted 4 lines
The armv8 stubs (firmware-supplied code in the low page that do the spinning) do actually implement arm64's spin-table method. It's the armv7 stubs that use these registers in the irqchip instead of plain addresses in system memory.
Let's put ARMv7 aside for the time being. If your firmware already implements spin-tables, why don't you simply use that at least on arm64?
We do.
Quoted 7 lines
Quoted 3 lines
Obviously not the way it is intended if you have to duplicate the core architectural code in the interrupt controller driver, which couldn't care less.
If we were using this method on arm64 then the other cores would not start up because
armstub8.Shas always included a wfe. Nothing in the commit mentions arm64 - this is an ARCH=arm fix.Thanks for the clarification, which you could have added to the commit message.
The question still remains: why do we have CPU bring-up code in an interrupt controller, instead of having it in the architecture code?
Quoted 7 lines
Quoted 3 lines
The RPi-2 is the *only* platform to have its SMP bringup code outside of
arch/arm, so the first course of action would be to move that code where it belongs.You were CC'd on the commit (
41f4988cc287e5f836d3f6620c9f900bc9b560e9) that introducedbcm2836_smp_boot_secondary- it seems strange to start objecting now.
Well, I'm far from being perfect. If I had noticed it, I'd have NACKed it.
Quoted 3 lines
Yes, I think it is odd that it didn't go into
arch/arm/mach-bcm, but in the interests of making changes in small, independent steps, do you have a problem with this commit?
On its own, no. I'm just not keen on adding more unrelated stuff to this file, so let's start with dealing with the original bug, and you can then add this fix on top.
Thanks,
M.
Signature · 1 line
Jazz is not dead, it just smell funny.
Quoted 89 lines · Marc Zyngier
Quoted 77 lines · Phil Elwell
Quoted 76 lines · Marc Zyngier
Quoted 61 lines · Phil Elwell
Quoted 60 lines · Marc Zyngier
Quoted 58 lines · Phil Elwell
Quoted 48 lines · Marc Zyngier
Quoted 47 lines · Eric Anholt
Quoted 44 lines · Marc Zyngier
Quoted 29 lines · Eric Anholt
Quoted 27 lines · Phil Elwell
In order to reduce power consumption and bus traffic, it is sensible for secondary cores to enter a low-power idle state when waiting to be started. The wfe instruction causes a core to wait until an event or interrupt arrives before continuing to the next instruction. The sev instruction sends a wakeup event to the other cores, so call it from
bcm2836_smp_boot_secondary, the function that wakes up the waiting cores during booting.It is harmless to use this patch without the corresponding change adding wfe to the ARMv7/ARMv8-32 stubs, but if the stubs are updated and this patch is not applied then the other cores will sleep forever.
See: https://github.com/raspberrypi/linux/issues/1989
Signed-off-by Phil Elwell <[email protected]>--- drivers/irqchip/irq-bcm2836.c | 3 +++ 1 file changed, 3 insertions(+)drivers/irqchip/irq-bcm2836.c +0
-0@@ -248,6 +248,9 @@ static int __init bcm2836_smp_boot_secondary(unsigned int cpu,writel (secondary_startup_phys, intc.base +LOCAL_MAILBOX3_SET0 +16 * cpu);Quoted 9 lines
drivers/irqchip/irq-bcm2836.c static int __init bcm2836_smp_boot_secondary(unsigned int cpu, · line 255
Quoted 5 lines
drivers/irqchip/irq-bcm2836.c +3
-0+dsb (sy);/* Ensure write has completed before waking the other CPUs */ +sev (); +return 0 ; }This is also the behavior that the standard arm64 spin-table method has,
which we unfortunately can't quite use.And why is that so? Why do you have to reinvent the wheel (and hide the cloned wheel in an interrupt controller driver)?
That doesn't seem right to me.
Quoted 7 lines
Quoted 4 lines
The armv8 stubs (firmware-supplied code in the low page that do the spinning) do actually implement arm64's spin-table method. It's the armv7 stubs that use these registers in the irqchip instead of plain addresses in system memory.
Let's put ARMv7 aside for the time being. If your firmware already implements spin-tables, why don't you simply use that at least on arm64?
We do.
Quoted 7 lines
Quoted 3 lines
Obviously not the way it is intended if you have to duplicate the core architectural code in the interrupt controller driver, which couldn't care less.
If we were using this method on arm64 then the other cores would not start up because
armstub8.Shas always included a wfe. Nothing in the commit mentions arm64 - this is an ARCH=arm fix.Thanks for the clarification, which you could have added to the commit message.
The question still remains: why do we have CPU bring-up code in an interrupt controller, instead of having it in the architecture code?
Quoted 7 lines
Quoted 3 lines
The RPi-2 is the *only* platform to have its SMP bringup code outside of
arch/arm, so the first course of action would be to move that code where it belongs.You were CC'd on the commit (
41f4988cc287e5f836d3f6620c9f900bc9b560e9) that introducedbcm2836_smp_boot_secondary- it seems strange to start objecting now.Well, I'm far from being perfect. If I had noticed it, I'd have NACKed it.
Quoted 7 lines
Quoted 3 lines
Yes, I think it is odd that it didn't go into
arch/arm/mach-bcm, but in the interests of making changes in small, independent steps, do you have a problem with this commit?On its own, no. I'm just not keen on adding more unrelated stuff to this file, so let's start with dealing with the original bug, and you can then add this fix on top.
That's an interesting use of the word "bug". From Wikipedia:
"A software bug is an error, flaw, failure or fault in a computer program or system that causes it to produce an incorrect or unexpected result, or to behave in unintended ways."
Although your concerns are valid, the faults you are objecting to are not causing a malfunction of any kind. If we were to update the RPi firmware before this patch was merged then upstream users would be left with one wheel on their wagon.
Phil
Quoted 98 lines · Phil Elwell
Quoted 89 lines · Marc Zyngier
Quoted 77 lines · Phil Elwell
Quoted 76 lines · Marc Zyngier
Quoted 61 lines · Phil Elwell
Quoted 60 lines · Marc Zyngier
Quoted 58 lines · Phil Elwell
Quoted 48 lines · Marc Zyngier
Quoted 47 lines · Eric Anholt
Quoted 44 lines · Marc Zyngier
Quoted 29 lines · Eric Anholt
Quoted 27 lines · Phil Elwell
In order to reduce power consumption and bus traffic, it is sensible for secondary cores to enter a low-power idle state when waiting to be started. The wfe instruction causes a core to wait until an event or interrupt arrives before continuing to the next instruction. The sev instruction sends a wakeup event to the other cores, so call it from
bcm2836_smp_boot_secondary, the function that wakes up the waiting cores during booting.It is harmless to use this patch without the corresponding change adding wfe to the ARMv7/ARMv8-32 stubs, but if the stubs are updated and this patch is not applied then the other cores will sleep forever.
See: https://github.com/raspberrypi/linux/issues/1989
Signed-off-by Phil Elwell <[email protected]>--- drivers/irqchip/irq-bcm2836.c | 3 +++ 1 file changed, 3 insertions(+)drivers/irqchip/irq-bcm2836.c +0
-0@@ -248,6 +248,9 @@ static int __init bcm2836_smp_boot_secondary(unsigned int cpu,writel (secondary_startup_phys, intc.base +LOCAL_MAILBOX3_SET0 +16 * cpu);Quoted 9 lines
drivers/irqchip/irq-bcm2836.c static int __init bcm2836_smp_boot_secondary(unsigned int cpu, · line 255
Quoted 5 lines
drivers/irqchip/irq-bcm2836.c +3
-0+dsb (sy);/* Ensure write has completed before waking the other CPUs */ +sev (); +return 0 ; }This is also the behavior that the standard arm64 spin-table method has,
which we unfortunately can't quite use.And why is that so? Why do you have to reinvent the wheel (and hide the cloned wheel in an interrupt controller driver)?
That doesn't seem right to me.
Quoted 7 lines
Quoted 4 lines
The armv8 stubs (firmware-supplied code in the low page that do the spinning) do actually implement arm64's spin-table method. It's the armv7 stubs that use these registers in the irqchip instead of plain addresses in system memory.
Let's put ARMv7 aside for the time being. If your firmware already implements spin-tables, why don't you simply use that at least on arm64?
We do.
Quoted 7 lines
Quoted 3 lines
Obviously not the way it is intended if you have to duplicate the core architectural code in the interrupt controller driver, which couldn't care less.
If we were using this method on arm64 then the other cores would not start up because
armstub8.Shas always included a wfe. Nothing in the commit mentions arm64 - this is an ARCH=arm fix.Thanks for the clarification, which you could have added to the commit message.
The question still remains: why do we have CPU bring-up code in an interrupt controller, instead of having it in the architecture code?
Quoted 7 lines
Quoted 3 lines
The RPi-2 is the *only* platform to have its SMP bringup code outside of
arch/arm, so the first course of action would be to move that code where it belongs.You were CC'd on the commit (
41f4988cc287e5f836d3f6620c9f900bc9b560e9) that introducedbcm2836_smp_boot_secondary- it seems strange to start objecting now.Well, I'm far from being perfect. If I had noticed it, I'd have NACKed it.
Quoted 7 lines
Quoted 3 lines
Yes, I think it is odd that it didn't go into
arch/arm/mach-bcm, but in the interests of making changes in small, independent steps, do you have a problem with this commit?On its own, no. I'm just not keen on adding more unrelated stuff to this file, so let's start with dealing with the original bug, and you can then add this fix on top.
Quoted 5 lines
That's an interesting use of the word "bug". From Wikipedia:
"A software bug is an error, flaw, failure or fault in a computer program or system that causes it to produce an incorrect or unexpected result, or to behave in unintended ways."
Whatever. Should I call it "pile of crap dumped in unsuitable locations" instead? What does Wikipedia says about it?
Quoted 3 lines
Although your concerns are valid, the faults you are objecting to are not causing a malfunction of any kind. If we were to update the RPi firmware before this patch was merged then upstream users would be left with one wheel on their wagon.
And that'd be your problem, not mine. Look, you can argue around this all day, or you can fix this mess. Your choice.
M.
Signature · 1 line
Jazz is not dead. It just smells funny...
Quoted 108 lines · Marc Zyngier
Quoted 98 lines · Phil Elwell
Quoted 89 lines · Marc Zyngier
Quoted 77 lines · Phil Elwell
Quoted 76 lines · Marc Zyngier
Quoted 61 lines · Phil Elwell
Quoted 60 lines · Marc Zyngier
Quoted 58 lines · Phil Elwell
Quoted 48 lines · Marc Zyngier
Quoted 47 lines · Eric Anholt
Quoted 44 lines · Marc Zyngier
Quoted 29 lines · Eric Anholt
Quoted 27 lines · Phil Elwell
In order to reduce power consumption and bus traffic, it is sensible
for secondary cores to enter a low-power idle state when waiting to
be started. The wfe instruction causes a core to wait until an event
or interrupt arrives before continuing to the next instruction.
The sev instruction sends a wakeup event to the other cores, so call
it from bcm2836_smp_boot_secondary, the function that wakes up the
waiting cores during booting.It is harmless to use this patch without the corresponding change
adding wfe to the ARMv7/ARMv8-32 stubs, but if the stubs are updated
and this patch is not applied then the other cores will sleep forever.See: https://github.com/raspberrypi/linux/issues/1989
Signed-off-by: Phil Elwell <[email protected]>
---
drivers/irqchip/irq-bcm2836.c | 3 +++
1 file changed, 3 insertions(+)diff --git a/drivers/irqchip/irq-bcm2836.c b/drivers/irqchip/irq-bcm2836.c
index e10597c..6dccdf9 100644
--- a/drivers/irqchip/irq-bcm2836.c
+++ b/drivers/irqchip/irq-bcm2836.c
@@ -248,6 +248,9 @@ static int __init bcm2836_smp_boot_secondary(unsigned int cpu,
writel(secondary_startup_phys,
intc.base + LOCAL_MAILBOX3_SET0 + 16 * cpu);Quoted 9 lines
Quoted 5 lines
+ dsb(sy); /* Ensure write has completed before waking the other CPUs */
+ sev();
+
return 0;
}This is also the behavior that the standard arm64 spin-table method has,
which we unfortunately can't quite use.And why is that so? Why do you have to reinvent the wheel (and hide the cloned wheel in an interrupt controller driver)?
That doesn't seem right to me.
Quoted 7 lines
Quoted 4 lines
The armv8 stubs (firmware-supplied code in the low page that do the spinning) do actually implement arm64's spin-table method. It's the armv7 stubs that use these registers in the irqchip instead of plain addresses in system memory.
Let's put ARMv7 aside for the time being. If your firmware already implements spin-tables, why don't you simply use that at least on arm64?
We do.
Quoted 7 lines
Quoted 3 lines
Obviously not the way it is intended if you have to duplicate the core architectural code in the interrupt controller driver, which couldn't care less.
If we were using this method on arm64 then the other cores would not start up because
armstub8.Shas always included a wfe. Nothing in the commit mentions arm64 - this is an ARCH=arm fix.Thanks for the clarification, which you could have added to the commit message.
The question still remains: why do we have CPU bring-up code in an interrupt controller, instead of having it in the architecture code?
Quoted 7 lines
Quoted 3 lines
The RPi-2 is the *only* platform to have its SMP bringup code outside of
arch/arm, so the first course of action would be to move that code where it belongs.You were CC'd on the commit (
41f4988cc287e5f836d3f6620c9f900bc9b560e9) that introducedbcm2836_smp_boot_secondary- it seems strange to start objecting now.Well, I'm far from being perfect. If I had noticed it, I'd have NACKed it.
Quoted 7 lines
Quoted 3 lines
Yes, I think it is odd that it didn't go into
arch/arm/mach-bcm, but in the interests of making changes in small, independent steps, do you have a problem with this commit?On its own, no. I'm just not keen on adding more unrelated stuff to this file, so let's start with dealing with the original bug, and you can then add this fix on top.
Quoted 5 lines
That's an interesting use of the word "bug". From Wikipedia:
"A software bug is an error, flaw, failure or fault in a computer program or system that causes it to produce an incorrect or unexpected result, or to behave in unintended ways."
Whatever. Should I call it "pile of crap dumped in unsuitable locations" instead? What does Wikipedia says about it?
Quoted 6 lines
Quoted 3 lines
Although your concerns are valid, the faults you are objecting to are not causing a malfunction of any kind. If we were to update the RPi firmware before this patch was merged then upstream users would be left with one wheel on their wagon.
And that'd be your problem, not mine. Look, you can argue around this all day, or you can fix this mess. Your choice.
Is that the opinion of all here?
Phil
Quoted 109 lines · Phil Elwell
Quoted 108 lines · Marc Zyngier
Quoted 98 lines · Phil Elwell
Quoted 89 lines · Marc Zyngier
Quoted 77 lines · Phil Elwell
Quoted 76 lines · Marc Zyngier
Quoted 61 lines · Phil Elwell
Quoted 60 lines · Marc Zyngier
Quoted 58 lines · Phil Elwell
Quoted 48 lines · Marc Zyngier
Quoted 47 lines · Eric Anholt
Quoted 44 lines · Marc Zyngier
Quoted 38 lines · Eric Anholt
Phil Elwell <[email protected]> writes:
In order to reduce power consumption and bus traffic, it is sensible
for secondary cores to enter a low-power idle state when waiting to
be started. The wfe instruction causes a core to wait until an event
or interrupt arrives before continuing to the next instruction.
The sev instruction sends a wakeup event to the other cores, so call
it from bcm2836_smp_boot_secondary, the function that wakes up the
waiting cores during booting.It is harmless to use this patch without the corresponding change
adding wfe to the ARMv7/ARMv8-32 stubs, but if the stubs are updated
and this patch is not applied then the other cores will sleep forever.See: https://github.com/raspberrypi/linux/issues/1989
Signed-off-by: Phil Elwell <[email protected]>
---
drivers/irqchip/irq-bcm2836.c | 3 +++
1 file changed, 3 insertions(+)diff --git a/drivers/irqchip/irq-bcm2836.c b/drivers/irqchip/irq-bcm2836.c
index e10597c..6dccdf9 100644
--- a/drivers/irqchip/irq-bcm2836.c
+++ b/drivers/irqchip/irq-bcm2836.c
@@ -248,6 +248,9 @@ static int __init bcm2836_smp_boot_secondary(unsigned int cpu,
writel(secondary_startup_phys,
intc.base + LOCAL_MAILBOX3_SET0 + 16 * cpu);+ dsb(sy); /* Ensure write has completed before waking the other CPUs */
+ sev();
+
return 0;
}This is also the behavior that the standard arm64 spin-table
method has,
which we unfortunately can't quite use.And why is that so? Why do you have to reinvent the wheel (and hide the cloned wheel in an interrupt controller driver)?
That doesn't seem right to me.
Quoted 7 lines
Quoted 4 lines
The armv8 stubs (firmware-supplied code in the low page that do the spinning) do actually implement arm64's spin-table method. It's the armv7 stubs that use these registers in the irqchip instead of plain addresses in system memory.
Let's put ARMv7 aside for the time being. If your firmware already implements spin-tables, why don't you simply use that at least on arm64?
We do.
Quoted 7 lines
Quoted 3 lines
Obviously not the way it is intended if you have to duplicate the core architectural code in the interrupt controller driver, which couldn't care less.
If we were using this method on arm64 then the other cores would not start up because
armstub8.Shas always included a wfe. Nothing in the commit mentions arm64 - this is an ARCH=arm fix.Thanks for the clarification, which you could have added to the commit message.
The question still remains: why do we have CPU bring-up code in an interrupt controller, instead of having it in the architecture code?
Quoted 7 lines
Quoted 3 lines
The RPi-2 is the *only* platform to have its SMP bringup code outside of
arch/arm, so the first course of action would be to move that code where it belongs.You were CC'd on the commit (
41f4988cc287e5f836d3f6620c9f900bc9b560e9) that introducedbcm2836_smp_boot_secondary- it seems strange to start objecting now.Well, I'm far from being perfect. If I had noticed it, I'd have NACKed it.
Quoted 7 lines
Quoted 3 lines
Yes, I think it is odd that it didn't go into
arch/arm/mach-bcm, but in the interests of making changes in small, independent steps, do you have a problem with this commit?On its own, no. I'm just not keen on adding more unrelated stuff to this file, so let's start with dealing with the original bug, and you can then add this fix on top.
Quoted 5 lines
That's an interesting use of the word "bug". From Wikipedia:
"A software bug is an error, flaw, failure or fault in a computer program or system that causes it to produce an incorrect or unexpected result, or to behave in unintended ways."
Whatever. Should I call it "pile of crap dumped in unsuitable locations" instead? What does Wikipedia says about it?
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Although your concerns are valid, the faults you are objecting to are not causing a malfunction of any kind. If we were to update the RPi firmware before this patch was merged then upstream users would be left with one wheel on their wagon.
And that'd be your problem, not mine. Look, you can argue around this all day, or you can fix this mess. Your choice.
Is that the opinion of all here?
The choice of word here got largely out of the original topic and I surely did eat a ton of popcorn here. There are two things that need fixing, and the time line and process for fixing these is clear:
- your bugfix (Phil) is something that should be applied now, and backported to -stable trees once the fix hits the irqchip tree (or Linus')
- relocating the code that does the secondary boot out of drivers/irqchip/ into arch/arm/mach-bcm/ needs to happen (Marc), and this is 4.13 material, there is no urgency in doing this *right now*, but it needs to happen
Does that work for everyone?
Signature · 1 line
Florian
Quoted 114 lines · Florian Fainelli
Quoted 113 lines · Phil Elwell
Quoted 112 lines · Marc Zyngier
Quoted 98 lines · Phil Elwell
Quoted 89 lines · Marc Zyngier
Quoted 77 lines · Phil Elwell
Quoted 76 lines · Marc Zyngier
Quoted 61 lines · Phil Elwell
Quoted 60 lines · Marc Zyngier
Quoted 58 lines · Phil Elwell
Quoted 48 lines · Marc Zyngier
Quoted 47 lines · Eric Anholt
Quoted 44 lines · Marc Zyngier
On 09/05/17 17:59, Eric Anholt wrote:
Phil Elwell <[email protected]> writes:In order to reduce power consumption and bus traffic, it is sensible
for secondary cores to enter a low-power idle state when waiting to
be started. The wfe instruction causes a core to wait until an event
or interrupt arrives before continuing to the next instruction.
The sev instruction sends a wakeup event to the other cores, so call
it from bcm2836_smp_boot_secondary, the function that wakes up the
waiting cores during booting.It is harmless to use this patch without the corresponding change
adding wfe to the ARMv7/ARMv8-32 stubs, but if the stubs are updated
and this patch is not applied then the other cores will sleep forever.See: https://github.com/raspberrypi/linux/issues/1989
Signed-off-by: Phil Elwell <[email protected]>
---
drivers/irqchip/irq-bcm2836.c | 3 +++
1 file changed, 3 insertions(+)diff --git a/drivers/irqchip/irq-bcm2836.c b/drivers/irqchip/irq-bcm2836.c
index e10597c..6dccdf9 100644
--- a/drivers/irqchip/irq-bcm2836.c
+++ b/drivers/irqchip/irq-bcm2836.c
@@ -248,6 +248,9 @@ static int __init bcm2836_smp_boot_secondary(unsigned int cpu,
writel(secondary_startup_phys,
intc.base + LOCAL_MAILBOX3_SET0 + 16 * cpu);+ dsb(sy); /* Ensure write has completed before waking the other CPUs */
+ sev();
+
return 0;
}This is also the behavior that the standard arm64 spin-table
method has,
which we unfortunately can't quite use.And why is that so? Why do you have to reinvent the wheel (and hide the
cloned wheel in an interrupt controller driver)?That doesn't seem right to me.
Quoted 7 lines
Quoted 4 lines
The armv8 stubs (firmware-supplied code in the low page that do the spinning) do actually implement arm64's spin-table method. It's the armv7 stubs that use these registers in the irqchip instead of plain addresses in system memory.
Let's put ARMv7 aside for the time being. If your firmware already implements spin-tables, why don't you simply use that at least on arm64?
We do.
Quoted 7 lines
Quoted 3 lines
Obviously not the way it is intended if you have to duplicate the core architectural code in the interrupt controller driver, which couldn't care less.
If we were using this method on arm64 then the other cores would not start up because
armstub8.Shas always included a wfe. Nothing in the commit mentions arm64 - this is an ARCH=arm fix.Thanks for the clarification, which you could have added to the commit message.
The question still remains: why do we have CPU bring-up code in an interrupt controller, instead of having it in the architecture code?
Quoted 7 lines
Quoted 3 lines
The RPi-2 is the *only* platform to have its SMP bringup code outside of
arch/arm, so the first course of action would be to move that code where it belongs.You were CC'd on the commit (
41f4988cc287e5f836d3f6620c9f900bc9b560e9) that introducedbcm2836_smp_boot_secondary- it seems strange to start objecting now.Well, I'm far from being perfect. If I had noticed it, I'd have NACKed it.
Quoted 7 lines
Quoted 3 lines
Yes, I think it is odd that it didn't go into
arch/arm/mach-bcm, but in the interests of making changes in small, independent steps, do you have a problem with this commit?On its own, no. I'm just not keen on adding more unrelated stuff to this file, so let's start with dealing with the original bug, and you can then add this fix on top.
Quoted 5 lines
That's an interesting use of the word "bug". From Wikipedia:
"A software bug is an error, flaw, failure or fault in a computer program or system that causes it to produce an incorrect or unexpected result, or to behave in unintended ways."
Whatever. Should I call it "pile of crap dumped in unsuitable locations" instead? What does Wikipedia says about it?
Quoted 3 lines
Although your concerns are valid, the faults you are objecting to are not causing a malfunction of any kind. If we were to update the RPi firmware before this patch was merged then upstream users would be left with one wheel on their wagon.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
And that'd be your problem, not mine. Look, you can argue around this all day, or you can fix this mess. Your choice.
Is that the opinion of all here?
The choice of word here got largely out of the original topic and I surely did eat a ton of popcorn here. There are two things that need
Always happy to entertain.
Quoted 4 lines
fixing, and the time line and process for fixing these is clear:
- your bugfix (Phil) is something that should be applied now, and backported to -stable trees once the fix hits the irqchip tree (or Linus')
Why? As far as I can tell, none of the current platforms are affected, since the corresponding firmware hasn't been distributed. So by the letter of this, this patch is 4.13 material too.
Quoted 6 lines
- relocating the code that does the secondary boot out of
drivers/irqchip/intoarch/arm/mach-bcm/needs to happen (Marc), and this is 4.13 material, there is no urgency in doing this *right now*, but it needs to happenDoes that work for everyone?
I'm strongly opposed to it. Moving code over to arch/arm/ has zero risk, and removes crap from the existing code. Hell, make the whole thing one patch if that makes it easier for you to backport. But piling more crap there? No, thank you.
M.
Signature · 1 line
Jazz is not dead. It just smells funny...
Quoted 125 lines · Marc Zyngier
Quoted 114 lines · Florian Fainelli
Quoted 113 lines · Phil Elwell
Quoted 112 lines · Marc Zyngier
Quoted 98 lines · Phil Elwell
Quoted 89 lines · Marc Zyngier
Quoted 77 lines · Phil Elwell
Quoted 76 lines · Marc Zyngier
Quoted 61 lines · Phil Elwell
Quoted 60 lines · Marc Zyngier
Quoted 58 lines · Phil Elwell
Quoted 48 lines · Marc Zyngier
Quoted 47 lines · Eric Anholt
Marc Zyngier <[email protected]> writes:
On 09/05/17 17:59, Eric Anholt wrote:
Phil Elwell <[email protected]> writes:In order to reduce power consumption and bus traffic, it is sensible
for secondary cores to enter a low-power idle state when waiting to
be started. The wfe instruction causes a core to wait until an event
or interrupt arrives before continuing to the next instruction.
The sev instruction sends a wakeup event to the other cores, so call
it from bcm2836_smp_boot_secondary, the function that wakes up the
waiting cores during booting.It is harmless to use this patch without the corresponding change
adding wfe to the ARMv7/ARMv8-32 stubs, but if the stubs are updated
and this patch is not applied then the other cores will sleep forever.See: https://github.com/raspberrypi/linux/issues/1989
Signed-off-by: Phil Elwell <[email protected]>
---
drivers/irqchip/irq-bcm2836.c | 3 +++
1 file changed, 3 insertions(+)diff --git a/drivers/irqchip/irq-bcm2836.c b/drivers/irqchip/irq-bcm2836.c
index e10597c..6dccdf9 100644
--- a/drivers/irqchip/irq-bcm2836.c
+++ b/drivers/irqchip/irq-bcm2836.c
@@ -248,6 +248,9 @@ static int __init bcm2836_smp_boot_secondary(unsigned int cpu,
writel(secondary_startup_phys,
intc.base + LOCAL_MAILBOX3_SET0 + 16 * cpu);+ dsb(sy); /* Ensure write has completed before waking the other CPUs */
+ sev();
+
return 0;
}This is also the behavior that the standard arm64 spin-table
method has,
which we unfortunately can't quite use.And why is that so? Why do you have to reinvent the wheel (and hide the
cloned wheel in an interrupt controller driver)?That doesn't seem right to me.
Quoted 7 lines
Quoted 4 lines
The armv8 stubs (firmware-supplied code in the low page that do the
spinning) do actually implement arm64's spin-table method. It's the
armv7 stubs that use these registers in the irqchip instead of plain
addresses in system memory.Let's put ARMv7 aside for the time being. If your firmware already implements spin-tables, why don't you simply use that at least on arm64?
We do.
Quoted 7 lines
Quoted 3 lines
Obviously not the way it is intended if you have to duplicate the core architectural code in the interrupt controller driver, which couldn't care less.
If we were using this method on arm64 then the other cores would not start up because
armstub8.Shas always included a wfe. Nothing in the commit mentions arm64 - this is an ARCH=arm fix.Thanks for the clarification, which you could have added to the commit message.
The question still remains: why do we have CPU bring-up code in an interrupt controller, instead of having it in the architecture code?
Quoted 7 lines
Quoted 3 lines
The RPi-2 is the *only* platform to have its SMP bringup code outside of
arch/arm, so the first course of action would be to move that code where it belongs.You were CC'd on the commit (
41f4988cc287e5f836d3f6620c9f900bc9b560e9) that introducedbcm2836_smp_boot_secondary- it seems strange to start objecting now.Well, I'm far from being perfect. If I had noticed it, I'd have NACKed it.
Quoted 7 lines
Quoted 3 lines
Yes, I think it is odd that it didn't go into
arch/arm/mach-bcm, but in the interests of making changes in small, independent steps, do you have a problem with this commit?On its own, no. I'm just not keen on adding more unrelated stuff to this file, so let's start with dealing with the original bug, and you can then add this fix on top.
Quoted 5 lines
That's an interesting use of the word "bug". From Wikipedia:
"A software bug is an error, flaw, failure or fault in a computer program or system that causes it to produce an incorrect or unexpected result, or to behave in unintended ways."
Whatever. Should I call it "pile of crap dumped in unsuitable locations" instead? What does Wikipedia says about it?
Quoted 3 lines
Although your concerns are valid, the faults you are objecting to are not causing a malfunction of any kind. If we were to update the RPi firmware before this patch was merged then upstream users would be left with one wheel on their wagon.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
And that'd be your problem, not mine. Look, you can argue around this all day, or you can fix this mess. Your choice.
Is that the opinion of all here?
The choice of word here got largely out of the original topic and I surely did eat a ton of popcorn here. There are two things that need
Always happy to entertain.
Quoted 8 lines
Quoted 4 lines
fixing, and the time line and process for fixing these is clear:
- your bugfix (Phil) is something that should be applied now, and backported to -stable trees once the fix hits the irqchip tree (or Linus')
Why? As far as I can tell, none of the current platforms are affected, since the corresponding firmware hasn't been distributed. So by the letter of this, this patch is 4.13 material too.
I presume that people updating their RPI firmware and using an older kernel can run into the scenario that Phil has described (secondary cores spinning forever), that IMHO is worth fixing on its own.
Quoted 6 lines
Quoted 6 lines
- relocating the code that does the secondary boot out of
drivers/irqchip/intoarch/arm/mach-bcm/needs to happen (Marc), and this is 4.13 material, there is no urgency in doing this *right now*, but it needs to happen
Quoted 6 lines
Quoted 1 line
Does that work for everyone?
I'm strongly opposed to it. Moving code over to
arch/arm/has zero risk, and removes crap from the existing code. Hell, make the whole thing one patch if that makes it easier for you to backport. But piling more crap there? No, thank you.
It sounds unlikely that we can get the ARM SOC maintainers to accept a change that late for 4.12 anyway, since it's not fixing anything but your aversion for where the code currently is, which most people here probably share as well (myself included), just with a different priority.
Anyway, I am just the guy taking Eric's pull requests on their way to the ARM SOC maintainers, so I will let him decide what to do.
Signature · 1 line
Florian
Quoted 126 lines · Florian Fainelli
Quoted 109 lines · Phil Elwell
Quoted 108 lines · Marc Zyngier
Quoted 98 lines · Phil Elwell
Quoted 89 lines · Marc Zyngier
Quoted 77 lines · Phil Elwell
Quoted 76 lines · Marc Zyngier
Quoted 61 lines · Phil Elwell
Quoted 60 lines · Marc Zyngier
Quoted 58 lines · Phil Elwell
Quoted 48 lines · Marc Zyngier
Quoted 47 lines · Eric Anholt
Quoted 44 lines · Marc Zyngier
On 09/05/17 17:59, Eric Anholt wrote:
Phil Elwell <[email protected]> writes:In order to reduce power consumption and bus traffic, it is sensible
for secondary cores to enter a low-power idle state when waiting to
be started. The wfe instruction causes a core to wait until an event
or interrupt arrives before continuing to the next instruction.
The sev instruction sends a wakeup event to the other cores, so call
it from bcm2836_smp_boot_secondary, the function that wakes up the
waiting cores during booting.It is harmless to use this patch without the corresponding change
adding wfe to the ARMv7/ARMv8-32 stubs, but if the stubs are updated
and this patch is not applied then the other cores will sleep forever.See: https://github.com/raspberrypi/linux/issues/1989
Signed-off-by: Phil Elwell <[email protected]>
---
drivers/irqchip/irq-bcm2836.c | 3 +++
1 file changed, 3 insertions(+)diff --git a/drivers/irqchip/irq-bcm2836.c b/drivers/irqchip/irq-bcm2836.c
index e10597c..6dccdf9 100644
--- a/drivers/irqchip/irq-bcm2836.c
+++ b/drivers/irqchip/irq-bcm2836.c
@@ -248,6 +248,9 @@ static int __init bcm2836_smp_boot_secondary(unsigned int cpu,
writel(secondary_startup_phys,
intc.base + LOCAL_MAILBOX3_SET0 + 16 * cpu);+ dsb(sy); /* Ensure write has completed before waking the other CPUs */
+ sev();
+
return 0;
}This is also the behavior that the standard arm64 spin-table
method has,
which we unfortunately can't quite use.And why is that so? Why do you have to reinvent the wheel (and hide the
cloned wheel in an interrupt controller driver)?That doesn't seem right to me.
Quoted 7 lines
Quoted 4 lines
The armv8 stubs (firmware-supplied code in the low page that do the spinning) do actually implement arm64's spin-table method. It's the armv7 stubs that use these registers in the irqchip instead of plain addresses in system memory.
Let's put ARMv7 aside for the time being. If your firmware already implements spin-tables, why don't you simply use that at least on arm64?
We do.
Quoted 7 lines
Quoted 3 lines
Obviously not the way it is intended if you have to duplicate the core architectural code in the interrupt controller driver, which couldn't care less.
If we were using this method on arm64 then the other cores would not start up because
armstub8.Shas always included a wfe. Nothing in the commit mentions arm64 - this is an ARCH=arm fix.Thanks for the clarification, which you could have added to the commit message.
The question still remains: why do we have CPU bring-up code in an interrupt controller, instead of having it in the architecture code?
Quoted 7 lines
Quoted 3 lines
The RPi-2 is the *only* platform to have its SMP bringup code outside of
arch/arm, so the first course of action would be to move that code where it belongs.You were CC'd on the commit (
41f4988cc287e5f836d3f6620c9f900bc9b560e9) that introducedbcm2836_smp_boot_secondary- it seems strange to start objecting now.Well, I'm far from being perfect. If I had noticed it, I'd have NACKed it.
Quoted 7 lines
Quoted 3 lines
Yes, I think it is odd that it didn't go into
arch/arm/mach-bcm, but in the interests of making changes in small, independent steps, do you have a problem with this commit?On its own, no. I'm just not keen on adding more unrelated stuff to this file, so let's start with dealing with the original bug, and you can then add this fix on top.
Quoted 5 lines
That's an interesting use of the word "bug". From Wikipedia:
"A software bug is an error, flaw, failure or fault in a computer program or system that causes it to produce an incorrect or unexpected result, or to behave in unintended ways."
Whatever. Should I call it "pile of crap dumped in unsuitable locations" instead? What does Wikipedia says about it?
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Although your concerns are valid, the faults you are objecting to are not causing a malfunction of any kind. If we were to update the RPi firmware before this patch was merged then upstream users would be left with one wheel on their wagon.
And that'd be your problem, not mine. Look, you can argue around this all day, or you can fix this mess. Your choice.
Is that the opinion of all here?
The choice of word here got largely out of the original topic and I surely did eat a ton of popcorn here. There are two things that need fixing, and the time line and process for fixing these is clear:
- your bugfix (Phil) is something that should be applied now, and backported to -stable trees once the fix hits the irqchip tree (or Linus')
Quoted 6 lines
- relocating the code that does the secondary boot out of
drivers/irqchip/intoarch/arm/mach-bcm/needs to happen (Marc), and this is 4.13 material, there is no urgency in doing this *right now*, but it needs to happenDoes that work for everyone?
Agreed. This patch, which we'll want to go to -stable, should clearly go in first. Marc's patch can go in after, since it's not a -stable candidate.
Thomas, could you add the cc to stable when picking this patch?
Quoted 147 lines · Eric Anholt
Quoted 145 lines · Florian Fainelli
Quoted 127 lines · Phil Elwell
Quoted 126 lines · Marc Zyngier
Quoted 108 lines · Phil Elwell
Quoted 98 lines · Marc Zyngier
Quoted 87 lines · Phil Elwell
Quoted 79 lines · Marc Zyngier
Quoted 69 lines · Phil Elwell
Quoted 62 lines · Marc Zyngier
Quoted 61 lines · Phil Elwell
Quoted 59 lines · Marc Zyngier
Quoted 54 lines · Eric Anholt
Marc Zyngier <[email protected]> writes:
On 09/05/17 17:59, Eric Anholt wrote:
Phil Elwell <[email protected]> writes:In order to reduce power consumption and bus traffic, it is sensible
for secondary cores to enter a low-power idle state when waiting to
be started. The wfe instruction causes a core to wait until an event
or interrupt arrives before continuing to the next instruction.
The sev instruction sends a wakeup event to the other cores, so call
it from bcm2836_smp_boot_secondary, the function that wakes up the
waiting cores during booting.It is harmless to use this patch without the corresponding change
adding wfe to the ARMv7/ARMv8-32 stubs, but if the stubs are updated
and this patch is not applied then the other cores will sleep forever.See: https://github.com/raspberrypi/linux/issues/1989
Signed-off-by: Phil Elwell <[email protected]>
---
drivers/irqchip/irq-bcm2836.c | 3 +++
1 file changed, 3 insertions(+)diff --git a/drivers/irqchip/irq-bcm2836.c b/drivers/irqchip/irq-bcm2836.c
index e10597c..6dccdf9 100644
--- a/drivers/irqchip/irq-bcm2836.c
+++ b/drivers/irqchip/irq-bcm2836.c
@@ -248,6 +248,9 @@ static int __init bcm2836_smp_boot_secondary(unsigned int cpu,
writel(secondary_startup_phys,
intc.base + LOCAL_MAILBOX3_SET0 + 16 * cpu);+ dsb(sy); /* Ensure write has completed before waking the other CPUs */
+ sev();
+
return 0;
}This is also the behavior that the standard arm64 spin-table
method has,
which we unfortunately can't quite use.And why is that so? Why do you have to reinvent the
wheel (and hide the
cloned wheel in an interrupt controller driver)?That doesn't seem right to me.
The armv8 stubs (firmware-supplied code in the low page that do the
spinning) do actually implement arm64's spin-table
method. It's the
armv7 stubs that use these registers in the irqchip
instead of plain
addresses in system memory.Let's put ARMv7 aside for the time being. If your firmware already implements spin-tables, why don't you simply use that at least on arm64?
Quoted 5 lines
Quoted 1 line
We do.
Obviously not the way it is intended if you have to duplicate the core architectural code in the interrupt controller driver, which couldn't care less.
Quoted 5 lines
If we were using this method on arm64 then the other cores would not start up
becausearmstub8.Shas always included a wfe. Nothing in the commit mentions
arm64 - this is an ARCH=arm fix.Thanks for the clarification, which you could have added to the commit message.
Quoted 6 lines
The question still remains: why do we have CPU bring-up code in an interrupt controller, instead of having it in the architecture code?
The RPi-2 is the *only* platform to have its SMP bringup code outside of
arch/arm, so the first course of action would be to move that code where it belongs.Quoted 5 lines
You were CC'd on the commit (
41f4988cc287e5f836d3f6620c9f900bc9b560e9) that introducedbcm2836_smp_boot_secondary- it seems strange to start objecting
now.Well, I'm far from being perfect. If I had noticed it, I'd have NACKed it.
Quoted 8 lines
Quoted 4 lines
Yes, I think it is odd that it didn't go into
arch/arm/mach-bcm, but in the interests of making changes in small, independent steps, do you have a
problem with this commit?On its own, no. I'm just not keen on adding more unrelated stuff to this file, so let's start with dealing with the original bug, and you can then add this fix on top.
Quoted 6 lines
That's an interesting use of the word "bug". From Wikipedia:
"A software bug is an error, flaw, failure or fault in a computer program or
system that causes it to produce an incorrect or unexpected result, or to behave in unintended ways."Whatever. Should I call it "pile of crap dumped in unsuitable locations" instead? What does Wikipedia says about it?
Quoted 6 lines
Although your concerns are valid, the faults you are objecting to are not causing
a malfunction of any kind. If we were to update the RPi firmware before this
patch was merged then upstream users would be left with one wheel on their wagon.Quoted 4 lines
Quoted 2 lines
And that'd be your problem, not mine. Look, you can argue around this all day, or you can fix this mess. Your choice.
Is that the opinion of all here?
The choice of word here got largely out of the original topic and I surely did eat a ton of popcorn here. There are two things that need fixing, and the time line and process for fixing these is clear:
- your bugfix (Phil) is something that should be applied now, and backported to -stable trees once the fix hits the irqchip tree (or Linus')
- relocating the code that does the secondary boot out of
drivers/irqchip/intoarch/arm/mach-bcm/needs to happen (Marc), and this is 4.13 material, there is no urgency in doing this *right now*, but it needs to happen
Quoted 7 lines
Quoted 1 line
Does that work for everyone?
Agreed. This patch, which we'll want to go to -stable, should clearly go in first. Marc's patch can go in after, since it's not a -stable candidate.
Thomas, could you add the cc to stable when picking this patch?
So, for the record, I'm clearly NAKing this patch. Thomas or Jason can pick it if they want to, I definitely won't.
M.
Signature · 1 line
Jazz is not dead, it just smell funny.