Allow relaxed MMIO writes in device IPI callbacks
cover Allow relaxed MMIO writes in device IPI callbacks
This series improves the barrier usage around device IPI callbacks such that platform IPI driver can now use relaxed MMIO writes in device IPI callbacks.
These patches can also be found in the sbi_ipi_barrier_imp_v1 branch
at: https://github.com/avpatel/opensbi.git
Anup Patel (2):
lib: sbi: Allow relaxed MMIO writes in device ipi_send() callback
lib: sbi: Allow relaxed MMIO writes in device ipi_clear() callback
lib/sbi/sbi_ipi.c | 31 +++++++++++++++++++++++++------ lib/utils/ipi/aclint_mswi.c | 4 ++-- lib/utils/irqchip/imsic.c | 4 ++-- 3 files changed, 29 insertions(+), 10 deletions(-)
Signature · 1 line
2.34.1
PATCH 1/2 lib: sbi: Allow relaxed MMIO writes in device ipi_send() callback
Currently, we have a smp_wmb() between atomic_raw_set_bit() and ipi_send() device callback whereas the MMIO writes done by the device ipi_send() callback will also include a barrier.
We can avoid unnecessary/redundant barriers described above by allowing relaxed MMIO writes in device ipi_send() callback. To achieve this, we simply use wmb() instead of smp_wmb() before calling device ipi_send().
--- lib/sbi/sbi_ipi.c | 15 +++++++++++---- lib/utils/ipi/aclint_mswi.c | 2 +- lib/utils/irqchip/imsic.c | 4 ++-- 3 files changed, 14 insertions(+), 7 deletions(-)
lib/sbi/sbi_ipi.c +11 -4
@@ -67,14 +67,12 @@ static int sbi_ipi_send(struct sbi_scratch *scratch, u32 remote_hartindex,* trigger the interrupt */ atomic_raw_set_bit (event, &ipi_data->ipi_type); -smp_wmb (); -if (ipi_dev && ipi_dev->ipi_send) - ipi_dev->ipi_send (remote_hartindex); + ret =sbi_ipi_raw_send (remote_hartindex);sbi_pmu_ctr_incr_fw (SBI_PMU_FW_IPI_SENT ); -return 0 ; +return ret; }static int sbi_ipi_sync (struct sbi_scratch *scratch,u32 event)
@@ -243,6 +241,15 @@ int sbi_ipi_raw_send(u32 hartindex)if (!ipi_dev || !ipi_dev->ipi_send)return SBI_EINVAL ; +/* +* Ensure that memory or MMIO writes done before +* this function are not observed after the memory +* or MMIO writes done by the ipi_send() device +* callback. This also allows the ipi_send() device +* callback to use relaxed MMIO writes. +*/ +wmb (); + ipi_dev->ipi_send (hartindex);return 0 ; }
lib/utils/ipi/aclint_mswi.c +1 -1
@@ -41,7 +41,7 @@ static void mswi_ipi_send(u32 hart_index)/* Set ACLINT IPI */ msip = (void *)mswi->addr; -writel (1 , &msip[sbi_hartindex_to_hartid (hart_index) - +writel_relaxed (1 , &msip[sbi_hartindex_to_hartid (hart_index) - mswi->first_hartid]); }
lib/utils/irqchip/imsic.c +2 -2
@@ -186,8 +186,8 @@ static void imsic_ipi_send(u32 hart_index) }if (regs->size && (reloff < regs->size)) -writel (IMSIC_IPI_ID , - (void *)(regs->addr + reloff +IMSIC_MMIO_PAGE_LE )); +writel_relaxed (IMSIC_IPI_ID , + (void *)(regs->addr + reloff +IMSIC_MMIO_PAGE_LE )); }static struct sbi_ipi_device imsic_ipi_device = {
Signature · 1 line
2.34.1
PATCH 2/2 lib: sbi: Allow relaxed MMIO writes in device ipi_clear() callback
In reply to Anup PatelCurrently, there are no barriers before or after the ipi_clear() device callback which forces ipi_clear() device callback to always use non-relaxed MMIO writes.
Instead of above, we use wmb() in after the ipi_clear() device callback which pairs with the wmb() done before the ipi_send() device callback. This also allows device ipi_clear() callback to use relaxed MMIO writes.
--- lib/sbi/sbi_ipi.c | 16 ++++++++++++++-- lib/utils/ipi/aclint_mswi.c | 2 +- 2 files changed, 15 insertions(+), 3 deletions(-)
lib/sbi/sbi_ipi.c +14 -2
@@ -220,8 +220,7 @@ void sbi_ipi_process(void)u32 hartindex =sbi_hartid_to_hartindex (current_hartid ());sbi_pmu_ctr_incr_fw (SBI_PMU_FW_IPI_RECVD ); -if (ipi_dev && ipi_dev->ipi_clear) - ipi_dev->ipi_clear (hartindex); +sbi_ipi_raw_clear (hartindex); ipi_type =atomic_raw_xchg_ulong (&ipi_data->ipi_type,0 ); ipi_event =0 ;
@@ -247,6 +246,8 @@ int sbi_ipi_raw_send(u32 hartindex)* or MMIO writes done by the ipi_send() device * callback. This also allows the ipi_send() device * callback to use relaxed MMIO writes. +* +* This pairs with the wmb() in sbi_ipi_raw_clear(). */ wmb ();
@@ -258,6 +259,17 @@ void sbi_ipi_raw_clear(u32 hartindex) {if (ipi_dev && ipi_dev->ipi_clear) ipi_dev->ipi_clear (hartindex); + +/* +* Ensure that memory or MMIO writes after this +* function returns are not observed before the +* memory or MMIO writes done by the ipi_clear() +* device callback. This also allows ipi_clear() +* device callback to use relaxed MMIO writes. +* +* This pairs with the wmb() in sbi_ipi_raw_send(). +*/ +wmb (); }const struct sbi_ipi_device *sbi_ipi_get_device (void )
lib/utils/ipi/aclint_mswi.c +1 -1
@@ -61,7 +61,7 @@ static void mswi_ipi_clear(u32 hart_index)/* Clear ACLINT IPI */ msip = (void *)mswi->addr; -writel (0 , &msip[sbi_hartindex_to_hartid (hart_index) - +writel_relaxed (0 , &msip[sbi_hartindex_to_hartid (hart_index) - mswi->first_hartid]); }
Signature · 1 line
2.34.1
cover Allow relaxed MMIO writes in device IPI callbacks
In reply to Anup Patel? 2023-11-21???? 21:47 +0530?Anup Patel???
Quoted 6 lines
This series improves the barrier usage around device IPI callbacks such that platform IPI driver can now use relaxed MMIO writes in device IPI callbacks.
These patches can also be found in the
sbi_ipi_barrier_imp_v1branch
at: https://github.com/avpatel/opensbi.git
This patch is partly similar to this one?
https://lists.infradead.org/pipermail/opensbi/2023-November/005931.html
Then we may also encounter the following case:
hart A hart B
send ipi--------->| |
| clear ipi |
| | set ipi_type
| |
| xchg ipi_type |
| |
|<-----------------| send ipi
| |
| |To deal with this case, my patch:
https://lists.infradead.org/pipermail/opensbi/2023-November/005932.html
Regards,
Xiang W
Quoted 14 lines
Anup Patel (2):
? lib: sbi: Allow relaxed MMIO writes in deviceipi_send()callback ? lib: sbi: Allow relaxed MMIO writes in deviceipi_clear()callback?lib/sbi/sbi_ipi.c?????????? | 31 +++++++++++++++++++++++++------ ?lib/utils/ipi/aclint_mswi.c |? 4 ++-- ?lib/utils/irqchip/imsic.c?? |? 4 ++--?3 files changed, 29 insertions(+), 10 deletions(-)
Signature · 1 line
2.34.1
cover Allow relaxed MMIO writes in device IPI callbacks
Quoted 14 lines · On Tue, Nov 21, 2023 at 10:18?PM Xiang W <[email protected]> wrote:
? 2023-11-21???? 21:47 +0530?Anup Patel???
Quoted 6 lines
This series improves the barrier usage around device IPI callbacks such that platform IPI driver can now use relaxed MMIO writes in device IPI callbacks.
These patches can also be found in the
sbi_ipi_barrier_imp_v1branch
at: https://github.com/avpatel/opensbi.gitThis patch is partly similar to this one?
https://lists.infradead.org/pipermail/opensbi/2023-November/005931.htmlThen we may also encounter the following case:
hart A hart B
Quoted 9 lines
send ipi--------->| |
| clear ipi |
| | set ipi_type
| |
| xchg ipi_type |
| |
|<-----------------| send ipi
| |
| |This case is handled.
Quoted 3 lines
To deal with this case, my patch:
https://lists.infradead.org/pipermail/opensbi/2023-November/005932.html
Using spinlock for IPI synchronization is very handed and reduces performance.
Functionally we have an IPI mux built on-top of one HW IPI which is functionally similar to the Linux generic IPI-mux.
Regards,
Anup
Quoted 18 lines
Regards,
Xiang WQuoted 14 lines
Anup Patel (2):
lib: sbi: Allow relaxed MMIO writes in deviceipi_send()callback
lib: sbi: Allow relaxed MMIO writes in deviceipi_clear()callbacklib/sbi/sbi_ipi.c | 31 +++++++++++++++++++++++++------ lib/utils/ipi/aclint_mswi.c | 4 ++-- lib/utils/irqchip/imsic.c | 4 ++-- 3 files changed, 29 insertions(+), 10 deletions(-)--
2.34.1
cover Allow relaxed MMIO writes in device IPI callbacks
? 2023-11-21???? 23:03 +0530?Anup Patel???
Quoted 28 lines · On Tue, Nov 21, 2023 at 10:18?PM Xiang W <[email protected]> wrote:
Quoted 14 lines
? 2023-11-21???? 21:47 +0530?Anup Patel???
Quoted 6 lines
This series improves the barrier usage around device IPI callbacks such that platform IPI driver can now use relaxed MMIO writes in device IPI callbacks.
These patches can also be found in the
sbi_ipi_barrier_imp_v1branch
at: https://github.com/avpatel/opensbi.gitThis patch is partly similar to this one?
https://lists.infradead.org/pipermail/opensbi/2023-November/005931.htmlThen we may also encounter the following case:
??????????????? hart A???????????? hart B
Quoted 9 lines
send ipi--------->|????????????????? | ????????????????? | clear ipi??????? | ????????????????? |????????????????? | set ipi_type ????????????????? |????????????????? | ????????????????? | xchg ipi_type??? | ????????????????? |????????????????? | ????????????????? |<-----------------| send ipi ????????????????? |????????????????? | ????????????????? |????????????????? |This case is handled.
Quoted 1 line
Quoted 8 lines
Quoted 2 lines
To deal with this case, my patch:
https://lists.infradead.org/pipermail/opensbi/2023-November/005932.htmlUsing spinlock for IPI synchronization is very handed and reduces performance.
Functionally we have an IPI mux built on-top of one HW IPI which is functionally similar to the Linux generic IPI-mux.
New patch at
https://lists.infradead.org/pipermail/opensbi/2023-November/005965.html
Regards,
Xiang W
Quoted 22 lines
Regards,
AnupQuoted 18 lines
Regards,
Xiang WQuoted 14 lines
Anup Patel (2):
? lib: sbi: Allow relaxed MMIO writes in deviceipi_send()callback ? lib: sbi: Allow relaxed MMIO writes in deviceipi_clear()callback?lib/sbi/sbi_ipi.c?????????? | 31 +++++++++++++++++++++++++------ ?lib/utils/ipi/aclint_mswi.c |? 4 ++-- ?lib/utils/irqchip/imsic.c?? |? 4 ++--?3 files changed, 29 insertions(+), 10 deletions(-)
--
2.34.1
cover Allow relaxed MMIO writes in device IPI callbacks
In reply to Anup PatelQuoted 7 lines · Anup Patel
This series improves the barrier usage around device IPI callbacks such that platform IPI driver can now use relaxed MMIO writes in device IPI callbacks.
These patches can also be found in the
sbi_ipi_barrier_imp_v1branch
at: https://github.com/avpatel/opensbi.git
Quoted 9 lines
Anup Patel (2):
lib: sbi: Allow relaxed MMIO writes in deviceipi_send()callback
lib: sbi: Allow relaxed MMIO writes in deviceipi_clear()callbacklib/sbi/sbi_ipi.c | 31 +++++++++++++++++++++++++------ lib/utils/ipi/aclint_mswi.c | 4 ++-- lib/utils/irqchip/imsic.c | 4 ++-- 3 files changed, 29 insertions(+), 10 deletions(-)
Hi Anup,
Thank you so much for fixing this. I hope my previous discussion helped. Would you mind having a Reported-by: Bo Gan <[email protected]> ? Really appreciate you helping me understand the code.
Bo
cover Allow relaxed MMIO writes in device IPI callbacks
Quoted 18 lines · On Wed, Nov 22, 2023 at 6:15?AM Bo Gan <[email protected]> wrote:
Quoted 7 lines · Anup Patel
This series improves the barrier usage around device IPI callbacks such that platform IPI driver can now use relaxed MMIO writes in device IPI callbacks.
These patches can also be found in the
sbi_ipi_barrier_imp_v1branch
at: https://github.com/avpatel/opensbi.gitQuoted 9 lines
Anup Patel (2):
lib: sbi: Allow relaxed MMIO writes in deviceipi_send()callback
lib: sbi: Allow relaxed MMIO writes in deviceipi_clear()callbacklib/sbi/sbi_ipi.c | 31 +++++++++++++++++++++++++------ lib/utils/ipi/aclint_mswi.c | 4 ++-- lib/utils/irqchip/imsic.c | 4 ++-- 3 files changed, 29 insertions(+), 10 deletions(-)
Quoted 5 lines
Hi Anup,
Thank you so much for fixing this. I hope my previous discussion helped. Would you mind having a Reported-by: Bo Gan <[email protected]> ? Really appreciate you helping me understand the code.
Sure, I will add Reported-by
Regards,
Anup
Anup Patel
Appliedcover Allow relaxed MMIO writes in device IPI callbacks
In reply to Anup PatelQuoted 11 lines · On Tue, Nov 21, 2023 at 9:48?PM Anup Patel <[email protected]> wrote:
This series improves the barrier usage around device IPI callbacks such that platform IPI driver can now use relaxed MMIO writes in device IPI callbacks.
These patches can also be found in the
sbi_ipi_barrier_imp_v1branch
at: https://github.com/avpatel/opensbi.gitAnup Patel (2):
lib: sbi: Allow relaxed MMIO writes in deviceipi_send()callback
lib: sbi: Allow relaxed MMIO writes in deviceipi_clear()callback
Applied this series to the riscv/opensbi repo.
Regards,
Anup
Quoted 9 lines
lib/sbi/sbi_ipi.c | 31 +++++++++++++++++++++++++------ lib/utils/ipi/aclint_mswi.c | 4 ++-- lib/utils/irqchip/imsic.c | 4 ++-- 3 files changed, 29 insertions(+), 10 deletions(-)--
2.34.1