Skip to contentSkip to conversation

rfence related question

OpenSBI Leo Liang 2 participants 3 messages Oct 14, 2021 – Oct 19, 2021

lore
Share & tools
lore

Leo Liang

author
#0 lore raw

Hi,

I am studying the rfence part of OpenSBI recently.

From what I understand the mechanism of the IPI framework is to do "update" to update the IPI information for remote harts, and then send IPI through "ipi_dev" device to remote harts sequentially.

After the first IPI is sent (let's say from hart 0 to hart 1),
hart 0 waits in the "sync" for the remote hart(hart 1) to do "process".
Hart 0 waits until the per hart tlb_sync flag is set from the remote hart(hart1) after "process" is done, and then keep on sending the next IPI to hart 2, and then hart 3 ... etc.

If I understand it correctly, then would the following scenario have the possibility to occur?

In Linux, Hart 0 executes remote I cache flush instruction and sends IPI to hart 1; At the meantime, hart 2 executes the same instruction and tries to send IPI to hart 0.

Both hart 0 and hart 2 step into m-mode, their mstatus.mie are disabled,
so, when hart 0 tries to send IPI to hart 2 and wait for the sync, deadlock happens.
Hart 0 is waiting for hart 2 and hart 2 is also waiting for hart 0 indefinitely because the interrupts being disabled.

Is this case likely to happen?

Best regards,
Leo

#1 lore raw
Quoted 16 lines · Leo Liang

Hi,

I am studying the rfence part of OpenSBI recently.

From what I understand the mechanism of the IPI framework is to do "update" to update the IPI information for remote harts, and then send IPI through "ipi_dev" device to remote harts sequentially.

After the first IPI is sent (let's say from hart 0 to hart 1),
hart 0 waits in the "sync" for the remote hart(hart 1) to do "process".
Hart 0 waits until the per hart tlb_sync flag is set from the remote hart(hart1) after "process" is done, and then keep on sending the next IPI to hart 2, and then hart 3 ... etc.

If I understand it correctly, then would the following scenario have the possibility to occur?

Quoted 9 lines

In Linux, Hart 0 executes remote I cache flush instruction and sends IPI to hart 1; At the meantime, hart 2 executes the same instruction and tries to send IPI to hart 0.

Both hart 0 and hart 2 step into m-mode, their mstatus.mie are disabled,
so, when hart 0 tries to send IPI to hart 2 and wait for the sync, deadlock happens.
Hart 0 is waiting for hart 2 and hart 2 is also waiting for hart 0 indefinitely because the interrupts being disabled.

Is this case likely to happen?

Nope because each hart uses a busy loop and consume the pending IPIs

https://github.com/riscv-software-src/opensbi/blob/master/lib/sbi/sbi_tlb.c#L268

Quoted 7 lines

Best regards,
Leo

--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

Signature · 2 lines
Regards,
Atish
author
#2 lore raw
Quoted 24 lines · Atish Patra
Quoted 23 lines · Leo Liang

Hi,

I am studying the rfence part of OpenSBI recently.

From what I understand the mechanism of the IPI framework is to do "update" to update the IPI information for remote harts, and then send IPI through "ipi_dev" device to remote harts sequentially.

After the first IPI is sent (let's say from hart 0 to hart 1),
hart 0 waits in the "sync" for the remote hart(hart 1) to do "process".
Hart 0 waits until the per hart tlb_sync flag is set from the remote hart(hart1) after "process" is done, and then keep on sending the next IPI to hart 2, and then hart 3 ... etc.

If I understand it correctly, then would the following scenario have the possibility to occur?

In Linux, Hart 0 executes remote I cache flush instruction and sends IPI to hart 1; At the meantime, hart 2 executes the same instruction and tries to send IPI to hart 0.

Both hart 0 and hart 2 step into m-mode, their mstatus.mie are disabled,
so, when hart 0 tries to send IPI to hart 2 and wait for the sync, deadlock happens.
Hart 0 is waiting for hart 2 and hart 2 is also waiting for hart 0 indefinitely because the interrupts being disabled.

Quoted 7 lines
Quoted 2 lines

Is this case likely to happen?

Nope because each hart uses a busy loop and consume the pending IPIs

https://github.com/riscv-software-src/opensbi/blob/master/lib/sbi/sbi_tlb.c#L268

Hi Atish,

Got it! Thanks for the pointer!

Best regards,
Leo

Quoted 13 lines
Quoted 7 lines

Best regards,
Leo

--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

Signature · 2 lines
Regards,
Atish