RISC-V: KVM: Delegate illegal instruction fault
Subsystems (3)
PATCH 1/1 RISC-V: KVM: Delegate illegal instruction fault
Delegate illegal instruction fault to VS mode in default to avoid such exceptions being trapped to HS and redirected back to VS.
--- arch/riscv/include/asm/kvm_host.h | 1 + 1 file changed, 1 insertion(+)
arch/riscv/include/asm/kvm_host.h +1 -0
@@ -48,6 +48,7 @@ BIT(EXC_SYSCALL) | \ BIT(EXC_INST_PAGE_FAULT) | \ BIT(EXC_LOAD_PAGE_FAULT) | \ + BIT(EXC_INST_ILLEGAL) | \ BIT(EXC_STORE_PAGE_FAULT))#define KVM_HIDELEG_DEFAULT (BIT(IRQ_VS_SOFT) | \
Signature · 1 line
2.20.1
2025-06-20T17:17:20+08:00, Xu Lu <[email protected]>:
arch/riscv/include/asm/kvm_host.h line 48
Quoted 8 lines
Delegate illegal instruction fault to VS mode in default to avoid such exceptions being trapped to HS and redirected back to VS.
Signed-off-by Xu Lu <[email protected]>---
arch/riscv/include/asm/kvm_host.h +1
-0@@ -48,6 +48,7 @@ + BIT(EXC_INST_ILLEGAL) | \
You should also remove the dead code in kvm_riscv_vcpu_exit.
And why not delegate the others as well? (EXC_LOAD_MISALIGNED, EXC_STORE_MISALIGNED, EXC_LOAD_ACCESS,
EXC_STORE_ACCESS, and EXC_INST_ACCESS.)
Thanks.
Hi Radim,
Quoted 5 lines · Radim Krčmář
2025-06-20T17:17:20+08:00, Xu Lu <[email protected]>:Quoted 3 lines
Delegate illegal instruction fault to VS mode in default to avoid such exceptions being trapped to HS and redirected back to VS.
Quoted 7 lines
arch/riscv/include/asm/kvm_host.h line 48
Quoted 5 lines
Signed-off-by Xu Lu <[email protected]>---
arch/riscv/include/asm/kvm_host.h +1
-0@@ -48,6 +48,7 @@ + BIT(EXC_INST_ILLEGAL) | \You should also remove the dead code in
kvm_riscv_vcpu_exit.
I only want to delegate it by default. And KVM may still want to delegate different exceptions for different VMs like what it does for EXC_BREAKPOINT. So maybe it is better to reserve these codes?
Quoted 4 lines
And why not delegate the others as well? (
EXC_LOAD_MISALIGNED,EXC_STORE_MISALIGNED,EXC_LOAD_ACCESS,
EXC_STORE_ACCESS, andEXC_INST_ACCESS.)
Thanks for the reminder. I will have a test and resend the patch if it works.
Quoted 2 lines
Thanks.
Best Regards,
Xu Lu
2025-06-22T18:11:49+08:00, Xu Lu <[email protected]>:
Quoted 16 lines
Hi Radim,
Quoted 5 lines · Radim Krčmář
2025-06-20T17:17:20+08:00, Xu Lu <[email protected]>:Quoted 3 lines
Delegate illegal instruction fault to VS mode in default to avoid such exceptions being trapped to HS and redirected back to VS.
Quoted 7 lines
arch/riscv/include/asm/kvm_host.h line 48
Quoted 5 lines
Signed-off-by Xu Lu <[email protected]>---
arch/riscv/include/asm/kvm_host.h +1
-0@@ -48,6 +48,7 @@ + BIT(EXC_INST_ILLEGAL) | \You should also remove the dead code in
kvm_riscv_vcpu_exit.
Quoted 3 lines
I only want to delegate it by default. And KVM may still want to delegate different exceptions for different VMs like what it does for
EXC_BREAKPOINT.
(I think we could easily reintroduce the code if KVM wants to do that in
the future. I also think that it's bad that this patch is doing an observable change without userspace involvement -- the counting of KVM SBI PMU events, but others will probably disagree with me on this.)
Quoted 1 line
So maybe it is better to reserve these codes?
Possibly, the current is acceptable if you have considered the implications on PMU events.
Quoted 5 lines
Quoted 3 lines
And why not delegate the others as well? (
EXC_LOAD_MISALIGNED,EXC_STORE_MISALIGNED,EXC_LOAD_ACCESS,
EXC_STORE_ACCESS, andEXC_INST_ACCESS.)Thanks for the reminder. I will have a test and resend the patch if it works.
The misaligned exceptions are already being worked on, so don't waste your time on them, sorry.
Hi Radim,
Quoted 27 lines · Radim Krčmář
2025-06-22T18:11:49+08:00, Xu Lu <[email protected]>:Quoted 16 lines
Hi Radim,
Quoted 5 lines · Radim Krčmář
2025-06-20T17:17:20+08:00, Xu Lu <[email protected]>:Quoted 3 lines
Delegate illegal instruction fault to VS mode in default to avoid such exceptions being trapped to HS and redirected back to VS.
Quoted 7 lines
arch/riscv/include/asm/kvm_host.h line 48
Quoted 5 lines
Signed-off-by Xu Lu <[email protected]>---
arch/riscv/include/asm/kvm_host.h +1
-0@@ -48,6 +48,7 @@ + BIT(EXC_INST_ILLEGAL) | \You should also remove the dead code in
kvm_riscv_vcpu_exit.Quoted 3 lines
I only want to delegate it by default. And KVM may still want to delegate different exceptions for different VMs like what it does for
EXC_BREAKPOINT.(I think we could easily reintroduce the code if KVM wants to do that in
the future. I also think that it's bad that this patch is doing an observable change without userspace involvement -- the counting of KVM SBI PMU events, but others will probably disagree with me on this.)
Quoted 4 lines
Quoted 1 line
So maybe it is better to reserve these codes?
Possibly, the current is acceptable if you have considered the implications on PMU events.
So maybe it comes back to our discussion on the difference between vs insn fault and illegal insn fault again~ In my personal opinion, it seems to be a waste of CPU resources to trap illegal instruction to HS-mode hypervisor, which does nothing but redirect it back to VS-mode guest kernel. I think it is OK (and maybe it should) to record 0 illegal instruction exits in KVM PMU. If someone wants illegal insn to trigger an vcpu exit, then an ioctl can be provided to remove the delegation like what KVM_SET_GUEST_DEBUG does.
Quoted 9 lines
Quoted 6 lines
Quoted 4 lines
And why not delegate the others as well? (
EXC_LOAD_MISALIGNED,EXC_STORE_MISALIGNED,EXC_LOAD_ACCESS,
EXC_STORE_ACCESS, andEXC_INST_ACCESS.)Thanks for the reminder. I will have a test and resend the patch if it works.
The misaligned exceptions are already being worked on, so don't waste your time on them, sorry.
Thanks for the reminder too. I did not consider this before. I will leave the MISALIGNED faults alone.
Best Regards,
Xu Lu
Quoted 12 lines · Radim Krčmář
2025-06-20T17:17:20+08:00, Xu Lu <[email protected]>:arch/riscv/include/asm/kvm_host.h line 48
Quoted 8 lines
Delegate illegal instruction fault to VS mode in default to avoid such exceptions being trapped to HS and redirected back to VS.
Signed-off-by Xu Lu <[email protected]>---
arch/riscv/include/asm/kvm_host.h +1
-0@@ -48,6 +48,7 @@ + BIT(EXC_INST_ILLEGAL) | \You should also remove the dead code in
kvm_riscv_vcpu_exit.
Quoted 3 lines
And why not delegate the others as well? (
EXC_LOAD_MISALIGNED,EXC_STORE_MISALIGNED,EXC_LOAD_ACCESS,
EXC_STORE_ACCESS, andEXC_INST_ACCESS.)
Currently, OpenSBI does not delegate misaligned exception by default and handles misaligned access by itself, this is (partially) why we added the FWFT SBI extension to request such delegation. Since some supervisor software expect that default, they do not have code to handle misaligned accesses emulation. So they should not be delegated by default.
Thanks,
Clément
Quoted 7 lines
Thanks.
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
2025-06-23T10:04:45+02:00, Clément Léger <[email protected]>:
Quoted 10 lines · Radim Krčmář
Quoted 3 lines
And why not delegate the others as well? (
EXC_LOAD_MISALIGNED,EXC_STORE_MISALIGNED,EXC_LOAD_ACCESS,
EXC_STORE_ACCESS, andEXC_INST_ACCESS.)Currently, OpenSBI does not delegate misaligned exception by default and handles misaligned access by itself, this is (partially) why we added the FWFT SBI extension to request such delegation. Since some supervisor software expect that default, they do not have code to handle misaligned accesses emulation. So they should not be delegated by default.
Yeah, I forgot about your patches that conflict with the change, thanks.
(The current KVM exception handler only forwards all the listed
exceptions, so the only observable difference this change would make is that the KVM SBI PMU event is not counted.)
Hi Clément,
Quoted 20 lines · Clément Léger
Quoted 12 lines · Radim Krčmář
2025-06-20T17:17:20+08:00, Xu Lu <[email protected]>:arch/riscv/include/asm/kvm_host.h line 48
Quoted 8 lines
Delegate illegal instruction fault to VS mode in default to avoid such exceptions being trapped to HS and redirected back to VS.
Signed-off-by Xu Lu <[email protected]>---
arch/riscv/include/asm/kvm_host.h +1
-0@@ -48,6 +48,7 @@ + BIT(EXC_INST_ILLEGAL) | \You should also remove the dead code in
kvm_riscv_vcpu_exit.Quoted 3 lines
And why not delegate the others as well? (
EXC_LOAD_MISALIGNED,EXC_STORE_MISALIGNED,EXC_LOAD_ACCESS,
EXC_STORE_ACCESS, andEXC_INST_ACCESS.)
Quoted 5 lines
Currently, OpenSBI does not delegate misaligned exception by default and handles misaligned access by itself, this is (partially) why we added the FWFT SBI extension to request such delegation. Since some supervisor software expect that default, they do not have code to handle misaligned accesses emulation. So they should not be delegated by default.
It doesn't matter whether these exceptions are delegated in medeleg. KVM in HS-mode does not handle illegal instruction or misaligned access and only redirects them back to VS-mode. Delegating such exceptions in hedeleg helps save CPU usage even when they are not delegated in medeleg: opensbi will check whether these exceptions are delegated to VS-mode and redirect them to VS-mode if possible. There seems to be no conflicts with SSE implementation. Please correct me if I missed anything.
Best Regards,
Xu Lu
Quoted 13 lines
Thanks,
Clément
Quoted 7 lines
Thanks.
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Quoted 23 lines · Xu Lu
Hi Clément,
Quoted 20 lines · Clément Léger
Quoted 15 lines · Radim Krčmář
2025-06-20T17:17:20+08:00, Xu Lu <[email protected]>:arch/riscv/include/asm/kvm_host.h line 48
Quoted 8 lines
Delegate illegal instruction fault to VS mode in default to avoid such exceptions being trapped to HS and redirected back to VS.
Signed-off-by Xu Lu <[email protected]>---
arch/riscv/include/asm/kvm_host.h +1
-0@@ -48,6 +48,7 @@ + BIT(EXC_INST_ILLEGAL) | \You should also remove the dead code in
kvm_riscv_vcpu_exit.And why not delegate the others as well? (
EXC_LOAD_MISALIGNED,EXC_STORE_MISALIGNED,EXC_LOAD_ACCESS,
EXC_STORE_ACCESS, andEXC_INST_ACCESS.)
Quoted 7 lines
Quoted 5 lines
Currently, OpenSBI does not delegate misaligned exception by default and handles misaligned access by itself, this is (partially) why we added the FWFT SBI extension to request such delegation. Since some supervisor software expect that default, they do not have code to handle misaligned accesses emulation. So they should not be delegated by default.
It doesn't matter whether these exceptions are delegated in medeleg.
Not sure to totally understand, but if the exceptions are not delegated in medeleg, they won't be delegated to VS-mode even though hedeleg bit is set right ? The spec says:
A synchronous trap that has been delegated to HS-mode (using medeleg) is further delegated to VS-mode if V=1 before the trap and the corresponding hedeleg bit is set.
Quoted 7 lines
KVM in HS-mode does not handle illegal instruction or misaligned access and only redirects them back to VS-mode. Delegating such exceptions in hedeleg helps save CPU usage even when they are not delegated in medeleg: opensbi will check whether these exceptions are delegated to VS-mode and redirect them to VS-mode if possible. There seems to be no conflicts with SSE implementation. Please correct me if I missed anything.
AFAIU, this means that since medeleg bit for misaligned accesses were not delegated up to the introduction of the FWFT extension, VS-mode generated misaligned accesses were handled by OpenSBI right ? Now that we are requesting openSBI to delegate misaligned accesses, HS-mode handles it's own misaligned accesses through the trap handler. With that configuration, if VS-mode generate a misaligned access, it will end up being redirected to VS-mode and won't be handle by HS-mode.
To summarize, prior to FWFT, medeleg wasn't delegating misaligned accesses to S-mode:
- VS-mode misaligned access -> trap to M-mode -> OpenSBI handle it -> Back to VS-mode, misaligned access fixed up by OpenSBI
Now that Linux uses SBI FWFT to delegates misaligned accesses (without hedeleg being set for misaligned delegation, but that doesn't really matter, the outcome is the same):
- VS-mode misaligned access -> trap to HS-mode -> redirection to VS-mode, needs to handle the misaligned access by itself
This means that previously, misaligned access were silently fixed up by OpenSBI for VS-mode and now that FWFT is used for delegation, this isn't true anymore. So, old kernel or sueprvisor software that included code to handle misaligned accesses will crash. Did I missed something ?
Note: this is not directly related to your series but my introduction of FWFT !
Thanks,
Clément
Quoted 17 lines
Best Regards,
Xu LuQuoted 13 lines
Thanks,
Clément
Quoted 7 lines
Thanks.
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Hi Clément,
Quoted 35 lines · Clément Léger
Quoted 23 lines · Xu Lu
Hi Clément,
Quoted 20 lines · Clément Léger
Quoted 15 lines · Radim Krčmář
2025-06-20T17:17:20+08:00, Xu Lu <[email protected]>:arch/riscv/include/asm/kvm_host.h line 48
Quoted 8 lines
Delegate illegal instruction fault to VS mode in default to avoid such exceptions being trapped to HS and redirected back to VS.
Signed-off-by Xu Lu <[email protected]>---
arch/riscv/include/asm/kvm_host.h +1
-0@@ -48,6 +48,7 @@ + BIT(EXC_INST_ILLEGAL) | \You should also remove the dead code in
kvm_riscv_vcpu_exit.And why not delegate the others as well? (
EXC_LOAD_MISALIGNED,EXC_STORE_MISALIGNED,EXC_LOAD_ACCESS,
EXC_STORE_ACCESS, andEXC_INST_ACCESS.)Quoted 7 lines
Quoted 5 lines
Currently, OpenSBI does not delegate misaligned exception by default and handles misaligned access by itself, this is (partially) why we added the FWFT SBI extension to request such delegation. Since some supervisor software expect that default, they do not have code to handle misaligned accesses emulation. So they should not be delegated by default.
It doesn't matter whether these exceptions are delegated in medeleg.
Quoted 7 lines
Not sure to totally understand, but if the exceptions are not delegated in medeleg, they won't be delegated to VS-mode even though hedeleg bit is set right ? The spec says:
A synchronous trap that has been delegated to HS-mode (using medeleg) is further delegated to VS-mode if V=1 before the trap and the corresponding hedeleg bit is set.
Yes, you are right. The illegal insn exception is still trapped in M mode if it is not delegated in medeleg. But delegating it in hedeleg is still useful. The opensbi will check CSR_HEDELEG in the function sbi_trap_redirect. If the exception has been delegated to VS-mode in CSR_HEDLEG, opensbi can directly return back to VS-mode, without the overhead of going back to HS-mode and then going back to VS-mode.
Quoted 19 lines
Quoted 10 lines
KVM in HS-mode does not handle illegal instruction or misaligned access and only redirects them back to VS-mode. Delegating such exceptions in hedeleg helps save CPU usage even when they are not delegated in medeleg: opensbi will check whether these exceptions are delegated to VS-mode and redirect them to VS-mode if possible. There seems to be no conflicts with SSE implementation. Please correct me if I missed anything.
AFAIU, this means that since medeleg bit for misaligned accesses were not delegated up to the introduction of the FWFT extension, VS-mode generated misaligned accesses were handled by OpenSBI right ? Now that we are requesting openSBI to delegate misaligned accesses, HS-mode handles it's own misaligned accesses through the trap handler. With that configuration, if VS-mode generate a misaligned access, it will end up being redirected to VS-mode and won't be handle by HS-mode.
Quoted 5 lines
To summarize, prior to FWFT, medeleg wasn't delegating misaligned accesses to S-mode:
- VS-mode misaligned access -> trap to M-mode -> OpenSBI handle it -> Back to VS-mode, misaligned access fixed up by OpenSBI
Yes, this is what I want the procedure of handling illegal insn exceptions to be. Actually it now behaves as:
VS-mode illegal insn exception -> trap to M-mode -> OpenSBI handles it -> Back to HS-mode, does nothing -> Back to VS-mode.
I want to avoid going through HS-mode.
Quoted 5 lines
Now that Linux uses SBI FWFT to delegates misaligned accesses (without hedeleg being set for misaligned delegation, but that doesn't really matter, the outcome is the same):
Quoted 8 lines
- VS-mode misaligned access -> trap to HS-mode -> redirection to VS-mode, needs to handle the misaligned access by itself
This means that previously, misaligned access were silently fixed up by OpenSBI for VS-mode and now that FWFT is used for delegation, this isn't true anymore. So, old kernel or sueprvisor software that included code to handle misaligned accesses will crash. Did I missed something ?
Great! You make it very clear! Thanks for your explanation. But even when misalign exceptions are delegated to HS-mode, KVM seems to do nothing but redirect to VS-mode when VM get trapped due to misalign exceptions. So maybe we can directly delegate the misaligned exceptions in hedeleg too before running VCPU and retrieve them after VCPU exists. And then the handling procedure will be:
VS-mode misaligned exception -> trap to VS-mode -> VS handles it -> Back to VU-mode.
Please correct me if I missed anything.
Best Regards,
Xu Lu
Quoted 26 lines
Note: this is not directly related to your series but my introduction of FWFT !
Thanks,
Clément
Quoted 17 lines
Best Regards,
Xu LuQuoted 13 lines
Thanks,
Clément
Quoted 7 lines
Thanks.
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Quoted 78 lines · Xu Lu
Hi Clément,
Quoted 35 lines · Clément Léger
Quoted 23 lines · Xu Lu
Hi Clément,
Quoted 20 lines · Clément Léger
Quoted 15 lines · Radim Krčmář
2025-06-20T17:17:20+08:00, Xu Lu <[email protected]>:arch/riscv/include/asm/kvm_host.h line 48
Quoted 8 lines
Delegate illegal instruction fault to VS mode in default to avoid such exceptions being trapped to HS and redirected back to VS.
Signed-off-by Xu Lu <[email protected]>---
arch/riscv/include/asm/kvm_host.h +1
-0@@ -48,6 +48,7 @@ + BIT(EXC_INST_ILLEGAL) | \You should also remove the dead code in
kvm_riscv_vcpu_exit.And why not delegate the others as well? (
EXC_LOAD_MISALIGNED,EXC_STORE_MISALIGNED,EXC_LOAD_ACCESS,
EXC_STORE_ACCESS, andEXC_INST_ACCESS.)Quoted 7 lines
Quoted 5 lines
Currently, OpenSBI does not delegate misaligned exception by default and handles misaligned access by itself, this is (partially) why we added the FWFT SBI extension to request such delegation. Since some supervisor software expect that default, they do not have code to handle misaligned accesses emulation. So they should not be delegated by default.
It doesn't matter whether these exceptions are delegated in medeleg.
Quoted 7 lines
Not sure to totally understand, but if the exceptions are not delegated in medeleg, they won't be delegated to VS-mode even though hedeleg bit is set right ? The spec says:
A synchronous trap that has been delegated to HS-mode (using medeleg) is further delegated to VS-mode if V=1 before the trap and the corresponding hedeleg bit is set.
Yes, you are right. The illegal insn exception is still trapped in M mode if it is not delegated in medeleg. But delegating it in hedeleg is still useful. The opensbi will check
CSR_HEDELEGin the functionsbi_trap_redirect. If the exception has been delegated to VS-mode inCSR_HEDLEG, opensbi can directly return back to VS-mode, without the overhead of going back to HS-mode and then going back to VS-mode.Quoted 19 lines
Quoted 10 lines
KVM in HS-mode does not handle illegal instruction or misaligned access and only redirects them back to VS-mode. Delegating such exceptions in hedeleg helps save CPU usage even when they are not delegated in medeleg: opensbi will check whether these exceptions are delegated to VS-mode and redirect them to VS-mode if possible. There seems to be no conflicts with SSE implementation. Please correct me if I missed anything.
AFAIU, this means that since medeleg bit for misaligned accesses were not delegated up to the introduction of the FWFT extension, VS-mode generated misaligned accesses were handled by OpenSBI right ? Now that we are requesting openSBI to delegate misaligned accesses, HS-mode handles it's own misaligned accesses through the trap handler. With that configuration, if VS-mode generate a misaligned access, it will end up being redirected to VS-mode and won't be handle by HS-mode.
Quoted 5 lines
To summarize, prior to FWFT, medeleg wasn't delegating misaligned accesses to S-mode:
- VS-mode misaligned access -> trap to M-mode -> OpenSBI handle it -> Back to VS-mode, misaligned access fixed up by OpenSBI
Quoted 7 lines
Yes, this is what I want the procedure of handling illegal insn exceptions to be. Actually it now behaves as:
VS-mode illegal insn exception -> trap to M-mode -> OpenSBI handles it -> Back to HS-mode, does nothing -> Back to VS-mode.
I want to avoid going through HS-mode.
Hi Xu,
Yeah, that make sense as well but that should only happen if the VS-mode requested misaligned access delegation via KVM SBI FWFT interface. I know that currently KVM does do anything useful from the misaligned exception except redirecting it to VS-mode but IMHO, that's a regression I introduced with FWFT misaligned requested delegation...
Quoted 15 lines
Quoted 6 lines
Now that Linux uses SBI FWFT to delegates misaligned accesses (without hedeleg being set for misaligned delegation, but that doesn't really matter, the outcome is the same):
Quoted 8 lines
- VS-mode misaligned access -> trap to HS-mode -> redirection to VS-mode, needs to handle the misaligned access by itself
This means that previously, misaligned access were silently fixed up by OpenSBI for VS-mode and now that FWFT is used for delegation, this isn't true anymore. So, old kernel or sueprvisor software that included code to handle misaligned accesses will crash. Did I missed something ?
Quoted 4 lines
Great! You make it very clear! Thanks for your explanation. But even when misalign exceptions are delegated to HS-mode, KVM seems to do nothing but redirect to VS-mode when VM get trapped due to misalign exceptions.
Exactly, which is why I said that either setting hedeleg by default or not will lead to the same outcome, ie: VS-mode needs to handle access by itself (which is a regression introduced by FWFT usage).
Quoted 6 lines
So maybe we can directly delegate the misaligned exceptions in hedeleg too before running VCPU and retrieve them after VCPU exists. And then the handling procedure will be:
VS-mode misaligned exception -> trap to VS-mode -> VS handles it -> Back to VU-mode.
I'd better want to let the HS-mode handle the misaligned accesses if not requested via the KVM SBI FWFT interface by VS-mode to keep HS-mode expected behavior. As you pointed out, this is not currently the case and the misaligned exceptions are directly redirected to VS-mode, this differs from what was actually done previously without FWFT (ie OpenSBI handles the misaligned access).
To summarize, I think HS-mode should fixup VS-mode misaligned accesses unless requested via KVM SBI FWFT interface, in which case it will delegates them (which is done by the FWFT series). This would match the HS-mode <-> OpenSBI behavior.
Thanks,
Clément
Quoted 33 lines
Please correct me if I missed anything.
Best Regards,
Xu Lu
Quoted 26 lines
Note: this is not directly related to your series but my introduction of FWFT !
Thanks,
Clément
Quoted 17 lines
Best Regards,
Xu LuQuoted 13 lines
Thanks,
Clément
Quoted 7 lines
Thanks.
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Quoted 137 lines · Clément Léger
Quoted 78 lines · Xu Lu
Hi Clément,
Quoted 35 lines · Clément Léger
Quoted 23 lines · Xu Lu
Hi Clément,
Quoted 20 lines · Clément Léger
Quoted 15 lines · Radim Krčmář
2025-06-20T17:17:20+08:00, Xu Lu <[email protected]>:arch/riscv/include/asm/kvm_host.h line 48
Quoted 8 lines
Delegate illegal instruction fault to VS mode in default to avoid such exceptions being trapped to HS and redirected back to VS.
Signed-off-by Xu Lu <[email protected]>---
arch/riscv/include/asm/kvm_host.h +1
-0@@ -48,6 +48,7 @@ + BIT(EXC_INST_ILLEGAL) | \You should also remove the dead code in
kvm_riscv_vcpu_exit.And why not delegate the others as well? (
EXC_LOAD_MISALIGNED,EXC_STORE_MISALIGNED,EXC_LOAD_ACCESS,
EXC_STORE_ACCESS, andEXC_INST_ACCESS.)Quoted 7 lines
Quoted 5 lines
Currently, OpenSBI does not delegate misaligned exception by default and handles misaligned access by itself, this is (partially) why we added the FWFT SBI extension to request such delegation. Since some supervisor software expect that default, they do not have code to handle misaligned accesses emulation. So they should not be delegated by default.
It doesn't matter whether these exceptions are delegated in medeleg.
Quoted 7 lines
Not sure to totally understand, but if the exceptions are not delegated in medeleg, they won't be delegated to VS-mode even though hedeleg bit is set right ? The spec says:
A synchronous trap that has been delegated to HS-mode (using medeleg) is further delegated to VS-mode if V=1 before the trap and the corresponding hedeleg bit is set.
Yes, you are right. The illegal insn exception is still trapped in M mode if it is not delegated in medeleg. But delegating it in hedeleg is still useful. The opensbi will check
CSR_HEDELEGin the functionsbi_trap_redirect. If the exception has been delegated to VS-mode inCSR_HEDLEG, opensbi can directly return back to VS-mode, without the overhead of going back to HS-mode and then going back to VS-mode.Quoted 19 lines
Quoted 10 lines
KVM in HS-mode does not handle illegal instruction or misaligned access and only redirects them back to VS-mode. Delegating such exceptions in hedeleg helps save CPU usage even when they are not delegated in medeleg: opensbi will check whether these exceptions are delegated to VS-mode and redirect them to VS-mode if possible. There seems to be no conflicts with SSE implementation. Please correct me if I missed anything.
AFAIU, this means that since medeleg bit for misaligned accesses were not delegated up to the introduction of the FWFT extension, VS-mode generated misaligned accesses were handled by OpenSBI right ? Now that we are requesting openSBI to delegate misaligned accesses, HS-mode handles it's own misaligned accesses through the trap handler. With that configuration, if VS-mode generate a misaligned access, it will end up being redirected to VS-mode and won't be handle by HS-mode.
Quoted 5 lines
To summarize, prior to FWFT, medeleg wasn't delegating misaligned accesses to S-mode:
- VS-mode misaligned access -> trap to M-mode -> OpenSBI handle it -> Back to VS-mode, misaligned access fixed up by OpenSBI
Quoted 7 lines
Yes, this is what I want the procedure of handling illegal insn exceptions to be. Actually it now behaves as:
VS-mode illegal insn exception -> trap to M-mode -> OpenSBI handles it -> Back to HS-mode, does nothing -> Back to VS-mode.
I want to avoid going through HS-mode.
Hi Xu,
Yeah, that make sense as well but that should only happen if the VS-mode requested misaligned access delegation via KVM SBI FWFT interface. I know that currently KVM does do anything useful from the misaligned exception except redirecting it to VS-mode but IMHO, that's a regression I introduced with FWFT misaligned requested delegation...
Quoted 15 lines
Quoted 6 lines
Now that Linux uses SBI FWFT to delegates misaligned accesses (without hedeleg being set for misaligned delegation, but that doesn't really matter, the outcome is the same):
Quoted 8 lines
- VS-mode misaligned access -> trap to HS-mode -> redirection to VS-mode, needs to handle the misaligned access by itself
This means that previously, misaligned access were silently fixed up by OpenSBI for VS-mode and now that FWFT is used for delegation, this isn't true anymore. So, old kernel or sueprvisor software that included code to handle misaligned accesses will crash. Did I missed something ?
Quoted 4 lines
Great! You make it very clear! Thanks for your explanation. But even when misalign exceptions are delegated to HS-mode, KVM seems to do nothing but redirect to VS-mode when VM get trapped due to misalign exceptions.
Exactly, which is why I said that either setting hedeleg by default or not will lead to the same outcome, ie: VS-mode needs to handle access by itself (which is a regression introduced by FWFT usage).
Quoted 6 lines
So maybe we can directly delegate the misaligned exceptions in hedeleg too before running VCPU and retrieve them after VCPU exists. And then the handling procedure will be:
VS-mode misaligned exception -> trap to VS-mode -> VS handles it -> Back to VU-mode.
I'd better want to let the HS-mode handle the misaligned accesses if not requested via the KVM SBI FWFT interface by VS-mode to keep HS-mode expected behavior. As you pointed out, this is not currently the case and the misaligned exceptions are directly redirected to VS-mode, this differs from what was actually done previously without FWFT (ie OpenSBI handles the misaligned access).
Quoted 4 lines
To summarize, I think HS-mode should fixup VS-mode misaligned accesses unless requested via KVM SBI FWFT interface, in which case it will delegates them (which is done by the FWFT series). This would match the HS-mode <-> OpenSBI behavior.
Great! Roger that. Hope it can be fixed in the future.
Quoted 39 lines
Thanks,
Clément
Quoted 33 lines
Please correct me if I missed anything.
Best Regards,
Xu Lu
Quoted 26 lines
Note: this is not directly related to your series but my introduction of FWFT !
Thanks,
Clément
Quoted 17 lines
Best Regards,
Xu LuQuoted 13 lines
Thanks,
Clément
Quoted 7 lines
Thanks.
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Quoted 138 lines · Xu Lu
Quoted 137 lines · Clément Léger
Quoted 78 lines · Xu Lu
Hi Clément,
Quoted 35 lines · Clément Léger
Quoted 23 lines · Xu Lu
Hi Clément,
Quoted 20 lines · Clément Léger
Quoted 15 lines · Radim Krčmář
2025-06-20T17:17:20+08:00, Xu Lu <[email protected]>:arch/riscv/include/asm/kvm_host.h line 48
Quoted 8 lines
Delegate illegal instruction fault to VS mode in default to avoid such exceptions being trapped to HS and redirected back to VS.
Signed-off-by Xu Lu <[email protected]>---
arch/riscv/include/asm/kvm_host.h +1
-0@@ -48,6 +48,7 @@ + BIT(EXC_INST_ILLEGAL) | \You should also remove the dead code in
kvm_riscv_vcpu_exit.And why not delegate the others as well? (
EXC_LOAD_MISALIGNED,EXC_STORE_MISALIGNED,EXC_LOAD_ACCESS,
EXC_STORE_ACCESS, andEXC_INST_ACCESS.)Quoted 7 lines
Quoted 5 lines
Currently, OpenSBI does not delegate misaligned exception by default and handles misaligned access by itself, this is (partially) why we added the FWFT SBI extension to request such delegation. Since some supervisor software expect that default, they do not have code to handle misaligned accesses emulation. So they should not be delegated by default.
It doesn't matter whether these exceptions are delegated in medeleg.
Quoted 7 lines
Not sure to totally understand, but if the exceptions are not delegated in medeleg, they won't be delegated to VS-mode even though hedeleg bit is set right ? The spec says:
A synchronous trap that has been delegated to HS-mode (using medeleg) is further delegated to VS-mode if V=1 before the trap and the corresponding hedeleg bit is set.
Yes, you are right. The illegal insn exception is still trapped in M mode if it is not delegated in medeleg. But delegating it in hedeleg is still useful. The opensbi will check
CSR_HEDELEGin the functionsbi_trap_redirect. If the exception has been delegated to VS-mode inCSR_HEDLEG, opensbi can directly return back to VS-mode, without the overhead of going back to HS-mode and then going back to VS-mode.Quoted 19 lines
Quoted 10 lines
KVM in HS-mode does not handle illegal instruction or misaligned access and only redirects them back to VS-mode. Delegating such exceptions in hedeleg helps save CPU usage even when they are not delegated in medeleg: opensbi will check whether these exceptions are delegated to VS-mode and redirect them to VS-mode if possible. There seems to be no conflicts with SSE implementation. Please correct me if I missed anything.
AFAIU, this means that since medeleg bit for misaligned accesses were not delegated up to the introduction of the FWFT extension, VS-mode generated misaligned accesses were handled by OpenSBI right ? Now that we are requesting openSBI to delegate misaligned accesses, HS-mode handles it's own misaligned accesses through the trap handler. With that configuration, if VS-mode generate a misaligned access, it will end up being redirected to VS-mode and won't be handle by HS-mode.
Quoted 5 lines
To summarize, prior to FWFT, medeleg wasn't delegating misaligned accesses to S-mode:
- VS-mode misaligned access -> trap to M-mode -> OpenSBI handle it -> Back to VS-mode, misaligned access fixed up by OpenSBI
Quoted 7 lines
Yes, this is what I want the procedure of handling illegal insn exceptions to be. Actually it now behaves as:
VS-mode illegal insn exception -> trap to M-mode -> OpenSBI handles it -> Back to HS-mode, does nothing -> Back to VS-mode.
I want to avoid going through HS-mode.
Hi Xu,
Yeah, that make sense as well but that should only happen if the VS-mode requested misaligned access delegation via KVM SBI FWFT interface. I know that currently KVM does do anything useful from the misaligned exception except redirecting it to VS-mode but IMHO, that's a regression I introduced with FWFT misaligned requested delegation...
Quoted 15 lines
Quoted 6 lines
Now that Linux uses SBI FWFT to delegates misaligned accesses (without hedeleg being set for misaligned delegation, but that doesn't really matter, the outcome is the same):
Quoted 8 lines
- VS-mode misaligned access -> trap to HS-mode -> redirection to VS-mode, needs to handle the misaligned access by itself
This means that previously, misaligned access were silently fixed up by OpenSBI for VS-mode and now that FWFT is used for delegation, this isn't true anymore. So, old kernel or sueprvisor software that included code to handle misaligned accesses will crash. Did I missed something ?
Quoted 4 lines
Great! You make it very clear! Thanks for your explanation. But even when misalign exceptions are delegated to HS-mode, KVM seems to do nothing but redirect to VS-mode when VM get trapped due to misalign exceptions.
Exactly, which is why I said that either setting hedeleg by default or not will lead to the same outcome, ie: VS-mode needs to handle access by itself (which is a regression introduced by FWFT usage).
Quoted 6 lines
So maybe we can directly delegate the misaligned exceptions in hedeleg too before running VCPU and retrieve them after VCPU exists. And then the handling procedure will be:
VS-mode misaligned exception -> trap to VS-mode -> VS handles it -> Back to VU-mode.
I'd better want to let the HS-mode handle the misaligned accesses if not requested via the KVM SBI FWFT interface by VS-mode to keep HS-mode expected behavior. As you pointed out, this is not currently the case and the misaligned exceptions are directly redirected to VS-mode, this differs from what was actually done previously without FWFT (ie OpenSBI handles the misaligned access).
Quoted 6 lines
Quoted 4 lines
To summarize, I think HS-mode should fixup VS-mode misaligned accesses unless requested via KVM SBI FWFT interface, in which case it will delegates them (which is done by the FWFT series). This would match the HS-mode <-> OpenSBI behavior.
Great! Roger that. Hope it can be fixed in the future.
Ahah ! I'll take a look at it if everyone agrees that it's a regression.
Thanks,
Clément
Quoted 40 lines
Quoted 40 lines
Thanks,
Clément
Quoted 33 lines
Please correct me if I missed anything.
Best Regards,
Xu Lu
Quoted 26 lines
Note: this is not directly related to your series but my introduction of FWFT !
Thanks,
Clément
Quoted 17 lines
Best Regards,
Xu LuQuoted 13 lines
Thanks,
Clément
Quoted 7 lines
Thanks.
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv