Skip to contentSkip to conversation

x86/apic: fix disabling LVT0 in disconnect_bsp_APIC

PATCH Superseded Roger Pau Monne 3 participants 8 messages Jan 17, 2020 – Jan 23, 2020

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 x86/apic: fix disabling LVT0 in disconnect_bsp_APIC

The Intel SDM states:

"When an illegal vector value (0 to 15) is written to a LVT entry and the delivery mode is Fixed (bits 8-11 equal 0), the APIC may signal an illegal vector error, without regard to whether the mask bit is set or whether an interrupt is actually seen on the input."

And that's exactly what's currently done in disconnect_bsp_APIC when virt_wire_setup is true and LVT LINT0 is being masked. By writing only APIC_LVT_MASKED Xen is actually setting the vector to 0 and the delivery mode to Fixed (0), and hence it triggers an APIC error even when the LVT entry is masked.

This would usually manifest when Xen is being shut down, as that's where disconnect_bsp_APIC is called:

(XEN) APIC error on CPU0: 40(00)

Fix this by reusing the current LVT LINT0 value and just adding the mask bit to it.

---
 xen/arch/x86/apic.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
xen/arch/x86/apic.c +2 -1
index a6a7754d77..e4363639bd 100644
@@ -281,7 +281,8 @@ void disconnect_bsp_APIC(int virt_wire_setup)
         }
         else {
             /* Disable LVT0 */
-            apic_write(APIC_LVT0, APIC_LVT_MASKED);
+            value = apic_read(APIC_LVT0);
+            apic_write(APIC_LVT0, value | APIC_LVT_MASKED);
         }
 
         /* For LVT1 make it edge triggered, active high, nmi and enabled */
Signature · 7 lines
2.25.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#1 lore raw
Quoted 32 lines · Roger Pau Monne

The Intel SDM states:

"When an illegal vector value (0 to 15) is written to a LVT entry and the delivery mode is Fixed (bits 8-11 equal 0), the APIC may signal an illegal vector error, without regard to whether the mask bit is set or whether an interrupt is actually seen on the input."

And that's exactly what's currently done in disconnect_bsp_APIC when virt_wire_setup is true and LVT LINT0 is being masked. By writing only APIC_LVT_MASKED Xen is actually setting the vector to 0 and the delivery mode to Fixed (0), and hence it triggers an APIC error even when the LVT entry is masked.

This would usually manifest when Xen is being shut down, as that's where disconnect_bsp_APIC is called:

(XEN) APIC error on CPU0: 40(00)

Fix this by reusing the current LVT LINT0 value and just adding the mask bit to it.

---
 xen/arch/x86/apic.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
xen/arch/x86/apic.c +0 -0
index a6a7754d77..e4363639bd 100644

xen/arch/x86/apic.c void disconnect_bsp_APIC(int virt_wire_setup) · line 286

Quoted 7 lines
xen/arch/x86/apic.c +2 -1
@@ -281,7 +281,8 @@ void disconnect_bsp_APIC(int virt_wire_setup)
         }
         else {
             /* Disable LVT0 */
-            apic_write(APIC_LVT0, APIC_LVT_MASKED);
+            value = apic_read(APIC_LVT0);
+            apic_write(APIC_LVT0, value | APIC_LVT_MASKED);
         }

This really is ugly.  It seems that we can't write LVT0 to the same state that it has after reset/INIT.

For the code however, both halves of the if() condition do a read/modify/write.  It would be nicer to have the read and write common, with modify alone having the if().

~Andrew

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#2 lore raw
Quoted 44 lines · Andrew Cooper
Quoted 32 lines · Roger Pau Monne

The Intel SDM states:

"When an illegal vector value (0 to 15) is written to a LVT entry and the delivery mode is Fixed (bits 8-11 equal 0), the APIC may signal an illegal vector error, without regard to whether the mask bit is set or whether an interrupt is actually seen on the input."

And that's exactly what's currently done in disconnect_bsp_APIC when virt_wire_setup is true and LVT LINT0 is being masked. By writing only APIC_LVT_MASKED Xen is actually setting the vector to 0 and the delivery mode to Fixed (0), and hence it triggers an APIC error even when the LVT entry is masked.

This would usually manifest when Xen is being shut down, as that's where disconnect_bsp_APIC is called:

(XEN) APIC error on CPU0: 40(00)

Fix this by reusing the current LVT LINT0 value and just adding the mask bit to it.

---
 xen/arch/x86/apic.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
xen/arch/x86/apic.c +0 -0
index a6a7754d77..e4363639bd 100644

xen/arch/x86/apic.c void disconnect_bsp_APIC(int virt_wire_setup) · line 286

Quoted 7 lines
xen/arch/x86/apic.c +2 -1
@@ -281,7 +281,8 @@ void disconnect_bsp_APIC(int virt_wire_setup)
         }
         else {
             /* Disable LVT0 */
-            apic_write(APIC_LVT0, APIC_LVT_MASKED);
+            value = apic_read(APIC_LVT0);
+            apic_write(APIC_LVT0, value | APIC_LVT_MASKED);
         }

This really is ugly.  It seems that we can't write LVT0 to the same state that it has after reset/INIT.

Quoted 3 lines

For the code however, both halves of the if() condition do a read/modify/write.  It would be nicer to have the read and write common, with modify alone having the if().

As said on my reply to Jan, we could do the same as clear_local_APIC if that seems preferable.

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#3 lore raw
In reply to Roger Pau Monne
Quoted 7 lines · Roger Pau Monne

The Intel SDM states:

"When an illegal vector value (0 to 15) is written to a LVT entry and the delivery mode is Fixed (bits 8-11 equal 0), the APIC may signal an illegal vector error, without regard to whether the mask bit is set or whether an interrupt is actually seen on the input."

Quoted 5 lines

And that's exactly what's currently done in disconnect_bsp_APIC when virt_wire_setup is true and LVT LINT0 is being masked. By writing only APIC_LVT_MASKED Xen is actually setting the vector to 0 and the delivery mode to Fixed (0), and hence it triggers an APIC error even when the LVT entry is masked.

But there are many more instances where we (have a risk to) do so, most notably in clear_local_APIC(). The two step logic there is anyway somewhat in conflict with the citation above.

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#4 lore raw
Quoted 14 lines · Jan Beulich
Quoted 7 lines · Roger Pau Monne

The Intel SDM states:

"When an illegal vector value (0 to 15) is written to a LVT entry and the delivery mode is Fixed (bits 8-11 equal 0), the APIC may signal an illegal vector error, without regard to whether the mask bit is set or whether an interrupt is actually seen on the input."

Quoted 5 lines

And that's exactly what's currently done in disconnect_bsp_APIC when virt_wire_setup is true and LVT LINT0 is being masked. By writing only APIC_LVT_MASKED Xen is actually setting the vector to 0 and the delivery mode to Fixed (0), and hence it triggers an APIC error even when the LVT entry is masked.

Quoted 3 lines

But there are many more instances where we (have a risk to) do so, most notably in clear_local_APIC(). The two step logic there is anyway somewhat in conflict with the citation above.

clear_local_APIC masks the error vector before doing any write, and clears ESR afterwards, there's a comment at the top:

"Masking an LVT entry on a P6 can trigger a local APIC error if the vector is zero. Mask LVTERR first to prevent this."

We could do the same (ie: mask LVTERR first and clear ESR afterwards) if that seems preferable. There's a maxlvt check in clear_local_APIC, but the sdm doesn't specify anyway to check if the lapic will accept a masked vector 0 write or not, so not sure whether we should replicate that check or just do it unconditionally on both disconnect_bsp_APIC and clear_local_APIC.

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#5 lore raw
Quoted 25 lines · Roger Pau Monné
Quoted 14 lines · Jan Beulich
Quoted 7 lines · Roger Pau Monne

The Intel SDM states:

"When an illegal vector value (0 to 15) is written to a LVT entry and the delivery mode is Fixed (bits 8-11 equal 0), the APIC may signal an illegal vector error, without regard to whether the mask bit is set or whether an interrupt is actually seen on the input."

Quoted 5 lines

And that's exactly what's currently done in disconnect_bsp_APIC when virt_wire_setup is true and LVT LINT0 is being masked. By writing only APIC_LVT_MASKED Xen is actually setting the vector to 0 and the delivery mode to Fixed (0), and hence it triggers an APIC error even when the LVT entry is masked.

Quoted 3 lines

But there are many more instances where we (have a risk to) do so, most notably in clear_local_APIC(). The two step logic there is anyway somewhat in conflict with the citation above.

clear_local_APIC masks the error vector before doing any write, and clears ESR afterwards, there's a comment at the top:

"Masking an LVT entry on a P6 can trigger a local APIC error if the vector is zero. Mask LVTERR first to prevent this."

Quoted 6 lines

We could do the same (ie: mask LVTERR first and clear ESR afterwards) if that seems preferable. There's a maxlvt check in clear_local_APIC, but the sdm doesn't specify anyway to check if the lapic will accept a masked vector 0 write or not, so not sure whether we should replicate that check or just do it unconditionally on both disconnect_bsp_APIC and clear_local_APIC.

I think doing it the most careful way is going to be best. I find it surprising anyway that disconnect_bsp_APIC() doesn't write LVTERR (or other LVTs except for LVT1) at all. The function looks to have a goal of putting the APIC back into the state that we found it when booting.

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#6 lore raw
Quoted 33 lines · Jan Beulich
Quoted 25 lines · Roger Pau Monné
Quoted 14 lines · Jan Beulich
Quoted 7 lines · Roger Pau Monne

The Intel SDM states:

"When an illegal vector value (0 to 15) is written to a LVT entry and the delivery mode is Fixed (bits 8-11 equal 0), the APIC may signal an illegal vector error, without regard to whether the mask bit is set or whether an interrupt is actually seen on the input."

Quoted 5 lines

And that's exactly what's currently done in disconnect_bsp_APIC when virt_wire_setup is true and LVT LINT0 is being masked. By writing only APIC_LVT_MASKED Xen is actually setting the vector to 0 and the delivery mode to Fixed (0), and hence it triggers an APIC error even when the LVT entry is masked.

Quoted 3 lines

But there are many more instances where we (have a risk to) do so, most notably in clear_local_APIC(). The two step logic there is anyway somewhat in conflict with the citation above.

clear_local_APIC masks the error vector before doing any write, and clears ESR afterwards, there's a comment at the top:

"Masking an LVT entry on a P6 can trigger a local APIC error if the vector is zero. Mask LVTERR first to prevent this."

Quoted 6 lines

We could do the same (ie: mask LVTERR first and clear ESR afterwards) if that seems preferable. There's a maxlvt check in clear_local_APIC, but the sdm doesn't specify anyway to check if the lapic will accept a masked vector 0 write or not, so not sure whether we should replicate that check or just do it unconditionally on both disconnect_bsp_APIC and clear_local_APIC.

Quoted 5 lines

I think doing it the most careful way is going to be best. I find it surprising anyway that disconnect_bsp_APIC() doesn't write LVTERR (or other LVTs except for LVT1) at all. The function looks to have a goal of putting the APIC back into the state that we found it when booting.

Maybe it would be better to just call clear_local_APIC before trying to configure LVT{0/1}, which will leave LVT0 in a reset state and thus no write would be required in the !virt_wire_setup case?

Roger.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#7 lore raw
Quoted 40 lines · Roger Pau Monné
Quoted 33 lines · Jan Beulich
Quoted 25 lines · Roger Pau Monné
Quoted 14 lines · Jan Beulich
Quoted 7 lines · Roger Pau Monne

The Intel SDM states:

"When an illegal vector value (0 to 15) is written to a LVT entry and the delivery mode is Fixed (bits 8-11 equal 0), the APIC may signal an illegal vector error, without regard to whether the mask bit is set or whether an interrupt is actually seen on the input."

Quoted 5 lines

And that's exactly what's currently done in disconnect_bsp_APIC when virt_wire_setup is true and LVT LINT0 is being masked. By writing only APIC_LVT_MASKED Xen is actually setting the vector to 0 and the delivery mode to Fixed (0), and hence it triggers an APIC error even when the LVT entry is masked.

Quoted 3 lines

But there are many more instances where we (have a risk to) do so, most notably in clear_local_APIC(). The two step logic there is anyway somewhat in conflict with the citation above.

clear_local_APIC masks the error vector before doing any write, and clears ESR afterwards, there's a comment at the top:

"Masking an LVT entry on a P6 can trigger a local APIC error if the vector is zero. Mask LVTERR first to prevent this."

Quoted 6 lines

We could do the same (ie: mask LVTERR first and clear ESR afterwards) if that seems preferable. There's a maxlvt check in clear_local_APIC, but the sdm doesn't specify anyway to check if the lapic will accept a masked vector 0 write or not, so not sure whether we should replicate that check or just do it unconditionally on both disconnect_bsp_APIC and clear_local_APIC.

Quoted 5 lines

I think doing it the most careful way is going to be best. I find it surprising anyway that disconnect_bsp_APIC() doesn't write LVTERR (or other LVTs except for LVT1) at all. The function looks to have a goal of putting the APIC back into the state that we found it when booting.

Quoted 3 lines

Maybe it would be better to just call clear_local_APIC before trying to configure LVT{0/1}, which will leave LVT0 in a reset state and thus no write would be required in the !virt_wire_setup case?

Half of me was implying this as on option from the earlier reply. The other half was thinking that this would be quite a lot of behavioral change in one step. But since you think so too, why don't we give this a try?

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel