xen/arm: vGICv3: Emulate properly 32-bit access on GICR_PENDBASER
PATCH v3 1/1 xen/arm: vGICv3: Emulate properly 32-bit access on GICR_PENDBASER
If a guest is running in 32 bit mode and it tries to access "GICR_PENDBASER + 4" mmio reg, it will be trapped to Xen. vreg_reg64_extract() will return the value stored "v->arch.vgic.rdist_pendbase + 4". This will be stored in a 64bit cpu register.
So now we have the top 32 bits of GICR_PENDBASER (a 64 bit MMIO register) stored in the lower 32 bits of the 64bit cpu register.
This 64bit cpu register is then modified bitwise with a mask (ie GICR_PENDBASER_PTZ, it clears the 62nd bit). But the PTZ (which is bit 30 in the 64 bit cpu register) is not cleared as expected by the specification.
The correct thing to do here is to store the value of "v->arch.vgic.rdist_pendbase" in a temporary 64 bit variable. This variable is then modified bitwise with GICR_PENDBASER_PTZ mask. It is then passed to vreg_reg64_extract() which will extract 32 bits from the given offset.
---
Changes from:-
v1 - 1. Extracted this fix from "[RFC PATCH v1 05/12] Arm: GICv3: Emulate GICR_PENDBASER and GICR_PROPBASER on AArch32" into a separate patch with an appropriate commit message.
v2 - 1. Removed spin_lock_irqsave(). Used read_atomic() to read v->arch.vgic.rdist_pendbase in an atomic context.
2. Rectified the commit message to state that the cpu register is 64 bit. (because currently, GICv3 is supported on Arm64 only). Reworded to make it clear.
xen/arch/arm/vgic-v3.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
xen/arch/arm/vgic-v3.c +5 -5
@@ -249,16 +249,16 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,case VREG64 (GICR_PENDBASER ): { -unsigned long flags; +uint64_t val;if ( !v->domain->arch.vgic.has_its ) goto read_as_zero_64;if ( !vgic_reg64_check_access (dabt) ) goto bad_width; -spin_lock_irqsave (&v->arch.vgic.lock, flags); - *r =vreg_reg64_extract (v->arch.vgic.rdist_pendbase, info); - *r &= ~GICR_PENDBASER_PTZ ;/* WO, reads as 0 */ -spin_unlock_irqrestore (&v->arch.vgic.lock, flags); + val =read_atomic (&v->arch.vgic.rdist_pendbase); + val = v->arch.vgic.rdist_pendbase; + val &= ~GICR_PENDBASER_PTZ ;/* WO, reads as 0 */ + *r =vreg_reg64_extract (val, info);return 1 ; }
Signature · 1 line
2.17.1
Hi Ayan,
Quoted 17 lines · Ayan Kumar Halder
If a guest is running in 32 bit mode and it tries to access "
GICR_PENDBASER+ 4" mmio reg, it will be trapped to Xen.vreg_reg64_extract()will return the value stored "v->arch.vgic.rdist_pendbase+ 4". This will be stored in a 64bit cpu register.So now we have the top 32 bits of
GICR_PENDBASER(a 64 bit MMIO register) stored in the lower 32 bits of the 64bit cpu register.This 64bit cpu register is then modified bitwise with a mask (ie
GICR_PENDBASER_PTZ, it clears the 62nd bit). But the PTZ (which is bit 30 in the 64 bit cpu register) is not cleared as expected by the specification.The correct thing to do here is to store the value of "
v->arch.vgic.rdist_pendbase" in a temporary 64 bit variable. This variable is then modified bitwise withGICR_PENDBASER_PTZmask. It is then passed tovreg_reg64_extract()which will extract 32 bits from the given offset.
Quoted 4 lines
Fixes fe7fa1332dabd9ce4 ("ARM: vGICv3: handle virtual LPI pending and property tables")Signed-off-by Ayan Kumar Halder <[email protected]>Reviewed-by Bertrand Marquis <[email protected]>Reviewed-by Andre Przywara <[email protected]>
Given the changes you made below, the reviewed-by tags below should not have been retained.
Quoted 11 lines
Release-acked-by Henry Wang <[email protected]>---
Changes from:-
v1 - 1. Extracted this fix from "[RFC PATCH v1 05/12] Arm: GICv3: Emulate
GICR_PENDBASERandGICR_PROPBASERon AArch32" into a separate patch with an appropriate commit message.v2 - 1. Removed
spin_lock_irqsave(). Usedread_atomic()to readv->arch.vgic.rdist_pendbasein an atomic context.
Please in the commit message why the lock is removed. But...
Quoted 17 lines
2. Rectified the commit message to state that the cpu register is 64 bit. (because currently, GICv3 is supported on Arm64 only). Reworded to make it clear.
xen/arch/arm/vgic-v3.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)xen/arch/arm/vgic-v3.c +1
-1@@ -249,16 +249,16 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,case VREG64 (GICR_PENDBASER ): { -unsigned long flags; +uint64_t val;
xen/arch/arm/vgic-v3.c static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info, · line 258
Quoted 10 lines
xen/arch/arm/vgic-v3.c +1
-4if ( !v->domain->arch.vgic.has_its ) goto read_as_zero_64;if ( !vgic_reg64_check_access (dabt) ) goto bad_width; -spin_lock_irqsave (&v->arch.vgic.lock, flags); - *r =vreg_reg64_extract (v->arch.vgic.rdist_pendbase, info); - *r &= ~GICR_PENDBASER_PTZ ;/* WO, reads as 0 */ -spin_unlock_irqrestore (&v->arch.vgic.lock, flags); + val =read_atomic (&v->arch.vgic.rdist_pendbase);
... you also need to ensure that the writers are atomically setting rdist_pendbase. Please correct if I am wrong, but the callers are not using write_atomic(). So how does that work?
Cheers,
Signature · 1 line
Julien Grall
Quoted 1 line · Julien Grall
Hi Ayan,
Hi Julien,
Quoted 19 lines · Ayan Kumar Halder
Quoted 17 lines
If a guest is running in 32 bit mode and it tries to access "
GICR_PENDBASER+ 4" mmio reg, it will be trapped to Xen.vreg_reg64_extract()will return the value stored "v->arch.vgic.rdist_pendbase+ 4". This will be stored in a 64bit cpu register.So now we have the top 32 bits of
GICR_PENDBASER(a 64 bit MMIO register) stored in the lower 32 bits of the 64bit cpu register.This 64bit cpu register is then modified bitwise with a mask (ie
GICR_PENDBASER_PTZ, it clears the 62nd bit). But the PTZ (which is bit 30 in the 64 bit cpu register) is not cleared as expected by the specification.The correct thing to do here is to store the value of "
v->arch.vgic.rdist_pendbase" in a temporary 64 bit variable. This variable is then modified bitwise withGICR_PENDBASER_PTZmask. It is then passed tovreg_reg64_extract()which will extract 32 bits from the given offset.
Quoted 6 lines
Quoted 4 lines
Fixes fe7fa1332dabd9ce4 ("ARM: vGICv3: handle virtual LPI pending and property tables")Signed-off-by Ayan Kumar Halder <[email protected]>Reviewed-by Bertrand Marquis <[email protected]>Reviewed-by Andre Przywara <[email protected]>Given the changes you made below, the reviewed-by tags below should not have been retained.
Sorry, I will take care of this henceforth.
Quoted 6 lines
Quoted 6 lines
Release-acked-by Henry Wang <[email protected]>---
Changes from:-
Quoted 8 lines
Quoted 6 lines
v1 - 1. Extracted this fix from "[RFC PATCH v1 05/12] Arm: GICv3: Emulate
GICR_PENDBASERandGICR_PROPBASERon AArch32" into a separate patch with an appropriate commit message.v2 - 1. Removed
spin_lock_irqsave(). Usedread_atomic()to readv->arch.vgic.rdist_pendbasein an atomic context.Please in the commit message why the lock is removed. But...
ok
Quoted 20 lines
Quoted 20 lines
2. Rectified the commit message to state that the cpu register is 64 bit. (because currently, GICv3 is supported on Arm64 only). Reworded to make it clear.
xen/arch/arm/vgic-v3.c| 10 +++++-----1 file changed, 5 insertions(+), 5 deletions(-)xen/arch/arm/vgic-v3.c +0
-0@@ -249,16 +249,16 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,case
VREG64(GICR_PENDBASER):
{
- unsigned long flags;
+uint64_tval;
if ( !v->domain->arch.vgic.has_its )
gotoread_as_zero_64;
Quoted 8 lines
Quoted 6 lines
if ( !vgic_reg64_check_access(dabt) ) goto
bad_width; -spin_lock_irqsave(&v->arch.vgic.lock, flags); - *r =vreg_reg64_extract(v->arch.vgic.rdist_pendbase, info); - *r &= ~GICR_PENDBASER_PTZ; /* WO, reads as 0 */ -spin_unlock_irqrestore(&v->arch.vgic.lock, flags); + val =read_atomic(&v->arch.vgic.rdist_pendbase);... you also need to ensure that the writers are atomically setting
rdist_pendbase. Please correct if I am wrong, but the callers are not usingwrite_atomic(). So how does that work?
I think read_atomic()/write_atomic() may not be the correct approach for the following reasons :-
1. __vgic_v3_rdistr_rd_mmio_read is a static function. So 'val' has a global lifetime. Thus, all the following three lines need to be protected from concurrent access.
val = read_atomic(&v->arch.vgic.rdist_pendbase); val &= ~GICR_PENDBASER_PTZ; /* WO, reads as 0 */
/* If a context switch happens here, then the 'val' below may potentially be incorrect. */
*r = vreg_reg64_extract(val, info);
2. The same holds true for 'reg' as well in __vgic_v3_rdistr_rd_mmio_write()
reg = v->arch.vgic.rdist_pendbase;
blah, blah
v->arch.vgic.rdist_pendbase = reg;
Thus, I am thinking of going back to spin_lock_irqsave()/spin_unlock_irqrestore(). Unless, you have some other opinions.
- Ayan
Quoted 3 lines
Cheers,
Quoted 6 lines · Ayan Kumar Halder
Quoted 2 lines
... you also need to ensure that the writers are atomically setting
rdist_pendbase. Please correct if I am wrong, but the callers are not usingwrite_atomic(). So how does that work?I think read_atomic()/write_atomic() may not be the correct approach for the following reasons :-
1.
__vgic_v3_rdistr_rd_mmio_readis a static function. So 'val' has a global lifetime. Thus, all the following three lines need to be protected from concurrent access.
I don't understand this argument. 'static' means the function is not exported. The local variables will still reside on the stack.
So why does the use of 'val' needs to be protected with the lock?
Quoted 7 lines
val =
read_atomic(&v->arch.vgic.rdist_pendbase); val &= ~GICR_PENDBASER_PTZ; /* WO, reads as 0 *//* If a context switch happens here, then the 'val' below may
potentially be incorrect. */
Quoted 7 lines
*r =
vreg_reg64_extract(val, info);2. The same holds true for 'reg' as well in
__vgic_v3_rdistr_rd_mmio_write()reg =
v->arch.vgic.rdist_pendbase;
blah, blah
v->arch.vgic.rdist_pendbase= reg;
Same here.
Cheers,
Signature · 1 line
Julien Grall
Hi Julien,
Quoted 6 lines · Julien Grall
Quoted 3 lines · Ayan Kumar Halder
Quoted 2 lines
... you also need to ensure that the writers are atomically setting
rdist_pendbase. Please correct if I am wrong, but the callers are not usingwrite_atomic(). So how does that work?
Quoted 7 lines
Quoted 3 lines
I think read_atomic()/write_atomic() may not be the correct approach for the following reasons :-
1.
__vgic_v3_rdistr_rd_mmio_readis a static function. So 'val' has a global lifetime. Thus, all the following three lines need to be protected from concurrent access.I don't understand this argument. 'static' means the function is not exported. The local variables will still reside on the stack.
So why does the use of 'val' needs to be protected with the lock?
Yes, you are correct. I was misunderstanding this as a static variable.
Also, I understood from Stefano that pre-emption does not occur in Xen. So there will be no context switch.
So, the only race is between __vgic_v3_rdistr_rd_mmio_read() and __vgic_v3_rdistr_rd_mmio_write() for reading/writing rdist_pendbase.
- Ayan
Quoted 19 lines
Quoted 7 lines
val =
read_atomic(&v->arch.vgic.rdist_pendbase); val &= ~GICR_PENDBASER_PTZ; /* WO, reads as 0 *//* If a context switch happens here, then the 'val' below may potentially be incorrect. */
Quoted 7 lines
*r =
vreg_reg64_extract(val, info);2. The same holds true for 'reg' as well in
__vgic_v3_rdistr_rd_mmio_write()reg =
v->arch.vgic.rdist_pendbase;
blah, blah
v->arch.vgic.rdist_pendbase= reg;Same here.
Cheers,