RISC-V: KVM: change get_reg/set_reg error code
Subsystems (4)
v4 cover RISC-V: KVM: change get_reg/set_reg error code
Hi,
This version includes a diff that Andrew mentioned in v2 [1] that I missed. They were squashed into patch 1.
No other changes made. Patches rebased on top of riscv_kvm_queue.
Changes from v3:
- patch 1:
- added missing EINVAL - ENOENT conversions
- v3 link: https://lore.kernel.org/kvm/20230803140022.399333-1-dbarboza at ventanamicro.com/
[1] https://lore.kernel.org/kvm/20230801222629.210929-1-dbarboza at ventanamicro.com/
Andrew Jones (1):
RISC-V: KVM: Improve vector save/restore errors
Daniel Henrique Barboza (9):
RISC-V: KVM: return ENOENT in *_one_reg() when reg is unknown
RISC-V: KVM: use ENOENT in *_one_reg() when extension is unavailable
RISC-V: KVM: do not EOPNOTSUPP in set_one_reg() zicbo(m|z)
RISC-V: KVM: do not EOPNOTSUPP in set KVM_REG_RISCV_TIMER_REG
RISC-V: KVM: use EBUSY when !vcpu->arch.ran_atleast_once
RISC-V: KVM: avoid EBUSY when writing same ISA val
RISC-V: KVM: avoid EBUSY when writing the same machine ID val
RISC-V: KVM: avoid EBUSY when writing the same isa_ext val
docs: kvm: riscv: document EBUSY in KVM_SET_ONE_REG
Documentation/virt/kvm/api.rst | 2 + arch/riscv/kvm/aia.c | 4 +- arch/riscv/kvm/vcpu_fp.c | 12 +++--- arch/riscv/kvm/vcpu_onereg.c | 74 ++++++++++++++++++++++------------ arch/riscv/kvm/vcpu_sbi.c | 16 ++++---- arch/riscv/kvm/vcpu_timer.c | 11 ++--- arch/riscv/kvm/vcpu_vector.c | 60 ++++++++++++++------------- 7 files changed, 107 insertions(+), 72 deletions(-)
Signature · 1 line
2.41.0
PATCH v4 1/10 RISC-V: KVM: return ENOENT in *_one_reg() when reg is unknown
get_one_reg() and set_one_reg() are returning EINVAL errors for almost
everything: if a reg doesn't exist, if a reg ID is malformatted, if the associated CPU extension that implements the reg isn't present in the host, and for set_one_reg() if the value being written is invalid.
This isn't wrong according to the existing KVM API docs (EINVAL can be used when there's no such register) but adding more ENOENT instances will make easier for userspace to understand what went wrong.
Existing userspaces can be affected by this error code change. We checked a few. As of current upstream code, crosvm doesn't check for any particular errno code when using kvm_(get|set)_one_reg(). Neither does QEMU. rust-vmm doesn't have kvm-riscv support yet. Thus we have a good chance of changing these error codes now while the KVM RISC-V ecosystem is still new, minimizing user impact.
Change all get_one_reg() and set_one_reg() implementations to return -ENOENT at all "no such register" cases.
--- arch/riscv/kvm/aia.c | 4 ++-- arch/riscv/kvm/vcpu_fp.c | 12 ++++++------ arch/riscv/kvm/vcpu_onereg.c | 36 ++++++++++++++++++------------------ arch/riscv/kvm/vcpu_sbi.c | 16 +++++++++------- arch/riscv/kvm/vcpu_timer.c | 8 ++++---- 5 files changed, 39 insertions(+), 37 deletions(-)
arch/riscv/kvm/aia.c +2 -2
@@ -176,7 +176,7 @@ int kvm_riscv_vcpu_aia_get_csr(struct kvm_vcpu *vcpu,struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;if (reg_num >=sizeof (struct kvm_riscv_aia_csr ) /sizeof (unsigned long )) -return -EINVAL ; +return -ENOENT ; *out_val =0 ;if (kvm_riscv_aia_available ())
@@ -192,7 +192,7 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;if (reg_num >=sizeof (struct kvm_riscv_aia_csr ) /sizeof (unsigned long )) -return -EINVAL ; +return -ENOENT ;if (kvm_riscv_aia_available ()) { ((unsigned long *)csr)[reg_num] = val;
arch/riscv/kvm/vcpu_fp.c +6 -6
@@ -96,7 +96,7 @@ int kvm_riscv_vcpu_get_reg_fp(struct kvm_vcpu *vcpu, reg_num <= KVM_REG_RISCV_FP_F_REG(f[31])) reg_val = &cntx->fp.f.f[reg_num]; else - return -EINVAL; + return -ENOENT; }else if ((rtype ==KVM_REG_RISCV_FP_D ) &&riscv_isa_extension_available (vcpu->arch.isa, d)) {if (reg_num ==KVM_REG_RISCV_FP_D_REG (fcsr)) {
@@ -109,9 +109,9 @@ int kvm_riscv_vcpu_get_reg_fp(struct kvm_vcpu *vcpu,return -EINVAL ; reg_val = &cntx->fp.d.f[reg_num]; }else -return -EINVAL ; +return -ENOENT ; }else -return -EINVAL ; +return -ENOENT ;if (copy_to_user (uaddr, reg_val,KVM_REG_SIZE (reg->id)))return -EFAULT ;
@@ -141,7 +141,7 @@ int kvm_riscv_vcpu_set_reg_fp(struct kvm_vcpu *vcpu, reg_num <= KVM_REG_RISCV_FP_F_REG(f[31])) reg_val = &cntx->fp.f.f[reg_num]; else - return -EINVAL; + return -ENOENT; }else if ((rtype ==KVM_REG_RISCV_FP_D ) &&riscv_isa_extension_available (vcpu->arch.isa, d)) {if (reg_num ==KVM_REG_RISCV_FP_D_REG (fcsr)) {
@@ -154,9 +154,9 @@ int kvm_riscv_vcpu_set_reg_fp(struct kvm_vcpu *vcpu,return -EINVAL ; reg_val = &cntx->fp.d.f[reg_num]; }else -return -EINVAL ; +return -ENOENT ; }else -return -EINVAL ; +return -ENOENT ;if (copy_from_user (reg_val, uaddr,KVM_REG_SIZE (reg->id)))return -EFAULT ;
arch/riscv/kvm/vcpu_onereg.c +18 -18
@@ -156,7 +156,7 @@ static int kvm_riscv_vcpu_get_reg_config(struct kvm_vcpu *vcpu, reg_val = satp_mode >>SATP_MODE_SHIFT ;break ;default : -return -EINVAL ; +return -ENOENT ; }if (copy_to_user (uaddr, ®_val,KVM_REG_SIZE (reg->id)))
@@ -242,7 +242,7 @@ static int kvm_riscv_vcpu_set_reg_config(struct kvm_vcpu *vcpu,return -EINVAL ;break ;default : -return -EINVAL ; +return -ENOENT ; }return 0 ;
@@ -262,7 +262,7 @@ static int kvm_riscv_vcpu_get_reg_core(struct kvm_vcpu *vcpu,if (KVM_REG_SIZE (reg->id) !=sizeof (unsigned long ))return -EINVAL ;if (reg_num >=sizeof (struct kvm_riscv_core ) /sizeof (unsigned long )) -return -EINVAL ; +return -ENOENT ;if (reg_num ==KVM_REG_RISCV_CORE_REG (regs.pc)) reg_val = cntx->sepc;
@@ -273,7 +273,7 @@ static int kvm_riscv_vcpu_get_reg_core(struct kvm_vcpu *vcpu, reg_val = (cntx->sstatus &SR_SPP ) ?KVM_RISCV_MODE_S :KVM_RISCV_MODE_U ; else - return -EINVAL; + return -ENOENT;if (copy_to_user (uaddr, ®_val,KVM_REG_SIZE (reg->id)))return -EFAULT ;
@@ -295,7 +295,7 @@ static int kvm_riscv_vcpu_set_reg_core(struct kvm_vcpu *vcpu,if (KVM_REG_SIZE (reg->id) !=sizeof (unsigned long ))return -EINVAL ;if (reg_num >=sizeof (struct kvm_riscv_core ) /sizeof (unsigned long )) -return -EINVAL ; +return -ENOENT ;if (copy_from_user (®_val, uaddr,KVM_REG_SIZE (reg->id)))return -EFAULT ;
@@ -311,7 +311,7 @@ static int kvm_riscv_vcpu_set_reg_core(struct kvm_vcpu *vcpu,else cntx->sstatus &= ~SR_SPP ; }else -return -EINVAL ; +return -ENOENT ;return 0 ; }
@@ -323,7 +323,7 @@ static int kvm_riscv_vcpu_general_get_csr(struct kvm_vcpu *vcpu,struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;if (reg_num >=sizeof (struct kvm_riscv_csr ) /sizeof (unsigned long )) -return -EINVAL ; +return -ENOENT ;if (reg_num ==KVM_REG_RISCV_CSR_REG (sip)) {kvm_riscv_vcpu_flush_interrupts (vcpu);
@@ -342,7 +342,7 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;if (reg_num >=sizeof (struct kvm_riscv_csr ) /sizeof (unsigned long )) -return -EINVAL ; +return -ENOENT ;if (reg_num ==KVM_REG_RISCV_CSR_REG (sip)) { reg_val &=VSIP_VALID_MASK ;
@@ -381,7 +381,7 @@ static int kvm_riscv_vcpu_get_reg_csr(struct kvm_vcpu *vcpu, rc =kvm_riscv_vcpu_aia_get_csr (vcpu, reg_num, ®_val);break ;default : - rc = -EINVAL ; + rc = -ENOENT ;break ; }if (rc)
@@ -420,7 +420,7 @@ static int kvm_riscv_vcpu_set_reg_csr(struct kvm_vcpu *vcpu, rc =kvm_riscv_vcpu_aia_set_csr (vcpu, reg_num, reg_val);break ;default : - rc = -EINVAL ; + rc = -ENOENT ;break ; }if (rc)
@@ -437,7 +437,7 @@ static int riscv_vcpu_get_isa_ext_single(struct kvm_vcpu *vcpu,if (reg_num >=KVM_RISCV_ISA_EXT_MAX || reg_num >=ARRAY_SIZE (kvm_isa_ext_arr)) -return -EINVAL ; +return -ENOENT ; *reg_val =0 ; host_isa_ext = kvm_isa_ext_arr[reg_num];
@@ -455,7 +455,7 @@ static int riscv_vcpu_set_isa_ext_single(struct kvm_vcpu *vcpu,if (reg_num >=KVM_RISCV_ISA_EXT_MAX || reg_num >=ARRAY_SIZE (kvm_isa_ext_arr)) -return -EINVAL ; +return -ENOENT ; host_isa_ext = kvm_isa_ext_arr[reg_num];if (!__riscv_isa_extension_available (NULL , host_isa_ext))
@@ -489,7 +489,7 @@ static int riscv_vcpu_get_isa_ext_multi(struct kvm_vcpu *vcpu,unsigned long i, ext_id, ext_val;if (reg_num >KVM_REG_RISCV_ISA_MULTI_REG_LAST ) -return -EINVAL ; +return -ENOENT ;for (i =0 ; i <BITS_PER_LONG ; i++) { ext_id = i + reg_num *BITS_PER_LONG ;
@@ -512,7 +512,7 @@ static int riscv_vcpu_set_isa_ext_multi(struct kvm_vcpu *vcpu,unsigned long i, ext_id;if (reg_num >KVM_REG_RISCV_ISA_MULTI_REG_LAST ) -return -EINVAL ; +return -ENOENT ;for_each_set_bit (i, ®_val,BITS_PER_LONG ) { ext_id = i + reg_num *BITS_PER_LONG ;
@@ -554,7 +554,7 @@ static int kvm_riscv_vcpu_get_reg_isa_ext(struct kvm_vcpu *vcpu, reg_val = ~reg_val;break ;default : - rc = -EINVAL ; + rc = -ENOENT ; }if (rc)return rc;
@@ -592,7 +592,7 @@ static int kvm_riscv_vcpu_set_reg_isa_ext(struct kvm_vcpu *vcpu,case KVM_REG_RISCV_SBI_MULTI_DIS :return riscv_vcpu_set_isa_ext_multi (vcpu, reg_num, reg_val, false);default : -return -EINVAL ; +return -ENOENT ; }return 0 ;
@@ -627,7 +627,7 @@ int kvm_riscv_vcpu_set_reg(struct kvm_vcpu *vcpu,break ; } -return -EINVAL ; +return -ENOENT ; } int kvm_riscv_vcpu_get_reg(struct kvm_vcpu *vcpu,
@@ -659,5 +659,5 @@ int kvm_riscv_vcpu_get_reg(struct kvm_vcpu *vcpu,break ; } -return -EINVAL ; +return -ENOENT ; }
arch/riscv/kvm/vcpu_sbi.c +9 -7
@@ -140,8 +140,10 @@ static int riscv_vcpu_set_sbi_ext_single(struct kvm_vcpu *vcpu,const struct kvm_riscv_sbi_extension_entry *sext =NULL ;struct kvm_vcpu_sbi_context *scontext = &vcpu->arch.sbi_context; -if (reg_num >=KVM_RISCV_SBI_EXT_MAX || - (reg_val !=1 && reg_val !=0 )) +if (reg_num >=KVM_RISCV_SBI_EXT_MAX ) +return -ENOENT ; + +if (reg_val !=1 && reg_val !=0 )return -EINVAL ;for (i =0 ; i <ARRAY_SIZE (sbi_ext); i++) {
@@ -175,7 +177,7 @@ static int riscv_vcpu_get_sbi_ext_single(struct kvm_vcpu *vcpu,struct kvm_vcpu_sbi_context *scontext = &vcpu->arch.sbi_context;if (reg_num >=KVM_RISCV_SBI_EXT_MAX ) -return -EINVAL ; +return -ENOENT ;for (i =0 ; i <ARRAY_SIZE (sbi_ext); i++) {if (sbi_ext[i].ext_idx == reg_num) {
@@ -206,7 +208,7 @@ static int riscv_vcpu_set_sbi_ext_multi(struct kvm_vcpu *vcpu,unsigned long i, ext_id;if (reg_num >KVM_REG_RISCV_SBI_MULTI_REG_LAST ) -return -EINVAL ; +return -ENOENT ;for_each_set_bit (i, ®_val,BITS_PER_LONG ) { ext_id = i + reg_num *BITS_PER_LONG ;
@@ -226,7 +228,7 @@ static int riscv_vcpu_get_sbi_ext_multi(struct kvm_vcpu *vcpu,unsigned long i, ext_id, ext_val;if (reg_num >KVM_REG_RISCV_SBI_MULTI_REG_LAST ) -return -EINVAL ; +return -ENOENT ;for (i =0 ; i <BITS_PER_LONG ; i++) { ext_id = i + reg_num *BITS_PER_LONG ;
@@ -272,7 +274,7 @@ int kvm_riscv_vcpu_set_reg_sbi_ext(struct kvm_vcpu *vcpu,case KVM_REG_RISCV_SBI_MULTI_DIS :return riscv_vcpu_set_sbi_ext_multi (vcpu, reg_num, reg_val, false);default : -return -EINVAL ; +return -ENOENT ; }return 0 ;
@@ -307,7 +309,7 @@ int kvm_riscv_vcpu_get_reg_sbi_ext(struct kvm_vcpu *vcpu, reg_val = ~reg_val;break ;default : - rc = -EINVAL ; + rc = -ENOENT ; }if (rc)return rc;
arch/riscv/kvm/vcpu_timer.c +4 -4
@@ -170,7 +170,7 @@ int kvm_riscv_vcpu_get_reg_timer(struct kvm_vcpu *vcpu,if (KVM_REG_SIZE (reg->id) !=sizeof (u64))return -EINVAL ;if (reg_num >=sizeof (struct kvm_riscv_timer ) /sizeof (u64)) -return -EINVAL ; +return -ENOENT ;switch (reg_num) {case KVM_REG_RISCV_TIMER_REG (frequency):
@@ -187,7 +187,7 @@ int kvm_riscv_vcpu_get_reg_timer(struct kvm_vcpu *vcpu,KVM_RISCV_TIMER_STATE_OFF ;break ;default : -return -EINVAL ; +return -ENOENT ; }if (copy_to_user (uaddr, ®_val,KVM_REG_SIZE (reg->id)))
@@ -211,7 +211,7 @@ int kvm_riscv_vcpu_set_reg_timer(struct kvm_vcpu *vcpu,if (KVM_REG_SIZE (reg->id) !=sizeof (u64))return -EINVAL ;if (reg_num >=sizeof (struct kvm_riscv_timer ) /sizeof (u64)) -return -EINVAL ; +return -ENOENT ;if (copy_from_user (®_val, uaddr,KVM_REG_SIZE (reg->id)))return -EFAULT ;
@@ -233,7 +233,7 @@ int kvm_riscv_vcpu_set_reg_timer(struct kvm_vcpu *vcpu, ret =kvm_riscv_vcpu_timer_cancel (t);break ;default : - ret = -EINVAL ; + ret = -ENOENT ;break ; }
Signature · 1 line
2.41.0
PATCH v4 2/10 RISC-V: KVM: use ENOENT in *_one_reg() when extension is unavailable
In reply to Daniel Henrique BarbozaFollowing a similar logic as the previous patch let's minimize the EINVAL usage in *_one_reg() APIs by using ENOENT when an extension that implements the reg is not available.
For consistency we're also replacing an EOPNOTSUPP instance that should be an ENOENT since it's an "extension is not available" error.
--- arch/riscv/kvm/vcpu_onereg.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
arch/riscv/kvm/vcpu_onereg.c +3 -3
@@ -135,12 +135,12 @@ static int kvm_riscv_vcpu_get_reg_config(struct kvm_vcpu *vcpu,break ;case KVM_REG_RISCV_CONFIG_REG (zicbom_block_size):if (!riscv_isa_extension_available (vcpu->arch.isa,ZICBOM )) -return -EINVAL ; +return -ENOENT ; reg_val = riscv_cbom_block_size;break ;case KVM_REG_RISCV_CONFIG_REG (zicboz_block_size):if (!riscv_isa_extension_available (vcpu->arch.isa,ZICBOZ )) -return -EINVAL ; +return -ENOENT ; reg_val = riscv_cboz_block_size;break ;case KVM_REG_RISCV_CONFIG_REG (mvendorid):
@@ -459,7 +459,7 @@ static int riscv_vcpu_set_isa_ext_single(struct kvm_vcpu *vcpu, host_isa_ext = kvm_isa_ext_arr[reg_num];if (!__riscv_isa_extension_available (NULL , host_isa_ext)) -return -EOPNOTSUPP ; +return -ENOENT ;if (!vcpu->arch.ran_atleast_once) {/*
Signature · 1 line
2.41.0
PATCH v4 3/10 RISC-V: KVM: do not EOPNOTSUPP in set_one_reg() zicbo(m|z)
In reply to Daniel Henrique Barbozazicbom_block_size and zicboz_block_size have a peculiar API: they can be read via get_one_reg() but any write will return a EOPNOTSUPP.
It makes sense to return a 'not supported' error since both values can't be changed, but as far as userspace goes they're regs that are throwing the same EOPNOTSUPP error even if they were read beforehand via get_one_reg(), even if the same read value is being written back. EOPNOTSUPP is also returned even if ZICBOM/ZICBOZ aren't enabled in the host.
Change both to work more like their counterparts in get_one_reg() and return -ENOENT if their respective extensions aren't available. After that, check if the userspace is written a valid value (i.e. the host value). Throw an -EINVAL if that's not case, let it slide otherwise.
This allows both regs to be read/written by userspace in a 'lazy' manner, as long as the userspace doesn't change the reg vals.
--- arch/riscv/kvm/vcpu_onereg.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)
arch/riscv/kvm/vcpu_onereg.c +10 -2
@@ -216,9 +216,17 @@ static int kvm_riscv_vcpu_set_reg_config(struct kvm_vcpu *vcpu, }break ;case KVM_REG_RISCV_CONFIG_REG (zicbom_block_size): -return -EOPNOTSUPP ; +if (!riscv_isa_extension_available (vcpu->arch.isa,ZICBOM )) +return -ENOENT ; +if (reg_val != riscv_cbom_block_size) +return -EINVAL ; +break ;case KVM_REG_RISCV_CONFIG_REG (zicboz_block_size): -return -EOPNOTSUPP ; +if (!riscv_isa_extension_available (vcpu->arch.isa,ZICBOZ )) +return -ENOENT ; +if (reg_val != riscv_cboz_block_size) +return -EINVAL ; +break ;case KVM_REG_RISCV_CONFIG_REG (mvendorid):if (!vcpu->arch.ran_atleast_once) vcpu->arch.mvendorid = reg_val;
Signature · 1 line
2.41.0
PATCH v4 4/10 RISC-V: KVM: do not EOPNOTSUPP in set KVM_REG_RISCV_TIMER_REG
In reply to Daniel Henrique BarbozaThe KVM_REG_RISCV_TIMER_REG can be read via get_one_reg(). But trying to write anything in this reg via set_one_reg() results in an EOPNOTSUPP.
Change the API to behave like cbom_block_size: instead of always erroring out with EOPNOTSUPP, allow userspace to write the same value (riscv_timebase) back, throwing an EINVAL if a different value is attempted.
--- arch/riscv/kvm/vcpu_timer.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/riscv/kvm/vcpu_timer.c +2 -1
@@ -218,7 +218,8 @@ int kvm_riscv_vcpu_set_reg_timer(struct kvm_vcpu *vcpu,switch (reg_num) {case KVM_REG_RISCV_TIMER_REG (frequency): - ret = -EOPNOTSUPP ; +if (reg_val != riscv_timebase) +return -EINVAL ;break ;case KVM_REG_RISCV_TIMER_REG (time): gt->time_delta = reg_val -get_cycles64 ();
Signature · 1 line
2.41.0
PATCH v4 5/10 RISC-V: KVM: use EBUSY when !vcpu->arch.ran_atleast_once
In reply to Daniel Henrique Barbozavcpu_set_reg_config() and vcpu_set_reg_isa_ext() is throwing an EOPNOTSUPP error when !vcpu->arch.ran_atleast_once. In similar cases we're throwing an EBUSY error, like in mvendorid/marchid/mimpid set_reg().
EOPNOTSUPP has a conotation of finality. EBUSY is more adequate in this case since its a condition/error related to the vcpu lifecycle.
Change these EOPNOTSUPP instances to EBUSY.
--- arch/riscv/kvm/vcpu_onereg.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/riscv/kvm/vcpu_onereg.c +2 -2
@@ -212,7 +212,7 @@ static int kvm_riscv_vcpu_set_reg_config(struct kvm_vcpu *vcpu, vcpu->arch.isa[0 ] = reg_val;kvm_riscv_vcpu_fp_reset (vcpu); }else { -return -EOPNOTSUPP ; +return -EBUSY ; }break ;case KVM_REG_RISCV_CONFIG_REG (zicbom_block_size):
@@ -484,7 +484,7 @@ static int riscv_vcpu_set_isa_ext_single(struct kvm_vcpu *vcpu,return -EINVAL ;kvm_riscv_vcpu_fp_reset (vcpu); }else { -return -EOPNOTSUPP ; +return -EBUSY ; }return 0 ;
Signature · 1 line
2.41.0
PATCH v4 6/10 RISC-V: KVM: avoid EBUSY when writing same ISA val
In reply to Daniel Henrique Barbozakvm_riscv_vcpu_set_reg_config() will return -EBUSY if the ISA config reg is being written after the VCPU ran at least once.
The same restriction isn't placed in kvm_riscv_vcpu_get_reg_config(), so there's a chance that we'll -EBUSY out on an ISA config reg write even if the userspace intended no changes to it.
We'll allow the same form of 'lazy writing' that registers such as zicbom/zicboz_block_size supports: avoid erroring out if userspace made no changes to the ISA config reg.
--- arch/riscv/kvm/vcpu_onereg.c | 7 +++++++ 1 file changed, 7 insertions(+)
arch/riscv/kvm/vcpu_onereg.c +7 -0
@@ -190,6 +190,13 @@ static int kvm_riscv_vcpu_set_reg_config(struct kvm_vcpu *vcpu,if (fls (reg_val) >=RISCV_ISA_EXT_BASE )return -EINVAL ; +/* +* Return early (i.e. do nothing) if reg_val is the same +* value retrievable via kvm_riscv_vcpu_get_reg_config(). +*/ +if (reg_val == (vcpu->arch.isa[0 ] &KVM_RISCV_BASE_ISA_MASK )) +break ; +if (!vcpu->arch.ran_atleast_once) {/* Ignore the enable/disable request for certain extensions */ for (i =0 ; i <RISCV_ISA_EXT_BASE ; i++) {
Signature · 1 line
2.41.0
PATCH v4 7/10 RISC-V: KVM: avoid EBUSY when writing the same machine ID val
In reply to Daniel Henrique BarbozaRight now we do not allow any write in mvendorid/marchid/mimpid if the vcpu already started, preventing these regs to be changed.
However, if userspace doesn't change them, an alternative is to consider the reg write a no-op and avoid erroring out altogether. Userpace can then be oblivious about KVM internals if no changes were intended in the first place.
Allow the same form of 'lazy writing' that registers such as zicbom/zicboz_block_size supports: avoid erroring out if userspace makes no changes in mvendorid/marchid/mimpid during reg write.
--- arch/riscv/kvm/vcpu_onereg.c | 6 ++++++ 1 file changed, 6 insertions(+)
arch/riscv/kvm/vcpu_onereg.c +6 -0
@@ -235,18 +235,24 @@ static int kvm_riscv_vcpu_set_reg_config(struct kvm_vcpu *vcpu,return -EINVAL ;break ;case KVM_REG_RISCV_CONFIG_REG (mvendorid): +if (reg_val == vcpu->arch.mvendorid) +break ;if (!vcpu->arch.ran_atleast_once) vcpu->arch.mvendorid = reg_val;else return -EBUSY ;break ;case KVM_REG_RISCV_CONFIG_REG (marchid): +if (reg_val == vcpu->arch.marchid) +break ;if (!vcpu->arch.ran_atleast_once) vcpu->arch.marchid = reg_val;else return -EBUSY ;break ;case KVM_REG_RISCV_CONFIG_REG (mimpid): +if (reg_val == vcpu->arch.mimpid) +break ;if (!vcpu->arch.ran_atleast_once) vcpu->arch.mimpid = reg_val;else
Signature · 1 line
2.41.0
PATCH v4 8/10 RISC-V: KVM: avoid EBUSY when writing the same isa_ext val
In reply to Daniel Henrique Barbozariscv_vcpu_set_isa_ext_single() will prevent any write of isa_ext regs if the vcpu already started spinning.
But if there's no extension state (enabled/disabled) made by the userspace, there's no need to -EBUSY out - we can treat the operation as a no-op.
zicbom/zicboz_block_size, ISA config reg and mvendorid/march/mimpid already works in a more permissive manner w.r.t userspace writes being a no-op, so let's do the same with isa_ext writes.
--- arch/riscv/kvm/vcpu_onereg.c | 3 +++ 1 file changed, 3 insertions(+)
arch/riscv/kvm/vcpu_onereg.c +3 -0
@@ -482,6 +482,9 @@ static int riscv_vcpu_set_isa_ext_single(struct kvm_vcpu *vcpu,if (!__riscv_isa_extension_available (NULL , host_isa_ext))return -ENOENT ; +if (reg_val ==test_bit (host_isa_ext, vcpu->arch.isa)) +return 0 ; +if (!vcpu->arch.ran_atleast_once) {/* * All multi-letter extension and a few single letter
Signature · 1 line
2.41.0
PATCH v4 9/10 RISC-V: KVM: Improve vector save/restore errors
In reply to Daniel Henrique BarbozaFrom: Andrew Jones <[email protected]>
kvm_riscv_vcpu_(get/set)_reg_vector() now returns ENOENT if V is not available, EINVAL if reg type is not of VECTOR type, and any error that might be thrown by kvm_riscv_vcpu_vreg_addr().
--- arch/riscv/kvm/vcpu_vector.c | 60 ++++++++++++++++++++---------------- 1 file changed, 33 insertions(+), 27 deletions(-)
arch/riscv/kvm/vcpu_vector.c +33 -27
@@ -91,44 +91,44 @@ void kvm_riscv_vcpu_free_vector_context(struct kvm_vcpu *vcpu) }#endif -static void *kvm_riscv_vcpu_vreg_addr (struct kvm_vcpu *vcpu, +static int kvm_riscv_vcpu_vreg_addr (struct kvm_vcpu *vcpu,unsigned long reg_num, -size_t reg_size) +size_t reg_size, +void **reg_val) {struct kvm_cpu_context *cntx = &vcpu->arch.guest_context; -void *reg_val;size_t vlenb = riscv_v_vsize /32 ;if (reg_num <KVM_REG_RISCV_VECTOR_REG (0 )) {if (reg_size !=sizeof (unsigned long )) -return NULL ; +return -EINVAL ;switch (reg_num) {case KVM_REG_RISCV_VECTOR_CSR_REG (vstart): - reg_val = &cntx->vector.vstart; + *reg_val = &cntx->vector.vstart;break ;case KVM_REG_RISCV_VECTOR_CSR_REG (vl): - reg_val = &cntx->vector.vl; + *reg_val = &cntx->vector.vl;break ;case KVM_REG_RISCV_VECTOR_CSR_REG (vtype): - reg_val = &cntx->vector.vtype; + *reg_val = &cntx->vector.vtype;break ;case KVM_REG_RISCV_VECTOR_CSR_REG (vcsr): - reg_val = &cntx->vector.vcsr; + *reg_val = &cntx->vector.vcsr;break ;case KVM_REG_RISCV_VECTOR_CSR_REG (datap):default : -return NULL ; +return -ENOENT ; } }else if (reg_num <=KVM_REG_RISCV_VECTOR_REG (31 )) {if (reg_size != vlenb) -return NULL ; - reg_val = cntx->vector.datap +return -EINVAL ; + *reg_val = cntx->vector.datap + (reg_num -KVM_REG_RISCV_VECTOR_REG (0 )) * vlenb; }else { -return NULL ; +return -ENOENT ; } -return reg_val; +return 0 ; } int kvm_riscv_vcpu_get_reg_vector(struct kvm_vcpu *vcpu,
@@ -141,17 +141,20 @@ int kvm_riscv_vcpu_get_reg_vector(struct kvm_vcpu *vcpu,unsigned long reg_num = reg->id & ~(KVM_REG_ARCH_MASK |KVM_REG_SIZE_MASK | rtype); -void *reg_val =NULL ;size_t reg_size =KVM_REG_SIZE (reg->id); +void *reg_val; +int rc; -if (rtype ==KVM_REG_RISCV_VECTOR && -riscv_isa_extension_available (isa, v)) { - reg_val =kvm_riscv_vcpu_vreg_addr (vcpu, reg_num, reg_size); - } - -if (!reg_val) +if (rtype !=KVM_REG_RISCV_VECTOR )return -EINVAL ; +if (!riscv_isa_extension_available (isa, v)) +return -ENOENT ; + + rc =kvm_riscv_vcpu_vreg_addr (vcpu, reg_num, reg_size, ®_val); +if (rc) +return rc; +if (copy_to_user (uaddr, reg_val, reg_size))return -EFAULT ;
@@ -168,17 +171,20 @@ int kvm_riscv_vcpu_set_reg_vector(struct kvm_vcpu *vcpu,unsigned long reg_num = reg->id & ~(KVM_REG_ARCH_MASK |KVM_REG_SIZE_MASK | rtype); -void *reg_val =NULL ;size_t reg_size =KVM_REG_SIZE (reg->id); +void *reg_val; +int rc; -if (rtype ==KVM_REG_RISCV_VECTOR && -riscv_isa_extension_available (isa, v)) { - reg_val =kvm_riscv_vcpu_vreg_addr (vcpu, reg_num, reg_size); - } - -if (!reg_val) +if (rtype !=KVM_REG_RISCV_VECTOR )return -EINVAL ; +if (!riscv_isa_extension_available (isa, v)) +return -ENOENT ; + + rc =kvm_riscv_vcpu_vreg_addr (vcpu, reg_num, reg_size, ®_val); +if (rc) +return rc; +if (copy_from_user (reg_val, uaddr, reg_size))return -EFAULT ;
Signature · 1 line
2.41.0
Re: [PATCH v4 09/10] RISC-V: KVM: Improve vector save/restore errors
Quoted 118 lines · Daniel Henrique Barboza
From: Andrew Jones <[email protected]>
kvm_riscv_vcpu_(get/set)_reg_vector() now returns ENOENT if V is not available, EINVAL if reg type is not of VECTOR type, and any error that might be thrown by
kvm_riscv_vcpu_vreg_addr().Signed-off-by Andrew Jones <[email protected]>--- arch/riscv/kvm/vcpu_vector.c | 60 ++++++++++++++++++++---------------- 1 file changed, 33 insertions(+), 27 deletions(-)arch/riscv/kvm/vcpu_vector.c +26
-27@@ -91,44 +91,44 @@ void kvm_riscv_vcpu_free_vector_context(struct kvm_vcpu *vcpu) }#endif -static void *kvm_riscv_vcpu_vreg_addr (struct kvm_vcpu *vcpu, +static int kvm_riscv_vcpu_vreg_addr (struct kvm_vcpu *vcpu,unsigned long reg_num, -size_t reg_size) +size_t reg_size, +void **reg_val) {struct kvm_cpu_context *cntx = &vcpu->arch.guest_context; -void *reg_val;size_t vlenb = riscv_v_vsize /32 ;if (reg_num <KVM_REG_RISCV_VECTOR_REG (0 )) {if (reg_size !=sizeof (unsigned long )) -return NULL ; +return -EINVAL ;switch (reg_num) {case KVM_REG_RISCV_VECTOR_CSR_REG (vstart): - reg_val = &cntx->vector.vstart; + *reg_val = &cntx->vector.vstart;break ;case KVM_REG_RISCV_VECTOR_CSR_REG (vl): - reg_val = &cntx->vector.vl; + *reg_val = &cntx->vector.vl;break ;case KVM_REG_RISCV_VECTOR_CSR_REG (vtype): - reg_val = &cntx->vector.vtype; + *reg_val = &cntx->vector.vtype;break ;case KVM_REG_RISCV_VECTOR_CSR_REG (vcsr): - reg_val = &cntx->vector.vcsr; + *reg_val = &cntx->vector.vcsr;break ;case KVM_REG_RISCV_VECTOR_CSR_REG (datap):default : -return NULL ; +return -ENOENT ; } }else if (reg_num <=KVM_REG_RISCV_VECTOR_REG (31 )) {if (reg_size != vlenb) -return NULL ; - reg_val = cntx->vector.datap +return -EINVAL ; + *reg_val = cntx->vector.datap + (reg_num -KVM_REG_RISCV_VECTOR_REG (0 )) * vlenb; }else { -return NULL ; +return -ENOENT ; } -return reg_val; +return 0 ; } int kvm_riscv_vcpu_get_reg_vector(struct kvm_vcpu *vcpu,@@ -141,17 +141,20 @@ int kvm_riscv_vcpu_get_reg_vector(struct kvm_vcpu *vcpu,unsigned long reg_num = reg->id & ~(KVM_REG_ARCH_MASK |KVM_REG_SIZE_MASK | rtype); -void *reg_val =NULL ;size_t reg_size =KVM_REG_SIZE (reg->id); +void *reg_val; +int rc; -if (rtype ==KVM_REG_RISCV_VECTOR && -riscv_isa_extension_available (isa, v)) { - reg_val =kvm_riscv_vcpu_vreg_addr (vcpu, reg_num, reg_size); - } - -if (!reg_val) +if (rtype !=KVM_REG_RISCV_VECTOR )return -EINVAL ; +if (!riscv_isa_extension_available (isa, v)) +return -ENOENT ; + + rc =kvm_riscv_vcpu_vreg_addr (vcpu, reg_num, reg_size, ®_val); +if (rc) +return rc; +if (copy_to_user (uaddr, reg_val, reg_size))return -EFAULT ;@@ -168,17 +171,20 @@ int kvm_riscv_vcpu_set_reg_vector(struct kvm_vcpu *vcpu,unsigned long reg_num = reg->id & ~(KVM_REG_ARCH_MASK |KVM_REG_SIZE_MASK | rtype); -void *reg_val =NULL ;size_t reg_size =KVM_REG_SIZE (reg->id); +void *reg_val; +int rc; -if (rtype ==KVM_REG_RISCV_VECTOR && -riscv_isa_extension_available (isa, v)) { - reg_val =kvm_riscv_vcpu_vreg_addr (vcpu, reg_num, reg_size); - } - -if (!reg_val) +if (rtype !=KVM_REG_RISCV_VECTOR )return -EINVAL ;
arch/riscv/kvm/vcpu_vector.c int kvm_riscv_vcpu_set_reg_vector(struct kvm_vcpu *vcpu, · line 189
Quoted 10 lines
arch/riscv/kvm/vcpu_vector.c +7
-0+if (!riscv_isa_extension_available (isa, v)) +return -ENOENT ; + + rc =kvm_riscv_vcpu_vreg_addr (vcpu, reg_num, reg_size, ®_val); +if (rc) +return rc; +if (copy_from_user (reg_val, uaddr, reg_size))return -EFAULT ;
Ugh, this is totally wrong. We no longer set the register. I need to rework this rework...
Re: [PATCH v4 09/10] RISC-V: KVM: Improve vector save/restore errors
Quoted 120 lines · Andrew Jones
Quoted 119 lines · Daniel Henrique Barboza
From: Andrew Jones <[email protected]>
kvm_riscv_vcpu_(get/set)_reg_vector() now returns ENOENT if V is not available, EINVAL if reg type is not of VECTOR type, and any error that might be thrown by
kvm_riscv_vcpu_vreg_addr().Signed-off-by Andrew Jones <[email protected]>--- arch/riscv/kvm/vcpu_vector.c | 60 ++++++++++++++++++++---------------- 1 file changed, 33 insertions(+), 27 deletions(-)arch/riscv/kvm/vcpu_vector.c +26
-27@@ -91,44 +91,44 @@ void kvm_riscv_vcpu_free_vector_context(struct kvm_vcpu *vcpu) }#endif -static void *kvm_riscv_vcpu_vreg_addr (struct kvm_vcpu *vcpu, +static int kvm_riscv_vcpu_vreg_addr (struct kvm_vcpu *vcpu,unsigned long reg_num, -size_t reg_size) +size_t reg_size, +void **reg_val) {struct kvm_cpu_context *cntx = &vcpu->arch.guest_context; -void *reg_val;size_t vlenb = riscv_v_vsize /32 ;if (reg_num <KVM_REG_RISCV_VECTOR_REG (0 )) {if (reg_size !=sizeof (unsigned long )) -return NULL ; +return -EINVAL ;switch (reg_num) {case KVM_REG_RISCV_VECTOR_CSR_REG (vstart): - reg_val = &cntx->vector.vstart; + *reg_val = &cntx->vector.vstart;break ;case KVM_REG_RISCV_VECTOR_CSR_REG (vl): - reg_val = &cntx->vector.vl; + *reg_val = &cntx->vector.vl;break ;case KVM_REG_RISCV_VECTOR_CSR_REG (vtype): - reg_val = &cntx->vector.vtype; + *reg_val = &cntx->vector.vtype;break ;case KVM_REG_RISCV_VECTOR_CSR_REG (vcsr): - reg_val = &cntx->vector.vcsr; + *reg_val = &cntx->vector.vcsr;break ;case KVM_REG_RISCV_VECTOR_CSR_REG (datap):default : -return NULL ; +return -ENOENT ; } }else if (reg_num <=KVM_REG_RISCV_VECTOR_REG (31 )) {if (reg_size != vlenb) -return NULL ; - reg_val = cntx->vector.datap +return -EINVAL ; + *reg_val = cntx->vector.datap + (reg_num -KVM_REG_RISCV_VECTOR_REG (0 )) * vlenb; }else { -return NULL ; +return -ENOENT ; } -return reg_val; +return 0 ; } int kvm_riscv_vcpu_get_reg_vector(struct kvm_vcpu *vcpu,@@ -141,17 +141,20 @@ int kvm_riscv_vcpu_get_reg_vector(struct kvm_vcpu *vcpu,unsigned long reg_num = reg->id & ~(KVM_REG_ARCH_MASK |KVM_REG_SIZE_MASK | rtype); -void *reg_val =NULL ;size_t reg_size =KVM_REG_SIZE (reg->id); +void *reg_val; +int rc; -if (rtype ==KVM_REG_RISCV_VECTOR && -riscv_isa_extension_available (isa, v)) { - reg_val =kvm_riscv_vcpu_vreg_addr (vcpu, reg_num, reg_size); - } - -if (!reg_val) +if (rtype !=KVM_REG_RISCV_VECTOR )return -EINVAL ; +if (!riscv_isa_extension_available (isa, v)) +return -ENOENT ; + + rc =kvm_riscv_vcpu_vreg_addr (vcpu, reg_num, reg_size, ®_val); +if (rc) +return rc; +if (copy_to_user (uaddr, reg_val, reg_size))return -EFAULT ;@@ -168,17 +171,20 @@ int kvm_riscv_vcpu_set_reg_vector(struct kvm_vcpu *vcpu,unsigned long reg_num = reg->id & ~(KVM_REG_ARCH_MASK |KVM_REG_SIZE_MASK | rtype); -void *reg_val =NULL ;size_t reg_size =KVM_REG_SIZE (reg->id); +void *reg_val; +int rc; -if (rtype ==KVM_REG_RISCV_VECTOR && -riscv_isa_extension_available (isa, v)) { - reg_val =kvm_riscv_vcpu_vreg_addr (vcpu, reg_num, reg_size); - } - -if (!reg_val) +if (rtype !=KVM_REG_RISCV_VECTOR )return -EINVAL ;
Quoted 12 lines
arch/riscv/kvm/vcpu_vector.c int kvm_riscv_vcpu_set_reg_vector(struct kvm_vcpu *vcpu, · line 189
Quoted 9 lines
arch/riscv/kvm/vcpu_vector.c +7
-0+if (!riscv_isa_extension_available (isa, v)) +return -ENOENT ; + + rc =kvm_riscv_vcpu_vreg_addr (vcpu, reg_num, reg_size, ®_val); +if (rc) +return rc; +if (copy_from_user (reg_val, uaddr, reg_size))return -EFAULT ;Ugh, this is totally wrong. We no longer set the register. I need to rework this rework...
Oh, never mind. Skimming over (my own patch) I forgot we were fetching the address not the value. I should have renamed 'reg_val' to 'reg_addr'. There's another change that can be made, which is to drop rtype and its check from these functions and just use KVM_REG_RISCV_VECTOR directly in the mask. I'll send a cleanup patch with that and the rename, but that's separate from this series.
Thanks,
drew
PATCH v4 10/10 docs: kvm: riscv: document EBUSY in KVM_SET_ONE_REG
In reply to Daniel Henrique BarbozaThe EBUSY errno is being used for KVM_SET_ONE_REG as a way to tell userspace that a given reg can't be changed after the vcpu started.
--- Documentation/virt/kvm/api.rst | 2 ++ 1 file changed, 2 insertions(+)
Documentation/virt/kvm/api.rst +2 -0
@@ -2259,6 +2259,8 @@ Errors: EINVAL invalid register ID, or no such register or used with VMs in protected virtualization mode on s390 EPERM (arm64) register access not allowed before vcpu finalization + EBUSY (riscv) changing register value not allowed after the vcpu + has run at least once ====== ============================================================ (These error codes are indicative only: do not rely on a specific error
Signature · 1 line
2.41.0
Quoted 5 lines · Daniel Henrique Barboza
Hi,
This version includes a diff that Andrew mentioned in v2 [1] that I missed. They were squashed into patch 1.
Quoted 10 lines
No other changes made. Patches rebased on top of
riscv_kvm_queue.Changes from v3:
- patch 1:
- added missing EINVAL - ENOENT conversions
- v3 link: https://lore.kernel.org/kvm/20230803140022.399333-1-dbarboza at ventanamicro.com/[1] https://lore.kernel.org/kvm/20230801222629.210929-1-dbarboza at ventanamicro.com/
For the series,
Anup Patel
maintainerQuoted 19 lines · On Thu, Aug 3, 2023 at 10:03?PM Daniel Henrique Barboza…
Hi,
This version includes a diff that Andrew mentioned in v2 [1] that I missed. They were squashed into patch 1.
No other changes made. Patches rebased on top of
riscv_kvm_queue.Changes from v3:
- patch 1:
- added missing EINVAL - ENOENT conversions
- v3 link: https://lore.kernel.org/kvm/20230803140022.399333-1-dbarboza at ventanamicro.com/[1] https://lore.kernel.org/kvm/20230801222629.210929-1-dbarboza at ventanamicro.com/
Andrew Jones (1):
RISC-V: KVM: Improve vector save/restore errors
Quoted 10 lines
Daniel Henrique Barboza (9):
RISC-V: KVM: return ENOENT in*_one_reg()when reg is unknown
RISC-V: KVM: use ENOENT in*_one_reg()when extension is unavailable
RISC-V: KVM: do not EOPNOTSUPP inset_one_reg()zicbo(m|z)
RISC-V: KVM: do not EOPNOTSUPP in setKVM_REG_RISCV_TIMER_REG
RISC-V: KVM: use EBUSY when !vcpu->arch.ran_atleast_once
RISC-V: KVM: avoid EBUSY when writing same ISA val
RISC-V: KVM: avoid EBUSY when writing the same machine ID val
RISC-V: KVM: avoid EBUSY when writing the sameisa_extval
docs: kvm: riscv: document EBUSY inKVM_SET_ONE_REG
I have queued all patches except PATCH9 for Linux-6.6.
Drew, please send v5 of PATCH.
Thanks,
Anup
Quoted 13 lines
Documentation/virt/kvm/api.rst | 2 + arch/riscv/kvm/aia.c | 4 +- arch/riscv/kvm/vcpu_fp.c | 12 +++--- arch/riscv/kvm/vcpu_onereg.c | 74 ++++++++++++++++++++++------------ arch/riscv/kvm/vcpu_sbi.c | 16 ++++---- arch/riscv/kvm/vcpu_timer.c | 11 ++--- arch/riscv/kvm/vcpu_vector.c | 60 ++++++++++++++------------- 7 files changed, 107 insertions(+), 72 deletions(-)--
2.41.0
Anup Patel
maintainerQuoted 33 lines · On Fri, Aug 4, 2023 at 2:41?PM Anup Patel <[email protected]> wrote:
Quoted 19 lines · On Thu, Aug 3, 2023 at 10:03?PM Daniel Henrique Barboza…
Hi,
This version includes a diff that Andrew mentioned in v2 [1] that I missed. They were squashed into patch 1.
No other changes made. Patches rebased on top of
riscv_kvm_queue.Changes from v3:
- patch 1:
- added missing EINVAL - ENOENT conversions
- v3 link: https://lore.kernel.org/kvm/20230803140022.399333-1-dbarboza at ventanamicro.com/[1] https://lore.kernel.org/kvm/20230801222629.210929-1-dbarboza at ventanamicro.com/
Andrew Jones (1):
RISC-V: KVM: Improve vector save/restore errorsQuoted 10 lines
Daniel Henrique Barboza (9):
RISC-V: KVM: return ENOENT in*_one_reg()when reg is unknown
RISC-V: KVM: use ENOENT in*_one_reg()when extension is unavailable
RISC-V: KVM: do not EOPNOTSUPP inset_one_reg()zicbo(m|z)
RISC-V: KVM: do not EOPNOTSUPP in setKVM_REG_RISCV_TIMER_REG
RISC-V: KVM: use EBUSY when !vcpu->arch.ran_atleast_once
RISC-V: KVM: avoid EBUSY when writing same ISA val
RISC-V: KVM: avoid EBUSY when writing the same machine ID val
RISC-V: KVM: avoid EBUSY when writing the sameisa_extval
docs: kvm: riscv: document EBUSY inKVM_SET_ONE_REG
Quoted 3 lines
I have queued all patches except PATCH9 for Linux-6.6.
Drew, please send v5 of PATCH.
I have queued PATCH9 as well.
Thanks,
Anup
Quoted 17 lines
Thanks,
AnupQuoted 13 lines
Documentation/virt/kvm/api.rst | 2 + arch/riscv/kvm/aia.c | 4 +- arch/riscv/kvm/vcpu_fp.c | 12 +++--- arch/riscv/kvm/vcpu_onereg.c | 74 ++++++++++++++++++++++------------ arch/riscv/kvm/vcpu_sbi.c | 16 ++++---- arch/riscv/kvm/vcpu_timer.c | 11 ++--- arch/riscv/kvm/vcpu_vector.c | 60 ++++++++++++++------------- 7 files changed, 107 insertions(+), 72 deletions(-)--
2.41.0