riscv: implement Ssqosid extension and CBQRI controllers
Subsystems (2)
v2 cover riscv: implement Ssqosid extension and CBQRI controllers
This RFC series implements the Ssqosid extension and the sqoscfg CSR as defined in the RISC-V Capacity and Bandwidth Controller QoS Register Interface (CBQRI) specification [1]. Quality of Service (QoS) in this context is concerned with shared resources on an SoC such as cache capacity and memory bandwidth.
These patches are available as a public git branch [2].
sqoscfg CSR
-----------
The sqoscfg CSR provides a mechanism by which a software workload (e.g. a process or a set of processes) can be associated with a resource control ID (RCID) and a monitoring counter ID (MCID) that accompanies each request made by the hart to shared resources like cache.
CBQRI defines operations to configure resource usage limits, in the form of capacity or bandwidth, for an RCID. CBQRI also defines operations to configure counters to track the resource utilization per MCID.
CBQRI controllers
-----------------
This series also implements an CBQRI capacity controller and an CBQRI bandwidth controller which can be configured from the command line:
$ qemu-system-riscv64 -M virt ... \
-device riscv.cbqri.capacity,mmio_base=0x04828000[,...] \
-device riscv.cbqri.bandwidth,mmio_base=0x04829000[,...]The mmio_base option is mandatory, the others are optional.
As many -device arguments as wanted can be provided as long as their mmio regions don't conflict.
To see all possible options:
$ qemu-system-riscv64 -device riscv.cbqri.capacity,help
riscv.cbqri.capacity options:
alloc_op_config_limit=<bool> - (default: true)
alloc_op_flush_rcid=<bool> - (default: true)
alloc_op_read_limit=<bool> - (default: true)
at_code=<bool> - (default: true)
at_data=<bool> - (default: true)
max_mcids=<uint16> - (default: 256)
max_rcids=<uint16> - (default: 64)
mmio_base=<uint64> - (default: 0)
mon_evt_id_none=<bool> - (default: true)
mon_evt_id_occupancy=<bool> - (default: true)
mon_op_config_event=<bool> - (default: true)
mon_op_read_counter=<bool> - (default: true)
ncblks=<uint16> - (default: 16)
target=<str> $ qemu-system-riscv64 -device riscv.cbqri.bandwidth,help
riscv.cbqri.bandwidth options:
alloc_op_config_limit=<bool> - (default: true)
alloc_op_read_limit=<bool> - (default: true)
at_code=<bool> - (default: true)
at_data=<bool> - (default: true)
max_mcids=<uint16> - (default: 256)
max_rcids=<uint16> - (default: 64)
mmio_base=<uint64> - (default: 0)
mon_evt_id_none=<bool> - (default: true)
mon_evt_id_rdonly_count=<bool> - (default: true)
mon_evt_id_rdwr_count=<bool> - (default: true)
mon_evt_id_wronly_count=<bool> - (default: true)
mon_op_config_event=<bool> - (default: true)
mon_op_read_counter=<bool> - (default: true)
nbwblks=<uint16> - (default: 1024)
target=<str>Boolean options correspond to hardware capabilities that can be disabled
CBQRI proof-of-concept
----------------------
A hypothetical SoC with the following CBQRI controller configuration is used for the CBQRI proof-of-concept:
- L2 cache controllers
- Resource type: Capacity
- Number of capacity blocks (NCBLKS): 12
- In the context of a set-associative cache, the number of capacity blocks can be thought of as the number of ways
- Number of access types: 2 (code and data)
- Usage monitoring not supported
- Capacity allocation operations: CONFIG_LIMIT, READ_LIMIT
- Last-level cache (LLC) controller
- Resource type: Capacity
- Number of capacity blocks (NCBLKS): 16
- Number of access types: 2 (code and data)
- Usage monitoring operations: CONFIG_EVENT, READ_COUNTER
- Event IDs supported: None, Occupancy
- Capacity allocation ops: CONFIG_LIMIT, READ_LIMIT, FLUSH_RCID
- Memory controllers
- Resource type: Bandwidth
- Number of bandwidth blocks (NBWBLKS): 1024
- Bandwidth blocks do not have a unit but instead represent a portion of the total bandwidth resource. For NWBLKS of 1024, each block represents about 0.1% of the bandwidth resource.
- Maximum reserved bandwidth blocks (MRBWB): 819 [80% of NBWBLKS]
- Number of access types: 1 (no code/data differentiation)
- Usage monitoring operations: CONFIG_EVENT, READ_COUNTER
- Event IDs supported: None, Total read/write byte count, Total read byte count, Total write byte count
- Bandwidth allocation operations: CONFIG_LIMIT, READ_LIMIT
The memory map used for the proof-of-concept:
Base addr Size 0x4820000 4KB Cluster 0 L2 cache controller 0x4821000 4KB Cluster 1 L2 cache controller 0x4828000 4KB Memory controller 0 0x4829000 4KB Memory controller 1 0x482a000 4KB Memory controller 2 0x482b000 4KB Shared LLC cache controller
This configuration is meant to provide a "concrete" example for software (like Linux) to test against. It represents just one of many possible ways for hardware to implement the CBQRI spec.
The CBQRI proof-of-concept configuration is created with the following:
qemu-system-riscv64 \
-M virt \
-nographic \
-smp 8 \
-device riscv.cbqri.capacity,max_mcids=256,max_rcids=64,ncblks=12,alloc_op_flush_rcid=false,mon_op_config_event=false,mon_op_read_counter=false,mon_evt_id_none=false,mon_evt_id_occupancy=false,mmio_base=0x04820000 \
-device riscv.cbqri.capacity,max_mcids=256,max_rcids=64,ncblks=12,alloc_op_flush_rcid=false,mon_op_config_event=false,mon_op_read_counter=false,mon_evt_id_none=false,mon_evt_id_occupancy=false,mmio_base=0x04821000 \
-device riscv.cbqri.capacity,max_mcids=256,max_rcids=64,ncblks=16,mmio_base=0x0482B000 \
-device riscv.cbqri.bandwidth,max_mcids=256,max_rcids=64,nbwblks=1024,mrbwb=819,mmio_base=0x04828000 \
-device riscv.cbqri.bandwidth,max_mcids=256,max_rcids=64,nbwblks=1024,mrbwb=819,mmio_base=0x04829000 \
-device riscv.cbqri.bandwidth,max_mcids=256,max_rcids=64,nbwblks=1024,mrbwb=819,mmio_base=0x0482a000In addition, please note that this RFC series only implements the register interface that CBQRI specifies. It does not attempt to emulate the performance impact of configuring limits on shared resources like cache and memory bandwidth. Similarly, the code does not attempt to emulate cache and memory bandwidth utilization, like what would be observed on a real hardware system implementing CBQRI.
Status of CBQRI
---------------
The CBQRI spec is still in a draft state and is undergoing review [3]. It is possible there will be changes to the Ssqosid extension and the CBQRI spec. For example, the sqoscfg CSR address is not yet finalized.
The goal of this Qemu patch series, along with complimentary Linux patch series [4][5], is to satisfy the software proof of concept requirement or CBQRI to be frozen.
Future work
-----------
In the future, the device tree generation will be expanded to include CBQRI-related properties. Currently, these are added to the dumped dtb that Linux consumes for the CBQRI proof-of-concept [5].
Changes since v1
----------------
- Rebase on current master (8.0.50) instead of 8.0.0-rc4
- Configure CBQRI controllers based on device properties in command line arguments instead of a fixed example SoC configuration.
- Move TYPE_RISCV_CBQRI_BC and TYPE_RISCV_CBQRI_CC into header so that machines may use it (suggested by Alistair).
- Change 'select RISC_CBQRI' to 'imply RISCV_CBQRI' for RISCV_VIRT.
- Patches 8/9 could be dropped as they are not needed for the CBQRI proof-of-concept to work. They are only meant to serve as an example for those implementing new machines.
[1] https://github.com/riscv-non-isa/riscv-cbqri/blob/main/riscv-cbqri.pdf
[2] https://gitlab.baylibre.com/baylibre/qemu/-/tree/riscv-cbqri-rfc-v2
[3] https://lists.riscv.org/g/tech-cbqri/message/38
[4] https://lore.kernel.org/linux-riscv/[email protected]/
[5] https://lore.kernel.org/linux-riscv/[email protected]/
RFC PATCH v2 1/9 riscv: implement Ssqosid extension and sqoscfg CSR
From: Kornel Dulęba <[email protected]>
Implement the sqoscfg CSR defined by the Ssqosid ISA extension (Supervisor-mode Quality of Service ID). The CSR contains two fields:
- Resource Control ID (RCID) used determine resource allocation
- Monitoring Counter ID (MCID) used to track resource usage
The CSR is defined for S-mode but accessing it when V=1 shall cause a virtual instruction exception. Implement this behavior by calling the hmode predicate.
---
Changes since v1:
- rebase on current master (v8.0.50) instead of 8.0.0-rc4
disas/riscv.c | 1 + target/riscv/cpu.c | 2 ++ target/riscv/cpu.h | 3 +++ target/riscv/cpu_bits.h | 5 +++++ target/riscv/csr.c | 34 ++++++++++++++++++++++++++++++++++ 5 files changed, 45 insertions(+)
disas/riscv.c +1 -0
@@ -2100,6 +2100,7 @@ static const char *csr_name(int csrno)case 0x0143 :return "stval" ;case 0x0144 :return "sip" ;case 0x0180 :return "satp" ; +case 0x0181 :return "sqoscfg" ;case 0x0200 :return "hstatus" ;case 0x0202 :return "hedeleg" ;case 0x0203 :return "hideleg" ;
target/riscv/cpu.c +2 -0
@@ -114,6 +114,7 @@ static const struct isa_ext_data isa_edata_arr[] = {ISA_EXT_DATA_ENTRY (smaia, true,PRIV_VERSION_1_12_0 , ext_smaia),ISA_EXT_DATA_ENTRY (ssaia, true,PRIV_VERSION_1_12_0 , ext_ssaia),ISA_EXT_DATA_ENTRY (sscofpmf, true,PRIV_VERSION_1_12_0 , ext_sscofpmf), +ISA_EXT_DATA_ENTRY (ssqosid, true,PRIV_VERSION_1_12_0 , ext_ssqosid),ISA_EXT_DATA_ENTRY (sstc, true,PRIV_VERSION_1_12_0 , ext_sstc),ISA_EXT_DATA_ENTRY (svadu, true,PRIV_VERSION_1_12_0 , ext_svadu), ISA_EXT_DATA_ENTRY(svinval, true, PRIV_VERSION_1_12_0, ext_svinval),
@@ -1397,6 +1398,7 @@ static Property riscv_cpu_extensions[] = {DEFINE_PROP_BOOL ("svadu" ,RISCVCPU , cfg.ext_svadu, true), +DEFINE_PROP_BOOL ("ssqosid" ,RISCVCPU , cfg.ext_ssqosid, true),DEFINE_PROP_BOOL ("svinval" ,RISCVCPU , cfg.ext_svinval, false),DEFINE_PROP_BOOL ("svnapot" ,RISCVCPU , cfg.ext_svnapot, false), DEFINE_PROP_BOOL("svpbmt", RISCVCPU, cfg.ext_svpbmt, false),
target/riscv/cpu.h +3 -0
@@ -222,6 +222,8 @@ struct CPUArchState {target_ulong mcause;target_ulong mtval;/* since: priv-1.10.0 */ +target_ulong sqoscfg; +/* Machine and Supervisor interrupt priorities */ uint8_t miprio[64 ];uint8_t siprio[64 ];
@@ -454,6 +456,7 @@ struct RISCVCPUConfig {bool ext_icboz;bool ext_zicond;bool ext_zihintpause; +bool ext_ssqosid;bool ext_smstateen;bool ext_sstc;bool ext_svadu;
target/riscv/cpu_bits.h +5 -0
@@ -217,6 +217,7 @@/* Supervisor Protection and Translation */ #define CSR_SPTBR 0x180#define CSR_SATP 0x180 +#define CSR_SQOSCFG 0x181/* Supervisor-Level Window to Indirectly Accessed Registers (AIA) */ #define CSR_SISELECT 0x150
@@ -898,4 +899,8 @@ typedef enum RISCVException {#define MHPMEVENT_IDX_MASK 0xFFFFF#define MHPMEVENT_SSCOF_RESVD 16 +/* SQOSCFG BITS (QOSID) */ +#define SQOSCFG_RCID 0x00000FFF +#define SQOSCFG_MCID 0x0FFF0000 +#endif
target/riscv/csr.c +34 -0
@@ -2700,6 +2700,37 @@ static RISCVException write_satp(CPURISCVState *env, int csrno,return RISCV_EXCP_NONE ; } +static RISCVException check_sqoscfg (CPURISCVState *env,int csrno) +{ +RISCVCPU *cpu =env_archcpu (env); + +if (!cpu->cfg.ext_ssqosid) { +return RISCV_EXCP_ILLEGAL_INST ; + } + +/* +* Even though this is an S-mode CSR the spec says that we need to throw +* and virt instruction fault if a guest tries to access it. +*/ +return hmode (env, csrno); +} + +static RISCVException read_sqoscfg (CPURISCVState *env,int csrno, +target_ulong *val) +{ + *val = env->sqoscfg; +return RISCV_EXCP_NONE ; +} + +static RISCVException write_sqoscfg (CPURISCVState *env,int csrno, +target_ulong val) +{ + env->sqoscfg = val & (SQOSCFG_RCID |SQOSCFG_MCID ); +return RISCV_EXCP_NONE ; +} + + +static int read_vstopi (CPURISCVState *env,int csrno,target_ulong *val) {int irq, ret;
@@ -4182,6 +4213,9 @@ riscv_csr_operations csr_ops[CSR_TABLE_SIZE] = { /* Supervisor Protection and Translation */ [CSR_SATP] = { "satp", smode, read_satp, write_satp }, + /* Supervisor-Level Quality of Service Identifier */ + [CSR_SQOSCFG] = { "sqoscfg", check_sqoscfg, read_sqoscfg, write_sqoscfg }, + /* Supervisor-Level Window to Indirectly Accessed Registers (AIA) */ [CSR_SISELECT] = { "siselect", aia_smode, NULL, NULL, rmw_xiselect }, [CSR_SIREG] = { "sireg", aia_smode, NULL, NULL, rmw_xireg },
Signature · 1 line
2.34.1
Re: [RFC PATCH v2 1/9] riscv: implement Ssqosid extension and sqoscfg CSR
Quoted 112 lines · Drew Fustini
From: Kornel Dulęba <[email protected]>
Implement the sqoscfg CSR defined by the Ssqosid ISA extension (Supervisor-mode Quality of Service ID). The CSR contains two fields:
- Resource Control ID (RCID) used determine resource allocation
- Monitoring Counter ID (MCID) used to track resource usageThe CSR is defined for S-mode but accessing it when V=1 shall cause a virtual instruction exception. Implement this behavior by calling the hmode predicate.
Signed-off-by Kornel Dulęba <[email protected]>[dfustini: rebase on v8.0.50, reword commit message]Signed-off-by Drew Fustini <[email protected]>---
Changes since v1:
- rebase on current master (v8.0.50) instead of 8.0.0-rc4disas/riscv.c | 1 + target/riscv/cpu.c | 2 ++ target/riscv/cpu.h | 3 +++ target/riscv/cpu_bits.h | 5 +++++ target/riscv/csr.c | 34 ++++++++++++++++++++++++++++++++++ 5 files changed, 45 insertions(+)disas/riscv.c +1
-0@@ -2100,6 +2100,7 @@ static const char *csr_name(int csrno)case 0x0143 :return "stval" ;case 0x0144 :return "sip" ;case 0x0180 :return "satp" ; +case 0x0181 :return "sqoscfg" ;case 0x0200 :return "hstatus" ;case 0x0202 :return "hedeleg" ;case 0x0203 :return "hideleg" ;target/riscv/cpu.c +2
-0@@ -114,6 +114,7 @@ static const struct isa_ext_data isa_edata_arr[] = {ISA_EXT_DATA_ENTRY (smaia, true,PRIV_VERSION_1_12_0 , ext_smaia),ISA_EXT_DATA_ENTRY (ssaia, true,PRIV_VERSION_1_12_0 , ext_ssaia),ISA_EXT_DATA_ENTRY (sscofpmf, true,PRIV_VERSION_1_12_0 , ext_sscofpmf), +ISA_EXT_DATA_ENTRY (ssqosid, true,PRIV_VERSION_1_12_0 , ext_ssqosid),ISA_EXT_DATA_ENTRY (sstc, true,PRIV_VERSION_1_12_0 , ext_sstc),ISA_EXT_DATA_ENTRY (svadu, true,PRIV_VERSION_1_12_0 , ext_svadu), ISA_EXT_DATA_ENTRY(svinval, true, PRIV_VERSION_1_12_0, ext_svinval),@@ -1397,6 +1398,7 @@ static Property riscv_cpu_extensions[] = {DEFINE_PROP_BOOL ("svadu" ,RISCVCPU , cfg.ext_svadu, true), +DEFINE_PROP_BOOL ("ssqosid" ,RISCVCPU , cfg.ext_ssqosid, true),DEFINE_PROP_BOOL ("svinval" ,RISCVCPU , cfg.ext_svinval, false),DEFINE_PROP_BOOL ("svnapot" ,RISCVCPU , cfg.ext_svnapot, false), DEFINE_PROP_BOOL("svpbmt", RISCVCPU, cfg.ext_svpbmt, false),target/riscv/cpu.h +3
-0@@ -222,6 +222,8 @@ struct CPUArchState {target_ulong mcause;target_ulong mtval;/* since: priv-1.10.0 */ +target_ulong sqoscfg; +/* Machine and Supervisor interrupt priorities */ uint8_t miprio[64 ];uint8_t siprio[64 ];@@ -454,6 +456,7 @@ struct RISCVCPUConfig {bool ext_icboz;bool ext_zicond;bool ext_zihintpause; +bool ext_ssqosid;bool ext_smstateen;bool ext_sstc;bool ext_svadu;target/riscv/cpu_bits.h +5
-0@@ -217,6 +217,7 @@/* Supervisor Protection and Translation */ #define CSR_SPTBR 0x180#define CSR_SATP 0x180 +#define CSR_SQOSCFG 0x181/* Supervisor-Level Window to Indirectly Accessed Registers (AIA) */ #define CSR_SISELECT 0x150@@ -898,4 +899,8 @@ typedef enum RISCVException {#define MHPMEVENT_IDX_MASK 0xFFFFF#define MHPMEVENT_SSCOF_RESVD 16 +/* SQOSCFG BITS (QOSID) */ +#define SQOSCFG_RCID 0x00000FFF +#define SQOSCFG_MCID 0x0FFF0000 +#endif target/riscv/csr.c +3
-0@@ -2700,6 +2700,37 @@ static RISCVException write_satp(CPURISCVState *env, int csrno,return RISCV_EXCP_NONE ; } +static RISCVException check_sqoscfg (CPURISCVState *env,int csrno) +{ +RISCVCPU *cpu =env_archcpu (env);
target/riscv/csr.c static RISCVException write_satp(CPURISCVState *env, int csrno, · line 2715
Quoted 10 lines
target/riscv/csr.c +10
-0+ +if (!cpu->cfg.ext_ssqosid) { +return RISCV_EXCP_ILLEGAL_INST ; + } + +/* +* Even though this is an S-mode CSR the spec says that we need to throw +* and virt instruction fault if a guest tries to access it. +*/ +return hmode (env, csrno);
If above comments is true, use hmode() here is not right, since it only check whether H extension is supported.
It need another check for guest mode access. And we should use smode() instead of hmode() here.
target/riscv/csr.c static RISCVException write_satp(CPURISCVState *env, int csrno, · line 2719
Quoted 4 lines
target/riscv/csr.c +4
-0+} + +static RISCVException read_sqoscfg (CPURISCVState *env,int csrno, +target_ulong *val)
'target_ulong' is better to align with 'CPURISCVState'.
Regards,
Weiwei Li
Quoted 27 lines
target/riscv/csr.c +17
-0+{ + *val = env->sqoscfg; +return RISCV_EXCP_NONE ; +} + +static RISCVException write_sqoscfg (CPURISCVState *env,int csrno, +target_ulong val) +{ + env->sqoscfg = val & (SQOSCFG_RCID |SQOSCFG_MCID ); +return RISCV_EXCP_NONE ; +} + + +static int read_vstopi (CPURISCVState *env,int csrno,target_ulong *val) {int irq, ret;@@ -4182,6 +4213,9 @@ riscv_csr_operations csr_ops[CSR_TABLE_SIZE] = { /* Supervisor Protection and Translation */ [CSR_SATP] = { "satp", smode, read_satp, write_satp }, + /* Supervisor-Level Quality of Service Identifier */ + [CSR_SQOSCFG] = { "sqoscfg", check_sqoscfg, read_sqoscfg, write_sqoscfg }, + /* Supervisor-Level Window to Indirectly Accessed Registers (AIA) */ [CSR_SISELECT] = { "siselect", aia_smode, NULL, NULL, rmw_xiselect }, [CSR_SIREG] = { "sireg", aia_smode, NULL, NULL, rmw_xireg },
RFC PATCH v2 2/9 hw/riscv: define capabilities of CBQRI controllers
In reply to Drew FustiniFrom: Nicolas Pitre <[email protected]>
Define structs to represent the hardware capabilities of capacity and bandwidth controllers according to the RISC-V Capacity and Bandwidth QoS Register Interface (CBQRI).
---
Changes since v1:
- Move defines TYPE_RISCV_CBQRI_CC from cbqri_capacity.c and TYPE_RISCV_CBQRI_BC from cbqri_bandwidth.c into include/hw/riscv.h so machines can include it (suggested by Alistair)
include/hw/riscv/cbqri.h | 81 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 81 insertions(+) create mode 100644 include/hw/riscv/cbqri.h
include/hw/riscv/cbqri.h new file +81 -0
@@ -0,0 +1,81 @@ +/* +* RISC-V Capacity and Bandwidth QoS Register Interface +* URL: https://github.com/riscv-non-isa/riscv-cbqri +* +* Copyright (c) 2023 BayLibre SAS +* +* This program is free software; you can redistribute it and/or modify it +* under the terms and conditions of the GNU General Public License, +* version 2 or later, as published by the Free Software Foundation. +* +* This program is distributed in the hope it will be useful, but WITHOUT +* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for +* more details. +* +* You should have received a copy of the GNU General Public License along with +* this program. If not, see <http://www.gnu.org/licenses/>. +*/ + +#ifndef HW_RISCV_CBQRI_H +#define HW_RISCV_CBQRI_H + +#include "qemu/typedefs.h" + +#define RISCV_CBQRI_VERSION_MAJOR 0 +#define RISCV_CBQRI_VERSION_MINOR 1 + +#define TYPE_RISCV_CBQRI_CC "riscv.cbqri.capacity" +#define TYPE_RISCV_CBQRI_BC "riscv.cbqri.bandwidth" + +/* Capacity Controller hardware capabilities */ +typedef struct RiscvCbqriCapacityCaps { +uint16_t nb_mcids; +uint16_t nb_rcids; + +uint16_t ncblks; + +bool supports_at_data:1 ; +bool supports_at_code:1 ; + +bool supports_alloc_op_config_limit:1 ; +bool supports_alloc_op_read_limit:1 ; +bool supports_alloc_op_flush_rcid:1 ; + +bool supports_mon_op_config_event:1 ; +bool supports_mon_op_read_counter:1 ; + +bool supports_mon_evt_id_none:1 ; +bool supports_mon_evt_id_occupancy:1 ; +}RiscvCbqriCapacityCaps ; + +/* Bandwidth Controller hardware capabilities */ +typedef struct RiscvCbqriBandwidthCaps { +uint16_t nb_mcids; +uint16_t nb_rcids; + +uint16_t nbwblks; +uint16_t mrbwb; + +bool supports_at_data:1 ; +bool supports_at_code:1 ; + +bool supports_alloc_op_config_limit:1 ; +bool supports_alloc_op_read_limit:1 ; + +bool supports_mon_op_config_event:1 ; +bool supports_mon_op_read_counter:1 ; + +bool supports_mon_evt_id_none:1 ; +bool supports_mon_evt_id_rdwr_count:1 ; +bool supports_mon_evt_id_rdonly_count:1 ; +bool supports_mon_evt_id_wronly_count:1 ; +}RiscvCbqriBandwidthCaps ; + +DeviceState *riscv_cbqri_cc_create (hwaddr addr, +const RiscvCbqriCapacityCaps *caps, +const char *target_name); +DeviceState *riscv_cbqri_bc_create (hwaddr addr, +const RiscvCbqriBandwidthCaps *caps, +const char *target_name); +#endif
Signature · 1 line
2.34.1
RFC PATCH v2 3/9 hw/riscv: implement CBQRI capacity controller
In reply to Drew FustiniFrom: Nicolas Pitre <[email protected]>
Implement a capacity controller according to the Capacity and Bandwidth QoS Register Interface (CBQRI) which supports these capabilities:
- Number of access types: 2 (code and data)
- Usage monitoring operations: CONFIG_EVENT, READ_COUNTER
- Event IDs supported: None, Occupancy
- Capacity allocation ops: CONFIG_LIMIT, READ_LIMIT, FLUSH_RCID
---
Changes since v1:
- Move TYPE_RISCV_CBQRI_CC to include/hw/riscv/cbqri.h so that machines can use it (suggested by Alistair)
- Add device properties so that the capacity controller can be fully configured from the command line
hw/riscv/cbqri_capacity.c | 614 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 614 insertions(+) create mode 100644 hw/riscv/cbqri_capacity.c
hw/riscv/cbqri_capacity.c new file +614 -0
@@ -0,0 +1,614 @@ +/* +* RISC-V Capacity and Bandwidth QoS Register Interface +* URL: https://github.com/riscv-non-isa/riscv-cbqri +* +* Copyright (c) 2023 BayLibre SAS +* +* This file contains the Capacity-controller QoS Register Interface. +* +* This program is free software; you can redistribute it and/or modify it +* under the terms and conditions of the GNU General Public License, +* version 2 or later, as published by the Free Software Foundation. +* +* This program is distributed in the hope it will be useful, but WITHOUT +* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for +* more details. +* +* You should have received a copy of the GNU General Public License along with +* this program. If not, see <http://www.gnu.org/licenses/>. +*/ + +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "qemu/error-report.h" +#include "qemu/log.h" +#include "qemu/module.h" +#include "qemu/bitmap.h" +#include "hw/qdev-properties.h" +#include "hw/sysbus.h" +#include "target/riscv/cpu.h" +#include "hw/riscv/cbqri.h" + +/* Encodings of `AT` field */ +enum { +CC_AT_DATA =0 , +CC_AT_CODE =1 , +}; + +/* Capabilities */ +REG64 (CC_CAPABILITIES ,0 ); +FIELD (CC_CAPABILITIES ,VER ,0 ,8 ); +FIELD (CC_CAPABILITIES ,VER_MINOR ,0 ,4 ); +FIELD (CC_CAPABILITIES ,VER_MAJOR ,4 ,4 ); +FIELD (CC_CAPABILITIES ,NCBLKS ,8 ,16 ); +FIELD (CC_CAPABILITIES ,FRCID ,24 ,1 ); + +/* Usage monitoring control */ +REG64 (CC_MON_CTL ,8 ); +FIELD (CC_MON_CTL ,OP ,0 ,5 ); +FIELD (CC_MON_CTL ,AT ,5 ,3 ); +FIELD (CC_MON_CTL ,MCID ,8 ,12 ); +FIELD (CC_MON_CTL ,EVT_ID ,20 ,8 ); +FIELD (CC_MON_CTL ,ATV ,28 ,1 ); +FIELD (CC_MON_CTL ,STATUS ,32 ,7 ); +FIELD (CC_MON_CTL ,BUSY ,39 ,1 ); + +/* Usage monitoring operations */ +enum { +CC_MON_OP_CONFIG_EVENT =1 , +CC_MON_OP_READ_COUNTER =2 , +}; + +/* Usage monitoring event ID */ +enum { + CC_EVT_ID_None =0 , + CC_EVT_ID_Occupancy =1 , +}; + +/* CC_MON_CTL.STATUS field encodings */ +enum { +CC_MON_CTL_STATUS_SUCCESS =1 , +CC_MON_CTL_STATUS_INVAL_OP =2 , +CC_MON_CTL_STATUS_INVAL_MCID =3 , +CC_MON_CTL_STATUS_INVAL_EVT_ID =4 , +CC_MON_CTL_STATUS_INVAL_AT =5 , +}; + +/* Monitoring counter value */ +REG64 (CC_MON_CTR_VAL ,16 ); +FIELD (CC_MON_CTR_VAL ,CTR ,0 ,63 ); +FIELD (CC_MON_CTR_VAL ,INVALID ,63 ,1 ); + +/* Capacity allocation control */ +REG64 (CC_ALLOC_CTL ,24 ); +FIELD (CC_ALLOC_CTL ,OP ,0 ,5 ); +FIELD (CC_ALLOC_CTL ,AT ,5 ,3 ); +FIELD (CC_ALLOC_CTL ,RCID ,8 ,12 ); +FIELD (CC_ALLOC_CTL ,STATUS ,32 ,7 ); +FIELD (CC_ALLOC_CTL ,BUSY ,39 ,1 ); + +/* Capacity allocation operations */ +enum { +CC_ALLOC_OP_CONFIG_LIMIT =1 , +CC_ALLOC_OP_READ_LIMIT =2 , +CC_ALLOC_OP_FLUSH_RCID =3 , +}; + +/* CC_ALLOC_CTL.STATUS field encodings */ +enum { +CC_ALLOC_STATUS_SUCCESS =1 , +CC_ALLOC_STATUS_INVAL_OP =2 , +CC_ALLOC_STATUS_INVAL_RCID =3 , +CC_ALLOC_STATUS_INVAL_AT =4 , +CC_ALLOC_STATUS_INVAL_BLKMASK =5 , +}; + +REG64 (CC_BLOCK_MASK ,32 ); + + +typedef struct MonitorCounter { +uint64_t ctr_val; +int at; +int evt_id; +bool active; +}MonitorCounter ; + +typedef struct RiscvCbqriCapacityState { +SysBusDevice parent_obj; +MemoryRegion mmio; + +/* cached value of some registers */ +uint64_t cc_mon_ctl; +uint64_t cc_mon_ctr_val; +uint64_t cc_alloc_ctl; + +/* monitoring counters */ +MonitorCounter *mon_counters; + +/* allocation blockmasks (1st one is the CC_BLOCK_MASK register) */ +uint64_t *alloc_blockmasks; + +/* properties */ + +uint64_t mmio_base; +char *target; +uint16_t nb_mcids; +uint16_t nb_rcids; + +uint16_t ncblks; + +bool supports_at_data; +bool supports_at_code; + +bool supports_alloc_op_config_limit; +bool supports_alloc_op_read_limit; +bool supports_alloc_op_flush_rcid; + +bool supports_mon_op_config_event; +bool supports_mon_op_read_counter; + +bool supports_mon_evt_id_none; +bool supports_mon_evt_id_occupancy; +}RiscvCbqriCapacityState ; + +#define RISCV_CBQRI_CC (obj) \ + OBJECT_CHECK(RiscvCbqriCapacityState, (obj), TYPE_RISCV_CBQRI_CC) + +static uint64_t *get_blockmask_location (RiscvCbqriCapacityState *cc, +uint32_t rcid,uint32_t at) +{ +/* +* All blockmasks are contiguous to simplify allocation. +* The first one is used to hold the CC_BLOCK_MASK register content, +* followed by respective blockmasks for each AT per RCID. +* Each blockmask is made of one or more uint64_t "slots". +*/ +unsigned int nb_ats =0 ; + nb_ats += !!cc->supports_at_data; + nb_ats += !!cc->supports_at_code; + nb_ats =MAX (nb_ats,1 ); +assert (at < nb_ats); + +unsigned int blockmask_slots = (cc->ncblks +63 ) /64 ; +unsigned int blockmask_offset = blockmask_slots * (1 + rcid * nb_ats + at); + +return cc->alloc_blockmasks + blockmask_offset; +} + +static uint32_t alloc_blockmask_config (RiscvCbqriCapacityState *cc, +uint32_t rcid,uint32_t at, +bool *busy) +{ +unsigned int blockmask_slots = (cc->ncblks +63 ) /64 ; + +if ((cc->ncblks %64 ) !=0 ) { +/* make sure provided mask isn't too large */ +uint64_t tail = cc->alloc_blockmasks[blockmask_slots -1 ]; +if ((tail >> (cc->ncblks %64 )) !=0 ) { +return CC_ALLOC_STATUS_INVAL_BLKMASK ; + } + } + +/* for now we only preserve the current CC_BLOCK_MASK register content */ +memcpy (get_blockmask_location (cc, rcid, at), + cc->alloc_blockmasks, blockmask_slots *8 ); +return CC_ALLOC_STATUS_SUCCESS ; +} + +static uint32_t alloc_blockmask_read (RiscvCbqriCapacityState *cc, +uint32_t rcid,uint32_t at, +bool *busy) +{ +unsigned int blockmask_slots = (cc->ncblks +63 ) /64 ; + +memcpy (cc->alloc_blockmasks, +get_blockmask_location (cc, rcid, at), + blockmask_slots *8 ); +return CC_ALLOC_STATUS_SUCCESS ; +} + +static uint32_t alloc_blockmask_init (RiscvCbqriCapacityState *cc, +uint32_t rcid,uint32_t at,bool set, +bool *busy) +{ +void *blockmask =get_blockmask_location (cc, rcid, at); + +if (set) { +bitmap_fill (blockmask, cc->ncblks); + }else { +bitmap_zero (blockmask, cc->ncblks); + } +return CC_ALLOC_STATUS_SUCCESS ; +} + +static bool is_valid_at (RiscvCbqriCapacityState *cc,uint32_t at) +{ +switch (at) { +case CC_AT_DATA : +return cc->supports_at_data; +case CC_AT_CODE : +return cc->supports_at_code; +default : +return false; + } +} + +static void riscv_cbqri_cc_write_mon_ctl (RiscvCbqriCapacityState *cc, +uint64_t value) +{ +if (!cc->supports_mon_op_config_event && + !cc->supports_mon_op_read_counter) { +/* monitoring not supported: leave mon_ctl set to 0 */ +return ; + } + +/* extract writable fields */ +uint32_t op =FIELD_EX64 (value,CC_MON_CTL ,OP ); +uint32_t at =FIELD_EX64 (value,CC_MON_CTL ,AT ); +uint32_t mcid =FIELD_EX64 (value,CC_MON_CTL ,MCID ); +uint32_t evt_id =FIELD_EX64 (value,CC_MON_CTL ,EVT_ID ); +bool atv =FIELD_EX64 (value,CC_MON_CTL ,ATV ); + +/* extract read-only fields */ +uint32_t status =FIELD_EX64 (cc->cc_mon_ctl,CC_MON_CTL ,STATUS ); +bool busy =FIELD_EX64 (cc->cc_mon_ctl,CC_MON_CTL ,BUSY ); + +if (busy) { +qemu_log_mask (LOG_GUEST_ERROR , +"%s: busy flag still set, ignored" , + __func__); +return ; + } + +if (!cc->supports_at_data && + !cc->supports_at_code) { +/* AT not supported: hardwire to 0 */ + at =0 ; + atv = false; + } + +if (mcid >= cc->nb_mcids) { + status =CC_MON_CTL_STATUS_INVAL_MCID ; + }else if (op ==CC_MON_OP_CONFIG_EVENT && + cc->supports_mon_op_config_event) { +if (evt_id == CC_EVT_ID_None && + cc->supports_mon_evt_id_none) { + cc->mon_counters[mcid].active = false; + status =CC_MON_CTL_STATUS_SUCCESS ; + }else if (evt_id == CC_EVT_ID_Occupancy && + cc->supports_mon_evt_id_occupancy) { +if (atv && !is_valid_at (cc, at)) { + status =CC_MON_CTL_STATUS_INVAL_AT ; + }else { + cc->mon_counters[mcid].ctr_val = +FIELD_DP64 (0 ,CC_MON_CTR_VAL ,INVALID ,1 ); + cc->mon_counters[mcid].evt_id = evt_id; + cc->mon_counters[mcid].at = atv ? at :-1 ; + cc->mon_counters[mcid].active = true; + status =CC_MON_CTL_STATUS_SUCCESS ; + } + }else { + status =CC_MON_CTL_STATUS_INVAL_EVT_ID ; + } + }else if (op ==CC_MON_OP_READ_COUNTER && + cc->supports_mon_op_read_counter) { + cc->cc_mon_ctr_val = cc->mon_counters[mcid].ctr_val; + status =CC_MON_CTL_STATUS_SUCCESS ; + }else { + status =CC_MON_CTL_STATUS_INVAL_OP ; + } + +/* reconstruct updated register value */ + value =0 ; + value =FIELD_DP64 (value,CC_MON_CTL ,OP , op); + value =FIELD_DP64 (value,CC_MON_CTL ,AT , at); + value =FIELD_DP64 (value,CC_MON_CTL ,MCID , mcid); + value =FIELD_DP64 (value,CC_MON_CTL ,EVT_ID , evt_id); + value =FIELD_DP64 (value,CC_MON_CTL ,ATV , atv); + value =FIELD_DP64 (value,CC_MON_CTL ,STATUS , status); + value =FIELD_DP64 (value,CC_MON_CTL ,BUSY , busy); + cc->cc_mon_ctl = value; +} + +static void riscv_cbqri_cc_write_alloc_ctl (RiscvCbqriCapacityState *cc, +uint64_t value) +{ +if (cc->ncblks ==0 || + (!cc->supports_alloc_op_config_limit && + !cc->supports_alloc_op_read_limit && + !cc->supports_alloc_op_flush_rcid)) { +/* capacity allocation not supported: leave alloc_ctl set to 0 */ +return ; + } + +/* extract writable fields */ +uint32_t op =FIELD_EX64 (value,CC_ALLOC_CTL ,OP ); +uint32_t at =FIELD_EX64 (value,CC_ALLOC_CTL ,AT ); +uint32_t rcid =FIELD_EX64 (value,CC_ALLOC_CTL ,RCID ); + +/* extract read-only fields */ +uint32_t status =FIELD_EX64 (cc->cc_alloc_ctl,CC_ALLOC_CTL ,STATUS ); +bool busy =FIELD_EX64 (cc->cc_alloc_ctl,CC_ALLOC_CTL ,BUSY ); + +if (busy) { +qemu_log_mask (LOG_GUEST_ERROR , +"%s: busy flag still set, ignored" , + __func__); +return ; + } + +bool atv = true; +if (!cc->supports_at_data && + !cc->supports_at_code) { +/* AT not supported: hardwire to 0 */ + at =0 ; + atv = false; + } + +if (rcid >= cc->nb_rcids) { + status =CC_ALLOC_STATUS_INVAL_RCID ; + }else if (atv && !is_valid_at (cc, at)) { + status =CC_ALLOC_STATUS_INVAL_AT ; + }else if (op ==CC_ALLOC_OP_CONFIG_LIMIT && + cc->supports_alloc_op_config_limit) { + status =alloc_blockmask_config (cc, rcid, at, &busy); + }else if (op ==CC_ALLOC_OP_READ_LIMIT && + cc->supports_alloc_op_read_limit) { + status =alloc_blockmask_read (cc, rcid, at, &busy); + }else if (op ==CC_ALLOC_OP_FLUSH_RCID && + cc->supports_alloc_op_flush_rcid) { + status =alloc_blockmask_init (cc, rcid, at, false, &busy); + }else { + status =CC_ALLOC_STATUS_INVAL_OP ; + } + +/* reconstruct updated register value */ + value =0 ; + value =FIELD_DP64 (value,CC_ALLOC_CTL ,OP , op); + value =FIELD_DP64 (value,CC_ALLOC_CTL ,AT , at); + value =FIELD_DP64 (value,CC_ALLOC_CTL ,RCID , rcid); + value =FIELD_DP64 (value,CC_ALLOC_CTL ,STATUS , status); + value =FIELD_DP64 (value,CC_ALLOC_CTL ,BUSY , busy); + cc->cc_alloc_ctl = value; +} + +static void riscv_cbqri_cc_write (void *opaque,hwaddr addr, +uint64_t value,unsigned size) +{ +RiscvCbqriCapacityState *cc = opaque; + +assert ((addr %8 ) ==0 ); +assert (size ==8 ); + +switch (addr) { +case A_CC_CAPABILITIES : +/* read-only register */ +break ; +case A_CC_MON_CTL : +riscv_cbqri_cc_write_mon_ctl (cc, value); +break ; +case A_CC_ALLOC_CTL : +riscv_cbqri_cc_write_alloc_ctl (cc, value); +break ; +case A_CC_MON_CTR_VAL : +/* read-only register */ +break ; +case A_CC_BLOCK_MASK : +if (cc->ncblks ==0 ) { +break ; + } +/* fallthrough */ +default : +uint32_t blkmask_slot = (addr -A_CC_BLOCK_MASK ) /8 ; +if (blkmask_slot >= (cc->ncblks +63 ) /64 ) { +qemu_log_mask (LOG_GUEST_ERROR , +"%s: out of bounds (addr=0x%x)" , + __func__, (uint32_t )addr); +break ; + } + cc->alloc_blockmasks[blkmask_slot] = value; + } +} + +static uint64_t riscv_cbqri_cc_read (void *opaque,hwaddr addr,unsigned size) +{ +RiscvCbqriCapacityState *cc = opaque; +uint64_t value =0 ; + +assert ((addr %8 ) ==0 ); +assert (size ==8 ); + +switch (addr) { +case A_CC_CAPABILITIES : + value =FIELD_DP64 (value,CC_CAPABILITIES ,VER_MAJOR , +RISCV_CBQRI_VERSION_MAJOR ); + value =FIELD_DP64 (value,CC_CAPABILITIES ,VER_MINOR , +RISCV_CBQRI_VERSION_MINOR ); + value =FIELD_DP64 (value,CC_CAPABILITIES ,NCBLKS , + cc->ncblks); + value =FIELD_DP64 (value,CC_CAPABILITIES ,FRCID , + cc->supports_alloc_op_flush_rcid); +break ; +case A_CC_MON_CTL : + value = cc->cc_mon_ctl; +break ; +case A_CC_ALLOC_CTL : + value = cc->cc_alloc_ctl; +break ; +case A_CC_MON_CTR_VAL : + value = cc->cc_mon_ctr_val; +break ; +case A_CC_BLOCK_MASK : +if (cc->ncblks ==0 ) { +break ; + } +/* fallthrough */ +default : +unsigned int blkmask_slot = (addr -A_CC_BLOCK_MASK ) /8 ; +if (blkmask_slot >= (cc->ncblks +63 ) /64 ) { +qemu_log_mask (LOG_GUEST_ERROR , +"%s: out of bounds (addr=0x%x)" , + __func__, (uint32_t )addr); +break ; + } + value = cc->alloc_blockmasks[blkmask_slot]; + } + +return value; +} + +static const MemoryRegionOps riscv_cbqri_cc_ops = { + .read = riscv_cbqri_cc_read, + .write = riscv_cbqri_cc_write, + .endianness =DEVICE_LITTLE_ENDIAN , + .valid.min_access_size =4 , + .valid.max_access_size =8 , + .impl.min_access_size =8 , + .impl.max_access_size =8 , +}; + +static void riscv_cbqri_cc_realize (DeviceState *dev,Error **errp) +{ +RiscvCbqriCapacityState *cc =RISCV_CBQRI_CC (dev); + +if (!cc->mmio_base) { +error_setg (errp,"mmio_base property not set" ); +return ; + } + +assert (cc->mon_counters ==NULL ); + cc->mon_counters =g_new0 (MonitorCounter, cc->nb_mcids); + +assert (cc->alloc_blockmasks ==NULL ); +uint64_t *end =get_blockmask_location (cc, cc->nb_rcids,0 ); +unsigned int blockmasks_size = end - cc->alloc_blockmasks; + cc->alloc_blockmasks =g_new0 (uint64_t, blockmasks_size); + +memory_region_init_io (&cc->mmio,OBJECT (dev), &riscv_cbqri_cc_ops, + cc,TYPE_RISCV_CBQRI_CC ".mmio" ,4 *1024 ); +sysbus_init_mmio (SYS_BUS_DEVICE (dev), &cc->mmio); +sysbus_mmio_map (SYS_BUS_DEVICE (dev),0 , cc->mmio_base); +} + +static void riscv_cbqri_cc_reset (DeviceState *dev) +{ +RiscvCbqriCapacityState *cc =RISCV_CBQRI_CC (dev); + + cc->cc_mon_ctl =0 ; + cc->cc_alloc_ctl =0 ; + +/* assign all capacity only to rcid0 */ +for (unsigned int rcid =0 ; rcid < cc->nb_rcids; rcid++) { +bool any_at = false; + +if (cc->supports_at_data) { +alloc_blockmask_init (cc, rcid,CC_AT_DATA , + rcid ==0 ,NULL ); + any_at = true; + } +if (cc->supports_at_code) { +alloc_blockmask_init (cc, rcid,CC_AT_CODE , + rcid ==0 ,NULL ); + any_at = true; + } +if (!any_at) { +alloc_blockmask_init (cc, rcid,0 , + rcid ==0 ,NULL ); + } + } +} + +static Property riscv_cbqri_cc_properties[] = { +DEFINE_PROP_UINT64 ("mmio_base" , RiscvCbqriCapacityState, mmio_base,0 ), +DEFINE_PROP_STRING ("target" , RiscvCbqriCapacityState, target), + +DEFINE_PROP_UINT16 ("max_mcids" , RiscvCbqriCapacityState, nb_mcids,256 ), +DEFINE_PROP_UINT16 ("max_rcids" , RiscvCbqriCapacityState, nb_rcids,64 ), +DEFINE_PROP_UINT16 ("ncblks" , RiscvCbqriCapacityState, ncblks,16 ), + +DEFINE_PROP_BOOL ("at_data" , RiscvCbqriCapacityState, + supports_at_data, true), +DEFINE_PROP_BOOL ("at_code" , RiscvCbqriCapacityState, + supports_at_code, true), + +DEFINE_PROP_BOOL ("alloc_op_config_limit" , RiscvCbqriCapacityState, + supports_alloc_op_config_limit, true), +DEFINE_PROP_BOOL ("alloc_op_read_limit" , RiscvCbqriCapacityState, + supports_alloc_op_read_limit, true), +DEFINE_PROP_BOOL ("alloc_op_flush_rcid" , RiscvCbqriCapacityState, + supports_alloc_op_flush_rcid, true), + +DEFINE_PROP_BOOL ("mon_op_config_event" , RiscvCbqriCapacityState, + supports_mon_op_config_event, true), +DEFINE_PROP_BOOL ("mon_op_read_counter" , RiscvCbqriCapacityState, + supports_mon_op_read_counter, true), + +DEFINE_PROP_BOOL ("mon_evt_id_none" , RiscvCbqriCapacityState, + supports_mon_evt_id_none, true), +DEFINE_PROP_BOOL ("mon_evt_id_occupancy" , RiscvCbqriCapacityState, + supports_mon_evt_id_occupancy, true), + +DEFINE_PROP_END_OF_LIST (), +}; + +static void riscv_cbqri_cc_class_init (ObjectClass *klass,void *data) +{ +DeviceClass *dc =DEVICE_CLASS (klass); + dc->realize = riscv_cbqri_cc_realize; +set_bit (DEVICE_CATEGORY_MISC , dc->categories); + dc->desc ="RISC-V CBQRI Capacity Controller" ; +device_class_set_props (dc, riscv_cbqri_cc_properties); + dc->reset = riscv_cbqri_cc_reset; + dc->user_creatable = true; +} + +static const TypeInfo riscv_cbqri_cc_info = { + .name =TYPE_RISCV_CBQRI_CC , + .parent =TYPE_SYS_BUS_DEVICE , + .instance_size =sizeof (RiscvCbqriCapacityState), + .class_init = riscv_cbqri_cc_class_init, +}; + +static void riscv_cbqri_cc_register_types (void ) +{ +type_register_static (&riscv_cbqri_cc_info); +} + +DeviceState *riscv_cbqri_cc_create (hwaddr addr, +const RiscvCbqriCapacityCaps *caps, +const char *target_name) +{ +DeviceState *dev =qdev_new (TYPE_RISCV_CBQRI_CC ); + +qdev_prop_set_uint64 (dev,"mmio_base" , addr); +qdev_prop_set_string (dev,"target" , target_name); +qdev_prop_set_uint16 (dev,"max_mcids" , caps->nb_mcids); +qdev_prop_set_uint16 (dev,"max_rcids" , caps->nb_rcids); +qdev_prop_set_uint16 (dev,"ncblks" , caps->ncblks); + +qdev_prop_set_bit (dev,"at_data" , + caps->supports_at_data); +qdev_prop_set_bit (dev,"at_code" , + caps->supports_at_code); +qdev_prop_set_bit (dev,"alloc_op_config_limit" , + caps->supports_alloc_op_config_limit); +qdev_prop_set_bit (dev,"alloc_op_read_limit" , + caps->supports_alloc_op_read_limit); +qdev_prop_set_bit (dev,"alloc_op_flush_rcid" , + caps->supports_alloc_op_flush_rcid); +qdev_prop_set_bit (dev,"mon_op_config_event" , + caps->supports_mon_op_config_event); +qdev_prop_set_bit (dev,"mon_op_read_counter" , + caps->supports_mon_op_read_counter); +qdev_prop_set_bit (dev,"mon_evt_id_none" , + caps->supports_mon_evt_id_none); +qdev_prop_set_bit (dev,"mon_evt_id_occupancy" , + caps->supports_mon_evt_id_occupancy); + +sysbus_realize_and_unref (SYS_BUS_DEVICE (dev), &error_fatal); + +return dev; +} + +type_init (riscv_cbqri_cc_register_types)
Signature · 1 line
2.34.1
RFC PATCH v2 4/9 hw/riscv: implement CBQRI bandwidth controller
In reply to Drew FustiniFrom: Nicolas Pitre <[email protected]>
Implement a bandwidth controller according to the Capacity and Bandwidth QoS Register Interface (CBQRI) which supports these capabilities:
- Number of access types: 2 (code and data)
- Usage monitoring operations: CONFIG_EVENT, READ_COUNTER
- Event IDs supported: None, Total read/write byte count, Total read byte count, Total write byte count
- Bandwidth allocation operations: CONFIG_LIMIT, READ_LIMIT
---
Changes since v1:
- Move TYPE_RISCV_CBQRI_BC into include/hw/riscv/cbqri.h so machines can to include it (suggested by Alistair)
- Add device properties so that the bandwidth controller can be fully configured from the command line
hw/riscv/cbqri_bandwidth.c | 600 +++++++++++++++++++++++++++++++++++++ 1 file changed, 600 insertions(+) create mode 100644 hw/riscv/cbqri_bandwidth.c
hw/riscv/cbqri_bandwidth.c new file +600 -0
@@ -0,0 +1,600 @@ +/* +* RISC-V Capacity and Bandwidth QoS Register Interface +* URL: https://github.com/riscv-non-isa/riscv-cbqri +* +* Copyright (c) 2023 BayLibre SAS +* +* This file contains the Bandwidth-controller QoS Register Interface. +* +* This program is free software; you can redistribute it and/or modify it +* under the terms and conditions of the GNU General Public License, +* version 2 or later, as published by the Free Software Foundation. +* +* This program is distributed in the hope it will be useful, but WITHOUT +* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for +* more details. +* +* You should have received a copy of the GNU General Public License along with +* this program. If not, see <http://www.gnu.org/licenses/>. +*/ + +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "qemu/error-report.h" +#include "qemu/log.h" +#include "qemu/module.h" +#include "hw/qdev-properties.h" +#include "hw/sysbus.h" +#include "target/riscv/cpu.h" +#include "hw/riscv/cbqri.h" + + +/* Encodings of `AT` field */ +enum { +BC_AT_DATA =0 , +BC_AT_CODE =1 , +}; + +/* Capabilities */ +REG64 (BC_CAPABILITIES ,0 ); +FIELD (BC_CAPABILITIES ,VER ,0 ,8 ); +FIELD (BC_CAPABILITIES ,VER_MINOR ,0 ,4 ); +FIELD (BC_CAPABILITIES ,VER_MAJOR ,4 ,4 ); +FIELD (BC_CAPABILITIES ,NBWBLKS ,8 ,16 ); +FIELD (BC_CAPABILITIES ,MRBWB ,32 ,16 ); + +/* Usage monitoring control */ +REG64 (BC_MON_CTL ,8 ); +FIELD (BC_MON_CTL ,OP ,0 ,5 ); +FIELD (BC_MON_CTL ,AT ,5 ,3 ); +FIELD (BC_MON_CTL ,MCID ,8 ,12 ); +FIELD (BC_MON_CTL ,EVT_ID ,20 ,8 ); +FIELD (BC_MON_CTL ,ATV ,28 ,1 ); +FIELD (BC_MON_CTL ,STATUS ,32 ,7 ); +FIELD (BC_MON_CTL ,BUSY ,39 ,1 ); + +/* Usage monitoring operations */ +enum { +BC_MON_OP_CONFIG_EVENT =1 , +BC_MON_OP_READ_COUNTER =2 , +}; + +/* Bandwidth monitoring event ID */ +enum { + BC_EVT_ID_None =0 , + BC_EVT_ID_RDWR_count =1 , + BC_EVT_ID_RDONLY_count =2 , + BC_EVT_ID_WRONLY_count =3 , +}; + +/* BC_MON_CTL.STATUS field encodings */ +enum { +BC_MON_CTL_STATUS_SUCCESS =1 , +BC_MON_CTL_STATUS_INVAL_OP =2 , +BC_MON_CTL_STATUS_INVAL_MCID =3 , +BC_MON_CTL_STATUS_INVAL_EVT_ID =4 , +BC_MON_CTL_STATUS_INVAL_AT =5 , +}; + +/* Monitoring counter value */ +REG64 (BC_MON_CTR_VAL ,16 ); +FIELD (BC_MON_CTR_VAL ,CTR ,0 ,62 ); +FIELD (BC_MON_CTR_VAL ,INVALID ,62 ,1 ); +FIELD (BC_MON_CTR_VAL ,OVF ,63 ,1 ); + +/* Bandwidth Allocation control */ +REG64 (BC_ALLOC_CTL ,24 ); +FIELD (BC_ALLOC_CTL ,OP ,0 ,5 ); +FIELD (BC_ALLOC_CTL ,AT ,5 ,3 ); +FIELD (BC_ALLOC_CTL ,RCID ,8 ,12 ); +FIELD (BC_ALLOC_CTL ,STATUS ,32 ,7 ); +FIELD (BC_ALLOC_CTL ,BUSY ,39 ,1 ); + +/* Bandwidth allocation operations */ +enum { +BC_ALLOC_OP_CONFIG_LIMIT =1 , +BC_ALLOC_OP_READ_LIMIT =2 , +}; + +/* BC_ALLOC_CTL.STATUS field encodings */ +enum { +BC_ALLOC_STATUS_SUCCESS =1 , +BC_ALLOC_STATUS_INVAL_OP =2 , +BC_ALLOC_STATUS_INVAL_RCID =3 , +BC_ALLOC_STATUS_INVAL_AT =4 , +BC_ALLOC_STATUS_INVAL_BLKS =5 , +}; + +/* Bandwidth allocation */ +REG64 (BC_BW_ALLOC ,32 ); +FIELD (BC_BW_ALLOC , Rbwb,0 ,16 ); +FIELD (BC_BW_ALLOC , Mweight,20 ,8 ); +FIELD (BC_BW_ALLOC , sharedAT,28 ,3 ); +FIELD (BC_BW_ALLOC , useShared,31 ,1 ); + + +typedef struct MonitorCounter { +uint64_t ctr_val; +int at; +int evt_id; +bool active; +}MonitorCounter ; + +typedef struct BandwidthAllocation { +uint32_t Rbwb:16 ; +uint32_t Mweight:8 ; +uint32_t sharedAT:3 ; +bool useShared:1 ; +}BandwidthAllocation ; + +typedef struct RiscvCbqriBandwidthState { +SysBusDevice parent_obj; +MemoryRegion mmio; + +/* cached value of some registers */ +uint64_t bc_mon_ctl; +uint64_t bc_mon_ctr_val; +uint64_t bc_alloc_ctl; +uint64_t bc_bw_alloc; + +MonitorCounter *mon_counters; +BandwidthAllocation *bw_allocations; + +/* properties */ + +uint64_t mmio_base; +char *target; +uint16_t nb_mcids; +uint16_t nb_rcids; + +uint16_t nbwblks; +uint16_t mrbwb; + +bool supports_at_data; +bool supports_at_code; + +bool supports_alloc_op_config_limit; +bool supports_alloc_op_read_limit; + +bool supports_mon_op_config_event; +bool supports_mon_op_read_counter; + +bool supports_mon_evt_id_none; +bool supports_mon_evt_id_rdwr_count; +bool supports_mon_evt_id_rdonly_count; +bool supports_mon_evt_id_wronly_count; +}RiscvCbqriBandwidthState ; + +#define RISCV_CBQRI_BC (obj) \ + OBJECT_CHECK(RiscvCbqriBandwidthState, (obj), TYPE_RISCV_CBQRI_BC) + +static BandwidthAllocation *get_bw_alloc (RiscvCbqriBandwidthState *bc, +uint32_t rcid,uint32_t at) +{ +/* +* All bandwidth allocation records are contiguous to simplify +* allocation. The first one is used to hold the BC_BW_ALLOC register +* content, followed by respective records for each AT per RCID. +*/ + +unsigned int nb_ats =0 ; + nb_ats += !!bc->supports_at_data; + nb_ats += !!bc->supports_at_code; + nb_ats =MAX (nb_ats,1 ); +assert (at < nb_ats); + +return &bc->bw_allocations[1 + rcid * nb_ats + at]; +} + +static uint32_t bandwidth_config (RiscvCbqriBandwidthState *bc, +uint32_t rcid,uint32_t at, +bool *busy) +{ +BandwidthAllocation *bw_alloc =get_bw_alloc (bc, rcid, at); + +/* for now we only preserve the current BC_BW_ALLOC register content */ + *bw_alloc = bc->bw_allocations[0 ]; +return BC_ALLOC_STATUS_SUCCESS ; +} + +static uint32_t bandwidth_read (RiscvCbqriBandwidthState *bc, +uint32_t rcid,uint32_t at, +bool *busy) +{ +BandwidthAllocation *bw_alloc =get_bw_alloc (bc, rcid, at); + +/* populate BC_BW_ALLOC register with selected content */ + bc->bw_allocations[0 ] = *bw_alloc; +return BC_ALLOC_STATUS_SUCCESS ; +} + +static bool is_valid_at (RiscvCbqriBandwidthState *bc,uint32_t at) +{ +switch (at) { +case BC_AT_DATA : +return bc->supports_at_data; +case BC_AT_CODE : +return bc->supports_at_code; +default : +return false; + } +} + +static void riscv_cbqri_bc_write_mon_ctl (RiscvCbqriBandwidthState *bc, +uint64_t value) +{ +if (!bc->supports_mon_op_config_event && + !bc->supports_mon_op_read_counter) { +/* monitoring not supported: leave mon_ctl set to 0 */ +return ; + } + +/* extract writable fields */ +uint32_t op =FIELD_EX64 (value,BC_MON_CTL ,OP ); +uint32_t at =FIELD_EX64 (value,BC_MON_CTL ,AT ); +uint32_t mcid =FIELD_EX64 (value,BC_MON_CTL ,MCID ); +uint32_t evt_id =FIELD_EX64 (value,BC_MON_CTL ,EVT_ID ); +bool atv =FIELD_EX64 (value,BC_MON_CTL ,ATV ); + +/* extract read-only fields */ +uint32_t status =FIELD_EX64 (bc->bc_mon_ctl,BC_MON_CTL ,STATUS ); +bool busy =FIELD_EX64 (bc->bc_mon_ctl,BC_MON_CTL ,BUSY ); + +if (busy) { +qemu_log_mask (LOG_GUEST_ERROR , +"%s: busy flag still set, ignored" , + __func__); +return ; + } + +if (!bc->supports_at_data && + !bc->supports_at_code) { +/* AT not supported: hardwire to 0 */ + at =0 ; + atv = false; + } + +if (mcid >= bc->nb_mcids) { + status =BC_MON_CTL_STATUS_INVAL_MCID ; + }else if (op ==BC_MON_OP_CONFIG_EVENT && + bc->supports_mon_op_config_event) { +if (evt_id == BC_EVT_ID_None && + bc->supports_mon_evt_id_none) { + bc->mon_counters[mcid].active = false; + status =BC_MON_CTL_STATUS_SUCCESS ; + }else if ((evt_id == BC_EVT_ID_RDWR_count && + bc->supports_mon_evt_id_rdwr_count) || + (evt_id == BC_EVT_ID_RDONLY_count && + bc->supports_mon_evt_id_rdonly_count) || + (evt_id == BC_EVT_ID_WRONLY_count && + bc->supports_mon_evt_id_wronly_count)) { +if (atv && !is_valid_at (bc, at)) { + status =BC_MON_CTL_STATUS_INVAL_AT ; + }else { + bc->mon_counters[mcid].ctr_val = +FIELD_DP64 (0 ,BC_MON_CTR_VAL ,INVALID ,1 ); + bc->mon_counters[mcid].evt_id = evt_id; + bc->mon_counters[mcid].at = atv ? at :-1 ; + bc->mon_counters[mcid].active = true; + status =BC_MON_CTL_STATUS_SUCCESS ; + } + }else { + status =BC_MON_CTL_STATUS_INVAL_EVT_ID ; + } + }else if (op ==BC_MON_OP_READ_COUNTER && + bc->supports_mon_op_read_counter) { + bc->bc_mon_ctr_val = bc->mon_counters[mcid].ctr_val; + status =BC_MON_CTL_STATUS_SUCCESS ; + }else { + status =BC_MON_CTL_STATUS_INVAL_OP ; + } + +/* reconstruct updated register value */ + value =0 ; + value =FIELD_DP64 (value,BC_MON_CTL ,OP , op); + value =FIELD_DP64 (value,BC_MON_CTL ,AT , at); + value =FIELD_DP64 (value,BC_MON_CTL ,MCID , mcid); + value =FIELD_DP64 (value,BC_MON_CTL ,EVT_ID , evt_id); + value =FIELD_DP64 (value,BC_MON_CTL ,ATV , atv); + value =FIELD_DP64 (value,BC_MON_CTL ,STATUS , status); + value =FIELD_DP64 (value,BC_MON_CTL ,BUSY , busy); + bc->bc_mon_ctl = value; +} + +static void riscv_cbqri_bc_write_alloc_ctl (RiscvCbqriBandwidthState *bc, +uint64_t value) +{ +if (bc->nbwblks ==0 || + (!bc->supports_alloc_op_config_limit && + !bc->supports_alloc_op_read_limit)) { +/* capacity allocation not supported: leave bc_alloc_ctl set to 0 */ +return ; + } + +/* extract writable fields */ +uint32_t op =FIELD_EX64 (value,BC_ALLOC_CTL ,OP ); +uint32_t at =FIELD_EX64 (value,BC_ALLOC_CTL ,AT ); +uint32_t rcid =FIELD_EX64 (value,BC_ALLOC_CTL ,RCID ); + +/* extract read-only fields */ +uint32_t status =FIELD_EX64 (bc->bc_alloc_ctl,BC_ALLOC_CTL ,STATUS ); +bool busy =FIELD_EX64 (bc->bc_alloc_ctl,BC_ALLOC_CTL ,BUSY ); + +if (busy) { +qemu_log_mask (LOG_GUEST_ERROR , +"%s: busy flag still set, ignored" , + __func__); +return ; + } + +bool atv = true; +if (!bc->supports_at_data && + !bc->supports_at_code) { +/* AT not supported: hardwire to 0 */ + at =0 ; + atv = false; + } + +if (rcid >= bc->nb_rcids) { + status =BC_ALLOC_STATUS_INVAL_RCID ; + }else if (atv && !is_valid_at (bc, at)) { + status =BC_ALLOC_STATUS_INVAL_AT ; + }else if (op ==BC_ALLOC_OP_CONFIG_LIMIT && + bc->supports_alloc_op_config_limit) { + status =bandwidth_config (bc, rcid, at, &busy); + }else if (op ==BC_ALLOC_OP_READ_LIMIT && + bc->supports_alloc_op_read_limit) { + status =bandwidth_read (bc, rcid, at, &busy); + }else { + status =BC_ALLOC_STATUS_INVAL_OP ; + } + +/* reconstruct updated register value */ + value =0 ; + value =FIELD_DP64 (value,BC_ALLOC_CTL ,OP , op); + value =FIELD_DP64 (value,BC_ALLOC_CTL ,AT , at); + value =FIELD_DP64 (value,BC_ALLOC_CTL ,RCID , rcid); + value =FIELD_DP64 (value,BC_ALLOC_CTL ,STATUS , status); + value =FIELD_DP64 (value,BC_ALLOC_CTL ,BUSY , busy); + bc->bc_alloc_ctl = value; +} + +static void riscv_cbqri_bc_write_bw_alloc (RiscvCbqriBandwidthState *bc, +uint64_t value) +{ +if (bc->nbwblks ==0 ) { +/* capacity allocation not supported: leave bw_alloc set to 0 */ +return ; + } + +BandwidthAllocation *bc_bw_alloc = &bc->bw_allocations[0 ]; + +/* extract writable fields */ + bc_bw_alloc->Rbwb =FIELD_EX64 (value,BC_BW_ALLOC , Rbwb); + bc_bw_alloc->Mweight =FIELD_EX64 (value,BC_BW_ALLOC , Mweight); + bc_bw_alloc->sharedAT =FIELD_EX64 (value,BC_BW_ALLOC , sharedAT); + bc_bw_alloc->useShared =FIELD_EX64 (value,BC_BW_ALLOC , useShared); + +if (!bc->supports_at_data && + !bc->supports_at_code) { +/* AT not supported: hardwire to 0 */ + bc_bw_alloc->sharedAT =0 ; + bc_bw_alloc->useShared = false; + } +} + +static void riscv_cbqri_bc_write (void *opaque,hwaddr addr, +uint64_t value,unsigned size) +{ +RiscvCbqriBandwidthState *bc = opaque; + +assert ((addr %8 ) ==0 ); +assert (size ==8 ); + +switch (addr) { +case A_BC_CAPABILITIES : +/* read-only register */ +break ; +case A_BC_MON_CTL : +riscv_cbqri_bc_write_mon_ctl (bc, value); +break ; +case A_BC_MON_CTR_VAL : +/* read-only register */ +break ; +case A_BC_ALLOC_CTL : +riscv_cbqri_bc_write_alloc_ctl (bc, value); +break ; +case A_BC_BW_ALLOC : +riscv_cbqri_bc_write_bw_alloc (bc, value); +break ; +default : +qemu_log_mask (LOG_GUEST_ERROR , +"%s: out of bounds (addr=0x%x)" , + __func__, (uint32_t )addr); + } +} + +static uint64_t riscv_cbqri_bc_read (void *opaque,hwaddr addr,unsigned size) +{ +RiscvCbqriBandwidthState *bc = opaque; +uint64_t value =0 ; + +assert ((addr %8 ) ==0 ); +assert (size ==8 ); + +switch (addr) { +case A_BC_CAPABILITIES : + value =FIELD_DP64 (value,BC_CAPABILITIES ,VER_MAJOR , +RISCV_CBQRI_VERSION_MAJOR ); + value =FIELD_DP64 (value,BC_CAPABILITIES ,VER_MINOR , +RISCV_CBQRI_VERSION_MINOR ); + value =FIELD_DP64 (value,BC_CAPABILITIES ,NBWBLKS , + bc->nbwblks); + value =FIELD_DP64 (value,BC_CAPABILITIES ,MRBWB , + bc->mrbwb); +break ; +case A_BC_MON_CTL : + value = bc->bc_mon_ctl; +break ; +case A_BC_MON_CTR_VAL : + value = bc->bc_mon_ctr_val; +break ; +case A_BC_ALLOC_CTL : + value = bc->bc_alloc_ctl; +break ; +case A_BC_BW_ALLOC : +BandwidthAllocation *bc_bw_alloc = &bc->bw_allocations[0 ]; + value =FIELD_DP64 (value,BC_BW_ALLOC , Rbwb, bc_bw_alloc->Rbwb); + value =FIELD_DP64 (value,BC_BW_ALLOC , Mweight, bc_bw_alloc->Mweight); + value =FIELD_DP64 (value,BC_BW_ALLOC , sharedAT, bc_bw_alloc->sharedAT); + value =FIELD_DP64 (value,BC_BW_ALLOC , useShared, bc_bw_alloc->useShared); +break ; +default : +qemu_log_mask (LOG_GUEST_ERROR , +"%s: out of bounds (addr=0x%x)" , + __func__, (uint32_t )addr); + } + +return value; +} + +static const MemoryRegionOps riscv_cbqri_bc_ops = { + .read = riscv_cbqri_bc_read, + .write = riscv_cbqri_bc_write, + .endianness =DEVICE_LITTLE_ENDIAN , + .valid.min_access_size =4 , + .valid.max_access_size =8 , + .impl.min_access_size =8 , + .impl.max_access_size =8 , +}; + +static void riscv_cbqri_bc_realize (DeviceState *dev,Error **errp) +{ +RiscvCbqriBandwidthState *bc =RISCV_CBQRI_BC (dev); + +if (!bc->mmio_base) { +error_setg (errp,"mmio_base property not set" ); +return ; + } + +assert (bc->mon_counters ==NULL ); + bc->mon_counters =g_new0 (MonitorCounter, bc->nb_mcids); + +assert (bc->bw_allocations ==NULL ); +BandwidthAllocation *bw_alloc_end =get_bw_alloc (bc, bc->nb_rcids,0 ); +unsigned int bw_alloc_size = bw_alloc_end - bc->bw_allocations; + bc->bw_allocations =g_new0 (BandwidthAllocation, bw_alloc_size); + +memory_region_init_io (&bc->mmio,OBJECT (dev), &riscv_cbqri_bc_ops, + bc,TYPE_RISCV_CBQRI_BC ".mmio" ,4 *1024 ); +sysbus_init_mmio (SYS_BUS_DEVICE (dev), &bc->mmio); +sysbus_mmio_map (SYS_BUS_DEVICE (dev),0 , bc->mmio_base); +} + +static void riscv_cbqri_bc_reset (DeviceState *dev) +{ +RiscvCbqriBandwidthState *bc =RISCV_CBQRI_BC (dev); + + bc->bc_mon_ctl =0 ; + bc->bc_alloc_ctl =0 ; +} + +static Property riscv_cbqri_bc_properties[] = { +DEFINE_PROP_UINT64 ("mmio_base" , RiscvCbqriBandwidthState, mmio_base,0 ), +DEFINE_PROP_STRING ("target" , RiscvCbqriBandwidthState, target), + +DEFINE_PROP_UINT16 ("max_mcids" , RiscvCbqriBandwidthState, nb_mcids,256 ), +DEFINE_PROP_UINT16 ("max_rcids" , RiscvCbqriBandwidthState, nb_rcids,64 ), +DEFINE_PROP_UINT16 ("nbwblks" , RiscvCbqriBandwidthState, nbwblks,1024 ), +DEFINE_PROP_UINT16 ("mrbwb" , RiscvCbqriBandwidthState, mrbwb,819 ), + +DEFINE_PROP_BOOL ("at_data" , RiscvCbqriBandwidthState, + supports_at_data, true), +DEFINE_PROP_BOOL ("at_code" , RiscvCbqriBandwidthState, + supports_at_code, true), + +DEFINE_PROP_BOOL ("alloc_op_config_limit" , RiscvCbqriBandwidthState, + supports_alloc_op_config_limit, true), +DEFINE_PROP_BOOL ("alloc_op_read_limit" , RiscvCbqriBandwidthState, + supports_alloc_op_read_limit, true), + +DEFINE_PROP_BOOL ("mon_op_config_event" , RiscvCbqriBandwidthState, + supports_mon_op_config_event, true), +DEFINE_PROP_BOOL ("mon_op_read_counter" , RiscvCbqriBandwidthState, + supports_mon_op_read_counter, true), + +DEFINE_PROP_BOOL ("mon_evt_id_none" , RiscvCbqriBandwidthState, + supports_mon_evt_id_none, true), +DEFINE_PROP_BOOL ("mon_evt_id_rdwr_count" , RiscvCbqriBandwidthState, + supports_mon_evt_id_rdwr_count, true), +DEFINE_PROP_BOOL ("mon_evt_id_rdonly_count" , RiscvCbqriBandwidthState, + supports_mon_evt_id_rdonly_count, true), +DEFINE_PROP_BOOL ("mon_evt_id_wronly_count" , RiscvCbqriBandwidthState, + supports_mon_evt_id_wronly_count, true), + +DEFINE_PROP_END_OF_LIST (), +}; + +static void riscv_cbqri_bc_class_init (ObjectClass *klass,void *data) +{ +DeviceClass *dc =DEVICE_CLASS (klass); + dc->realize = riscv_cbqri_bc_realize; +set_bit (DEVICE_CATEGORY_MISC , dc->categories); + dc->desc ="RISC-V CBQRI Bandwidth Controller" ; +device_class_set_props (dc, riscv_cbqri_bc_properties); + dc->reset = riscv_cbqri_bc_reset; + dc->user_creatable = true; +} + +static const TypeInfo riscv_cbqri_bc_info = { + .name =TYPE_RISCV_CBQRI_BC , + .parent =TYPE_SYS_BUS_DEVICE , + .instance_size =sizeof (RiscvCbqriBandwidthState), + .class_init = riscv_cbqri_bc_class_init, +}; + +static void riscv_cbqri_bc_register_types (void ) +{ +type_register_static (&riscv_cbqri_bc_info); +} + +DeviceState *riscv_cbqri_bc_create (hwaddr addr, +const RiscvCbqriBandwidthCaps *caps, +const char *target_name) +{ +DeviceState *dev =qdev_new (TYPE_RISCV_CBQRI_BC ); + +qdev_prop_set_uint64 (dev,"mmio_base" , addr); +qdev_prop_set_string (dev,"target" , target_name); +qdev_prop_set_uint16 (dev,"max_mcids" , caps->nb_mcids); +qdev_prop_set_uint16 (dev,"max_rcids" , caps->nb_rcids); +qdev_prop_set_uint16 (dev,"nbwblks" , caps->nbwblks); + +qdev_prop_set_bit (dev,"at_data" , + caps->supports_at_data); +qdev_prop_set_bit (dev,"at_code" , + caps->supports_at_code); +qdev_prop_set_bit (dev,"alloc_op_config_limit" , + caps->supports_alloc_op_config_limit); +qdev_prop_set_bit (dev,"alloc_op_read_limit" , + caps->supports_alloc_op_read_limit); +qdev_prop_set_bit (dev,"mon_op_config_event" , + caps->supports_mon_op_config_event); +qdev_prop_set_bit (dev,"mon_op_read_counter" , + caps->supports_mon_op_read_counter); +qdev_prop_set_bit (dev,"mon_evt_id_none" , + caps->supports_mon_evt_id_none); +qdev_prop_set_bit (dev,"mon_evt_id_rdwr_count" , + caps->supports_mon_evt_id_rdwr_count); +qdev_prop_set_bit (dev,"mon_evt_id_rdonly_count" , + caps->supports_mon_evt_id_rdonly_count); +qdev_prop_set_bit (dev,"mon_evt_id_wronly_count" , + caps->supports_mon_evt_id_wronly_count); + +sysbus_realize_and_unref (SYS_BUS_DEVICE (dev), &error_fatal); + +return dev; +} + +type_init (riscv_cbqri_bc_register_types)
Signature · 1 line
2.34.1
RFC PATCH v2 5/9 hw/riscv: Kconfig: add CBQRI options
In reply to Drew FustiniFrom: Nicolas Pitre <[email protected]>
Add boolean property for CBQRI and imply it should be enabled for the RISC-V virt machine.
---
Changes since v1:
- remove example SoC now that command line arguments supported for CBQRI
- change 'select RISC_CBQRI' to 'imply RISCV_CBQRI' for RISCV_VIRT
hw/riscv/Kconfig | 4 ++++ 1 file changed, 4 insertions(+)
hw/riscv/Kconfig +4 -0
@@ -1,3 +1,6 @@ +config RISCV_CBQRI + bool + config RISCV_NUMA bool
@@ -45,6 +48,7 @@ config RISCV_VIRT select FW_CFG_DMA select PLATFORM_BUS select ACPI + imply RISCV_CBQRI config SHAKTI_C bool
Signature · 1 line
2.34.1
RFC PATCH v2 6/9 hw/riscv: meson: add CBQRI controllers to the build
In reply to Drew FustiniFrom: Nicolas Pitre <[email protected]>
Build the CBQRI controllers when RISC-V CBQRI is enabled by Kconfig.
---
Changes since v1:
- remove example SoC now that command line arguments supported for CBQRI
hw/riscv/meson.build | 2 ++ 1 file changed, 2 insertions(+)
hw/riscv/meson.build +2 -0
@@ -10,5 +10,7 @@ riscv_ss.add(when: 'CONFIG_SIFIVE_U', if_true: files('sifive_u.c')) riscv_ss.add(when: 'CONFIG_SPIKE', if_true: files('spike.c')) riscv_ss.add(when: 'CONFIG_MICROCHIP_PFSOC', if_true: files('microchip_pfsoc.c')) riscv_ss.add(when: 'CONFIG_ACPI', if_true: files('virt-acpi-build.c')) +riscv_ss.add(when: 'CONFIG_RISCV_CBQRI', + if_true: files('cbqri_capacity.c', 'cbqri_bandwidth.c')) hw_arch += {'riscv': riscv_ss}
Signature · 1 line
2.34.1
RFC PATCH v2 7/9 hw/riscv: add CBQRI controllers to virt machine
In reply to Drew FustiniFrom: Nicolas Pitre <[email protected]>
Add CBQRI controllers to the RISC-V virt machine. The device properties can be fully configured from the command line:
$ qemu-system-riscv64 -M virt ... \
-device riscv.cbqri.capacity,mmio_base=0x04828000[,...]
-device riscv.cbqri.bandwidth,mmio_base=0x04829000[,...]The mmio_base option is mandatory, the others are optional.
Many -device arguments as wanted can be provided as long as their mmio regions don't conflict.
To see all possible options:
$ qemu-system-riscv64 -device riscv.cbqri.capacity,help
riscv.cbqri.capacity options:
alloc_op_config_limit=<bool> - (default: true)
alloc_op_flush_rcid=<bool> - (default: true)
alloc_op_read_limit=<bool> - (default: true)
at_code=<bool> - (default: true)
at_data=<bool> - (default: true)
max_mcids=<uint16> - (default: 256)
max_rcids=<uint16> - (default: 64)
mmio_base=<uint64> - (default: 0)
mon_evt_id_none=<bool> - (default: true)
mon_evt_id_occupancy=<bool> - (default: true)
mon_op_config_event=<bool> - (default: true)
mon_op_read_counter=<bool> - (default: true)
ncblks=<uint16> - (default: 16)
target=<str> $ qemu-system-riscv64 -device riscv.cbqri.bandwidth,help
riscv.cbqri.bandwidth options:
alloc_op_config_limit=<bool> - (default: true)
alloc_op_read_limit=<bool> - (default: true)
at_code=<bool> - (default: true)
at_data=<bool> - (default: true)
max_mcids=<uint16> - (default: 256)
max_rcids=<uint16> - (default: 64)
mmio_base=<uint64> - (default: 0)
mon_evt_id_none=<bool> - (default: true)
mon_evt_id_rdonly_count=<bool> - (default: true)
mon_evt_id_rdwr_count=<bool> - (default: true)
mon_evt_id_wronly_count=<bool> - (default: true)
mon_op_config_event=<bool> - (default: true)
mon_op_read_counter=<bool> - (default: true)
nbwblks=<uint16> - (default: 1024)
target=<str>Boolean options correspond to hardware capabilities that can be disabled
---
Changes since v1:
- remove initialization of the example SoC now that device properties can be use to configure controllers from the command line
hw/riscv/virt.c | 3 +++ 1 file changed, 3 insertions(+)
hw/riscv/virt.c +3 -0
@@ -50,6 +50,7 @@#include "hw/pci-host/gpex.h" #include "hw/display/ramfb.h" #include "hw/acpi/aml-build.h" +#include "hw/riscv/cbqri.h" #include "qapi/qapi-visit-common.h" /*
@@ -1688,6 +1689,8 @@ static void virt_machine_class_init(ObjectClass *oc, void *data)#ifdef CONFIG_TPM machine_class_allow_dynamic_sysbus_dev (mc,TYPE_TPM_TIS_SYSBUS );#endif +machine_class_allow_dynamic_sysbus_dev (mc,TYPE_RISCV_CBQRI_BC ); +machine_class_allow_dynamic_sysbus_dev (mc,TYPE_RISCV_CBQRI_CC );object_class_property_add_bool (oc,"aclint" , virt_get_aclint, virt_set_aclint);
Signature · 1 line
2.34.1
RFC PATCH v2 8/9 hw/riscv: instantiate CBQRI controllers for an example SoC
In reply to Drew FustiniFrom: Nicolas Pitre <[email protected]>
Instantiate a hypothetical CBQRI configuration for testing purposes with these properties:
- L2 cache controllers
- Resource type: Capacity
- NCBLKS: 12
- Number of access types: 2 (code and data)
- Usage monitoring not supported
- Capacity allocation operations: CONFIG_LIMIT, READ_LIMIT
- Last-level cache (LLC) controller
- Resource type: Capacity
- NCBLKS: 16
- Number of access types: 2 (code and data)
- Usage monitoring operations: CONFIG_EVENT, READ_COUNTER
- Event IDs supported: None, Occupancy
- Capacity allocation ops: CONFIG_LIMIT, READ_LIMIT, FLUSH_RCID
- Memory controllers
- Resource type: Bandwidth
- NBWBLKS: 1024
- MRBWB: 819 (80% of NBWBLKS)
- Number of access types: 1 (no code/data differentiation)
- Usage monitoring operations: CONFIG_EVENT, READ_COUNTER
- Event IDs supported: None, Total read/write byte count, Total read byte count, Total write byte count
- Bandwidth allocation operations: CONFIG_LIMIT, READ_LIMIT
The memory map for the CBQRI controllers in this example SoC:
Base addr Size 0x4820000 4KB Cluster 0 L2 cache controller 0x4821000 4KB Cluster 1 L2 cache controller 0x4828000 4KB Memory controller 0 0x4829000 4KB Memory controller 1 0x482A000 4KB Memory controller 2 0x482B000 4KB Shared LLC cache controller
---
Note: this example SoC instantiation code is only included for reference and it is not required anymore for the CBQRI proof-of-concept to work. The CBQRI controllers can be fully configured with device properties in the Qemu command line.
hw/riscv/cbqri_example_soc.c | 124 +++++++++++++++++++++++++++++++++++ include/hw/riscv/cbqri.h | 2 + 2 files changed, 126 insertions(+) create mode 100644 hw/riscv/cbqri_example_soc.c
hw/riscv/cbqri_example_soc.c new file +124 -0
@@ -0,0 +1,124 @@ +/* +* RISC-V Capacity and Bandwidth QoS Register Interface +* URL: https://github.com/riscv-non-isa/riscv-cbqri +* +* Copyright (c) 2023 BayLibre SAS +* +* This file contains an hypothetical CBQRI configuration instantiation +* for testing purposes. This may also be configured from the command +* line. +* +* This program is free software; you can redistribute it and/or modify it +* under the terms and conditions of the GNU General Public License, +* version 2 or later, as published by the Free Software Foundation. +* +* This program is distributed in the hope it will be useful, but WITHOUT +* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for +* more details. +* +* You should have received a copy of the GNU General Public License along with +* this program. If not, see <http://www.gnu.org/licenses/>. +*/ + +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "qemu/error-report.h" +#include "qemu/log.h" +#include "qemu/module.h" +#include "hw/sysbus.h" +#include "target/riscv/cpu.h" +#include "hw/riscv/cbqri.h" + +/* +* Example hardware: +* +* - Global +* - Number of RCIDs - 64 +* - Number of MCIDs - 256 +* - L2 cache +* - NCBLKS - 12 +* - Number of access types - 2 (code and data) +* - Usage monitoring not supported +* - Capacity allocation operations - CONFIG_LIMIT, READ_LIMIT +* - LLC +* - NCBLKS - 16 +* - Number of access types - 2 (code and data) +* - Usage monitoring operations - CONFIG_EVENT, READ_COUNTER +* - Event IDs supported - None, Occupancy +* - Capacity allocation operations - CONFIG_LIMIT, READ_LIMIT, FLUSH_RCID +* - Memory controllers +* - NBWBLKS - 1024 +* - MRBWB - 80 (80%) +* - Usage monitoring operations - CONFIG_EVENT, READ_COUNTER +* - Event IDs supported - None, Total read/write byte count, +* - total read byte count, total write byte count +* - Bandwidth allocation operations - CONFIG_LIMIT, READ_LIMIT +* - Number of access types - 1 (no code/data differentiation) +* +* 0x04820000 Cluster 0 L2 cache controller +* 0x04821000 Cluster 1 L2 cache controller +* 0x0482B000 Shared LLC controller +* 0x04828000 Memory controller 0 +* 0x04829000 Memory controller 1 +* 0x0482A000 Memory controller 2 +*/ + +#define CBQRI_NB_MCIDS 256 +#define CBQRI_NB_RCIDS 64 + +static const RiscvCbqriCapacityCaps example_soc_L2_cluster = { + .nb_mcids =CBQRI_NB_MCIDS , + .nb_rcids =CBQRI_NB_RCIDS , + .ncblks =12 , + .supports_at_data = true, + .supports_at_code = true, + .supports_alloc_op_config_limit = true, + .supports_alloc_op_read_limit = true, +}; + +static const RiscvCbqriCapacityCaps example_soc_LLC = { + .nb_mcids =CBQRI_NB_MCIDS , + .nb_rcids =CBQRI_NB_RCIDS , + .ncblks =16 , + .supports_at_data = true, + .supports_at_code = true, + .supports_alloc_op_config_limit = true, + .supports_alloc_op_read_limit = true, + .supports_alloc_op_flush_rcid = true, + .supports_mon_op_config_event = true, + .supports_mon_op_read_counter = true, + .supports_mon_evt_id_none = true, + .supports_mon_evt_id_occupancy = true, +}; + +static const RiscvCbqriBandwidthCaps example_soc_memory = { + .nb_mcids =CBQRI_NB_MCIDS , + .nb_rcids =CBQRI_NB_RCIDS , + .nbwblks =1024 , + .mrbwb =1024 *80 /100 , + .supports_alloc_op_config_limit = true, + .supports_alloc_op_read_limit = true, + .supports_mon_op_config_event = true, + .supports_mon_op_read_counter = true, + .supports_mon_evt_id_none = true, + .supports_mon_evt_id_rdwr_count = true, + .supports_mon_evt_id_rdonly_count = true, + .supports_mon_evt_id_wronly_count = true, +}; + +void example_soc_cbqri_init (void ) +{ +riscv_cbqri_cc_create (0x04820000 , &example_soc_L2_cluster, +"cluster 0 L2 cache controller" ); +riscv_cbqri_cc_create (0x04821000 , &example_soc_L2_cluster, +"cluster 1 L2 cache controller" ); +riscv_cbqri_cc_create (0x0482B000 , &example_soc_LLC, +"shared LLC controller" ); +riscv_cbqri_bc_create (0x04828000 , &example_soc_memory, +"memory controller 0" ); +riscv_cbqri_bc_create (0x04829000 , &example_soc_memory, +"memory controller 1" ); +riscv_cbqri_bc_create (0x0482a000 , &example_soc_memory, +"memory controller 2" ); +}
include/hw/riscv/cbqri.h +2 -0
@@ -78,4 +78,6 @@ DeviceState *riscv_cbqri_cc_create(hwaddr addr,DeviceState *riscv_cbqri_bc_create (hwaddr addr,const RiscvCbqriBandwidthCaps *caps,const char *target_name); + +void example_soc_cbqri_init (void );#endif
Signature · 1 line
2.34.1
RFC PATCH v2 9/9 hw/riscv: build example SoC when CBQRI_EXAMPLE_SOC enabled
In reply to Drew FustiniBuild the example SoC instantiation code when CBQRI_EXAMPLE_SOC is enabled.
---
Note: the example SoC instantiation code is only included for reference and it is not required anymore for the CBQRI proof-of-concept to work. The CBQRI controllers can be fully configured with device properties in the Qemu command line.
hw/riscv/Kconfig | 5 +++++ hw/riscv/meson.build | 1 + 2 files changed, 6 insertions(+)
hw/riscv/Kconfig +5 -0
@@ -49,6 +49,7 @@ config RISCV_VIRT select PLATFORM_BUS select ACPI imply RISCV_CBQRI + imply CBQRI_EXAMPLE_SOC config SHAKTI_C bool
@@ -88,3 +89,7 @@ config SPIKE select HTIF select RISCV_ACLINT select SIFIVE_PLIC + +config CBQRI_EXAMPLE_SOC + bool + select RISCV_CBQRI
hw/riscv/meson.build +1 -0
@@ -12,5 +12,6 @@ riscv_ss.add(when: 'CONFIG_MICROCHIP_PFSOC', if_true: files('microchip_pfsoc.c') riscv_ss.add(when: 'CONFIG_ACPI', if_true: files('virt-acpi-build.c')) riscv_ss.add(when: 'CONFIG_RISCV_CBQRI', if_true: files('cbqri_capacity.c', 'cbqri_bandwidth.c')) +riscv_ss.add(when: 'CONFIG_CBQRI_EXAMPLE_SOC', if_true: files('cbqri_example_soc.c')) hw_arch += {'riscv': riscv_ss}
Signature · 1 line
2.34.1