hw/block/nvme: zoned namespace command set
v2 cover hw/block/nvme: zoned namespace command set
From: Klaus Jensen <[email protected]>
So, things escalated a bit.
I'm removing the old cover letter. I don't think we need it anymore - you all know the reason I keep posting versions of this implementation ;)
Based-on: <[email protected]>
Changes for v2
~~~~~~~~~~~~~~
* "hw/block/nvme: add support for dulbe and block utilization tracking"
- Factor out pstate init/load into separate functions.
- Fixed a stupid off-by-1 bug that would trigger when resetting the last zone.
- I added a more formalized pstate file format that includes a header. This is pretty similar to what is done in Dmitry's series, but with fewer fields. The device parameters for nvme-ns are still the "authoritative" ones, so if any parameters that influence LBA size, number of zones, etc. do not match, an error indicating the discrepancy will be produced. IIRC, Dmitry's version does the same.
It is set up such that newer versions can load pstate files from older versions. The file format header is not unlike a standard nvme datastructure with reserved areas. This means that when adding new command sets that require persistent state, it is not needed to bump the version number, unless the header has to change dramatically. This is demonstrated when the zoned namespace command set support is added in "hw/block/nvme: add basic read/write for zoned namespaces".
* "hw/block/nvme: add basic read/write for zoned namespaces"
- The device will now transition all opened zones to Closed on "normal shutdown". You can force the "transition to Full" behavior by killing QEMU from the monitor.
* "hw/block/nvme: add the zone append command"
- Slightly reordered the logic so a LBA Out of Range error is returned before Invalid Field in Command for normal read/write commands.
* "hw/block/nvme: support zone active excursions"
- Dropped. Optional and non-critical.
* "hw/block/nvme: support reset/finish recommended limits"
- Dropped. Optional and non-critical.
Gollu Appalanaidu (1):
hw/block/nvme: add commands supported and effects log page
Klaus Jensen (13):
hw/block/nvme: add nsid to get/setfeat trace events hw/block/nvme: add trace event for requests with non-zero status code hw/block/nvme: make lba data size configurable hw/block/nvme: reject io commands if only admin command set selected hw/block/nvme: consolidate read, write and write zeroes hw/block/nvme: add support for dulbe and block utilization tracking hw/block/nvme: support namespace types
hw/block/nvme: add basic read/write for zoned namespaces hw/block/nvme: add the zone management receive command hw/block/nvme: add the zone management send command hw/block/nvme: add the zone append command
hw/block/nvme: track and enforce zone resources hw/block/nvme: allow open to close transitions by controller
docs/specs/nvme.txt | 47 +- hw/block/nvme-ns.h | 180 ++++- hw/block/nvme.h | 22 + include/block/nvme.h | 230 +++++- block/nvme.c | 4 +- hw/block/nvme-ns.c | 456 +++++++++++- hw/block/nvme.c | 1560 +++++++++++++++++++++++++++++++++++++++-- hw/block/trace-events | 42 +- 8 files changed, 2443 insertions(+), 98 deletions(-)
Signature · 1 line
2.28.0
PATCH v2 3/14 hw/block/nvme: make lba data size configurable
From: Klaus Jensen <[email protected]>
Allos the LBA data size (lbads) to be set between 9 and 12.
--- docs/specs/nvme.txt | 11 ++++++++++- hw/block/nvme-ns.h | 1 + hw/block/nvme-ns.c | 8 +++++++- hw/block/nvme.c | 1 + 4 files changed, 19 insertions(+), 2 deletions(-)
docs/specs/nvme.txt +10 -1
@@ -1,7 +1,16 @@ NVM Express Controller ====================== -The nvme device (-device nvme) emulates an NVM Express Controller. +The nvme device (-device nvme) emulates an NVM Express Controller. It is used +together with nvme-ns devices (-device nvme-ns) which emulates an NVM Express +Namespace. + +nvme-ns Options +--------------- + + `lbads`; The "LBA Data Size (LBADS)" indicates the LBA data size used by the + namespace. It is specified in terms of a power of two. Only values between + 9 and 12 (both inclusive) are supported. Reference Specifications
hw/block/nvme-ns.h +1 -0
@@ -21,6 +21,7 @@typedef struct NvmeNamespaceParams {uint32_t nsid; +uint8_t lbads; }NvmeNamespaceParams ; typedef struct NvmeNamespace {
hw/block/nvme-ns.c +7 -1
@@ -36,7 +36,7 @@ static void nvme_ns_init(NvmeNamespace *ns) ns->id_ns.dlfeat =0x9 ; } - id_ns->lbaf[0 ].ds =BDRV_SECTOR_BITS ; + id_ns->lbaf[0 ].ds = ns->params.lbads; id_ns->nsze =cpu_to_le64 (nvme_ns_nlbas (ns));
@@ -77,6 +77,11 @@ static int nvme_ns_check_constraints(NvmeNamespace *ns, Error **errp)return -1 ; } +if (ns->params.lbads <9 || ns->params.lbads >12 ) { +error_setg (errp,"unsupported lbads (supported: 9-12)" ); +return -1 ; + } +return 0 ; }
@@ -125,6 +130,7 @@ static void nvme_ns_realize(DeviceState *dev, Error **errp)static Property nvme_ns_props[] = {DEFINE_BLOCK_PROPERTIES (NvmeNamespace, blkconf),DEFINE_PROP_UINT32 ("nsid" , NvmeNamespace, params.nsid,0 ), +DEFINE_PROP_UINT8 ("lbads" , NvmeNamespace, params.lbads,BDRV_SECTOR_BITS ),DEFINE_PROP_END_OF_LIST (), };
hw/block/nvme.c +1 -0
@@ -2812,6 +2812,7 @@ static void nvme_realize(PCIDevice *pci_dev, Error **errp)if (n->namespace.blkconf.blk) { ns = &n->namespace; ns->params.nsid =1 ; + ns->params.lbads =BDRV_SECTOR_BITS ;if (nvme_ns_setup (n, ns, errp)) {return ;
Signature · 1 line
2.28.0
PATCH v2 1/14 hw/block/nvme: add nsid to get/setfeat trace events
In reply to Klaus JensenFrom: Klaus Jensen <[email protected]>
Include the namespace id in the pci_nvme_{get,set}feat trace events.
--- hw/block/nvme.c | 4 ++-- hw/block/trace-events | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-)
hw/block/nvme.c +2 -2
@@ -1660,7 +1660,7 @@ static uint16_t nvme_get_feature(NvmeCtrl *n, NvmeRequest *req) [NVME_ARBITRATION ] =NVME_ARB_AB_NOLIMIT , }; -trace_pci_nvme_getfeat (nvme_cid (req), fid, sel, dw11); +trace_pci_nvme_getfeat (nvme_cid (req), nsid, fid, sel, dw11);if (!nvme_feature_support[fid]) {return NVME_INVALID_FIELD |NVME_DNR ;
@@ -1798,7 +1798,7 @@ static uint16_t nvme_set_feature(NvmeCtrl *n, NvmeRequest *req)uint8_t fid =NVME_GETSETFEAT_FID (dw10);uint8_t save =NVME_SETFEAT_SAVE (dw10); -trace_pci_nvme_setfeat (nvme_cid (req), fid, save, dw11); +trace_pci_nvme_setfeat (nvme_cid (req), nsid, fid, save, dw11);if (save) {return NVME_FID_NOT_SAVEABLE |NVME_DNR ;
hw/block/trace-events +2 -2
@@ -53,8 +53,8 @@ pci_nvme_identify_ns(uint32_t ns) "nsid %"PRIu32"" pci_nvme_identify_nslist(uint32_t ns) "nsid %"PRIu32"" pci_nvme_identify_ns_descr_list(uint32_t ns) "nsid %"PRIu32"" pci_nvme_get_log(uint16_t cid, uint8_t lid, uint8_t lsp, uint8_t rae, uint32_t len, uint64_t off) "cid %"PRIu16" lid 0x%"PRIx8" lsp 0x%"PRIx8" rae 0x%"PRIx8" len %"PRIu32" off %"PRIu64"" -pci_nvme_getfeat(uint16_t cid, uint8_t fid, uint8_t sel, uint32_t cdw11) "cid %"PRIu16" fid 0x%"PRIx8" sel 0x%"PRIx8" cdw11 0x%"PRIx32"" -pci_nvme_setfeat(uint16_t cid, uint8_t fid, uint8_t save, uint32_t cdw11) "cid %"PRIu16" fid 0x%"PRIx8" save 0x%"PRIx8" cdw11 0x%"PRIx32"" +pci_nvme_getfeat(uint16_t cid, uint32_t nsid, uint8_t fid, uint8_t sel, uint32_t cdw11) "cid %"PRIu16" nsid 0x%"PRIx32" fid 0x%"PRIx8" sel 0x%"PRIx8" cdw11 0x%"PRIx32"" +pci_nvme_setfeat(uint16_t cid, uint32_t nsid, uint8_t fid, uint8_t save, uint32_t cdw11) "cid %"PRIu16" nsid 0x%"PRIx32" fid 0x%"PRIx8" save 0x%"PRIx8" cdw11 0x%"PRIx32"" pci_nvme_getfeat_vwcache(const char* result) "get feature volatile write cache, result=%s" pci_nvme_getfeat_numq(int result) "get feature number of queues, result=%d" pci_nvme_setfeat_numq(int reqcq, int reqsq, int gotcq, int gotsq) "requested cq_count=%d sq_count=%d, responding with cq_count=%d sq_count=%d"
Signature · 1 line
2.28.0
PATCH v2 2/14 hw/block/nvme: add trace event for requests with non-zero status code
In reply to Klaus JensenFrom: Klaus Jensen <[email protected]>
If a command results in a non-zero status code, trace it.
--- hw/block/nvme.c | 6 ++++++ hw/block/trace-events | 1 + 2 files changed, 7 insertions(+)
hw/block/nvme.c +6 -0
@@ -777,6 +777,12 @@ static void nvme_enqueue_req_completion(NvmeCQueue *cq, NvmeRequest *req)assert (cq->cqid == req->sq->cqid);trace_pci_nvme_enqueue_req_completion (nvme_cid (req), cq->cqid, req->status); + +if (req->status) { +trace_pci_nvme_err_req_status (nvme_cid (req),nvme_nsid (req->ns), + req->status, req->cmd.opcode); + } +QTAILQ_REMOVE (&req->sq->out_req_list, req, entry);QTAILQ_INSERT_TAIL (&cq->req_list, req, entry);timer_mod (cq->timer,qemu_clock_get_ns (QEMU_CLOCK_VIRTUAL ) +500 );
hw/block/trace-events +1 -0
@@ -89,6 +89,7 @@ pci_nvme_mmio_shutdown_cleared(void) "shutdown bit cleared" # nvme traces for error conditions pci_nvme_err_mdts(uint16_t cid, size_t len) "cid %"PRIu16" len %zu" +pci_nvme_err_req_status(uint16_t cid, uint32_t nsid, uint16_t status, uint8_t opc) "cid %"PRIu16" nsid %"PRIu32" status 0x%"PRIx16" opc 0x%"PRIx8"" pci_nvme_err_addr_read(uint64_t addr) "addr 0x%"PRIx64"" pci_nvme_err_addr_write(uint64_t addr) "addr 0x%"PRIx64"" pci_nvme_err_cfs(void) "controller fatal status"
Signature · 1 line
2.28.0
PATCH v2 7/14 hw/block/nvme: add commands supported and effects log page
In reply to Klaus JensenFrom: Gollu Appalanaidu <[email protected]>
This is to support for the Commands Supported and Effects log page. See NVM Express Spec 1.3d, sec. 5.14.1.5 ("Commands Supported and Effects")
--- include/block/nvme.h | 21 +++++++++++++ hw/block/nvme.c | 75 +++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 95 insertions(+), 1 deletion(-)
include/block/nvme.h +21 -0
@@ -734,6 +734,24 @@ typedef struct QEMU_PACKED NvmeSmartLog {uint8_t reserved2[320 ]; } NvmeSmartLog; +typedef struct QEMU_PACKED NvmeEffectsLog { +uint32_t acs[256 ]; +uint32_t iocs[256 ]; +uint8_t rsvd2048[2048 ]; +} NvmeEffectsLog; + +enum { +NVME_EFFECTS_CSUPP =1 <<0 , +NVME_EFFECTS_LBCC =1 <<1 , +NVME_EFFECTS_NCC =1 <<2 , +NVME_EFFECTS_NIC =1 <<3 , +NVME_EFFECTS_CCC =1 <<4 , +NVME_EFFECTS_CSE_SINGLE =1 <<16 , +NVME_EFFECTS_CSE_MULTI =1 <<17 , +NVME_EFFECTS_CSE_MASK =3 <<16 , +NVME_EFFECTS_UUID_SEL =1 <<19 , +}; +enum NvmeSmartWarn {NVME_SMART_SPARE =1 <<0 ,NVME_SMART_TEMPERATURE =1 <<1 ,
@@ -746,6 +764,7 @@ enum NvmeLogIdentifier {NVME_LOG_ERROR_INFO =0x01 ,NVME_LOG_SMART_INFO =0x02 ,NVME_LOG_FW_SLOT_INFO =0x03 , + NVME_LOG_EFFECTS = 0x05, }; typedef struct QEMU_PACKED NvmePSD {
@@ -857,6 +876,7 @@ enum NvmeIdCtrlFrmw { };enum NvmeIdCtrlLpa { +NVME_LPA_EFFECTS_LOG =1 <<1 ,NVME_LPA_EXTENDED =1 <<2 , };
@@ -1064,5 +1084,6 @@ static inline void _nvme_check_size(void)QEMU_BUILD_BUG_ON (sizeof (NvmeIdNs) !=4096 );QEMU_BUILD_BUG_ON (sizeof (NvmeSglDescriptor) !=16 );QEMU_BUILD_BUG_ON (sizeof (NvmeIdNsDescr) !=4 ); + QEMU_BUILD_BUG_ON(sizeof(NvmeEffectsLog) != 4096); }#endif
hw/block/nvme.c +74 -1
105 lines not on this page · raw message
Signature · 1 line
2.28.0
PATCH v2 6/14 hw/block/nvme: add support for dulbe and block utilization tracking
In reply to Klaus JensenFrom: Klaus Jensen <[email protected]>
This adds support for reporting the Deallocated or Unwritten Logical Block error (DULBE). This requires tracking the allocated/deallocated status of all logical blocks.
Introduce a bitmap that does this. The bitmap is persisted on the new 'pstate' blockdev that is associated with a namespace. If no such drive is attached, the controller will not indicate support for DULBE.
--- docs/specs/nvme.txt | 19 +++++ hw/block/nvme-ns.h | 32 ++++++++ include/block/nvme.h | 5 ++ hw/block/nvme-ns.c | 186 ++++++++++++++++++++++++++++++++++++++++++ hw/block/nvme.c | 130 ++++++++++++++++++++++++++++- hw/block/trace-events | 2 + 6 files changed, 371 insertions(+), 3 deletions(-)
docs/specs/nvme.txt +19 -0
@@ -12,6 +12,25 @@ nvme-ns Options namespace. It is specified in terms of a power of two. Only values between 9 and 12 (both inclusive) are supported. + `pstate`; This parameter specifies another blockdev to be used for storing + persistent state such as logical block allocation tracking. Adding this + parameter enables various optional features of the device. + + -drive id=pstate,file=pstate.img,format=raw + -device nvme-ns,pstate=pstate,... + + To reset (or initialize) state, the blockdev image should be of zero size: + + qemu-img create -f raw pstate.img 0 + + The image will be intialized with a file format header and truncated to + the required size. + + If the pstate given is of non-zero size, it will be assumed to contain + previous saved state and will be checked for consistency. If any stored + values (such as lbads) differs from those specified for the nvme-ns + device an error is produced. + Reference Specifications ------------------------
hw/block/nvme-ns.h +32 -0
@@ -19,6 +19,20 @@#define NVME_NS (obj) \ OBJECT_CHECK(NvmeNamespace, (obj), TYPE_NVME_NS) +#define NVME_PSTATE_MAGIC ((0x00 << 24) | ('S' << 16) | ('P' << 8) | 'N') +#define NVME_PSTATE_V1 1 + +typedef struct NvmePstateHeader { +uint32_t magic; +uint32_t version; + +int64_t blk_len; + +uint8_t lbads; + +uint8_t rsvd17[4079 ]; +} QEMU_PACKED NvmePstateHeader; + typedef struct NvmeNamespaceParams { uint32_t nsid; uint8_t lbads;
@@ -31,7 +45,20 @@ typedef struct NvmeNamespace {int64_t size;NvmeIdNs id_ns; +struct { +BlockBackend *blk; + +struct { +unsigned long *map; +int64_t offset; + } utilization; + } pstate; +NvmeNamespaceParams params; + +struct { +uint32_t err_rec; + } features; } NvmeNamespace;static inline uint32_t nvme_nsid (NvmeNamespace *ns)
@@ -72,4 +99,9 @@ int nvme_ns_setup(NvmeCtrl *n, NvmeNamespace *ns, Error **errp);void nvme_ns_drain (NvmeNamespace *ns);void nvme_ns_flush (NvmeNamespace *ns); +static inline void _nvme_ns_check_size (void ) +{ +QEMU_BUILD_BUG_ON (sizeof (NvmePstateHeader) !=4096 ); +} +#endif /* NVME_NS_H */
include/block/nvme.h +5 -0
@@ -683,6 +683,7 @@ enum NvmeStatusCodes {NVME_E2E_REF_ERROR =0x0284 ,NVME_CMP_FAILURE =0x0285 ,NVME_ACCESS_DENIED =0x0286 , +NVME_DULB =0x0287 ,NVME_MORE =0x2000 ,NVME_DNR =0x4000 , NVME_NO_COMPLETE = 0xffff,
@@ -898,6 +899,9 @@ enum NvmeIdCtrlLpa {#define NVME_AEC_NS_ATTR (aec) ((aec >> 8) & 0x1)#define NVME_AEC_FW_ACTIVATION (aec) ((aec >> 9) & 0x1) +#define NVME_ERR_REC_TLER (err_rec) (err_rec & 0xffff) +#define NVME_ERR_REC_DULBE (err_rec) (err_rec & 0x10000) +enum NvmeFeatureIds {NVME_ARBITRATION =0x1 ,NVME_POWER_MANAGEMENT =0x2 ,
@@ -1018,6 +1022,7 @@ enum NvmeNsIdentifierType {#define NVME_ID_NS_NSFEAT_THIN (nsfeat) ((nsfeat & 0x1)) +#define NVME_ID_NS_NSFEAT_DULBE (nsfeat) ((nsfeat >> 2) & 0x1)#define NVME_ID_NS_FLBAS_EXTENDED (flbas) ((flbas >> 4) & 0x1)#define NVME_ID_NS_FLBAS_INDEX (flbas) ((flbas & 0xf))#define NVME_ID_NS_MC_SEPARATE (mc) ((mc >> 1) & 0x1)
hw/block/nvme-ns.c +186 -0
224 lines not on this page · raw message
hw/block/nvme.c +127 -3
188 lines not on this page · raw message
hw/block/trace-events +2 -0
@@ -42,6 +42,7 @@ pci_nvme_io_cmd(uint16_t cid, uint32_t nsid, uint16_t sqid, uint8_t opcode, cons pci_nvme_admin_cmd(uint16_t cid, uint16_t sqid, uint8_t opcode, const char *opname) "cid %"PRIu16" sqid %"PRIu16" opc 0x%"PRIx8" opname '%s'" pci_nvme_rwz(uint16_t cid, const char *verb, uint32_t nsid, uint32_t nlb, uint64_t len, uint64_t lba) "cid %"PRIu16" opname '%s' nsid %"PRIu32" nlb %"PRIu32" len %"PRIu64" lba 0x%"PRIx64"" pci_nvme_rw_cb(uint16_t cid, const char *blkname) "cid %"PRIu16" blk '%s'" +pci_nvme_allocate(uint32_t ns, uint64_t slba, uint32_t nlb) "nsid %"PRIu32" slba 0x%"PRIx64" nlb %"PRIu32"" pci_nvme_do_aio(uint16_t cid, uint8_t opc, const char *opname, const char *blkname, int64_t offset, size_t len) "cid %"PRIu16" opc 0x%"PRIx8" opname '%s' blk '%s' offset %"PRId64" len %zu" pci_nvme_create_sq(uint64_t addr, uint16_t sqid, uint16_t cqid, uint16_t qsize, uint16_t qflags) "create submission queue, addr=0x%"PRIx64", sqid=%"PRIu16", cqid=%"PRIu16", qsize=%"PRIu16", qflags=%"PRIu16"" pci_nvme_create_cq(uint64_t addr, uint16_t cqid, uint16_t vector, uint16_t size, uint16_t qflags, int ien) "create completion queue, addr=0x%"PRIx64", cqid=%"PRIu16", vector=%"PRIu16", qsize=%"PRIu16", qflags=%"PRIu16", ien=%d"
@@ -89,6 +90,7 @@ pci_nvme_mmio_shutdown_cleared(void) "shutdown bit cleared" # nvme traces for error conditions pci_nvme_err_mdts(uint16_t cid, size_t len) "cid %"PRIu16" len %zu" pci_nvme_err_req_status(uint16_t cid, uint32_t nsid, uint16_t status, uint8_t opc) "cid %"PRIu16" nsid %"PRIu32" status 0x%"PRIx16" opc 0x%"PRIx8"" +pci_nvme_err_dulbe(uint16_t cid, uint64_t slba, uint32_t nlb) "cid %"PRIu16" slba 0x%"PRIx64" nlb %"PRIu32"" pci_nvme_err_addr_read(uint64_t addr) "addr 0x%"PRIx64"" pci_nvme_err_addr_write(uint64_t addr) "addr 0x%"PRIx64"" pci_nvme_err_cfs(void) "controller fatal status"
Signature · 1 line
2.28.0
PATCH v2 4/14 hw/block/nvme: reject io commands if only admin command set selected
In reply to Klaus JensenFrom: Klaus Jensen <[email protected]>
If the host sets CC.CSS to 111b, all commands submitted to I/O queues should be completed with status Invalid Command Opcode.
Note that this is technically a v1.4 feature, but it does not hurt to implement before we finally bump the reported version implemented.
--- include/block/nvme.h | 5 +++++ hw/block/nvme.c | 4 ++++ 2 files changed, 9 insertions(+)
include/block/nvme.h +5 -0
@@ -110,6 +110,11 @@ enum NvmeCcMask {#define NVME_CC_IOSQES (cc) ((cc >> CC_IOSQES_SHIFT) & CC_IOSQES_MASK)#define NVME_CC_IOCQES (cc) ((cc >> CC_IOCQES_SHIFT) & CC_IOCQES_MASK) +enum NvmeCcCss { +NVME_CC_CSS_NVM =0x0 , +NVME_CC_CSS_ADMIN_ONLY =0x7 , +}; +enum NvmeCstsShift {CSTS_RDY_SHIFT =0 ,CSTS_CFS_SHIFT =1 ,
hw/block/nvme.c +4 -0
@@ -1065,6 +1065,10 @@ static uint16_t nvme_io_cmd(NvmeCtrl *n, NvmeRequest *req)trace_pci_nvme_io_cmd (nvme_cid (req), nsid,nvme_sqid (req), req->cmd.opcode,nvme_io_opc_str (req->cmd.opcode)); +if (NVME_CC_CSS (n->bar.cc) ==NVME_CC_CSS_ADMIN_ONLY ) { +return NVME_INVALID_OPCODE |NVME_DNR ; + } +if (!nvme_nsid_valid (n, nsid)) {return NVME_INVALID_NSID |NVME_DNR ; }
Signature · 1 line
2.28.0
PATCH v2 9/14 hw/block/nvme: add basic read/write for zoned namespaces
In reply to Klaus JensenFrom: Klaus Jensen <[email protected]>
This adds basic read and write for zoned namespaces.
A zoned namespace is created by setting the iocs namespace parameter to 0x2 and specifying the zns.zcap parameter (zone capacity) in number of logical blocks per zone. If a zone size (zns.zsze) is not specified, the namespace device will set the zone size to be the next power of two and fit in as many zones as possible on the underlying namespace blockdev. This behavior is not required by the specification, but ensures that the device can be initialized by the Linux kernel nvme driver, which requires a power of two zone size.
If the namespace has an associated 'pstate' blockdev it will be used to store the zone states persistently. Only zone state changes are persisted, that is, zone write pointer updates are only persistent if the zone is explicitly closed. On boot up, any zones that were in an Opened state will be transitioned to Full.
--- docs/specs/nvme.txt | 7 ++ hw/block/nvme-ns.h | 116 +++++++++++++++++++- include/block/nvme.h | 59 ++++++++++- hw/block/nvme-ns.c | 184 +++++++++++++++++++++++++++++++- hw/block/nvme.c | 239 +++++++++++++++++++++++++++++++++++++++++- hw/block/trace-events | 8 ++ 6 files changed, 604 insertions(+), 9 deletions(-)
docs/specs/nvme.txt +7 -0
@@ -34,6 +34,13 @@ nvme-ns Options values (such as lbads) differs from those specified for the nvme-ns device an error is produced. + `zns.zcap`; If `iocs` is 0x2, this specifies the zone capacity. It is + specified in units of logical blocks. + + `zns.zsze`; If `iocs` is 0x2, this specifies the zone size. It is specified + in units of the logical blocks. If not specified, the value depends on + zns.zcap; if the zone capacity is a power of two, the zone size will be + set to that, otherwise it will default to the next power of two. Reference Specifications ------------------------
hw/block/nvme-ns.h +115 -1
173 lines not on this page · raw message
include/block/nvme.h +56 -3
99 lines not on this page · raw message
hw/block/nvme-ns.c +180 -4
265 lines not on this page · raw message
hw/block/nvme.c +238 -1
305 lines not on this page · raw message
hw/block/trace-events +8 -0
@@ -72,6 +72,7 @@ pci_nvme_enqueue_event_noqueue(int queued) "queued %d" pci_nvme_enqueue_event_masked(uint8_t typ) "type 0x%"PRIx8"" pci_nvme_no_outstanding_aers(void) "ignoring event; no outstanding AERs" pci_nvme_enqueue_req_completion(uint16_t cid, uint16_t cqid, uint16_t status) "cid %"PRIu16" cqid %"PRIu16" status 0x%"PRIx16"" +pci_nvme_zns_commit_zone(uint32_t nsid, uint64_t zslba) "nsid 0x%"PRIx32" zslba 0x%"PRIx64"" pci_nvme_mmio_read(uint64_t addr) "addr 0x%"PRIx64"" pci_nvme_mmio_write(uint64_t addr, uint64_t data) "addr 0x%"PRIx64" data 0x%"PRIx64"" pci_nvme_mmio_doorbell_cq(uint16_t cqid, uint16_t new_head) "cqid %"PRIu16" new_head %"PRIu16""
@@ -97,6 +98,11 @@ pci_nvme_err_addr_read(uint64_t addr) "addr 0x%"PRIx64"" pci_nvme_err_addr_write(uint64_t addr) "addr 0x%"PRIx64"" pci_nvme_err_cfs(void) "controller fatal status" pci_nvme_err_aio(uint16_t cid, const char *errname, uint16_t status) "cid %"PRIu16" err '%s' status 0x%"PRIx16"" +pci_nvme_err_zone_is_full(uint16_t cid, uint64_t slba) "cid %"PRIu16" lba 0x%"PRIx64"" +pci_nvme_err_zone_is_read_only(uint16_t cid, uint64_t slba) "cid %"PRIu16" lba 0x%"PRIx64"" +pci_nvme_err_zone_invalid_write(uint16_t cid, uint64_t slba, uint64_t wp) "cid %"PRIu16" lba 0x%"PRIx64" wp 0x%"PRIx64"" +pci_nvme_err_zone_boundary(uint16_t cid, uint64_t slba, uint32_t nlb, uint64_t zcap) "cid %"PRIu16" lba 0x%"PRIx64" nlb %"PRIu32" zcap 0x%"PRIx64"" +pci_nvme_err_zone_pending_writes(uint16_t cid, uint64_t zslba, uint64_t wp, uint64_t wp_staging) "cid %"PRIu16" zslba 0x%"PRIx64" wp 0x%"PRIx64" wp_staging 0x%"PRIx64"" pci_nvme_err_invalid_sgld(uint16_t cid, uint8_t typ) "cid %"PRIu16" type 0x%"PRIx8"" pci_nvme_err_invalid_num_sgld(uint16_t cid, uint8_t typ) "cid %"PRIu16" type 0x%"PRIx8"" pci_nvme_err_invalid_sgl_excess_length(uint16_t cid) "cid %"PRIu16""
@@ -125,6 +131,8 @@ pci_nvme_err_invalid_identify_cns(uint16_t cns) "identify, invalid cns=0x%"PRIx1 pci_nvme_err_invalid_getfeat(int dw10) "invalid get features, dw10=0x%"PRIx32"" pci_nvme_err_invalid_setfeat(uint32_t dw10) "invalid set features, dw10=0x%"PRIx32"" pci_nvme_err_invalid_log_page(uint16_t cid, uint16_t lid) "cid %"PRIu16" lid 0x%"PRIx16"" +pci_nvme_err_invalid_zone(uint16_t cid, uint64_t lba) "cid %"PRIu16" lba 0x%"PRIx64"" +pci_nvme_err_invalid_zone_condition(uint16_t cid, uint64_t zslba, uint8_t condition) "cid %"PRIu16" zslba 0x%"PRIx64" condition 0x%"PRIx8"" pci_nvme_err_startfail_cq(void) "nvme_start_ctrl failed because there are non-admin completion queues" pci_nvme_err_startfail_sq(void) "nvme_start_ctrl failed because there are non-admin submission queues" pci_nvme_err_startfail_nbarasq(void) "nvme_start_ctrl failed because the admin submission queue address is null"
Signature · 1 line
2.28.0
PATCH v2 8/14 hw/block/nvme: support namespace types
In reply to Klaus JensenFrom: Klaus Jensen <[email protected]>
Implement support for TP 4056 ("Namespace Types"). This adds the 'iocs' (I/O Command Set) device parameter to the nvme-ns device.
--- docs/specs/nvme.txt | 3 + hw/block/nvme-ns.h | 14 ++- hw/block/nvme.h | 3 + include/block/nvme.h | 58 +++++++++-- block/nvme.c | 4 +- hw/block/nvme-ns.c | 29 +++++- hw/block/nvme.c | 228 ++++++++++++++++++++++++++++++++++-------- hw/block/trace-events | 6 +- 8 files changed, 285 insertions(+), 60 deletions(-)
docs/specs/nvme.txt +3 -0
@@ -12,6 +12,9 @@ nvme-ns Options namespace. It is specified in terms of a power of two. Only values between 9 and 12 (both inclusive) are supported. + `iocs`; The "I/O Command Set" associated with the namespace. E.g. 0x0 for the + NVM Command Set (the default), or 0x2 for the Zoned Namespace Command Set. + `pstate`; This parameter specifies another blockdev to be used for storing persistent state such as logical block allocation tracking. Adding this parameter enables various optional features of the device.
hw/block/nvme-ns.h +11 -3
@@ -29,12 +29,14 @@ typedef struct NvmePstateHeader {int64_t blk_len;uint8_t lbads; +uint8_t iocs; -uint8_t rsvd17[4079 ]; +uint8_t rsvd18[4078 ]; }QEMU_PACKED NvmePstateHeader;typedef struct NvmeNamespaceParams {uint32_t nsid; +uint8_t iocs;uint8_t lbads; }NvmeNamespaceParams ;
@@ -43,7 +45,8 @@ typedef struct NvmeNamespace {BlockConf blkconf;int32_t bootindex;int64_t size; -NvmeIdNs id_ns; +uint8_t iocs; +void *id_ns[NVME_IOCS_MAX ];struct {BlockBackend *blk;
@@ -70,9 +73,14 @@ static inline uint32_t nvme_nsid(NvmeNamespace *ns)return -1 ; } +static inline NvmeIdNsNvm *nvme_ns_id_nvm (NvmeNamespace *ns) +{ +return ns->id_ns[NVME_IOCS_NVM ]; +} +static inline NvmeLBAF *nvme_ns_lbaf (NvmeNamespace *ns) { -NvmeIdNs *id_ns = &ns->id_ns; +NvmeIdNsNvm *id_ns =nvme_ns_id_nvm (ns);return &id_ns->lbaf[NVME_ID_NS_FLBAS_INDEX (id_ns->flbas)]; }
hw/block/nvme.h +3 -0
@@ -123,6 +123,7 @@ typedef struct NvmeFeatureVal { };uint32_t async_config;uint32_t vwc; +uint32_t iocsci; } NvmeFeatureVal; typedef struct NvmeCtrl {
@@ -150,6 +151,7 @@ typedef struct NvmeCtrl {uint64_t timestamp_set_qemu_clock_ms;/* QEMU clock time */ uint64_t starttime_ms;uint16_t temperature; +uint64_t iocscs[512 ];HostMemoryBackend *pmrdev;
@@ -165,6 +167,7 @@ typedef struct NvmeCtrl {NvmeSQueue admin_sq;NvmeCQueue admin_cq;NvmeIdCtrl id_ctrl; +void *id_ctrl_iocss[NVME_IOCS_MAX ];NvmeFeatureVal features; } NvmeCtrl;
include/block/nvme.h +47 -11
142 lines not on this page · raw message
block/nvme.c +2 -2
@@ -336,7 +336,7 @@ static inline int nvme_translate_error(const NvmeCqe *c) {uint16_t status = (le16_to_cpu (c->status) >>1 ) &0xFF ;if (status) { - trace_nvme_error(le32_to_cpu(c->result), + trace_nvme_error(le32_to_cpu(c->dw0), le16_to_cpu(c->sq_head), le16_to_cpu(c->sq_id), le16_to_cpu(c->cid),
@@ -503,7 +503,7 @@ static void nvme_identify(BlockDriverState *bs, int namespace, Error **errp)BDRVNVMeState *s = bs->opaque;union {NvmeIdCtrl ctrl; -NvmeIdNs ns; +NvmeIdNsNvm ns; } *id;NvmeLBAF *lbaf;uint16_t oncs;
hw/block/nvme-ns.c +26 -3
@@ -57,10 +57,15 @@ static int nvme_blk_truncate(BlockBackend *blk, size_t len, Error **errp)static void nvme_ns_init (NvmeNamespace *ns) { -NvmeIdNs *id_ns = &ns->id_ns; +NvmeIdNsNvm *id_ns; + + ns->id_ns[NVME_IOCS_NVM ] =g_new0 (NvmeIdNsNvm,1 ); + id_ns =nvme_ns_id_nvm (ns); + + ns->iocs = ns->params.iocs;if (blk_get_flags (ns->blkconf.blk) &BDRV_O_UNMAP ) { - ns->id_ns.dlfeat =0x9 ; + id_ns->dlfeat =0x9 ; } id_ns->lbaf[0 ].ds = ns->params.lbads;
@@ -90,6 +95,7 @@ static int nvme_ns_pstate_init(NvmeNamespace *ns, Error **errp) .version = cpu_to_le32(NVME_PSTATE_V1), .blk_len = cpu_to_le64(ns->size), .lbads = ns->params.lbads, + .iocs = ns->params.iocs, }; ret =blk_pwrite (blk,0 , &header,sizeof (header),0 );
@@ -147,6 +153,13 @@ static int nvme_ns_pstate_load(NvmeNamespace *ns, size_t len, Error **errp)return -1 ; } +if (header.iocs != ns->params.iocs) { +error_setg (errp,"iocs parameter inconsistent with pstate " +"(pstate %u; parameter %u)" , + header.iocs, ns->params.iocs); +return -1 ; + } + bitmap_len =DIV_ROUND_UP (nlbas,sizeof (unsigned long )); pstate_len =ROUND_UP (sizeof (NvmePstateHeader) + bitmap_len,BDRV_SECTOR_SIZE );
@@ -246,6 +259,14 @@ static int nvme_ns_check_constraints(NvmeNamespace *ns, Error **errp)return -1 ; } +switch (ns->params.iocs) { +case NVME_IOCS_NVM : +break ; +default : +error_setg (errp,"unsupported iocs" ); +return -1 ; + } +return 0 ; }
@@ -270,7 +291,8 @@ int nvme_ns_setup(NvmeCtrl *n, NvmeNamespace *ns, Error **errp)* With a pstate file in place we can enable the Deallocated or * Unwritten Logical Block Error feature. */ - ns->id_ns.nsfeat |=0x4 ; +NvmeIdNsNvm *id_ns =nvme_ns_id_nvm (ns); + id_ns->nsfeat |=0x4 ; }if (nvme_register_namespace (n, ns, errp)) {
@@ -317,6 +339,7 @@ static Property nvme_ns_props[] = {DEFINE_PROP_UINT32 ("nsid" , NvmeNamespace, params.nsid,0 ),DEFINE_PROP_UINT8 ("lbads" , NvmeNamespace, params.lbads,BDRV_SECTOR_BITS ),DEFINE_PROP_DRIVE ("pstate" , NvmeNamespace, pstate.blk), +DEFINE_PROP_UINT8 ("iocs" , NvmeNamespace, params.iocs,NVME_IOCS_NVM ), DEFINE_PROP_END_OF_LIST(), };
hw/block/nvme.c +189 -39
470 lines not on this page · raw message
hw/block/trace-events +4 -2
@@ -48,10 +48,12 @@ pci_nvme_create_sq(uint64_t addr, uint16_t sqid, uint16_t cqid, uint16_t qsize, pci_nvme_create_cq(uint64_t addr, uint16_t cqid, uint16_t vector, uint16_t size, uint16_t qflags, int ien) "create completion queue, addr=0x%"PRIx64", cqid=%"PRIu16", vector=%"PRIu16", qsize=%"PRIu16", qflags=%"PRIu16", ien=%d" pci_nvme_del_sq(uint16_t qid) "deleting submission queue sqid=%"PRIu16"" pci_nvme_del_cq(uint16_t cqid) "deleted completion queue, cqid=%"PRIu16"" +pci_nvme_identify(uint16_t cid, uint32_t nsid, uint16_t cntid, uint8_t cns, uint8_t csi, uint16_t nvmsetid) "cid %"PRIu16" nsid %"PRIu32" cntid 0x%"PRIx16" cns 0x%"PRIx8" csi 0x%"PRIx8" nvmsetid %"PRIu16"" pci_nvme_identify_ctrl(void) "identify controller" -pci_nvme_identify_ns(uint32_t ns) "nsid %"PRIu32"" -pci_nvme_identify_nslist(uint32_t ns) "nsid %"PRIu32"" +pci_nvme_identify_ns(uint32_t ns, uint8_t csi) "nsid %"PRIu32" csi 0x%"PRIx8"" +pci_nvme_identify_nslist(uint32_t ns, uint8_t csi) "nsid %"PRIu32" csi 0x%"PRIx8"" pci_nvme_identify_ns_descr_list(uint32_t ns) "nsid %"PRIu32"" +pci_nvme_identify_io_cmd_set(uint16_t cid) "cid %"PRIu16"" pci_nvme_get_log(uint16_t cid, uint8_t lid, uint8_t lsp, uint8_t rae, uint32_t len, uint64_t off) "cid %"PRIu16" lid 0x%"PRIx8" lsp 0x%"PRIx8" rae 0x%"PRIx8" len %"PRIu32" off %"PRIu64"" pci_nvme_getfeat(uint16_t cid, uint32_t nsid, uint8_t fid, uint8_t sel, uint32_t cdw11) "cid %"PRIu16" nsid 0x%"PRIx32" fid 0x%"PRIx8" sel 0x%"PRIx8" cdw11 0x%"PRIx32"" pci_nvme_setfeat(uint16_t cid, uint32_t nsid, uint8_t fid, uint8_t save, uint32_t cdw11) "cid %"PRIu16" nsid 0x%"PRIx32" fid 0x%"PRIx8" save 0x%"PRIx8" cdw11 0x%"PRIx32""
Signature · 1 line
2.28.0
PATCH v2 5/14 hw/block/nvme: consolidate read, write and write zeroes
In reply to Klaus JensenFrom: Klaus Jensen <[email protected]>
Consolidate the read/write and write zeroes functions.
--- hw/block/nvme.h | 11 ++++++++ include/block/nvme.h | 2 ++ hw/block/nvme.c | 65 +++++++++++++++---------------------------- hw/block/trace-events | 3 +- 4 files changed, 37 insertions(+), 44 deletions(-)
hw/block/nvme.h +11 -0
@@ -36,6 +36,17 @@ typedef struct NvmeRequest {QTAILQ_ENTRY (NvmeRequest)entry; } NvmeRequest; +static inline bool nvme_req_is_write (NvmeRequest *req) +{ +switch (req->cmd.opcode) { +case NVME_CMD_WRITE : +case NVME_CMD_WRITE_ZEROES : +return true; +default : +return false; + } +} +static inline const char *nvme_adm_opc_str (uint8_t opc) {switch (opc) {
include/block/nvme.h +2 -0
@@ -438,6 +438,8 @@ typedef struct QEMU_PACKED NvmeCmd {uint32_t cdw15; } NvmeCmd; +#define NVME_CMD_OPCODE_DATA_TRANSFER_MASK 0x3 +#define NVME_CMD_FLAGS_FUSE (flags) (flags & 0x3)#define NVME_CMD_FLAGS_PSDT (flags) ((flags >> 6) & 0x3)
hw/block/nvme.c +23 -42
96 lines not on this page · raw message
hw/block/trace-events +1 -2
@@ -40,9 +40,8 @@ pci_nvme_map_prp(uint64_t trans_len, uint32_t len, uint64_t prp1, uint64_t prp2, pci_nvme_map_sgl(uint16_t cid, uint8_t typ, uint64_t len) "cid %"PRIu16" type 0x%"PRIx8" len %"PRIu64"" pci_nvme_io_cmd(uint16_t cid, uint32_t nsid, uint16_t sqid, uint8_t opcode, const char *opname) "cid %"PRIu16" nsid %"PRIu32" sqid %"PRIu16" opc 0x%"PRIx8" opname '%s'" pci_nvme_admin_cmd(uint16_t cid, uint16_t sqid, uint8_t opcode, const char *opname) "cid %"PRIu16" sqid %"PRIu16" opc 0x%"PRIx8" opname '%s'" -pci_nvme_rw(uint16_t cid, const char *verb, uint32_t nsid, uint32_t nlb, uint64_t count, uint64_t lba) "cid %"PRIu16" opname '%s' nsid %"PRIu32" nlb %"PRIu32" count %"PRIu64" lba 0x%"PRIx64"" +pci_nvme_rwz(uint16_t cid, const char *verb, uint32_t nsid, uint32_t nlb, uint64_t len, uint64_t lba) "cid %"PRIu16" opname '%s' nsid %"PRIu32" nlb %"PRIu32" len %"PRIu64" lba 0x%"PRIx64"" pci_nvme_rw_cb(uint16_t cid, const char *blkname) "cid %"PRIu16" blk '%s'" -pci_nvme_write_zeroes(uint16_t cid, uint32_t nsid, uint64_t slba, uint32_t nlb) "cid %"PRIu16" nsid %"PRIu32" slba %"PRIu64" nlb %"PRIu32"" pci_nvme_do_aio(uint16_t cid, uint8_t opc, const char *opname, const char *blkname, int64_t offset, size_t len) "cid %"PRIu16" opc 0x%"PRIx8" opname '%s' blk '%s' offset %"PRId64" len %zu" pci_nvme_create_sq(uint64_t addr, uint16_t sqid, uint16_t cqid, uint16_t qsize, uint16_t qflags) "create submission queue, addr=0x%"PRIx64", sqid=%"PRIu16", cqid=%"PRIu16", qsize=%"PRIu16", qflags=%"PRIu16"" pci_nvme_create_cq(uint64_t addr, uint16_t cqid, uint16_t vector, uint16_t size, uint16_t qflags, int ien) "create completion queue, addr=0x%"PRIx64", cqid=%"PRIu16", vector=%"PRIu16", qsize=%"PRIu16", qflags=%"PRIu16", ien=%d"
Signature · 1 line
2.28.0
PATCH v2 10/14 hw/block/nvme: add the zone management receive command
In reply to Klaus JensenFrom: Klaus Jensen <[email protected]>
Add the Zone Management Receive command.
--- hw/block/nvme-ns.h | 11 +++- hw/block/nvme.h | 1 + include/block/nvme.h | 46 ++++++++++++++ hw/block/nvme-ns.c | 49 ++++++++++++--- hw/block/nvme.c | 135 ++++++++++++++++++++++++++++++++++++++++++ hw/block/trace-events | 1 + 6 files changed, 233 insertions(+), 10 deletions(-)
hw/block/nvme-ns.h +10 -1
@@ -37,9 +37,10 @@ typedef struct NvmePstateHeader {struct {uint64_t zcap;uint64_t zsze; +uint8_t zdes; } QEMU_PACKED zns; -uint8_t rsvd3088[1008 ]; +uint8_t rsvd3089[1007 ]; }QEMU_PACKED NvmePstateHeader; typedef struct NvmeNamespaceParams {
@@ -50,11 +51,13 @@ typedef struct NvmeNamespaceParams {struct {uint64_t zcap;uint64_t zsze; +uint8_t zdes; } zns; } NvmeNamespaceParams;typedef struct NvmeZone {NvmeZoneDescriptor *zd; +uint8_t *zde;uint64_t wp_staging; }NvmeZone ;
@@ -91,6 +94,7 @@ typedef struct NvmeNamespace {NvmeZone *zones;NvmeZoneDescriptor *zd; +uint8_t *zde; } zns; } NvmeNamespace;
@@ -183,6 +187,11 @@ static inline void nvme_zs_set(NvmeZone *zone, NvmeZoneState zs) zone->zd->zs = zs <<4 ; } +static inline size_t nvme_ns_zdes_bytes (NvmeNamespace *ns) +{ +return ns->params.zns.zdes <<6 ; +} +static inline bool nvme_ns_zone_wp_valid (NvmeZone *zone) {switch (nvme_zs (zone)) {
hw/block/nvme.h +1 -0
@@ -71,6 +71,7 @@ static inline const char *nvme_io_opc_str(uint8_t opc)case NVME_CMD_WRITE :return "NVME_NVM_CMD_WRITE" ;case NVME_CMD_READ :return "NVME_NVM_CMD_READ" ;case NVME_CMD_WRITE_ZEROES :return "NVME_NVM_CMD_WRITE_ZEROES" ; +case NVME_CMD_ZONE_MGMT_RECV :return "NVME_ZONED_CMD_ZONE_MGMT_RECV" ;default :return "NVME_NVM_CMD_UNKNOWN" ; } }
include/block/nvme.h +46 -0
70 lines not on this page · raw message
hw/block/nvme-ns.c +40 -9
127 lines not on this page · raw message
hw/block/nvme.c +135 -0
159 lines not on this page · raw message
hw/block/trace-events +1 -0
@@ -42,6 +42,7 @@ pci_nvme_io_cmd(uint16_t cid, uint32_t nsid, uint16_t sqid, uint8_t opcode, cons pci_nvme_admin_cmd(uint16_t cid, uint16_t sqid, uint8_t opcode, const char *opname) "cid %"PRIu16" sqid %"PRIu16" opc 0x%"PRIx8" opname '%s'" pci_nvme_rwz(uint16_t cid, const char *verb, uint32_t nsid, uint32_t nlb, uint64_t len, uint64_t lba) "cid %"PRIu16" opname '%s' nsid %"PRIu32" nlb %"PRIu32" len %"PRIu64" lba 0x%"PRIx64"" pci_nvme_rw_cb(uint16_t cid, const char *blkname) "cid %"PRIu16" blk '%s'" +pci_nvme_zone_mgmt_recv(uint16_t cid, uint32_t nsid, uint64_t slba, uint64_t len, uint8_t zra, uint8_t zrasp, uint8_t zrasf) "cid %"PRIu16" nsid %"PRIu32" slba 0x%"PRIx64" len %"PRIu64" zra 0x%"PRIx8" zrasp 0x%"PRIx8" zrasf 0x%"PRIx8"" pci_nvme_allocate(uint32_t ns, uint64_t slba, uint32_t nlb) "nsid %"PRIu32" slba 0x%"PRIx64" nlb %"PRIu32"" pci_nvme_do_aio(uint16_t cid, uint8_t opc, const char *opname, const char *blkname, int64_t offset, size_t len) "cid %"PRIu16" opc 0x%"PRIx8" opname '%s' blk '%s' offset %"PRId64" len %zu" pci_nvme_create_sq(uint64_t addr, uint16_t sqid, uint16_t cqid, uint16_t qsize, uint16_t qflags) "create submission queue, addr=0x%"PRIx64", sqid=%"PRIu16", cqid=%"PRIu16", qsize=%"PRIu16", qflags=%"PRIu16""
Signature · 1 line
2.28.0
PATCH v2 13/14 hw/block/nvme: track and enforce zone resources
In reply to Klaus JensenFrom: Klaus Jensen <[email protected]>
Track number of open/active resources.
--- docs/specs/nvme.txt | 7 +++++ hw/block/nvme-ns.h | 7 +++++ include/block/nvme.h | 2 ++ hw/block/nvme-ns.c | 25 +++++++++++++++-- hw/block/nvme.c | 67 +++++++++++++++++++++++++++++++++++++++++++- 5 files changed, 104 insertions(+), 4 deletions(-)
docs/specs/nvme.txt +7 -0
@@ -42,6 +42,13 @@ nvme-ns Options zns.zcap; if the zone capacity is a power of two, the zone size will be set to that, otherwise it will default to the next power of two. + `zns.mar`; Specifies the number of active resources available. This is a 0s + based value. + + `zns.mor`; Specifies the number of open resources available. This is a 0s + based value. + + Reference Specifications ------------------------
hw/block/nvme-ns.h +7 -0
@@ -52,6 +52,8 @@ typedef struct NvmeNamespaceParams {uint64_t zcap;uint64_t zsze;uint8_t zdes; +uint32_t mar; +uint32_t mor; } zns; } NvmeNamespaceParams;
@@ -95,6 +97,11 @@ typedef struct NvmeNamespace {NvmeZone *zones;NvmeZoneDescriptor *zd;uint8_t *zde; + +struct { +uint32_t open; +uint32_t active; + } resources; } zns; } NvmeNamespace;
include/block/nvme.h +2 -0
@@ -776,6 +776,8 @@ enum NvmeStatusCodes {NVME_ZONE_IS_READ_ONLY =0x01ba ,NVME_ZONE_IS_OFFLINE =0x01bb ,NVME_ZONE_INVALID_WRITE =0x01bc , +NVME_TOO_MANY_ACTIVE_ZONES =0x01bd , +NVME_TOO_MANY_OPEN_ZONES =0x01be ,NVME_INVALID_ZONE_STATE_TRANSITION =0x01bf ,NVME_WRITE_FAULT =0x0280 , NVME_UNRECOVERED_READ = 0x0281,
hw/block/nvme-ns.c +22 -3
@@ -119,8 +119,8 @@ static void nvme_ns_init_zoned(NvmeNamespace *ns) ns->zns.zde =g_malloc0_n (ns->zns.num_zones,nvme_ns_zdes_bytes (ns)); } - id_ns_zns->mar =0xffffffff ; - id_ns_zns->mor =0xffffffff ; + id_ns_zns->mar =cpu_to_le32 (ns->params.zns.mar); + id_ns_zns->mor =cpu_to_le32 (ns->params.zns.mor); }static void nvme_ns_init (NvmeNamespace *ns)
@@ -220,6 +220,11 @@ static int nvme_ns_pstate_init(NvmeNamespace *ns, Error **errp)void nvme_ns_zns_init_zone_state (NvmeNamespace *ns) { + ns->zns.resources.active = ns->params.zns.mar !=0xffffffff ? + ns->params.zns.mar +1 : ns->zns.num_zones; + ns->zns.resources.open = ns->params.zns.mor !=0xffffffff ? + ns->params.zns.mor +1 : ns->zns.num_zones; +for (int i =0 ; i < ns->zns.num_zones; i++) {NvmeZone *zone = &ns->zns.zones[i]; zone->zd = &ns->zns.zd[i];
@@ -238,9 +243,15 @@ void nvme_ns_zns_init_zone_state(NvmeNamespace *ns)if (nvme_wp (zone) ==nvme_zslba (zone) && !(zone->zd->za &NVME_ZA_ZDEV )) {nvme_zs_set (zone,NVME_ZS_ZSE ); +continue ; } -continue ; +if (ns->zns.resources.active) { + ns->zns.resources.active--; +continue ; + } + +/* fallthrough */ case NVME_ZS_ZSIO :case NVME_ZS_ZSEO :
@@ -462,6 +473,12 @@ static int nvme_ns_check_constraints(NvmeNamespace *ns, Error **errp)return -1 ; } +if (ns->params.zns.mor > ns->params.zns.mar) { +error_setg (errp,"maximum open resources (zns.mor) must be less " +"than or equal to maximum active resources (zns.mar)" ); +return -1 ; + } +break ;default :
@@ -547,6 +564,8 @@ static Property nvme_ns_props[] = {DEFINE_PROP_UINT64 ("zns.zcap" , NvmeNamespace, params.zns.zcap,0 ),DEFINE_PROP_UINT64 ("zns.zsze" , NvmeNamespace, params.zns.zsze,0 ),DEFINE_PROP_UINT8 ("zns.zdes" , NvmeNamespace, params.zns.zdes,0 ), +DEFINE_PROP_UINT32 ("zns.mar" , NvmeNamespace, params.zns.mar,0xffffffff ), +DEFINE_PROP_UINT32 ("zns.mor" , NvmeNamespace, params.zns.mor,0xffffffff ), DEFINE_PROP_END_OF_LIST(), };
hw/block/nvme.c +66 -1
106 lines not on this page · raw message
Signature · 1 line
2.28.0
PATCH v2 12/14 hw/block/nvme: add the zone append command
In reply to Klaus JensenFrom: Klaus Jensen <[email protected]>
Add the Zone Append command.
--- hw/block/nvme.h | 6 ++++ include/block/nvme.h | 7 +++++ hw/block/nvme.c | 71 +++++++++++++++++++++++++++++++++++++++---- hw/block/trace-events | 1 + 4 files changed, 79 insertions(+), 6 deletions(-)
hw/block/nvme.h +6 -0
@@ -16,6 +16,10 @@ typedef struct NvmeParams {uint32_t aer_max_queued;uint8_t mdts;bool use_intel_id; + +struct { +uint8_t zasl; + } zns; } NvmeParams; typedef struct NvmeAsyncEvent {
@@ -41,6 +45,7 @@ static inline bool nvme_req_is_write(NvmeRequest *req)switch (req->cmd.opcode) {case NVME_CMD_WRITE :case NVME_CMD_WRITE_ZEROES : +case NVME_CMD_ZONE_APPEND :return true;default :return false;
@@ -73,6 +78,7 @@ static inline const char *nvme_io_opc_str(uint8_t opc)case NVME_CMD_WRITE_ZEROES :return "NVME_NVM_CMD_WRITE_ZEROES" ;case NVME_CMD_ZONE_MGMT_SEND :return "NVME_ZONED_CMD_ZONE_MGMT_SEND" ;case NVME_CMD_ZONE_MGMT_RECV :return "NVME_ZONED_CMD_ZONE_MGMT_RECV" ; +case NVME_CMD_ZONE_APPEND :return "NVME_ZONED_CMD_ZONE_APPEND" ;default :return "NVME_NVM_CMD_UNKNOWN" ; } }
include/block/nvme.h +7 -0
@@ -483,6 +483,7 @@ enum NvmeIoCommands {NVME_CMD_DSM =0x09 ,NVME_CMD_ZONE_MGMT_SEND =0x79 ,NVME_CMD_ZONE_MGMT_RECV =0x7a , + NVME_CMD_ZONE_APPEND = 0x7d, }; typedef struct QEMU_PACKED NvmeDeleteQ {
@@ -1018,6 +1019,11 @@ enum NvmeIdCtrlLpa { NVME_LPA_EXTENDED = 1 << 2, }; +typedef struct QEMU_PACKED NvmeIdCtrlZns { +uint8_t zasl; +uint8_t rsvd1[4095 ]; +} NvmeIdCtrlZns; +#define NVME_CTRL_SQES_MIN (sqes) ((sqes) & 0xf)#define NVME_CTRL_SQES_MAX (sqes) (((sqes) >> 4) & 0xf)#define NVME_CTRL_CQES_MIN (cqes) ((cqes) & 0xf)
@@ -1242,6 +1248,7 @@ static inline void _nvme_check_size(void)QEMU_BUILD_BUG_ON (sizeof (NvmeFwSlotInfoLog) !=512 );QEMU_BUILD_BUG_ON (sizeof (NvmeSmartLog) !=512 );QEMU_BUILD_BUG_ON (sizeof (NvmeIdCtrl) !=4096 ); +QEMU_BUILD_BUG_ON (sizeof (NvmeIdCtrlZns) !=4096 );QEMU_BUILD_BUG_ON (sizeof (NvmeIdNsNvm) !=4096 );QEMU_BUILD_BUG_ON (sizeof (NvmeIdNsZns) !=4096 );QEMU_BUILD_BUG_ON (sizeof (NvmeSglDescriptor) !=16 );
hw/block/nvme.c +65 -6
143 lines not on this page · raw message
hw/block/trace-events +1 -0
@@ -103,6 +103,7 @@ pci_nvme_mmio_shutdown_cleared(void) "shutdown bit cleared" # nvme traces for error conditions pci_nvme_err_mdts(uint16_t cid, size_t len) "cid %"PRIu16" len %zu" +pci_nvme_err_zasl(uint16_t cid, size_t len) "cid %"PRIu16" len %zu" pci_nvme_err_req_status(uint16_t cid, uint32_t nsid, uint16_t status, uint8_t opc) "cid %"PRIu16" nsid %"PRIu32" status 0x%"PRIx16" opc 0x%"PRIx8"" pci_nvme_err_dulbe(uint16_t cid, uint64_t slba, uint32_t nlb) "cid %"PRIu16" slba 0x%"PRIx64" nlb %"PRIu32"" pci_nvme_err_addr_read(uint64_t addr) "addr 0x%"PRIx64""
Signature · 1 line
2.28.0
PATCH v2 11/14 hw/block/nvme: add the zone management send command
In reply to Klaus JensenFrom: Klaus Jensen <[email protected]>
Add the Zone Management Send command.
--- hw/block/nvme.h | 1 + include/block/nvme.h | 29 +++ hw/block/nvme.c | 552 ++++++++++++++++++++++++++++++++++++++++-- hw/block/trace-events | 11 + 4 files changed, 574 insertions(+), 19 deletions(-)
hw/block/nvme.h +1 -0
@@ -71,6 +71,7 @@ static inline const char *nvme_io_opc_str(uint8_t opc)case NVME_CMD_WRITE :return "NVME_NVM_CMD_WRITE" ;case NVME_CMD_READ :return "NVME_NVM_CMD_READ" ;case NVME_CMD_WRITE_ZEROES :return "NVME_NVM_CMD_WRITE_ZEROES" ; +case NVME_CMD_ZONE_MGMT_SEND :return "NVME_ZONED_CMD_ZONE_MGMT_SEND" ;case NVME_CMD_ZONE_MGMT_RECV :return "NVME_ZONED_CMD_ZONE_MGMT_RECV" ;default :return "NVME_NVM_CMD_UNKNOWN" ; }
include/block/nvme.h +29 -0
@@ -481,6 +481,7 @@ enum NvmeIoCommands {NVME_CMD_COMPARE =0x05 ,NVME_CMD_WRITE_ZEROES =0x08 ,NVME_CMD_DSM =0x09 , +NVME_CMD_ZONE_MGMT_SEND =0x79 , NVME_CMD_ZONE_MGMT_RECV = 0x7a, };
@@ -594,6 +595,32 @@ enum { NVME_RW_PRINFO_PRCHK_REF = 1 << 10, }; +typedef struct QEMU_PACKED NvmeZoneManagementSendCmd { +uint8_t opcode; +uint8_t flags; +uint16_t cid; +uint32_t nsid; +uint32_t rsvd8[4 ]; +NvmeCmdDptr dptr; +uint64_t slba; +uint32_t rsvd48; +uint8_t zsa; +uint8_t zsflags; +uint16_t rsvd54; +uint32_t rsvd56[2 ]; +} NvmeZoneManagementSendCmd; + +#define NVME_CMD_ZONE_MGMT_SEND_SELECT_ALL (zsflags) ((zsflags) & 0x1) + +typedef enum NvmeZoneManagementSendAction { +NVME_CMD_ZONE_MGMT_SEND_CLOSE =0x1 , +NVME_CMD_ZONE_MGMT_SEND_FINISH =0x2 , +NVME_CMD_ZONE_MGMT_SEND_OPEN =0x3 , +NVME_CMD_ZONE_MGMT_SEND_RESET =0x4 , +NVME_CMD_ZONE_MGMT_SEND_OFFLINE =0x5 , +NVME_CMD_ZONE_MGMT_SEND_SET_ZDE =0x10 , +}NvmeZoneManagementSendAction ; + typedef struct QEMU_PACKED NvmeZoneManagementRecvCmd {uint8_t opcode;uint8_t flags;
@@ -748,6 +775,7 @@ enum NvmeStatusCodes {NVME_ZONE_IS_READ_ONLY =0x01ba ,NVME_ZONE_IS_OFFLINE =0x01bb ,NVME_ZONE_INVALID_WRITE =0x01bc , +NVME_INVALID_ZONE_STATE_TRANSITION =0x01bf ,NVME_WRITE_FAULT =0x0280 ,NVME_UNRECOVERED_READ =0x0281 , NVME_E2E_GUARD_ERROR = 0x0282,
@@ -1207,6 +1235,7 @@ static inline void _nvme_check_size(void)QEMU_BUILD_BUG_ON (sizeof (NvmeIdentify) !=64 );QEMU_BUILD_BUG_ON (sizeof (NvmeRwCmd) !=64 );QEMU_BUILD_BUG_ON (sizeof (NvmeDsmCmd) !=64 ); +QEMU_BUILD_BUG_ON (sizeof (NvmeZoneManagementSendCmd) !=64 );QEMU_BUILD_BUG_ON (sizeof (NvmeZoneManagementRecvCmd) !=64 );QEMU_BUILD_BUG_ON (sizeof (NvmeRangeType) !=64 );QEMU_BUILD_BUG_ON (sizeof (NvmeErrorLog) !=64 );
hw/block/nvme.c +533 -19
619 lines not on this page · raw message
hw/block/trace-events +11 -0
@@ -42,8 +42,18 @@ pci_nvme_io_cmd(uint16_t cid, uint32_t nsid, uint16_t sqid, uint8_t opcode, cons pci_nvme_admin_cmd(uint16_t cid, uint16_t sqid, uint8_t opcode, const char *opname) "cid %"PRIu16" sqid %"PRIu16" opc 0x%"PRIx8" opname '%s'" pci_nvme_rwz(uint16_t cid, const char *verb, uint32_t nsid, uint32_t nlb, uint64_t len, uint64_t lba) "cid %"PRIu16" opname '%s' nsid %"PRIu32" nlb %"PRIu32" len %"PRIu64" lba 0x%"PRIx64"" pci_nvme_rw_cb(uint16_t cid, const char *blkname) "cid %"PRIu16" blk '%s'" +pci_nvme_zone_mgmt_send(uint16_t cid, uint32_t nsid, uint64_t zslba, uint8_t zsa, uint8_t zsflags) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64" zsa 0x%"PRIx8" zsflags 0x%"PRIx8"" +pci_nvme_zone_mgmt_send_all(uint16_t cid, uint32_t nsid, uint8_t za) "cid %"PRIu16" nsid %"PRIu32" za 0x%"PRIx8"" +pci_nvme_zone_mgmt_send_close(uint16_t cid, uint32_t nsid, uint64_t zslba, const char *zc) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64" zc \"%s\"" +pci_nvme_zone_mgmt_send_finish(uint16_t cid, uint32_t nsid, uint64_t zslba, const char *zc) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64" zc \"%s\"" +pci_nvme_zone_mgmt_send_open(uint16_t cid, uint32_t nsid, uint64_t zslba, const char *zc) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64" zc \"%s\"" +pci_nvme_zone_mgmt_send_reset(uint16_t cid, uint32_t nsid, uint64_t zslba, const char *zc) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64" zc \"%s\"" +pci_nvme_zone_mgmt_send_reset_cb(uint16_t cid, uint32_t nsid) "cid %"PRIu16" nsid %"PRIu32"" +pci_nvme_zone_mgmt_send_offline(uint16_t cid, uint32_t nsid, uint64_t zslba, const char *zc) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64" zc \"%s\"" +pci_nvme_zone_mgmt_send_set_zde(uint16_t cid, uint32_t nsid, uint64_t zslba, const char *zc) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64" zc \"%s\"" pci_nvme_zone_mgmt_recv(uint16_t cid, uint32_t nsid, uint64_t slba, uint64_t len, uint8_t zra, uint8_t zrasp, uint8_t zrasf) "cid %"PRIu16" nsid %"PRIu32" slba 0x%"PRIx64" len %"PRIu64" zra 0x%"PRIx8" zrasp 0x%"PRIx8" zrasf 0x%"PRIx8"" pci_nvme_allocate(uint32_t ns, uint64_t slba, uint32_t nlb) "nsid %"PRIu32" slba 0x%"PRIx64" nlb %"PRIu32"" +pci_nvme_deallocate(uint32_t ns, uint64_t slba, uint32_t nlb) "nsid %"PRIu32" slba 0x%"PRIx64" nlb %"PRIu32"" pci_nvme_do_aio(uint16_t cid, uint8_t opc, const char *opname, const char *blkname, int64_t offset, size_t len) "cid %"PRIu16" opc 0x%"PRIx8" opname '%s' blk '%s' offset %"PRId64" len %zu" pci_nvme_create_sq(uint64_t addr, uint16_t sqid, uint16_t cqid, uint16_t qsize, uint16_t qflags) "create submission queue, addr=0x%"PRIx64", sqid=%"PRIu16", cqid=%"PRIu16", qsize=%"PRIu16", qflags=%"PRIu16"" pci_nvme_create_cq(uint64_t addr, uint16_t cqid, uint16_t vector, uint16_t size, uint16_t qflags, int ien) "create completion queue, addr=0x%"PRIx64", cqid=%"PRIu16", vector=%"PRIu16", qsize=%"PRIu16", qflags=%"PRIu16", ien=%d"
@@ -134,6 +144,7 @@ pci_nvme_err_invalid_setfeat(uint32_t dw10) "invalid set features, dw10=0x%"PRIx pci_nvme_err_invalid_log_page(uint16_t cid, uint16_t lid) "cid %"PRIu16" lid 0x%"PRIx16"" pci_nvme_err_invalid_zone(uint16_t cid, uint64_t lba) "cid %"PRIu16" lba 0x%"PRIx64"" pci_nvme_err_invalid_zone_condition(uint16_t cid, uint64_t zslba, uint8_t condition) "cid %"PRIu16" zslba 0x%"PRIx64" condition 0x%"PRIx8"" +pci_nvme_err_invalid_zslba(uint16_t cid, uint64_t zslba) "cid %"PRIu16" zslba 0x%"PRIx64"" pci_nvme_err_startfail_cq(void) "nvme_start_ctrl failed because there are non-admin completion queues" pci_nvme_err_startfail_sq(void) "nvme_start_ctrl failed because there are non-admin submission queues" pci_nvme_err_startfail_nbarasq(void) "nvme_start_ctrl failed because the admin submission queue address is null"
Signature · 1 line
2.28.0
PATCH v2 14/14 hw/block/nvme: allow open to close transitions by controller
In reply to Klaus JensenFrom: Klaus Jensen <[email protected]>
Allow the controller to release open resources by transitioning implicitly and explicitly opened zones to closed. This is done using a naive "least recently opened" strategy.
--- hw/block/nvme-ns.h | 5 ++ hw/block/nvme-ns.c | 5 ++ hw/block/nvme.c | 105 +++++++++++++++++++++++++++++++++++------- hw/block/trace-events | 5 ++ 4 files changed, 103 insertions(+), 17 deletions(-)
hw/block/nvme-ns.h +5 -0
@@ -62,6 +62,8 @@ typedef struct NvmeZone {uint8_t *zde;uint64_t wp_staging; + +QTAILQ_ENTRY (NvmeZone) lru_entry; } NvmeZone; typedef struct NvmeNamespace {
@@ -101,6 +103,9 @@ typedef struct NvmeNamespace {struct {uint32_t open;uint32_t active; + + QTAILQ_HEAD(, NvmeZone) lru_open; + QTAILQ_HEAD(, NvmeZone) lru_active; } resources; } zns; } NvmeNamespace;
hw/block/nvme-ns.c +5 -0
@@ -225,6 +225,9 @@ void nvme_ns_zns_init_zone_state(NvmeNamespace *ns) ns->zns.resources.open = ns->params.zns.mor !=0xffffffff ? ns->params.zns.mor +1 : ns->zns.num_zones; +QTAILQ_INIT (&ns->zns.resources.lru_open); +QTAILQ_INIT (&ns->zns.resources.lru_active); +for (int i =0 ; i < ns->zns.num_zones; i++) {NvmeZone *zone = &ns->zns.zones[i]; zone->zd = &ns->zns.zd[i];
@@ -248,6 +251,8 @@ void nvme_ns_zns_init_zone_state(NvmeNamespace *ns)if (ns->zns.resources.active) { ns->zns.resources.active--; +QTAILQ_INSERT_TAIL (&ns->zns.resources.lru_active, zone, + lru_entry);continue ; }
hw/block/nvme.c +88 -17
238 lines not on this page · raw message
hw/block/trace-events +5 -0
@@ -88,6 +88,9 @@ pci_nvme_mmio_read(uint64_t addr) "addr 0x%"PRIx64"" pci_nvme_mmio_write(uint64_t addr, uint64_t data) "addr 0x%"PRIx64" data 0x%"PRIx64"" pci_nvme_mmio_doorbell_cq(uint16_t cqid, uint16_t new_head) "cqid %"PRIu16" new_head %"PRIu16"" pci_nvme_mmio_doorbell_sq(uint16_t sqid, uint16_t new_tail) "sqid %"PRIu16" new_tail %"PRIu16"" +pci_nvme_zone_zrm_transition(uint16_t cid, uint32_t nsid, uint64_t zslba, uint8_t from, uint8_t to) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64" from 0x%"PRIx8" to 0x%"PRIx8"" +pci_nvme_zone_zrm_candidate(uint16_t cid, uint32_t nsid, uint64_t zslba, uint64_t wp, uint8_t zc) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64" wp 0x%"PRIx64" zc 0x%"PRIx8"" +pci_nvme_zone_zrm_release_open(uint16_t cid, uint32_t nsid) "cid %"PRIu16" nsid %"PRIu32"" pci_nvme_mmio_intm_set(uint64_t data, uint64_t new_mask) "wrote MMIO, interrupt mask set, data=0x%"PRIx64", new_mask=0x%"PRIx64"" pci_nvme_mmio_intm_clr(uint64_t data, uint64_t new_mask) "wrote MMIO, interrupt mask clr, data=0x%"PRIx64", new_mask=0x%"PRIx64"" pci_nvme_mmio_cfg(uint64_t data) "wrote MMIO, config controller config=0x%"PRIx64""
@@ -115,6 +118,8 @@ pci_nvme_err_zone_is_read_only(uint16_t cid, uint64_t slba) "cid %"PRIu16" lba 0 pci_nvme_err_zone_invalid_write(uint16_t cid, uint64_t slba, uint64_t wp) "cid %"PRIu16" lba 0x%"PRIx64" wp 0x%"PRIx64"" pci_nvme_err_zone_boundary(uint16_t cid, uint64_t slba, uint32_t nlb, uint64_t zcap) "cid %"PRIu16" lba 0x%"PRIx64" nlb %"PRIu32" zcap 0x%"PRIx64"" pci_nvme_err_zone_pending_writes(uint16_t cid, uint64_t zslba, uint64_t wp, uint64_t wp_staging) "cid %"PRIu16" zslba 0x%"PRIx64" wp 0x%"PRIx64" wp_staging 0x%"PRIx64"" +pci_nvme_err_too_many_active_zones(uint16_t cid) "cid %"PRIu16"" +pci_nvme_err_too_many_open_zones(uint16_t cid) "cid %"PRIu16"" pci_nvme_err_invalid_sgld(uint16_t cid, uint8_t typ) "cid %"PRIu16" type 0x%"PRIx8"" pci_nvme_err_invalid_num_sgld(uint16_t cid, uint8_t typ) "cid %"PRIu16" type 0x%"PRIx8"" pci_nvme_err_invalid_sgl_excess_length(uint16_t cid) "cid %"PRIu16""
Signature · 1 line
2.28.0