hw/block/nvme: zoned namespace command set
v5 cover hw/block/nvme: zoned namespace command set
From: Klaus Jensen <[email protected]>
Oi,
Things seem to have slowed down a bit on the zoned front, so in the interest of moving things forward and catalyzing the process, I'm posting a v5 of my series (v4 never got posted, but it's in my repo for posterity, see below). I picked some preparatory patches from Dmitry and rebased on that.
These patches apply on top of the "pmr/cmb coexist", "compare" and "simple copy" series that I've posted previously.
An applied version is available here:
https://irrelevant.dk/g/pci-nvme.git/tag/?h=zoned-v5
Changed for v5
~~~~~~~~~~~~~~
* Rebased to work with the DULBE support, recently merged in nvme-next.
* "hw/block/nvme: Separate read and write handlers"
* "hw/block/nvme: Merge nvme_write_zeroes() with nvme_write()"
* "hw/block/nvme: Generate namespace UUIDs"
- Picked from Dmitry' series.
* "hw/block/nvme: add commands supported and effects log page"
* "hw/block/nvme: support namespace types"
- Refactored and cribbed Keith's approach to CSE from the kb-zns branch and getting rid of the ugly macro initializers.
* "hw/block/nvme: add persistence for zone info"
- Marked RFC for obvious reasons.
Changes for v4
~~~~~~~~~~~~~~
* "hw/block/nvme: add the zone management send command"
- Use a pwrite_zeroes for zone reset instead of a discard.
Changes for v3
~~~~~~~~~~~~~~
* Rebased on nvme-next with "[PATCH v2] hw/block/nvme: add dulbe support" applied.
* "hw/block/nvme: add support for dulbe and block utilization tracking"
- Dropped from this series. This series instead builds on the support for DULBE that I added in "[PATCH v2] hw/block/nvme: add dulbe support", previously posted.
* "hw/block/nvme: add the zone management send command"
- Use asynchronous discards.
* "hw/block/nvme: add basic read/write for zoned namespaces"
* "hw/block/nvme: add the zone management receive command"
* "hw/block/nvme: add the zone management send command"
* "hw/block/nvme: add the zone append command"
* "hw/block/nvme: track and enforce zone resources"
* "hw/block/nvme: allow open to close zone transitions by controller"
- In compliance with the concensus I dropped zone persistence support from all patches.
Changes for v2
~~~~~~~~~~~~~~
* "hw/block/nvme: add support for dulbe and block utilization tracking"
- Factor out pstate init/load into separate functions.
- Fixed a stupid off-by-1 bug that would trigger when resetting the last zone.
- I added a more formalized pstate file format that includes a header. This is pretty similar to what is done in Dmitry's series, but with fewer fields. The device parameters for nvme-ns are still the "authoritative" ones, so if any parameters that influence LBA size, number of zones, etc. do not match, an error indicating the discrepancy will be produced. IIRC, Dmitry's version does the same.
It is set up such that newer versions can load pstate files from older versions. The file format header is not unlike a standard nvme datastructure with reserved areas. This means that when adding new command sets that require persistent state, it is not needed to bump the version number, unless the header has to change dramatically. This is demonstrated when the zoned namespace command set support is added in "hw/block/nvme: add basic read/write for zoned namespaces".
* "hw/block/nvme: add basic read/write for zoned namespaces"
- The device will now transition all opened zones to Closed on "normal shutdown". You can force the "transition to Full" behavior by killing QEMU from the monitor.
* "hw/block/nvme: add the zone append command"
- Slightly reordered the logic so a LBA Out of Range error is returned before Invalid Field in Command for normal read/write commands.
* "hw/block/nvme: support zone active excursions"
- Dropped. Optional and non-critical.
* "hw/block/nvme: support reset/finish recommended limits"
- Dropped. Optional and non-critical.
Dmitry Fomichev (3):
hw/block/nvme: Separate read and write handlers hw/block/nvme: Merge nvme_write_zeroes() with nvme_write() hw/block/nvme: Generate namespace UUIDs
Gollu Appalanaidu (1):
hw/block/nvme: add commands supported and effects log page
Klaus Jensen (8):
hw/block/nvme: support namespace types
hw/block/nvme: add basic read/write for zoned namespaces hw/block/nvme: add the zone management receive command hw/block/nvme: add the zone management send command hw/block/nvme: add the zone append command hw/block/nvme: track and enforce zone resources hw/block/nvme: allow open to close zone transitions by controller hw/block/nvme: add persistence for zone info
docs/specs/nvme.txt | 32 + hw/block/nvme-ns.h | 127 +++- hw/block/nvme.h | 10 + include/block/nvme.h | 218 +++++- block/nvme.c | 4 +- hw/block/nvme-ns.c | 341 ++++++++- hw/block/nvme.c | 1613 ++++++++++++++++++++++++++++++++++++++--- hw/block/trace-events | 35 +- 8 files changed, 2246 insertions(+), 134 deletions(-)
Signature · 1 line
2.29.2
PATCH v5 1/12 hw/block/nvme: Separate read and write handlers
From: Dmitry Fomichev <[email protected]>
With ZNS support in place, the majority of code in nvme_rw() has become read- or write-specific. Move these parts to two separate handlers, nvme_read() and nvme_write() to make the code more readable and to remove multiple is_write checks that so far existed in the i/o path.
This is a refactoring patch, no change in functionality.
--- hw/block/nvme.c | 105 ++++++++++++++++++++++++++++-------------- hw/block/trace-events | 3 +- 2 files changed, 73 insertions(+), 35 deletions(-)
hw/block/nvme.c +71 -34
@@ -1394,6 +1394,61 @@ static uint16_t nvme_flush(NvmeCtrl *n, NvmeRequest *req)return NVME_NO_COMPLETE ; } +static uint16_t nvme_read (NvmeCtrl *n,NvmeRequest *req) +{ +NvmeRwCmd *rw = (NvmeRwCmd *)&req->cmd; +NvmeNamespace *ns = req->ns; +uint64_t slba =le64_to_cpu (rw->slba); +uint32_t nlb = (uint32_t )le16_to_cpu (rw->nlb) +1 ; +uint64_t data_size =nvme_l2b (ns, nlb); +uint64_t data_offset; +BlockBackend *blk = ns->blkconf.blk; +uint16_t status; + +trace_pci_nvme_read (nvme_cid (req),nvme_nsid (ns), nlb, data_size, slba); + + status =nvme_check_mdts (n, data_size); +if (status) { +trace_pci_nvme_err_mdts (nvme_cid (req), data_size); + goto invalid; + } + + status =nvme_check_bounds (ns, slba, nlb); +if (status) { +trace_pci_nvme_err_invalid_lba_range (slba, nlb, ns->id_ns.nsze); + goto invalid; + } + +if (NVME_ERR_REC_DULBE (ns->features.err_rec)) { + status =nvme_check_dulbe (ns, slba, nlb); +if (status) { + goto invalid; + } + } + + status =nvme_map_dptr (n, data_size, req); +if (status) { + goto invalid; + } + + data_offset =nvme_l2b (ns, slba); + +block_acct_start (blk_get_stats (blk), &req->acct, data_size, +BLOCK_ACCT_READ ); +if (req->qsg.sg) { + req->aiocb =dma_blk_read (blk, &req->qsg, data_offset, +BDRV_SECTOR_SIZE , nvme_rw_cb, req); + }else { + req->aiocb =blk_aio_preadv (blk, data_offset, &req->iov,0 , + nvme_rw_cb, req); + } +return NVME_NO_COMPLETE ; + +invalid: +block_acct_invalid (blk_get_stats (blk),BLOCK_ACCT_READ ); +return status; +} +static uint16_t nvme_write_zeroes (NvmeCtrl *n,NvmeRequest *req) {NvmeRwCmd *rw = (NvmeRwCmd *)&req->cmd;
@@ -1419,22 +1474,19 @@ static uint16_t nvme_write_zeroes(NvmeCtrl *n, NvmeRequest *req)return NVME_NO_COMPLETE ; } -static uint16_t nvme_rw (NvmeCtrl *n,NvmeRequest *req) +static uint16_t nvme_write (NvmeCtrl *n,NvmeRequest *req) {NvmeRwCmd *rw = (NvmeRwCmd *)&req->cmd;NvmeNamespace *ns = req->ns; -uint32_t nlb = (uint32_t )le16_to_cpu (rw->nlb) +1 ;uint64_t slba =le64_to_cpu (rw->slba); - +uint32_t nlb = (uint32_t )le16_to_cpu (rw->nlb) +1 ;uint64_t data_size =nvme_l2b (ns, nlb); -uint64_t data_offset =nvme_l2b (ns, slba); -enum BlockAcctType acct = req->cmd.opcode ==NVME_CMD_WRITE ? -BLOCK_ACCT_WRITE :BLOCK_ACCT_READ ; +uint64_t data_offset;BlockBackend *blk = ns->blkconf.blk;uint16_t status; -trace_pci_nvme_rw (nvme_cid (req),nvme_io_opc_str (rw->opcode), -nvme_nsid (ns), nlb, data_size, slba); +trace_pci_nvme_write (nvme_cid (req),nvme_io_opc_str (rw->opcode), +nvme_nsid (ns), nlb, data_size, slba); status =nvme_check_mdts (n, data_size);if (status) {
@@ -1448,42 +1500,26 @@ static uint16_t nvme_rw(NvmeCtrl *n, NvmeRequest *req) goto invalid; } -if (acct ==BLOCK_ACCT_READ ) { -if (NVME_ERR_REC_DULBE (ns->features.err_rec)) { - status =nvme_check_dulbe (ns, slba, nlb); -if (status) { - goto invalid; - } - } - } - status =nvme_map_dptr (n, data_size, req);if (status) { goto invalid; } -block_acct_start (blk_get_stats (blk), &req->acct, data_size, acct); + data_offset =nvme_l2b (ns, slba); + +block_acct_start (blk_get_stats (blk), &req->acct, data_size, +BLOCK_ACCT_WRITE );if (req->qsg.sg) { -if (acct ==BLOCK_ACCT_WRITE ) { - req->aiocb =dma_blk_write (blk, &req->qsg, data_offset, -BDRV_SECTOR_SIZE , nvme_rw_cb, req); - }else { - req->aiocb =dma_blk_read (blk, &req->qsg, data_offset, -BDRV_SECTOR_SIZE , nvme_rw_cb, req); - } + req->aiocb =dma_blk_write (blk, &req->qsg, data_offset, +BDRV_SECTOR_SIZE , nvme_rw_cb, req); }else { -if (acct ==BLOCK_ACCT_WRITE ) { - req->aiocb =blk_aio_pwritev (blk, data_offset, &req->iov,0 , - nvme_rw_cb, req); - }else { - req->aiocb =blk_aio_preadv (blk, data_offset, &req->iov,0 , - nvme_rw_cb, req); - } + req->aiocb =blk_aio_pwritev (blk, data_offset, &req->iov,0 , + nvme_rw_cb, req); }return NVME_NO_COMPLETE ; invalid: -block_acct_invalid (blk_get_stats (ns->blkconf.blk), acct); +block_acct_invalid (blk_get_stats (blk),BLOCK_ACCT_WRITE );return status; }
@@ -1513,8 +1549,9 @@ static uint16_t nvme_io_cmd(NvmeCtrl *n, NvmeRequest *req)case NVME_CMD_WRITE_ZEROES :return nvme_write_zeroes (n, req);case NVME_CMD_WRITE : +return nvme_write (n, req);case NVME_CMD_READ : -return nvme_rw (n, req); +return nvme_read (n, req);case NVME_CMD_COMPARE :return nvme_compare (n, req);case NVME_CMD_DSM :
hw/block/trace-events +2 -1
@@ -40,7 +40,8 @@ pci_nvme_map_prp(uint64_t trans_len, uint32_t len, uint64_t prp1, uint64_t prp2, pci_nvme_map_sgl(uint16_t cid, uint8_t typ, uint64_t len) "cid %"PRIu16" type 0x%"PRIx8" len %"PRIu64"" pci_nvme_io_cmd(uint16_t cid, uint32_t nsid, uint16_t sqid, uint8_t opcode, const char *opname) "cid %"PRIu16" nsid %"PRIu32" sqid %"PRIu16" opc 0x%"PRIx8" opname '%s'" pci_nvme_admin_cmd(uint16_t cid, uint16_t sqid, uint8_t opcode, const char *opname) "cid %"PRIu16" sqid %"PRIu16" opc 0x%"PRIx8" opname '%s'" -pci_nvme_rw(uint16_t cid, const char *verb, uint32_t nsid, uint32_t nlb, uint64_t count, uint64_t lba) "cid %"PRIu16" opname '%s' nsid %"PRIu32" nlb %"PRIu32" count %"PRIu64" lba 0x%"PRIx64"" +pci_nvme_read(uint16_t cid, uint32_t nsid, uint32_t nlb, uint64_t count, uint64_t lba) "cid %"PRIu16" nsid %"PRIu32" nlb %"PRIu32" count %"PRIu64" lba 0x%"PRIx64"" +pci_nvme_write(uint16_t cid, const char *verb, uint32_t nsid, uint32_t nlb, uint64_t count, uint64_t lba) "cid %"PRIu16" opname '%s' nsid %"PRIu32" nlb %"PRIu32" count %"PRIu64" lba 0x%"PRIx64"" pci_nvme_rw_cb(uint16_t cid, const char *blkname) "cid %"PRIu16" blk '%s'" pci_nvme_copy(uint16_t cid, uint32_t nsid, uint16_t nr, uint8_t format) "cid %"PRIu16" nsid %"PRIu32" nr %"PRIu16" format 0x%"PRIx8"" pci_nvme_copy_source_range(uint64_t slba, uint32_t nlb) "slba 0x%"PRIx64" nlb %"PRIu32""
Signature · 1 line
2.29.2
PATCH v5 2/12 hw/block/nvme: Merge nvme_write_zeroes() with nvme_write()
In reply to Klaus JensenFrom: Dmitry Fomichev <[email protected]>
nvme_write() now handles WRITE, WRITE ZEROES and ZONE_APPEND.
--- hw/block/nvme.c | 69 +++++++++++++++++-------------------------- hw/block/trace-events | 1 - 2 files changed, 27 insertions(+), 43 deletions(-)
hw/block/nvme.c +27 -42
@@ -1449,31 +1449,6 @@ invalid:return status; } -static uint16_t nvme_write_zeroes (NvmeCtrl *n,NvmeRequest *req) -{ -NvmeRwCmd *rw = (NvmeRwCmd *)&req->cmd; -NvmeNamespace *ns = req->ns; -uint64_t slba =le64_to_cpu (rw->slba); -uint32_t nlb = (uint32_t )le16_to_cpu (rw->nlb) +1 ; -uint64_t offset =nvme_l2b (ns, slba); -uint32_t count =nvme_l2b (ns, nlb); -uint16_t status; - -trace_pci_nvme_write_zeroes (nvme_cid (req),nvme_nsid (ns), slba, nlb); - - status =nvme_check_bounds (ns, slba, nlb); -if (status) { -trace_pci_nvme_err_invalid_lba_range (slba, nlb, ns->id_ns.nsze); -return status; - } - -block_acct_start (blk_get_stats (req->ns->blkconf.blk), &req->acct,0 , -BLOCK_ACCT_WRITE ); - req->aiocb =blk_aio_pwrite_zeroes (req->ns->blkconf.blk, offset, count, -BDRV_REQ_MAY_UNMAP , nvme_rw_cb, req); -return NVME_NO_COMPLETE ; -} -static uint16_t nvme_write (NvmeCtrl *n,NvmeRequest *req) {NvmeRwCmd *rw = (NvmeRwCmd *)&req->cmd;
@@ -1483,15 +1458,18 @@ static uint16_t nvme_write(NvmeCtrl *n, NvmeRequest *req)uint64_t data_size =nvme_l2b (ns, nlb);uint64_t data_offset;BlockBackend *blk = ns->blkconf.blk; +bool wrz = rw->opcode ==NVME_CMD_WRITE_ZEROES ;uint16_t status;trace_pci_nvme_write (nvme_cid (req),nvme_io_opc_str (rw->opcode),nvme_nsid (ns), nlb, data_size, slba); - status =nvme_check_mdts (n, data_size); -if (status) { -trace_pci_nvme_err_mdts (nvme_cid (req), data_size); - goto invalid; +if (!wrz) { + status =nvme_check_mdts (n, data_size); +if (status) { +trace_pci_nvme_err_mdts (nvme_cid (req), data_size); + goto invalid; + } } status =nvme_check_bounds (ns, slba, nlb);
@@ -1500,22 +1478,30 @@ static uint16_t nvme_write(NvmeCtrl *n, NvmeRequest *req) goto invalid; } - status =nvme_map_dptr (n, data_size, req); -if (status) { - goto invalid; - } - data_offset =nvme_l2b (ns, slba); -block_acct_start (blk_get_stats (blk), &req->acct, data_size, -BLOCK_ACCT_WRITE ); -if (req->qsg.sg) { - req->aiocb =dma_blk_write (blk, &req->qsg, data_offset, -BDRV_SECTOR_SIZE , nvme_rw_cb, req); +if (!wrz) { + status =nvme_map_dptr (n, data_size, req); +if (status) { + goto invalid; + } + +block_acct_start (blk_get_stats (blk), &req->acct, data_size, +BLOCK_ACCT_WRITE ); +if (req->qsg.sg) { + req->aiocb =dma_blk_write (blk, &req->qsg, data_offset, +BDRV_SECTOR_SIZE , nvme_rw_cb, req); + }else { + req->aiocb =blk_aio_pwritev (blk, data_offset, &req->iov,0 , + nvme_rw_cb, req); + } }else { - req->aiocb =blk_aio_pwritev (blk, data_offset, &req->iov,0 , - nvme_rw_cb, req); +block_acct_start (blk_get_stats (blk), &req->acct,0 ,BLOCK_ACCT_WRITE ); + req->aiocb =blk_aio_pwrite_zeroes (blk, data_offset, data_size, +BDRV_REQ_MAY_UNMAP , nvme_rw_cb, + req); } +return NVME_NO_COMPLETE ; invalid:
@@ -1547,7 +1533,6 @@ static uint16_t nvme_io_cmd(NvmeCtrl *n, NvmeRequest *req)case NVME_CMD_FLUSH :return nvme_flush (n, req);case NVME_CMD_WRITE_ZEROES : -return nvme_write_zeroes (n, req);case NVME_CMD_WRITE :return nvme_write (n, req);case NVME_CMD_READ :
hw/block/trace-events +0 -1
@@ -47,7 +47,6 @@ pci_nvme_copy(uint16_t cid, uint32_t nsid, uint16_t nr, uint8_t format) "cid %"P pci_nvme_copy_source_range(uint64_t slba, uint32_t nlb) "slba 0x%"PRIx64" nlb %"PRIu32"" pci_nvme_copy_in_complete(uint16_t cid) "cid %"PRIu16"" pci_nvme_copy_cb(uint16_t cid) "cid %"PRIu16"" -pci_nvme_write_zeroes(uint16_t cid, uint32_t nsid, uint64_t slba, uint32_t nlb) "cid %"PRIu16" nsid %"PRIu32" slba %"PRIu64" nlb %"PRIu32"" pci_nvme_block_status(int64_t offset, int64_t bytes, int64_t pnum, int ret, bool zeroed) "offset %"PRId64" bytes %"PRId64" pnum %"PRId64" ret 0x%x zeroed %d" pci_nvme_dsm(uint16_t cid, uint32_t nsid, uint32_t nr, uint32_t attr) "cid %"PRIu16" nsid %"PRIu32" nr %"PRIu32" attr 0x%"PRIx32"" pci_nvme_dsm_deallocate(uint16_t cid, uint32_t nsid, uint64_t slba, uint32_t nlb) "cid %"PRIu16" nsid %"PRIu32" slba %"PRIu64" nlb %"PRIu32""
Signature · 1 line
2.29.2
PATCH v5 3/12 hw/block/nvme: add commands supported and effects log page
In reply to Klaus JensenFrom: Gollu Appalanaidu <[email protected]>
This is to support for the Commands Supported and Effects log page. See NVM Express Spec 1.3d, sec. 5.14.1.5 ("Commands Supported and Effects")
--- include/block/nvme.h | 25 ++++++++++++++++-- hw/block/nvme.c | 61 +++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 83 insertions(+), 3 deletions(-)
include/block/nvme.h +23 -2
@@ -773,6 +773,24 @@ typedef struct QEMU_PACKED NvmeSmartLog {uint8_t reserved2[320 ]; } NvmeSmartLog; +typedef struct NvmeEffectsLog { +uint32_t acs[256 ]; +uint32_t iocs[256 ]; +uint8_t rsvd2048[2048 ]; +}NvmeEffectsLog ; + +enum { +NVME_EFFECTS_CSUPP =1 <<0 , +NVME_EFFECTS_LBCC =1 <<1 , +NVME_EFFECTS_NCC =1 <<2 , +NVME_EFFECTS_NIC =1 <<3 , +NVME_EFFECTS_CCC =1 <<4 , +NVME_EFFECTS_CSE_SINGLE =1 <<16 , +NVME_EFFECTS_CSE_MULTI =1 <<17 , +NVME_EFFECTS_CSE_MASK =3 <<16 , +NVME_EFFECTS_UUID_SEL =1 <<19 , +}; +enum NvmeSmartWarn {NVME_SMART_SPARE =1 <<0 ,NVME_SMART_TEMPERATURE =1 <<1 ,
@@ -785,6 +803,7 @@ enum NvmeLogIdentifier {NVME_LOG_ERROR_INFO =0x01 ,NVME_LOG_SMART_INFO =0x02 ,NVME_LOG_FW_SLOT_INFO =0x03 , + NVME_LOG_EFFECTS = 0x05, }; typedef struct QEMU_PACKED NvmePSD {
@@ -901,8 +920,9 @@ enum NvmeIdCtrlFrmw { };enum NvmeIdCtrlLpa { -NVME_LPA_NS_SMART =1 <<0 , -NVME_LPA_EXTENDED =1 <<2 , +NVME_LPA_NS_SMART =1 <<0 , +NVME_LPA_EFFECTS_LOG =1 <<1 , +NVME_LPA_EXTENDED =1 <<2 , };#define NVME_CTRL_SQES_MIN (sqes) ((sqes) & 0xf)
@@ -1119,5 +1139,6 @@ static inline void _nvme_check_size(void)QEMU_BUILD_BUG_ON (sizeof (NvmeIdNs) !=4096 );QEMU_BUILD_BUG_ON (sizeof (NvmeSglDescriptor) !=16 );QEMU_BUILD_BUG_ON (sizeof (NvmeIdNsDescr) !=4 ); + QEMU_BUILD_BUG_ON(sizeof(NvmeEffectsLog) != 4096); }#endif
hw/block/nvme.c +60 -1
79 lines not on this page · raw message
Signature · 1 line
2.29.2
PATCH v5 4/12 hw/block/nvme: Generate namespace UUIDs
In reply to Klaus JensenFrom: Dmitry Fomichev <[email protected]>
In NVMe 1.4, a namespace must report an ID descriptor of UUID type if it doesn't support EUI64 or NGUID. Add a new namespace property, "uuid", that provides the user the option to either specify the UUID explicitly or have a UUID generated automatically every time a namespace is initialized.
--- hw/block/nvme-ns.h | 1 + hw/block/nvme-ns.c | 1 + hw/block/nvme.c | 9 +++++---- 3 files changed, 7 insertions(+), 4 deletions(-)
hw/block/nvme-ns.h +1 -0
@@ -21,6 +21,7 @@ typedef struct NvmeNamespaceParams { uint32_t nsid; + QemuUUID uuid; uint16_t mssrl; uint32_t mcl;
hw/block/nvme-ns.c +1 -0
@@ -155,6 +155,7 @@ static void nvme_ns_realize(DeviceState *dev, Error **errp)static Property nvme_ns_props[] = {DEFINE_BLOCK_PROPERTIES (NvmeNamespace, blkconf),DEFINE_PROP_UINT32 ("nsid" , NvmeNamespace, params.nsid,0 ), +DEFINE_PROP_UUID ("uuid" , NvmeNamespace, params.uuid),DEFINE_PROP_UINT16 ("mssrl" , NvmeNamespace, params.mssrl,128 ),DEFINE_PROP_UINT32 ("mcl" , NvmeNamespace, params.mcl,128 ),DEFINE_PROP_UINT8 ("msrc" , NvmeNamespace, params.msrc,255 ),
hw/block/nvme.c +5 -4
@@ -2068,6 +2068,7 @@ static uint16_t nvme_identify_nslist(NvmeCtrl *n, NvmeRequest *req)static uint16_t nvme_identify_ns_descr_list (NvmeCtrl *n,NvmeRequest *req) { +NvmeNamespace *ns;NvmeIdentify *c = (NvmeIdentify *)&req->cmd;uint32_t nsid =le32_to_cpu (c->nsid);uint8_t list[NVME_IDENTIFY_DATA_SIZE ];
@@ -2087,7 +2088,8 @@ static uint16_t nvme_identify_ns_descr_list(NvmeCtrl *n, NvmeRequest *req)return NVME_INVALID_NSID |NVME_DNR ; } -if (unlikely (!nvme_ns (n, nsid))) { + ns =nvme_ns (n, nsid); +if (unlikely (!ns)) {return NVME_INVALID_FIELD |NVME_DNR ; }
@@ -2096,12 +2098,11 @@ static uint16_t nvme_identify_ns_descr_list(NvmeCtrl *n, NvmeRequest *req)/* * Because the NGUID and EUI64 fields are 0 in the Identify Namespace data * structure, a Namespace UUID (nidt = 0x3) must be reported in the -* Namespace Identification Descriptor. Add a very basic Namespace UUID -* here. +* Namespace Identification Descriptor. Add the namespace UUID here. */ ns_descrs->uuid.hdr.nidt =NVME_NIDT_UUID ; ns_descrs->uuid.hdr.nidl =NVME_NIDT_UUID_LEN ; -stl_be_p (&ns_descrs->uuid.v, nsid); +memcpy (&ns_descrs->uuid.v, ns->params.uuid.data,NVME_NIDT_UUID_LEN );return nvme_dma (n, list,NVME_IDENTIFY_DATA_SIZE ,DMA_DIRECTION_FROM_DEVICE , req);
Signature · 1 line
2.29.2
PATCH v5 5/12 hw/block/nvme: support namespace types
In reply to Klaus JensenFrom: Klaus Jensen <[email protected]>
Implement support for TP 4056 ("Namespace Types"). This adds the 'iocs' (I/O Command Set) device parameter to the nvme-ns device.
--- docs/specs/nvme.txt | 3 + hw/block/nvme-ns.h | 12 ++- hw/block/nvme.h | 3 + include/block/nvme.h | 52 ++++++++--- block/nvme.c | 4 +- hw/block/nvme-ns.c | 20 +++- hw/block/nvme.c | 209 +++++++++++++++++++++++++++++++++++------- hw/block/trace-events | 6 +- 8 files changed, 256 insertions(+), 53 deletions(-)
docs/specs/nvme.txt +3 -0
@@ -3,6 +3,9 @@ NVM Express Controller The nvme device (-device nvme) emulates an NVM Express Controller. + `iocs`; The "I/O Command Set" associated with the namespace. E.g. 0x0 for the + NVM Command Set (the default), or 0x2 for the Zoned Namespace Command Set. + Reference Specifications ------------------------
hw/block/nvme-ns.h +10 -2
@@ -21,6 +21,7 @@ typedef struct NvmeNamespaceParams { uint32_t nsid; + uint8_t iocs; QemuUUID uuid; uint16_t mssrl;
@@ -33,7 +34,8 @@ typedef struct NvmeNamespace {BlockConf blkconf;int32_t bootindex;int64_t size; -NvmeIdNs id_ns; +uint8_t iocs; +void *id_ns[NVME_IOCS_MAX ];NvmeNamespaceParams params;
@@ -53,7 +55,7 @@ static inline uint32_t nvme_nsid(NvmeNamespace *ns)static inline NvmeLBAF *nvme_ns_lbaf (NvmeNamespace *ns) { -NvmeIdNs *id_ns = &ns->id_ns; +NvmeIdNsNvm *id_ns = ns->id_ns[NVME_IOCS_NVM ];return &id_ns->lbaf[NVME_ID_NS_FLBAS_INDEX (id_ns->flbas)]; }
@@ -68,6 +70,12 @@ static inline uint64_t nvme_ns_nlbas(NvmeNamespace *ns)return ns->size >>nvme_ns_lbads (ns); } +static inline uint64_t nvme_ns_nsze (NvmeNamespace *ns) +{ +NvmeIdNsNvm *id_ns = ns->id_ns[NVME_IOCS_NVM ]; +return le64_to_cpu (id_ns->nsze); +} +/* convert an LBA to the equivalent in bytes */ static inline size_t nvme_l2b (NvmeNamespace *ns,uint64_t lba) {
hw/block/nvme.h +3 -0
@@ -115,6 +115,7 @@ typedef struct NvmeFeatureVal { };uint32_t async_config;uint32_t vwc; +uint32_t iocsci; } NvmeFeatureVal; typedef struct NvmeCtrl {
@@ -143,6 +144,7 @@ typedef struct NvmeCtrl {uint64_t timestamp_set_qemu_clock_ms;/* QEMU clock time */ uint64_t starttime_ms;uint16_t temperature; +uint64_t iocscs[512 ];HostMemoryBackend *pmrdev;
@@ -158,6 +160,7 @@ typedef struct NvmeCtrl {NvmeSQueue admin_sq;NvmeCQueue admin_cq;NvmeIdCtrl id_ctrl; +void *id_ctrl_iocss[NVME_IOCS_MAX ];NvmeFeatureVal features; } NvmeCtrl;
include/block/nvme.h +41 -11
130 lines not on this page · raw message
block/nvme.c +2 -2
@@ -337,7 +337,7 @@ static inline int nvme_translate_error(const NvmeCqe *c) {uint16_t status = (le16_to_cpu (c->status) >>1 ) &0xFF ;if (status) { - trace_nvme_error(le32_to_cpu(c->result), + trace_nvme_error(le32_to_cpu(c->dw0), le16_to_cpu(c->sq_head), le16_to_cpu(c->sq_id), le16_to_cpu(c->cid),
@@ -507,7 +507,7 @@ static void nvme_identify(BlockDriverState *bs, int namespace, Error **errp)BDRVNVMeState *s = bs->opaque;union {NvmeIdCtrl ctrl; -NvmeIdNs ns; +NvmeIdNsNvm ns; } *id;NvmeLBAF *lbaf;uint16_t oncs;
hw/block/nvme-ns.c +17 -3
@@ -33,11 +33,16 @@static int nvme_ns_init (NvmeNamespace *ns,Error **errp) {BlockDriverInfo bdi; -NvmeIdNs *id_ns = &ns->id_ns; -int lba_index =NVME_ID_NS_FLBAS_INDEX (ns->id_ns.flbas); +NvmeIdNsNvm *id_ns; +int lba_index;int npdg; - ns->id_ns.dlfeat =0x9 ; + id_ns = ns->id_ns[NVME_IOCS_NVM ] =g_new0 (NvmeIdNsNvm,1 ); + + ns->iocs = ns->params.iocs; + lba_index =NVME_ID_NS_FLBAS_INDEX (id_ns->flbas); + + id_ns->dlfeat =0x9 ; id_ns->lbaf[lba_index].ds =31 -clz32 (ns->blkconf.logical_block_size);
@@ -104,6 +109,14 @@ static int nvme_ns_check_constraints(NvmeNamespace *ns, Error **errp)return -1 ; } +switch (ns->params.iocs) { +case NVME_IOCS_NVM : +break ; +default : +error_setg (errp,"unsupported iocs" ); +return -1 ; + } +return 0 ; }
@@ -155,6 +168,7 @@ static void nvme_ns_realize(DeviceState *dev, Error **errp)static Property nvme_ns_props[] = {DEFINE_BLOCK_PROPERTIES (NvmeNamespace, blkconf),DEFINE_PROP_UINT32 ("nsid" , NvmeNamespace, params.nsid,0 ), +DEFINE_PROP_UINT8 ("iocs" , NvmeNamespace, params.iocs,NVME_IOCS_NVM ),DEFINE_PROP_UUID ("uuid" , NvmeNamespace, params.uuid),DEFINE_PROP_UINT16 ("mssrl" , NvmeNamespace, params.mssrl,128 ),DEFINE_PROP_UINT32 ("mcl" , NvmeNamespace, params.mcl,128 ),
hw/block/nvme.c +176 -33
436 lines not on this page · raw message
hw/block/trace-events +4 -2
@@ -58,10 +58,12 @@ pci_nvme_create_sq(uint64_t addr, uint16_t sqid, uint16_t cqid, uint16_t qsize, pci_nvme_create_cq(uint64_t addr, uint16_t cqid, uint16_t vector, uint16_t size, uint16_t qflags, int ien) "create completion queue, addr=0x%"PRIx64", cqid=%"PRIu16", vector=%"PRIu16", qsize=%"PRIu16", qflags=%"PRIu16", ien=%d" pci_nvme_del_sq(uint16_t qid) "deleting submission queue sqid=%"PRIu16"" pci_nvme_del_cq(uint16_t cqid) "deleted completion queue, cqid=%"PRIu16"" +pci_nvme_identify(uint16_t cid, uint32_t nsid, uint16_t cntid, uint8_t cns, uint8_t csi, uint16_t nvmsetid) "cid %"PRIu16" nsid %"PRIu32" cntid 0x%"PRIx16" cns 0x%"PRIx8" csi 0x%"PRIx8" nvmsetid %"PRIu16"" pci_nvme_identify_ctrl(void) "identify controller" -pci_nvme_identify_ns(uint32_t ns) "nsid %"PRIu32"" -pci_nvme_identify_nslist(uint32_t ns) "nsid %"PRIu32"" +pci_nvme_identify_ns(uint32_t ns, uint8_t csi) "nsid %"PRIu32" csi 0x%"PRIx8"" +pci_nvme_identify_nslist(uint32_t ns, uint8_t csi) "nsid %"PRIu32" csi 0x%"PRIx8"" pci_nvme_identify_ns_descr_list(uint32_t ns) "nsid %"PRIu32"" +pci_nvme_identify_io_cmd_set(uint16_t cid) "cid %"PRIu16"" pci_nvme_get_log(uint16_t cid, uint8_t lid, uint8_t lsp, uint8_t rae, uint32_t len, uint64_t off) "cid %"PRIu16" lid 0x%"PRIx8" lsp 0x%"PRIx8" rae 0x%"PRIx8" len %"PRIu32" off %"PRIu64"" pci_nvme_getfeat(uint16_t cid, uint32_t nsid, uint8_t fid, uint8_t sel, uint32_t cdw11) "cid %"PRIu16" nsid 0x%"PRIx32" fid 0x%"PRIx8" sel 0x%"PRIx8" cdw11 0x%"PRIx32"" pci_nvme_setfeat(uint16_t cid, uint32_t nsid, uint8_t fid, uint8_t save, uint32_t cdw11) "cid %"PRIu16" nsid 0x%"PRIx32" fid 0x%"PRIx8" save 0x%"PRIx8" cdw11 0x%"PRIx32""
Signature · 1 line
2.29.2
PATCH v5 6/12 hw/block/nvme: add basic read/write for zoned namespaces
In reply to Klaus JensenFrom: Klaus Jensen <[email protected]>
This adds basic read and write for zoned namespaces.
A zoned namespace is created by setting the iocs namespace parameter to 0x2 and specifying the zns.zcap parameter (zone capacity) in number of logical blocks per zone. If a zone size (zns.zsze) is not specified, the namespace device will set the zone size to be the next power of two and fit in as many zones as possible on the underlying namespace blockdev. This behavior is not required by the specification, but ensures that the device can be initialized by the Linux kernel nvme driver, which requires a power of two zone size.
--- docs/specs/nvme.txt | 8 + hw/block/nvme-ns.h | 78 ++++++++ include/block/nvme.h | 60 +++++- hw/block/nvme-ns.c | 86 +++++++++ hw/block/nvme.c | 415 ++++++++++++++++++++++++++++++++++++++++-- hw/block/trace-events | 8 + 6 files changed, 635 insertions(+), 20 deletions(-)
docs/specs/nvme.txt +8 -0
@@ -6,6 +6,14 @@ The nvme device (-device nvme) emulates an NVM Express Controller. `iocs`; The "I/O Command Set" associated with the namespace. E.g. 0x0 for the NVM Command Set (the default), or 0x2 for the Zoned Namespace Command Set. + `zns.zcap`; If `iocs` is 0x2, this specifies the zone capacity. It is + specified in units of logical blocks. + + `zns.zsze`; If `iocs` is 0x2, this specifies the zone size. It is specified + in units of the logical blocks. If not specified, the value depends on + zns.zcap; if the zone capacity is a power of two, the zone size will be + set to that, otherwise it will default to the next power of two. + Reference Specifications ------------------------
hw/block/nvme-ns.h +78 -0
113 lines not on this page · raw message
include/block/nvme.h +57 -3
100 lines not on this page · raw message
hw/block/nvme-ns.c +86 -0
116 lines not on this page · raw message
hw/block/nvme.c +398 -17
587 lines not on this page · raw message
hw/block/trace-events +8 -0
@@ -82,6 +82,8 @@ pci_nvme_enqueue_event_noqueue(int queued) "queued %d" pci_nvme_enqueue_event_masked(uint8_t typ) "type 0x%"PRIx8"" pci_nvme_no_outstanding_aers(void) "ignoring event; no outstanding AERs" pci_nvme_enqueue_req_completion(uint16_t cid, uint16_t cqid, uint16_t status) "cid %"PRIu16" cqid %"PRIu16" status 0x%"PRIx16"" +pci_nvme_zrm_transition(uint32_t nsid, uint64_t zslba, const char *s_from, uint8_t from, const char *s_to, uint8_t to) "nsid %"PRIu32" zslba 0x%"PRIx64" from '%s' (%"PRIu8") to '%s' (%"PRIu8")" +pci_nvme_zns_advance_wp(uint32_t nsid, uint64_t zslba, uint64_t wp_orig, uint32_t nlb) "nsid 0x%"PRIx32" zslba 0x%"PRIx64" wp_orig 0x%"PRIx64" nlb %"PRIu32"" pci_nvme_mmio_read(uint64_t addr) "addr 0x%"PRIx64"" pci_nvme_mmio_write(uint64_t addr, uint64_t data) "addr 0x%"PRIx64" data 0x%"PRIx64"" pci_nvme_mmio_doorbell_cq(uint16_t cqid, uint16_t new_head) "cqid %"PRIu16" new_head %"PRIu16""
@@ -107,6 +109,11 @@ pci_nvme_err_addr_write(uint64_t addr) "addr 0x%"PRIx64"" pci_nvme_err_cfs(void) "controller fatal status" pci_nvme_err_aio(uint16_t cid, const char *errname, uint16_t status) "cid %"PRIu16" err '%s' status 0x%"PRIx16"" pci_nvme_err_copy_invalid_format(uint8_t format) "format 0x%"PRIx8"" +pci_nvme_err_zone_is_full(uint64_t zslba) "zslba 0x%"PRIx64"" +pci_nvme_err_zone_is_read_only(uint64_t zslba) "zslba 0x%"PRIx64"" +pci_nvme_err_zone_is_offline(uint64_t zslba) "zslba 0x%"PRIx64"" +pci_nvme_err_zone_invalid_write(uint64_t slba, uint64_t wp) "lba 0x%"PRIx64" wp 0x%"PRIx64"" +pci_nvme_err_zone_boundary(uint64_t slba, uint32_t nlb, uint64_t zcap) "lba 0x%"PRIx64" nlb %"PRIu32" zcap 0x%"PRIx64"" pci_nvme_err_invalid_sgld(uint16_t cid, uint8_t typ) "cid %"PRIu16" type 0x%"PRIx8"" pci_nvme_err_invalid_num_sgld(uint16_t cid, uint8_t typ) "cid %"PRIu16" type 0x%"PRIx8"" pci_nvme_err_invalid_sgl_excess_length(uint16_t cid) "cid %"PRIu16""
@@ -133,6 +140,7 @@ pci_nvme_err_invalid_identify_cns(uint16_t cns) "identify, invalid cns=0x%"PRIx1 pci_nvme_err_invalid_getfeat(int dw10) "invalid get features, dw10=0x%"PRIx32"" pci_nvme_err_invalid_setfeat(uint32_t dw10) "invalid set features, dw10=0x%"PRIx32"" pci_nvme_err_invalid_log_page(uint16_t cid, uint16_t lid) "cid %"PRIu16" lid 0x%"PRIx16"" +pci_nvme_err_invalid_zone_state(uint64_t zslba, const char *zs_str, uint8_t zs) "zslba 0x%"PRIx64" zs '%s' (%"PRIu8")" pci_nvme_err_startfail_cq(void) "nvme_start_ctrl failed because there are non-admin completion queues" pci_nvme_err_startfail_sq(void) "nvme_start_ctrl failed because there are non-admin submission queues" pci_nvme_err_startfail_nbarasq(void) "nvme_start_ctrl failed because the admin submission queue address is null"
Signature · 1 line
2.29.2
PATCH v5 7/12 hw/block/nvme: add the zone management receive command
In reply to Klaus JensenFrom: Klaus Jensen <[email protected]>
Add the Zone Management Receive command.
--- hw/block/nvme-ns.h | 8 +++ hw/block/nvme.h | 1 + include/block/nvme.h | 46 +++++++++++++ hw/block/nvme-ns.c | 8 +++ hw/block/nvme.c | 150 ++++++++++++++++++++++++++++++++++++++++++ hw/block/trace-events | 1 + 6 files changed, 214 insertions(+)
hw/block/nvme-ns.h +8 -0
@@ -31,11 +31,13 @@ typedef struct NvmeNamespaceParams {struct {uint64_t zcap;uint64_t zsze; +uint8_t zdes; } zns; } NvmeNamespaceParams;typedef struct NvmeZone {NvmeZoneDescriptor *zd; +uint8_t *zde;uint64_t wp_staging; }NvmeZone ;
@@ -59,6 +61,7 @@ typedef struct NvmeNamespace {NvmeZone *zones;NvmeZoneDescriptor *zd; +uint8_t *zde; } zns; } NvmeNamespace;
@@ -99,6 +102,11 @@ static inline uint64_t nvme_ns_zsze(NvmeNamespace *ns)return nvme_ns_lbafe (ns)->zsze; } +static inline size_t nvme_ns_zdes_bytes (NvmeNamespace *ns) +{ +return ns->params.zns.zdes <<6 ; +} +/* calculate the number of LBAs that the namespace can accomodate */ static inline uint64_t nvme_ns_nlbas (NvmeNamespace *ns) {
hw/block/nvme.h +1 -0
@@ -63,6 +63,7 @@ static inline const char *nvme_io_opc_str(uint8_t opc)case NVME_CMD_WRITE_ZEROES :return "NVME_NVM_CMD_WRITE_ZEROES" ;case NVME_CMD_DSM :return "NVME_NVM_CMD_DSM" ;case NVME_CMD_COPY :return "NVME_NVM_CMD_COPY" ; +case NVME_CMD_ZONE_MGMT_RECV :return "NVME_ZONED_CMD_ZONE_MGMT_RECV" ;default :return "NVME_NVM_CMD_UNKNOWN" ; } }
include/block/nvme.h +46 -0
70 lines not on this page · raw message
hw/block/nvme-ns.c +8 -0
@@ -61,6 +61,9 @@ static void nvme_ns_zns_init_zones(NvmeNamespace *ns) zone = &ns->zns.zones[i]; zone->zd = &ns->zns.zd[i]; +if (ns->params.zns.zdes) { + zone->zde = &ns->zns.zde[i]; + } zone->wp_staging = zslba; zd = zone->zd;
@@ -81,11 +84,15 @@ static void nvme_ns_init_zoned(NvmeNamespace *ns) id_ns_zns->lbafe[i].zsze = ns->params.zns.zsze ?cpu_to_le64 (ns->params.zns.zsze) :cpu_to_le64 (pow2ceil (ns->params.zns.zcap)); + id_ns_zns->lbafe[i].zdes = ns->params.zns.zdes; } ns->zns.num_zones =nvme_ns_nlbas (ns) /nvme_ns_zsze (ns); ns->zns.zones =g_malloc0_n (ns->zns.num_zones,sizeof (NvmeZone)); ns->zns.zd =g_malloc0_n (ns->zns.num_zones,sizeof (NvmeZoneDescriptor)); +if (ns->params.zns.zdes) { + ns->zns.zde =g_malloc0_n (ns->zns.num_zones,nvme_ns_zdes_bytes (ns)); + } id_ns_zns->mar =0xffffffff ; id_ns_zns->mor =0xffffffff ;
@@ -259,6 +266,7 @@ static Property nvme_ns_props[] = {DEFINE_PROP_UINT8 ("msrc" , NvmeNamespace, params.msrc,255 ),DEFINE_PROP_UINT64 ("zns.zcap" , NvmeNamespace, params.zns.zcap,0 ),DEFINE_PROP_UINT64 ("zns.zsze" , NvmeNamespace, params.zns.zsze,0 ), +DEFINE_PROP_UINT8 ("zns.zdes" , NvmeNamespace, params.zns.zdes,0 ), DEFINE_PROP_END_OF_LIST(), };
hw/block/nvme.c +150 -0
210 lines not on this page · raw message
hw/block/trace-events +1 -0
@@ -54,6 +54,7 @@ pci_nvme_compare(uint16_t cid, uint32_t nsid, uint64_t slba, uint32_t nlb) "cid pci_nvme_compare_cb(uint16_t cid) "cid %"PRIu16"" pci_nvme_aio_discard_cb(uint16_t cid) "cid %"PRIu16"" pci_nvme_aio_copy_in_cb(uint16_t cid) "cid %"PRIu16"" +pci_nvme_zone_mgmt_recv(uint16_t cid, uint32_t nsid, uint64_t slba, uint64_t len, uint8_t zra, uint8_t zfeat, uint8_t zflags) "cid %"PRIu16" nsid %"PRIu32" slba 0x%"PRIx64" len %"PRIu64" zra 0x%"PRIx8" zrasf 0x%"PRIx8" pr 0x%"PRIx8"" pci_nvme_create_sq(uint64_t addr, uint16_t sqid, uint16_t cqid, uint16_t qsize, uint16_t qflags) "create submission queue, addr=0x%"PRIx64", sqid=%"PRIu16", cqid=%"PRIu16", qsize=%"PRIu16", qflags=%"PRIu16"" pci_nvme_create_cq(uint64_t addr, uint16_t cqid, uint16_t vector, uint16_t size, uint16_t qflags, int ien) "create completion queue, addr=0x%"PRIx64", cqid=%"PRIu16", vector=%"PRIu16", qsize=%"PRIu16", qflags=%"PRIu16", ien=%d" pci_nvme_del_sq(uint16_t qid) "deleting submission queue sqid=%"PRIu16""
Signature · 1 line
2.29.2
PATCH v5 8/12 hw/block/nvme: add the zone management send command
In reply to Klaus JensenFrom: Klaus Jensen <[email protected]>
Add the Zone Management Send command.
The spec specifies that "All logical blocks in a zone *shall* be marked as deallocated when [the zone is reset]". Since the device guarantees 0x00 to be read from deallocated blocks we have to issue a pwrite_zeroes since we cannot be sure that a discard actually does anything. Typically, this will be converted to an unmap/discard operation, but if the underlying blockdev doesn't support that a reset can be pretty expensive.
--- hw/block/nvme.h | 1 + include/block/nvme.h | 30 ++++ hw/block/nvme.c | 383 ++++++++++++++++++++++++++++++++++++++++++ hw/block/trace-events | 12 ++ 4 files changed, 426 insertions(+)
hw/block/nvme.h +1 -0
@@ -63,6 +63,7 @@ static inline const char *nvme_io_opc_str(uint8_t opc)case NVME_CMD_WRITE_ZEROES :return "NVME_NVM_CMD_WRITE_ZEROES" ;case NVME_CMD_DSM :return "NVME_NVM_CMD_DSM" ;case NVME_CMD_COPY :return "NVME_NVM_CMD_COPY" ; +case NVME_CMD_ZONE_MGMT_SEND :return "NVME_ZONED_CMD_ZONE_MGMT_SEND" ;case NVME_CMD_ZONE_MGMT_RECV :return "NVME_ZONED_CMD_ZONE_MGMT_RECV" ;default :return "NVME_NVM_CMD_UNKNOWN" ; }
include/block/nvme.h +30 -0
@@ -485,6 +485,7 @@ enum NvmeIoCommands {NVME_CMD_WRITE_ZEROES =0x08 ,NVME_CMD_DSM =0x09 ,NVME_CMD_COPY =0x19 , +NVME_CMD_ZONE_MGMT_SEND =0x79 , NVME_CMD_ZONE_MGMT_RECV = 0x7a, };
@@ -598,6 +599,34 @@ enum { NVME_RW_PRINFO_PRCHK_REF = 1 << 10, }; +typedef struct QEMU_PACKED NvmeZoneMgmtSendCmd { +uint8_t opcode; +uint8_t flags; +uint16_t cid; +uint32_t nsid; +uint32_t rsvd8[4 ]; +NvmeCmdDptr dptr; +uint64_t slba; +uint32_t rsvd48; +uint8_t zsa; +uint8_t select_all; +uint8_t rsvd54[2 ]; +uint32_t rsvd56[2 ]; +} NvmeZoneMgmtSendCmd; + +enum { +NVME_ZMS_SELECT_ALL =0x1 , +}; + +enum { +NVME_ZMS_CLOSE =0x1 , +NVME_ZMS_FINISH =0x2 , +NVME_ZMS_OPEN =0x3 , +NVME_ZMS_RESET =0x4 , +NVME_ZMS_OFFLINE =0x5 , +NVME_ZMS_SET_ZDE =0x10 , +}; + typedef struct QEMU_PACKED NvmeZoneMgmtRecvCmd {uint8_t opcode;uint8_t flags;
@@ -1257,6 +1286,7 @@ static inline void _nvme_check_size(void)QEMU_BUILD_BUG_ON (sizeof (NvmeRwCmd) !=64 );QEMU_BUILD_BUG_ON (sizeof (NvmeDsmCmd) !=64 );QEMU_BUILD_BUG_ON (sizeof (NvmeCopyCmd) !=64 ); +QEMU_BUILD_BUG_ON (sizeof (NvmeZoneMgmtSendCmd) !=64 );QEMU_BUILD_BUG_ON (sizeof (NvmeZoneMgmtRecvCmd) !=64 );QEMU_BUILD_BUG_ON (sizeof (NvmeRangeType) !=64 );QEMU_BUILD_BUG_ON (sizeof (NvmeErrorLog) !=64 );
hw/block/nvme.c +383 -0
425 lines not on this page · raw message
hw/block/trace-events +12 -0
@@ -54,6 +54,16 @@ pci_nvme_compare(uint16_t cid, uint32_t nsid, uint64_t slba, uint32_t nlb) "cid pci_nvme_compare_cb(uint16_t cid) "cid %"PRIu16"" pci_nvme_aio_discard_cb(uint16_t cid) "cid %"PRIu16"" pci_nvme_aio_copy_in_cb(uint16_t cid) "cid %"PRIu16"" +pci_nvme_aio_zone_reset_cb(uint16_t cid, uint64_t zslba) "cid %"PRIu16" zslba 0x%"PRIx64"" +pci_nvme_zone_mgmt_send(uint16_t cid, uint32_t nsid, uint64_t zslba, uint8_t zsa, uint8_t select_all) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64" zsa 0x%"PRIx8" select_all 0x%"PRIx8"" +pci_nvme_zone_mgmt_send_all(uint16_t cid, uint32_t nsid, uint8_t za) "cid %"PRIu16" nsid %"PRIu32" za 0x%"PRIx8"" +pci_nvme_zone_mgmt_send_close(uint16_t cid, uint32_t nsid, uint64_t zslba, const char *zc) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64" zc \"%s\"" +pci_nvme_zone_mgmt_send_finish(uint16_t cid, uint32_t nsid, uint64_t zslba, const char *zc) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64" zc \"%s\"" +pci_nvme_zone_mgmt_send_open(uint16_t cid, uint32_t nsid, uint64_t zslba, const char *zc) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64" zc \"%s\"" +pci_nvme_zone_mgmt_send_reset(uint16_t cid, uint32_t nsid, uint64_t zslba, const char *zc) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64" zc \"%s\"" +pci_nvme_zone_mgmt_send_reset_cb(uint16_t cid, uint32_t nsid) "cid %"PRIu16" nsid %"PRIu32"" +pci_nvme_zone_mgmt_send_offline(uint16_t cid, uint32_t nsid, uint64_t zslba, const char *zc) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64" zc \"%s\"" +pci_nvme_zone_mgmt_send_set_zde(uint16_t cid, uint32_t nsid, uint64_t zslba, const char *zc) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64" zc \"%s\"" pci_nvme_zone_mgmt_recv(uint16_t cid, uint32_t nsid, uint64_t slba, uint64_t len, uint8_t zra, uint8_t zfeat, uint8_t zflags) "cid %"PRIu16" nsid %"PRIu32" slba 0x%"PRIx64" len %"PRIu64" zra 0x%"PRIx8" zrasf 0x%"PRIx8" pr 0x%"PRIx8"" pci_nvme_create_sq(uint64_t addr, uint16_t sqid, uint16_t cqid, uint16_t qsize, uint16_t qflags) "create submission queue, addr=0x%"PRIx64", sqid=%"PRIu16", cqid=%"PRIu16", qsize=%"PRIu16", qflags=%"PRIu16"" pci_nvme_create_cq(uint64_t addr, uint16_t cqid, uint16_t vector, uint16_t size, uint16_t qflags, int ien) "create completion queue, addr=0x%"PRIx64", cqid=%"PRIu16", vector=%"PRIu16", qsize=%"PRIu16", qflags=%"PRIu16", ien=%d"
@@ -142,6 +152,8 @@ pci_nvme_err_invalid_getfeat(int dw10) "invalid get features, dw10=0x%"PRIx32"" pci_nvme_err_invalid_setfeat(uint32_t dw10) "invalid set features, dw10=0x%"PRIx32"" pci_nvme_err_invalid_log_page(uint16_t cid, uint16_t lid) "cid %"PRIu16" lid 0x%"PRIx16"" pci_nvme_err_invalid_zone_state(uint64_t zslba, const char *zs_str, uint8_t zs) "zslba 0x%"PRIx64" zs '%s' (%"PRIu8")" +pci_nvme_err_invalid_zone(uint64_t lba) "lba 0x%"PRIx64"" +pci_nvme_err_invalid_zslba(uint64_t lba) "lba 0x%"PRIx64"" pci_nvme_err_startfail_cq(void) "nvme_start_ctrl failed because there are non-admin completion queues" pci_nvme_err_startfail_sq(void) "nvme_start_ctrl failed because there are non-admin submission queues" pci_nvme_err_startfail_nbarasq(void) "nvme_start_ctrl failed because the admin submission queue address is null"
Signature · 1 line
2.29.2
PATCH v5 9/12 hw/block/nvme: add the zone append command
In reply to Klaus JensenFrom: Klaus Jensen <[email protected]>
Add the Zone Append command.
--- hw/block/nvme.h | 5 ++++ include/block/nvme.h | 7 ++++++ hw/block/nvme.c | 53 +++++++++++++++++++++++++++++++++++++++++++ hw/block/trace-events | 1 + 4 files changed, 66 insertions(+)
hw/block/nvme.h +5 -0
@@ -16,6 +16,10 @@ typedef struct NvmeParams {uint32_t aer_max_queued;uint8_t mdts;bool use_intel_id; + +struct { +uint8_t zasl; + } zns; } NvmeParams; typedef struct NvmeAsyncEvent {
@@ -65,6 +69,7 @@ static inline const char *nvme_io_opc_str(uint8_t opc)case NVME_CMD_COPY :return "NVME_NVM_CMD_COPY" ;case NVME_CMD_ZONE_MGMT_SEND :return "NVME_ZONED_CMD_ZONE_MGMT_SEND" ;case NVME_CMD_ZONE_MGMT_RECV :return "NVME_ZONED_CMD_ZONE_MGMT_RECV" ; +case NVME_CMD_ZONE_APPEND :return "NVME_ZONED_CMD_ZONE_APPEND" ;default :return "NVME_NVM_CMD_UNKNOWN" ; } }
include/block/nvme.h +7 -0
@@ -487,6 +487,7 @@ enum NvmeIoCommands {NVME_CMD_COPY =0x19 ,NVME_CMD_ZONE_MGMT_SEND =0x79 ,NVME_CMD_ZONE_MGMT_RECV =0x7a , + NVME_CMD_ZONE_APPEND = 0x7d, }; typedef struct QEMU_PACKED NvmeDeleteQ {
@@ -1059,6 +1060,11 @@ enum NvmeIdCtrlLpa { NVME_LPA_EXTENDED = 1 << 2, }; +typedef struct QEMU_PACKED NvmeIdCtrlZns { +uint8_t zasl; +uint8_t rsvd1[4095 ]; +} NvmeIdCtrlZns; +#define NVME_CTRL_SQES_MIN (sqes) ((sqes) & 0xf)#define NVME_CTRL_SQES_MAX (sqes) (((sqes) >> 4) & 0xf)#define NVME_CTRL_CQES_MIN (cqes) ((cqes) & 0xf)
@@ -1293,6 +1299,7 @@ static inline void _nvme_check_size(void)QEMU_BUILD_BUG_ON (sizeof (NvmeFwSlotInfoLog) !=512 );QEMU_BUILD_BUG_ON (sizeof (NvmeSmartLog) !=512 );QEMU_BUILD_BUG_ON (sizeof (NvmeIdCtrl) !=4096 ); +QEMU_BUILD_BUG_ON (sizeof (NvmeIdCtrlZns) !=4096 );QEMU_BUILD_BUG_ON (sizeof (NvmeIdNsNvm) !=4096 );QEMU_BUILD_BUG_ON (sizeof (NvmeIdNsZns) !=4096 );QEMU_BUILD_BUG_ON (sizeof (NvmeSglDescriptor) !=16 );
hw/block/nvme.c +53 -0
121 lines not on this page · raw message
hw/block/trace-events +1 -0
@@ -114,6 +114,7 @@ pci_nvme_mmio_shutdown_cleared(void) "shutdown bit cleared" # nvme traces for error conditions pci_nvme_err_mdts(uint16_t cid, size_t len) "cid %"PRIu16" len %zu" +pci_nvme_err_zasl(uint16_t cid, size_t len) "cid %"PRIu16" len %zu" pci_nvme_err_req_status(uint16_t cid, uint32_t nsid, uint16_t status, uint8_t opc) "cid %"PRIu16" nsid %"PRIu32" status 0x%"PRIx16" opc 0x%"PRIx8"" pci_nvme_err_addr_read(uint64_t addr) "addr 0x%"PRIx64"" pci_nvme_err_addr_write(uint64_t addr) "addr 0x%"PRIx64""
Signature · 1 line
2.29.2
PATCH v5 10/12 hw/block/nvme: track and enforce zone resources
In reply to Klaus JensenFrom: Klaus Jensen <[email protected]>
Track number of open/active resources.
--- docs/specs/nvme.txt | 6 ++++ hw/block/nvme-ns.h | 7 +++++ include/block/nvme.h | 2 ++ hw/block/nvme-ns.c | 17 ++++++++++-- hw/block/nvme.c | 65 ++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 95 insertions(+), 2 deletions(-)
docs/specs/nvme.txt +6 -0
@@ -14,6 +14,12 @@ The nvme device (-device nvme) emulates an NVM Express Controller. zns.zcap; if the zone capacity is a power of two, the zone size will be set to that, otherwise it will default to the next power of two. + `zns.mar`; Specifies the number of active resources available. This is a 0s + based value. + + `zns.mor`; Specifies the number of open resources available. This is a 0s + based value. + Reference Specifications ------------------------
hw/block/nvme-ns.h +7 -0
@@ -32,6 +32,8 @@ typedef struct NvmeNamespaceParams {uint64_t zcap;uint64_t zsze;uint8_t zdes; +uint32_t mar; +uint32_t mor; } zns; } NvmeNamespaceParams;
@@ -62,6 +64,11 @@ typedef struct NvmeNamespace {NvmeZone *zones;NvmeZoneDescriptor *zd;uint8_t *zde; + +struct { +uint32_t open; +uint32_t active; + } resources; } zns; } NvmeNamespace;
include/block/nvme.h +2 -0
@@ -813,6 +813,8 @@ enum NvmeStatusCodes {NVME_ZONE_IS_READ_ONLY =0x01ba ,NVME_ZONE_IS_OFFLINE =0x01bb ,NVME_ZONE_INVALID_WRITE =0x01bc , +NVME_TOO_MANY_ACTIVE_ZONES =0x01bd , +NVME_TOO_MANY_OPEN_ZONES =0x01be ,NVME_INVALID_ZONE_STATE_TRANSITION =0x01bf ,NVME_WRITE_FAULT =0x0280 , NVME_UNRECOVERED_READ = 0x0281,
hw/block/nvme-ns.c +15 -2
@@ -94,8 +94,13 @@ static void nvme_ns_init_zoned(NvmeNamespace *ns) ns->zns.zde =g_malloc0_n (ns->zns.num_zones,nvme_ns_zdes_bytes (ns)); } - id_ns_zns->mar =0xffffffff ; - id_ns_zns->mor =0xffffffff ; + id_ns_zns->mar =cpu_to_le32 (ns->params.zns.mar); + id_ns_zns->mor =cpu_to_le32 (ns->params.zns.mor); + + ns->zns.resources.active = ns->params.zns.mar !=0xffffffff ? + ns->params.zns.mar +1 : ns->zns.num_zones; + ns->zns.resources.open = ns->params.zns.mor !=0xffffffff ? + ns->params.zns.mor +1 : ns->zns.num_zones; }static int nvme_ns_init (NvmeNamespace *ns,Error **errp)
@@ -197,6 +202,12 @@ static int nvme_ns_check_constraints(NvmeNamespace *ns, Error **errp)return -1 ; } +if (ns->params.zns.mor > ns->params.zns.mar) { +error_setg (errp,"maximum open resources (zns.mor) must be less " +"than or equal to maximum active resources (zns.mar)" ); +return -1 ; + } +break ;default :
@@ -267,6 +278,8 @@ static Property nvme_ns_props[] = {DEFINE_PROP_UINT64 ("zns.zcap" , NvmeNamespace, params.zns.zcap,0 ),DEFINE_PROP_UINT64 ("zns.zsze" , NvmeNamespace, params.zns.zsze,0 ),DEFINE_PROP_UINT8 ("zns.zdes" , NvmeNamespace, params.zns.zdes,0 ), +DEFINE_PROP_UINT32 ("zns.mar" , NvmeNamespace, params.zns.mar,0xffffffff ), +DEFINE_PROP_UINT32 ("zns.mor" , NvmeNamespace, params.zns.mor,0xffffffff ), DEFINE_PROP_END_OF_LIST(), };
hw/block/nvme.c +65 -0
98 lines not on this page · raw message
Signature · 1 line
2.29.2
PATCH v5 11/12 hw/block/nvme: allow open to close zone transitions by controller
In reply to Klaus JensenFrom: Klaus Jensen <[email protected]>
Allow the controller to release open resources by transitioning implicitly and explicitly opened zones to closed. This is done using a naive "least recently opened" strategy.
--- hw/block/nvme-ns.h | 5 ++++ hw/block/nvme-ns.c | 3 +++ hw/block/nvme.c | 57 ++++++++++++++++++++++++++++++++++++++++--- hw/block/trace-events | 1 + 4 files changed, 63 insertions(+), 3 deletions(-)
hw/block/nvme-ns.h +5 -0
@@ -42,6 +42,8 @@ typedef struct NvmeZone {uint8_t *zde;uint64_t wp_staging; + +QTAILQ_ENTRY (NvmeZone) lru_entry; } NvmeZone; typedef struct NvmeNamespace {
@@ -68,6 +70,9 @@ typedef struct NvmeNamespace {struct {uint32_t open;uint32_t active; + + QTAILQ_HEAD(, NvmeZone) lru_open; + QTAILQ_HEAD(, NvmeZone) lru_active; } resources; } zns; } NvmeNamespace;
hw/block/nvme-ns.c +3 -0
@@ -101,6 +101,9 @@ static void nvme_ns_init_zoned(NvmeNamespace *ns) ns->params.zns.mar + 1 : ns->zns.num_zones; ns->zns.resources.open = ns->params.zns.mor !=0xffffffff ? ns->params.zns.mor +1 : ns->zns.num_zones; + +QTAILQ_INIT (&ns->zns.resources.lru_open); +QTAILQ_INIT (&ns->zns.resources.lru_active); }static int nvme_ns_init (NvmeNamespace *ns,Error **errp)
hw/block/nvme.c +54 -3
109 lines not on this page · raw message
hw/block/trace-events +1 -0
@@ -94,6 +94,7 @@ pci_nvme_enqueue_event_masked(uint8_t typ) "type 0x%"PRIx8"" pci_nvme_no_outstanding_aers(void) "ignoring event; no outstanding AERs" pci_nvme_enqueue_req_completion(uint16_t cid, uint16_t cqid, uint16_t status) "cid %"PRIu16" cqid %"PRIu16" status 0x%"PRIx16"" pci_nvme_zrm_transition(uint32_t nsid, uint64_t zslba, const char *s_from, uint8_t from, const char *s_to, uint8_t to) "nsid %"PRIu32" zslba 0x%"PRIx64" from '%s' (%"PRIu8") to '%s' (%"PRIu8")" +pci_nvme_zrm_release_open(uint32_t nsid) "nsid %"PRIu32"" pci_nvme_zns_advance_wp(uint32_t nsid, uint64_t zslba, uint64_t wp_orig, uint32_t nlb) "nsid 0x%"PRIx32" zslba 0x%"PRIx64" wp_orig 0x%"PRIx64" nlb %"PRIu32"" pci_nvme_mmio_read(uint64_t addr) "addr 0x%"PRIx64"" pci_nvme_mmio_write(uint64_t addr, uint64_t data) "addr 0x%"PRIx64" data 0x%"PRIx64""
Signature · 1 line
2.29.2
RFC PATCH v5 12/12 hw/block/nvme: add persistence for zone info
In reply to Klaus JensenFrom: Klaus Jensen <[email protected]>
--- docs/specs/nvme.txt | 15 +++ hw/block/nvme-ns.h | 16 ++++ hw/block/nvme-ns.c | 212 +++++++++++++++++++++++++++++++++++++++++- hw/block/nvme.c | 87 +++++++++++++++++ hw/block/trace-events | 2 + 5 files changed, 331 insertions(+), 1 deletion(-)
docs/specs/nvme.txt +15 -0
@@ -20,6 +20,21 @@ The nvme device (-device nvme) emulates an NVM Express Controller. `zns.mor`; Specifies the number of open resources available. This is a 0s based value. + `zns.pstate`; This parameter specifies another blockdev to be used for + storing zone state persistently. + + -drive id=zns-pstate,file=zns-pstate.img,format=raw + -device nvme-ns,zns.pstate=zns-pstate,... + + To reset (or initialize) state, the blockdev image should be of zero size: + + qemu-img create -f raw zns-pstate.img 0 + + The image will be intialized with a file format header and truncated to + the required size. If the pstate given is of non-zero size, it will be + assumed to already contain zone state information and the header will be + checked. + Reference Specifications ------------------------
hw/block/nvme-ns.h +16 -0
@@ -19,6 +19,15 @@#define NVME_NS (obj) \ OBJECT_CHECK(NvmeNamespace, (obj), TYPE_NVME_NS) +#define NVME_ZONE_PSTATE_MAGIC ((0x00 << 24) | ('S' << 16) | ('N' << 8) | 'Z') +#define NVME_ZONE_PSTATE_V1 1 + +typedef struct NvmeZonePStateHeader { +uint32_t magic; +uint32_t version; +uint8_t rsvd8[4088 ]; +} QEMU_PACKED NvmeZonePStateHeader; + typedef struct NvmeNamespaceParams { uint32_t nsid; uint8_t iocs;
@@ -74,6 +83,8 @@ typedef struct NvmeNamespace { QTAILQ_HEAD(, NvmeZone) lru_open; QTAILQ_HEAD(, NvmeZone) lru_active; } resources; + +BlockBackend *pstate; } zns; } NvmeNamespace;
@@ -186,4 +197,9 @@ int nvme_ns_setup(NvmeCtrl *n, NvmeNamespace *ns, Error **errp);void nvme_ns_drain (NvmeNamespace *ns);void nvme_ns_flush (NvmeNamespace *ns); +static inline void _nvme_ns_check_size (void ) +{ +QEMU_BUILD_BUG_ON (sizeof (NvmeZonePStateHeader) !=4096 ); +} +#endif /* NVME_NS_H */
hw/block/nvme-ns.c +211 -1
247 lines not on this page · raw message
hw/block/nvme.c +87 -0
165 lines not on this page · raw message
hw/block/trace-events +2 -0
@@ -96,6 +96,8 @@ pci_nvme_enqueue_req_completion(uint16_t cid, uint16_t cqid, uint16_t status) "c pci_nvme_zrm_transition(uint32_t nsid, uint64_t zslba, const char *s_from, uint8_t from, const char *s_to, uint8_t to) "nsid %"PRIu32" zslba 0x%"PRIx64" from '%s' (%"PRIu8") to '%s' (%"PRIu8")" pci_nvme_zrm_release_open(uint32_t nsid) "nsid %"PRIu32"" pci_nvme_zns_advance_wp(uint32_t nsid, uint64_t zslba, uint64_t wp_orig, uint32_t nlb) "nsid 0x%"PRIx32" zslba 0x%"PRIx64" wp_orig 0x%"PRIx64" nlb %"PRIu32"" +pci_nvme_zns_commit_zone(uint32_t nsid, uint64_t zslba) "nsid 0x%"PRIx32" zslba 0x%"PRIx64"" +pci_nvme_zns_commit_zde(uint32_t nsid, uint64_t zslba) "nsid 0x%"PRIx32" zslba 0x%"PRIx64"" pci_nvme_mmio_read(uint64_t addr) "addr 0x%"PRIx64"" pci_nvme_mmio_write(uint64_t addr, uint64_t data) "addr 0x%"PRIx64" data 0x%"PRIx64"" pci_nvme_mmio_doorbell_cq(uint16_t cqid, uint16_t new_head) "cqid %"PRIu16" new_head %"PRIu16""
Signature · 1 line
2.29.2
Re: [PATCH RFC v5 12/12] hw/block/nvme: add persistence for zone info
Quoted 23 lines · Klaus Jensen
From: Klaus Jensen <[email protected]>
Signed-off-by Klaus Jensen <[email protected]>--- docs/specs/nvme.txt | 15 +++ hw/block/nvme-ns.h | 16 ++++ hw/block/nvme-ns.c | 212 +++++++++++++++++++++++++++++++++++++++++- hw/block/nvme.c | 87 +++++++++++++++++ hw/block/trace-events | 2 + 5 files changed, 331 insertions(+), 1 deletion(-)docs/specs/nvme.txt +4
-0@@ -20,6 +20,21 @@ The nvme device (-device nvme) emulates an NVM Express Controller. `zns.mor`; Specifies the number of open resources available. This is a 0s based value. + `zns.pstate`; This parameter specifies another blockdev to be used for + storing zone state persistently. + + -drive id=zns-pstate,file=zns-pstate.img,format=raw
docs/specs/nvme.txt The nvme device (-device nvme) emulates an NVM Express Controller. · line 36
Quoted 10 lines
docs/specs/nvme.txt +10
-0+ -device nvme-ns,zns.pstate=zns-pstate,... + + To reset (or initialize) state, the blockdev image should be of zero size: + + qemu-img create -f raw zns-pstate.img 0 + + The image will be intialized with a file format header and truncated to + the required size. If the pstate given is of non-zero size, it will be + assumed to already contain zone state information and the header will be + checked.
In principle this makes sense but at first glance it looks like the code is synchronous - it blocks the vCPU if zone metadata I/O is necessary. That works for test/bring-up code but can't be used in production due to the performance impact.
Is the expectation that the QEMU NVMe device emulation will only be used for test/bring-up now and in the future?
Stefan
Re: [PATCH RFC v5 12/12] hw/block/nvme: add persistence for zone info
Quoted 40 lines · Stefan Hajnoczi
Quoted 23 lines · Klaus Jensen
From: Klaus Jensen <[email protected]>
Signed-off-by Klaus Jensen <[email protected]>--- docs/specs/nvme.txt | 15 +++ hw/block/nvme-ns.h | 16 ++++ hw/block/nvme-ns.c | 212 +++++++++++++++++++++++++++++++++++++++++- hw/block/nvme.c | 87 +++++++++++++++++ hw/block/trace-events | 2 + 5 files changed, 331 insertions(+), 1 deletion(-)docs/specs/nvme.txt +4
-0@@ -20,6 +20,21 @@ The nvme device (-device nvme) emulates an NVM Express Controller. `zns.mor`; Specifies the number of open resources available. This is a 0s based value. + `zns.pstate`; This parameter specifies another blockdev to be used for + storing zone state persistently. + + -drive id=zns-pstate,file=zns-pstate.img,format=rawdocs/specs/nvme.txt The nvme device (-device nvme) emulates an NVM Express Controller. · line 36
Quoted 10 lines
docs/specs/nvme.txt +10
-0+ -device nvme-ns,zns.pstate=zns-pstate,... + + To reset (or initialize) state, the blockdev image should be of zero size: + + qemu-img create -f raw zns-pstate.img 0 + + The image will be intialized with a file format header and truncated to + the required size. If the pstate given is of non-zero size, it will be + assumed to already contain zone state information and the header will be + checked.In principle this makes sense but at first glance it looks like the code is synchronous - it blocks the vCPU if zone metadata I/O is necessary. That works for test/bring-up code but can't be used in production due to the performance impact.
Quoted 3 lines
Is the expectation that the QEMU NVMe device emulation will only be used for test/bring-up now and in the future?
Hi Stefan,
Thanks for taking a look at this.
I could see why someone would maybe use the core nvme emulation in production (but I'm not aware of anyone doing it), but the zoned emulation is *probably* not for production (and that is where the zone updates are needed). But someone could surprise me with a use case I guess.
And yes, I know this is synchronous in this version. I have not extensively evaluated the performance impact, but crucially the blocking only happens when the zone changes state (i.e. every write does NOT trigger a blk_pwrite just to persist the updated write pointer).
I know this can be done asynchronously (I have implemented it like so previously), but I wanted to get an opinion on the general stategry before adding that. The opposing strategy, is to use a some form of mmap/msync, but I, for one, pushed back against that because I'd like this to work on as many platforms as possible. Hence the RFC for this blockdev based approach.
But if you think a blockdev approach like this is a reasonable QEMU-esce way of doing it, then I'll proceed to do a v2 with asynchronous updates.
Re: [PATCH RFC v5 12/12] hw/block/nvme: add persistence for zone info
Quoted 67 lines · Klaus Jensen
Quoted 40 lines · Stefan Hajnoczi
Quoted 23 lines · Klaus Jensen
From: Klaus Jensen <[email protected]>
Signed-off-by Klaus Jensen <[email protected]>--- docs/specs/nvme.txt | 15 +++ hw/block/nvme-ns.h | 16 ++++ hw/block/nvme-ns.c | 212 +++++++++++++++++++++++++++++++++++++++++- hw/block/nvme.c | 87 +++++++++++++++++ hw/block/trace-events | 2 + 5 files changed, 331 insertions(+), 1 deletion(-)docs/specs/nvme.txt +4
-0@@ -20,6 +20,21 @@ The nvme device (-device nvme) emulates an NVM Express Controller. `zns.mor`; Specifies the number of open resources available. This is a 0s based value. + `zns.pstate`; This parameter specifies another blockdev to be used for + storing zone state persistently. + + -drive id=zns-pstate,file=zns-pstate.img,format=rawdocs/specs/nvme.txt The nvme device (-device nvme) emulates an NVM Express Controller. · line 36
Quoted 10 lines
docs/specs/nvme.txt +10
-0+ -device nvme-ns,zns.pstate=zns-pstate,... + + To reset (or initialize) state, the blockdev image should be of zero size: + + qemu-img create -f raw zns-pstate.img 0 + + The image will be intialized with a file format header and truncated to + the required size. If the pstate given is of non-zero size, it will be + assumed to already contain zone state information and the header will be + checked.In principle this makes sense but at first glance it looks like the code is synchronous - it blocks the vCPU if zone metadata I/O is necessary. That works for test/bring-up code but can't be used in production due to the performance impact.
Quoted 3 lines
Is the expectation that the QEMU NVMe device emulation will only be used for test/bring-up now and in the future?
Hi Stefan,
Thanks for taking a look at this.
I could see why someone would maybe use the core nvme emulation in production (but I'm not aware of anyone doing it), but the zoned emulation is *probably* not for production (and that is where the zone updates are needed). But someone could surprise me with a use case I guess.
And yes, I know this is synchronous in this version. I have not extensively evaluated the performance impact, but crucially the blocking only happens when the zone changes state (i.e. every write does NOT trigger a
blk_pwritejust to persist the updated write pointer).I know this can be done asynchronously (I have implemented it like so previously), but I wanted to get an opinion on the general stategry before adding that. The opposing strategy, is to use a some form of mmap/msync, but I, for one, pushed back against that because I'd like this to work on as many platforms as possible. Hence the RFC for this blockdev based approach.
Quoted 2 lines
But if you think a blockdev approach like this is a reasonable QEMU-esce way of doing it, then I'll proceed to do a v2 with asynchronous updates.
Let me rephrase that I will most likely wait to do the v2 until we have conconsus and reviews on a zoned series.
Re: [PATCH RFC v5 12/12] hw/block/nvme: add persistence for zone info
In reply to Klaus JensenQuoted 67 lines · Klaus Jensen
Quoted 40 lines · Stefan Hajnoczi
Quoted 23 lines · Klaus Jensen
From: Klaus Jensen <[email protected]>
Signed-off-by Klaus Jensen <[email protected]>--- docs/specs/nvme.txt | 15 +++ hw/block/nvme-ns.h | 16 ++++ hw/block/nvme-ns.c | 212 +++++++++++++++++++++++++++++++++++++++++- hw/block/nvme.c | 87 +++++++++++++++++ hw/block/trace-events | 2 + 5 files changed, 331 insertions(+), 1 deletion(-)docs/specs/nvme.txt +4
-0@@ -20,6 +20,21 @@ The nvme device (-device nvme) emulates an NVM Express Controller. `zns.mor`; Specifies the number of open resources available. This is a 0s based value. + `zns.pstate`; This parameter specifies another blockdev to be used for + storing zone state persistently. + + -drive id=zns-pstate,file=zns-pstate.img,format=rawdocs/specs/nvme.txt The nvme device (-device nvme) emulates an NVM Express Controller. · line 36
Quoted 10 lines
docs/specs/nvme.txt +10
-0+ -device nvme-ns,zns.pstate=zns-pstate,... + + To reset (or initialize) state, the blockdev image should be of zero size: + + qemu-img create -f raw zns-pstate.img 0 + + The image will be intialized with a file format header and truncated to + the required size. If the pstate given is of non-zero size, it will be + assumed to already contain zone state information and the header will be + checked.In principle this makes sense but at first glance it looks like the code is synchronous - it blocks the vCPU if zone metadata I/O is necessary. That works for test/bring-up code but can't be used in production due to the performance impact.
Quoted 3 lines
Is the expectation that the QEMU NVMe device emulation will only be used for test/bring-up now and in the future?
Hi Stefan,
Thanks for taking a look at this.
I could see why someone would maybe use the core nvme emulation in production (but I'm not aware of anyone doing it), but the zoned emulation is *probably* not for production (and that is where the zone updates are needed). But someone could surprise me with a use case I guess.
And yes, I know this is synchronous in this version. I have not extensively evaluated the performance impact, but crucially the blocking only happens when the zone changes state (i.e. every write does NOT trigger a
blk_pwritejust to persist the updated write pointer).I know this can be done asynchronously (I have implemented it like so previously), but I wanted to get an opinion on the general stategry before adding that. The opposing strategy, is to use a some form of mmap/msync, but I, for one, pushed back against that because I'd like this to work on as many platforms as possible. Hence the RFC for this blockdev based approach.
Quoted 2 lines
But if you think a blockdev approach like this is a reasonable QEMU-esce way of doing it, then I'll proceed to do a v2 with asynchronous updates.
I don't have a strong opinion, especially since I haven't investigated how NVMe Zoned Namespaces work (besides watching your interesting KVM Forum presentation).
If you want to do the mmap approach, please use HostMemoryBackend for that instead of open coding mmap(2) calls. That way it will support several types of backing memory, including MAP_SHARED files, persistent memory, etc.
Stefan