hw/nvme: Support for Namespaces Management from guest OS
v3 cover hw/nvme: Support for Namespaces Management from guest OS
From: Michael Kropaczek <[email protected]>
Description:
Currently namespaces could be configured as follows:
1. Legacy Namespace - just one namespace within Nvme controller's where the back-end was specified for nvme device by -drive parameter pointing directly to the image file.
2. Additional Namespaces - specified by nvme-ns devices each having its own back-end. To have multiple namespaces each needed to be specified at Qemu's command line being associated with the most recently defined nvme-bus from nvme device.
If a such additional namespace should be attached and/or detached by the guest OS, nvme controller has to be linked with another device nvme-subsys.
All that have a static nature, all need to be specified at the Qemu's command line, all specified virtual nvme entities will be processed during Qemu's start-up then created and provided to the guest OS.
To have a support for nvme create-ns and delete-ns commands with specified parameters a different approach is needed.
Virtual devices representing namespaces need to be created and/or deleted during Qemu's running session, at anytime. The back-end image sizes for a namespace must accommodate the payload size and size of metadata resulted from specified parameters. The total capacity of the nvme controller altogether with un-allocated capacity needs to be taken into account and updated according to nvme create-ns and delete-ns commands respectively.
Here is the approach:
The nvme device will get new parameter:
- auto-ns-path, which specifies the path to the storage area where back-end image and necessary config files located stored.
The virtual devices representing namespaces will be created dynamically during the Qemu running session following issuance of nvme create-ns and delete-ns commands from the guest OS. QOM classes and instances will be created utilizing existing configuration scheme used during Qemu's start-up. Back-end image files will be neither created nor deleted during Qemu's startup or running session. Instead a set of back-end image files and relevant config will be created by qemu-img tool with createns sub-command prior to Qemu's session. Required parameters are: -S serial number which must match serial parameter of qemu-system-xx -device nvme command line specification, -C total capacity, and optional -N that will set a maximal limit on number of allowed namespaces (default 256) which will be followed by path name pointing to storage location corresponding to auto-ns-path of qemu-system-xx -device nvme parameter.
Those created back-end image files will be pre-loaded during Qemu's start-up and then during running Qemu's session will be associated or disassociated with QOM namespaces virtual instances, as a result of issuing nvme create-ns or delete-ns commands. The associated back-end image file for relevant namespace will be re-sized as follows: delete-ns command will truncate image file to the size of 0, whereas create-ns command will re-size the image file to the size provided by nvme create-ns command parameters. Truncating/re-sizing is a result of blk_truncate() API which utilizes co-routines and should not block Qemu main thread while scheduling AIO operations. It is assured that all settings will retain over Qemu's start-ups and shutdowns. The implementation makes it possible to combine the existing "Additional Namespace" implementation with the new "Managed Namespaces". Those will coexist with obvious restrictions, like both will share the same NsIds space, "static" namespaces cannot be deleted or if its NsId specified at Qemu's command line will conflicts with previously created one by nvme create-ns (and retained), this will lead to an abort of Qemu at its start up.
Michael Kropaczek (2):
hw/nvme: Support for Namespaces Management from guest OS - create-ns hw/nvme: Support for Namespaces Management from guest OS - delete-ns
docs/system/devices/nvme.rst | 56 ++++++- hw/nvme/cfg_key_checker.c | 51 ++++++ hw/nvme/ctrl-cfg.c | 181 ++++++++++++++++++++ hw/nvme/ctrl.c | 282 +++++++++++++++++++++++++++++++- hw/nvme/meson.build | 2 +- hw/nvme/ns-backend.c | 239 +++++++++++++++++++++++++++ hw/nvme/ns.c | 308 +++++++++++++++++++++++++++++++---- hw/nvme/nvme.h | 33 +++- hw/nvme/trace-events | 3 + include/block/nvme.h | 31 ++++ include/hw/nvme/ctrl-cfg.h | 24 +++ include/hw/nvme/ns-cfg.h | 28 ++++ include/hw/nvme/nvme-cfg.h | 201 +++++++++++++++++++++++ qemu-img-cmds.hx | 6 + qemu-img.c | 134 +++++++++++++++ 15 files changed, 1542 insertions(+), 37 deletions(-) create mode 100644 hw/nvme/cfg_key_checker.c create mode 100644 hw/nvme/ctrl-cfg.c create mode 100644 hw/nvme/ns-backend.c create mode 100644 include/hw/nvme/ctrl-cfg.h create mode 100644 include/hw/nvme/ns-cfg.h create mode 100644 include/hw/nvme/nvme-cfg.h
Signature · 1 line
2.37.3
PATCH v3 1/2 hw/nvme: Support for Namespaces Management from guest OS - create-ns
From: Michael Kropaczek <[email protected]>
Added support for NVMEe NameSpaces Mangement allowing the guest OS to create namespaces by issuing nvme create-ns command.
It is an extension to currently implemented Qemu nvme virtual device. Virtual devices representing namespaces will be created and/or deleted during Qemu's running session, at anytime.
--- docs/system/devices/nvme.rst | 55 +++++++- hw/nvme/cfg_key_checker.c | 51 ++++++++ hw/nvme/ctrl-cfg.c | 181 +++++++++++++++++++++++++++ hw/nvme/ctrl.c | 204 +++++++++++++++++++++++++++++- hw/nvme/meson.build | 2 +- hw/nvme/ns-backend.c | 234 +++++++++++++++++++++++++++++++++++ hw/nvme/ns.c | 234 ++++++++++++++++++++++++++++++----- hw/nvme/nvme.h | 31 ++++- hw/nvme/trace-events | 2 + include/block/nvme.h | 30 +++++ include/hw/nvme/ctrl-cfg.h | 24 ++++ include/hw/nvme/ns-cfg.h | 28 +++++ include/hw/nvme/nvme-cfg.h | 201 ++++++++++++++++++++++++++++++ qemu-img-cmds.hx | 6 + qemu-img.c | 134 ++++++++++++++++++++ 15 files changed, 1380 insertions(+), 37 deletions(-) create mode 100644 hw/nvme/cfg_key_checker.c create mode 100644 hw/nvme/ctrl-cfg.c create mode 100644 hw/nvme/ns-backend.c create mode 100644 include/hw/nvme/ctrl-cfg.h create mode 100644 include/hw/nvme/ns-cfg.h create mode 100644 include/hw/nvme/nvme-cfg.h
docs/system/devices/nvme.rst +54 -1
@@ -92,6 +92,59 @@ There are a number of parameters available: attach the namespace to a specific ``nvme`` device (identified by an ``id`` parameter on the controller device). +Additional Namespaces managed by guest OS Namespaces Management +--------------------------------------------------------------------- + +.. code-block:: console + + -device nvme,id=nvme-ctrl,serial=1234,subsys=nvme-subsys,auto-ns-path=path + +Parameters: + +``auto-ns-path=<path to nvme storage location>`` + If specified indicates a support for dynamic management of nvme namespaces + by means of nvme create-ns command. This path points + to the storage area for backend images must exist. Additionally it requires + that parameter `ns-subsys` must be specified whereas parameter `drive` + must not. The legacy namespace backend is disabled, instead, a pair of + files 'nvme_<ctrl SN>_ns_<NS-ID>.cfg' and 'nvme_<ctrl SN>_ns_<NS-ID>.img' + will refer to respective namespaces. The create-ns, attach-ns + and detach-ns commands, issued at the guest side, will make changes to + those files accordingly. + For each namespace exists an image file in raw format and a config file + containing namespace parameters and state of the attachment allowing QEMU + to configure namespaces accordingly during start up. If for instance an + image file has a size of 0 bytes this will be interpreted as non existent + namespace. Issuing create-ns command will change the status in the config + files and and will re-size the image file accordingly so the image file + will be associated with the respective namespace. The main config file + nvme_<ctrl SN>_ctrl.cfg keeps the track of allocated capacity to the + namespaces within the nvme controller. + As it is the case of a typical hard drive, backend images together with + config files need to be created. For this reason the qemu-img tool has + been extended by adding createns command. + + qemu-img createns {-S <serial> -C <total NVMe capacity>} + [-N <NsId max>] {<path>} + + Parameters: + -S and -C and <path> are mandatory, `-S` must match `serial` parameter + and <path> must match `auto-ns-path` parameter of "-device nvme,..." + specification. + -N is optional, if specified it will set a limit to the number of potential + namespaces and will reduce the number of backend images and config files + accordingly. As a default, a set of images of 0 bytes size and default + config files for 256 namespaces will be created, a total of 513 files. + +Please note that ``nvme-ns`` device is not required to support of dynamic +namespaces management feature. It is not prohibited to assign a such device to +``nvme`` device specified to support dynamic namespace management if one has +an use case to do so, however, it will only coexist and be out of the scope of +Namespaces Management. NsIds will be consistently managed, creation (create-ns) +of a namespace will not allocate the NsId already being taken. If ``nvme-ns`` +device conflicts with previously created one by create-ns (the same NsId), +it will break QEMU's start up. + NVM Subsystems --------------
@@ -320,4 +373,4 @@ controller are: .. code-block:: console - echo 0000:01:00.1 > /sys/bus/pci/drivers/nvme/bind + echo 0000:01:00.1 > /sys/bus/pci/drivers/nvme/bind
hw/nvme/cfg_key_checker.c new file +51 -0
51 lines not on this page · raw message
hw/nvme/ctrl-cfg.c new file +181 -0
181 lines not on this page · raw message
hw/nvme/ctrl.c +201 -3
276 lines not on this page · raw message
hw/nvme/meson.build +1 -1
@@ -1 +1 @@ -softmmu_ss.add(when: 'CONFIG_NVME_PCI', if_true: files('ctrl.c', 'dif.c', 'ns.c', 'subsys.c')) +softmmu_ss.add(when: 'CONFIG_NVME_PCI', if_true: files('ctrl.c', 'dif.c', 'ns.c', 'subsys.c', 'ns-backend.c', 'cfg_key_checker.c', 'ctrl-cfg.c'))
hw/nvme/ns-backend.c new file +234 -0
234 lines not on this page · raw message
hw/nvme/ns.c +205 -29
284 lines not on this page · raw message
hw/nvme/nvme.h +28 -3
@@ -23,9 +23,8 @@#include "hw/block/block.h" #include "block/nvme.h" +#include "hw/nvme/ctrl-cfg.h" -#define NVME_MAX_CONTROLLERS 256 -#define NVME_MAX_NAMESPACES 256#define NVME_EUI64_DEFAULT ((uint64_t)0x5254000000000000)QEMU_BUILD_BUG_ON (NVME_MAX_NAMESPACES >NVME_NSID_BROADCAST -1 );
@@ -279,6 +278,8 @@ int nvme_ns_setup(NvmeNamespace *ns, Error **errp);void nvme_ns_drain (NvmeNamespace *ns);void nvme_ns_shutdown (NvmeNamespace *ns);void nvme_ns_cleanup (NvmeNamespace *ns); +void nvme_validate_flbas (uint8_t flbas,Error **errp); +NvmeNamespace *nvme_ns_create (NvmeCtrl *n,uint32_t nsid,NvmeIdNsMgmt *id_ns,Error **errp); typedef struct NvmeAsyncEvent { QTAILQ_ENTRY(NvmeAsyncEvent) entry;
@@ -339,6 +340,7 @@ static inline const char *nvme_adm_opc_str(uint8_t opc)case NVME_ADM_CMD_SET_FEATURES :return "NVME_ADM_CMD_SET_FEATURES" ;case NVME_ADM_CMD_GET_FEATURES :return "NVME_ADM_CMD_GET_FEATURES" ;case NVME_ADM_CMD_ASYNC_EV_REQ :return "NVME_ADM_CMD_ASYNC_EV_REQ" ; +case NVME_ADM_CMD_NS_MGMT :return "NVME_ADM_CMD_NS_MGMT" ;case NVME_ADM_CMD_NS_ATTACHMENT :return "NVME_ADM_CMD_NS_ATTACHMENT" ;case NVME_ADM_CMD_VIRT_MNGMT :return "NVME_ADM_CMD_VIRT_MNGMT" ;case NVME_ADM_CMD_DBBUF_CONFIG :return "NVME_ADM_CMD_DBBUF_CONFIG" ;
@@ -427,6 +429,7 @@ typedef struct NvmeParams {uint16_t sriov_vi_flexible;uint8_t sriov_max_vq_per_vf;uint8_t sriov_max_vi_per_vf; +char *ns_directory;/* if empty (default) one legacy ns will be created */ } NvmeParams; typedef struct NvmeCtrl {
@@ -485,8 +488,9 @@ typedef struct NvmeCtrl {NvmeSubsystem *subsys; -NvmeNamespace namespace; +NvmeNamespace namespace;/* if ns_directory is empty this will be used */ NvmeNamespace *namespaces[NVME_MAX_NAMESPACES +1 ]; +BlockBackend *preloaded_blk[NVME_MAX_NAMESPACES +1 ];NvmeSQueue **sq;NvmeCQueue **cq;NvmeSQueue admin_sq;
@@ -509,6 +513,7 @@ typedef struct NvmeCtrl {uint16_t vqrfap;uint16_t virfap; } next_pri_ctrl_cap;/* These override pri_ctrl_cap after reset */ +uint16_t nsidMax; } NvmeCtrl; typedef enum NvmeResetType {
@@ -575,6 +580,9 @@ static inline NvmeSecCtrlEntry *nvme_sctrl_for_cntlid(NvmeCtrl *n,return NULL ; } +BlockBackend *ns_blockdev_init (const char *file,Error **errp); +void ns_blockdev_activate (BlockBackend *blk,uint64_t image_size,Error **errp); +int nvme_ns_backend_setup (NvmeCtrl *n,Error **errp);void nvme_attach_ns (NvmeCtrl *n,NvmeNamespace *ns);uint16_t nvme_bounce_data (NvmeCtrl *n,void *ptr,uint32_t len,NvmeTxDirection dir,NvmeRequest *req);
@@ -583,5 +591,22 @@ uint16_t nvme_bounce_mdata(NvmeCtrl *n, void *ptr, uint32_t len,void nvme_rw_complete_cb (void *opaque,int ret);uint16_t nvme_map_dptr (NvmeCtrl *n,NvmeSg *sg,size_t len,NvmeCmd *cmd); +char *ns_create_image_name (NvmeCtrl *n,uint32_t nsid,Error **errp); +int ns_storage_path_check (NvmeCtrl *n,Error **errp); +int ns_auto_check (NvmeCtrl *n,NvmeNamespace *ns,uint32_t nsid); +int ns_cfg_save (NvmeCtrl *n,NvmeNamespace *ns,uint32_t nsid); +int ns_cfg_load (NvmeCtrl *n,NvmeNamespace *ns,uint32_t nsid); +int64_t qdict_get_int_chkd (const QDict *qdict,const char *key,Error **errp); +bool qdict_get_bool_chkd (const QDict *qdict,const char *key,Error **errp); +void ns_cfg_clear (NvmeNamespace *ns); +int nvme_cfg_save (NvmeCtrl *n); +int nvme_cfg_load (NvmeCtrl *n); + +typedef enum NvmeNsAllocAction { +NVME_NS_ALLOC_CHK , +NVME_NS_ALLOC , +NVME_NS_DEALLOC , +}NvmeNsAllocAction ; +int nvme_cfg_update (NvmeCtrl *n,uint64_t ammount,NvmeNsAllocAction action);#endif /* HW_NVME_NVME_H */
hw/nvme/trace-events +2 -0
@@ -77,6 +77,8 @@ pci_nvme_aer(uint16_t cid) "cid %"PRIu16"" pci_nvme_aer_aerl_exceeded(void) "aerl exceeded" pci_nvme_aer_masked(uint8_t type, uint8_t mask) "type 0x%"PRIx8" mask 0x%"PRIx8"" pci_nvme_aer_post_cqe(uint8_t typ, uint8_t info, uint8_t log_page) "type 0x%"PRIx8" info 0x%"PRIx8" lid 0x%"PRIx8"" +pci_nvme_ns_mgmt(uint16_t cid, uint32_t nsid, uint8_t sel, uint8_t csi, uint8_t psdt) "cid %"PRIu16", nsid=%"PRIu32", sel=0x%"PRIx8", csi=0x%"PRIx8", psdt=0x%"PRIx8"" +pci_nvme_ns_create(uint16_t nsid, uint64_t nsze, uint64_t ncap, uint8_t flbas) "nsid %"PRIu16", nsze=%"PRIu64", ncap=%"PRIu64", flbas=%"PRIu8"" pci_nvme_ns_attachment(uint16_t cid, uint8_t sel) "cid %"PRIu16", sel=0x%"PRIx8"" pci_nvme_ns_attachment_attach(uint16_t cntlid, uint32_t nsid) "cntlid=0x%"PRIx16", nsid=0x%"PRIx32"" pci_nvme_enqueue_event(uint8_t typ, uint8_t info, uint8_t log_page) "type 0x%"PRIx8" info 0x%"PRIx8" lid 0x%"PRIx8""
include/block/nvme.h +30 -0
@@ -592,6 +592,7 @@ enum NvmeAdminCommands {NVME_ADM_CMD_SET_FEATURES =0x09 ,NVME_ADM_CMD_GET_FEATURES =0x0a ,NVME_ADM_CMD_ASYNC_EV_REQ =0x0c , +NVME_ADM_CMD_NS_MGMT =0x0d ,NVME_ADM_CMD_ACTIVATE_FW =0x10 ,NVME_ADM_CMD_DOWNLOAD_FW =0x11 , NVME_ADM_CMD_NS_ATTACHMENT = 0x15,
@@ -897,14 +898,18 @@ enum NvmeStatusCodes {NVME_FEAT_NOT_CHANGEABLE =0x010e ,NVME_FEAT_NOT_NS_SPEC =0x010f ,NVME_FW_REQ_SUSYSTEM_RESET =0x0110 , +NVME_NS_INSUFFICIENT_CAPAC =0x0115 , +NVME_NS_IDNTIFIER_UNAVAIL =0x0116 ,NVME_NS_ALREADY_ATTACHED =0x0118 ,NVME_NS_PRIVATE =0x0119 ,NVME_NS_NOT_ATTACHED =0x011a , +NVME_THIN_PROVISION_NOTSPRD =0x011b ,NVME_NS_CTRL_LIST_INVALID =0x011c ,NVME_INVALID_CTRL_ID =0x011f ,NVME_INVALID_SEC_CTRL_STATE =0x0120 ,NVME_INVALID_NUM_RESOURCES =0x0121 ,NVME_INVALID_RESOURCE_ID =0x0122 , +NVME_NS_ATTACH_MGMT_NOTSPRD =0x0129 ,NVME_CONFLICTING_ATTRS =0x0180 ,NVME_INVALID_PROT_INFO =0x0181 , NVME_WRITE_TO_RO = 0x0182,
@@ -1184,6 +1189,10 @@ enum NvmeIdCtrlCmic { NVME_CMIC_MULTI_CTRL = 1 << 1, }; +enum NvmeNsManagementOperation { +NVME_NS_MANAGEMENT_CREATE =0x0 , +}; +enum NvmeNsAttachmentOperation {NVME_NS_ATTACHMENT_ATTACH =0x0 ,NVME_NS_ATTACHMENT_DETACH =0x1 ,
@@ -1345,6 +1354,26 @@ typedef struct QEMU_PACKED NvmeIdNs {uint8_t vs[3712 ]; } NvmeIdNs; +typedef struct QEMU_PACKED NvmeIdNsMgmt { +uint64_t nsze; +uint64_t ncap; +uint8_t rsvd16[10 ]; +uint8_t flbas; +uint8_t rsvd27[2 ]; +uint8_t dps; +uint8_t nmic; +uint8_t rsvd31[61 ]; +uint32_t anagrpid; +uint8_t rsvd96[4 ]; +uint16_t nvmsetid; +uint16_t endgid; +uint8_t rsvd104[280 ]; +uint64_t lbstm; +uint8_t rsvd392[120 ]; +uint8_t rsvd512[512 ]; +uint8_t vs[3072 ]; +} NvmeIdNsMgmt; +#define NVME_ID_NS_NVM_ELBAF_PIF (elbaf) (((elbaf) >> 7) & 0x3) typedef struct QEMU_PACKED NvmeIdNsNvm {
@@ -1646,6 +1675,7 @@ static inline void _nvme_check_size(void)QEMU_BUILD_BUG_ON (sizeof (NvmeLBAF) !=4 );QEMU_BUILD_BUG_ON (sizeof (NvmeLBAFE) !=16 );QEMU_BUILD_BUG_ON (sizeof (NvmeIdNs) !=4096 ); +QEMU_BUILD_BUG_ON (sizeof (NvmeIdNsMgmt) !=4096 );QEMU_BUILD_BUG_ON (sizeof (NvmeIdNsNvm) !=4096 );QEMU_BUILD_BUG_ON (sizeof (NvmeIdNsZoned) !=4096 );QEMU_BUILD_BUG_ON (sizeof (NvmeSglDescriptor) !=16 );
include/hw/nvme/ctrl-cfg.h new file +24 -0
@@ -0,0 +1,24 @@ +/* +* QEMU NVM Express Virtual Dynamic Namespace Management +* Common configuration handling for qemu-img tool and and qemu-system-xx +* +* +* Copyright (c) 2022 Solidigm +* +* Authors: +* Michael Kropaczek <[email protected]> +* +* This work is licensed under the terms of the GNU GPL, version 2. See the +* COPYING file in the top-level directory. +* +*/ + +#ifndef CTRL_CFG_DEF +#define NVME_STR_ (s) #s +#define NVME_STRINGIFY (s) NVME_STR_(s) +#define NVME_MAX_NAMESPACES 256 +#define NVME_MAX_CONTROLLERS 256 +#else +CTRL_CFG_DEF(int, "tnvmcap", int128_get64(tnvmcap128), tnvmcap64) +CTRL_CFG_DEF(int, "unvmcap", int128_get64(unvmcap128), unvmcap64) +#endif
include/hw/nvme/ns-cfg.h new file +28 -0
@@ -0,0 +1,28 @@ +/* +* QEMU NVM Express Virtual Dynamic Namespace Management +* Common configuration handling for qemu-img tool and qemu-system-xx +* +* +* Copyright (c) 2022 Solidigm +* +* Authors: +* Michael Kropaczek <[email protected]> +* +* This work is licensed under the terms of the GNU GPL, version 2. See the +* COPYING file in the top-level directory. +* +*/ + +#ifdef NS_CFG_DEF +NS_CFG_DEF(int, "params.nsid", (int64_t)ns->params.nsid, nsid) +NS_CFG_DEF(bool, "params.detached", ns->params.detached, true) +NS_CFG_DEF(int, "params.pi", (int64_t)ns->params.pi, 0) +NS_CFG_DEF(int, "lbasz", (int64_t)ns->lbasz, 0) +NS_CFG_DEF(int, "id_ns.nsze", le64_to_cpu(ns->id_ns.nsze), 0) +NS_CFG_DEF(int, "id_ns.ncap", le64_to_cpu(ns->id_ns.ncap), 0) +NS_CFG_DEF(int, "id_ns.nuse", le64_to_cpu(ns->id_ns.nuse), 0) +NS_CFG_DEF(int, "id_ns.nsfeat", (int64_t)ns->id_ns.nsfeat, 0) +NS_CFG_DEF(int, "id_ns.flbas", (int64_t)ns->id_ns.flbas, 0) +NS_CFG_DEF(int, "id_ns.nmic", (int64_t)ns->id_ns.nmic, 0) +NS_CFG_DEF(int, "ns_size", ns->size, 0) +#endif
include/hw/nvme/nvme-cfg.h new file +201 -0
201 lines not on this page · raw message
qemu-img-cmds.hx +6 -0
@@ -57,6 +57,12 @@ SRST .. option:: create [--object OBJECTDEF] [-q] [-f FMT] [-b BACKING_FILE [-F BACKING_FMT]] [-u] [-o OPTIONS] FILENAME [SIZE] ERST +DEF("createns", nsimgs_create, + "createns -S nvme_ctrl_serial_number -C nvme_ctrl_total_capacity [-N <NsId_max>] pathname") +SRST +.. option:: createns -S SERIAL_NUMBER -C TOTAL_CAPACITY [-N NSID_MAX] PATHNAME +ERST + DEF("dd", img_dd, "dd [--image-opts] [-U] [-f fmt] [-O output_fmt] [bs=block_size] [count=blocks] [skip=blocks] if=input of=output") SRST
qemu-img.c +134 -0
156 lines not on this page · raw message
Signature · 1 line
2.37.3
Re: [PATCH v3 1/2] hw/nvme: Support for Namespaces Management from guest OS - create-ns
Quoted 35 lines · Jonathan Derrick
docs/system/devices/nvme.rst +35
-0+Parameters: + +``auto-ns-path=<path to nvme storage location>`` + If specified indicates a support for dynamic management of nvme namespaces + by means of nvme create-ns command. This path points + to the storage area for backend images must exist. Additionally it requires + that parameter `ns-subsys` must be specified whereas parameter `drive` + must not. The legacy namespace backend is disabled, instead, a pair of + files 'nvme_<ctrl SN>_ns_<NS-ID>.cfg' and 'nvme_<ctrl SN>_ns_<NS-ID>.img' + will refer to respective namespaces. The create-ns, attach-ns + and detach-ns commands, issued at the guest side, will make changes to + those files accordingly. + For each namespace exists an image file in raw format and a config file + containing namespace parameters and state of the attachment allowing QEMU + to configure namespaces accordingly during start up. If for instance an + image file has a size of 0 bytes this will be interpreted as non existent + namespace. Issuing create-ns command will change the status in the config + files and and will re-size the image file accordingly so the image file + will be associated with the respective namespace. The main config file + nvme_<ctrl SN>_ctrl.cfg keeps the track of allocated capacity to the + namespaces within the nvme controller. + As it is the case of a typical hard drive, backend images together with + config files need to be created. For this reason the qemu-img tool has + been extended by adding createns command. + + qemu-img createns {-S <serial> -C <total NVMe capacity>} + [-N <NsId max>] {<path>} + + Parameters: + -S and -C and <path> are mandatory, `-S` must match `serial` parameter + and <path> must match `auto-ns-path` parameter of "-device nvme,..." + specification. + -N is optional, if specified it will set a limit to the number of potential + namespaces and will reduce the number of backend images and config files + accordingly. As a default, a set of images of 0 bytes size and default
docs/system/devices/nvme.rst There are a number of parameters available: · line 146
Quoted 10 lines
docs/system/devices/nvme.rst +10
-0+ config files for 256 namespaces will be created, a total of 513 files. + +Please note that ``nvme-ns`` device is not required to support of dynamic +namespaces management feature. It is not prohibited to assign a such device to +``nvme`` device specified to support dynamic namespace management if one has +an use case to do so, however, it will only coexist and be out of the scope of +Namespaces Management. NsIds will be consistently managed, creation (create-ns) +of a namespace will not allocate the NsId already being taken. If ``nvme-ns`` +device conflicts with previously created one by create-ns (the same NsId), +it will break QEMU's start up.
By requiring the controller's serial number up-front, does this mean we can't share dynamic namespaces with other controllers in the subsystem?
include/hw/nvme/nvme-cfg.h line 70
Quoted 7 lines
include/hw/nvme/nvme-cfg.h new file +7
-0+static inline char *create_fmt_name (const char *fmt,char *ns_directory,char *serial,uint32_t nsid,Error **errp) +{ +char *file_name =NULL ; +Error *local_err =NULL ; + +storage_path_check (ns_directory, serial, errp); +if (local_err) {
How is 'local_err' ever *not* NULL here? Are you meaning to check "*errp" instead?
Quoted 21 lines
include/hw/nvme/nvme-cfg.h new file +21
-0+error_propagate (errp, local_err); + }else { + file_name =g_strdup_printf (fmt, ns_directory, serial, nsid); + } + +return file_name; +} + +static inline char *create_cfg_name (char *ns_directory,char *serial,uint32_t nsid,Error **errp) +{ + return create_fmt_name(NS_FILE_FMT NS_CFG_EXT, ns_directory, serial, nsid, errp); +} + + +static inline char *create_image_name (char *ns_directory,char *serial,uint32_t nsid,Error **errp) +{ + return create_fmt_name(NS_FILE_FMT NS_IMG_EXT, ns_directory, serial, nsid, errp); +} + +static inline int nsid_cfg_save (char *ns_directory,char *serial,QDict *ns_cfg,uint32_t nsid) +{
include/hw/nvme/nvme-cfg.h line 101
Quoted 10 lines
include/hw/nvme/nvme-cfg.h new file +10
-0+GString *json =NULL ; +char *filename; +FILE *fp; +int ret =0 ; +Error *local_err =NULL ; + + json =qobject_to_json_pretty (QOBJECT (ns_cfg), false); + +if (strlen (json->str) +2 /* '\n'+'\0' */ >NS_CFG_MAXSIZE ) { +error_setg (&local_err,"ns-cfg allowed max size %d exceeded" ,NS_CFG_MAXSIZE );
I find this whole "local_err" control flow unpleasant to follow. The local_error gets set in this above condition only to be overwritten in the very next step without ever being used? Surely that can't be right.
I'm just picking on this one example here, but the pattern appears to repeat. I think this would be easier to read if the error conditions took 'goto' paths to unwind so that the good path doesn't require such deep 'if/else' nesting.
Quoted 29 lines
include/hw/nvme/nvme-cfg.h new file +29
-0+ } + + filename =create_cfg_name (ns_directory, serial, nsid, &local_err); +if (!local_err) { + fp =fopen (filename,"w" ); +if (fp ==NULL ) { +error_setg (&local_err,"open %s: %s" , filename, +strerror (errno)); + }else { +chmod (filename,0644 ); +if (!fprintf (fp,"%s\n" , json->str)) { +error_setg (&local_err,"could not write ns-cfg %s: %s" , filename, +strerror (errno)); + } +fclose (fp); + } + } + +if (local_err) { +error_report_err (local_err); + ret =-1 ; + } + +g_string_free (json, true); +g_free (filename); +qobject_unref (ns_cfg); + +return ret; +}
RE: [PATCH v3 1/2] hw/nvme: Support for Namespaces Management from guest OS - create-ns
-----Original Message-----
From: Keith Busch <[email protected]>
Sent: Thursday, October 27, 2022 1:21 PM
To: Jonathan Derrick <[email protected]>
Cc: [email protected]; Michael Kropaczek (CW) <[email protected]>; [email protected]; Klaus Jensen <[email protected]>; Kevin Wolf <[email protected]>; Hanna Reitz <[email protected]>
Subject: Re: [PATCH v3 1/2] hw/nvme: Support for Namespaces Management from guest OS - create-ns
[You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
Caution: External Email
Quoted 50 lines · Jonathan Derrick
(quoted diff) +50
-0+Parameters: + +``auto-ns-path=<path to nvme storage location>`` + If specified indicates a support for dynamic management of nvme +namespaces + by means of nvme create-ns command. This path points + to the storage area for backend images must exist. Additionally it +requires + that parameter `ns-subsys` must be specified whereas parameter +`drive` + must not. The legacy namespace backend is disabled, instead, a pair +of + files 'nvme_<ctrl SN>_ns_<NS-ID>.cfg' and 'nvme_<ctrl SN>_ns_<NS-ID>.img' + will refer to respective namespaces. The create-ns, attach-ns + and detach-ns commands, issued at the guest side, will make changes +to + those files accordingly. + For each namespace exists an image file in raw format and a config +file + containing namespace parameters and state of the attachment +allowing QEMU + to configure namespaces accordingly during start up. If for +instance an + image file has a size of 0 bytes this will be interpreted as non +existent + namespace. Issuing create-ns command will change the status in the +config + files and and will re-size the image file accordingly so the image +file + will be associated with the respective namespace. The main config +file + nvme_<ctrl SN>_ctrl.cfg keeps the track of allocated capacity to +the + namespaces within the nvme controller. + As it is the case of a typical hard drive, backend images together +with + config files need to be created. For this reason the qemu-img tool +has + been extended by adding createns command. + + qemu-img createns {-S <serial> -C <total NVMe capacity>} + [-N <NsId max>] {<path>} + + Parameters: + -S and -C and <path> are mandatory, `-S` must match `serial` + parameter and <path> must match `auto-ns-path` parameter of "-device nvme,..." + specification. + -N is optional, if specified it will set a limit to the number of + potential namespaces and will reduce the number of backend images + and config files accordingly. As a default, a set of images of 0
Review
Quoted 10 lines
(quoted diff) +10
-0+ bytes size and default config files for 256 namespaces will be created, a total of 513 files. + +Please note that ``nvme-ns`` device is not required to support of +dynamic namespaces management feature. It is not prohibited to assign +a such device to ``nvme`` device specified to support dynamic +namespace management if one has an use case to do so, however, it +will only coexist and be out of the scope of Namespaces Management. +NsIds will be consistently managed, creation (create-ns) of a +namespace will not allocate the NsId already being taken. If +``nvme-ns`` device conflicts with previously created one by create-ns (the same NsId), it will break QEMU's start up.
By requiring the controller's serial number up-front, does this mean we can't share dynamic namespaces with other controllers in the subsystem? [MK]: At this point namespaces of one controller can be attached to the subsystem.
[MK]: More controllers would require additional parameter specifying the number of controllers and expanding the naming pattern of backend files.
include/hw/nvme/nvme-cfg.h line 70
Quoted 7 lines
include/hw/nvme/nvme-cfg.h new file +7
-0+static inline char *create_fmt_name (const char *fmt,char +*ns_directory,char *serial,uint32_t nsid,Error **errp) { +char *file_name =NULL ; +Error *local_err =NULL ; + +storage_path_check (ns_directory, serial, errp); +if (local_err) {
How is 'local_err' ever *not* NULL here? Are you meaning to check "*errp" instead? [MK]: This will be corrected, indeed a dead code.
Quoted 22 lines
include/hw/nvme/nvme-cfg.h new file +22
-0+error_propagate (errp, local_err); + }else { + file_name =g_strdup_printf (fmt, ns_directory, serial, nsid); + } + +return file_name; +} + +static inline char *create_cfg_name (char *ns_directory,char *serial, +uint32_t nsid,Error **errp) { + return create_fmt_name(NS_FILE_FMT NS_CFG_EXT, ns_directory, +serial, nsid, errp); } + + +static inline char *create_image_name (char *ns_directory,char +*serial,uint32_t nsid,Error **errp) { + return create_fmt_name(NS_FILE_FMT NS_IMG_EXT, ns_directory, +serial, nsid, errp); } + +static inline int nsid_cfg_save (char *ns_directory,char *serial, +QDict *ns_cfg,uint32_t nsid) { +GString *json =NULL ;
include/hw/nvme/nvme-cfg.h
Quoted 10 lines
include/hw/nvme/nvme-cfg.h new file +10
-0+char *filename; +FILE *fp; +int ret =0 ; +Error *local_err =NULL ; + + json =qobject_to_json_pretty (QOBJECT (ns_cfg), false); + +if (strlen (json->str) +2 /* '\n'+'\0' */ >NS_CFG_MAXSIZE ) { +error_setg (&local_err,"ns-cfg allowed max size %d exceeded" , +NS_CFG_MAXSIZE );
I find this whole "local_err" control flow unpleasant to follow. The local_error gets set in this above condition only to be overwritten in the very next step without ever being used? Surely that can't be right.
I'm just picking on this one example here, but the pattern appears to repeat. I think this would be easier to read if the error conditions took 'goto' paths to unwind so that the good path doesn't require such deep 'if/else' nesting. [MK]: this will be corrected.
Quoted 29 lines
include/hw/nvme/nvme-cfg.h new file +29
-0+ } + + filename =create_cfg_name (ns_directory, serial, nsid, &local_err); +if (!local_err) { + fp =fopen (filename,"w" ); +if (fp ==NULL ) { +error_setg (&local_err,"open %s: %s" , filename, +strerror (errno)); + }else { +chmod (filename,0644 ); +if (!fprintf (fp,"%s\n" , json->str)) { +error_setg (&local_err,"could not write ns-cfg %s: %s" , filename, +strerror (errno)); + } +fclose (fp); + } + } + +if (local_err) { +error_report_err (local_err); + ret =-1 ; + } + +g_string_free (json, true); +g_free (filename); +qobject_unref (ns_cfg); + +return ret; +}
PATCH v3 2/2 hw/nvme: Support for Namespaces Management from guest OS - delete-ns
In reply to Jonathan DerrickFrom: Michael Kropaczek <[email protected]>
Added support for NVMEe NameSpaces Mangement allowing the guest OS to delete namespaces by issuing nvme delete-ns command.
It is an extension to currently implemented Qemu nvme virtual device. Virtual devices representing namespaces will be created and/or deleted during Qemu's running session, at anytime.
--- docs/system/devices/nvme.rst | 9 ++-- hw/nvme/ctrl.c | 86 ++++++++++++++++++++++++++++++++++-- hw/nvme/ns-backend.c | 5 +++ hw/nvme/ns.c | 74 +++++++++++++++++++++++++++++++ hw/nvme/nvme.h | 2 + hw/nvme/trace-events | 1 + include/block/nvme.h | 1 + 7 files changed, 170 insertions(+), 8 deletions(-)
docs/system/devices/nvme.rst +5 -4
@@ -103,12 +103,12 @@ Parameters: ``auto-ns-path=<path to nvme storage location>`` If specified indicates a support for dynamic management of nvme namespaces - by means of nvme create-ns command. This path points + by means of nvme create-ns and nvme delete-ns commands. This path points to the storage area for backend images must exist. Additionally it requires that parameter `ns-subsys` must be specified whereas parameter `drive` must not. The legacy namespace backend is disabled, instead, a pair of files 'nvme_<ctrl SN>_ns_<NS-ID>.cfg' and 'nvme_<ctrl SN>_ns_<NS-ID>.img' - will refer to respective namespaces. The create-ns, attach-ns + will refer to respective namespaces. The create-ns, delete-ns, attach-ns and detach-ns commands, issued at the guest side, will make changes to those files accordingly. For each namespace exists an image file in raw format and a config file
@@ -140,8 +140,9 @@ Please note that ``nvme-ns`` device is not required to support of dynamic namespaces management feature. It is not prohibited to assign a such device to ``nvme`` device specified to support dynamic namespace management if one has an use case to do so, however, it will only coexist and be out of the scope of -Namespaces Management. NsIds will be consistently managed, creation (create-ns) -of a namespace will not allocate the NsId already being taken. If ``nvme-ns`` +Namespaces Management. Deletion (delete-ns) will render an error for this +namespace. NsIds will be consistently managed, creation (create-ns) of +a namespace will not allocate the NsId already being taken. If ``nvme-ns`` device conflicts with previously created one by create-ns (the same NsId), it will break QEMU's start up.
hw/nvme/ctrl.c +82 -4
126 lines not on this page · raw message
hw/nvme/ns-backend.c +5 -0
@@ -76,6 +76,11 @@ void ns_blockdev_activate(BlockBackend *blk, uint64_t image_size, Error **errp) errp); } +void ns_blockdev_deactivate (BlockBackend *blk,Error **errp) +{ +ns_blockdev_activate (blk,0 , errp); +} +int ns_storage_path_check (NvmeCtrl *n,Error **errp) {return storage_path_check (n->params.ns_directory, n->params.serial, errp);
hw/nvme/ns.c +74 -0
86 lines not on this page · raw message
hw/nvme/nvme.h +2 -0
@@ -280,6 +280,7 @@ void nvme_ns_shutdown(NvmeNamespace *ns);void nvme_ns_cleanup (NvmeNamespace *ns);void nvme_validate_flbas (uint8_t flbas,Error **errp);NvmeNamespace *nvme_ns_create (NvmeCtrl *n,uint32_t nsid,NvmeIdNsMgmt *id_ns,Error **errp); +void nvme_ns_delete (NvmeCtrl *n,uint32_t nsid,Error **errp); typedef struct NvmeAsyncEvent { QTAILQ_ENTRY(NvmeAsyncEvent) entry;
@@ -582,6 +583,7 @@ static inline NvmeSecCtrlEntry *nvme_sctrl_for_cntlid(NvmeCtrl *n,BlockBackend *ns_blockdev_init (const char *file,Error **errp);void ns_blockdev_activate (BlockBackend *blk,uint64_t image_size,Error **errp); +void ns_blockdev_deactivate (BlockBackend *blk,Error **errp);int nvme_ns_backend_setup (NvmeCtrl *n,Error **errp);void nvme_attach_ns (NvmeCtrl *n,NvmeNamespace *ns); uint16_t nvme_bounce_data(NvmeCtrl *n, void *ptr, uint32_t len,
hw/nvme/trace-events +1 -0
@@ -79,6 +79,7 @@ pci_nvme_aer_masked(uint8_t type, uint8_t mask) "type 0x%"PRIx8" mask 0x%"PRIx8" pci_nvme_aer_post_cqe(uint8_t typ, uint8_t info, uint8_t log_page) "type 0x%"PRIx8" info 0x%"PRIx8" lid 0x%"PRIx8"" pci_nvme_ns_mgmt(uint16_t cid, uint32_t nsid, uint8_t sel, uint8_t csi, uint8_t psdt) "cid %"PRIu16", nsid=%"PRIu32", sel=0x%"PRIx8", csi=0x%"PRIx8", psdt=0x%"PRIx8"" pci_nvme_ns_create(uint16_t nsid, uint64_t nsze, uint64_t ncap, uint8_t flbas) "nsid %"PRIu16", nsze=%"PRIu64", ncap=%"PRIu64", flbas=%"PRIu8"" +pci_nvme_ns_delete(uint16_t nsid) "nsid %"PRIu16"" pci_nvme_ns_attachment(uint16_t cid, uint8_t sel) "cid %"PRIu16", sel=0x%"PRIx8"" pci_nvme_ns_attachment_attach(uint16_t cntlid, uint32_t nsid) "cntlid=0x%"PRIx16", nsid=0x%"PRIx32"" pci_nvme_enqueue_event(uint8_t typ, uint8_t info, uint8_t log_page) "type 0x%"PRIx8" info 0x%"PRIx8" lid 0x%"PRIx8""
include/block/nvme.h +1 -0
@@ -1191,6 +1191,7 @@ enum NvmeIdCtrlCmic {enum NvmeNsManagementOperation {NVME_NS_MANAGEMENT_CREATE =0x0 , +NVME_NS_MANAGEMENT_DELETE =0x1 , };enum NvmeNsAttachmentOperation {
Signature · 1 line
2.37.3