Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18183C001B0 for ; Fri, 11 Aug 2023 07:47:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234213AbjHKHrV (ORCPT ); Fri, 11 Aug 2023 03:47:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57954 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234209AbjHKHrO (ORCPT ); Fri, 11 Aug 2023 03:47:14 -0400 Received: from mailout3.samsung.com (mailout3.samsung.com [203.254.224.33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 72898211B for ; Fri, 11 Aug 2023 00:47:13 -0700 (PDT) Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout3.samsung.com (KnoxPortal) with ESMTP id 20230811074711epoutp03e59d2857b03a511896a88c5fef7e91c5~6RJh2Kk780526805268epoutp03D for ; Fri, 11 Aug 2023 07:47:11 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout3.samsung.com 20230811074711epoutp03e59d2857b03a511896a88c5fef7e91c5~6RJh2Kk780526805268epoutp03D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1691740031; bh=G2psaaFjP1ydxkfkl23H4WaUz3pNBWk9HwVL3ryBMvM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jXM2mcIJ5rxhEeht3LkakyIAV8otCaOZiRDJTrhybJV56ocCZRgy+BQgX+2tgJKfN zzLTKVq/5N6LOvxPZHumlocYO0+cwCFOApEU8Z/LnXyqjcnbIfbOM1AKBk1W/eeczv NERVZwDp2WTxD0B2WCytvwduGbvAhT6T39nx2OFs= Received: from epsnrtp1.localdomain (unknown [182.195.42.162]) by epcas5p1.samsung.com (KnoxPortal) with ESMTP id 20230811074711epcas5p1e47bb48ccfbb004ba48e709967d5ba63~6RJhmMkiP2270122701epcas5p1y; Fri, 11 Aug 2023 07:47:11 +0000 (GMT) Received: from epsmges5p2new.samsung.com (unknown [182.195.38.175]) by epsnrtp1.localdomain (Postfix) with ESMTP id 4RMbV51lFmz4x9Q7; Fri, 11 Aug 2023 07:47:09 +0000 (GMT) Received: from epcas5p3.samsung.com ( [182.195.41.41]) by epsmges5p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 9A.4E.44250.D77E5D46; Fri, 11 Aug 2023 16:47:09 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p4.samsung.com (KnoxPortal) with ESMTPA id 20230811060644epcas5p45a369165d04425b5efdb6c5e9f8ca265~6Px0c5Qcu1652616526epcas5p45; Fri, 11 Aug 2023 06:06:44 +0000 (GMT) Received: from epsmgms1p2new.samsung.com (unknown [182.195.42.42]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20230811060644epsmtrp2dd743a27e08896265378b9fd210e5758~6Px0cOmgK2376523765epsmtrp2x; Fri, 11 Aug 2023 06:06:44 +0000 (GMT) X-AuditID: b6c32a4a-c4fff7000000acda-d1-64d5e77d343f Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgms1p2new.samsung.com (Symantec Messaging Gateway) with SMTP id AB.60.30535.4FFC5D46; Fri, 11 Aug 2023 15:06:44 +0900 (KST) Received: from unvme-desktop.sa.corp.samsungelectronics.net (unknown [107.99.41.39]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20230811060642epsmtip16bbcc0de2d74c26e4a899fcdf7216355~6PxzBYrlL2768827688epsmtip1T; Fri, 11 Aug 2023 06:06:42 +0000 (GMT) From: Ankit Kumar To: axboe@kernel.dk, vincentfu@gmail.com Cc: fio@vger.kernel.org, kbusch@kernel.org, joshi.k@samsung.com, martin.petersen@oracle.com, Ankit Kumar Subject: [PATCH v2 07/10] engines:io_uring: generate and verify pi for 16b guard Date: Fri, 11 Aug 2023 16:55:44 +0530 Message-Id: <20230811112547.92345-8-ankit.kumar@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230811112547.92345-1-ankit.kumar@samsung.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprIJsWRmVeSWpSXmKPExsWy7bCmpm7t86spBl1fOS3WXPnNbrH6bj+b xcdZL5gtjv5/y2Yx6dA1Rovlx/8xWWxoE3Rg99g56y67x+WzpR6bVnWyeXx8eovFo2/LKkaP z5vkAtiism0yUhNTUosUUvOS81My89JtlbyD453jTc0MDHUNLS3MlRTyEnNTbZVcfAJ03TJz gC5RUihLzCkFCgUkFhcr6dvZFOWXlqQqZOQXl9gqpRak5BSYFOgVJ+YWl+al6+WlllgZGhgY mQIVJmRnvLwjWLDPv+LwpXtsDYwL7LsYOTkkBEwkrlw8wt7FyMUhJLCbUWJWwzdWCOcTo8SB vZ0sEM43RoneiXOYYVq+7n/LCJHYyygxb8crqP5OJonnmxpYQKrYBLQlXr29AdTBwSEioCGx 5m4qSA2zQAujxO7u2awgNcICQRKnt69gA7FZBFQlzmy+yA5i8wrYSLz8N5EdYpu8xMxL39lB 5nAK2EosWiEGUSIocXLmE7BVzEAlzVtnM4PMlxD4yC4xcXEjG0Svi8T+V+eg5ghLvDq+BcqW kvj8bi9UTbbEpoc/mSDsAokjL3qhvrSXaD3VD3Y/s4CmxPpd+hBhWYmpp9YxQezlk+j9/QSq lVdixzwYW1Xi773bLBC2tMTNd1ehbA+Jf7MawE4QEpjAKPH0r+AERoVZSN6ZheSdWQibFzAy r2KUTC0ozk1PLTYtMMpLLYfHcXJ+7iZGcNrU8trB+PDBB71DjEwcjIcYJTiYlUR4bYMvpQjx piRWVqUW5ccXleakFh9iNAUG90RmKdHkfGDiziuJNzSxNDAxMzMzsTQ2M1QS533dOjdFSCA9 sSQ1OzW1ILUIpo+Jg1OqgUnqsPS7enM5+a2mXhfOMy28eXJVz6ZHidW5druE01M4p5ya8H0x i76yEJ+1XtNmr7QU/gVXJyu4VjeLfJ11oiVoGnetX5HTeWO9RYF5t3uNNszjtvkpEtDqWuqy 9bnHn58NPJ73S+MV9/lPXXB8r9vE5yxRH+6GrHTnDjzLvtq65bZqp+PU15t/L5AxlSv1io9r dzmsnK4hfOoDk3DJLBkVL9HswmcR33ULeoN/RuxozpV6vDZ3jYGXYxTHfslqhpV3hbb6WfAE 1v/QcFtu2W5eIni67Glzzf5et/8bPDTDDoWxemhHliTOStnG8t150xNmAabA+FXczPKBXLXv 0nefPh0SV8Riy25010WJpTgj0VCLuag4EQCvYLDBJAQAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrBLMWRmVeSWpSXmKPExsWy7bCSnO6X81dTDH6/YLNYc+U3u8Xqu/1s Fh9nvWC2OPr/LZvFpEPXGC2WH//HZLGhTdCB3WPnrLvsHpfPlnpsWtXJ5vHx6S0Wj74tqxg9 Pm+SC2CL4rJJSc3JLEst0rdL4Mp4eUewYJ9/xeFL99gaGBfYdzFyckgImEh83f+WsYuRi0NI YDejxM7t71m7GDmAEtISC9cnQtQIS6z895wdoqadSeJh+w52kASbgLbEq7c3mEFsEQEtiWPr mthAipgFuhgl9u6exgaSEBYIkHjavokVxGYRUJU4s/kiWDOvgI3Ey38T2SE2yEvMvPSdHWQx p4CtxKIVYiBhIaCSVzN/s0GUC0qcnPmEBcRmBipv3jqbeQKjwCwkqVlIUgsYmVYxSqYWFOem 5xYbFhjlpZbrFSfmFpfmpesl5+duYgQHtpbWDsY9qz7oHWJk4mA8xCjBwawkwmsbfClFiDcl sbIqtSg/vqg0J7X4EKM0B4uSOO+3170pQgLpiSWp2ampBalFMFkmDk6pBibHpLTqZZ6rlvHs 3VJqfOSy3w5BiYiWmNVRy1nezzyjc+3Zs97SDc277eJW7VDbuFM98iuvjdUlhQ3MP+yUjb+u 5lriM0UuY6Yry99dS+UTJK+bfpl+j8990UKL8kcba42e72OLiC5d4ZjA6iRlalD17XVBd3jN H2nj8qkR+3ZUdN9wUWHZ/na3xzqDzcpSxXLdtZ9ZfL+n+gnImqe5ny2JD+fkeXjI891ebhXZ hXMy/xfOv9jHfS1h02n7tIAPIdlt/DcDpWZrTntetlridPL7HUc7+9KnZU10WuLh+Vl2xk3f 3kPLluU9NPOeZKm8YNf5+g1fls7SnKO2hjfBjPfE+44s4z3WO/OSIvxDg5VYijMSDbWYi4oT AUYI2H/bAgAA X-CMS-MailID: 20230811060644epcas5p45a369165d04425b5efdb6c5e9f8ca265 X-Msg-Generator: CA Content-Type: text/plain; charset="utf-8" X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20230811060644epcas5p45a369165d04425b5efdb6c5e9f8ca265 References: <20230811112547.92345-1-ankit.kumar@samsung.com> Precedence: bulk List-ID: X-Mailing-List: fio@vger.kernel.org Generate and verify protection information for 16 bit guard format, for the nvme backend of io_uring_cmd ioengine. The support is there for both the cases where metadata is transferred in separate buffer, or transferred at the end of logical block creating an extended logical block. This support also takes into consideration when protection information resides in last or first 8 bytes of metadata. Signed-off-by: Ankit Kumar --- engines/io_uring.c | 34 +++++++++ engines/nvme.c | 173 +++++++++++++++++++++++++++++++++++++++++++++ engines/nvme.h | 12 ++++ 3 files changed, 219 insertions(+) diff --git a/engines/io_uring.c b/engines/io_uring.c index 376a2a27..7ac7c755 100644 --- a/engines/io_uring.c +++ b/engines/io_uring.c @@ -456,7 +456,9 @@ static struct io_u *fio_ioring_cmd_event(struct thread_data *td, int event) struct ioring_options *o = td->eo; struct io_uring_cqe *cqe; struct io_u *io_u; + struct nvme_data *data; unsigned index; + int ret; index = (event + ld->cq_ring_off) & ld->cq_ring_mask; if (o->cmd_type == FIO_URING_CMD_NVME) @@ -470,6 +472,15 @@ static struct io_u *fio_ioring_cmd_event(struct thread_data *td, int event) else io_u->error = 0; + if (o->cmd_type == FIO_URING_CMD_NVME) { + data = FILE_ENG_DATA(io_u->file); + if (data->pi_type && (io_u->ddir == DDIR_READ) && !o->pi_act) { + ret = fio_nvme_pi_verify(data, io_u); + if (ret) + io_u->error = ret; + } + } + return io_u; } @@ -1190,6 +1201,7 @@ static int fio_ioring_io_u_init(struct thread_data *td, struct io_u *io_u) { struct ioring_data *ld = td->io_ops_data; struct ioring_options *o = td->eo; + struct nvme_pi_data *pi_data; char *p; ld->io_u_index[io_u->index] = io_u; @@ -1198,11 +1210,32 @@ static int fio_ioring_io_u_init(struct thread_data *td, struct io_u *io_u) p = PTR_ALIGN(ld->md_buf, page_mask) + td->o.mem_align; p += o->md_per_io_size * io_u->index; io_u->mmap_data = p; + + if (!o->pi_act) { + pi_data = calloc(1, sizeof(*pi_data)); + pi_data->io_flags |= o->prchk; + pi_data->apptag_mask = o->apptag_mask; + pi_data->apptag = o->apptag; + io_u->engine_data = pi_data; + } } return 0; } +static void fio_ioring_io_u_free(struct thread_data *td, struct io_u *io_u) +{ + struct ioring_options *o = td->eo; + struct nvme_pi *pi; + + if (!strcmp(td->io_ops->name, "io_uring_cmd") && + (o->cmd_type == FIO_URING_CMD_NVME)) { + pi = io_u->engine_data; + free(pi); + io_u->engine_data = NULL; + } +} + static int fio_ioring_open_file(struct thread_data *td, struct fio_file *f) { struct ioring_data *ld = td->io_ops_data; @@ -1411,6 +1444,7 @@ static struct ioengine_ops ioengine_uring_cmd = { .init = fio_ioring_init, .post_init = fio_ioring_cmd_post_init, .io_u_init = fio_ioring_io_u_init, + .io_u_free = fio_ioring_io_u_free, .prep = fio_ioring_cmd_prep, .queue = fio_ioring_queue, .commit = fio_ioring_commit, diff --git a/engines/nvme.c b/engines/nvme.c index 8793d742..e308331f 100644 --- a/engines/nvme.c +++ b/engines/nvme.c @@ -4,6 +4,7 @@ */ #include "nvme.h" +#include "../crc/crc-t10dif.h" static inline __u64 get_slba(struct nvme_data *data, struct io_u *io_u) { @@ -21,6 +22,158 @@ static inline __u32 get_nlb(struct nvme_data *data, struct io_u *io_u) return (io_u->xfer_buflen >> data->lba_shift) - 1; } +static void fio_nvme_generate_pi_16b_guard(struct nvme_data *data, + struct io_u *io_u, + struct nvme_cmd_ext_io_opts *opts) +{ + struct nvme_pi_data *pi_data = io_u->engine_data; + struct nvme_16b_guard_pif *pi; + unsigned char *buf = io_u->xfer_buf; + unsigned char *md_buf = io_u->mmap_data; + __u64 slba = get_slba(data, io_u); + __u32 nlb = get_nlb(data, io_u) + 1; + __u32 lba_num = 0; + __u16 guard = 0; + + if (data->pi_loc) { + if (data->lba_ext) + pi_data->interval = data->lba_ext - data->ms; + else + pi_data->interval = 0; + } else { + if (data->lba_ext) + pi_data->interval = data->lba_ext - sizeof(struct nvme_16b_guard_pif); + else + pi_data->interval = data->ms - sizeof(struct nvme_16b_guard_pif); + } + + if (io_u->ddir != DDIR_WRITE) + return; + + while (lba_num < nlb) { + if (data->lba_ext) + pi = (struct nvme_16b_guard_pif *)(buf + pi_data->interval); + else + pi = (struct nvme_16b_guard_pif *)(md_buf + pi_data->interval); + + if (opts->io_flags & NVME_IO_PRINFO_PRCHK_GUARD) { + if (data->lba_ext) { + guard = fio_crc_t10dif_generic(0, buf, pi_data->interval); + } else { + guard = fio_crc_t10dif_generic(0, buf, data->lba_size); + guard = fio_crc_t10dif_generic(guard, md_buf, pi_data->interval); + } + pi->guard = cpu_to_be16(guard); + } + + if (opts->io_flags & NVME_IO_PRINFO_PRCHK_APP) + pi->apptag = cpu_to_be16(pi_data->apptag); + + if (opts->io_flags & NVME_IO_PRINFO_PRCHK_REF) { + switch (data->pi_type) { + case NVME_NS_DPS_PI_TYPE1: + case NVME_NS_DPS_PI_TYPE2: + pi->srtag = cpu_to_be32((__u32)slba + lba_num); + break; + case NVME_NS_DPS_PI_TYPE3: + break; + } + } + if (data->lba_ext) { + buf += data->lba_ext; + } else { + buf += data->lba_size; + md_buf += data->ms; + } + lba_num++; + } +} + +static int fio_nvme_verify_pi_16b_guard(struct nvme_data *data, + struct io_u *io_u) +{ + struct nvme_pi_data *pi_data = io_u->engine_data; + struct nvme_16b_guard_pif *pi; + struct fio_file *f = io_u->file; + unsigned char *buf = io_u->xfer_buf; + unsigned char *md_buf = io_u->mmap_data; + __u64 slba = get_slba(data, io_u); + __u32 nlb = get_nlb(data, io_u) + 1; + __u32 lba_num = 0; + __u16 unmask_app, unmask_app_exp, guard = 0; + + while (lba_num < nlb) { + if (data->lba_ext) + pi = (struct nvme_16b_guard_pif *)(buf + pi_data->interval); + else + pi = (struct nvme_16b_guard_pif *)(md_buf + pi_data->interval); + + if (data->pi_type == NVME_NS_DPS_PI_TYPE3) { + if (pi->apptag == NVME_PI_APP_DISABLE && + pi->srtag == NVME_PI_REF_DISABLE) + goto next; + } else if (data->pi_type == NVME_NS_DPS_PI_TYPE1 || + data->pi_type == NVME_NS_DPS_PI_TYPE2) { + if (pi->apptag == NVME_PI_APP_DISABLE) + goto next; + } + + if (pi_data->io_flags & NVME_IO_PRINFO_PRCHK_GUARD) { + if (data->lba_ext) { + guard = fio_crc_t10dif_generic(0, buf, pi_data->interval); + } else { + guard = fio_crc_t10dif_generic(0, buf, data->lba_size); + guard = fio_crc_t10dif_generic(guard, md_buf, pi_data->interval); + } + if (be16_to_cpu(pi->guard) != guard) { + log_err("%s: Guard compare error: LBA: %llu Expected=%x, Actual=%x\n", + f->file_name, (unsigned long long)slba, + guard, be16_to_cpu(pi->guard)); + return -EIO; + } + } + + if (pi_data->io_flags & NVME_IO_PRINFO_PRCHK_APP) { + unmask_app = be16_to_cpu(pi->apptag) & pi_data->apptag_mask; + unmask_app_exp = pi_data->apptag & pi_data->apptag_mask; + if (unmask_app != unmask_app_exp) { + log_err("%s: APPTAG compare error: LBA: %llu Expected=%x, Actual=%x\n", + f->file_name, (unsigned long long)slba, + unmask_app_exp, unmask_app); + return -EIO; + } + } + + if (pi_data->io_flags & NVME_IO_PRINFO_PRCHK_REF) { + switch (data->pi_type) { + case NVME_NS_DPS_PI_TYPE1: + case NVME_NS_DPS_PI_TYPE2: + if (be32_to_cpu(pi->srtag) != + ((__u32)slba + lba_num)) { + log_err("%s: REFTAG compare error: LBA: %llu Expected=%x, Actual=%x\n", + f->file_name, (unsigned long long)slba, + (__u32)slba + lba_num, + be32_to_cpu(pi->srtag)); + return -EIO; + } + break; + case NVME_NS_DPS_PI_TYPE3: + break; + } + } +next: + if (data->lba_ext) { + buf += data->lba_ext; + } else { + buf += data->lba_size; + md_buf += data->ms; + } + lba_num++; + } + + return 0; +} + void fio_nvme_uring_cmd_trim_prep(struct nvme_uring_cmd *cmd, struct io_u *io_u, struct nvme_dsm_range *dsm) { @@ -96,6 +249,11 @@ void fio_nvme_pi_fill(struct nvme_uring_cmd *cmd, struct io_u *io_u, slba = get_slba(data, io_u); cmd->cdw12 |= opts->io_flags; + if (data->pi_type && !(opts->io_flags & NVME_IO_PRINFO_PRACT)) { + if (data->guard_type == NVME_NVM_NS_16B_GUARD) + fio_nvme_generate_pi_16b_guard(data, io_u, opts); + } + switch (data->pi_type) { case NVME_NS_DPS_PI_TYPE1: case NVME_NS_DPS_PI_TYPE2: @@ -120,6 +278,21 @@ void fio_nvme_pi_fill(struct nvme_uring_cmd *cmd, struct io_u *io_u, } } +int fio_nvme_pi_verify(struct nvme_data *data, struct io_u *io_u) +{ + int ret = 0; + + switch (data->guard_type) { + case NVME_NVM_NS_16B_GUARD: + ret = fio_nvme_verify_pi_16b_guard(data, io_u); + break; + default: + break; + } + + return ret; +} + static int nvme_identify(int fd, __u32 nsid, enum nvme_identify_cns cns, enum nvme_csi csi, void *data) { diff --git a/engines/nvme.h b/engines/nvme.h index f3598352..a1102dfe 100644 --- a/engines/nvme.h +++ b/engines/nvme.h @@ -44,6 +44,9 @@ struct nvme_uring_cmd { #define NVME_IDENTIFY_CSI_SHIFT 24 #define NVME_NQN_LENGTH 256 +#define NVME_PI_APP_DISABLE 0xFFFF +#define NVME_PI_REF_DISABLE 0xFFFFFFFF + #define NVME_ZNS_ZRA_REPORT_ZONES 0 #define NVME_ZNS_ZRAS_FEAT_ERZ (1 << 16) #define NVME_ZNS_ZSA_RESET 0x4 @@ -131,6 +134,13 @@ enum nvme_io_control_flags { NVME_IO_PRINFO_PRACT = 1U << 29, }; +struct nvme_pi_data { + __u32 interval; + __u32 io_flags; + __u16 apptag; + __u16 apptag_mask; +}; + struct nvme_lbaf { __le16 ms; __u8 ds; @@ -415,6 +425,8 @@ int fio_nvme_uring_cmd_prep(struct nvme_uring_cmd *cmd, struct io_u *io_u, void fio_nvme_pi_fill(struct nvme_uring_cmd *cmd, struct io_u *io_u, struct nvme_cmd_ext_io_opts *opts); +int fio_nvme_pi_verify(struct nvme_data *data, struct io_u *io_u); + int fio_nvme_get_zoned_model(struct thread_data *td, struct fio_file *f, enum zbd_zoned_model *model); -- 2.25.1