Received: from mail-106120.protonmail.ch (mail-106120.protonmail.ch [79.135.106.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CBBEB3C4B8A for ; Thu, 4 Jun 2026 19:37:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=79.135.106.120 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780601838; cv=none; b=exhV0xs0WRkSHTm1K7zHm8v3qEtTSqc9Qnn93BF1+UJyG7niQCCjrj2lsestsUhkHF4Rv2lA5yhIFIbbrt9uOjKkUAUZjweHUZmkmrq8NV8Uhig3Wuso9CIgKyvInoWtOBjoqzFd3xf9H/iDgJQ9HLXK/8hA952IgkokWg12L3c= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780601838; c=relaxed/simple; bh=oS7NxEacAgL0HYtv5WvCgJkETMPNe0vdRBVK6cP564w=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=H7uoyVhbeEqgkXWB5ZUFemcgdGvL6v6e/joE3S6Y8TzQIRjmouXZdo6tPf3hGlFksjjNaWsUKY/L1WvkplEp2s57fn3qdWgA3JRFK+Eddmnx2n5QiYuEBD1Yi7uuxgJYuFg1UbzhdFuWKQ2zU1Y9b/jBNPapsFkpwdZy6VPzd98= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=proton.me; spf=pass smtp.mailfrom=proton.me; dkim=pass (2048-bit key) header.d=proton.me header.i=@proton.me header.b=gO6rlClG; arc=none smtp.client-ip=79.135.106.120 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=proton.me Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=proton.me Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=proton.me header.i=@proton.me header.b="gO6rlClG" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=proton.me; s=pidt2ukzyrggdauwg4b2q6m3ty.protonmail; t=1780601819; x=1780861019; bh=ujDKy6SWWWi8FQFzqh264wUeyQqzjQjB5x1dbv93N9A=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector; b=gO6rlClGX2rUj1P/zjADI2XSlrzTs/Z8DIcwM6+vGcKbbPTHF944ed7AHU/svnDAB lKZdpDbDXFiErt27KaRWARK7g37jnGakUYP7SKuatzc9d0/q6F5fX9mbTOgUVyJJTg gtxameNyvkRyftqodHv98Tg6ahoZgSu03F8XjQC3OIXEo4EwWSDA2l04HKKxrhNl5n 6STdQgx4i9RvHBTJ1vZCaZmMX95BR9wgLKc8OichtUO1J0CntObHzHt44mnZdWi0DD 76zOssE3t82y4aGjHPGsrN+xhu4EGdGtlYd4O81cIQ26+01LiJeay2tg0pnjRJdfII G5BSGnLCnUQdw== Date: Thu, 04 Jun 2026 19:36:54 +0000 To: Christoph Hellwig , Sagi Grimberg , Keith Busch , Chaitanya Kulkarni From: Bryam Vargas Cc: linux-nvme@lists.infradead.org, linux-rdma@vger.kernel.org, linux-block@vger.kernel.org Subject: [PATCH v2] nvmet-rdma: handle inline data with a nonzero offset Message-ID: <20260604193645.178350-1-hexlabsecurity@proton.me> In-Reply-To: References: <20260604084624.120032-1-hexlabsecurity@proton.me> Feedback-ID: 199661219:user:proton X-Pm-Message-ID: e669ea10f8438003e78f5e4cf1408c764e65c494 Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable nvmet_rdma_use_inline_sg() maps the host-controlled inline data offset into the per-command inline scatterlist. The bounds check admits any offset with off + len <=3D inline_data_size, but the mapping still assumes the data begins in the first inline page: =09sg->offset =3D off; =09sg->length =3D min_t(int, len, PAGE_SIZE - off); When a port is configured with inline_data_size > PAGE_SIZE (settable up to max(SZ_16K, PAGE_SIZE)), an offset in (PAGE_SIZE, inline_data_size] makes "PAGE_SIZE - off" underflow, so sg->length is set to ~4 GiB and the block backend reads far past the first inline page. num_pages(len) also ignores the offset, so an in-bounds offset whose [off, off+len) span crosses a page boundary under-counts the scatterlist. Map the offset properly: split it into a page index and an in-page offset, start the scatterlist at that page, and size the page count from page_off + len. Because the request scatterlist may now start at inline_sg[page_idx] rather than inline_sg[0], generalize the inline-SGL identity test in nvmet_rdma_release_rsp() to a range test; otherwise the persistent inline scatterlist is mistaken for an allocated one and nvmet_req_free_sgls() frees an inline page (and warns in free_large_kmalloc()). Fixes: 0d5ee2b2ab4f ("nvmet-rdma: support max(16KB, PAGE_SIZE) inline data"= ) Cc: stable@vger.kernel.org Suggested-by: Keith Busch Reported-by: Bryam Vargas Signed-off-by: Bryam Vargas --- v1 rejected a nonzero offset; per Keith's note a nonzero in-capsule SGL offset is legitimate (it is the per-command SGL Offset field, distinct from the controller ICDOFF attribute that nvme_rdma_setup_ctrl() refuses when nonzero), so v2 handles it instead, using Keith's suggested page_idx/page_off form for nvmet_rdma_use_inline_sg(). Review context (not for the commit log): Bound safety: with off + len <=3D inline_data_size the highest inline_sg[] index touched is page_idx + sg_count - 1 =3D floor((off + len - 1) / PAGE_SIZE) <=3D num_pages(inline_data_size) - 1 =3D inline_page_count - 1 (<=3D NVMET_RDMA_MAX_INLINE_SGE - 1), and page_off < PAGE_SIZE so PAGE_SIZE - page_off cannot underflow. The release_rsp range test is a strict generalization of the old "!=3D inline_sg" test: inline_sg[0] is in range (unchanged: not freed), allocated/keyed SGLs are outside it (still freed), and only the new inline_sg[1..] starts are additionally treated as inline. Decides identically on 32- and 64-bit builds: off is u64, so the offset arithmetic and PAGE_SIZE - page_off are evaluated in 64-bit on both ABIs; num_pages() sees page_off + len <=3D 16384 (positive, int-safe on both); the release_rsp comparison is a pointer comparison, identical semantics on ILP32 and LP64. (-m32/-m64 model output identical.) A/B on a KASAN build (inline_data_size =3D 16384) over an rdma_rxe loopback nvmet-rdma target with a block backend, inline write: - offset 0: succeeds, clean (control + no regression). - offset 8192: before this patch the block backend reads out of bounds BUG: KASAN: slab-out-of-bounds in copy_folio_from_iter_atomic (sg->length =3D 0xfffff000); with this patch it is served from the correct inline page, in bounds, no KASAN and no free_large_kmalloc warning. - the use_inline_sg() rework alone (without the release_rsp change) trips on offset 8192: WARNING: ... free_large_kmalloc ... Not a kmalloc allocation nvmet_req_free_sgls <- nvmet_rdma_release_rsp <- nvmet_rdma_send_do= ne drivers/nvme/target/rdma.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c index 565183a20007..eb975fbd74a1 100644 --- a/drivers/nvme/target/rdma.c +++ b/drivers/nvme/target/rdma.c @@ -666,7 +666,8 @@ static void nvmet_rdma_release_rsp(struct nvmet_rdma_rs= p *rsp) =09if (rsp->n_rdma) =09=09nvmet_rdma_rw_ctx_destroy(rsp); -=09if (rsp->req.sg !=3D rsp->cmd->inline_sg) +=09if (rsp->req.sg < rsp->cmd->inline_sg || +=09 rsp->req.sg >=3D rsp->cmd->inline_sg + queue->dev->inline_page_coun= t) =09=09nvmet_req_free_sgls(&rsp->req); =09if (unlikely(!list_empty_careful(&queue->rsp_wr_wait_list))) @@ -821,24 +822,25 @@ static void nvmet_rdma_write_data_done(struct ib_cq *= cq, struct ib_wc *wc) static void nvmet_rdma_use_inline_sg(struct nvmet_rdma_rsp *rsp, u32 len, =09=09u64 off) { -=09int sg_count =3D num_pages(len); +=09u64 page_off =3D off % PAGE_SIZE; +=09u64 page_idx =3D off / PAGE_SIZE; +=09int sg_count =3D num_pages(page_off + len); =09struct scatterlist *sg; =09int i; -=09sg =3D rsp->cmd->inline_sg; +=09sg =3D &rsp->cmd->inline_sg[page_idx]; =09for (i =3D 0; i < sg_count; i++, sg++) { =09=09if (i < sg_count - 1) =09=09=09sg_unmark_end(sg); =09=09else =09=09=09sg_mark_end(sg); -=09=09sg->offset =3D off; -=09=09sg->length =3D min_t(int, len, PAGE_SIZE - off); +=09=09sg->offset =3D page_off; +=09=09sg->length =3D min_t(u64, len, PAGE_SIZE - page_off); =09=09len -=3D sg->length; -=09=09if (!i) -=09=09=09off =3D 0; +=09=09page_off =3D 0; =09} -=09rsp->req.sg =3D rsp->cmd->inline_sg; +=09rsp->req.sg =3D &rsp->cmd->inline_sg[page_idx]; =09rsp->req.sg_cnt =3D sg_count; } -- 2.43.0