Skip to contentSkip to conversation

nvmet: add support reading with offset from ANA log

PATCH In review Daniel Wagner 2 participants 4 messages Jan 10, 2022 – Jan 19, 2022

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 nvmet: add support reading with offset from ANA log

Add support to read with offsets from ANA log buffer.

The controller claims to support extended data for the Get Log Page command (including extended Number of Dwords and Log Page Offset 2 fields):

lpa : 0x7

  [2:2] : 0x1   Extended data for Get Log Page Supported
  [1:1] : 0x1   Command Effects Log Page Supported
  [0:0] : 0x1   SMART/Health Log Page per NS Supported

---
v2:
- fixed lenght calculation and limit response lenght
- some more testing with KASAN enabled
v1:
- https://lore.kernel.org/linux-nvme/[email protected]/

 drivers/nvme/target/admin-cmd.c | 41 +++++++++++++++++++--------------
 1 file changed, 24 insertions(+), 17 deletions(-)
drivers/nvme/target/admin-cmd.c +24 -17
index 6fb24746de06..c4f1bbb9fb8a 100644
@@ -263,35 +263,42 @@ static u32 nvmet_format_ana_group(struct nvmet_req *req, u32 grpid,
 	desc->nnsids = cpu_to_le32(count);
 	desc->chgcnt = cpu_to_le64(nvmet_ana_chgcnt);
 	desc->state = req->port->ana_state[grpid];
-	memset(desc->rsvd17, 0, sizeof(desc->rsvd17));
 	return struct_size(desc, nsids, count);
 }
 
 static void nvmet_execute_get_log_page_ana(struct nvmet_req *req)
 {
-	struct nvme_ana_rsp_hdr hdr = { 0, };
+	struct nvme_ana_rsp_hdr *hdr;
 	struct nvme_ana_group_desc *desc;
-	size_t offset = sizeof(struct nvme_ana_rsp_hdr); /* start beyond hdr */
-	size_t len;
+	u64 offset = nvmet_get_log_page_offset(req->cmd);
+	u32 len;
+	void *buffer;
 	u32 grpid;
 	u16 ngrps = 0;
 	u16 status;
 
+	if (offset & 0x3) {
+		req->error_loc =
+			offsetof(struct nvme_get_log_page_command, lpo);
+		status = NVME_SC_INVALID_FIELD | NVME_SC_DNR;
+		goto out;
+	}
+
 	status = NVME_SC_INTERNAL;
-	desc = kmalloc(struct_size(desc, nsids, NVMET_MAX_NAMESPACES),
-		       GFP_KERNEL);
-	if (!desc)
+	len = sizeof(*hdr) + struct_size(desc, nsids, NVMET_MAX_NAMESPACES);
+	buffer = kzalloc(len, GFP_KERNEL);
+	if (!buffer)
 		goto out;
 
+	len = sizeof(*hdr);
+	hdr = buffer;
+	desc = buffer + sizeof(*hdr);
+
 	down_read(&nvmet_ana_sem);
 	for (grpid = 1; grpid <= NVMET_MAX_ANAGRPS; grpid++) {
 		if (!nvmet_ana_group_enabled[grpid])
 			continue;
-		len = nvmet_format_ana_group(req, grpid, desc);
-		status = nvmet_copy_to_sgl(req, offset, desc, len);
-		if (status)
-			break;
-		offset += len;
+		len += nvmet_format_ana_group(req, grpid, desc);
 		ngrps++;
 	}
 	for ( ; grpid <= NVMET_MAX_ANAGRPS; grpid++) {
@@ -299,15 +306,15 @@ static void nvmet_execute_get_log_page_ana(struct nvmet_req *req)
 			ngrps++;
 	}
 
-	hdr.chgcnt = cpu_to_le64(nvmet_ana_chgcnt);
-	hdr.ngrps = cpu_to_le16(ngrps);
+	hdr->chgcnt = cpu_to_le64(nvmet_ana_chgcnt);
+	hdr->ngrps = cpu_to_le16(ngrps);
 	nvmet_clear_aen_bit(req, NVME_AEN_BIT_ANA_CHANGE);
 	up_read(&nvmet_ana_sem);
 
-	kfree(desc);
+	status = nvmet_copy_to_sgl(req, 0, buffer + offset,
+				   min(len, nvmet_get_log_page_len(req->cmd)));
 
-	/* copy the header last once we know the number of groups */
-	status = nvmet_copy_to_sgl(req, 0, &hdr, sizeof(hdr));
+	kfree(buffer);
 out:
 	nvmet_req_complete(req, status);
 }
Signature · 1 line
2.34.1
#1 lore raw
Quoted 11 lines · Daniel Wagner

Add support to read with offsets from ANA log buffer.

The controller claims to support extended data for the Get Log Page command (including extended Number of Dwords and Log Page Offset 2 fields):

lpa : 0x7

  [2:2] : 0x1   Extended data for Get Log Page Supported
  [1:1] : 0x1   Command Effects Log Page Supported
  [0:0] : 0x1   SMART/Health Log Page per NS Supported
Quoted 7 lines

---
v2:
- fixed lenght calculation and limit response lenght
- some more testing with KASAN enabled
v1:
- https://lore.kernel.org/linux-nvme/[email protected]/

Please wait with this patch. I think there is something wrong with my calculation. I want to test some more just didn't have time for it yet.

#2 lore raw
In reply to Daniel Wagner

Daniel,

Quoted 6 lines · Daniel Wagner

Add support to read with offsets from ANA log buffer.

The controller claims to support extended data for the Get Log Page command (including extended Number of Dwords and Log Page Offset 2 fields):

Quoted 7 lines

lpa : 0x7

   [2:2] : 0x1   Extended data for Get Log Page Supported
   [1:1] : 0x1   Command Effects Log Page Supported
   [0:0] : 0x1   SMART/Health Log Page per NS Supported

---

I'm not against having this code but do you really have a client side usecase for this one that you can please explain ?

Existing code should work fine to read the log pages and I've not received any complaints so far ..

#3 lore raw

Hi Chaitanya,

Quoted 6 lines · Chaitanya Kulkarni

I'm not against having this code but do you really have a client side usecase for this one that you can please explain ?

Existing code should work fine to read the log pages and I've not received any complaints so far ..

All started with this call trace:

nvme nvme17: new ctrl: NQN "nqn.2014-08.org.nvmexpress.discovery", addr 100.102.252.12:8009
nvme: page allocation failure: order:7, mode:0x40cc0(GFP_KERNEL|__GFP_COMP), nodemask=(null),cpuset=/,mems_allowed=0-1

CPU: 1 PID: 96790 Comm: nvme Kdump: loaded Tainted: G           OE  X    5.3.18-59.27-default #1 SLE15-SP3
Call Trace:
 dump_stack+0x66/0x8b
 warn_alloc+0x10b/0x190
 ? __alloc_pages_direct_compact+0x15e/0x170
 __alloc_pages_slowpath+0xcea/0xd20
 __alloc_pages_nodemask+0x2cb/0x320
 kmalloc_order+0x18/0x70
 kmalloc_order_trace+0x1d/0xa0
 __kmalloc+0x257/0x2b0
 nvme_mpath_init_identify+0xe5/0x190 [nvme_core]
 ? blk_queue_write_cache+0x30/0x60
 nvme_init_ctrl_finish+0x340/0xde0 [nvme_core]
 ? blk_mq_run_hw_queue+0x77/0x100
 nvme_tcp_setup_ctrl+0x396/0x680 [nvme_tcp]
 nvme_tcp_create_ctrl+0x31e/0x3c7 [nvme_tcp]
 nvmf_dev_write+0xb04/0xd70 [nvme_fabrics]
 ? common_file_perm+0x64/0x190
 ? vfs_write+0xad/0x1b0
 vfs_write+0xad/0x1b0
 ksys_write+0xa1/0xe0
 do_syscall_64+0x5b/0x1e0
 entry_SYSCALL_64_after_hwframe+0x44/0xa9
RIP: 0033:0x7fb44848bc30

The target says it support 49 ANA Group Descriptors and 65536 namespaces. The current code in Linux is allocating an ANA buffer large enough to read the complete ANA log in one operation.

With the above numbers the buffer is

16 + 16 * 49 + 65365 * 4 = 262260 bytes

large, ca 64.028 4k pages. This means when allocating the buffer, the smallest bucket to allocated from is the 128kB (2^7).

So my idea was to limit the ana log buffer and read the it in chunks if needed.

Anyway, I started with replicating the situation and found out that the target code does not support reads with offsets hence this patch.

Daniel