nvmet: add support reading with offset from ANA log
Subsystems (2)
PATCH v2 1/1 nvmet: add support reading with offset from ANA log
Add support to read with offsets from ANA log buffer.
The controller claims to support extended data for the Get Log Page command (including extended Number of Dwords and Log Page Offset 2 fields):
lpa : 0x7
[2:2] : 0x1 Extended data for Get Log Page Supported [1:1] : 0x1 Command Effects Log Page Supported [0:0] : 0x1 SMART/Health Log Page per NS Supported
---
v2:
- fixed lenght calculation and limit response lenght
- some more testing with KASAN enabled
v1:
- https://lore.kernel.org/linux-nvme/[email protected]/
drivers/nvme/target/admin-cmd.c | 41 +++++++++++++++++++-------------- 1 file changed, 24 insertions(+), 17 deletions(-)
drivers/nvme/target/admin-cmd.c +24 -17
@@ -263,35 +263,42 @@ static u32 nvmet_format_ana_group(struct nvmet_req *req, u32 grpid, desc->nnsids =cpu_to_le32 (count); desc->chgcnt =cpu_to_le64 (nvmet_ana_chgcnt); desc->state = req->port->ana_state[grpid]; -memset (desc->rsvd17,0 ,sizeof (desc->rsvd17));return struct_size (desc, nsids, count); }static void nvmet_execute_get_log_page_ana (struct nvmet_req *req) { -struct nvme_ana_rsp_hdr hdr = {0 , }; +struct nvme_ana_rsp_hdr *hdr;struct nvme_ana_group_desc *desc; -size_t offset =sizeof (struct nvme_ana_rsp_hdr );/* start beyond hdr */ -size_t len; +u64 offset =nvmet_get_log_page_offset (req->cmd); +u32 len; +void *buffer;u32 grpid;u16 ngrps =0 ;u16 status; +if (offset &0x3 ) { + req->error_loc = + offsetof(struct nvme_get_log_page_command , lpo); + status =NVME_SC_INVALID_FIELD |NVME_SC_DNR ; + goto out; + } + status =NVME_SC_INTERNAL ; - desc =kmalloc (struct_size (desc, nsids,NVMET_MAX_NAMESPACES ), -GFP_KERNEL ); -if (!desc) + len =sizeof (*hdr) +struct_size (desc, nsids,NVMET_MAX_NAMESPACES ); + buffer =kzalloc (len,GFP_KERNEL ); +if (!buffer) goto out; + len =sizeof (*hdr); + hdr = buffer; + desc = buffer +sizeof (*hdr); +down_read (&nvmet_ana_sem);for (grpid =1 ; grpid <=NVMET_MAX_ANAGRPS ; grpid++) {if (!nvmet_ana_group_enabled[grpid])continue ; - len =nvmet_format_ana_group (req, grpid, desc); - status =nvmet_copy_to_sgl (req, offset, desc, len); -if (status) -break ; - offset += len; + len +=nvmet_format_ana_group (req, grpid, desc); ngrps++; }for ( ; grpid <=NVMET_MAX_ANAGRPS ; grpid++) {
@@ -299,15 +306,15 @@ static void nvmet_execute_get_log_page_ana(struct nvmet_req *req) ngrps++; } - hdr.chgcnt =cpu_to_le64 (nvmet_ana_chgcnt); - hdr.ngrps =cpu_to_le16 (ngrps); + hdr->chgcnt =cpu_to_le64 (nvmet_ana_chgcnt); + hdr->ngrps =cpu_to_le16 (ngrps);nvmet_clear_aen_bit (req,NVME_AEN_BIT_ANA_CHANGE );up_read (&nvmet_ana_sem); -kfree (desc); + status =nvmet_copy_to_sgl (req,0 , buffer + offset, +min (len,nvmet_get_log_page_len (req->cmd))); -/* copy the header last once we know the number of groups */ - status =nvmet_copy_to_sgl (req,0 , &hdr,sizeof (hdr)); +kfree (buffer); out:nvmet_req_complete (req, status); }
Signature · 1 line
2.34.1
Quoted 11 lines · Daniel Wagner
Add support to read with offsets from ANA log buffer.
The controller claims to support extended data for the Get Log Page command (including extended Number of Dwords and Log Page Offset 2 fields):
lpa :
0x7[2:2] : 0x1 Extended data for Get Log Page Supported [1:1] : 0x1 Command Effects Log Page Supported [0:0] : 0x1 SMART/Health Log Page per NS Supported
Quoted 7 lines
Signed-off-by Daniel Wagner <[email protected]>---
v2:
- fixed lenght calculation and limit response lenght
- some more testing with KASAN enabled
v1:
- https://lore.kernel.org/linux-nvme/[email protected]/
Please wait with this patch. I think there is something wrong with my calculation. I want to test some more just didn't have time for it yet.
Daniel,
Quoted 6 lines · Daniel Wagner
Add support to read with offsets from ANA log buffer.
The controller claims to support extended data for the Get Log Page command (including extended Number of Dwords and Log Page Offset 2 fields):
Quoted 7 lines
lpa :
0x7[2:2] : 0x1 Extended data for Get Log Page Supported [1:1] : 0x1 Command Effects Log Page Supported [0:0] : 0x1 SMART/Health Log Page per NS SupportedSigned-off-by Daniel Wagner <[email protected]>---
I'm not against having this code but do you really have a client side usecase for this one that you can please explain ?
Existing code should work fine to read the log pages and I've not received any complaints so far ..
Hi Chaitanya,
Quoted 6 lines · Chaitanya Kulkarni
I'm not against having this code but do you really have a client side usecase for this one that you can please explain ?
Existing code should work fine to read the log pages and I've not received any complaints so far ..
All started with this call trace:
nvme nvme17: new ctrl: NQN "nqn.2014-08.org.nvmexpress.discovery", addr 100.102.252.12:8009
nvme: page allocation failure: order:7, mode:0x40cc0(GFP_KERNEL|__GFP_COMP), nodemask=(null),cpuset=/,mems_allowed=0-1
CPU: 1 PID: 96790 Comm: nvme Kdump: loaded Tainted: G OE X 5.3.18-59.27-default #1 SLE15-SP3 Call Trace: dump_stack+0x66/0x8b warn_alloc+0x10b/0x190 ? __alloc_pages_direct_compact+0x15e/0x170 __alloc_pages_slowpath+0xcea/0xd20 __alloc_pages_nodemask+0x2cb/0x320 kmalloc_order+0x18/0x70 kmalloc_order_trace+0x1d/0xa0 __kmalloc+0x257/0x2b0 nvme_mpath_init_identify+0xe5/0x190 [nvme_core] ? blk_queue_write_cache+0x30/0x60 nvme_init_ctrl_finish+0x340/0xde0 [nvme_core] ? blk_mq_run_hw_queue+0x77/0x100 nvme_tcp_setup_ctrl+0x396/0x680 [nvme_tcp] nvme_tcp_create_ctrl+0x31e/0x3c7 [nvme_tcp] nvmf_dev_write+0xb04/0xd70 [nvme_fabrics] ? common_file_perm+0x64/0x190 ? vfs_write+0xad/0x1b0 vfs_write+0xad/0x1b0 ksys_write+0xa1/0xe0 do_syscall_64+0x5b/0x1e0 entry_SYSCALL_64_after_hwframe+0x44/0xa9 RIP: 0033:0x7fb44848bc30
The target says it support 49 ANA Group Descriptors and 65536 namespaces. The current code in Linux is allocating an ANA buffer large enough to read the complete ANA log in one operation.
With the above numbers the buffer is
16 + 16 * 49 + 65365 * 4 = 262260 bytes
large, ca 64.028 4k pages. This means when allocating the buffer, the smallest bucket to allocated from is the 128kB (2^7).
So my idea was to limit the ana log buffer and read the it in chunks if needed.
Anyway, I started with replicating the situation and found out that the target code does not support reads with offsets hence this patch.
Daniel