Fix some kernel panics due to the host sending invalid PDUs
Subsystems (2)
cover Fix some kernel panics due to the host sending invalid PDUs
I used the reproducer that was published here:
https://lore.kernel.org/linux-nvme/[email protected]/T/#m57f25abc944dc706125fed02cb99d86b56c5d36e
I didn't get the same call trace, however.
Maurizio Lombardi (2):
nvmet-tcp: Fix a kernel panic when host sends an invalid H2C PDU
length
nvmet-tcp: fix a crash in nvmet_req_complete()
drivers/nvme/target/tcp.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-)
Signature · 1 line
2.39.3
PATCH 1/2 nvmet-tcp: Fix a kernel panic when host sends an invalid H2C PDU length
If the host sends an H2CData command with a DATAL field equal to 0, the kernel crashes in nvmet_tcp_build_pdu_iovec().
Unable to handle kernel NULL pointer dereference at virtual address 0000000000000000 lr : nvmet_tcp_io_work+0x6ac/0x718 [nvmet_tcp] Call trace: process_one_work+0x174/0x3c8 worker_thread+0x2d0/0x3e8 kthread+0x104/0x110
Getting an H2CData Packet with DATAL=0 is unexpected, treat it like a fatal transport error.
Also, the PDU length should never exceed the MAXH2CDATA parameter which has been communicated to the host in nvmet_tcp_handle_icreq().
--- drivers/nvme/target/tcp.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
drivers/nvme/target/tcp.c +9 -1
@@ -24,6 +24,7 @@#include "nvmet.h" #define NVMET_TCP_DEF_INLINE_DATA_SIZE (4 * PAGE_SIZE) +#define NVMET_TCP_MAXH2CDATA 0x400000/* 16M arbitrary limit */ static int param_store_val (const char *str,int *val,int min,int max) {
@@ -923,7 +924,7 @@ static int nvmet_tcp_handle_icreq(struct nvmet_tcp_queue *queue) icresp->hdr.pdo =0 ; icresp->hdr.plen =cpu_to_le32 (icresp->hdr.hlen); icresp->pfv =cpu_to_le16 (NVME_TCP_PFV_1_0 ); - icresp->maxdata =cpu_to_le32 (0x400000 );/* 16M arbitrary limit */ + icresp->maxdata =cpu_to_le32 (NVMET_TCP_MAXH2CDATA ); icresp->cpda =0 ;if (queue->hdr_digest) icresp->digest |=NVME_TCP_HDR_DIGEST_ENABLE ;
@@ -1002,6 +1003,13 @@ static int nvmet_tcp_handle_h2c_data_pdu(struct nvmet_tcp_queue *queue) } cmd->pdu_len =le32_to_cpu (data->data_length); +if (unlikely (cmd->pdu_len ==0 || + cmd->pdu_len >NVMET_TCP_MAXH2CDATA )) { +pr_err ("H2CData PDU len out of range\n" ); +/* FIXME: use proper transport errors */ +nvmet_tcp_fatal_error (queue); +return -EPROTO ; + } cmd->pdu_recv =0 ;nvmet_tcp_build_pdu_iovec (cmd); queue->cmd = cmd;
Signature · 1 line
2.39.3
Re: [PATCH 1/2] nvmet-tcp: Fix a kernel panic when host sends an invalid H2C PDU length
drivers/nvme/target/tcp.c static int nvmet_tcp_handle_h2c_data_pdu(struct nvmet_tcp_queue *queue) · line 1008
Quoted 7 lines · Maurizio Lombardi
drivers/nvme/target/tcp.c +3
-0@@ -1002,6 +1003,13 @@ static int nvmet_tcp_handle_h2c_data_pdu(struct nvmet_tcp_queue *queue) } cmd->pdu_len =le32_to_cpu (data->data_length); +if (unlikely (cmd->pdu_len ==0 || + cmd->pdu_len >NVMET_TCP_MAXH2CDATA )) { +pr_err ("H2CData PDU len out of range\n" );
Please add the 'pdu_len' to the pr_err since that info sounds useful to knowing which type of target error we're dealing with.
Otherwise looks good!
Quoted 7 lines
drivers/nvme/target/tcp.c +4
-0+/* FIXME: use proper transport errors */ +nvmet_tcp_fatal_error (queue); +return -EPROTO ; + } cmd->pdu_recv =0 ;nvmet_tcp_build_pdu_iovec (cmd); queue->cmd = cmd;
Re: [PATCH 1/2] nvmet-tcp: Fix a kernel panic when host sends an invalid H2C PDU length
čt 21. 12. 2023 v 18:42 odesílatel Keith Busch <[email protected]> napsal:
Quoted 9 lines · Maurizio Lombardi
Please add the '
pdu_len' to thepr_errsince that info sounds useful to knowing which type of target error we're dealing with.Otherwise looks good!
Reviewed-by Keith Busch <[email protected]>
Thanks,
I realized that there is still a possible problem, the data_length sent by the host
could be inside the allowed range, but still higher than the real size of the packet.
This could make the kernel crash.
Today I will send a V2 and add more fixes to this patchset.
Maurizio
PATCH 2/2 nvmet-tcp: fix a crash in nvmet_req_complete()
In reply to Maurizio Lombardiin nvmet_tcp_handle_h2c_data_pdu(), if the host sends a data_offset different from rbytes_done, the driver ends up calling nvmet_req_complete() passing a status error.
The problem is that at this point cmd->req is not yet initialized, the kernel will crash after dereferencing a NULL pointer.
Fix the bug by replacing the call to nvmet_req_complete() with nvmet_tcp_fatal_error().
--- drivers/nvme/target/tcp.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
drivers/nvme/target/tcp.c +1 -2
@@ -997,8 +997,7 @@ static int nvmet_tcp_handle_h2c_data_pdu(struct nvmet_tcp_queue *queue) data->ttag,le32_to_cpu (data->data_offset), cmd->rbytes_done);/* FIXME: use path and transport errors */ -nvmet_req_complete (&cmd->req, -NVME_SC_INVALID_FIELD |NVME_SC_DNR ); +nvmet_tcp_fatal_error (queue);return -EPROTO ; }
Signature · 1 line
2.39.3
Re: [PATCH 2/2] nvmet-tcp: fix a crash in nvmet_req_complete()
Quoted 8 lines · Maurizio Lombardi
in
nvmet_tcp_handle_h2c_data_pdu(), if the host sends adata_offsetdifferent fromrbytes_done, the driver ends up callingnvmet_req_complete()passing a status error.
The problem is that at this pointcmd->reqis not yet initialized, the kernel will crash after dereferencing a NULL pointer.Fix the bug by replacing the call to
nvmet_req_complete()withnvmet_tcp_fatal_error().
Looks good. The bug this fixes goes back to the beginning: