Skip to contentSkip to conversation

scsi: iscsi: publish endpoints after transport setup

PATCH In review Ruoyu Wang 3 participants 3 messages Jun 17

Subsystems (6)
lore
Share & tools
lore

Ruoyu Wang

author
#0 lore raw

PATCH 1/1 scsi: iscsi: publish endpoints after transport setup

iscsi_create_endpoint() inserts a new endpoint into iscsi_ep_idr before transport drivers have initialized their endpoint-private data. The endpoint handle is returned only after ep_connect() completes, but handles are allocated from a predictable IDR and iscsi_lookup_endpoint() looks them up directly.

Reserve the endpoint ID with a NULL IDR entry, add iscsi_register_endpoint() for the publish step, and call it from the in-tree transport drivers after private endpoint setup has completed. Until registration, endpoint lookup keeps returning NULL for the reserved handle.

---
 drivers/scsi/be2iscsi/be_iscsi.c    |  6 ++++++
 drivers/scsi/bnx2i/bnx2i_iscsi.c    |  4 ++++
 drivers/scsi/cxgbi/libcxgbi.c       |  6 ++++++
 drivers/scsi/qedi/qedi_iscsi.c      |  4 ++++
 drivers/scsi/qla4xxx/ql4_os.c       |  6 ++++++
 drivers/scsi/scsi_transport_iscsi.c | 24 +++++++++++++++++++++++-
 include/scsi/scsi_transport_iscsi.h |  1 +
 7 files changed, 50 insertions(+), 1 deletion(-)
drivers/scsi/be2iscsi/be_iscsi.c +6 -0
index 8d374ae863ba2..1abb2bc695923 100644
@@ -1188,6 +1188,12 @@ beiscsi_ep_connect(struct Scsi_Host *shost, struct sockaddr *dst_addr,
 		goto free_ep;
 	}
 
+	ret = iscsi_register_endpoint(ep);
+	if (ret) {
+		beiscsi_ep_disconnect(ep);
+		return ERR_PTR(ret);
+	}
+
 	return ep;
 
 free_ep:
drivers/scsi/bnx2i/bnx2i_iscsi.c +4 -0
index 6c80e5b514fda..2c84b4864e240 100644
@@ -1914,6 +1914,10 @@ static struct iscsi_endpoint *bnx2i_ep_connect(struct Scsi_Host *shost,
 	if (rc)
 		goto del_active_ep;
 
+	rc = iscsi_register_endpoint(ep);
+	if (rc)
+		goto del_active_ep;
+
 	mutex_unlock(&hba->net_dev_lock);
 	return ep;
 
drivers/scsi/cxgbi/libcxgbi.c +6 -0
index ea9631bfe2e23..c1b2a401ddca3 100644
@@ -2937,11 +2937,17 @@ check_route:
 	cep->csk = csk;
 	cep->chba = hba;
 
+	err = iscsi_register_endpoint(ep);
+	if (err)
+		goto release_ep;
+
 	log_debug(1 << CXGBI_DBG_ISCSI | 1 << CXGBI_DBG_SOCK,
 		"ep 0x%p, cep 0x%p, csk 0x%p, hba 0x%p,%s.\n",
 		ep, cep, csk, hba, hba->ndev->name);
 	return ep;
 
+release_ep:
+	iscsi_destroy_endpoint(ep);
 release_conn:
 	cxgbi_sock_put(csk);
 	cxgbi_sock_closed(csk);
drivers/scsi/qedi/qedi_iscsi.c +4 -0
index 6ab3a989d2817..60c37566f768c 100644
@@ -1014,6 +1014,10 @@ qedi_ep_connect(struct Scsi_Host *shost, struct sockaddr *dst_addr,
 		goto ep_rel_conn;
 	}
 
+	ret = iscsi_register_endpoint(ep);
+	if (ret)
+		goto ep_rel_conn;
+
 	atomic_inc(&qedi->num_offloads);
 	return ep;
 
drivers/scsi/qla4xxx/ql4_os.c +6 -0
index d598ab4126f80..8b30620d0e8dc 100644
@@ -1749,6 +1749,12 @@ qla4xxx_ep_connect(struct Scsi_Host *shost, struct sockaddr *dst_addr,
 
 	qla_ep->host = shost;
 
+	ret = iscsi_register_endpoint(ep);
+	if (ret) {
+		iscsi_destroy_endpoint(ep);
+		return ERR_PTR(ret);
+	}
+
 	return ep;
 }
 
drivers/scsi/scsi_transport_iscsi.c +23 -1
index 8aa76f813bcdb..a8c2a32c0f134 100644
@@ -217,7 +217,7 @@ iscsi_create_endpoint(int dd_size)
 	 * First endpoint id should be 1 to comply with user space
 	 * applications (iscsid).
 	 */
-	id = idr_alloc(&iscsi_ep_idr, ep, 1, -1, GFP_NOIO);
+	id = idr_alloc(&iscsi_ep_idr, NULL, 1, -1, GFP_NOIO);
 	if (id < 0) {
 		mutex_unlock(&iscsi_ep_idr_mutex);
 		printk(KERN_ERR "Could not allocate endpoint ID. Error %d.\n",
@@ -257,6 +257,28 @@ free_ep:
 }
 EXPORT_SYMBOL_GPL(iscsi_create_endpoint);
 
+int iscsi_register_endpoint(struct iscsi_endpoint *ep)
+{
+	void *old;
+	int err = 0;
+
+	mutex_lock(&iscsi_ep_idr_mutex);
+	old = idr_find(&iscsi_ep_idr, ep->id);
+	if (old) {
+		err = -EBUSY;
+		goto unlock;
+	}
+
+	old = idr_replace(&iscsi_ep_idr, ep, ep->id);
+	if (IS_ERR(old))
+		err = PTR_ERR(old);
+unlock:
+	mutex_unlock(&iscsi_ep_idr_mutex);
+
+	return err;
+}
+EXPORT_SYMBOL_GPL(iscsi_register_endpoint);
+
 void iscsi_destroy_endpoint(struct iscsi_endpoint *ep)
 {
 	sysfs_remove_group(&ep->dev.kobj, &iscsi_endpoint_group);
include/scsi/scsi_transport_iscsi.h +1 -0
index 76de2b662f4fb..96299d25e0f73 100644
@@ -459,6 +459,7 @@ extern void iscsi_get_conn(struct iscsi_cls_conn *conn);
 extern void iscsi_unblock_session(struct iscsi_cls_session *session);
 extern void iscsi_block_session(struct iscsi_cls_session *session);
 extern struct iscsi_endpoint *iscsi_create_endpoint(int dd_size);
+int iscsi_register_endpoint(struct iscsi_endpoint *ep);
 extern void iscsi_destroy_endpoint(struct iscsi_endpoint *ep);
 extern struct iscsi_endpoint *iscsi_lookup_endpoint(u64 handle);
 extern void iscsi_put_endpoint(struct iscsi_endpoint *ep);
Signature · 1 line
2.51.0
maintainer
#1 lore raw
Quoted 2 lines · Ruoyu Wang

iscsi_create_endpoint() inserts a new endpoint into iscsi_ep_idr before transport drivers have initialized their endpoint-private data. The

What is the bug you are fixing with this patch?

We should normally be doing iscsi_lookup_endpoint calls from interface calls done under the rx_mutex. The ep creation is also done under the mutex so we should never see a partially setup endpoint.

Is there an async error path where we do a lookup from?

Quoted 10 lines

endpoint handle is returned only after ep_connect() completes, but handles are allocated from a predictable IDR and iscsi_lookup_endpoint() looks them up directly.

Reserve the endpoint ID with a NULL IDR entry, add iscsi_register_endpoint() for the publish step, and call it from the in-tree transport drivers after private endpoint setup has completed. Until registration, endpoint lookup keeps returning NULL for the reserved handle.

#2 lore raw
In reply to Ruoyu Wang
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

New issues:
- [High] The iscsi_iser transport driver was not updated to call iscsi_register_endpoint(), completely breaking connection binding.

Pre-existing issues:
- [High] If device_register() fails inside iscsi_create_endpoint(), a double idr_remove() occurs with a dropped lock in between, incorrectly deleting a concurrent allocation. --

commit d4c9226c25ea1f5ce70c62be37b16435ed05458c
Author: Ruoyu Wang <[email protected]>

scsi: iscsi: publish endpoints after transport setup

iscsi_create_endpoint() inserts a new endpoint into iscsi_ep_idr before transport drivers have initialized their endpoint-private data. The endpoint handle is returned only after ep_connect() completes, but handles are allocated from a predictable IDR and iscsi_lookup_endpoint() looks them up directly.

Reserve the endpoint ID with a NULL IDR entry, add iscsi_register_endpoint() for the publish step, and call it from the in-tree transport drivers after private endpoint setup has completed. Until registration, endpoint lookup keeps returning NULL for the reserved handle.

drivers/scsi/scsi_transport_iscsi.c iscsi_create_endpoint(int dd_size) · line 221

Quoted 11 lines
drivers/scsi/scsi_transport_iscsi.c +1 -1
index 8aa76f813bcdb..a8c2a32c0f134 100644
@@ -217,7 +217,7 @@ iscsi_create_endpoint(int dd_size)
 	 * First endpoint id should be 1 to comply with user space
 	 * applications (iscsid).
 	 */
-	id = idr_alloc(&iscsi_ep_idr, ep, 1, -1, GFP_NOIO);
+	id = idr_alloc(&iscsi_ep_idr, NULL, 1, -1, GFP_NOIO);
 	if (id < 0) {

[Severity: High]
This is a pre-existing issue, but does the put_dev error path in iscsi_create_endpoint() result in a double idr_remove() for the same ID?

If device_register() fails, the code jumps to put_dev:

	put_dev:
		mutex_lock(&iscsi_ep_idr_mutex);
		idr_remove(&iscsi_ep_idr, id);
		mutex_unlock(&iscsi_ep_idr_mutex);
		put_device(&ep->dev);
		return NULL;

Calling put_device() drops the kobject refcount to zero, which triggers the release callback:

drivers/scsi/scsi_transport_iscsi.c:iscsi_endpoint_release() {
	...
	mutex_lock(&iscsi_ep_idr_mutex);
	idr_remove(&iscsi_ep_idr, ep->id);
	mutex_unlock(&iscsi_ep_idr_mutex);
	...
}

Since the lock is dropped before put_device(), could a concurrent thread successfully allocate the same ID via iscsi_create_endpoint(), only for the second idr_remove() to incorrectly delete the newly allocated ID belonging to the concurrent thread?

[ ... ]

drivers/scsi/scsi_transport_iscsi.c iscsi_create_endpoint(int dd_size) · line 263

Quoted 8 lines
drivers/scsi/scsi_transport_iscsi.c +4 -0
@@ -257,6 +257,28 @@ iscsi_create_endpoint(int dd_size)
 }
 EXPORT_SYMBOL_GPL(iscsi_create_endpoint);
 
+int iscsi_register_endpoint(struct iscsi_endpoint *ep)
+{
+	void *old;
+	int err = 0;

[Severity: High]
Does the iscsi_iser transport driver need to be updated to call iscsi_register_endpoint()?

The iscsi_iser driver in drivers/infiniband/ulp/iser/iscsi_iser.c allocates the endpoint:

drivers/infiniband/ulp/iser/iscsi_iser.c:iscsi_iser_ep_connect() {
	...
	ep = iscsi_create_endpoint(0);
	if (!ep)
		return ERR_PTR(-ENOMEM);
	...
	return ep;
}

But it doesn't seem to publish it to the IDR. When userspace subsequently calls iscsi_iser_conn_bind() to bind the connection, the lookup uses idr_find(), which correctly returns NULL:

drivers/infiniband/ulp/iser/iscsi_iser.c:iscsi_iser_conn_bind() {
	...
	ep = iscsi_lookup_endpoint(transport_eph);
	if (!ep) {
		iser_err("can't bind eph %llx\n",
			 (unsigned long long)transport_eph);
		return -EINVAL;
	}
	...
}

Would this cause the binding lookup to always fail with -EINVAL, preventing the iSER transport from working?

Quoted 17 lines
drivers/scsi/scsi_transport_iscsi.c +17 -0
+
+	mutex_lock(&iscsi_ep_idr_mutex);
+	old = idr_find(&iscsi_ep_idr, ep->id);
+	if (old) {
+		err = -EBUSY;
+		goto unlock;
+	}
+
+	old = idr_replace(&iscsi_ep_idr, ep, ep->id);
+	if (IS_ERR(old))
+		err = PTR_ERR(old);
+unlock:
+	mutex_unlock(&iscsi_ep_idr_mutex);
+
+	return err;
+}
+EXPORT_SYMBOL_GPL(iscsi_register_endpoint);
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1