Skip to contentSkip to conversation

target: fix a race condition between login_work and the login thread

PATCH Superseded Maurizio Lombardi 1 participant 3 messages Nov 11, 2022 – Nov 15, 2022 SCSI target

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 target: fix a race condition between login_work and the login thread

In case a malicious initiator sends some random data immediately after a login PDU; the iscsi_target_sk_data_ready() callback will schedule the login_work and, at the same time,
the negotiation may end without clearing the LOGIN_FLAGS_INITIAL_PDU flag (because no additional PDU exchanges are required to complete the login).

The login has been completed but the login_work function will find the LOGIN_FLAGS_INITIAL_PDU flag set and will never stop from rescheduling itself;
at this point, if the initiator drops the connection, the iscsit_conn structure will be freed, login_work will dereference a released socket structure and the kernel crashes.

BUG: kernel NULL pointer dereference, address: 0000000000000230
PF: supervisor write access in kernel mode
PF: error_code(0x0002) - not-present page
Workqueue: events iscsi_target_do_login_rx [iscsi_target_mod]
RIP: 0010:_raw_read_lock_bh+0x15/0x30
Call trace:
 iscsi_target_do_login_rx+0x75/0x3f0 [iscsi_target_mod]
 process_one_work+0x1e8/0x3c0

Fix this bug by forcing login_work to stop after the login has been completed and the socket callbacks have been restored. Also fix other potential race conditions in the error paths.

V2: remove an unnecessary call to cancel_delayed_work();
fix a potential race condition in iscsi_start_negotiation() and in iscsi_target_do_login_rx()'s error paths

---
 drivers/target/iscsi/iscsi_target_nego.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/target/iscsi/iscsi_target_nego.c +3 -3
index f2919319ad38..465c53b9ddb3 100644
@@ -645,7 +645,6 @@ static void iscsi_target_do_login_rx(struct work_struct *work)
 						    LOGIN_FLAGS_WRITE_ACTIVE))
 			goto err;
 	} else if (rc == 1) {
-		cancel_delayed_work(&conn->login_work);
 		iscsi_target_nego_release(conn);
 		iscsi_post_login_handler(np, conn, zero_tsih);
 		iscsit_deaccess_np(np, tpg, tpg_np);
@@ -654,7 +653,7 @@ static void iscsi_target_do_login_rx(struct work_struct *work)
 
 err:
 	iscsi_target_restore_sock_callbacks(conn);
-	cancel_delayed_work(&conn->login_work);
+	cancel_delayed_work_sync(&conn->login_work);
 	iscsi_target_login_drop(conn, login);
 	iscsit_deaccess_np(np, tpg, tpg_np);
 }
@@ -1058,6 +1057,7 @@ static int iscsi_target_do_login(struct iscsit_conn *conn, struct iscsi_login *l
 				login->tsih = conn->sess->tsih;
 				login->login_complete = 1;
 				iscsi_target_restore_sock_callbacks(conn);
+				cancel_delayed_work_sync(&conn->login_work);
 				if (iscsi_target_do_tx_login_io(conn,
 						login) < 0)
 					return -1;
@@ -1363,8 +1363,8 @@ int iscsi_target_start_negotiation(
 		ret = -1;
 
 	if (ret < 0) {
-		cancel_delayed_work_sync(&conn->login_work);
 		iscsi_target_restore_sock_callbacks(conn);
+		cancel_delayed_work_sync(&conn->login_work);
 		iscsi_remove_failed_auth_entry(conn);
 	}
 	if (ret != 0)
Signature · 1 line
2.31.1
#1 lore raw

pá 11. 11. 2022 v 16:47 odesílatel Maurizio Lombardi <[email protected]> napsal:

Quoted 5 lines

Fix this bug by forcing login_work to stop after the login has been completed and the socket callbacks have been restored. Also fix other potential race conditions in the error paths.

Self-NACK

I can't call cancel_delayed_work_sync() in iscsi_target_do_login() because the latter could be called from the login_work context and this will cause a deadlock.

Will submit a V3 when ready

Quoted 52 lines

V2: remove an unnecessary call to cancel_delayed_work();
fix a potential race condition in iscsi_start_negotiation() and in iscsi_target_do_login_rx()'s error paths

---
 drivers/target/iscsi/iscsi_target_nego.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/target/iscsi/iscsi_target_nego.c +3 -4
index f2919319ad38..465c53b9ddb3 100644
@@ -645,7 +645,6 @@ static void iscsi_target_do_login_rx(struct work_struct *work)
                                                    LOGIN_FLAGS_WRITE_ACTIVE))
                        goto err;
        } else if (rc == 1) {
-               cancel_delayed_work(&conn->login_work);
                iscsi_target_nego_release(conn);
                iscsi_post_login_handler(np, conn, zero_tsih);
                iscsit_deaccess_np(np, tpg, tpg_np);
@@ -654,7 +653,7 @@ static void iscsi_target_do_login_rx(struct work_struct *work)
 
 err:
        iscsi_target_restore_sock_callbacks(conn);
-       cancel_delayed_work(&conn->login_work);
+       cancel_delayed_work_sync(&conn->login_work);
        iscsi_target_login_drop(conn, login);
        iscsit_deaccess_np(np, tpg, tpg_np);
 }
@@ -1058,6 +1057,7 @@ static int iscsi_target_do_login(struct iscsit_conn *conn, struct iscsi_login *l
                                login->tsih = conn->sess->tsih;
                                login->login_complete = 1;
                                iscsi_target_restore_sock_callbacks(conn);
+                               cancel_delayed_work_sync(&conn->login_work);
                                if (iscsi_target_do_tx_login_io(conn,
                                                login) < 0)
                                        return -1;
@@ -1363,8 +1363,8 @@ int iscsi_target_start_negotiation(
                ret = -1;
 
        if (ret < 0) {
-               cancel_delayed_work_sync(&conn->login_work);
                iscsi_target_restore_sock_callbacks(conn);
+               cancel_delayed_work_sync(&conn->login_work);
                iscsi_remove_failed_auth_entry(conn);
        }
        if (ret != 0)
--

2.31.1

#2 lore raw

út 15. 11. 2022 v 9:33 odesílatel Maurizio Lombardi <[email protected]> napsal:

Quoted 9 lines

pá 11. 11. 2022 v 16:47 odesílatel Maurizio Lombardi <[email protected]> napsal:

Quoted 5 lines

Fix this bug by forcing login_work to stop after the login has been completed and the socket callbacks have been restored. Also fix other potential race conditions in the error paths.

Quoted 7 lines

Self-NACK

I can't call cancel_delayed_work_sync() in iscsi_target_do_login() because the latter could be called from the login_work context and this will cause a deadlock.

Will submit a V3 when ready

The correct place where to call cancel_delayed_work_sync() is in iscsi_target_start_negotiation(), because the latter is only called in the login_thread context.

I'm going to test the following:

(quoted diff) +0 -0

b/drivers/target/iscsi/iscsi_target_nego.c
index f2919319ad38..b9bd77f41c8d 100644

drivers/target/iscsi/iscsi_target_nego.c +4 -2
@@ -1363,12 +1363,14 @@ int iscsi_target_start_negotiation(
                ret = -1;
 
        if (ret < 0) {
-               cancel_delayed_work_sync(&conn->login_work);
                iscsi_target_restore_sock_callbacks(conn);
+               cancel_delayed_work_sync(&conn->login_work);
                iscsi_remove_failed_auth_entry(conn);
        }
-       if (ret != 0)
+       if (ret != 0) {
+               cancel_delayed_work_sync(&conn->login_work);
                iscsi_target_nego_release(conn);
+       }
 
        return ret;
 }

Maurizio