Revert "scsi: target/iscsi: Detect conn_cmd_list corruption early"
Subsystems (2)
PATCH 1/1 Revert "scsi: target/iscsi: Detect conn_cmd_list corruption early"
This reverts commit b0055acaedf56a2717a6e2a4b700f1959a1b60df.
This warning is duplicated because the very same condition is already checked in __iscsit_free_cmd().
--- drivers/target/iscsi/iscsi_target_util.c | 2 -- 1 file changed, 2 deletions(-)
drivers/target/iscsi/iscsi_target_util.c +0 -2
@@ -764,8 +764,6 @@ void iscsit_free_cmd(struct iscsi_cmd *cmd, bool shutdown)struct se_cmd *se_cmd = cmd->se_cmd.se_tfo ? &cmd->se_cmd :NULL ;int rc; -WARN_ON (!list_empty (&cmd->i_conn_node)); -__iscsit_free_cmd (cmd, shutdown);if (se_cmd) { rc =transport_generic_free_cmd (se_cmd, shutdown);
Signature · 1 line
2.26.2
Quoted 16 lines · Maurizio Lombardi
This reverts commit
b0055acaedf56a2717a6e2a4b700f1959a1b60df.This warning is duplicated because the very same condition is already checked in
__iscsit_free_cmd().Signed-off-by Maurizio Lombardi <[email protected]>--- drivers/target/iscsi/iscsi_target_util.c | 2 -- 1 file changed, 2 deletions(-)drivers/target/iscsi/iscsi_target_util.c +0
-0
drivers/target/iscsi/iscsi_target_util.c void iscsit_free_cmd(struct iscsi_cmd *cmd, bool shutdown) · line 769
Quoted 8 lines
drivers/target/iscsi/iscsi_target_util.c +0
-2@@ -764,8 +764,6 @@ void iscsit_free_cmd(struct iscsi_cmd *cmd, bool shutdown)struct se_cmd *se_cmd = cmd->se_cmd.se_tfo ? &cmd->se_cmd :NULL ;int rc; -WARN_ON (!list_empty (&cmd->i_conn_node)); -__iscsit_free_cmd (cmd, shutdown);if (se_cmd) { rc =transport_generic_free_cmd (se_cmd, shutdown);
Maurizio, are you hitting these WARN()s?
Patch looks ok.
Dne 02. 10. 20 v 20:06 Mike Christie napsal(a):
Quoted 19 lines · Maurizio Lombardi
Quoted 18 lines
This reverts commit
b0055acaedf56a2717a6e2a4b700f1959a1b60df.This warning is duplicated because the very same condition is already checked in
__iscsit_free_cmd().Signed-off-by Maurizio Lombardi <[email protected]>--- drivers/target/iscsi/iscsi_target_util.c | 2 -- 1 file changed, 2 deletions(-)drivers/target/iscsi/iscsi_target_util.c +0
-0@@ -764,8 +764,6 @@ void iscsit_free_cmd(struct iscsi_cmd *cmd, bool shutdown)struct se_cmd *se_cmd = cmd->se_cmd.se_tfo ? &cmd->se_cmd :NULL ;int rc;
Quoted 8 lines
drivers/target/iscsi/iscsi_target_util.c void iscsit_free_cmd(struct iscsi_cmd *cmd, bool shutdown) · line 769
Quoted 6 lines
drivers/target/iscsi/iscsi_target_util.c +0
-2-WARN_ON (!list_empty (&cmd->i_conn_node)); -__iscsit_free_cmd (cmd, shutdown);if (se_cmd) { rc =transport_generic_free_cmd (se_cmd, shutdown);Maurizio, are you hitting these WARN()s?
We received a number of bug reports against RHEL7 about warnings like the following:
Feb 18 12:41:01 hostname kernel: ------------[ cut here ]------------ Feb 18 12:41:01 hostname kernel: WARNING: CPU: 1 PID: 21173 at drivers/target/iscsi/iscsi_target_util.c:720 __iscsit_free_cmd+0x26e/0x290 [iscsi_target_mod] ... Feb 18 12:41:01 hostname kernel: CPU: 1 PID: 21173 Comm: kworker/u8:3 Kdump: loaded Not tainted 3.10.0-1062.4.1.el7.x86_64 #1 Feb 18 12:41:01 hostname kernel: Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/17/2015 Feb 18 12:41:01 hostname kernel: Workqueue: tmr-user target_tmr_work [target_core_mod] Feb 18 12:41:01 hostname kernel: Call Trace: Feb 18 12:41:01 hostname kernel: [<ffffffff91d78ba4>] dump_stack+0x19/0x1b Feb 18 12:41:01 hostname kernel: [<ffffffff9169a958>] __warn+0xd8/0x100 Feb 18 12:41:01 hostname kernel: [<ffffffff9169aa9d>] warn_slowpath_null+0x1d/0x20 Feb 18 12:41:01 hostname kernel: [<ffffffffc0a6f69e>] __iscsit_free_cmd+0x26e/0x290 [iscsi_target_mod] Feb 18 12:41:01 hostname kernel: [<ffffffffc0a70bc4>] iscsit_aborted_task+0x64/0x70 [iscsi_target_mod] Feb 18 12:41:01 hostname kernel: [<ffffffffc0a7830a>] lio_aborted_task+0x2a/0x30 [iscsi_target_mod] Feb 18 12:41:01 hostname kernel: [<ffffffffc09fa516>] transport_cmd_finish_abort+0x66/0xb0 [target_core_mod] Feb 18 12:41:01 hostname kernel: [<ffffffffc09f4d92>] core_tmr_abort_task+0x102/0x180 [target_core_mod] Feb 18 12:41:01 hostname kernel: [<ffffffffc09f7bb2>] target_tmr_work+0x152/0x170 [target_core_mod] Feb 18 12:41:01 hostname kernel: [<ffffffff916bd1df>] process_one_work+0x17f/0x440 Feb 18 12:41:01 hostname kernel: [<ffffffff916be2f6>] worker_thread+0x126/0x3c0 Feb 18 12:41:01 hostname kernel: [<ffffffff916be1d0>] ? manage_workers.isra.26+0x2a0/0x2a0 Feb 18 12:41:01 hostname kernel: [<ffffffff916c51b1>] kthread+0xd1/0xe0 Feb 18 12:41:01 hostname kernel: [<ffffffff916c50e0>] ? insert_kthread_work+0x40/0x40 Feb 18 12:41:01 hostname kernel: [<ffffffff91d8bd37>] ret_from_fork_nospec_begin+0x21/0x21 Feb 18 12:41:01 hostname kernel: [<ffffffff916c50e0>] ? insert_kthread_work+0x40/0x40 Feb 18 12:41:01 hostname kernel: ---[ end trace ed2119501826ec7a ]---
I was discussing the matter with Bart Van Assche in private and maybe I found the bug, this is the content of the last email I sent him:
The iscsit_release_commands_from_conn() function does the following operations:
1) locks the cmd_lock spinlock
2) Scans the list of commands and sets the CMD_T_FABRIC_STOP flag
3) Releases the cmd_lock spinlock
4) Rescans the list again and clears the i_conn_node link of each command
But what happens if an abort timer is fired between points 3 and 4?
void iscsit_aborted_task(struct iscsi_conn *conn, struct iscsi_cmd *cmd)
{
spin_lock_bh(&conn->cmd_lock);
if (!list_empty(&cmd->i_conn_node) &&
!(cmd->se_cmd.transport_state & CMD_T_FABRIC_STOP))
list_del_init(&cmd->i_conn_node);
spin_unlock_bh(&conn->cmd_lock);
__iscsit_free_cmd(cmd, true);
}iscsit_aborted_task() will find the cmd_lock spinlock unlocked.
will also find a non-empty i_conn_node link but with the CMD_T_FABRIC_STOP flag set. Therefore it will not call list_del_init(i_conn_node) and will trigger the warning in __iscsit_free_cmd().
Sounds it possible to you?
If I am right, this has been introduced by commit 064cdd2d91c2805d788876082f31cc63506f22c3
Maurizio
Quoted 15 lines · Maurizio Lombardi
This reverts commit
b0055acaedf56a2717a6e2a4b700f1959a1b60df.This warning is duplicated because the very same condition is already checked in
__iscsit_free_cmd().Signed-off-by Maurizio Lombardi <[email protected]>--- drivers/target/iscsi/iscsi_target_util.c | 2 -- 1 file changed, 2 deletions(-)drivers/target/iscsi/iscsi_target_util.c +0
-0
drivers/target/iscsi/iscsi_target_util.c void iscsit_free_cmd(struct iscsi_cmd *cmd, bool shutdown) · line 769
Quoted 8 lines
drivers/target/iscsi/iscsi_target_util.c +0
-2@@ -764,8 +764,6 @@ void iscsit_free_cmd(struct iscsi_cmd *cmd, bool shutdown)struct se_cmd *se_cmd = cmd->se_cmd.se_tfo ? &cmd->se_cmd :NULL ;int rc; -WARN_ON (!list_empty (&cmd->i_conn_node)); -__iscsit_free_cmd (cmd, shutdown);if (se_cmd) { rc =transport_generic_free_cmd (se_cmd, shutdown);
Hi Maurizio,
I agree that the same WARN_ON() occurs inside __iscsit_free_cmd(). What is not clear to me is how removing the WARN_ON() statement from iscsit_free_cmd() can help since an identical statement occurs inside __iscsit_free_cmd()?
Thanks,
Bart.
Dne 03. 10. 20 v 2:23 Bart Van Assche napsal(a):
Quoted 5 lines
Hi Maurizio,
I agree that the same
WARN_ON()occurs inside__iscsit_free_cmd(). What is not clear to me is how removing theWARN_ON()statement fromiscsit_free_cmd()can help since an identical statement occurs inside__iscsit_free_cmd()?
It doesn't help indeed, this patch just removes one duplicate warning but doesn't really change anything.
The bug I am trying to fix will need a different patch to prevent the race condition.
Maurizio
Quoted 9 lines · Maurizio Lombardi
Dne 03. 10. 20 v 2:23 Bart Van Assche napsal(a):
Quoted 3 lines
I agree that the same
WARN_ON()occurs inside__iscsit_free_cmd(). What is not clear to me is how removing theWARN_ON()statement fromiscsit_free_cmd()can help since an identical statement occurs inside__iscsit_free_cmd()?It doesn't help indeed, this patch just removes one duplicate warning but doesn't really change anything.
The bug I am trying to fix will need a different patch to prevent the race condition.
How about addressing both issues with a single patch?
Thanks,
Bart.
Dne 03. 10. 20 v 17:17 Bart Van Assche napsal(a):
Quoted 6 lines · Maurizio Lombardi
Quoted 5 lines
Dne 03. 10. 20 v 2:23 Bart Van Assche napsal(a):
Quoted 3 lines
I agree that the same
WARN_ON()occurs inside__iscsit_free_cmd(). What is not clear to me is how removing theWARN_ON()statement fromiscsit_free_cmd()can help since an identical statement occurs inside__iscsit_free_cmd()?
Quoted 6 lines
Quoted 4 lines
It doesn't help indeed, this patch just removes one duplicate warning but doesn't really change anything.
The bug I am trying to fix will need a different patch to prevent the race condition.
How about addressing both issues with a single patch?
I've nothing against it, I will just need a bit more time to come up with a patch.
Maurizio