scsi: mpi3mr: Fix start_time and error return in mpi3mr_bring_ioc_ready()
Subsystems (2)
PATCH 1/1 scsi: mpi3mr: Fix start_time and error return in mpi3mr_bring_ioc_ready()
mpi3mr_bring_ioc_ready() has three bugs in its failure and retry handling:
1. start_time is only assigned after MPI3_SYSIF_IOC_CONFIG_ENABLE_IOC when retry == 0, after three earlier 'goto out_failed' paths. If soft reset, reset-state polling, or admin queue setup fails on the first attempt, out_failed computes elapsed_time_sec from an uninitialized start_time, and if a retry occurs (retry == 1), start_time remains uninitialized for the ready-polling loop as well.
2. In 'elapsed_time_sec < (mrioc->ready_timeout - 60)', ready_timeout is u32, so values below 60 wrap around in 32-bit unsigned arithmetic before zero-extending to u64, spuriously retrying past the timeout.
3. After mpi3mr_setup_admin_qpair() sets retval = 0 (or on entry where retval is initialized to 0), controller fault/reset-history checks, ready-state timeouts, and early unrecoverable or device-not-present exits return retval == 0 (success) to mpi3mr_init_ioc() and mpi3mr_reinit_ioc() even though controller bringup failed.
Initialize start_time = jiffies at function entry for pre-enable failure exits, reset start_time = jiffies the first time MPI3_SYSIF_IOC_CONFIG_ENABLE_IOC is written so pre-enable waits do not drain ready_timeout, rewrite the retry window check as 'elapsed_time_sec + 60 < mrioc->ready_timeout', and ensure retval = -1 on all failure exits.
Found via Clang -Wconditional-uninitialized on an x86_64 allyesconfig build. Verified by building drivers/scsi/mpi3mr/mpi3mr_fw.o with LLVM=1 W=1 (-Werror) and -Wconditional-uninitialized, and with a standalone C logic reproducer under ASan/UBSan; not runtime-tested on physical MPI3 hardware.
--- drivers/scsi/mpi3mr/mpi3mr_fw.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-)
drivers/scsi/mpi3mr/mpi3mr_fw.c +11 -3
@@ -1582,8 +1582,9 @@ static int mpi3mr_bring_ioc_ready(struct mpi3mr_ioc *mrioc)int retval =0 ;enum mpi3mr_iocstate ioc_state;u64 base_info; +bool enable_started = false;u8 retry =0 ; -u64 start_time, elapsed_time_sec; +u64 start_time = jiffies, elapsed_time_sec; retry_bring_ioc_ready:
@@ -1595,6 +1596,7 @@ static int mpi3mr_bring_ioc_ready(struct mpi3mr_ioc *mrioc)if (!mpi3mr_is_fault_recoverable (mrioc)) { mrioc->unrecoverable =1 ; + retval =-1 ; goto out_device_not_present; }
@@ -1621,6 +1623,7 @@ static int mpi3mr_bring_ioc_ready(struct mpi3mr_ioc *mrioc)if (!pci_device_is_present (mrioc->pdev)) { mrioc->unrecoverable =1 ;ioc_err (mrioc,"controller is not present while waiting to reset\n" ); + retval =-1 ; goto out_device_not_present; }
@@ -1654,6 +1657,7 @@ static int mpi3mr_bring_ioc_ready(struct mpi3mr_ioc *mrioc)if (!pci_device_is_present (mrioc->pdev)) { mrioc->unrecoverable =1 ;ioc_err (mrioc,"controller is not present at the bringup\n" ); + retval =-1 ; goto out_device_not_present; }msleep (100 );
@@ -1690,8 +1694,10 @@ static int mpi3mr_bring_ioc_ready(struct mpi3mr_ioc *mrioc) ioc_config |=MPI3_SYSIF_IOC_CONFIG_ENABLE_IOC ;writel (ioc_config, &mrioc->sysif_regs->ioc_configuration); -if (retry ==0 ) +if (!enable_started) { start_time = jiffies; + enable_started = true; + } timeout = mrioc->ready_timeout *10 ; do {
@@ -1731,7 +1737,7 @@ static int mpi3mr_bring_ioc_ready(struct mpi3mr_ioc *mrioc) out_failed: elapsed_time_sec =jiffies_to_msecs (jiffies - start_time)/1000 ; -if ((retry <2 ) && (elapsed_time_sec < (mrioc->ready_timeout -60 ))) { +if (retry <2 && elapsed_time_sec +60 < mrioc->ready_timeout) { retry++; ioc_warn(mrioc, "retrying to bring IOC ready, retry_count:%d\n"
@@ -1739,6 +1745,8 @@ static int mpi3mr_bring_ioc_ready(struct mpi3mr_ioc *mrioc) goto retry_bring_ioc_ready; } +if (!retval) + retval =-1 ; ioc_state =mpi3mr_get_iocstate (mrioc); ioc_err(mrioc, "failed to bring to ready state, current state: %s\n",
Signature · 1 line
2.56.1.708.g77570a6a1d-goog
Quoted 138 lines · Ian Rogers
mpi3mr_bring_ioc_ready()has three bugs in its failure and retry handling:1.
start_timeis only assigned afterMPI3_SYSIF_IOC_CONFIG_ENABLE_IOCwhen retry == 0, after three earlier 'gotoout_failed' paths. If soft reset, reset-state polling, or admin queue setup fails on the first attempt,out_failedcomputeselapsed_time_secfrom an uninitializedstart_time, and if a retry occurs (retry == 1),start_timeremains uninitialized for the ready-polling loop as well.2. In '
elapsed_time_sec< (mrioc->ready_timeout- 60)',ready_timeoutis u32, so values below 60 wrap around in 32-bit unsigned arithmetic
before zero-extending to u64, spuriously retrying past the timeout.3. After
mpi3mr_setup_admin_qpair()sets retval = 0 (or on entry where
retval is initialized to 0), controller fault/reset-history checks,
ready-state timeouts, and early unrecoverable or device-not- present
exits return retval == 0 (success) tompi3mr_init_ioc()andmpi3mr_reinit_ioc()even though controller bringup failed.Initialize
start_time= jiffies at function entry for pre-enable failure
exits, resetstart_time= jiffies the first timeMPI3_SYSIF_IOC_CONFIG_ENABLE_IOCis written so pre-enable waits do not
drainready_timeout, rewrite the retry window check as 'elapsed_time_sec+ 60 <mrioc->ready_timeout', and ensure retval = - 1
on all failure exits.Found via Clang -Wconditional-uninitialized on an
x86_64allyesconfig build. Verified by buildingdrivers/scsi/mpi3mr/mpi3mr_fw.owith LLVM=1 W=1 (-Werror) and -Wconditional-uninitialized, and with a standalone C logic reproducer under ASan/UBSan; not runtime-tested on physical MPI3 hardware.Fixes 59bd9cfe3fa0 ("scsi: mpi3mr: Code refactor of IOC init -part2")
Fixes 0a319f162949 ("scsi: mpi3mr: Wait for diagnostic save duringcontroller init")
Fixes 9634bb07083c ("scsi: mpi3mr: Enhance the Enable Controllerretry logic")
Fixes 4616a4b3cb8a ("scsi: mpi3mr: Improve wait logic whilecontroller transitions to READY state")
Fixes fb6eb98f3965 ("scsi: mpi3mr: Handling of fault code forinsufficient power")
Signed-off-by Ian Rogers <[email protected]>---
drivers/scsi/mpi3mr/mpi3mr_fw.c | 14 +++++++++++---1 file changed, 11 insertions(+), 3 deletions(-)drivers/scsi/mpi3mr/mpi3mr_fw.c +2
-1@@ -1582,8 +1582,9 @@ static int mpi3mr_bring_ioc_ready(struct mpi3mr_ioc *mrioc) int retval = 0; enum mpi3mr_iocstate ioc_state; u64 base_info; +bool enable_started = false; u8 retry = 0; -u64 start_time, elapsed_time_sec; +u64 start_time = jiffies, elapsed_time_sec;
retry_bring_ioc_ready:drivers/scsi/mpi3mr/mpi3mr_fw.c +0
-0@@ -1595,6 +1596,7 @@ static int mpi3mr_bring_ioc_ready(struct mpi3mr_ioc *mrioc)
if (!mpi3mr_is_fault_recoverable(mrioc)) {mrioc->unrecoverable= 1;drivers/scsi/mpi3mr/mpi3mr_fw.c +1
-0+ retval =-1 ;goto
out_device_not_present;
}drivers/scsi/mpi3mr/mpi3mr_fw.c +0
-0@@ -1621,6 +1623,7 @@ static int mpi3mr_bring_ioc_ready(struct mpi3mr_ioc *mrioc) if (!pci_device_is_present(mrioc->pdev)) { mrioc->unrecoverable = 1;
ioc_err(mrioc, "controller is not present while waiting to reset\n");drivers/scsi/mpi3mr/mpi3mr_fw.c +1
-0+ retval =-1 ;goto
out_device_not_present; }drivers/scsi/mpi3mr/mpi3mr_fw.c +0
-0@@ -1654,6 +1657,7 @@ static int mpi3mr_bring_ioc_ready(struct mpi3mr_ioc *mrioc)if (!pci_device_is_present(mrioc-
Quoted 1 line
pdev)) {
mrioc->unrecoverable= 1;ioc_err(mrioc, "controller is not present at the bringup\n");drivers/scsi/mpi3mr/mpi3mr_fw.c +1
-0+ retval =-1 ;goto
out_device_not_present; }
msleep(100);drivers/scsi/mpi3mr/mpi3mr_fw.c +0
-0@@ -1690,8 +1694,10 @@ static int mpi3mr_bring_ioc_ready(struct mpi3mr_ioc *mrioc) ioc_config |= MPI3_SYSIF_IOC_CONFIG_ENABLE_IOC; writel(ioc_config, &mrioc->sysif_regs->ioc_configuration);drivers/scsi/mpi3mr/mpi3mr_fw.c +1
-1-if (retry ==0 ) +if (!enable_started) {
start_time= jiffies;drivers/scsi/mpi3mr/mpi3mr_fw.c +2
-0+ enable_started = true; + }
timeout =mrioc->ready_timeout* 10;
do {drivers/scsi/mpi3mr/mpi3mr_fw.c +0
-0@@ -1731,7 +1737,7 @@ static int mpi3mr_bring_ioc_ready(struct mpi3mr_ioc *mrioc)
out_failed:
elapsed_time_sec=jiffies_to_msecs(jiffies - start_time)/1000;drivers/scsi/mpi3mr/mpi3mr_fw.c +1
-2-if ((retry <2 ) && (elapsed_time_sec < (mrioc->ready_timeout - 60))) { + if (retry < 2 && elapsed_time_sec + 60 < mrioc-Quoted 1 line
ready_timeout) {retry++;
ioc_warn(mrioc, "retrying to bring IOC ready, retry_count:%d\n"drivers/scsi/mpi3mr/mpi3mr_fw.c +0
-0@@ -1739,6 +1745,8 @@ static int mpi3mr_bring_ioc_ready(struct mpi3mr_ioc *mrioc)
gotoretry_bring_ioc_ready;
}drivers/scsi/mpi3mr/mpi3mr_fw.c +2
-0+if (!retval) + retval =-1 ;
Quoted 3 lines
ioc_state=mpi3mr_get_iocstate(mrioc); ioc_err(mrioc,
"failed to bring to ready state, current state: %s\n",
Hello Ian
Thanks for this. I checked it against current mainline and all three issues are real. Some comments below.
Quoted 1 line
3. After
mpi3mr_setup_admin_qpair()sets retval = 0 (or on entry
where
Quoted 1 line
retval is initialized to 0), controller fault/reset-history
checks,
Quoted 1 line
ready-state timeouts, and early unrecoverable or device-not-
present
Quoted 2 lines
exits return retval == 0 (success) to
mpi3mr_init_ioc()andmpi3mr_reinit_ioc()even though controller bringup failed.
This is the most serious of the three. Both callers only check "if (retval)", so with the !mpi3mr_is_fault_recoverable() exit we can set mrioc->unrecoverable = 1 and still report success, and init then carries on against a dead controller.
Quoted 2 lines
2. In '
elapsed_time_sec< (mrioc->ready_timeout- 60)',ready_timeoutis u32, so values below 60 wrap around in 32-bit unsigned
arithmetic
Quoted 1 line
before zero-extending to u64, spuriously retrying past the
timeout.
Agreed that the arithmetic is wrong, but note the fix changes behavior:
today any firmware advertising a ready timeout below 60s always gets its two retries; after this patch it gets none.
Sathya/Sreekanth, can you confirm whether any shipping MPI3 firmware reports a ready timeout under 60 seconds, and if so whether dropping the retries entirely is the intended behavior?
Quoted 1 line
Initialize
start_time= jiffies at function entry for pre-enable
failure
Quoted 2 lines
exits, reset
start_time= jiffies the first timeMPI3_SYSIF_IOC_CONFIG_ENABLE_IOCis written so pre-enable waits do
not
Quoted 2 lines
drain
ready_timeout, rewrite the retry window check as 'elapsed_time_sec+ 60 <mrioc->ready_timeout', and ensure retval = -
1
Quoted 1 line
on all failure exits.
Would you consider splitting this into three patches, one per issue, each with only its own Fixes: tag? The retval fix stands on its own and is the one stable and distro kernels will want first; the ready_timeout change may need to wait for Broadcom's answer above. Five Fixes: tags across different commits on one patch also makes backporting harder.
Quoted 5 lines
Found via Clang -Wconditional-uninitialized on an
x86_64allyesconfig build. Verified by buildingdrivers/scsi/mpi3mr/mpi3mr_fw.owith LLVM=1 W=1 (-Werror) and -Wconditional-uninitialized, and with a standalone C logic reproducer under ASan/UBSan; not runtime-tested on physical MPI3 hardware.
I have access to mpi3mr hardware and can run normal bring-up, controller reset and host-reset reinit on a v2 to make sure the good path does not regress, and will send a Tested-by.
With the split, for the retval and start_time fixes:
Regards
Laurence