ipmi: si: Fix NULL pointer dereference in start_kcs_transaction()
PATCH 1/1 ipmi: si: Fix NULL pointer dereference in start_kcs_transaction()
try_smi_init() allocates new_smi->si_sm and later calls ipmi_register_smi(), which maps to ipmi_add_smi().
During ipmi_add_smi(), the upper IPMI message handler obtains the initial BMC device information through __bmc_get_device_id(). This can fail if the BMC does not return a successful response to the Get Device ID command.
When the BMC returns a nonzero completion code, the device-id helper retries the command and eventually returns -EIO if the device ID still cannot be fetched.
On this failure path, ipmi_add_smi() logs "Unable to get the device id" and goes to out_err_started, where it invokes the lower driver's shutdown callback. try_smi_init() then logs the returned registration failure:
ipmi_si IPI0001:00: IPMI message handler: Unable to get the device id: -5 ipmi_si IPI0001:00: Unable to register device: error -5
For ipmi_si, the shutdown callback is shutdown_smi(), which cleans up the SI state machine data, frees smi_info->si_sm, and sets smi_info->si_sm and smi_info->intf to NULL.
kfree(smi_info->si_sm);
smi_info->si_sm = NULL;
smi_info->intf = NULL;However, the smi_info can still be reached later. In the observed case, the redo_bmc_reg work item retried BMC device-id probing after shutdown_smi() had already cleared smi_info->si_sm. The retry path reached start_next_msg(), which passed the NULL smi_info->si_sm pointer to the selected KCS state machine handler:
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000 Workqueue: events redo_bmc_reg [ipmi_msghandler] RIP: start_kcs_transaction+0x2c/0x190 [ipmi_si] Call Trace: start_next_msg+0x50/0x80 [ipmi_si] check_start_timer_thread.part.9+0x3b/0x50 [ipmi_si] sender+0x69/0x80 [ipmi_si] i_ipmi_request+0x2ac/0x9d0 [ipmi_msghandler] __get_device_id.isra.29+0xaa/0x180 [ipmi_msghandler] __bmc_get_device_id+0xef/0x950 [ipmi_msghandler] redo_bmc_reg+0x52/0x60 [ipmi_msghandler] process_one_work+0x1a7/0x360
start_next_msg() and smi_event_handler() both invoke the selected SI state machine handlers with smi_info->si_sm without checking whether the state machine data has already been freed and cleared.
Add NULL checks before calling into the selected SI state machine handlers. If the state machine data has already been removed, treat the interface as idle and avoid dereferencing the cleared state.
--- drivers/char/ipmi/ipmi_si_intf.c | 6 ++++++ 1 file changed, 6 insertions(+)
drivers/char/ipmi/ipmi_si_intf.c +6 -0
@@ -320,6 +320,9 @@ static enum si_sm_result start_next_msg(struct smi_info *smi_info) {int rv; +if (unlikely (!smi_info->si_sm)) +return SI_SM_IDLE ; +if (!smi_info->waiting_msg) { smi_info->curr_msg =NULL ; rv =SI_SM_IDLE ;
@@ -800,6 +803,9 @@ static enum si_sm_result smi_event_handler(struct smi_info *smi_info, {enum si_sm_result si_sm_result; +if (unlikely (!smi_info->si_sm)) +return SI_SM_IDLE ; + restart:if (smi_info->si_state ==SI_HOSED )/* Just in case, hosed state is only left from the timeout. */
Signature · 1 line
2.54.0
Corey Minyard
maintainerQuoted 2 lines · Seiji Nishikawa
try_smi_init()allocatesnew_smi->si_smand later callsipmi_register_smi(), which maps toipmi_add_smi().
Thank you, the analysis is correct and very good.
I don't think the fix is right, though. I believe the right way to handle this is to set intf->in_shutdown right after out_err_started:
in ipmi_add_smi(). That was the intent, this case was missed.
And that way it's fixed for the other IPMI interfaces.
Do you agree?
-corey
Quoted 84 lines
During
ipmi_add_smi(), the upper IPMI message handler obtains the initial BMC device information through__bmc_get_device_id(). This can fail if the BMC does not return a successful response to the Get Device ID command.When the BMC returns a nonzero completion code, the device-id helper retries the command and eventually returns
-EIOif the device ID still cannot be fetched.On this failure path,
ipmi_add_smi()logs "Unable to get the device id" and goes toout_err_started, where it invokes the lower driver's shutdown callback.try_smi_init()then logs the returned registration failure:
ipmi_siIPI0001:00: IPMI message handler: Unable to get the device id: -5ipmi_siIPI0001:00: Unable to register device: error -5For
ipmi_si, the shutdown callback isshutdown_smi(), which cleans up the SI state machine data, freessmi_info->si_sm, and setssmi_info->si_smandsmi_info->intfto NULL.kfree(smi_info->si_sm); smi_info->si_sm = NULL; smi_info->intf = NULL;However, the
smi_infocan still be reached later. In the observed case, theredo_bmc_regwork item retried BMC device-id probing aftershutdown_smi()had already clearedsmi_info->si_sm. The retry path reachedstart_next_msg(), which passed the NULLsmi_info->si_smpointer to the selected KCS state machine handler:BUG: unable to handle kernel NULL pointer dereference at 0000000000000000 Workqueue: events redo_bmc_reg [ipmi_msghandler] RIP: start_kcs_transaction+0x2c/0x190 [ipmi_si] Call Trace: start_next_msg+0x50/0x80 [ipmi_si] check_start_timer_thread.part.9+0x3b/0x50 [ipmi_si] sender+0x69/0x80 [ipmi_si] i_ipmi_request+0x2ac/0x9d0 [ipmi_msghandler] __get_device_id.isra.29+0xaa/0x180 [ipmi_msghandler] __bmc_get_device_id+0xef/0x950 [ipmi_msghandler] redo_bmc_reg+0x52/0x60 [ipmi_msghandler] process_one_work+0x1a7/0x360
start_next_msg()andsmi_event_handler()both invoke the selected SI state machine handlers withsmi_info->si_smwithout checking whether the state machine data has already been freed and cleared.Add NULL checks before calling into the selected SI state machine handlers. If the state machine data has already been removed, treat the interface as idle and avoid dereferencing the cleared state.
Signed-off-by Seiji Nishikawa <[email protected]>--- drivers/char/ipmi/ipmi_si_intf.c | 6 ++++++ 1 file changed, 6 insertions(+)drivers/char/ipmi/ipmi_si_intf.c +6
-0@@ -320,6 +320,9 @@ static enum si_sm_result start_next_msg(struct smi_info *smi_info) {int rv; +if (unlikely (!smi_info->si_sm)) +return SI_SM_IDLE ; +if (!smi_info->waiting_msg) { smi_info->curr_msg =NULL ; rv =SI_SM_IDLE ;@@ -800,6 +803,9 @@ static enum si_sm_result smi_event_handler(struct smi_info *smi_info, {enum si_sm_result si_sm_result; +if (unlikely (!smi_info->si_sm)) +return SI_SM_IDLE ; + restart:if (smi_info->si_state ==SI_HOSED )/* Just in case, hosed state is only left from the timeout. */ Signature · 1 line
2.54.0
PATCH v2 1/1 ipmi: si: Fix NULL pointer dereference after failed registration
try_smi_init() allocates new_smi->si_sm and later calls ipmi_register_smi_mod(), which maps to ipmi_add_smi().
During ipmi_add_smi(), the upper IPMI message handler obtains the initial BMC device information through __bmc_get_device_id(). This can fail if the BMC does not return a successful response to the Get Device ID command.
When the BMC returns a nonzero completion code, the device-id helper retries the command and eventually returns -EIO if the device ID still cannot be fetched.
On this failure path, ipmi_add_smi() logs "Unable to get the device id" and goes to out_err_started, where it invokes the lower driver's shutdown callback. try_smi_init() then logs the returned registration failure:
ipmi_si IPI0001:00: IPMI message handler: Unable to get the device id: -5 ipmi_si IPI0001:00: Unable to register device: error -5
For ipmi_si, the shutdown callback is shutdown_smi(), which cleans up the SI state machine data, frees smi_info->si_sm, and sets smi_info->si_sm and smi_info->intf to NULL.
However, intf->in_shutdown is not set on this failed-registration rollback path. Therefore, the asynchronous redo_bmc_reg work item can still retry BMC device-id probing after the lower driver has already cleared its SI state machine data. In the observed case, that retry path reached start_next_msg(), which passed the NULL smi_info->si_sm pointer to the selected KCS state machine handler:
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000 Workqueue: events redo_bmc_reg [ipmi_msghandler] RIP: start_kcs_transaction+0x2c/0x190 [ipmi_si] Call Trace: start_next_msg+0x50/0x80 [ipmi_si] check_start_timer_thread.part.9+0x3b/0x50 [ipmi_si] sender+0x69/0x80 [ipmi_si] i_ipmi_request+0x2ac/0x9d0 [ipmi_msghandler] __get_device_id.isra.29+0xaa/0x180 [ipmi_msghandler] __bmc_get_device_id+0xef/0x950 [ipmi_msghandler] redo_bmc_reg+0x52/0x60 [ipmi_msghandler] process_one_work+0x1a7/0x360
Set intf->in_shutdown on the out_err_started path before invoking the lower driver's shutdown callback. This prevents later redo_bmc_reg retries from using an interface whose lower driver state has been cleaned up, and applies the same shutdown state to other IPMI interfaces as well.
---
Thanks, I agree. I moved the fix to the ipmi_add_smi() rollback path in v2 and dropped the SI-side NULL checks.
Changes in v2:
- Move the fix to ipmi_add_smi() by setting intf->in_shutdown on the out_err_started path, as suggested by Corey.
- Drop the NULL checks in ipmi_si_intf.c.
drivers/char/ipmi/ipmi_msghandler.c | 1 + 1 file changed, 1 insertion(+)
drivers/char/ipmi/ipmi_msghandler.c +1 -0
@@ -3757,6 +3757,7 @@ int ipmi_add_smi(struct module *owner, out_err_bmc_reg:ipmi_bmc_unregister (intf); out_err_started: + intf->in_shutdown = true;if (intf->handlers->shutdown) intf->handlers->shutdown (intf->send_info); out_err:
Signature · 1 line
2.54.0
Corey Minyard
maintainerRe: [PATCH v2] ipmi: si: Fix NULL pointer dereference after failed registration
Quoted 50 lines · Seiji Nishikawa
try_smi_init()allocatesnew_smi->si_smand later callsipmi_register_smi_mod(), which maps toipmi_add_smi().During
ipmi_add_smi(), the upper IPMI message handler obtains the initial BMC device information through__bmc_get_device_id(). This can fail if the BMC does not return a successful response to the Get Device ID command.When the BMC returns a nonzero completion code, the device-id helper retries the command and eventually returns
-EIOif the device ID still cannot be fetched.On this failure path,
ipmi_add_smi()logs "Unable to get the device id" and goes toout_err_started, where it invokes the lower driver's shutdown callback.try_smi_init()then logs the returned registration failure:
ipmi_siIPI0001:00: IPMI message handler: Unable to get the device id: -5ipmi_siIPI0001:00: Unable to register device: error -5For
ipmi_si, the shutdown callback isshutdown_smi(), which cleans up the SI state machine data, freessmi_info->si_sm, and setssmi_info->si_smandsmi_info->intfto NULL.However,
intf->in_shutdownis not set on this failed-registration rollback path. Therefore, the asynchronousredo_bmc_regwork item can still retry BMC device-id probing after the lower driver has already cleared its SI state machine data. In the observed case, that retry path reachedstart_next_msg(), which passed the NULLsmi_info->si_smpointer to the selected KCS state machine handler:BUG: unable to handle kernel NULL pointer dereference at 0000000000000000 Workqueue: events redo_bmc_reg [ipmi_msghandler] RIP: start_kcs_transaction+0x2c/0x190 [ipmi_si] Call Trace: start_next_msg+0x50/0x80 [ipmi_si] check_start_timer_thread.part.9+0x3b/0x50 [ipmi_si] sender+0x69/0x80 [ipmi_si] i_ipmi_request+0x2ac/0x9d0 [ipmi_msghandler] __get_device_id.isra.29+0xaa/0x180 [ipmi_msghandler] __bmc_get_device_id+0xef/0x950 [ipmi_msghandler] redo_bmc_reg+0x52/0x60 [ipmi_msghandler] process_one_work+0x1a7/0x360Set
intf->in_shutdownon theout_err_startedpath before invoking the lower driver's shutdown callback. This prevents laterredo_bmc_regretries from using an interface whose lower driver state has been cleaned up, and applies the same shutdown state to other IPMI interfaces as well.
Quoted 4 lines
Signed-off-by Seiji Nishikawa <[email protected]>---
Thanks, I agree. I moved the fix to theipmi_add_smi()rollback path in v2 and dropped the SI-side NULL checks.
Thanks, this is in my for-next tree.
-corey
Quoted 24 lines
Changes in v2:
- Move the fix toipmi_add_smi()by settingintf->in_shutdownon theout_err_startedpath, as suggested by Corey.
- Drop the NULL checks inipmi_si_intf.c.drivers/char/ipmi/ipmi_msghandler.c | 1 + 1 file changed, 1 insertion(+)drivers/char/ipmi/ipmi_msghandler.c +1
-0@@ -3757,6 +3757,7 @@ int ipmi_add_smi(struct module *owner, out_err_bmc_reg:ipmi_bmc_unregister (intf); out_err_started: + intf->in_shutdown = true;if (intf->handlers->shutdown) intf->handlers->shutdown (intf->send_info); out_err:Signature · 1 line
2.54.0