Skip to contentSkip to conversation

ipmi: si: Fix NULL pointer dereference in start_kcs_transaction()

PATCH In review Seiji Nishikawa 2 participants 4 messages Jun 30 – Jul 2 IPMI

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 ipmi: si: Fix NULL pointer dereference in start_kcs_transaction()

try_smi_init() allocates new_smi->si_sm and later calls ipmi_register_smi(), which maps to ipmi_add_smi().

During ipmi_add_smi(), the upper IPMI message handler obtains the initial BMC device information through __bmc_get_device_id(). This can fail if the BMC does not return a successful response to the Get Device ID command.

When the BMC returns a nonzero completion code, the device-id helper retries the command and eventually returns -EIO if the device ID still cannot be fetched.

On this failure path, ipmi_add_smi() logs "Unable to get the device id" and goes to out_err_started, where it invokes the lower driver's shutdown callback. try_smi_init() then logs the returned registration failure:

ipmi_si IPI0001:00: IPMI message handler: Unable to get the device id: -5 ipmi_si IPI0001:00: Unable to register device: error -5

For ipmi_si, the shutdown callback is shutdown_smi(), which cleans up the SI state machine data, frees smi_info->si_sm, and sets smi_info->si_sm and smi_info->intf to NULL.

        kfree(smi_info->si_sm);
        smi_info->si_sm = NULL;

        smi_info->intf = NULL;

However, the smi_info can still be reached later. In the observed case, the redo_bmc_reg work item retried BMC device-id probing after shutdown_smi() had already cleared smi_info->si_sm. The retry path reached start_next_msg(), which passed the NULL smi_info->si_sm pointer to the selected KCS state machine handler:

BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
Workqueue: events redo_bmc_reg [ipmi_msghandler]
RIP: start_kcs_transaction+0x2c/0x190 [ipmi_si]
Call Trace:
 start_next_msg+0x50/0x80 [ipmi_si]
 check_start_timer_thread.part.9+0x3b/0x50 [ipmi_si]
 sender+0x69/0x80 [ipmi_si]
 i_ipmi_request+0x2ac/0x9d0 [ipmi_msghandler]
 __get_device_id.isra.29+0xaa/0x180 [ipmi_msghandler]
 __bmc_get_device_id+0xef/0x950 [ipmi_msghandler]
 redo_bmc_reg+0x52/0x60 [ipmi_msghandler]
 process_one_work+0x1a7/0x360

start_next_msg() and smi_event_handler() both invoke the selected SI state machine handlers with smi_info->si_sm without checking whether the state machine data has already been freed and cleared.

Add NULL checks before calling into the selected SI state machine handlers. If the state machine data has already been removed, treat the interface as idle and avoid dereferencing the cleared state.

---
 drivers/char/ipmi/ipmi_si_intf.c | 6 ++++++
 1 file changed, 6 insertions(+)
drivers/char/ipmi/ipmi_si_intf.c +6 -0
index 9a9d12be9bf7..d13d5024352a 100644
@@ -320,6 +320,9 @@ static enum si_sm_result start_next_msg(struct smi_info *smi_info)
 {
 	int rv;
 
+	if (unlikely(!smi_info->si_sm))
+		return SI_SM_IDLE;
+
 	if (!smi_info->waiting_msg) {
 		smi_info->curr_msg = NULL;
 		rv = SI_SM_IDLE;
@@ -800,6 +803,9 @@ static enum si_sm_result smi_event_handler(struct smi_info *smi_info,
 {
 	enum si_sm_result si_sm_result;
 
+	if (unlikely(!smi_info->si_sm))
+		return SI_SM_IDLE;
+
 restart:
 	if (smi_info->si_state == SI_HOSED)
 		/* Just in case, hosed state is only left from the timeout. */
Signature · 1 line
2.54.0
maintainer
#1 lore raw
Quoted 2 lines · Seiji Nishikawa

try_smi_init() allocates new_smi->si_sm and later calls ipmi_register_smi(), which maps to ipmi_add_smi().

Thank you, the analysis is correct and very good.

I don't think the fix is right, though. I believe the right way to handle this is to set intf->in_shutdown right after out_err_started:
in ipmi_add_smi(). That was the intent, this case was missed.

And that way it's fixed for the other IPMI interfaces.

Do you agree?

-corey

Quoted 84 lines

During ipmi_add_smi(), the upper IPMI message handler obtains the initial BMC device information through __bmc_get_device_id(). This can fail if the BMC does not return a successful response to the Get Device ID command.

When the BMC returns a nonzero completion code, the device-id helper retries the command and eventually returns -EIO if the device ID still cannot be fetched.

On this failure path, ipmi_add_smi() logs "Unable to get the device id" and goes to out_err_started, where it invokes the lower driver's shutdown callback. try_smi_init() then logs the returned registration failure:

ipmi_si IPI0001:00: IPMI message handler: Unable to get the device id: -5 ipmi_si IPI0001:00: Unable to register device: error -5

For ipmi_si, the shutdown callback is shutdown_smi(), which cleans up the SI state machine data, frees smi_info->si_sm, and sets smi_info->si_sm and smi_info->intf to NULL.

        kfree(smi_info->si_sm);
        smi_info->si_sm = NULL;

        smi_info->intf = NULL;

However, the smi_info can still be reached later. In the observed case, the redo_bmc_reg work item retried BMC device-id probing after shutdown_smi() had already cleared smi_info->si_sm. The retry path reached start_next_msg(), which passed the NULL smi_info->si_sm pointer to the selected KCS state machine handler:

BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
Workqueue: events redo_bmc_reg [ipmi_msghandler]
RIP: start_kcs_transaction+0x2c/0x190 [ipmi_si]
Call Trace:
 start_next_msg+0x50/0x80 [ipmi_si]
 check_start_timer_thread.part.9+0x3b/0x50 [ipmi_si]
 sender+0x69/0x80 [ipmi_si]
 i_ipmi_request+0x2ac/0x9d0 [ipmi_msghandler]
 __get_device_id.isra.29+0xaa/0x180 [ipmi_msghandler]
 __bmc_get_device_id+0xef/0x950 [ipmi_msghandler]
 redo_bmc_reg+0x52/0x60 [ipmi_msghandler]
 process_one_work+0x1a7/0x360

start_next_msg() and smi_event_handler() both invoke the selected SI state machine handlers with smi_info->si_sm without checking whether the state machine data has already been freed and cleared.

Add NULL checks before calling into the selected SI state machine handlers. If the state machine data has already been removed, treat the interface as idle and avoid dereferencing the cleared state.

---
 drivers/char/ipmi/ipmi_si_intf.c | 6 ++++++
 1 file changed, 6 insertions(+)
drivers/char/ipmi/ipmi_si_intf.c +6 -0
index 9a9d12be9bf7..d13d5024352a 100644
@@ -320,6 +320,9 @@ static enum si_sm_result start_next_msg(struct smi_info *smi_info)
 {
 	int rv;
 
+	if (unlikely(!smi_info->si_sm))
+		return SI_SM_IDLE;
+
 	if (!smi_info->waiting_msg) {
 		smi_info->curr_msg = NULL;
 		rv = SI_SM_IDLE;
@@ -800,6 +803,9 @@ static enum si_sm_result smi_event_handler(struct smi_info *smi_info,
 {
 	enum si_sm_result si_sm_result;
 
+	if (unlikely(!smi_info->si_sm))
+		return SI_SM_IDLE;
+
 restart:
 	if (smi_info->si_state == SI_HOSED)
 		/* Just in case, hosed state is only left from the timeout. */
Signature · 1 line
2.54.0
#2 lore raw

PATCH v2 1/1 ipmi: si: Fix NULL pointer dereference after failed registration

try_smi_init() allocates new_smi->si_sm and later calls ipmi_register_smi_mod(), which maps to ipmi_add_smi().

During ipmi_add_smi(), the upper IPMI message handler obtains the initial BMC device information through __bmc_get_device_id(). This can fail if the BMC does not return a successful response to the Get Device ID command.

When the BMC returns a nonzero completion code, the device-id helper retries the command and eventually returns -EIO if the device ID still cannot be fetched.

On this failure path, ipmi_add_smi() logs "Unable to get the device id" and goes to out_err_started, where it invokes the lower driver's shutdown callback. try_smi_init() then logs the returned registration failure:

ipmi_si IPI0001:00: IPMI message handler: Unable to get the device id: -5 ipmi_si IPI0001:00: Unable to register device: error -5

For ipmi_si, the shutdown callback is shutdown_smi(), which cleans up the SI state machine data, frees smi_info->si_sm, and sets smi_info->si_sm and smi_info->intf to NULL.

However, intf->in_shutdown is not set on this failed-registration rollback path. Therefore, the asynchronous redo_bmc_reg work item can still retry BMC device-id probing after the lower driver has already cleared its SI state machine data. In the observed case, that retry path reached start_next_msg(), which passed the NULL smi_info->si_sm pointer to the selected KCS state machine handler:

BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
Workqueue: events redo_bmc_reg [ipmi_msghandler]
RIP: start_kcs_transaction+0x2c/0x190 [ipmi_si]
Call Trace:
 start_next_msg+0x50/0x80 [ipmi_si]
 check_start_timer_thread.part.9+0x3b/0x50 [ipmi_si]
 sender+0x69/0x80 [ipmi_si]
 i_ipmi_request+0x2ac/0x9d0 [ipmi_msghandler]
 __get_device_id.isra.29+0xaa/0x180 [ipmi_msghandler]
 __bmc_get_device_id+0xef/0x950 [ipmi_msghandler]
 redo_bmc_reg+0x52/0x60 [ipmi_msghandler]
 process_one_work+0x1a7/0x360

Set intf->in_shutdown on the out_err_started path before invoking the lower driver's shutdown callback. This prevents later redo_bmc_reg retries from using an interface whose lower driver state has been cleaned up, and applies the same shutdown state to other IPMI interfaces as well.

---
Thanks, I agree. I moved the fix to the ipmi_add_smi() rollback path in v2 and dropped the SI-side NULL checks.

Changes in v2:
- Move the fix to ipmi_add_smi() by setting intf->in_shutdown on the out_err_started path, as suggested by Corey.
- Drop the NULL checks in ipmi_si_intf.c.

 drivers/char/ipmi/ipmi_msghandler.c | 1 +
 1 file changed, 1 insertion(+)
drivers/char/ipmi/ipmi_msghandler.c +1 -0
index ab4c85f3d6fe..8d9f2e647d9b 100644
@@ -3757,6 +3757,7 @@ int ipmi_add_smi(struct module         *owner,
  out_err_bmc_reg:
 	ipmi_bmc_unregister(intf);
  out_err_started:
+	intf->in_shutdown = true;
 	if (intf->handlers->shutdown)
 		intf->handlers->shutdown(intf->send_info);
  out_err:
Signature · 1 line
2.54.0
maintainer
#3 lore raw

Re: [PATCH v2] ipmi: si: Fix NULL pointer dereference after failed registration

Quoted 50 lines · Seiji Nishikawa

try_smi_init() allocates new_smi->si_sm and later calls ipmi_register_smi_mod(), which maps to ipmi_add_smi().

During ipmi_add_smi(), the upper IPMI message handler obtains the initial BMC device information through __bmc_get_device_id(). This can fail if the BMC does not return a successful response to the Get Device ID command.

When the BMC returns a nonzero completion code, the device-id helper retries the command and eventually returns -EIO if the device ID still cannot be fetched.

On this failure path, ipmi_add_smi() logs "Unable to get the device id" and goes to out_err_started, where it invokes the lower driver's shutdown callback. try_smi_init() then logs the returned registration failure:

ipmi_si IPI0001:00: IPMI message handler: Unable to get the device id: -5 ipmi_si IPI0001:00: Unable to register device: error -5

For ipmi_si, the shutdown callback is shutdown_smi(), which cleans up the SI state machine data, frees smi_info->si_sm, and sets smi_info->si_sm and smi_info->intf to NULL.

However, intf->in_shutdown is not set on this failed-registration rollback path. Therefore, the asynchronous redo_bmc_reg work item can still retry BMC device-id probing after the lower driver has already cleared its SI state machine data. In the observed case, that retry path reached start_next_msg(), which passed the NULL smi_info->si_sm pointer to the selected KCS state machine handler:

BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
Workqueue: events redo_bmc_reg [ipmi_msghandler]
RIP: start_kcs_transaction+0x2c/0x190 [ipmi_si]
Call Trace:
 start_next_msg+0x50/0x80 [ipmi_si]
 check_start_timer_thread.part.9+0x3b/0x50 [ipmi_si]
 sender+0x69/0x80 [ipmi_si]
 i_ipmi_request+0x2ac/0x9d0 [ipmi_msghandler]
 __get_device_id.isra.29+0xaa/0x180 [ipmi_msghandler]
 __bmc_get_device_id+0xef/0x950 [ipmi_msghandler]
 redo_bmc_reg+0x52/0x60 [ipmi_msghandler]
 process_one_work+0x1a7/0x360

Set intf->in_shutdown on the out_err_started path before invoking the lower driver's shutdown callback. This prevents later redo_bmc_reg retries from using an interface whose lower driver state has been cleaned up, and applies the same shutdown state to other IPMI interfaces as well.

Quoted 4 lines

---
Thanks, I agree. I moved the fix to the ipmi_add_smi() rollback path in v2 and dropped the SI-side NULL checks.

Thanks, this is in my for-next tree.

-corey

Quoted 24 lines

Changes in v2:
- Move the fix to ipmi_add_smi() by setting intf->in_shutdown on the out_err_started path, as suggested by Corey.
- Drop the NULL checks in ipmi_si_intf.c.

 drivers/char/ipmi/ipmi_msghandler.c | 1 +
 1 file changed, 1 insertion(+)
drivers/char/ipmi/ipmi_msghandler.c +1 -0
index ab4c85f3d6fe..8d9f2e647d9b 100644
@@ -3757,6 +3757,7 @@ int ipmi_add_smi(struct module         *owner,
  out_err_bmc_reg:
 	ipmi_bmc_unregister(intf);
  out_err_started:
+	intf->in_shutdown = true;
 	if (intf->handlers->shutdown)
 		intf->handlers->shutdown(intf->send_info);
  out_err:
Signature · 1 line
2.54.0