ALSA: hda: check RIRB to avoid use NULL pointer
PATCH 1/1 ALSA: hda: check RIRB to avoid use NULL pointer
From: Liwei Song <[email protected]>
Fix the following BUG:
BUG: unable to handle kernel NULL pointer dereference at 000000000000000c Workqueue: events azx_probe_work [snd_hda_intel] RIP: 0010:snd_hdac_bus_update_rirb+0x80/0x160 [snd_hda_core] Call Trace: <IRQ> azx_interrupt+0x78/0x140 [snd_hda_codec] __handle_irq_event_percpu+0x49/0x300 handle_irq_event_percpu+0x23/0x60 handle_irq_event+0x3c/0x60 handle_edge_irq+0xdb/0x180 handle_irq+0x23/0x30 do_IRQ+0x6a/0x140 common_interrupt+0xf/0xf
The Call Trace happened when run kdump on a NFS rootfs system. Exist the following calling sequence when boot the second kernel:
azx_first_init()
--> azx_acquire_irq()
<-- interrupt come in, azx_interrupt() was called
--> hda_intel_init_chip()
--> azx_init_chip()
--> snd_hdac_bus_init_chip()
--> snd_hdac_bus_init_cmd_io();
--> init rirb.buf and corb.bufInterrupt happened after azx_acquire_irq() while RIRB still didn't got initialized, then NULL pointer will be used when process the interrupt.
Check the value of RIRB to ensure it is not NULL, to aviod some special case may hang the system.
--- sound/hda/hdac_controller.c | 3 +++ 1 file changed, 3 insertions(+)
sound/hda/hdac_controller.c +3 -0
@@ -195,6 +195,9 @@ void snd_hdac_bus_update_rirb(struct hdac_bus *bus)return ; bus->rirb.wp = wp; +if (!bus->rirb.buf) +return ; +while (bus->rirb.rp != wp) { bus->rirb.rp++; bus->rirb.rp %=AZX_MAX_RIRB_ENTRIES ;
Signature · 1 line
2.7.4
Quoted 31 lines · Song liwei
From: Liwei Song <[email protected]>
Fix the following BUG:
BUG: unable to handle kernel NULL pointer dereference at 000000000000000c Workqueue: events azx_probe_work [snd_hda_intel] RIP: 0010:snd_hdac_bus_update_rirb+0x80/0x160 [snd_hda_core] Call Trace: <IRQ> azx_interrupt+0x78/0x140 [snd_hda_codec] __handle_irq_event_percpu+0x49/0x300 handle_irq_event_percpu+0x23/0x60 handle_irq_event+0x3c/0x60 handle_edge_irq+0xdb/0x180 handle_irq+0x23/0x30 do_IRQ+0x6a/0x140 common_interrupt+0xf/0xfThe Call Trace happened when run kdump on a NFS rootfs system. Exist the following calling sequence when boot the second kernel:
azx_first_init()
-->azx_acquire_irq()
<-- interrupt come in,azx_interrupt()was called
-->hda_intel_init_chip()--> azx_init_chip() --> snd_hdac_bus_init_chip() --> snd_hdac_bus_init_cmd_io(); --> init rirb.buf and corb.buf
Quoted 8 lines
Interrupt happened after
azx_acquire_irq()while RIRB still didn't got initialized, then NULL pointer will be used when process the interrupt.Check the value of RIRB to ensure it is not NULL, to aviod some special case may hang the system.
Fixes 14752412721c ("ALSA: hda - Add the controller helper codes to hda-core module")Signed-off-by Liwei Song <[email protected]>
Oh, that's indeed a race there.
But I guess the check introduced by the patch is still error-prone. Basically the interrupt handling should be moved after the chip initialization. I suppose that your platform uses the shared interrupt, not the MSI?
In anyway, alternative (and likely more certain) fix would be to move the azx_acquir_irq() call like the patch below (note: totally untested). Could you check whether it works?
thanks,
Takashi
sound/pci/hda/hda_intel.c +3 -3
@@ -1788,9 +1788,6 @@ static int azx_first_init(struct azx *chip) chip->msi =0 ; } -if (azx_acquire_irq (chip,0 ) <0 ) -return -EBUSY ; -pci_set_master (pci);synchronize_irq (bus->irq);
@@ -1904,6 +1901,9 @@ static int azx_first_init(struct azx *chip)return -ENODEV ; } +if (azx_acquire_irq (chip,0 ) <0 ) +return -EBUSY ; +strcpy (card->driver,"HDA-Intel" );strlcpy (card->shortname, driver_short_names[chip->driver_type],sizeof (card->shortname));
Quoted 42 lines · Takashi Iwai
Quoted 31 lines · Song liwei
From: Liwei Song <[email protected]>
Fix the following BUG:
BUG: unable to handle kernel NULL pointer dereference at 000000000000000c Workqueue: events azx_probe_work [snd_hda_intel] RIP: 0010:snd_hdac_bus_update_rirb+0x80/0x160 [snd_hda_core] Call Trace: <IRQ> azx_interrupt+0x78/0x140 [snd_hda_codec] __handle_irq_event_percpu+0x49/0x300 handle_irq_event_percpu+0x23/0x60 handle_irq_event+0x3c/0x60 handle_edge_irq+0xdb/0x180 handle_irq+0x23/0x30 do_IRQ+0x6a/0x140 common_interrupt+0xf/0xfThe Call Trace happened when run kdump on a NFS rootfs system. Exist the following calling sequence when boot the second kernel:
azx_first_init()
-->azx_acquire_irq()
<-- interrupt come in,azx_interrupt()was called
-->hda_intel_init_chip()--> azx_init_chip() --> snd_hdac_bus_init_chip() --> snd_hdac_bus_init_cmd_io(); --> init rirb.buf and corb.bufQuoted 8 lines
Interrupt happened after
azx_acquire_irq()while RIRB still didn't got initialized, then NULL pointer will be used when process the interrupt.Check the value of RIRB to ensure it is not NULL, to aviod some special case may hang the system.
Fixes 14752412721c ("ALSA: hda - Add the controller helper codes to hda-core module")Signed-off-by Liwei Song <[email protected]>
Quoted 6 lines
Oh, that's indeed a race there.
But I guess the check introduced by the patch is still error-prone. Basically the interrupt handling should be moved after the chip initialization. I suppose that your platform uses the shared interrupt, not the MSI?
This is the information from /proc/interrupt
134: 0 102 0 0 IR-PCI-MSI 514048-edge snd_hda_intel:card0
Quoted 4 lines
In anyway, alternative (and likely more certain) fix would be to move the
azx_acquir_irq()call like the patch below (note: totally untested). Could you check whether it works?
Yes, It works.
Considering a previous patch like the one you provide will import some issue, so I choose check the invalid value to low the risk, but just as you mentioned, It is not a good solution.
commit 542cedec53c9e8b73f3f05bf8468823598c50489
Author: Yu Zhao <[email protected]> Date: Tue Sep 11 15:12:46 2018 -0600
Revert "ASoC: Intel: Skylake: Acquire irq after RIRB allocation"
This reverts commit 12eeeb4f4733bbc4481d01df35933fc15beb8b19.
The patch doesn't fix accessing memory with null pointer in skl_interrupt().
There are two problems: 1) skl_init_chip() is called twice, before and after dma buffer is allocate. The first call sets bus->chip_init which prevents the second from initializing bus->corb.buf and rirb.buf from bus->rb.area. 2) snd_hdac_bus_init_chip() enables interrupt before snd_hdac_bus_init_cmd_io() initializing dma buffers. There is a small window which skl_interrupt() can be called if irq has been acquired. If so, it crashes when using null dma buffer pointers.
Thanks,
Liwei.
Quoted 31 lines
thanks,
Takashi
sound/pci/hda/hda_intel.c +3
-3@@ -1788,9 +1788,6 @@ static int azx_first_init(struct azx *chip) chip->msi =0 ; } -if (azx_acquire_irq (chip,0 ) <0 ) -return -EBUSY ; -pci_set_master (pci);synchronize_irq (bus->irq);@@ -1904,6 +1901,9 @@ static int azx_first_init(struct azx *chip)return -ENODEV ; } +if (azx_acquire_irq (chip,0 ) <0 ) +return -EBUSY ; +strcpy (card->driver,"HDA-Intel" );strlcpy (card->shortname, driver_short_names[chip->driver_type],sizeof (card->shortname));
Quoted 48 lines · Liwei Song
Quoted 44 lines · Takashi Iwai
Quoted 31 lines · Song liwei
From: Liwei Song <[email protected]>
Fix the following BUG:
BUG: unable to handle kernel NULL pointer dereference at 000000000000000c Workqueue: events azx_probe_work [snd_hda_intel] RIP: 0010:snd_hdac_bus_update_rirb+0x80/0x160 [snd_hda_core] Call Trace: <IRQ> azx_interrupt+0x78/0x140 [snd_hda_codec] __handle_irq_event_percpu+0x49/0x300 handle_irq_event_percpu+0x23/0x60 handle_irq_event+0x3c/0x60 handle_edge_irq+0xdb/0x180 handle_irq+0x23/0x30 do_IRQ+0x6a/0x140 common_interrupt+0xf/0xfThe Call Trace happened when run kdump on a NFS rootfs system. Exist the following calling sequence when boot the second kernel:
azx_first_init()
-->azx_acquire_irq()
<-- interrupt come in,azx_interrupt()was called
-->hda_intel_init_chip()--> azx_init_chip() --> snd_hdac_bus_init_chip() --> snd_hdac_bus_init_cmd_io(); --> init rirb.buf and corb.bufQuoted 8 lines
Interrupt happened after
azx_acquire_irq()while RIRB still didn't got initialized, then NULL pointer will be used when process the interrupt.Check the value of RIRB to ensure it is not NULL, to aviod some special case may hang the system.
Fixes 14752412721c ("ALSA: hda - Add the controller helper codes to hda-core module")Signed-off-by Liwei Song <[email protected]>Oh, that's indeed a race there.
Quoted 7 lines
Quoted 4 lines
But I guess the check introduced by the patch is still error-prone. Basically the interrupt handling should be moved after the chip initialization. I suppose that your platform uses the shared interrupt, not the MSI?
This is the information from
/proc/interrupt
134: 0 102 0 0 IR-PCI-MSI 514048-edge snd_hda_intel:card0
Hm, then it's interesting...
Quoted 21 lines
Quoted 3 lines
In anyway, alternative (and likely more certain) fix would be to move the
azx_acquir_irq()call like the patch below (note: totally untested). Could you check whether it works?Yes, It works.
Considering a previous patch like the one you provide will import some issue, so I choose check the invalid value to low the risk, but just as you mentioned, It is not a good solution.
commit
542cedec53c9e8b73f3f05bf8468823598c50489Author: Yu Zhao <[email protected]> Date: Tue Sep 11 15:12:46 2018 -0600Revert "ASoC: Intel: Skylake: Acquire irq after RIRB allocation"
This reverts commit
12eeeb4f4733bbc4481d01df35933fc15beb8b19.The patch doesn't fix accessing memory with null pointer in
skl_interrupt().
Quoted 8 lines
There are two problems: 1)
skl_init_chip()is called twice, before and after dma buffer is allocate. The first call setsbus->chip_initwhich prevents the second from initializingbus->corb.bufand rirb.buf frombus->rb.area. 2)snd_hdac_bus_init_chip()enables interrupt beforesnd_hdac_bus_init_cmd_io()initializing dma buffers. There is a small window whichskl_interrupt()can be called if irq has been acquired. If so, it crashes when using null dma buffer pointers.
Actually this followed by another fix b61749a89f82,
sound: enable interrupt after dma buffer initialization
and this moved the IRQ enablement after snd_hdac_bus_init_cmd_io().
So I wonder how the irq gets triggered in your case.
If it were a shared irq, it's understandable. But for MSI, it should have been the isolated source.
In anyway, for the latest tree, the change I suggested would cover better although it's more radical as you pointed.
thanks,
Takashi
Quoted 91 lines · Takashi Iwai
Quoted 48 lines · Liwei Song
Quoted 44 lines · Takashi Iwai
Quoted 31 lines · Song liwei
From: Liwei Song <[email protected]>
Fix the following BUG:
BUG: unable to handle kernel NULL pointer dereference at 000000000000000c Workqueue: events azx_probe_work [snd_hda_intel] RIP: 0010:snd_hdac_bus_update_rirb+0x80/0x160 [snd_hda_core] Call Trace: <IRQ> azx_interrupt+0x78/0x140 [snd_hda_codec] __handle_irq_event_percpu+0x49/0x300 handle_irq_event_percpu+0x23/0x60 handle_irq_event+0x3c/0x60 handle_edge_irq+0xdb/0x180 handle_irq+0x23/0x30 do_IRQ+0x6a/0x140 common_interrupt+0xf/0xfThe Call Trace happened when run kdump on a NFS rootfs system. Exist the following calling sequence when boot the second kernel:
azx_first_init()
-->azx_acquire_irq()
<-- interrupt come in,azx_interrupt()was called
-->hda_intel_init_chip()--> azx_init_chip() --> snd_hdac_bus_init_chip() --> snd_hdac_bus_init_cmd_io(); --> init rirb.buf and corb.bufQuoted 8 lines
Interrupt happened after
azx_acquire_irq()while RIRB still didn't got initialized, then NULL pointer will be used when process the interrupt.Check the value of RIRB to ensure it is not NULL, to aviod some special case may hang the system.
Fixes 14752412721c ("ALSA: hda - Add the controller helper codes to hda-core module")Signed-off-by Liwei Song <[email protected]>Oh, that's indeed a race there.
Quoted 7 lines
Quoted 4 lines
But I guess the check introduced by the patch is still error-prone. Basically the interrupt handling should be moved after the chip initialization. I suppose that your platform uses the shared interrupt, not the MSI?
This is the information from
/proc/interrupt
134: 0 102 0 0 IR-PCI-MSI 514048-edge snd_hda_intel:card0Hm, then it's interesting...
Quoted 21 lines
Quoted 3 lines
In anyway, alternative (and likely more certain) fix would be to move the
azx_acquir_irq()call like the patch below (note: totally untested). Could you check whether it works?Yes, It works.
Considering a previous patch like the one you provide will import some issue, so I choose check the invalid value to low the risk, but just as you mentioned, It is not a good solution.
commit
542cedec53c9e8b73f3f05bf8468823598c50489Author: Yu Zhao <[email protected]> Date: Tue Sep 11 15:12:46 2018 -0600Revert "ASoC: Intel: Skylake: Acquire irq after RIRB allocation"
This reverts commit
12eeeb4f4733bbc4481d01df35933fc15beb8b19.The patch doesn't fix accessing memory with null pointer in
skl_interrupt().Quoted 8 lines
There are two problems: 1)
skl_init_chip()is called twice, before and after dma buffer is allocate. The first call setsbus->chip_initwhich prevents the second from initializingbus->corb.bufand rirb.buf frombus->rb.area. 2)snd_hdac_bus_init_chip()enables interrupt beforesnd_hdac_bus_init_cmd_io()initializing dma buffers. There is a small window whichskl_interrupt()can be called if irq has been acquired. If so, it crashes when using null dma buffer pointers.
Quoted 8 lines
Actually this followed by another fix
b61749a89f82,
sound: enable interrupt after dma buffer initializationand this moved the IRQ enablement after
snd_hdac_bus_init_cmd_io().So I wonder how the irq gets triggered in your case.
If it were a shared irq, it's understandable. But for MSI, it should have been the isolated source.
I'm still working on how the irq was triggered,
it is a little complex to reproduce it, first it must run with NFS rootfs, without NFS rootfs it can not reproduced.
Then with kdump enabled, after "echo c > /proc/sysrq-trigger" crash the kernel, the kernel specified by kdump will boot, then interrupt will trigger soon after azx interrupt was register.
Quoted 3 lines
In anyway, for the latest tree, the change I suggested would cover better although it's more radical as you pointed.
Got it, Thanks.
Liwei.
Quoted 7 lines
thanks,
Takashi
Quoted 104 lines · Liwei Song
Quoted 91 lines · Takashi Iwai
Quoted 48 lines · Liwei Song
Quoted 44 lines · Takashi Iwai
Quoted 31 lines · Song liwei
From: Liwei Song <[email protected]>
Fix the following BUG:
BUG: unable to handle kernel NULL pointer dereference at 000000000000000c Workqueue: events azx_probe_work [snd_hda_intel] RIP: 0010:snd_hdac_bus_update_rirb+0x80/0x160 [snd_hda_core] Call Trace: <IRQ> azx_interrupt+0x78/0x140 [snd_hda_codec] __handle_irq_event_percpu+0x49/0x300 handle_irq_event_percpu+0x23/0x60 handle_irq_event+0x3c/0x60 handle_edge_irq+0xdb/0x180 handle_irq+0x23/0x30 do_IRQ+0x6a/0x140 common_interrupt+0xf/0xfThe Call Trace happened when run kdump on a NFS rootfs system. Exist the following calling sequence when boot the second kernel:
azx_first_init()
-->azx_acquire_irq()
<-- interrupt come in,azx_interrupt()was called
-->hda_intel_init_chip()--> azx_init_chip() --> snd_hdac_bus_init_chip() --> snd_hdac_bus_init_cmd_io(); --> init rirb.buf and corb.bufQuoted 8 lines
Interrupt happened after
azx_acquire_irq()while RIRB still didn't got initialized, then NULL pointer will be used when process the interrupt.Check the value of RIRB to ensure it is not NULL, to aviod some special case may hang the system.
Fixes 14752412721c ("ALSA: hda - Add the controller helper codes to hda-core module")Signed-off-by Liwei Song <[email protected]>Oh, that's indeed a race there.
Quoted 7 lines
Quoted 4 lines
But I guess the check introduced by the patch is still error-prone. Basically the interrupt handling should be moved after the chip initialization. I suppose that your platform uses the shared interrupt, not the MSI?
This is the information from
/proc/interrupt
134: 0 102 0 0 IR-PCI-MSI 514048-edge snd_hda_intel:card0Hm, then it's interesting...
Quoted 21 lines
Quoted 3 lines
In anyway, alternative (and likely more certain) fix would be to move the
azx_acquir_irq()call like the patch below (note: totally untested). Could you check whether it works?Yes, It works.
Considering a previous patch like the one you provide will import some issue, so I choose check the invalid value to low the risk, but just as you mentioned, It is not a good solution.
commit
542cedec53c9e8b73f3f05bf8468823598c50489Author: Yu Zhao <[email protected]> Date: Tue Sep 11 15:12:46 2018 -0600Revert "ASoC: Intel: Skylake: Acquire irq after RIRB allocation"
This reverts commit
12eeeb4f4733bbc4481d01df35933fc15beb8b19.The patch doesn't fix accessing memory with null pointer in
skl_interrupt().Quoted 8 lines
There are two problems: 1)
skl_init_chip()is called twice, before and after dma buffer is allocate. The first call setsbus->chip_initwhich prevents the second from initializingbus->corb.bufand rirb.buf frombus->rb.area. 2)snd_hdac_bus_init_chip()enables interrupt beforesnd_hdac_bus_init_cmd_io()initializing dma buffers. There is a small window whichskl_interrupt()can be called if irq has been acquired. If so, it crashes when using null dma buffer pointers.Quoted 8 lines
Actually this followed by another fix
b61749a89f82,
sound: enable interrupt after dma buffer initializationand this moved the IRQ enablement after
snd_hdac_bus_init_cmd_io().So I wonder how the irq gets triggered in your case.
If it were a shared irq, it's understandable. But for MSI, it should have been the isolated source.
Quoted 6 lines
I'm still working on how the irq was triggered,
it is a little complex to reproduce it, first it must run with NFS rootfs, without NFS rootfs it can not reproduced.
Then with kdump enabled, after "echo c >/proc/sysrq-trigger" crash the kernel, the kernel specified by kdump will boot, then interrupt will trigger soon after azx interrupt was register.
Ah, so it happens in a kdump kernel? It implies that the interrupt line may be still active (or confused). Then it's no wonder a stale interrupt comes up.
Quoted 4 lines
Quoted 2 lines
In anyway, for the latest tree, the change I suggested would cover better although it's more radical as you pointed.
Got it, Thanks.
OK, I'm going to submit and apply the proper patch.
thanks,
Takashi