Skip to contentSkip to conversation

NOC (interconnect) lockup on i.MX8MP during resume

PATCH Alexander Stein 3 participants 5 messages Mar 9, 2023 – Mar 14, 2023

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi,

while debugging another issue regarding suspend/resume I noticed that the imx interconnect driver locks up the system when writing into registers. This happens during resume of the 'g2' power domain. imx8m_blk_ctrl_power_on() will eventually call down to imx_icc_node_set(). Using the debug diff furthe down I get this output:

[   71.493062] imx_icc_node_set: node: VPU G2
[   71.497163] imx_icc_node_set: peak_bw: 1
[   71.501091] imx_icc_node_set: prio: 0x80000303

<freeze here>

Unfortunately there is absolutely no documentation regarding NOC in the reference manual, so no idea whats happening here.
This is on next-20230309 using imx8mp-tqma8mpql-mba8mpxl.dts with USB disabled (prevents suspend) and Ethernet-PHY-IRQ disabled (the issue I'm tackling).
Is there some idea what do in order to support resume on i.MX8MP?

Best regards,
Alexander

---8<---

drivers/interconnect/imx/imx.c +5 -0
index 979ed610f704..93ec79cc7c06 100644
@@ -44,13 +44,18 @@ static int imx_icc_node_set(struct icc_node *node)
        u32 prio;
        u64 freq;
 
+       pr_info("%s: node: %s\n", __func__, node->name);
+       pr_info("%s: peak_bw: %u\n", __func__, node->peak_bw);
        if (node_data->setting && node->peak_bw) {
                base = node_data->setting->reg + node_data->imx_provider->noc_base;
                if (node_data->setting->mode == IMX_NOC_MODE_FIXED) {
                        prio = node_data->setting->prio_level;
                        prio = PRIORITY_COMP_MARK | (prio << 8) | prio;
+                       pr_info("%s: prio: %#x\n", __func__, prio);
                        writel(prio, base + IMX_NOC_PRIO_REG);
+                       pr_info("%s: mode: %#x\n", __func__, node_data->setting->mode);
                        writel(node_data->setting->mode, base + IMX_NOC_MODE_REG);
+                       pr_info("%s: ext_control: #%x\n", __func__, node_data->setting->ext_control);
                        writel(node_data->setting->ext_control, base + IMX_NOC_EXT_CTL_REG);
                        dev_dbg(dev, "%s: mode: 0x%x, prio: 0x%x, ext_control: 0x%x\n",
                                node_data->desc->name, node_data->setting->mode, prio,

---8<---

Signature · 4 lines
TQ-Systems GmbH | Mühlstraße 2, Gut Delling | 82229 Seefeld, Germany
Amtsgericht München, HRB 105018
Geschäftsführer: Detlef Schneider, Rüdiger Stahl, Stefan Schneider
http://www.tq-group.com/
#1 lore raw

Hello Alexander,

Quoted 11 lines · Alexander Stein

Hi,

while debugging another issue regarding suspend/resume I noticed that the imx interconnect driver locks up the system when writing into registers. This happens during resume of the 'g2' power domain. imx8m_blk_ctrl_power_on() will eventually call down to imx_icc_node_set(). Using the debug diff furthe down I get this output:

[   71.493062] imx_icc_node_set: node: VPU G2
[   71.497163] imx_icc_node_set: peak_bw: 1
[   71.501091] imx_icc_node_set: prio: 0x80000303
Quoted 8 lines

<freeze here>

Unfortunately there is absolutely no documentation regarding NOC in the reference manual, so no idea whats happening here.
This is on next-20230309 using imx8mp-tqma8mpql-mba8mpxl.dts with USB disabled (prevents suspend) and Ethernet-PHY-IRQ disabled (the issue I'm tackling).
Is there some idea what do in order to support resume on i.MX8MP?

Could it be that you don't have a fully featured 8MP, but a 8MPUL or 8MPL, which lacks VPUs? In that case, it's not enough to disable the VPU nodes in the DT, but you need to disable the VPU power domains as well.

Cheers,
Ahmad

Quoted 30 lines

Best regards,
Alexander

---8<---

drivers/interconnect/imx/imx.c +5 -1
index 979ed610f704..93ec79cc7c06 100644
@@ -44,13 +44,18 @@ static int imx_icc_node_set(struct icc_node *node)
        u32 prio;
        u64 freq;
 
+       pr_info("%s: node: %s\n", __func__, node->name);
+       pr_info("%s: peak_bw: %u\n", __func__, node->peak_bw);
        if (node_data->setting && node->peak_bw) {
                base = node_data->setting->reg + node_data->imx_provider->noc_base;
                if (node_data->setting->mode == IMX_NOC_MODE_FIXED) {
                        prio = node_data->setting->prio_level;
                        prio = PRIORITY_COMP_MARK | (prio << 8) | prio;
+                       pr_info("%s: prio: %#x\n", __func__, prio);
                        writel(prio, base + IMX_NOC_PRIO_REG);
+                       pr_info("%s: mode: %#x\n", __func__, node_data->setting->mode);
                        writel(node_data->setting->mode, base + IMX_NOC_MODE_REG);
+                       pr_info("%s: ext_control: #%x\n", __func__, node_data->setting->ext_control);
                        writel(node_data->setting->ext_control, base + IMX_NOC_EXT_CTL_REG);
                        dev_dbg(dev, "%s: mode: 0x%x, prio: 0x%x, ext_control: 0x%x\n",
                                node_data->desc->name, node_data->setting->mode, prio,
---8<---
Signature · 4 lines
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
#2 lore raw

Hi Ahmad,

Am Donnerstag, 9. März 2023, 16:03:26 CET schrieb Ahmad Fatoum:

Quoted 23 lines

Hello Alexander,

Quoted 11 lines · Alexander Stein

Hi,

while debugging another issue regarding suspend/resume I noticed that the imx interconnect driver locks up the system when writing into registers. This happens during resume of the 'g2' power domain. imx8m_blk_ctrl_power_on() will eventually call down to imx_icc_node_set(). Using the debug diff furthe down I get this output:

[   71.493062] imx_icc_node_set: node: VPU G2
[   71.497163] imx_icc_node_set: peak_bw: 1
[   71.501091] imx_icc_node_set: prio: 0x80000303
Quoted 8 lines

<freeze here>

Unfortunately there is absolutely no documentation regarding NOC in the reference manual, so no idea whats happening here.
This is on next-20230309 using imx8mp-tqma8mpql-mba8mpxl.dts with USB disabled (prevents suspend) and Ethernet-PHY-IRQ disabled (the issue I'm tackling).
Is there some idea what do in order to support resume on i.MX8MP?

Quoted 4 lines

Could it be that you don't have a fully featured 8MP, but a 8MPUL or 8MPL, which lacks VPUs? In that case, it's not enough to disable the VPU nodes in the DT, but you need to disable the VPU power domains as well.

That's reasonable to question this, but in this case this is a full-fledged 8MP. I just noticed that during boot those settings are done as well and there I get the (expected) output:

[   12.696371] imx_icc_node_set: node: VPU G2
[   12.700472] imx_icc_node_set: peak_bw: 1
[   12.704401] imx_icc_node_set: prio: 0x80000303
[   12.708850] imx_icc_node_set: mode: 0x0
[   12.712691] imx_icc_node_set: ext_control: #0
[   12.717053] imx_icc_node_set: node: NOC_VIDEO
[   12.721419] imx_icc_node_set: peak_bw: 1

So there is something wrong during resume. Both clk_ignore_unused pd_ignore_unused did not help.

Best regards,
Alexander

Quoted 48 lines

Cheers,
Ahmad

Quoted 45 lines

Best regards,
Alexander

---8<---

(quoted diff) +0 -0

b/drivers/interconnect/imx/imx.c index 979ed610f704..93ec79cc7c06 100644

drivers/interconnect/imx/imx.c +4 -0
@@ -44,13 +44,18 @@ static int imx_icc_node_set(struct icc_node *node)
 
        u32 prio;
        u64 freq;
 
+       pr_info("%s: node: %s\n", __func__, node->name);
+       pr_info("%s: peak_bw: %u\n", __func__, node->peak_bw);
 
        if (node_data->setting && node->peak_bw) {
 
                base = node_data->setting->reg +
                node_data->imx_provider->noc_base;
                if (node_data->setting->mode == IMX_NOC_MODE_FIXED) {
 
                        prio = node_data->setting->prio_level;
                        prio = PRIORITY_COMP_MARK | (prio << 8) | prio;
 
+                       pr_info("%s: prio: %#x\n", __func__, prio);
 
                        writel(prio, base + IMX_NOC_PRIO_REG);
 
+                       pr_info("%s: mode: %#x\n", __func__,

node_data->setting->mode);>

                        writel(node_data->setting->mode, base +
                        IMX_NOC_MODE_REG);

+ pr_info("%s: ext_control: #%x\n", __func__,
node_data->setting->ext_control);>

                        writel(node_data->setting->ext_control, base +
                        IMX_NOC_EXT_CTL_REG);
                        dev_dbg(dev, "%s: mode: 0x%x, prio: 0x%x,
                        ext_control: 0x%x\n",
                        
                                node_data->desc->name,
                                node_data->setting->mode, prio,

---8<---

Signature · 4 lines
TQ-Systems GmbH | Mühlstraße 2, Gut Delling | 82229 Seefeld, Germany
Amtsgericht München, HRB 105018
Geschäftsführer: Detlef Schneider, Rüdiger Stahl, Stefan Schneider
http://www.tq-group.com/
#3 lore raw
In reply to Alexander Stein
Quoted 12 lines · Alexander Stein

Hi,

while debugging another issue regarding suspend/resume I noticed that the imx interconnect driver locks up the system when writing into registers. This happens during resume of the 'g2' power domain. imx8m_blk_ctrl_power_on() will eventually call down to imx_icc_node_set(). Using the debug diff furthe down I get this output:

[   71.493062] imx_icc_node_set: node: VPU G2
[   71.497163] imx_icc_node_set: peak_bw: 1
[   71.501091] imx_icc_node_set: prio: 0x80000303
Quoted 8 lines

<freeze here>

Unfortunately there is absolutely no documentation regarding NOC in the reference manual, so no idea whats happening here.
This is on next-20230309 using imx8mp-tqma8mpql-mba8mpxl.dts with USB disabled (prevents suspend) and Ethernet-PHY-IRQ disabled (the issue I'm tackling).
Is there some idea what do in order to support resume on i.MX8MP?

I haven't played with suspend-resume myself, but there was a thread
[1] with discussions about fixing a race condition on the imx8mp interconnect.

adam

[1] - https://lore.kernel.org/lkml/[email protected]/

Quoted 31 lines

Best regards,
Alexander

---8<---

drivers/interconnect/imx/imx.c +5 -2
index 979ed610f704..93ec79cc7c06 100644
@@ -44,13 +44,18 @@ static int imx_icc_node_set(struct icc_node *node)
        u32 prio;
        u64 freq;
 
+       pr_info("%s: node: %s\n", __func__, node->name);
+       pr_info("%s: peak_bw: %u\n", __func__, node->peak_bw);
        if (node_data->setting && node->peak_bw) {
                base = node_data->setting->reg + node_data->imx_provider->noc_base;
                if (node_data->setting->mode == IMX_NOC_MODE_FIXED) {
                        prio = node_data->setting->prio_level;
                        prio = PRIORITY_COMP_MARK | (prio << 8) | prio;
+                       pr_info("%s: prio: %#x\n", __func__, prio);
                        writel(prio, base + IMX_NOC_PRIO_REG);
+                       pr_info("%s: mode: %#x\n", __func__, node_data->setting->mode);
                        writel(node_data->setting->mode, base + IMX_NOC_MODE_REG);
+                       pr_info("%s: ext_control: #%x\n", __func__, node_data->setting->ext_control);
                        writel(node_data->setting->ext_control, base + IMX_NOC_EXT_CTL_REG);
                        dev_dbg(dev, "%s: mode: 0x%x, prio: 0x%x, ext_control: 0x%x\n",
                                node_data->desc->name, node_data->setting->mode, prio,
---8<---
 
--
Quoted 6 lines

TQ-Systems GmbH | Mühlstraße 2, Gut Delling | 82229 Seefeld, Germany Amtsgericht München, HRB 105018
Geschäftsführer: Detlef Schneider, Rüdiger Stahl, Stefan Schneider
http://www.tq-group.com/

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

#4 lore raw

Hi Adam,

Am Montag, 13. März 2023, 20:17:55 CET schrieb Adam Ford:

Quoted 23 lines

On Thu, Mar 9, 2023 at 9:11 AM Alexander Stein

Quoted 11 lines · <[email protected]> wrote:

Hi,

while debugging another issue regarding suspend/resume I noticed that the imx interconnect driver locks up the system when writing into registers. This happens during resume of the 'g2' power domain. imx8m_blk_ctrl_power_on() will eventually call down to imx_icc_node_set(). Using the debug diff furthe down I get this output:

[   71.493062] imx_icc_node_set: node: VPU G2
[   71.497163] imx_icc_node_set: peak_bw: 1
[   71.501091] imx_icc_node_set: prio: 0x80000303
Quoted 8 lines

<freeze here>

Unfortunately there is absolutely no documentation regarding NOC in the reference manual, so no idea whats happening here.
This is on next-20230309 using imx8mp-tqma8mpql-mba8mpxl.dts with USB disabled (prevents suspend) and Ethernet-PHY-IRQ disabled (the issue I'm tackling).
Is there some idea what do in order to support resume on i.MX8MP?

Quoted 3 lines

I haven't played with suspend-resume myself, but there was a thread
[1] with discussions about fixing a race condition on the imx8mp interconnect.

Thanks for this link, but I think this is addressing a race condition during (un)registering. This works for me so far. I'm dealing with a freezing hardware write upon resume.

Best regards,
Alexander

Quoted 56 lines

adam

[1] -
https://lore.kernel.org/lkml/[email protected]
/

Quoted 51 lines

Best regards,
Alexander

---8<---

(quoted diff) +0 -0

b/drivers/interconnect/imx/imx.c index 979ed610f704..93ec79cc7c06 100644

drivers/interconnect/imx/imx.c +4 -0
@@ -44,13 +44,18 @@ static int imx_icc_node_set(struct icc_node *node)
 
        u32 prio;
        u64 freq;
 
+       pr_info("%s: node: %s\n", __func__, node->name);
+       pr_info("%s: peak_bw: %u\n", __func__, node->peak_bw);
 
        if (node_data->setting && node->peak_bw) {
 
                base = node_data->setting->reg +
                node_data->imx_provider->noc_base;
                if (node_data->setting->mode == IMX_NOC_MODE_FIXED) {
 
                        prio = node_data->setting->prio_level;
                        prio = PRIORITY_COMP_MARK | (prio << 8) | prio;
 
+                       pr_info("%s: prio: %#x\n", __func__, prio);
 
                        writel(prio, base + IMX_NOC_PRIO_REG);
 
+                       pr_info("%s: mode: %#x\n", __func__,

node_data->setting->mode);>

                        writel(node_data->setting->mode, base +
                        IMX_NOC_MODE_REG);

+ pr_info("%s: ext_control: #%x\n", __func__,
node_data->setting->ext_control);>

                        writel(node_data->setting->ext_control, base +
                        IMX_NOC_EXT_CTL_REG);
                        dev_dbg(dev, "%s: mode: 0x%x, prio: 0x%x,
                        ext_control: 0x%x\n",
                        
                                node_data->desc->name,
                                node_data->setting->mode, prio,

---8<---

--
TQ-Systems GmbH | Mühlstraße 2, Gut Delling | 82229 Seefeld, Germany Amtsgericht München, HRB 105018
Geschäftsführer: Detlef Schneider, Rüdiger Stahl, Stefan Schneider
http://www.tq-group.com/

Signature · 4 lines
TQ-Systems GmbH | Mühlstraße 2, Gut Delling | 82229 Seefeld, Germany
Amtsgericht München, HRB 105018
Geschäftsführer: Detlef Schneider, Rüdiger Stahl, Stefan Schneider
http://www.tq-group.com/