Skip to contentSkip to conversation

Tegra234 Memory interconnect support

PATCH Superseded Sumit Gupta 2 participants 17 messages Apr 24, 2023 – May 9, 2023

Subsystems (5)

v7 of 8 Superseded Latest is v8 →

lore
Share & tools
lore
#0 lore raw

v7 cover Tegra234 Memory interconnect support

Hi All,

Thank you for the review.
Requesting for ACK on the remaining patches and please consider for merging.

The patches are divided into below groups:

- Patch [6-7]: Memory Interconnect support in PCI (MC client) -- Added ACK from Lorenzo.

- Patch [3-5, 8]: Memory Interconnect support in CPUFREQ (MC client) -- 'Patch 3, 4 & 5': have ACK from Krzysztof.
-- 'Patch 8: Need ACK.

- Patch [1-2]: Memory Interconnect base support
-- 'Patch 2': has ACK from Krzysztof.
-- 'Patch 1': Need ACK.

Both the MC client patches are dependent on the 'Patch [1-2]'.

Thank you,
Sumit Gupta
============

This patch series adds memory interconnect support for Tegra234 SoC. It is used to dynamically scale DRAM Frequency as per the bandwidth requests from different Memory Controller (MC) clients.
MC Clients use ICC Framework's icc_set_bw() api to dynamically request for the DRAM bandwidth (BW). As per path, the request will be routed from MC to the EMC driver. MC driver passes the request info like the Client ID, type, and frequency request info to the BPMP-FW which will set the final DRAM freq considering all exisiting requests.

MC and EMC are the ICC providers. Nodes in path for a request will be:

     Client[1-n] -> MC -> EMC -> EMEM/DRAM

The patch series also adds interconnect support in below clients:
1) CPUFREQ driver for scaling bandwidth with CPU frequency. For that, add per cluster OPP table which will be used in the CPUFREQ driver by requesting the minimum BW respective to the given CPU frequency in the OPP table of it's cluster.
2) PCIE driver to request BW required for different modes.

---
v6[6] -> v7:
- removed 'Patch 6' from v6 as it has been applied in next by 'Viresh'.
- changed commit message of PCI patches as suggested by 'Lorenzo'.
- added ACK from 'Lorenzo' in both PCI patches.

v5[5] -> v6:
- split PCI patch into two 'patch 7 & 8' as suggested by 'Lorenzo'.
- added more info into the commit description of PCI patches.

v4[4] -> v5:
- dropped 'patch 1 & 2' from v4 which added "nvidia,bpmp" in MC node.
- save BPMP ref from EMC node into 'mc->bpmp' and use it in MC driver.
- added check for array bounds violation in pci suggested by Bjorn.
- moved DT patch having OPP table to the last 'Patch 8'.
- did multiple small changes and cleanup suggested by Krzysztof in v4.

v3[3] -> v4:
- dropped 'patch 1' from v3 which returns 'struct tegra_bpmp *'.
- added 'patch 1 & 2' to get bpmp ref using 'nvidia,bpmp' property.
- dropped 'patch 10 & 11' from v3 and added those changes in 'patch 3'.
- added static to prototype of 'tegra_cpufreq_init_cpufreq_table()' to fix the warning reported by 'kernel test robot'.

v2[2] -> v3:
- in 'patch 7', set 'icc_dram_bw_scaling' to false if set_opp call failed to avoid flooding of uart with 'Failed to set bw' messages.
- added 'patch 10' to handle if the bpmp-fw is old and not support bwmgr mrq.
- added 'patch 11' to fix interconnect registration in tegra186-emc. ref patch link in linux next:
[https://lore.kernel.org/all/[email protected]/]

v1[1] -> v2:
- moved BW setting to tegra234_mc_icc_set() from EMC driver.
- moved sw clients to the 'tegra_mc_clients' table.
- point 'node->data' to the entry within 'tegra_mc_clients'.
- removed 'struct tegra_icc_node' and get client info using 'node->data'.
- changed error handling in and around tegra_emc_interconnect_init().
- moved 'tegra-icc.h' from 'include/soc/tegra' to 'include/linux'.
- added interconnect support to PCIE driver in 'Patch 9'.
- merged 'Patch 9 & 10' from [1] to get num_channels and use.
- merged 'Patch 2 & 3' from [1] to add ISO and NISO clients.
- added 'Acked-by' of Krzysztof from 'Patch 05/10' of [1].
- Removed 'Patch 7' from [1] as that is merged now.

Sumit Gupta (8):
memory: tegra: add interconnect support for DRAM scaling in Tegra234
memory: tegra: add mc clients for Tegra234
memory: tegra: add software mc clients in Tegra234
dt-bindings: tegra: add icc ids for dummy MC clients
memory: tegra: make cpu cluster bw request a multiple of mc channels
PCI: tegra194: Fix possible array out of bounds access
PCI: tegra194: Add interconnect support in Tegra234
arm64: tegra: Add cpu OPP tables and interconnects property

 arch/arm64/boot/dts/nvidia/tegra234.dtsi   | 276 ++++++++++
 drivers/memory/tegra/mc.c                  |  24 +
 drivers/memory/tegra/mc.h                  |   1 +
 drivers/memory/tegra/tegra186-emc.c        | 130 +++++
 drivers/memory/tegra/tegra234.c            | 595 ++++++++++++++++++++-
 drivers/pci/controller/dwc/pcie-tegra194.c |  44 +-
 include/dt-bindings/memory/tegra234-mc.h   |   5 +
 include/linux/tegra-icc.h                  |  65 +++
 include/soc/tegra/mc.h                     |   8 +
 9 files changed, 1139 insertions(+), 9 deletions(-)
 create mode 100644 include/linux/tegra-icc.h

[1] https://lore.kernel.org/lkml/[email protected]/
[2] https://lore.kernel.org/lkml/[email protected]/
[3] https://lore.kernel.org/lkml/[email protected]/
[4] https://lore.kernel.org/lkml/[email protected]/
[5] https://lore.kernel.org/lkml/[email protected]/
[6] https://lore.kernel.org/lkml/[email protected]/

Signature · 1 line
2.17.1
#1 lore raw

PATCH v7 1/8 memory: tegra: add interconnect support for DRAM scaling in Tegra234

Add Interconnect framework support to dynamically set the DRAM bandwidth from different clients. Both the MC and EMC drivers are added as ICC providers. The path for any request is:
MC-Client[1-n] -> MC -> EMC -> EMEM/DRAM

MC client's request for bandwidth will go to the MC driver which passes the client request info like BPMP Client ID, Client type and the Bandwidth to the BPMP-FW. The final DRAM freq to achieve the requested bandwidth is set by the BPMP-FW based on the passed parameters.

---
 drivers/memory/tegra/mc.c           |   5 +
 drivers/memory/tegra/tegra186-emc.c | 130 ++++++++++++++++++++++++++
 drivers/memory/tegra/tegra234.c     | 138 +++++++++++++++++++++++++++-
 include/linux/tegra-icc.h           |  65 +++++++++++++
 include/soc/tegra/mc.h              |   7 ++
 5 files changed, 344 insertions(+), 1 deletion(-)
 create mode 100644 include/linux/tegra-icc.h
drivers/memory/tegra/mc.c +5 -0
index 9082b6c3763d..983455b1f98d 100644
@@ -15,6 +15,7 @@
 #include <linux/platform_device.h>
 #include <linux/slab.h>
 #include <linux/sort.h>
+#include <linux/tegra-icc.h>
 
 #include <soc/tegra/fuse.h>
 
@@ -792,6 +793,8 @@ static int tegra_mc_interconnect_setup(struct tegra_mc *mc)
 	mc->provider.data = &mc->provider;
 	mc->provider.set = mc->soc->icc_ops->set;
 	mc->provider.aggregate = mc->soc->icc_ops->aggregate;
+	mc->provider.get_bw = mc->soc->icc_ops->get_bw;
+	mc->provider.xlate = mc->soc->icc_ops->xlate;
 	mc->provider.xlate_extended = mc->soc->icc_ops->xlate_extended;
 
 	icc_provider_init(&mc->provider);
@@ -824,6 +827,8 @@ static int tegra_mc_interconnect_setup(struct tegra_mc *mc)
 		err = icc_link_create(node, TEGRA_ICC_MC);
 		if (err)
 			goto remove_nodes;
+
+		node->data = (struct tegra_mc_client *)&(mc->soc->clients[i]);
 	}
 
 	err = icc_provider_register(&mc->provider);
drivers/memory/tegra/tegra186-emc.c +130 -0

176 lines not on this page · raw message

drivers/memory/tegra/tegra234.c +137 -1

233 lines not on this page · raw message

include/linux/tegra-icc.h new file +65 -0

65 lines not on this page · raw message

include/soc/tegra/mc.h +7 -0
index 51a2263e1bc5..900d88b26fae 100644
@@ -13,6 +13,7 @@
 #include <linux/irq.h>
 #include <linux/reset-controller.h>
 #include <linux/types.h>
+#include <linux/tegra-icc.h>
 
 struct clk;
 struct device;
@@ -26,6 +27,8 @@ struct tegra_mc_timing {
 
 struct tegra_mc_client {
 	unsigned int id;
+	unsigned int bpmp_id;
+	enum tegra_icc_client_type type;
 	const char *name;
 	/*
 	 * For Tegra210 and earlier, this is the SWGROUP ID used for IOVA translations in the
@@ -166,8 +169,10 @@ struct tegra_mc_icc_ops {
 	int (*set)(struct icc_node *src, struct icc_node *dst);
 	int (*aggregate)(struct icc_node *node, u32 tag, u32 avg_bw,
 			 u32 peak_bw, u32 *agg_avg, u32 *agg_peak);
+	struct icc_node* (*xlate)(struct of_phandle_args *spec, void *data);
 	struct icc_node_data *(*xlate_extended)(struct of_phandle_args *spec,
 						void *data);
+	int (*get_bw)(struct icc_node *node, u32 *avg, u32 *peak);
 };
 
 struct tegra_mc_ops {
@@ -214,6 +219,7 @@ struct tegra_mc_soc {
 };
 
 struct tegra_mc {
+	struct tegra_bpmp *bpmp;
 	struct device *dev;
 	struct tegra_smmu *smmu;
 	struct gart_device *gart;
@@ -229,6 +235,7 @@ struct tegra_mc {
 	struct tegra_mc_timing *timings;
 	unsigned int num_timings;
 
+	bool bwmgr_mrq_supported;
 	struct reset_controller_dev reset;
 
 	struct icc_provider provider;
Signature · 1 line
2.17.1
#2 lore raw

Re: [Patch v7 1/8] memory: tegra: add interconnect support for DRAM scaling in Tegra234

Quoted 5 lines · Sumit Gupta

Add Interconnect framework support to dynamically set the DRAM bandwidth from different clients. Both the MC and EMC drivers are added as ICC providers. The path for any request is:
MC-Client[1-n] -> MC -> EMC -> EMEM/DRAM

Thank you for your patch. There is something to discuss/improve.

Quoted 75 lines
drivers/memory/tegra/tegra186-emc.c +66 -0
+static int tegra_emc_interconnect_init(struct tegra186_emc *emc)
+{
+	struct tegra_mc *mc = dev_get_drvdata(emc->dev->parent);
+	const struct tegra_mc_soc *soc = mc->soc;
+	struct icc_node *node;
+	int err;
+
+	emc->provider.dev = emc->dev;
+	emc->provider.set = tegra_emc_icc_set_bw;
+	emc->provider.data = &emc->provider;
+	emc->provider.aggregate = soc->icc_ops->aggregate;
+	emc->provider.xlate = tegra_emc_of_icc_xlate;
+	emc->provider.get_bw = tegra_emc_icc_get_init_bw;
+
+	icc_provider_init(&emc->provider);
+
+	/* create External Memory Controller node */
+	node = icc_node_create(TEGRA_ICC_EMC);
+	if (IS_ERR(node)) {
+		err = PTR_ERR(node);
+		goto err_msg;
+	}
+
+	node->name = "External Memory Controller";
+	icc_node_add(node, &emc->provider);
+
+	/* link External Memory Controller to External Memory (DRAM) */
+	err = icc_link_create(node, TEGRA_ICC_EMEM);
+	if (err)
+		goto remove_nodes;
+
+	/* create External Memory node */
+	node = icc_node_create(TEGRA_ICC_EMEM);
+	if (IS_ERR(node)) {
+		err = PTR_ERR(node);
+		goto remove_nodes;
+	}
+
+	node->name = "External Memory (DRAM)";
+	icc_node_add(node, &emc->provider);
+
+	err = icc_provider_register(&emc->provider);
+	if (err)
+		goto remove_nodes;
+
+	return 0;
+
+remove_nodes:
+	icc_nodes_remove(&emc->provider);
+err_msg:
+	dev_err(emc->dev, "failed to initialize ICC: %d\n", err);
+
+	return err;
+}
+
 static int tegra186_emc_probe(struct platform_device *pdev)
 {
+	struct tegra_mc *mc = dev_get_drvdata(pdev->dev.parent);
 	struct mrq_emc_dvfs_latency_response response;
 	struct tegra_bpmp_message msg;
 	struct tegra186_emc *emc;
@@ -236,6 +339,29 @@ static int tegra186_emc_probe(struct platform_device *pdev)
 	debugfs_create_file("max_rate", S_IRUGO | S_IWUSR, emc->debugfs.root,
 			    emc, &tegra186_emc_debug_max_rate_fops);
 
+	if (mc && mc->soc->icc_ops) {
+		/*
+		 * Initialize the ICC even if BPMP-FW doesn't support 'MRQ_BWMGR_INT'.
+		 * Use the flag 'mc->bwmgr_mrq_supported' within MC driver and return
+		 * EINVAL instead of passing the request to BPMP-FW later when the BW
+		 * request is made by client with 'icc_set_bw()' call.
+		 */
+		err = tegra_emc_interconnect_init(emc);
+		if (err)
+			goto put_bpmp;

drivers/memory/tegra/tegra186-emc.c static int tegra186_emc_probe(struct platform_device *pdev) · line 361

Quoted 10 lines
drivers/memory/tegra/tegra186-emc.c +10 -0
+
+		if (tegra_bpmp_mrq_is_supported(emc->bpmp, MRQ_BWMGR_INT)) {
+			mc->bwmgr_mrq_supported = true;
+
+			/*
+			 * MC driver probe can't get BPMP reference as it gets probed
+			 * earlier than BPMP. So, save the BPMP ref got from the EMC
+			 * DT node in the mc->bpmp and use it in MC's icc_set hook.
+			 */
+			mc->bpmp = emc->bpmp;

This (and ()) are called without any locking. You register first the interconnect, so set() callback can be used, right? Then set() could be called anytime between tegra_emc_interconnect_init() and assignment above. How do you synchronize these?

Best regards,
Krzysztof

#3 lore raw

Re: [Patch v7 1/8] memory: tegra: add interconnect support for DRAM scaling in Tegra234

Quoted 103 lines · Krzysztof Kozlowski

External email: Use caution opening links or attachments

Quoted 5 lines · Sumit Gupta

Add Interconnect framework support to dynamically set the DRAM bandwidth from different clients. Both the MC and EMC drivers are added as ICC providers. The path for any request is:
MC-Client[1-n] -> MC -> EMC -> EMEM/DRAM

Thank you for your patch. There is something to discuss/improve.

Quoted 75 lines
drivers/memory/tegra/tegra186-emc.c +66 -0
+static int tegra_emc_interconnect_init(struct tegra186_emc *emc)
+{
+     struct tegra_mc *mc = dev_get_drvdata(emc->dev->parent);
+     const struct tegra_mc_soc *soc = mc->soc;
+     struct icc_node *node;
+     int err;
+
+     emc->provider.dev = emc->dev;
+     emc->provider.set = tegra_emc_icc_set_bw;
+     emc->provider.data = &emc->provider;
+     emc->provider.aggregate = soc->icc_ops->aggregate;
+     emc->provider.xlate = tegra_emc_of_icc_xlate;
+     emc->provider.get_bw = tegra_emc_icc_get_init_bw;
+
+     icc_provider_init(&emc->provider);
+
+     /* create External Memory Controller node */
+     node = icc_node_create(TEGRA_ICC_EMC);
+     if (IS_ERR(node)) {
+             err = PTR_ERR(node);
+             goto err_msg;
+     }
+
+     node->name = "External Memory Controller";
+     icc_node_add(node, &emc->provider);
+
+     /* link External Memory Controller to External Memory (DRAM) */
+     err = icc_link_create(node, TEGRA_ICC_EMEM);
+     if (err)
+             goto remove_nodes;
+
+     /* create External Memory node */
+     node = icc_node_create(TEGRA_ICC_EMEM);
+     if (IS_ERR(node)) {
+             err = PTR_ERR(node);
+             goto remove_nodes;
+     }
+
+     node->name = "External Memory (DRAM)";
+     icc_node_add(node, &emc->provider);
+
+     err = icc_provider_register(&emc->provider);
+     if (err)
+             goto remove_nodes;
+
+     return 0;
+
+remove_nodes:
+     icc_nodes_remove(&emc->provider);
+err_msg:
+     dev_err(emc->dev, "failed to initialize ICC: %d\n", err);
+
+     return err;
+}
+
  static int tegra186_emc_probe(struct platform_device *pdev)
  {
+     struct tegra_mc *mc = dev_get_drvdata(pdev->dev.parent);
       struct mrq_emc_dvfs_latency_response response;
       struct tegra_bpmp_message msg;
       struct tegra186_emc *emc;
@@ -236,6 +339,29 @@ static int tegra186_emc_probe(struct platform_device *pdev)
       debugfs_create_file("max_rate", S_IRUGO | S_IWUSR, emc->debugfs.root,
                           emc, &tegra186_emc_debug_max_rate_fops);
 
+     if (mc && mc->soc->icc_ops) {
+             /*
+              * Initialize the ICC even if BPMP-FW doesn't support 'MRQ_BWMGR_INT'.
+              * Use the flag 'mc->bwmgr_mrq_supported' within MC driver and return
+              * EINVAL instead of passing the request to BPMP-FW later when the BW
+              * request is made by client with 'icc_set_bw()' call.
+              */
+             err = tegra_emc_interconnect_init(emc);
+             if (err)
+                     goto put_bpmp;

drivers/memory/tegra/tegra186-emc.c static int tegra186_emc_probe(struct platform_device *pdev) · line 361

Quoted 10 lines
drivers/memory/tegra/tegra186-emc.c +10 -0
+
+             if (tegra_bpmp_mrq_is_supported(emc->bpmp, MRQ_BWMGR_INT)) {
+                     mc->bwmgr_mrq_supported = true;
+
+                     /*
+                      * MC driver probe can't get BPMP reference as it gets probed
+                      * earlier than BPMP. So, save the BPMP ref got from the EMC
+                      * DT node in the mc->bpmp and use it in MC's icc_set hook.
+                      */
+                     mc->bpmp = emc->bpmp;

This (and ()) are called without any locking. You register first the interconnect, so set() callback can be used, right? Then set() could be called anytime between tegra_emc_interconnect_init() and assignment above. How do you synchronize these?

Quoted 3 lines

Best regards,
Krzysztof

Currently, the tegra234_mc_icc_set() has NULL check. So, it will give this error.

 if (!mc->bpmp) {
         dev_err(mc->dev, "BPMP reference NULL\n");
         return -ENOENT;

}

To fix, we can do the below change and swap the order. Please let me know if this sounds fine ?

 if (mc && mc->soc->icc_ops) {
         if (tegra_bpmp_mrq_is_supported(emc->bpmp, MRQ_BWMGR_INT)) {
                 mc->bwmgr_mrq_supported = true;
                 mc->bpmp = emc->bpmp;
         }

         err = tegra_emc_interconnect_init(emc);
         if (err)
                 goto put_bpmp;

}

Thank you,
Sumit Gupta

#4 lore raw

Re: [Patch v7 1/8] memory: tegra: add interconnect support for DRAM scaling in Tegra234

Quoted 131 lines · Sumit Gupta
Quoted 111 lines · Krzysztof Kozlowski

External email: Use caution opening links or attachments

Quoted 5 lines · Sumit Gupta

Add Interconnect framework support to dynamically set the DRAM bandwidth from different clients. Both the MC and EMC drivers are added as ICC providers. The path for any request is:
  MC-Client[1-n] -> MC -> EMC -> EMEM/DRAM

Thank you for your patch. There is something to discuss/improve.

Quoted 85 lines
drivers/memory/tegra/tegra186-emc.c +55 -0
+static int tegra_emc_interconnect_init(struct tegra186_emc *emc)
+{
+     struct tegra_mc *mc = dev_get_drvdata(emc->dev->parent);
+     const struct tegra_mc_soc *soc = mc->soc;
+     struct icc_node *node;
+     int err;
+
+     emc->provider.dev = emc->dev;
+     emc->provider.set = tegra_emc_icc_set_bw;
+     emc->provider.data = &emc->provider;
+     emc->provider.aggregate = soc->icc_ops->aggregate;
+     emc->provider.xlate = tegra_emc_of_icc_xlate;
+     emc->provider.get_bw = tegra_emc_icc_get_init_bw;
+
+     icc_provider_init(&emc->provider);
+
+     /* create External Memory Controller node */
+     node = icc_node_create(TEGRA_ICC_EMC);
+     if (IS_ERR(node)) {
+             err = PTR_ERR(node);
+             goto err_msg;
+     }
+
+     node->name = "External Memory Controller";
+     icc_node_add(node, &emc->provider);
+
+     /* link External Memory Controller to External Memory (DRAM) */
+     err = icc_link_create(node, TEGRA_ICC_EMEM);
+     if (err)
+             goto remove_nodes;
+
+     /* create External Memory node */
+     node = icc_node_create(TEGRA_ICC_EMEM);
+     if (IS_ERR(node)) {
+             err = PTR_ERR(node);
+             goto remove_nodes;
+     }
+
+     node->name = "External Memory (DRAM)";
+     icc_node_add(node, &emc->provider);
+
+     err = icc_provider_register(&emc->provider);
+     if (err)
+             goto remove_nodes;
+
+     return 0;
+
+remove_nodes:
+     icc_nodes_remove(&emc->provider);
+err_msg:
+     dev_err(emc->dev, "failed to initialize ICC: %d\n", err);
+
+     return err;
+}
+

  static int tegra186_emc_probe(struct platform_device *pdev) {
+     struct tegra_mc *mc = dev_get_drvdata(pdev->dev.parent); struct mrq_emc_dvfs_latency_response response; struct tegra_bpmp_message msg;
       struct tegra186_emc *emc;

drivers/memory/tegra/tegra186-emc.c +3 -0
@@ -236,6 +339,29 @@ static int tegra186_emc_probe(struct 
 platform_device *pdev)
        debugfs_create_file("max_rate", S_IRUGO | S_IWUSR, emc->debugfs.root,
                            emc, &tegra186_emc_debug_max_rate_fops);
 
+     if (mc && mc->soc->icc_ops) {
+             /*
+              * Initialize the ICC even if BPMP-FW doesn't support 

'MRQ_BWMGR_INT'.
+              * Use the flag 'mc->bwmgr_mrq_supported' within MC driver and return
+              * EINVAL instead of passing the request to BPMP-FW later when the BW
+              * request is made by client with 'icc_set_bw()' call. +              */
+             err = tegra_emc_interconnect_init(emc);
+             if (err)
+                     goto put_bpmp;
+
+             if (tegra_bpmp_mrq_is_supported(emc->bpmp, MRQ_BWMGR_INT)) {
+                     mc->bwmgr_mrq_supported = true;
+
+                     /*

Quoted 8 lines

+                      * MC driver probe can't get BPMP reference as it gets probed
+                      * earlier than BPMP. So, save the BPMP ref got from the EMC
+                      * DT node in the mc->bpmp and use it in MC's icc_set hook.
+                      */
+                     mc->bpmp = emc->bpmp;

This (and ()) are called without any locking. You register first the interconnect, so set() callback can be used, right? Then set() could be called anytime between tegra_emc_interconnect_init() and assignment above. How do you synchronize these?

Quoted 3 lines

Best regards,
Krzysztof

Currently, the tegra234_mc_icc_set() has NULL check. So, it will give this error.
 if (!mc->bpmp) {
         dev_err(mc->dev, "BPMP reference NULL\n");
         return -ENOENT;
 }

To fix, we can do the below change and swap the order. Please let me know if this sounds fine ?
 if (mc && mc->soc->icc_ops) {
         if (tegra_bpmp_mrq_is_supported(emc->bpmp, MRQ_BWMGR_INT)) { mc->bwmgr_mrq_supported = true;
                 mc->bpmp = emc->bpmp;
         }

Quoted 5 lines

         err = tegra_emc_interconnect_init(emc);
         if (err)
                 goto put_bpmp;

 }

Sorry, replied too soon. Missed setting "mc->bpmp" to NULL. Sharing new proposed change. Please let me know if this looks OK.

 if (mc && mc->soc->icc_ops) {
    if (tegra_bpmp_mrq_is_supported(emc->bpmp, MRQ_BWMGR_INT)) {
            mc->bwmgr_mrq_supported = true;
            mc->bpmp = emc->bpmp;
    }

    err = tegra_emc_interconnect_init(emc);
    if (err) {
            mc->bpmp = NULL;
            goto put_bpmp;
    }

}

Quoted 3 lines

Thank you,
Sumit Gupta

#5 lore raw

Re: [Patch v7 1/8] memory: tegra: add interconnect support for DRAM scaling in Tegra234

Quoted 118 lines · Sumit Gupta
Quoted 115 lines · Sumit Gupta
Quoted 112 lines · Krzysztof Kozlowski

External email: Use caution opening links or attachments

Quoted 5 lines · Sumit Gupta

Add Interconnect framework support to dynamically set the DRAM bandwidth from different clients. Both the MC and EMC drivers are added as ICC providers. The path for any request is:
  MC-Client[1-n] -> MC -> EMC -> EMEM/DRAM

Thank you for your patch. There is something to discuss/improve.

Quoted 86 lines
drivers/memory/tegra/tegra186-emc.c +55 -0
+static int tegra_emc_interconnect_init(struct tegra186_emc *emc)
+{
+     struct tegra_mc *mc = dev_get_drvdata(emc->dev->parent);
+     const struct tegra_mc_soc *soc = mc->soc;
+     struct icc_node *node;
+     int err;
+
+     emc->provider.dev = emc->dev;
+     emc->provider.set = tegra_emc_icc_set_bw;
+     emc->provider.data = &emc->provider;
+     emc->provider.aggregate = soc->icc_ops->aggregate;
+     emc->provider.xlate = tegra_emc_of_icc_xlate;
+     emc->provider.get_bw = tegra_emc_icc_get_init_bw;
+
+     icc_provider_init(&emc->provider);
+
+     /* create External Memory Controller node */
+     node = icc_node_create(TEGRA_ICC_EMC);
+     if (IS_ERR(node)) {
+             err = PTR_ERR(node);
+             goto err_msg;
+     }
+
+     node->name = "External Memory Controller";
+     icc_node_add(node, &emc->provider);
+
+     /* link External Memory Controller to External Memory (DRAM) */
+     err = icc_link_create(node, TEGRA_ICC_EMEM);
+     if (err)
+             goto remove_nodes;
+
+     /* create External Memory node */
+     node = icc_node_create(TEGRA_ICC_EMEM);
+     if (IS_ERR(node)) {
+             err = PTR_ERR(node);
+             goto remove_nodes;
+     }
+
+     node->name = "External Memory (DRAM)";
+     icc_node_add(node, &emc->provider);
+
+     err = icc_provider_register(&emc->provider);
+     if (err)
+             goto remove_nodes;
+
+     return 0;
+
+remove_nodes:
+     icc_nodes_remove(&emc->provider);
+err_msg:
+     dev_err(emc->dev, "failed to initialize ICC: %d\n", err);
+
+     return err;
+}
+

  static int tegra186_emc_probe(struct platform_device *pdev) {
+     struct tegra_mc *mc = dev_get_drvdata(pdev->dev.parent); struct mrq_emc_dvfs_latency_response response; struct tegra_bpmp_message msg;
       struct tegra186_emc *emc;

drivers/memory/tegra/tegra186-emc.c +3 -0
@@ -236,6 +339,29 @@ static int tegra186_emc_probe(struct 
 platform_device *pdev)
        debugfs_create_file("max_rate", S_IRUGO | S_IWUSR, 
 emc->debugfs.root,
                            emc, &tegra186_emc_debug_max_rate_fops);
 
+     if (mc && mc->soc->icc_ops) {
+             /*
+              * Initialize the ICC even if BPMP-FW doesn't support 

'MRQ_BWMGR_INT'.
+              * Use the flag 'mc->bwmgr_mrq_supported' within MC driver and return
+              * EINVAL instead of passing the request to BPMP-FW later when the BW
+              * request is made by client with 'icc_set_bw()' call. +              */
+             err = tegra_emc_interconnect_init(emc);
+             if (err)
+                     goto put_bpmp;
+
+             if (tegra_bpmp_mrq_is_supported(emc->bpmp, MRQ_BWMGR_INT)) {
+                     mc->bwmgr_mrq_supported = true;
+
+                     /*

Quoted 8 lines

+                      * MC driver probe can't get BPMP reference as it gets probed
+                      * earlier than BPMP. So, save the BPMP ref got from the EMC
+                      * DT node in the mc->bpmp and use it in MC's icc_set hook.
+                      */
+                     mc->bpmp = emc->bpmp;

This (and ()) are called without any locking. You register first the interconnect, so set() callback can be used, right? Then set() could be called anytime between tegra_emc_interconnect_init() and assignment above. How do you synchronize these?

Quoted 7 lines
Quoted 7 lines
Quoted 3 lines

Best regards,
Krzysztof

Currently, the tegra234_mc_icc_set() has NULL check. So, it will give this error.
 if (!mc->bpmp) {

How does it solve concurrent accesses and re-ordering of instructions by compiler or CPU?

Quoted 26 lines
Quoted 12 lines

         dev_err(mc->dev, "BPMP reference NULL\n"); return -ENOENT;
 }

To fix, we can do the below change and swap the order. Please let me know if this sounds fine ?
 if (mc && mc->soc->icc_ops) {
         if (tegra_bpmp_mrq_is_supported(emc->bpmp, MRQ_BWMGR_INT)) { mc->bwmgr_mrq_supported = true;
                 mc->bpmp = emc->bpmp;
         }

Quoted 5 lines

         err = tegra_emc_interconnect_init(emc);
         if (err)
                 goto put_bpmp;

 }

Sorry, replied too soon. Missed setting "mc->bpmp" to NULL. Sharing new proposed change. Please let me know if this looks OK.

  if (mc && mc->soc->icc_ops) {
     if (tegra_bpmp_mrq_is_supported(emc->bpmp, MRQ_BWMGR_INT)) {
             mc->bwmgr_mrq_supported = true;
             mc->bpmp = emc->bpmp;
     }
Quoted 4 lines
     err = tegra_emc_interconnect_init(emc);
     if (err) {
             mc->bpmp = NULL;
             goto put_bpmp;

Sorry, I didn't check the code. I assume you should do it...

Best regards,
Krzysztof

#6 lore raw

Re: [Patch v7 1/8] memory: tegra: add interconnect support for DRAM scaling in Tegra234

Quoted 126 lines · Krzysztof Kozlowski

External email: Use caution opening links or attachments

Quoted 122 lines · Sumit Gupta
Quoted 119 lines · Sumit Gupta
Quoted 115 lines · Krzysztof Kozlowski

External email: Use caution opening links or attachments

Quoted 5 lines · Sumit Gupta

Add Interconnect framework support to dynamically set the DRAM bandwidth from different clients. Both the MC and EMC drivers are added as ICC providers. The path for any request is:

   MC-Client[1-n] -> MC -> EMC -> EMEM/DRAM

Thank you for your patch. There is something to discuss/improve.

Quoted 92 lines
drivers/memory/tegra/tegra186-emc.c +59 -0
+static int tegra_emc_interconnect_init(struct tegra186_emc *emc)
+{
+     struct tegra_mc *mc = dev_get_drvdata(emc->dev->parent);
+     const struct tegra_mc_soc *soc = mc->soc;
+     struct icc_node *node;
+     int err;
+
+     emc->provider.dev = emc->dev;
+     emc->provider.set = tegra_emc_icc_set_bw;
+     emc->provider.data = &emc->provider;
+     emc->provider.aggregate = soc->icc_ops->aggregate;
+     emc->provider.xlate = tegra_emc_of_icc_xlate;
+     emc->provider.get_bw = tegra_emc_icc_get_init_bw;
+
+     icc_provider_init(&emc->provider);
+
+     /* create External Memory Controller node */
+     node = icc_node_create(TEGRA_ICC_EMC);
+     if (IS_ERR(node)) {
+             err = PTR_ERR(node);
+             goto err_msg;
+     }
+
+     node->name = "External Memory Controller";
+     icc_node_add(node, &emc->provider);
+
+     /* link External Memory Controller to External Memory (DRAM) */
+     err = icc_link_create(node, TEGRA_ICC_EMEM);
+     if (err)
+             goto remove_nodes;
+
+     /* create External Memory node */
+     node = icc_node_create(TEGRA_ICC_EMEM);
+     if (IS_ERR(node)) {
+             err = PTR_ERR(node);
+             goto remove_nodes;
+     }
+
+     node->name = "External Memory (DRAM)";
+     icc_node_add(node, &emc->provider);
+
+     err = icc_provider_register(&emc->provider);
+     if (err)
+             goto remove_nodes;
+
+     return 0;
+
+remove_nodes:
+     icc_nodes_remove(&emc->provider);
+err_msg:
+     dev_err(emc->dev, "failed to initialize ICC: %d\n", err);
+
+     return err;
+}
+
   static int tegra186_emc_probe(struct platform_device *pdev)
   {
+     struct tegra_mc *mc = dev_get_drvdata(pdev->dev.parent);
        struct mrq_emc_dvfs_latency_response response;
        struct tegra_bpmp_message msg;
        struct tegra186_emc *emc;
@@ -236,6 +339,29 @@ static int tegra186_emc_probe(struct
 platform_device *pdev)
        debugfs_create_file("max_rate", S_IRUGO | S_IWUSR,
 emc->debugfs.root,
                            emc, &tegra186_emc_debug_max_rate_fops);
 
+     if (mc && mc->soc->icc_ops) {
+             /*
+              * Initialize the ICC even if BPMP-FW doesn't support

'MRQ_BWMGR_INT'.
+ * Use the flag 'mc->bwmgr_mrq_supported' within MC
driver and return
+ * EINVAL instead of passing the request to BPMP-FW
later when the BW

drivers/memory/tegra/tegra186-emc.c +7 -0
+              * request is made by client with 'icc_set_bw()' call.
+              */
+             err = tegra_emc_interconnect_init(emc);
+             if (err)
+                     goto put_bpmp;
+
+             if (tegra_bpmp_mrq_is_supported(emc->bpmp,

MRQ_BWMGR_INT)) {

drivers/memory/tegra/tegra186-emc.c +4 -0
+                     mc->bwmgr_mrq_supported = true;
+
+                     /*
+                      * MC driver probe can't get BPMP reference as

it gets probed
+ * earlier than BPMP. So, save the BPMP ref got
from the EMC
+ * DT node in the mc->bpmp and use it in MC's
icc_set hook.

Review

Quoted 2 lines
(quoted diff) +2 -0
+                      */
+                     mc->bpmp = emc->bpmp;

This (and ()) are called without any locking. You register first the interconnect, so set() callback can be used, right? Then set() could be called anytime between tegra_emc_interconnect_init() and assignment above. How do you synchronize these?

Best regards,
Krzysztof

Quoted 7 lines
Quoted 3 lines
Quoted 3 lines

Currently, the tegra234_mc_icc_set() has NULL check. So, it will give this error.
if (!mc->bpmp) {

How does it solve concurrent accesses and re-ordering of instructions by compiler or CPU?

Now, the "mc->bpmp" is set before tegra_emc_interconnect_init(). So, until the EMC interconnect initializes, set() won't be called as the devm_of_icc_get() call will fail.

Added the change in v8.

Thank you,
Sumit Gupta

Quoted 36 lines
Quoted 26 lines
Quoted 16 lines
           dev_err(mc->dev, "BPMP reference NULL\n");
           return -ENOENT;

}

To fix, we can do the below change and swap the order. Please let me know if this sounds fine ?

   if (mc && mc->soc->icc_ops) {
           if (tegra_bpmp_mrq_is_supported(emc->bpmp, MRQ_BWMGR_INT)) {
                   mc->bwmgr_mrq_supported = true;
                   mc->bpmp = emc->bpmp;
           }

           err = tegra_emc_interconnect_init(emc);
           if (err)
                   goto put_bpmp;
Quoted 1 line
   }

Sorry, replied too soon. Missed setting "mc->bpmp" to NULL. Sharing new proposed change. Please let me know if this looks OK.

   if (mc && mc->soc->icc_ops) {
      if (tegra_bpmp_mrq_is_supported(emc->bpmp, MRQ_BWMGR_INT)) {
              mc->bwmgr_mrq_supported = true;
              mc->bpmp = emc->bpmp;
      }
Quoted 4 lines
      err = tegra_emc_interconnect_init(emc);
      if (err) {
              mc->bpmp = NULL;
              goto put_bpmp;

Sorry, I didn't check the code. I assume you should do it...

Best regards,
Krzysztof

#7 lore raw

Re: [Patch v7 1/8] memory: tegra: add interconnect support for DRAM scaling in Tegra234

Quoted 23 lines · Sumit Gupta
Quoted 23 lines
Quoted 22 lines
Quoted 22 lines
Quoted 10 lines

Review

Quoted 9 lines
(quoted diff) +2 -0
+                     /*
+                      * MC driver probe can't get BPMP reference as

it gets probed
+ * earlier than BPMP. So, save the BPMP ref got
from the EMC
+ * DT node in the mc->bpmp and use it in MC's
icc_set hook.

(quoted diff) +2 -0
+                      */
+                     mc->bpmp = emc->bpmp;
Quoted 8 lines

This (and ()) are called without any locking. You register first the interconnect, so set() callback can be used, right? Then set() could be called anytime between tegra_emc_interconnect_init() and assignment above. How do you synchronize these?

Best regards,
Krzysztof

Currently, the tegra234_mc_icc_set() has NULL check. So, it will give this error.
if (!mc->bpmp) {

Quoted 7 lines
Quoted 3 lines

How does it solve concurrent accesses and re-ordering of instructions by compiler or CPU?

Now, the "mc->bpmp" is set before tegra_emc_interconnect_init(). So, until the EMC interconnect initializes, set() won't be called as the devm_of_icc_get() call will fail.

What if compiler puts "mc->bpmp" assignment after
tegra_emc_interconnect_init()?

What if CPU executes above assignment also after
tegra_emc_interconnect_init()?

Considering amount of code inside tegra_emc_interconnect_init() second case is rather unlikely, but first possible, right?

Best regards,
Krzysztof

#8 lore raw

Re: [Patch v7 1/8] memory: tegra: add interconnect support for DRAM scaling in Tegra234

Quoted 38 lines · Krzysztof Kozlowski

External email: Use caution opening links or attachments

Quoted 23 lines · Sumit Gupta
Quoted 23 lines
Quoted 22 lines
Quoted 22 lines
Quoted 10 lines

Review

Quoted 9 lines
(quoted diff) +2 -0
+                     /*
+                      * MC driver probe can't get BPMP reference as

it gets probed
+ * earlier than BPMP. So, save the BPMP ref got
from the EMC
+ * DT node in the mc->bpmp and use it in MC's
icc_set hook.

(quoted diff) +2 -0
+                      */
+                     mc->bpmp = emc->bpmp;
Quoted 8 lines

This (and ()) are called without any locking. You register first the interconnect, so set() callback can be used, right? Then set() could be called anytime between tegra_emc_interconnect_init() and assignment above. How do you synchronize these?

Best regards,
Krzysztof

Currently, the tegra234_mc_icc_set() has NULL check. So, it will give this error.

    if (!mc->bpmp) {
Quoted 7 lines
Quoted 3 lines

How does it solve concurrent accesses and re-ordering of instructions by compiler or CPU?

Now, the "mc->bpmp" is set before tegra_emc_interconnect_init(). So, until the EMC interconnect initializes, set() won't be called as the devm_of_icc_get() call will fail.

What if compiler puts "mc->bpmp" assignment after
tegra_emc_interconnect_init()?

Quoted 9 lines

What if CPU executes above assignment also after
tegra_emc_interconnect_init()?

Considering amount of code inside tegra_emc_interconnect_init() second case is rather unlikely, but first possible, right?

Best regards,
Krzysztof

Yes, for protection from compiler re-ordering I can add barrier() after the assignment.

  mc->bpmp = emc->bpmp;
  barrier();

Thank you,
Sumit Gupta

#9 lore raw

PATCH v7 2/8 memory: tegra: add mc clients for Tegra234

In reply to Sumit Gupta

Add few Isochronous (ISO) and Non-ISO MC clients. ISO clients have guaranteed bandwidth requirement. PCIE clients added to the mc_clients table represent each controller in Tegra234.

---
 drivers/memory/tegra/tegra234.c | 420 ++++++++++++++++++++++++++++++++
 1 file changed, 420 insertions(+)
drivers/memory/tegra/tegra234.c +420 -0
index 56d911926d54..a4ea181e1090 100644
@@ -14,6 +14,30 @@
 
 static const struct tegra_mc_client tegra234_mc_clients[] = {
 	{
+		.id = TEGRA234_MEMORY_CLIENT_HDAR,
+		.name = "hdar",
+		.bpmp_id = TEGRA_ICC_BPMP_HDA,
+		.type = TEGRA_ICC_ISO_AUDIO,
+		.sid = TEGRA234_SID_HDA,
+		.regs = {
+			.sid = {
+				.override = 0xa8,
+				.security = 0xac,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_HDAW,
+		.name = "hdaw",
+		.bpmp_id = TEGRA_ICC_BPMP_HDA,
+		.type = TEGRA_ICC_ISO_AUDIO,
+		.sid = TEGRA234_SID_HDA,
+		.regs = {
+			.sid = {
+				.override = 0x1a8,
+				.security = 0x1ac,
+			},
+		},
+	}, {
 		.id = TEGRA234_MEMORY_CLIENT_MGBEARD,
 		.name = "mgbeard",
 		.bpmp_id = TEGRA_ICC_BPMP_EQOS,
@@ -133,6 +157,90 @@ static const struct tegra_mc_client tegra234_mc_clients[] = {
 				.security = 0x33c,
 			},
 		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_VI2W,
+		.name = "vi2w",
+		.bpmp_id = TEGRA_ICC_BPMP_VI2,
+		.type = TEGRA_ICC_ISO_VI,
+		.sid = TEGRA234_SID_ISO_VI2,
+		.regs = {
+			.sid = {
+				.override = 0x380,
+				.security = 0x384,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_VI2FALR,
+		.name = "vi2falr",
+		.bpmp_id = TEGRA_ICC_BPMP_VI2FAL,
+		.type = TEGRA_ICC_ISO_VIFAL,
+		.sid = TEGRA234_SID_ISO_VI2FALC,
+		.regs = {
+			.sid = {
+				.override = 0x388,
+				.security = 0x38c,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_VI2FALW,
+		.name = "vi2falw",
+		.bpmp_id = TEGRA_ICC_BPMP_VI2FAL,
+		.type = TEGRA_ICC_ISO_VIFAL,
+		.sid = TEGRA234_SID_ISO_VI2FALC,
+		.regs = {
+			.sid = {
+				.override = 0x3e0,
+				.security = 0x3e4,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_APER,
+		.name = "aper",
+		.bpmp_id = TEGRA_ICC_BPMP_APE,
+		.type = TEGRA_ICC_ISO_AUDIO,
+		.sid = TEGRA234_SID_APE,
+		.regs = {
+			.sid = {
+				.override = 0x3d0,
+				.security = 0x3d4,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_APEW,
+		.name = "apew",
+		.bpmp_id = TEGRA_ICC_BPMP_APE,
+		.type = TEGRA_ICC_ISO_AUDIO,
+		.sid = TEGRA234_SID_APE,
+		.regs = {
+			.sid = {
+				.override = 0x3d8,
+				.security = 0x3dc,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_NVDISPLAYR,
+		.name = "nvdisplayr",
+		.bpmp_id = TEGRA_ICC_BPMP_DISPLAY,
+		.type = TEGRA_ICC_ISO_DISPLAY,
+		.sid = TEGRA234_SID_ISO_NVDISPLAY,
+		.regs = {
+			.sid = {
+				.override = 0x490,
+				.security = 0x494,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_NVDISPLAYR1,
+		.name = "nvdisplayr1",
+		.bpmp_id = TEGRA_ICC_BPMP_DISPLAY,
+		.type = TEGRA_ICC_ISO_DISPLAY,
+		.sid = TEGRA234_SID_ISO_NVDISPLAY,
+		.regs = {
+			.sid = {
+				.override = 0x508,
+				.security = 0x50c,
+			},
+		},
 	}, {
 		.id = TEGRA234_MEMORY_CLIENT_BPMPR,
 		.name = "bpmpr",
@@ -357,6 +465,318 @@ static const struct tegra_mc_client tegra234_mc_clients[] = {
 				.security = 0x37c,
 			},
 		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_PCIE0R,
+		.name = "pcie0r",
+		.bpmp_id = TEGRA_ICC_BPMP_PCIE_0,
+		.type = TEGRA_ICC_NISO,
+		.sid = TEGRA234_SID_PCIE0,
+		.regs = {
+			.sid = {
+				.override = 0x6c0,
+				.security = 0x6c4,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_PCIE0W,
+		.name = "pcie0w",
+		.bpmp_id = TEGRA_ICC_BPMP_PCIE_0,
+		.type = TEGRA_ICC_NISO,
+		.sid = TEGRA234_SID_PCIE0,
+		.regs = {
+			.sid = {
+				.override = 0x6c8,
+				.security = 0x6cc,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_PCIE1R,
+		.name = "pcie1r",
+		.bpmp_id = TEGRA_ICC_BPMP_PCIE_1,
+		.type = TEGRA_ICC_NISO,
+		.sid = TEGRA234_SID_PCIE1,
+		.regs = {
+			.sid = {
+				.override = 0x6d0,
+				.security = 0x6d4,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_PCIE1W,
+		.name = "pcie1w",
+		.bpmp_id = TEGRA_ICC_BPMP_PCIE_1,
+		.type = TEGRA_ICC_NISO,
+		.sid = TEGRA234_SID_PCIE1,
+		.regs = {
+			.sid = {
+				.override = 0x6d8,
+				.security = 0x6dc,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_PCIE2AR,
+		.name = "pcie2ar",
+		.bpmp_id = TEGRA_ICC_BPMP_PCIE_2,
+		.type = TEGRA_ICC_NISO,
+		.sid = TEGRA234_SID_PCIE2,
+		.regs = {
+			.sid = {
+				.override = 0x6e0,
+				.security = 0x6e4,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_PCIE2AW,
+		.name = "pcie2aw",
+		.bpmp_id = TEGRA_ICC_BPMP_PCIE_2,
+		.type = TEGRA_ICC_NISO,
+		.sid = TEGRA234_SID_PCIE2,
+		.regs = {
+			.sid = {
+				.override = 0x6e8,
+				.security = 0x6ec,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_PCIE3R,
+		.name = "pcie3r",
+		.bpmp_id = TEGRA_ICC_BPMP_PCIE_3,
+		.type = TEGRA_ICC_NISO,
+		.sid = TEGRA234_SID_PCIE3,
+		.regs = {
+			.sid = {
+				.override = 0x6f0,
+				.security = 0x6f4,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_PCIE3W,
+		.name = "pcie3w",
+		.bpmp_id = TEGRA_ICC_BPMP_PCIE_3,
+		.type = TEGRA_ICC_NISO,
+		.sid = TEGRA234_SID_PCIE3,
+		.regs = {
+			.sid = {
+				.override = 0x6f8,
+				.security = 0x6fc,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_PCIE4R,
+		.name = "pcie4r",
+		.bpmp_id = TEGRA_ICC_BPMP_PCIE_4,
+		.type = TEGRA_ICC_NISO,
+		.sid = TEGRA234_SID_PCIE4,
+		.regs = {
+			.sid = {
+				.override = 0x700,
+				.security = 0x704,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_PCIE4W,
+		.name = "pcie4w",
+		.bpmp_id = TEGRA_ICC_BPMP_PCIE_4,
+		.type = TEGRA_ICC_NISO,
+		.sid = TEGRA234_SID_PCIE4,
+		.regs = {
+			.sid = {
+				.override = 0x708,
+				.security = 0x70c,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_PCIE5R,
+		.name = "pcie5r",
+		.bpmp_id = TEGRA_ICC_BPMP_PCIE_5,
+		.type = TEGRA_ICC_NISO,
+		.sid = TEGRA234_SID_PCIE5,
+		.regs = {
+			.sid = {
+				.override = 0x710,
+				.security = 0x714,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_PCIE5W,
+		.name = "pcie5w",
+		.bpmp_id = TEGRA_ICC_BPMP_PCIE_5,
+		.type = TEGRA_ICC_NISO,
+		.sid = TEGRA234_SID_PCIE5,
+		.regs = {
+			.sid = {
+				.override = 0x718,
+				.security = 0x71c,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_PCIE5R1,
+		.name = "pcie5r1",
+		.bpmp_id = TEGRA_ICC_BPMP_PCIE_5,
+		.type = TEGRA_ICC_NISO,
+		.sid = TEGRA234_SID_PCIE5,
+		.regs = {
+			.sid = {
+				.override = 0x778,
+				.security = 0x77c,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_PCIE6AR,
+		.name = "pcie6ar",
+		.bpmp_id = TEGRA_ICC_BPMP_PCIE_6,
+		.type = TEGRA_ICC_NISO,
+		.sid = TEGRA234_SID_PCIE6,
+		.regs = {
+			.sid = {
+				.override = 0x140,
+				.security = 0x144,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_PCIE6AW,
+		.name = "pcie6aw",
+		.bpmp_id = TEGRA_ICC_BPMP_PCIE_6,
+		.type = TEGRA_ICC_NISO,
+		.sid = TEGRA234_SID_PCIE6,
+		.regs = {
+			.sid = {
+				.override = 0x148,
+				.security = 0x14c,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_PCIE6AR1,
+		.name = "pcie6ar1",
+		.bpmp_id = TEGRA_ICC_BPMP_PCIE_6,
+		.type = TEGRA_ICC_NISO,
+		.sid = TEGRA234_SID_PCIE6,
+		.regs = {
+			.sid = {
+				.override = 0x1e8,
+				.security = 0x1ec,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_PCIE7AR,
+		.name = "pcie7ar",
+		.bpmp_id = TEGRA_ICC_BPMP_PCIE_7,
+		.type = TEGRA_ICC_NISO,
+		.sid = TEGRA234_SID_PCIE7,
+		.regs = {
+			.sid = {
+				.override = 0x150,
+				.security = 0x154,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_PCIE7AW,
+		.name = "pcie7aw",
+		.bpmp_id = TEGRA_ICC_BPMP_PCIE_7,
+		.type = TEGRA_ICC_NISO,
+		.sid = TEGRA234_SID_PCIE7,
+		.regs = {
+			.sid = {
+				.override = 0x180,
+				.security = 0x184,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_PCIE7AR1,
+		.name = "pcie7ar1",
+		.bpmp_id = TEGRA_ICC_BPMP_PCIE_7,
+		.type = TEGRA_ICC_NISO,
+		.sid = TEGRA234_SID_PCIE7,
+		.regs = {
+			.sid = {
+				.override = 0x248,
+				.security = 0x24c,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_PCIE8AR,
+		.name = "pcie8ar",
+		.bpmp_id = TEGRA_ICC_BPMP_PCIE_8,
+		.type = TEGRA_ICC_NISO,
+		.sid = TEGRA234_SID_PCIE8,
+		.regs = {
+			.sid = {
+				.override = 0x190,
+				.security = 0x194,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_PCIE8AW,
+		.name = "pcie8aw",
+		.bpmp_id = TEGRA_ICC_BPMP_PCIE_8,
+		.type = TEGRA_ICC_NISO,
+		.sid = TEGRA234_SID_PCIE8,
+		.regs = {
+			.sid = {
+				.override = 0x1d8,
+				.security = 0x1dc,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_PCIE9AR,
+		.name = "pcie9ar",
+		.bpmp_id = TEGRA_ICC_BPMP_PCIE_9,
+		.type = TEGRA_ICC_NISO,
+		.sid = TEGRA234_SID_PCIE9,
+		.regs = {
+			.sid = {
+				.override = 0x1e0,
+				.security = 0x1e4,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_PCIE9AW,
+		.name = "pcie9aw",
+		.bpmp_id = TEGRA_ICC_BPMP_PCIE_9,
+		.type = TEGRA_ICC_NISO,
+		.sid = TEGRA234_SID_PCIE9,
+		.regs = {
+			.sid = {
+				.override = 0x1f0,
+				.security = 0x1f4,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_PCIE10AR,
+		.name = "pcie10ar",
+		.bpmp_id = TEGRA_ICC_BPMP_PCIE_10,
+		.type = TEGRA_ICC_NISO,
+		.sid = TEGRA234_SID_PCIE10,
+		.regs = {
+			.sid = {
+				.override = 0x1f8,
+				.security = 0x1fc,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_PCIE10AW,
+		.name = "pcie10aw",
+		.bpmp_id = TEGRA_ICC_BPMP_PCIE_10,
+		.type = TEGRA_ICC_NISO,
+		.sid = TEGRA234_SID_PCIE10,
+		.regs = {
+			.sid = {
+				.override = 0x200,
+				.security = 0x204,
+			},
+		},
+	}, {
+		.id = TEGRA234_MEMORY_CLIENT_PCIE10AR1,
+		.name = "pcie10ar1",
+		.bpmp_id = TEGRA_ICC_BPMP_PCIE_10,
+		.type = TEGRA_ICC_NISO,
+		.sid = TEGRA234_SID_PCIE10,
+		.regs = {
+			.sid = {
+				.override = 0x240,
+				.security = 0x244,
+			},
+		},
 	},
 };
 
Signature · 1 line
2.17.1
#10 lore raw

PATCH v7 3/8 memory: tegra: add software mc clients in Tegra234

In reply to Sumit Gupta

Add dummy Memory Controller clients to represent CPU clusters. They will be used by the CPUFREQ driver to scale DRAM FREQ with the CPU FREQ.

---
 drivers/memory/tegra/tegra234.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
drivers/memory/tegra/tegra234.c +15 -0
index a4ea181e1090..0fa68862749f 100644
@@ -777,6 +777,21 @@ static const struct tegra_mc_client tegra234_mc_clients[] = {
 				.security = 0x244,
 			},
 		},
+	}, {
+		.id = TEGRA_ICC_MC_CPU_CLUSTER0,
+		.name = "sw_cluster0",
+		.bpmp_id = TEGRA_ICC_BPMP_CPU_CLUSTER0,
+		.type = TEGRA_ICC_NISO,
+	}, {
+		.id = TEGRA_ICC_MC_CPU_CLUSTER1,
+		.name = "sw_cluster1",
+		.bpmp_id = TEGRA_ICC_BPMP_CPU_CLUSTER1,
+		.type = TEGRA_ICC_NISO,
+	}, {
+		.id = TEGRA_ICC_MC_CPU_CLUSTER2,
+		.name = "sw_cluster2",
+		.bpmp_id = TEGRA_ICC_BPMP_CPU_CLUSTER2,
+		.type = TEGRA_ICC_NISO,
 	},
 };
 
Signature · 1 line
2.17.1
#11 lore raw

PATCH v7 4/8 dt-bindings: tegra: add icc ids for dummy MC clients

In reply to Sumit Gupta

Add ICC id's for dummy software clients representing CCPLEX clusters.

---
 include/dt-bindings/memory/tegra234-mc.h | 5 +++++
 1 file changed, 5 insertions(+)
include/dt-bindings/memory/tegra234-mc.h +5 -0
index 347e55e89a2a..6e60d55491b3 100644
@@ -536,4 +536,9 @@
 #define TEGRA234_MEMORY_CLIENT_NVJPG1SRD 0x123
 #define TEGRA234_MEMORY_CLIENT_NVJPG1SWR 0x124
 
+/* ICC ID's for dummy MC clients used to represent CPU Clusters */
+#define TEGRA_ICC_MC_CPU_CLUSTER0       1003
+#define TEGRA_ICC_MC_CPU_CLUSTER1       1004
+#define TEGRA_ICC_MC_CPU_CLUSTER2       1005
+
 #endif
Signature · 1 line
2.17.1
#12 lore raw

PATCH v7 5/8 memory: tegra: make cpu cluster bw request a multiple of mc channels

In reply to Sumit Gupta

Make CPU cluster's bandwidth (BW) request a multiple of MC channels. CPU OPP tables have BW info per MC channel. But, the actual BW depends on the number of MC channels which can change as per the boot config. Get the number of MC channels which are actually enabled in current boot configuration and multiply the BW request from a CPU cluster with the number of enabled MC channels. This is not required to be done for other MC clients.

---
 drivers/memory/tegra/mc.c       | 19 +++++++++++++++++++
 drivers/memory/tegra/mc.h       |  1 +
 drivers/memory/tegra/tegra234.c | 24 +++++++++++++++++++++++-
 include/soc/tegra/mc.h          |  1 +
 4 files changed, 44 insertions(+), 1 deletion(-)
drivers/memory/tegra/mc.c +19 -0
index 983455b1f98d..4a750da1c12a 100644
@@ -843,6 +843,23 @@ static int tegra_mc_interconnect_setup(struct tegra_mc *mc)
 	return err;
 }
 
+static void tegra_mc_num_channel_enabled(struct tegra_mc *mc)
+{
+	unsigned int i;
+	u32 value;
+
+	value = mc_ch_readl(mc, 0, MC_EMEM_ADR_CFG_CHANNEL_ENABLE);
+	if (value <= 0) {
+		mc->num_channels = mc->soc->num_channels;
+		return;
+	}
+
+	for (i = 0; i < 32; i++) {
+		if (value & BIT(i))
+			mc->num_channels++;
+	}
+}
+
 static int tegra_mc_probe(struct platform_device *pdev)
 {
 	struct tegra_mc *mc;
@@ -881,6 +898,8 @@ static int tegra_mc_probe(struct platform_device *pdev)
 			return err;
 	}
 
+	tegra_mc_num_channel_enabled(mc);
+
 	if (mc->soc->ops && mc->soc->ops->handle_irq) {
 		mc->irq = platform_get_irq(pdev, 0);
 		if (mc->irq < 0)
drivers/memory/tegra/mc.h +1 -0
index bc01586b6560..c3f6655bec60 100644
@@ -53,6 +53,7 @@
 #define MC_ERR_ROUTE_SANITY_ADR				0x9c4
 #define MC_ERR_GENERALIZED_CARVEOUT_STATUS		0xc00
 #define MC_ERR_GENERALIZED_CARVEOUT_ADR			0xc04
+#define MC_EMEM_ADR_CFG_CHANNEL_ENABLE			0xdf8
 #define MC_GLOBAL_INTSTATUS				0xf24
 #define MC_ERR_ADR_HI					0x11fc
 
drivers/memory/tegra/tegra234.c +23 -1
index 0fa68862749f..8e873a7bc34f 100644
@@ -867,6 +867,28 @@ static int tegra234_mc_icc_set(struct icc_node *src, struct icc_node *dst)
 	return ret;
 }
 
+static int tegra234_mc_icc_aggregate(struct icc_node *node, u32 tag, u32 avg_bw,
+				     u32 peak_bw, u32 *agg_avg, u32 *agg_peak)
+{
+	struct icc_provider *p = node->provider;
+	struct tegra_mc *mc = icc_provider_to_tegra_mc(p);
+
+	if (!mc->bwmgr_mrq_supported)
+		return -EINVAL;
+
+	if (node->id == TEGRA_ICC_MC_CPU_CLUSTER0 ||
+	    node->id == TEGRA_ICC_MC_CPU_CLUSTER1 ||
+	    node->id == TEGRA_ICC_MC_CPU_CLUSTER2) {
+		if (mc)
+			peak_bw = peak_bw * mc->num_channels;
+	}
+
+	*agg_avg += avg_bw;
+	*agg_peak = max(*agg_peak, peak_bw);
+
+	return 0;
+}
+
 static struct icc_node*
 tegra234_mc_of_icc_xlate(struct of_phandle_args *spec, void *data)
 {
@@ -898,7 +920,7 @@ static int tegra234_mc_icc_get_init_bw(struct icc_node *node, u32 *avg, u32 *pea
 
 static const struct tegra_mc_icc_ops tegra234_mc_icc_ops = {
 	.xlate = tegra234_mc_of_icc_xlate,
-	.aggregate = icc_std_aggregate,
+	.aggregate = tegra234_mc_icc_aggregate,
 	.get_bw = tegra234_mc_icc_get_init_bw,
 	.set = tegra234_mc_icc_set,
 };
include/soc/tegra/mc.h +1 -0
index 900d88b26fae..fc3001483e62 100644
@@ -234,6 +234,7 @@ struct tegra_mc {
 
 	struct tegra_mc_timing *timings;
 	unsigned int num_timings;
+	unsigned int num_channels;
 
 	bool bwmgr_mrq_supported;
 	struct reset_controller_dev reset;
Signature · 1 line
2.17.1
#13 lore raw

PATCH v7 6/8 PCI: tegra194: Fix possible array out of bounds access

In reply to Sumit Gupta

Add check to fix the possible array out of bounds violation by making speed equal to GEN1_CORE_CLK_FREQ when its value is more than the size of "pcie_gen_freq" array. This array has size of four but possible speed (CLS) values are from "0 to 0xF". So, "speed - 1" values are "-1 to 0xE".

---
 drivers/pci/controller/dwc/pcie-tegra194.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
drivers/pci/controller/dwc/pcie-tegra194.c +11 -2
index 09825b4a075e..e6eec85480ca 100644
@@ -223,6 +223,7 @@
 #define EP_STATE_ENABLED	1
 
 static const unsigned int pcie_gen_freq[] = {
+	GEN1_CORE_CLK_FREQ,	/* PCI_EXP_LNKSTA_CLS == 0; undefined */
 	GEN1_CORE_CLK_FREQ,
 	GEN2_CORE_CLK_FREQ,
 	GEN3_CORE_CLK_FREQ,
@@ -459,7 +460,11 @@ static irqreturn_t tegra_pcie_ep_irq_thread(int irq, void *arg)
 
 	speed = dw_pcie_readw_dbi(pci, pcie->pcie_cap_base + PCI_EXP_LNKSTA) &
 		PCI_EXP_LNKSTA_CLS;
-	clk_set_rate(pcie->core_clk, pcie_gen_freq[speed - 1]);
+
+	if (speed >= ARRAY_SIZE(pcie_gen_freq))
+		speed = 0;
+
+	clk_set_rate(pcie->core_clk, pcie_gen_freq[speed]);
 
 	if (pcie->of_data->has_ltr_req_fix)
 		return IRQ_HANDLED;
@@ -1020,7 +1025,11 @@ static int tegra_pcie_dw_start_link(struct dw_pcie *pci)
 
 	speed = dw_pcie_readw_dbi(pci, pcie->pcie_cap_base + PCI_EXP_LNKSTA) &
 		PCI_EXP_LNKSTA_CLS;
-	clk_set_rate(pcie->core_clk, pcie_gen_freq[speed - 1]);
+
+	if (speed >= ARRAY_SIZE(pcie_gen_freq))
+		speed = 0;
+
+	clk_set_rate(pcie->core_clk, pcie_gen_freq[speed]);
 
 	tegra_pcie_enable_interrupts(pp);
 
Signature · 1 line
2.17.1
#14 lore raw

PATCH v7 7/8 PCI: tegra194: Add interconnect support in Tegra234

In reply to Sumit Gupta

Add support to request DRAM bandwidth (BW) with Memory Interconnect in Tegra234 SoC. The DRAM BW required for different modes depends on the link speed (Gen-1/2/3/4) and width/lanes (x1/x2/x4/x8). Currently, the DRAM frequency is always set to the maximum available but that results in the highest power consumption.
The Memory Interconnect is a software feature which uses Interconnect framework (ICC). It adds the capability for Memory Controller (MC) clients to request bandwidth and therefore scale DRAM frequency dynamically depending on the required link speed so that the DRAM energy consumption can be optimized.

---
 drivers/pci/controller/dwc/pcie-tegra194.c | 51 +++++++++++++++-------
 1 file changed, 35 insertions(+), 16 deletions(-)
drivers/pci/controller/dwc/pcie-tegra194.c +35 -16
index e6eec85480ca..4fdadc7b045f 100644
@@ -14,6 +14,7 @@
 #include <linux/delay.h>
 #include <linux/gpio.h>
 #include <linux/gpio/consumer.h>
+#include <linux/interconnect.h>
 #include <linux/interrupt.h>
 #include <linux/iopoll.h>
 #include <linux/kernel.h>
@@ -288,6 +289,7 @@ struct tegra_pcie_dw {
 	unsigned int pex_rst_irq;
 	int ep_state;
 	long link_status;
+	struct icc_path *icc_path;
 };
 
 static inline struct tegra_pcie_dw *to_tegra_pcie(struct dw_pcie *pci)
@@ -310,6 +312,27 @@ struct tegra_pcie_soc {
 	enum dw_pcie_device_mode mode;
 };
 
+static void tegra_pcie_icc_set(struct tegra_pcie_dw *pcie)
+{
+	struct dw_pcie *pci = &pcie->pci;
+	u32 val, speed, width;
+
+	val = dw_pcie_readw_dbi(pci, pcie->pcie_cap_base + PCI_EXP_LNKSTA);
+
+	speed = FIELD_GET(PCI_EXP_LNKSTA_CLS, val);
+	width = FIELD_GET(PCI_EXP_LNKSTA_NLW, val);
+
+	val = width * (PCIE_SPEED2MBS_ENC(pcie_link_speed[speed]) / BITS_PER_BYTE);
+
+	if (icc_set_bw(pcie->icc_path, MBps_to_icc(val), 0))
+		dev_err(pcie->dev, "can't set bw[%u]\n", val);
+
+	if (speed >= ARRAY_SIZE(pcie_gen_freq))
+		speed = 0;
+
+	clk_set_rate(pcie->core_clk, pcie_gen_freq[speed]);
+}
+
 static void apply_bad_link_workaround(struct dw_pcie_rp *pp)
 {
 	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
@@ -453,18 +476,12 @@ static irqreturn_t tegra_pcie_ep_irq_thread(int irq, void *arg)
 	struct tegra_pcie_dw *pcie = arg;
 	struct dw_pcie_ep *ep = &pcie->pci.ep;
 	struct dw_pcie *pci = &pcie->pci;
-	u32 val, speed;
+	u32 val;
 
 	if (test_and_clear_bit(0, &pcie->link_status))
 		dw_pcie_ep_linkup(ep);
 
-	speed = dw_pcie_readw_dbi(pci, pcie->pcie_cap_base + PCI_EXP_LNKSTA) &
-		PCI_EXP_LNKSTA_CLS;
-
-	if (speed >= ARRAY_SIZE(pcie_gen_freq))
-		speed = 0;
-
-	clk_set_rate(pcie->core_clk, pcie_gen_freq[speed]);
+	tegra_pcie_icc_set(pcie);
 
 	if (pcie->of_data->has_ltr_req_fix)
 		return IRQ_HANDLED;
@@ -950,9 +967,9 @@ static int tegra_pcie_dw_host_init(struct dw_pcie_rp *pp)
 
 static int tegra_pcie_dw_start_link(struct dw_pcie *pci)
 {
-	u32 val, offset, speed, tmp;
 	struct tegra_pcie_dw *pcie = to_tegra_pcie(pci);
 	struct dw_pcie_rp *pp = &pci->pp;
+	u32 val, offset, tmp;
 	bool retry = true;
 
 	if (pcie->of_data->mode == DW_PCIE_EP_TYPE) {
@@ -1023,13 +1040,7 @@ static int tegra_pcie_dw_start_link(struct dw_pcie *pci)
 		goto retry_link;
 	}
 
-	speed = dw_pcie_readw_dbi(pci, pcie->pcie_cap_base + PCI_EXP_LNKSTA) &
-		PCI_EXP_LNKSTA_CLS;
-
-	if (speed >= ARRAY_SIZE(pcie_gen_freq))
-		speed = 0;
-
-	clk_set_rate(pcie->core_clk, pcie_gen_freq[speed]);
+	tegra_pcie_icc_set(pcie);
 
 	tegra_pcie_enable_interrupts(pp);
 
@@ -2233,6 +2244,14 @@ static int tegra_pcie_dw_probe(struct platform_device *pdev)
 
 	platform_set_drvdata(pdev, pcie);
 
+	pcie->icc_path = devm_of_icc_get(&pdev->dev, "write");
+	ret = PTR_ERR_OR_ZERO(pcie->icc_path);
+	if (ret) {
+		tegra_bpmp_put(pcie->bpmp);
+		dev_err_probe(&pdev->dev, ret, "failed to get write interconnect\n");
+		return ret;
+	}
+
 	switch (pcie->of_data->mode) {
 	case DW_PCIE_RC_TYPE:
 		ret = devm_request_irq(dev, pp->irq, tegra_pcie_rp_irq_handler,
Signature · 1 line
2.17.1
#15 lore raw

PATCH v7 8/8 arm64: tegra: Add cpu OPP tables and interconnects property

In reply to Sumit Gupta

Add OPP table and interconnects property to scale DDR frequency with CPU frequency for better performance. Each operating point entry of the OPP table has CPU freq to per MC channel bandwidth mapping. One table is added for each cluster even though the table data is same because the bandwidth request is per cluster. This is done because OPP framework creates a single icc path and hence single bandwidth request if the table is marked as 'opp-shared' and shared among all clusters. For us, the OPP table data is same but the MC Client ID argument to interconnects property is different for each cluster. So, having per cluster table makes different icc path for each cluster and helps to make per cluster BW requests.

---
 arch/arm64/boot/dts/nvidia/tegra234.dtsi | 276 +++++++++++++++++++++++
 1 file changed, 276 insertions(+)
arch/arm64/boot/dts/nvidia/tegra234.dtsi +276 -0
index 5d354f8923b4..8d0b89b5733e 100644
@@ -3011,6 +3011,9 @@
 
 			enable-method = "psci";
 
+			operating-points-v2 = <&cl0_opp_tbl>;
+			interconnects = <&mc TEGRA_ICC_MC_CPU_CLUSTER0 &emc>;
+
 			i-cache-size = <65536>;
 			i-cache-line-size = <64>;
 			i-cache-sets = <256>;
@@ -3027,6 +3030,9 @@
 
 			enable-method = "psci";
 
+			operating-points-v2 = <&cl0_opp_tbl>;
+			interconnects = <&mc TEGRA_ICC_MC_CPU_CLUSTER0 &emc>;
+
 			i-cache-size = <65536>;
 			i-cache-line-size = <64>;
 			i-cache-sets = <256>;
@@ -3043,6 +3049,9 @@
 
 			enable-method = "psci";
 
+			operating-points-v2 = <&cl0_opp_tbl>;
+			interconnects = <&mc TEGRA_ICC_MC_CPU_CLUSTER0 &emc>;
+
 			i-cache-size = <65536>;
 			i-cache-line-size = <64>;
 			i-cache-sets = <256>;
@@ -3059,6 +3068,9 @@
 
 			enable-method = "psci";
 
+			operating-points-v2 = <&cl0_opp_tbl>;
+			interconnects = <&mc TEGRA_ICC_MC_CPU_CLUSTER0 &emc>;
+
 			i-cache-size = <65536>;
 			i-cache-line-size = <64>;
 			i-cache-sets = <256>;
@@ -3075,6 +3087,9 @@
 
 			enable-method = "psci";
 
+			operating-points-v2 = <&cl1_opp_tbl>;
+			interconnects = <&mc TEGRA_ICC_MC_CPU_CLUSTER1 &emc>;
+
 			i-cache-size = <65536>;
 			i-cache-line-size = <64>;
 			i-cache-sets = <256>;
@@ -3091,6 +3106,9 @@
 
 			enable-method = "psci";
 
+			operating-points-v2 = <&cl1_opp_tbl>;
+			interconnects = <&mc TEGRA_ICC_MC_CPU_CLUSTER1 &emc>;
+
 			i-cache-size = <65536>;
 			i-cache-line-size = <64>;
 			i-cache-sets = <256>;
@@ -3107,6 +3125,9 @@
 
 			enable-method = "psci";
 
+			operating-points-v2 = <&cl1_opp_tbl>;
+			interconnects = <&mc TEGRA_ICC_MC_CPU_CLUSTER1 &emc>;
+
 			i-cache-size = <65536>;
 			i-cache-line-size = <64>;
 			i-cache-sets = <256>;
@@ -3123,6 +3144,9 @@
 
 			enable-method = "psci";
 
+			operating-points-v2 = <&cl1_opp_tbl>;
+			interconnects = <&mc TEGRA_ICC_MC_CPU_CLUSTER1 &emc>;
+
 			i-cache-size = <65536>;
 			i-cache-line-size = <64>;
 			i-cache-sets = <256>;
@@ -3139,6 +3163,9 @@
 
 			enable-method = "psci";
 
+			operating-points-v2 = <&cl2_opp_tbl>;
+			interconnects = <&mc TEGRA_ICC_MC_CPU_CLUSTER2 &emc>;
+
 			i-cache-size = <65536>;
 			i-cache-line-size = <64>;
 			i-cache-sets = <256>;
@@ -3155,6 +3182,9 @@
 
 			enable-method = "psci";
 
+			operating-points-v2 = <&cl2_opp_tbl>;
+			interconnects = <&mc TEGRA_ICC_MC_CPU_CLUSTER2 &emc>;
+
 			i-cache-size = <65536>;
 			i-cache-line-size = <64>;
 			i-cache-sets = <256>;
@@ -3171,6 +3201,9 @@
 
 			enable-method = "psci";
 
+			operating-points-v2 = <&cl2_opp_tbl>;
+			interconnects = <&mc TEGRA_ICC_MC_CPU_CLUSTER2 &emc>;
+
 			i-cache-size = <65536>;
 			i-cache-line-size = <64>;
 			i-cache-sets = <256>;
@@ -3187,6 +3220,9 @@
 
 			enable-method = "psci";
 
+			operating-points-v2 = <&cl2_opp_tbl>;
+			interconnects = <&mc TEGRA_ICC_MC_CPU_CLUSTER2 &emc>;
+
 			i-cache-size = <65536>;
 			i-cache-line-size = <64>;
 			i-cache-sets = <256>;
@@ -3461,4 +3497,244 @@
 		interrupt-parent = <&gic>;
 		always-on;
 	};
+
+	cl0_opp_tbl: opp-table-cluster0 {
+		compatible = "operating-points-v2";
+		opp-shared;
+
+		cl0_ch1_opp1: opp-115200000 {
+			  opp-hz = /bits/ 64 <115200000>;
+			  opp-peak-kBps = <816000>;
+		};
+
+		cl0_ch1_opp2: opp-268800000 {
+			opp-hz = /bits/ 64 <268800000>;
+			opp-peak-kBps = <816000>;
+		};
+
+		cl0_ch1_opp3: opp-422400000 {
+			opp-hz = /bits/ 64 <422400000>;
+			opp-peak-kBps = <816000>;
+		};
+
+		cl0_ch1_opp4: opp-576000000 {
+			opp-hz = /bits/ 64 <576000000>;
+			opp-peak-kBps = <816000>;
+		};
+
+		cl0_ch1_opp5: opp-729600000 {
+			opp-hz = /bits/ 64 <729600000>;
+			opp-peak-kBps = <816000>;
+		};
+
+		cl0_ch1_opp6: opp-883200000 {
+			opp-hz = /bits/ 64 <883200000>;
+			opp-peak-kBps = <816000>;
+		};
+
+		cl0_ch1_opp7: opp-1036800000 {
+			opp-hz = /bits/ 64 <1036800000>;
+			opp-peak-kBps = <816000>;
+		};
+
+		cl0_ch1_opp8: opp-1190400000 {
+			opp-hz = /bits/ 64 <1190400000>;
+			opp-peak-kBps = <816000>;
+		};
+
+		cl0_ch1_opp9: opp-1344000000 {
+			opp-hz = /bits/ 64 <1344000000>;
+			opp-peak-kBps = <1632000>;
+		};
+
+		cl0_ch1_opp10: opp-1497600000 {
+			opp-hz = /bits/ 64 <1497600000>;
+			opp-peak-kBps = <1632000>;
+		};
+
+		cl0_ch1_opp11: opp-1651200000 {
+			opp-hz = /bits/ 64 <1651200000>;
+			opp-peak-kBps = <2660000>;
+		};
+
+		cl0_ch1_opp12: opp-1804800000 {
+			opp-hz = /bits/ 64 <1804800000>;
+			opp-peak-kBps = <2660000>;
+		};
+
+		cl0_ch1_opp13: opp-1958400000 {
+			opp-hz = /bits/ 64 <1958400000>;
+			opp-peak-kBps = <3200000>;
+		};
+
+		cl0_ch1_opp14: opp-2112000000 {
+			opp-hz = /bits/ 64 <2112000000>;
+			opp-peak-kBps = <6400000>;
+		};
+
+		cl0_ch1_opp15: opp-2201600000 {
+			opp-hz = /bits/ 64 <2201600000>;
+			opp-peak-kBps = <6400000>;
+		};
+	};
+
+	cl1_opp_tbl: opp-table-cluster1 {
+		compatible = "operating-points-v2";
+		opp-shared;
+
+		cl1_ch1_opp1: opp-115200000 {
+			  opp-hz = /bits/ 64 <115200000>;
+			  opp-peak-kBps = <816000>;
+		};
+
+		cl1_ch1_opp2: opp-268800000 {
+			opp-hz = /bits/ 64 <268800000>;
+			opp-peak-kBps = <816000>;
+		};
+
+		cl1_ch1_opp3: opp-422400000 {
+			opp-hz = /bits/ 64 <422400000>;
+			opp-peak-kBps = <816000>;
+		};
+
+		cl1_ch1_opp4: opp-576000000 {
+			opp-hz = /bits/ 64 <576000000>;
+			opp-peak-kBps = <816000>;
+		};
+
+		cl1_ch1_opp5: opp-729600000 {
+			opp-hz = /bits/ 64 <729600000>;
+			opp-peak-kBps = <816000>;
+		};
+
+		cl1_ch1_opp6: opp-883200000 {
+			opp-hz = /bits/ 64 <883200000>;
+			opp-peak-kBps = <816000>;
+		};
+
+		cl1_ch1_opp7: opp-1036800000 {
+			opp-hz = /bits/ 64 <1036800000>;
+			opp-peak-kBps = <816000>;
+		};
+
+		cl1_ch1_opp8: opp-1190400000 {
+			opp-hz = /bits/ 64 <1190400000>;
+			opp-peak-kBps = <816000>;
+		};
+
+		cl1_ch1_opp9: opp-1344000000 {
+			opp-hz = /bits/ 64 <1344000000>;
+			opp-peak-kBps = <1632000>;
+		};
+
+		cl1_ch1_opp10: opp-1497600000 {
+			opp-hz = /bits/ 64 <1497600000>;
+			opp-peak-kBps = <1632000>;
+		};
+
+		cl1_ch1_opp11: opp-1651200000 {
+			opp-hz = /bits/ 64 <1651200000>;
+			opp-peak-kBps = <2660000>;
+		};
+
+		cl1_ch1_opp12: opp-1804800000 {
+			opp-hz = /bits/ 64 <1804800000>;
+			opp-peak-kBps = <2660000>;
+		};
+
+		cl1_ch1_opp13: opp-1958400000 {
+			opp-hz = /bits/ 64 <1958400000>;
+			opp-peak-kBps = <3200000>;
+		};
+
+		cl1_ch1_opp14: opp-2112000000 {
+			opp-hz = /bits/ 64 <2112000000>;
+			opp-peak-kBps = <6400000>;
+		};
+
+		cl1_ch1_opp15: opp-2201600000 {
+			opp-hz = /bits/ 64 <2201600000>;
+			opp-peak-kBps = <6400000>;
+		};
+	};
+
+	cl2_opp_tbl: opp-table-cluster2 {
+		compatible = "operating-points-v2";
+		opp-shared;
+
+		cl2_ch1_opp1: opp-115200000 {
+			  opp-hz = /bits/ 64 <115200000>;
+			  opp-peak-kBps = <816000>;
+		};
+
+		cl2_ch1_opp2: opp-268800000 {
+			opp-hz = /bits/ 64 <268800000>;
+			opp-peak-kBps = <816000>;
+		};
+
+		cl2_ch1_opp3: opp-422400000 {
+			opp-hz = /bits/ 64 <422400000>;
+			opp-peak-kBps = <816000>;
+		};
+
+		cl2_ch1_opp4: opp-576000000 {
+			opp-hz = /bits/ 64 <576000000>;
+			opp-peak-kBps = <816000>;
+		};
+
+		cl2_ch1_opp5: opp-729600000 {
+			opp-hz = /bits/ 64 <729600000>;
+			opp-peak-kBps = <816000>;
+		};
+
+		cl2_ch1_opp6: opp-883200000 {
+			opp-hz = /bits/ 64 <883200000>;
+			opp-peak-kBps = <816000>;
+		};
+
+		cl2_ch1_opp7: opp-1036800000 {
+			opp-hz = /bits/ 64 <1036800000>;
+			opp-peak-kBps = <816000>;
+		};
+
+		cl2_ch1_opp8: opp-1190400000 {
+			opp-hz = /bits/ 64 <1190400000>;
+			opp-peak-kBps = <816000>;
+		};
+
+		cl2_ch1_opp9: opp-1344000000 {
+			opp-hz = /bits/ 64 <1344000000>;
+			opp-peak-kBps = <1632000>;
+		};
+
+		cl2_ch1_opp10: opp-1497600000 {
+			opp-hz = /bits/ 64 <1497600000>;
+			opp-peak-kBps = <1632000>;
+		};
+
+		cl2_ch1_opp11: opp-1651200000 {
+			opp-hz = /bits/ 64 <1651200000>;
+			opp-peak-kBps = <2660000>;
+		};
+
+		cl2_ch1_opp12: opp-1804800000 {
+			opp-hz = /bits/ 64 <1804800000>;
+			opp-peak-kBps = <2660000>;
+		};
+
+		cl2_ch1_opp13: opp-1958400000 {
+			opp-hz = /bits/ 64 <1958400000>;
+			opp-peak-kBps = <3200000>;
+		};
+
+		cl2_ch1_opp14: opp-2112000000 {
+			opp-hz = /bits/ 64 <2112000000>;
+			opp-peak-kBps = <6400000>;
+		};
+
+		cl2_ch1_opp15: opp-2201600000 {
+			opp-hz = /bits/ 64 <2201600000>;
+			opp-peak-kBps = <6400000>;
+		};
+	};
 };
Signature · 1 line
2.17.1
#16 lore raw
In reply to Sumit Gupta
Quoted 5 lines · Sumit Gupta

Hi All,

Thank you for the review.
Requesting for ACK on the remaining patches and please consider for merging.

You know, if you haven't make it a dependencies and request to merge everything through arm-soc, the memory controller bits would be already in v6.3.

Best regards,
Krzysztof