Skip to contentSkip to conversation

drm/lima: Convert to clk_bulk API

PATCH In review Marek Vasut 5 participants 10 messages Jul 16, 2021 – Jul 20, 2021

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 drm/lima: Convert to clk_bulk API

Instead of requesting two separate clock and then handling them separately in various places of the driver, use clk_bulk_*() API. This permits handling devices with more than "bus"/"core" clock, like ZynqMP, which has "gpu"/"gpu_pp0"/"gpu_pp1" all as separate clock.

---
 drivers/gpu/drm/lima/lima_devfreq.c | 17 +++++++++---
 drivers/gpu/drm/lima/lima_devfreq.h |  1 +
 drivers/gpu/drm/lima/lima_device.c  | 42 +++++++++++------------------
 drivers/gpu/drm/lima/lima_device.h  |  4 +--
 4 files changed, 32 insertions(+), 32 deletions(-)
drivers/gpu/drm/lima/lima_devfreq.c +14 -3
index 8989e215dfc9..533b36932f79 100644
@@ -58,7 +58,7 @@ static int lima_devfreq_get_dev_status(struct device *dev,
 	struct lima_devfreq *devfreq = &ldev->devfreq;
 	unsigned long irqflags;
 
-	status->current_frequency = clk_get_rate(ldev->clk_gpu);
+	status->current_frequency = clk_get_rate(devfreq->clk_gpu);
 
 	spin_lock_irqsave(&devfreq->lock, irqflags);
 
@@ -110,12 +110,23 @@ int lima_devfreq_init(struct lima_device *ldev)
 	struct lima_devfreq *ldevfreq = &ldev->devfreq;
 	struct dev_pm_opp *opp;
 	unsigned long cur_freq;
-	int ret;
+	int i, ret;
 
 	if (!device_property_present(dev, "operating-points-v2"))
 		/* Optional, continue without devfreq */
 		return 0;
 
+	/* Find first clock which are not "bus" clock */
+	for (i = 0; i < ldev->nr_clks; i++) {
+		if (!strcmp(ldev->clks[i].id, "bus"))
+			continue;
+		ldevfreq->clk_gpu = ldev->clks[i].clk;
+		break;
+	}
+
+	if (!ldevfreq->clk_gpu)
+		return -ENODEV;
+
 	spin_lock_init(&ldevfreq->lock);
 
 	ret = devm_pm_opp_set_clkname(dev, "core");
@@ -135,7 +146,7 @@ int lima_devfreq_init(struct lima_device *ldev)
 
 	lima_devfreq_reset(ldevfreq);
 
-	cur_freq = clk_get_rate(ldev->clk_gpu);
+	cur_freq = clk_get_rate(ldevfreq->clk_gpu);
 
 	opp = devfreq_recommended_opp(dev, &cur_freq, 0);
 	if (IS_ERR(opp))
drivers/gpu/drm/lima/lima_devfreq.h +1 -0
index b8e50feaeab6..ffef5c91795d 100644
@@ -17,6 +17,7 @@ struct lima_devfreq {
 	struct devfreq *devfreq;
 	struct thermal_cooling_device *cooling;
 	struct devfreq_simple_ondemand_data gov_data;
+	struct clk *clk_gpu;
 
 	ktime_t busy_time;
 	ktime_t idle_time;
drivers/gpu/drm/lima/lima_device.c +15 -27

83 lines not on this page · raw message

drivers/gpu/drm/lima/lima_device.h +2 -2
index 41b9d7b4bcc7..de11570c9903 100644
@@ -85,8 +85,8 @@ struct lima_device {
 	int num_pp;
 
 	void __iomem *iomem;
-	struct clk *clk_bus;
-	struct clk *clk_gpu;
+	struct clk_bulk_data *clks;
+	int nr_clks;
 	struct reset_control *reset;
 	struct regulator *regulator;
 
Signature · 1 line
2.30.2
maintainer
#1 lore raw
Quoted 6 lines · Marek Vasut

Instead of requesting two separate clock and then handling them separately in various places of the driver, use clk_bulk_*() API. This permits handling devices with more than "bus"/"core" clock, like ZynqMP, which has "gpu"/"gpu_pp0"/"gpu_pp1" all as separate clock.

I can't find the ZynqMP DTS file under arch/arm64/boot/dts/xilinx which has mali GPU node with an upstream kernel, where is it?

So what's the relationship between "gpu" clk and "gpu_pp0"/"gpu_pp1" clk? Do they need to be controlled separately or we can just control the "gpu" clk? Because the devfreq code just controls a single module clk.

Quoted 32 lines
---
 drivers/gpu/drm/lima/lima_devfreq.c | 17 +++++++++---
 drivers/gpu/drm/lima/lima_devfreq.h |  1 +
 drivers/gpu/drm/lima/lima_device.c  | 42 +++++++++++------------------
 drivers/gpu/drm/lima/lima_device.h  |  4 +--
 4 files changed, 32 insertions(+), 32 deletions(-)
drivers/gpu/drm/lima/lima_devfreq.c +2 -2
index 8989e215dfc9..533b36932f79 100644
@@ -58,7 +58,7 @@ static int lima_devfreq_get_dev_status(struct device *dev,
        struct lima_devfreq *devfreq = &ldev->devfreq;
        unsigned long irqflags;
 
-       status->current_frequency = clk_get_rate(ldev->clk_gpu);
+       status->current_frequency = clk_get_rate(devfreq->clk_gpu);
 
        spin_lock_irqsave(&devfreq->lock, irqflags);
 
@@ -110,12 +110,23 @@ int lima_devfreq_init(struct lima_device *ldev)
        struct lima_devfreq *ldevfreq = &ldev->devfreq;
        struct dev_pm_opp *opp;
        unsigned long cur_freq;
-       int ret;
+       int i, ret;
 
        if (!device_property_present(dev, "operating-points-v2"))

drivers/gpu/drm/lima/lima_devfreq.c int lima_devfreq_init(struct lima_device *ldev) · line 125

Quoted 10 lines
drivers/gpu/drm/lima/lima_devfreq.c +7 -0
                /* Optional, continue without devfreq */
                return 0;
 
+       /* Find first clock which are not "bus" clock */
+       for (i = 0; i < ldev->nr_clks; i++) {
+               if (!strcmp(ldev->clks[i].id, "bus"))
+                       continue;
+               ldevfreq->clk_gpu = ldev->clks[i].clk;
+               break;
+       }

I'd prefer an explicit name for the required clk name. If some DTS has different name other than "core" for the module clk (ie. "gpu"), it should be changed to "core".

Quoted 136 lines
drivers/gpu/drm/lima/lima_devfreq.c +5 -1
+
+       if (!ldevfreq->clk_gpu)
+               return -ENODEV;
+
        spin_lock_init(&ldevfreq->lock);
 
        ret = devm_pm_opp_set_clkname(dev, "core");
@@ -135,7 +146,7 @@ int lima_devfreq_init(struct lima_device *ldev)
 
        lima_devfreq_reset(ldevfreq);
 
-       cur_freq = clk_get_rate(ldev->clk_gpu);
+       cur_freq = clk_get_rate(ldevfreq->clk_gpu);
 
        opp = devfreq_recommended_opp(dev, &cur_freq, 0);
        if (IS_ERR(opp))
drivers/gpu/drm/lima/lima_devfreq.h +1 -0
index b8e50feaeab6..ffef5c91795d 100644
@@ -17,6 +17,7 @@ struct lima_devfreq {
        struct devfreq *devfreq;
        struct thermal_cooling_device *cooling;
        struct devfreq_simple_ondemand_data gov_data;
+       struct clk *clk_gpu;
 
        ktime_t busy_time;
        ktime_t idle_time;
drivers/gpu/drm/lima/lima_device.c +15 -27
index 65fdca366e41..9f7bde7e9d22 100644
@@ -85,29 +85,23 @@ static int lima_clk_enable(struct lima_device *dev)
 {
        int err;
 
-       err = clk_prepare_enable(dev->clk_bus);
+       err = clk_bulk_prepare_enable(dev->nr_clks, dev->clks);
        if (err)
                return err;
 
-       err = clk_prepare_enable(dev->clk_gpu);
-       if (err)
-               goto error_out0;
-
        if (dev->reset) {
                err = reset_control_deassert(dev->reset);
                if (err) {
                        dev_err(dev->dev,
                                "reset controller deassert failed %d\n", err);
-                       goto error_out1;
+                       goto error;
                }
        }
 
        return 0;
 
-error_out1:
-       clk_disable_unprepare(dev->clk_gpu);
-error_out0:
-       clk_disable_unprepare(dev->clk_bus);
+error:
+       clk_bulk_disable_unprepare(dev->nr_clks, dev->clks);
        return err;
 }
 
@@ -115,31 +109,23 @@ static void lima_clk_disable(struct lima_device *dev)
 {
        if (dev->reset)
                reset_control_assert(dev->reset);
-       clk_disable_unprepare(dev->clk_gpu);
-       clk_disable_unprepare(dev->clk_bus);
+       clk_bulk_disable_unprepare(dev->nr_clks, dev->clks);
 }
 
 static int lima_clk_init(struct lima_device *dev)
 {
        int err;
 
-       dev->clk_bus = devm_clk_get(dev->dev, "bus");
-       if (IS_ERR(dev->clk_bus)) {
-               err = PTR_ERR(dev->clk_bus);
+       err = devm_clk_bulk_get_all(dev->dev, &dev->clks);
+       if (err < 1) {
+               if (err == 0)   /* No clock at all is an error too */
+                       err = -ENODEV;
                if (err != -EPROBE_DEFER)
-                       dev_err(dev->dev, "get bus clk failed %d\n", err);
-               dev->clk_bus = NULL;
+                       dev_err(dev->dev, "get clk failed %d\n", err);
                return err;
        }
 
-       dev->clk_gpu = devm_clk_get(dev->dev, "core");
-       if (IS_ERR(dev->clk_gpu)) {
-               err = PTR_ERR(dev->clk_gpu);
-               if (err != -EPROBE_DEFER)
-                       dev_err(dev->dev, "get core clk failed %d\n", err);
-               dev->clk_gpu = NULL;
-               return err;
-       }
+       dev->nr_clks = err;
 
        dev->reset = devm_reset_control_array_get_optional_shared(dev->dev);
        if (IS_ERR(dev->reset)) {
@@ -412,8 +398,10 @@ int lima_device_init(struct lima_device *ldev)
        INIT_LIST_HEAD(&ldev->error_task_list);
        mutex_init(&ldev->error_task_list_lock);
 
-       dev_info(ldev->dev, "bus rate = %lu\n", clk_get_rate(ldev->clk_bus));
-       dev_info(ldev->dev, "mod rate = %lu", clk_get_rate(ldev->clk_gpu));
+       for (i = 0; i < ldev->nr_clks; i++) {
+               dev_info(ldev->dev, "clk %s = %lu Hz\n", ldev->clks[i].id,
+                        clk_get_rate(ldev->clks[i].clk));
+       }
 
        return 0;
 
drivers/gpu/drm/lima/lima_device.h +2 -3
index 41b9d7b4bcc7..de11570c9903 100644
@@ -85,8 +85,8 @@ struct lima_device {
        int num_pp;
 
        void __iomem *iomem;
-       struct clk *clk_bus;
-       struct clk *clk_gpu;
+       struct clk_bulk_data *clks;
+       int nr_clks;
        struct reset_control *reset;
        struct regulator *regulator;
 
--

2.30.2

#2 lore raw
Quoted 10 lines · Qiang Yu
Quoted 6 lines · Marek Vasut

Instead of requesting two separate clock and then handling them separately in various places of the driver, use clk_bulk_*() API. This permits handling devices with more than "bus"/"core" clock, like ZynqMP, which has "gpu"/"gpu_pp0"/"gpu_pp1" all as separate clock.

I can't find the ZynqMP DTS file under arch/arm64/boot/dts/xilinx which has mali GPU node with an upstream kernel, where is it?

Posted here:
https://patchwork.kernel.org/project/linux-arm-kernel/patch/[email protected]/

Quoted 3 lines

So what's the relationship between "gpu" clk and "gpu_pp0"/"gpu_pp1" clk? Do they need to be controlled separately or we can just control the "gpu" clk? Because the devfreq code just controls a single module clk.

Per the docs, they are separate enable bits and the zynqmp clock controller exports them as separate clock, see bits 24..26 here:

https://www.xilinx.com/html_docs/registers/ug1087/crf_apb___gpu_ref_ctrl.html

Quoted 34 lines
Quoted 34 lines
---
  drivers/gpu/drm/lima/lima_devfreq.c | 17 +++++++++---
  drivers/gpu/drm/lima/lima_devfreq.h |  1 +
  drivers/gpu/drm/lima/lima_device.c  | 42 +++++++++++------------------
  drivers/gpu/drm/lima/lima_device.h  |  4 +--
  4 files changed, 32 insertions(+), 32 deletions(-)
drivers/gpu/drm/lima/lima_devfreq.c +2 -2
index 8989e215dfc9..533b36932f79 100644
@@ -58,7 +58,7 @@ static int lima_devfreq_get_dev_status(struct device *dev,
         struct lima_devfreq *devfreq = &ldev->devfreq;
         unsigned long irqflags;
 
-       status->current_frequency = clk_get_rate(ldev->clk_gpu);
+       status->current_frequency = clk_get_rate(devfreq->clk_gpu);
 
         spin_lock_irqsave(&devfreq->lock, irqflags);
 
@@ -110,12 +110,23 @@ int lima_devfreq_init(struct lima_device *ldev)
         struct lima_devfreq *ldevfreq = &ldev->devfreq;
         struct dev_pm_opp *opp;
         unsigned long cur_freq;
-       int ret;
+       int i, ret;
 
         if (!device_property_present(dev, "operating-points-v2"))
                 /* Optional, continue without devfreq */
                 return 0;
 
Quoted 11 lines

drivers/gpu/drm/lima/lima_devfreq.c int lima_devfreq_init(struct lima_device *ldev) · line 125

Quoted 7 lines
drivers/gpu/drm/lima/lima_devfreq.c +7 -0
+       /* Find first clock which are not "bus" clock */
+       for (i = 0; i < ldev->nr_clks; i++) {
+               if (!strcmp(ldev->clks[i].id, "bus"))
+                       continue;
+               ldevfreq->clk_gpu = ldev->clks[i].clk;
+               break;
+       }

I'd prefer an explicit name for the required clk name. If some DTS has different name other than "core" for the module clk (ie. "gpu"), it should be changed to "core".

The problem here is, the zynqmp has no core clock, it has "gpu and both pixel pipes" super-clock-gate which controls everything, and then per-pixel-pipe sub-clock-gates.

maintainer
#3 lore raw
Quoted 71 lines · Marek Vasut
Quoted 10 lines · Qiang Yu
Quoted 6 lines · Marek Vasut

Instead of requesting two separate clock and then handling them separately in various places of the driver, use clk_bulk_*() API. This permits handling devices with more than "bus"/"core" clock, like ZynqMP, which has "gpu"/"gpu_pp0"/"gpu_pp1" all as separate clock.

I can't find the ZynqMP DTS file under arch/arm64/boot/dts/xilinx which has mali GPU node with an upstream kernel, where is it?

Posted here:
https://patchwork.kernel.org/project/linux-arm-kernel/patch/[email protected]/

Quoted 3 lines

So what's the relationship between "gpu" clk and "gpu_pp0"/"gpu_pp1" clk? Do they need to be controlled separately or we can just control the "gpu" clk? Because the devfreq code just controls a single module clk.

Per the docs, they are separate enable bits and the zynqmp clock controller exports them as separate clock, see bits 24..26 here:

https://www.xilinx.com/html_docs/registers/ug1087/crf_apb___gpu_ref_ctrl.html

Quoted 34 lines
Quoted 34 lines
---
  drivers/gpu/drm/lima/lima_devfreq.c | 17 +++++++++---
  drivers/gpu/drm/lima/lima_devfreq.h |  1 +
  drivers/gpu/drm/lima/lima_device.c  | 42 +++++++++++------------------
  drivers/gpu/drm/lima/lima_device.h  |  4 +--
  4 files changed, 32 insertions(+), 32 deletions(-)
drivers/gpu/drm/lima/lima_devfreq.c +2 -2
index 8989e215dfc9..533b36932f79 100644
@@ -58,7 +58,7 @@ static int lima_devfreq_get_dev_status(struct device *dev,
         struct lima_devfreq *devfreq = &ldev->devfreq;
         unsigned long irqflags;
 
-       status->current_frequency = clk_get_rate(ldev->clk_gpu);
+       status->current_frequency = clk_get_rate(devfreq->clk_gpu);
 
         spin_lock_irqsave(&devfreq->lock, irqflags);
 
@@ -110,12 +110,23 @@ int lima_devfreq_init(struct lima_device *ldev)
         struct lima_devfreq *ldevfreq = &ldev->devfreq;
         struct dev_pm_opp *opp;
         unsigned long cur_freq;
-       int ret;
+       int i, ret;
 
         if (!device_property_present(dev, "operating-points-v2"))
                 /* Optional, continue without devfreq */
                 return 0;
 
Quoted 11 lines

drivers/gpu/drm/lima/lima_devfreq.c int lima_devfreq_init(struct lima_device *ldev) · line 125

Quoted 7 lines
drivers/gpu/drm/lima/lima_devfreq.c +7 -0
+       /* Find first clock which are not "bus" clock */
+       for (i = 0; i < ldev->nr_clks; i++) {
+               if (!strcmp(ldev->clks[i].id, "bus"))
+                       continue;
+               ldevfreq->clk_gpu = ldev->clks[i].clk;
+               break;
+       }

I'd prefer an explicit name for the required clk name. If some DTS has different name other than "core" for the module clk (ie. "gpu"), it should be changed to "core".

Quoted 3 lines

The problem here is, the zynqmp has no core clock, it has "gpu and both pixel pipes" super-clock-gate which controls everything, and then per-pixel-pipe sub-clock-gates.

So the "gpu" clk can gate both "gpu_pp0" and "gpu_pp1" clk, how about frequency? Can we set clock rate for "gpu" then "gpu_pp0" and "gpu_pp1" pass through the same
rate? If so, "gpu" works just like "core".

#4 lore raw
Quoted 72 lines · Qiang Yu
Quoted 71 lines · Marek Vasut
Quoted 10 lines · Qiang Yu
Quoted 6 lines · Marek Vasut

Instead of requesting two separate clock and then handling them separately in various places of the driver, use clk_bulk_*() API. This permits handling devices with more than "bus"/"core" clock, like ZynqMP, which has "gpu"/"gpu_pp0"/"gpu_pp1" all as separate clock.

I can't find the ZynqMP DTS file under arch/arm64/boot/dts/xilinx which has mali GPU node with an upstream kernel, where is it?

Posted here:
https://patchwork.kernel.org/project/linux-arm-kernel/patch/[email protected]/

Quoted 3 lines

So what's the relationship between "gpu" clk and "gpu_pp0"/"gpu_pp1" clk? Do they need to be controlled separately or we can just control the "gpu" clk? Because the devfreq code just controls a single module clk.

Per the docs, they are separate enable bits and the zynqmp clock controller exports them as separate clock, see bits 24..26 here:

https://www.xilinx.com/html_docs/registers/ug1087/crf_apb___gpu_ref_ctrl.html

Quoted 45 lines
Quoted 31 lines
---
   drivers/gpu/drm/lima/lima_devfreq.c | 17 +++++++++---
   drivers/gpu/drm/lima/lima_devfreq.h |  1 +
   drivers/gpu/drm/lima/lima_device.c  | 42 +++++++++++------------------
   drivers/gpu/drm/lima/lima_device.h  |  4 +--
   4 files changed, 32 insertions(+), 32 deletions(-)
drivers/gpu/drm/lima/lima_devfreq.c +2 -2
index 8989e215dfc9..533b36932f79 100644
@@ -58,7 +58,7 @@ static int lima_devfreq_get_dev_status(struct device *dev,
          struct lima_devfreq *devfreq = &ldev->devfreq;
          unsigned long irqflags;
 
-       status->current_frequency = clk_get_rate(ldev->clk_gpu);
+       status->current_frequency = clk_get_rate(devfreq->clk_gpu);
 
          spin_lock_irqsave(&devfreq->lock, irqflags);
 
@@ -110,12 +110,23 @@ int lima_devfreq_init(struct lima_device *ldev)
          struct lima_devfreq *ldevfreq = &ldev->devfreq;
          struct dev_pm_opp *opp;
          unsigned long cur_freq;
-       int ret;
+       int i, ret;
 
          if (!device_property_present(dev, "operating-points-v2"))

drivers/gpu/drm/lima/lima_devfreq.c int lima_devfreq_init(struct lima_device *ldev) · line 125

Quoted 10 lines
drivers/gpu/drm/lima/lima_devfreq.c +7 -0
                  /* Optional, continue without devfreq */
                  return 0;
 
+       /* Find first clock which are not "bus" clock */
+       for (i = 0; i < ldev->nr_clks; i++) {
+               if (!strcmp(ldev->clks[i].id, "bus"))
+                       continue;
+               ldevfreq->clk_gpu = ldev->clks[i].clk;
+               break;
+       }

I'd prefer an explicit name for the required clk name. If some DTS has different name other than "core" for the module clk (ie. "gpu"), it should be changed to "core".

Quoted 5 lines
Quoted 3 lines

The problem here is, the zynqmp has no core clock, it has "gpu and both pixel pipes" super-clock-gate which controls everything, and then per-pixel-pipe sub-clock-gates.

So the "gpu" clk can gate both "gpu_pp0" and "gpu_pp1" clk, how about frequency?

I don't think it is a good idea to just gate off the root clock while the sub-clock are still enabled. That might lead to latch ups (+CC Michal, he might know more).

And who would enable the sub-clock anyway, it should be the GPU driver, no?

Quoted 3 lines

Can we set clock rate for "gpu" then "gpu_pp0" and "gpu_pp1" pass through the same
rate? If so, "gpu" works just like "core".

I don't think the zynqmp is capable of any DVFS on the GPU at all, it just runs at fixed frequency.

maintainer
#5 lore raw
Quoted 78 lines · Marek Vasut
Quoted 76 lines · Qiang Yu
Quoted 74 lines · Marek Vasut
Quoted 10 lines · Qiang Yu
Quoted 6 lines · Marek Vasut

Instead of requesting two separate clock and then handling them separately in various places of the driver, use clk_bulk_*() API. This permits handling devices with more than "bus"/"core" clock, like ZynqMP, which has "gpu"/"gpu_pp0"/"gpu_pp1" all as separate clock.

I can't find the ZynqMP DTS file under arch/arm64/boot/dts/xilinx which has mali GPU node with an upstream kernel, where is it?

Posted here:
https://patchwork.kernel.org/project/linux-arm-kernel/patch/[email protected]/

Quoted 3 lines

So what's the relationship between "gpu" clk and "gpu_pp0"/"gpu_pp1" clk? Do they need to be controlled separately or we can just control the "gpu" clk? Because the devfreq code just controls a single module clk.

Per the docs, they are separate enable bits and the zynqmp clock controller exports them as separate clock, see bits 24..26 here:

https://www.xilinx.com/html_docs/registers/ug1087/crf_apb___gpu_ref_ctrl.html

Quoted 34 lines
Quoted 34 lines
---
   drivers/gpu/drm/lima/lima_devfreq.c | 17 +++++++++---
   drivers/gpu/drm/lima/lima_devfreq.h |  1 +
   drivers/gpu/drm/lima/lima_device.c  | 42 +++++++++++------------------
   drivers/gpu/drm/lima/lima_device.h  |  4 +--
   4 files changed, 32 insertions(+), 32 deletions(-)
drivers/gpu/drm/lima/lima_devfreq.c +2 -2
index 8989e215dfc9..533b36932f79 100644
@@ -58,7 +58,7 @@ static int lima_devfreq_get_dev_status(struct device *dev,
          struct lima_devfreq *devfreq = &ldev->devfreq;
          unsigned long irqflags;
 
-       status->current_frequency = clk_get_rate(ldev->clk_gpu);
+       status->current_frequency = clk_get_rate(devfreq->clk_gpu);
 
          spin_lock_irqsave(&devfreq->lock, irqflags);
 
@@ -110,12 +110,23 @@ int lima_devfreq_init(struct lima_device *ldev)
          struct lima_devfreq *ldevfreq = &ldev->devfreq;
          struct dev_pm_opp *opp;
          unsigned long cur_freq;
-       int ret;
+       int i, ret;
 
          if (!device_property_present(dev, "operating-points-v2"))
                  /* Optional, continue without devfreq */
                  return 0;
 
Quoted 11 lines

drivers/gpu/drm/lima/lima_devfreq.c int lima_devfreq_init(struct lima_device *ldev) · line 125

Quoted 7 lines
drivers/gpu/drm/lima/lima_devfreq.c +7 -0
+       /* Find first clock which are not "bus" clock */
+       for (i = 0; i < ldev->nr_clks; i++) {
+               if (!strcmp(ldev->clks[i].id, "bus"))
+                       continue;
+               ldevfreq->clk_gpu = ldev->clks[i].clk;
+               break;
+       }

I'd prefer an explicit name for the required clk name. If some DTS has different name other than "core" for the module clk (ie. "gpu"), it should be changed to "core".

The problem here is, the zynqmp has no core clock, it has "gpu and both pixel pipes" super-clock-gate which controls everything, and then per-pixel-pipe sub-clock-gates.

Quoted 8 lines
Quoted 1 line

So the "gpu" clk can gate both "gpu_pp0" and "gpu_pp1" clk, how about frequency?

I don't think it is a good idea to just gate off the root clock while the sub-clock are still enabled. That might lead to latch ups (+CC Michal, he might know more).

And who would enable the sub-clock anyway, it should be the GPU driver, no?

Right, I understand it's not proper either by HW or SW point of view to just use root clk gate.

Quoted 6 lines
Quoted 3 lines

Can we set clock rate for "gpu" then "gpu_pp0" and "gpu_pp1" pass through the same
rate? If so, "gpu" works just like "core".

I don't think the zynqmp is capable of any DVFS on the GPU at all, it just runs at fixed frequency.

I see the GPU_REF_CTRL register 13:8 is a divisor, is this for all "gpu"/"gpu_pp0"/"gpu_pp1" clk rating? If so, can we use it to dynamically change the GPU clk freq because other SoC also use system clock to do GPU DVFS, see sun8i-h3.dtsi. If we can't then zynqmp won't finish lima_devfreq_init() and get here at all because it does not have an OPP table.

#6 lore raw
Quoted 97 lines · Qiang Yu
Quoted 78 lines · Marek Vasut
Quoted 76 lines · Qiang Yu
Quoted 74 lines · Marek Vasut
Quoted 10 lines · Qiang Yu
Quoted 6 lines · Marek Vasut

Instead of requesting two separate clock and then handling them separately in various places of the driver, use clk_bulk_*() API. This permits handling devices with more than "bus"/"core" clock, like ZynqMP, which has "gpu"/"gpu_pp0"/"gpu_pp1" all as separate clock.

I can't find the ZynqMP DTS file under arch/arm64/boot/dts/xilinx which has mali GPU node with an upstream kernel, where is it?

Posted here:
https://patchwork.kernel.org/project/linux-arm-kernel/patch/[email protected]/

Quoted 3 lines

So what's the relationship between "gpu" clk and "gpu_pp0"/"gpu_pp1" clk? Do they need to be controlled separately or we can just control the "gpu" clk? Because the devfreq code just controls a single module clk.

Per the docs, they are separate enable bits and the zynqmp clock controller exports them as separate clock, see bits 24..26 here:

https://www.xilinx.com/html_docs/registers/ug1087/crf_apb___gpu_ref_ctrl.html

Quoted 34 lines
Quoted 34 lines
---
   drivers/gpu/drm/lima/lima_devfreq.c | 17 +++++++++---
   drivers/gpu/drm/lima/lima_devfreq.h |  1 +
   drivers/gpu/drm/lima/lima_device.c  | 42 +++++++++++------------------
   drivers/gpu/drm/lima/lima_device.h  |  4 +--
   4 files changed, 32 insertions(+), 32 deletions(-)
drivers/gpu/drm/lima/lima_devfreq.c +2 -2
index 8989e215dfc9..533b36932f79 100644
@@ -58,7 +58,7 @@ static int lima_devfreq_get_dev_status(struct device *dev,
          struct lima_devfreq *devfreq = &ldev->devfreq;
          unsigned long irqflags;
 
-       status->current_frequency = clk_get_rate(ldev->clk_gpu);
+       status->current_frequency = clk_get_rate(devfreq->clk_gpu);
 
          spin_lock_irqsave(&devfreq->lock, irqflags);
 
@@ -110,12 +110,23 @@ int lima_devfreq_init(struct lima_device *ldev)
          struct lima_devfreq *ldevfreq = &ldev->devfreq;
          struct dev_pm_opp *opp;
          unsigned long cur_freq;
-       int ret;
+       int i, ret;
 
          if (!device_property_present(dev, "operating-points-v2"))
                  /* Optional, continue without devfreq */
                  return 0;
 
Quoted 11 lines

drivers/gpu/drm/lima/lima_devfreq.c int lima_devfreq_init(struct lima_device *ldev) · line 125

Quoted 7 lines
drivers/gpu/drm/lima/lima_devfreq.c +7 -0
+       /* Find first clock which are not "bus" clock */
+       for (i = 0; i < ldev->nr_clks; i++) {
+               if (!strcmp(ldev->clks[i].id, "bus"))
+                       continue;
+               ldevfreq->clk_gpu = ldev->clks[i].clk;
+               break;
+       }

I'd prefer an explicit name for the required clk name. If some DTS has different name other than "core" for the module clk (ie. "gpu"), it should be changed to "core".

The problem here is, the zynqmp has no core clock, it has "gpu and both pixel pipes" super-clock-gate which controls everything, and then per-pixel-pipe sub-clock-gates.

Quoted 8 lines
Quoted 1 line

So the "gpu" clk can gate both "gpu_pp0" and "gpu_pp1" clk, how about frequency?

I don't think it is a good idea to just gate off the root clock while the sub-clock are still enabled. That might lead to latch ups (+CC Michal, he might know more).

And who would enable the sub-clock anyway, it should be the GPU driver, no?

Right, I understand it's not proper either by HW or SW point of view to just use root clk gate.

Quoted 6 lines
Quoted 3 lines

Can we set clock rate for "gpu" then "gpu_pp0" and "gpu_pp1" pass through the same
rate? If so, "gpu" works just like "core".

I don't think the zynqmp is capable of any DVFS on the GPU at all, it just runs at fixed frequency.

Quoted 7 lines

I see the GPU_REF_CTRL register 13:8 is a divisor, is this for all "gpu"/"gpu_pp0"/"gpu_pp1" clk rating? If so, can we use it to dynamically change the GPU clk freq because other SoC also use system clock to do GPU DVFS, see sun8i-h3.dtsi. If we can't then zynqmp won't finish lima_devfreq_init() and get here at all because it does not have an OPP table.

Jianqiang: Please take a look at this from zynqmp point of view.

Thanks,
Michal

#7 lore raw

Add Mads.

Thanks,
Jason

Quoted 129 lines

-----Original Message-----
From: Michal Simek <[email protected]>
Sent: Monday, July 19, 2021 12:53 AM
To: Qiang Yu <[email protected]>; Marek Vasut <[email protected]>; Jianqiang Chen <[email protected]>
Cc: dri-devel <[email protected]>; [email protected]; Michal Simek <[email protected]>; Michal Simek <[email protected]>
Subject: Re: [PATCH] drm/lima: Convert to clk_bulk API

Quoted 14 lines · Qiang Yu
Quoted 13 lines · Marek Vasut
Quoted 11 lines · Qiang Yu
Quoted 10 lines · Marek Vasut
Quoted 8 lines · Qiang Yu
Quoted 6 lines · Marek Vasut

Instead of requesting two separate clock and then handling them separately in various places of the driver, use clk_bulk_*() API. This permits handling devices with more than "bus"/"core" clock, like ZynqMP, which has "gpu"/"gpu_pp0"/"gpu_pp1" all as separate clock.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines

I can't find the ZynqMP DTS file under arch/arm64/boot/dts/xilinx which has mali GPU node with an upstream kernel, where is it?

Posted here:
https://patchwork.kernel.org/project/linux-arm-kernel/patch/2021071
[email protected]/

Quoted 1 line

So what's the relationship between "gpu" clk and

"gpu_pp0"/"gpu_pp1"

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

clk? Do they need to be controlled separately or we can just control the "gpu" clk? Because the devfreq code just controls a single

module clk.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Per the docs, they are separate enable bits and the zynqmp clock controller exports them as separate clock, see bits 24..26 here:

https://www.xilinx.com/html_docs/registers/ug1087/crf_apb___gpu_ref

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

_ctrl.html

Quoted 7 lines
Quoted 7 lines
---
   drivers/gpu/drm/lima/lima_devfreq.c | 17 +++++++++---
   drivers/gpu/drm/lima/lima_devfreq.h |  1 +
   drivers/gpu/drm/lima/lima_device.c  | 42 +++++++++++-------------

-----

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

drivers/gpu/drm/lima/lima_devfreq.c static int lima_devfreq_get_dev_status(struct

Quoted 9 lines
   drivers/gpu/drm/lima/lima_device.h  |  4 +--
   4 files changed, 32 insertions(+), 32 deletions(-)
drivers/gpu/drm/lima/lima_devfreq.c +0 -0
index 8989e215dfc9..533b36932f79 100644
@@ -58,7 +58,7 @@ static int lima_devfreq_get_dev_status(struct

device *dev,

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

drivers/gpu/drm/lima/lima_devfreq.c int lima_devfreq_init(struct lima_device

Quoted 10 lines
drivers/gpu/drm/lima/lima_devfreq.c +2 -1
          struct lima_devfreq *devfreq = &ldev->devfreq;
          unsigned long irqflags;
 
-       status->current_frequency = clk_get_rate(ldev->clk_gpu);
+       status->current_frequency =
+ clk_get_rate(devfreq->clk_gpu);
 
          spin_lock_irqsave(&devfreq->lock, irqflags);
@@ -110,12 +110,23 @@ int lima_devfreq_init(struct lima_device

*ldev)

Quoted 22 lines
Quoted 22 lines
Quoted 22 lines
Quoted 22 lines
Quoted 21 lines
Quoted 7 lines
drivers/gpu/drm/lima/lima_devfreq.c +1 -1
          struct lima_devfreq *ldevfreq = &ldev->devfreq;
          struct dev_pm_opp *opp;
          unsigned long cur_freq;
-       int ret;
+       int i, ret;
 
          if (!device_property_present(dev, "operating-points-v2"))

drivers/gpu/drm/lima/lima_devfreq.c int lima_devfreq_init(struct lima_device · line 125

Quoted 10 lines
drivers/gpu/drm/lima/lima_devfreq.c +7 -0
                  /* Optional, continue without devfreq */
                  return 0;
 
+       /* Find first clock which are not "bus" clock */
+       for (i = 0; i < ldev->nr_clks; i++) {
+               if (!strcmp(ldev->clks[i].id, "bus"))
+                       continue;
+               ldevfreq->clk_gpu = ldev->clks[i].clk;
+               break;
+       }

I'd prefer an explicit name for the required clk name. If some DTS has different name other than "core" for the module clk (ie. "gpu"), it should be changed to "core".

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

The problem here is, the zynqmp has no core clock, it has "gpu and both pixel pipes" super-clock-gate which controls everything, and then per-pixel-pipe sub-clock-gates.

So the "gpu" clk can gate both "gpu_pp0" and "gpu_pp1" clk, how about

frequency?

Quoted 6 lines
Quoted 6 lines

I don't think it is a good idea to just gate off the root clock while the sub-clock are still enabled. That might lead to latch ups (+CC Michal, he might know more).

And who would enable the sub-clock anyway, it should be the GPU driver,

no?

Quoted 10 lines
Quoted 1 line

Right, I understand it's not proper either by HW or SW point of view to just use root clk gate.

Quoted 5 lines
Quoted 2 lines

Can we set clock rate for "gpu" then "gpu_pp0" and "gpu_pp1" pass through the same rate? If so, "gpu" works just like "core".

I don't think the zynqmp is capable of any DVFS on the GPU at all, it just runs at fixed frequency.

Quoted 8 lines

I see the GPU_REF_CTRL register 13:8 is a divisor, is this for all "gpu"/"gpu_pp0"/"gpu_pp1" clk rating? If so, can we use it to dynamically change the GPU clk freq because other SoC also use system clock to do GPU DVFS, see sun8i-h3.dtsi. If we can't then zynqmp won't finish
lima_devfreq_init() and get here at all because it does not have an OPP table.

Quoted 1 line

Jianqiang: Please take a look at this from zynqmp point of view.

Hi Mads, can you comment on this?

Quoted 3 lines

Thanks,
Michal

#8 lore raw

Hi,

I had created a patch sometimes ago to test on our platform. That is correct, we have three clocks going to gp,pp0 and pp1 (Although all are at same rate). DVFS is not supported primarily because at zynqmp clocks are shared, for example it could come from VPLL or IOPLL.

All zynqmp specific patches can be found here: https://github.com/Xilinx/meta-xilinx/tree/rel-v2021.1/meta-xilinx-bsp/recipes-graphics/mali/kernel-module-mali

-Mads

On 7/19/21, 10:38 AM, "Jianqiang Chen" <[email protected]> wrote:

    Add Mads.

    Thanks,
    Jason

    > -----Original Message-----
    > From: Michal Simek <[email protected]>
    > Sent: Monday, July 19, 2021 12:53 AM
    > To: Qiang Yu <[email protected]>; Marek Vasut <[email protected]>;
    > Jianqiang Chen <[email protected]>
    > Cc: dri-devel <[email protected]>; [email protected];
    > Michal Simek <[email protected]>; Michal Simek <[email protected]>
    > Subject: Re: [PATCH] drm/lima: Convert to clk_bulk API
    > 
    > 
    > 
    > On 7/18/21 4:56 AM, Qiang Yu wrote:
    > > On Sat, Jul 17, 2021 at 10:52 PM Marek Vasut <[email protected]> wrote:
    > >>
    > >> On 7/17/21 4:21 PM, Qiang Yu wrote:
    > >>> On Sat, Jul 17, 2021 at 9:08 PM Marek Vasut <[email protected]> wrote:
    > >>>>
    > >>>> On 7/17/21 2:34 PM, Qiang Yu wrote:
    > >>>>> On Sat, Jul 17, 2021 at 2:20 AM Marek Vasut <[email protected]> wrote:
    > >>>>>>
    > >>>>>> Instead of requesting two separate clock and then handling them
    > >>>>>> separately in various places of the driver, use clk_bulk_*() API.
    > >>>>>> This permits handling devices with more than "bus"/"core" clock,
    > >>>>>> like ZynqMP, which has "gpu"/"gpu_pp0"/"gpu_pp1" all as separate
    > >>>>>> clock.
    > >>>>>
    > >>>>> I can't find the ZynqMP DTS file under arch/arm64/boot/dts/xilinx
    > >>>>> which has mali GPU node with an upstream kernel, where is it?
    > >>>>
    > >>>> Posted here:
    > >>>> https://patchwork.kernel.org/project/linux-arm-kernel/patch/2021071
    > >>>> [email protected]/
    > >>>>
    > >>>>> So what's the relationship between "gpu" clk and
    > "gpu_pp0"/"gpu_pp1"
    > >>>>> clk? Do they need to be controlled separately or we can just
    > >>>>> control the "gpu" clk? Because the devfreq code just controls a single
    > module clk.
    > >>>>
    > >>>> Per the docs, they are separate enable bits and the zynqmp clock
    > >>>> controller exports them as separate clock, see bits 24..26 here:
    > >>>>
    > >>>>
    > https://www.xilinx.com/html_docs/registers/ug1087/crf_apb___gpu_ref
    > >>>> _ctrl.html
    > >>>>
    > >>>>>> Signed-off-by: Marek Vasut <[email protected]>
    > >>>>>> Cc: Qiang Yu <[email protected]>
    > >>>>>> Cc: [email protected]
    > >>>>>> ---
    > >>>>>>    drivers/gpu/drm/lima/lima_devfreq.c | 17 +++++++++---
    > >>>>>>    drivers/gpu/drm/lima/lima_devfreq.h |  1 +
    > >>>>>>    drivers/gpu/drm/lima/lima_device.c  | 42 +++++++++++-------------
    > -----
    > >>>>>>    drivers/gpu/drm/lima/lima_device.h  |  4 +--
    > >>>>>>    4 files changed, 32 insertions(+), 32 deletions(-)
    > >>>>>>
    > >>>>>> diff --git a/drivers/gpu/drm/lima/lima_devfreq.c
    > >>>>>> b/drivers/gpu/drm/lima/lima_devfreq.c
    > >>>>>> index 8989e215dfc9..533b36932f79 100644
    > >>>>>> --- a/drivers/gpu/drm/lima/lima_devfreq.c
    > >>>>>> +++ b/drivers/gpu/drm/lima/lima_devfreq.c
    > >>>>>> @@ -58,7 +58,7 @@ static int lima_devfreq_get_dev_status(struct
    > device *dev,
    > >>>>>>           struct lima_devfreq *devfreq = &ldev->devfreq;
    > >>>>>>           unsigned long irqflags;
    > >>>>>>
    > >>>>>> -       status->current_frequency = clk_get_rate(ldev->clk_gpu);
    > >>>>>> +       status->current_frequency =
    > >>>>>> + clk_get_rate(devfreq->clk_gpu);
    > >>>>>>
    > >>>>>>           spin_lock_irqsave(&devfreq->lock, irqflags);
    > >>>>>>
    > >>>>>> @@ -110,12 +110,23 @@ int lima_devfreq_init(struct lima_device
    > *ldev)
    > >>>>>>           struct lima_devfreq *ldevfreq = &ldev->devfreq;
    > >>>>>>           struct dev_pm_opp *opp;
    > >>>>>>           unsigned long cur_freq;
    > >>>>>> -       int ret;
    > >>>>>> +       int i, ret;
    > >>>>>>
    > >>>>>>           if (!device_property_present(dev, "operating-points-v2"))
    > >>>>>>                   /* Optional, continue without devfreq */
    > >>>>>>                   return 0;
    > >>>>>>
    > >>>>>> +       /* Find first clock which are not "bus" clock */
    > >>>>>> +       for (i = 0; i < ldev->nr_clks; i++) {
    > >>>>>> +               if (!strcmp(ldev->clks[i].id, "bus"))
    > >>>>>> +                       continue;
    > >>>>>> +               ldevfreq->clk_gpu = ldev->clks[i].clk;
    > >>>>>> +               break;
    > >>>>>> +       }
    > >>>>>
    > >>>>> I'd prefer an explicit name for the required clk name. If some DTS
    > >>>>> has different name other than "core" for the module clk (ie.
    > >>>>> "gpu"), it should be changed to "core".
    > >>>>
    > >>>> The problem here is, the zynqmp has no core clock, it has "gpu and
    > >>>> both pixel pipes" super-clock-gate which controls everything, and
    > >>>> then per-pixel-pipe sub-clock-gates.
    > >>>
    > >>> So the "gpu" clk can gate both "gpu_pp0" and "gpu_pp1" clk, how about
    > frequency?
    > >>
    > >> I don't think it is a good idea to just gate off the root clock while
    > >> the sub-clock are still enabled. That might lead to latch ups (+CC
    > >> Michal, he might know more).
    > >>
    > >> And who would enable the sub-clock anyway, it should be the GPU driver,
    > no?
    > >>
    > > Right, I understand it's not proper either by HW or SW point of view
    > > to just use root clk gate.
    > >
    > >>> Can we set clock rate for "gpu" then "gpu_pp0" and "gpu_pp1" pass
    > >>> through the same rate? If so, "gpu" works just like "core".
    > >>
    > >> I don't think the zynqmp is capable of any DVFS on the GPU at all, it
    > >> just runs at fixed frequency.
    > >
    > > I see the GPU_REF_CTRL register 13:8 is a divisor, is this for all
    > > "gpu"/"gpu_pp0"/"gpu_pp1" clk rating? If so, can we use it to
    > > dynamically change the GPU clk freq because other SoC also use system
    > > clock to do GPU DVFS, see sun8i-h3.dtsi. If we can't then zynqmp won't
    > > finish
    > > lima_devfreq_init() and get here at all because it does not have an
    > > OPP table.
    > >
    > 
    > Jianqiang: Please take a look at this from zynqmp point of view.

    Hi Mads, can you comment on this?

    > 
    > Thanks,
    > Michal

From cc3ec45f39be1956698c9bc41b0c100e6320ece1 Mon Sep 17 00:00:00 2001
From: Madhurkiran Harikrishnan <[email protected]>
Date: Tue, 11 Aug 2020 11:27:34 -0700
Subject: [PATCH 1/2] lima: Add ZynqMP specific changes for Lima

This patch does following Xilinx specific modifications:
1) Add zynqmp specific clock support for GP, PP0 and PP1.
2) Update the device tree to use proper interrupt names.
3) Do not initialize the regulators for zynqmp devices.

---
 arch/arm64/boot/dts/xilinx/zynqmp.dtsi |  2 +-
 drivers/gpu/drm/lima/lima_device.c     | 58 ++++++++++++++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_device.h     |  6 ++++
 3 files changed, 65 insertions(+), 1 deletion(-)
arch/arm64/boot/dts/xilinx/zynqmp.dtsi +1 -1
index b0b306e..97e7762 100644
@@ -462,7 +462,7 @@
 			reg = <0x0 0xfd4b0000 0x0 0x10000>;
 			interrupt-parent = <&gic>;
 			interrupts = <0 132 4>, <0 132 4>, <0 132 4>, <0 132 4>, <0 132 4>, <0 132 4>;
-			interrupt-names = "IRQGP", "IRQGPMMU", "IRQPP0", "IRQPPMMU0", "IRQPP1", "IRQPPMMU1";
+			interrupt-names = "gp", "gpmmu", "pp0", "ppmmu0", "pp1", "ppmmu1";
 			clock-names = "gpu", "gpu_pp0", "gpu_pp1";
 			power-domains = <&zynqmp_firmware PD_GPU>;
 		};
drivers/gpu/drm/lima/lima_device.c +58 -0

95 lines not on this page · raw message

drivers/gpu/drm/lima/lima_device.h +6 -0
index 31158d8..fad2b15 100644
@@ -80,8 +80,14 @@ struct lima_device {
 	int num_pp;
 
 	void __iomem *iomem;
+#if defined(CONFIG_ARCH_ZYNQMP)
+	struct clk *clk_gpu_gp;
+	struct clk *clk_gpu_pp0;
+	struct clk *clk_gpu_pp1;
+#else
 	struct clk *clk_bus;
 	struct clk *clk_gpu;
+#endif
 	struct reset_control *reset;
 	struct regulator *regulator;
 
Signature · 1 line
2.7.4
maintainer
#9 lore raw
Quoted 5 lines · Madhurkiran Harikrishnan

Hi,

I had created a patch sometimes ago to test on our platform. That is correct, we have three clocks going to gp,pp0 and pp1 (Although all are at same rate). DVFS is not supported primarily because at zynqmp clocks are shared, for example it could come from VPLL or IOPLL.

It's not necessary to change the root PLL clock rate to support DVFS. You can also change a divisor to a sub clock if the sub clock is not shared by another module. For example if root PLL is 700MHz, you can set the divisor to GPU clock to 2 to make GPU run at 350MHz. My question is if the GPU_REF_CTRL[13:8] is just such a divisor and can be used to support DVFS (an OPP table in DTS)?

Quoted 148 lines

All zynqmp specific patches can be found here: https://github.com/Xilinx/meta-xilinx/tree/rel-v2021.1/meta-xilinx-bsp/recipes-graphics/mali/kernel-module-mali

-Mads

On 7/19/21, 10:38 AM, "Jianqiang Chen" <[email protected]> wrote:

    Add Mads.

    Thanks,
    Jason

    > -----Original Message-----
    > From: Michal Simek <[email protected]>
    > Sent: Monday, July 19, 2021 12:53 AM
    > To: Qiang Yu <[email protected]>; Marek Vasut <[email protected]>;
    > Jianqiang Chen <[email protected]>
    > Cc: dri-devel <[email protected]>; [email protected];
    > Michal Simek <[email protected]>; Michal Simek <[email protected]>
    > Subject: Re: [PATCH] drm/lima: Convert to clk_bulk API
    >
    >
    >
    > On 7/18/21 4:56 AM, Qiang Yu wrote:
    > > On Sat, Jul 17, 2021 at 10:52 PM Marek Vasut <[email protected]> wrote:
    > >>
    > >> On 7/17/21 4:21 PM, Qiang Yu wrote:
    > >>> On Sat, Jul 17, 2021 at 9:08 PM Marek Vasut <[email protected]> wrote:
    > >>>>
    > >>>> On 7/17/21 2:34 PM, Qiang Yu wrote:
    > >>>>> On Sat, Jul 17, 2021 at 2:20 AM Marek Vasut <[email protected]> wrote:
    > >>>>>>
    > >>>>>> Instead of requesting two separate clock and then handling them
    > >>>>>> separately in various places of the driver, use clk_bulk_*() API.
    > >>>>>> This permits handling devices with more than "bus"/"core" clock,
    > >>>>>> like ZynqMP, which has "gpu"/"gpu_pp0"/"gpu_pp1" all as separate
    > >>>>>> clock.
    > >>>>>
    > >>>>> I can't find the ZynqMP DTS file under arch/arm64/boot/dts/xilinx
    > >>>>> which has mali GPU node with an upstream kernel, where is it?
    > >>>>
    > >>>> Posted here:
    > >>>> https://patchwork.kernel.org/project/linux-arm-kernel/patch/2021071
    > >>>> [email protected]/
    > >>>>
    > >>>>> So what's the relationship between "gpu" clk and
    > "gpu_pp0"/"gpu_pp1"
    > >>>>> clk? Do they need to be controlled separately or we can just
    > >>>>> control the "gpu" clk? Because the devfreq code just controls a single
    > module clk.
    > >>>>
    > >>>> Per the docs, they are separate enable bits and the zynqmp clock
    > >>>> controller exports them as separate clock, see bits 24..26 here:
    > >>>>
    > >>>>
    > https://www.xilinx.com/html_docs/registers/ug1087/crf_apb___gpu_ref
    > >>>> _ctrl.html
    > >>>>
    > >>>>>> Signed-off-by: Marek Vasut <[email protected]>
    > >>>>>> Cc: Qiang Yu <[email protected]>
    > >>>>>> Cc: [email protected]
    > >>>>>> ---
    > >>>>>>    drivers/gpu/drm/lima/lima_devfreq.c | 17 +++++++++---
    > >>>>>>    drivers/gpu/drm/lima/lima_devfreq.h |  1 +
    > >>>>>>    drivers/gpu/drm/lima/lima_device.c  | 42 +++++++++++-------------
    > -----
    > >>>>>>    drivers/gpu/drm/lima/lima_device.h  |  4 +--
    > >>>>>>    4 files changed, 32 insertions(+), 32 deletions(-)
    > >>>>>>
    > >>>>>> diff --git a/drivers/gpu/drm/lima/lima_devfreq.c
    > >>>>>> b/drivers/gpu/drm/lima/lima_devfreq.c
    > >>>>>> index 8989e215dfc9..533b36932f79 100644
    > >>>>>> --- a/drivers/gpu/drm/lima/lima_devfreq.c
    > >>>>>> +++ b/drivers/gpu/drm/lima/lima_devfreq.c
    > >>>>>> @@ -58,7 +58,7 @@ static int lima_devfreq_get_dev_status(struct
    > device *dev,
    > >>>>>>           struct lima_devfreq *devfreq = &ldev->devfreq;
    > >>>>>>           unsigned long irqflags;
    > >>>>>>
    > >>>>>> -       status->current_frequency = clk_get_rate(ldev->clk_gpu);
    > >>>>>> +       status->current_frequency =
    > >>>>>> + clk_get_rate(devfreq->clk_gpu);
    > >>>>>>
    > >>>>>>           spin_lock_irqsave(&devfreq->lock, irqflags);
    > >>>>>>
    > >>>>>> @@ -110,12 +110,23 @@ int lima_devfreq_init(struct lima_device
    > *ldev)
    > >>>>>>           struct lima_devfreq *ldevfreq = &ldev->devfreq;
    > >>>>>>           struct dev_pm_opp *opp;
    > >>>>>>           unsigned long cur_freq;
    > >>>>>> -       int ret;
    > >>>>>> +       int i, ret;
    > >>>>>>
    > >>>>>>           if (!device_property_present(dev, "operating-points-v2"))
    > >>>>>>                   /* Optional, continue without devfreq */
    > >>>>>>                   return 0;
    > >>>>>>
    > >>>>>> +       /* Find first clock which are not "bus" clock */
    > >>>>>> +       for (i = 0; i < ldev->nr_clks; i++) {
    > >>>>>> +               if (!strcmp(ldev->clks[i].id, "bus"))
    > >>>>>> +                       continue;
    > >>>>>> +               ldevfreq->clk_gpu = ldev->clks[i].clk;
    > >>>>>> +               break;
    > >>>>>> +       }
    > >>>>>
    > >>>>> I'd prefer an explicit name for the required clk name. If some DTS
    > >>>>> has different name other than "core" for the module clk (ie.
    > >>>>> "gpu"), it should be changed to "core".
    > >>>>
    > >>>> The problem here is, the zynqmp has no core clock, it has "gpu and
    > >>>> both pixel pipes" super-clock-gate which controls everything, and
    > >>>> then per-pixel-pipe sub-clock-gates.
    > >>>
    > >>> So the "gpu" clk can gate both "gpu_pp0" and "gpu_pp1" clk, how about
    > frequency?
    > >>
    > >> I don't think it is a good idea to just gate off the root clock while
    > >> the sub-clock are still enabled. That might lead to latch ups (+CC
    > >> Michal, he might know more).
    > >>
    > >> And who would enable the sub-clock anyway, it should be the GPU driver,
    > no?
    > >>
    > > Right, I understand it's not proper either by HW or SW point of view
    > > to just use root clk gate.
    > >
    > >>> Can we set clock rate for "gpu" then "gpu_pp0" and "gpu_pp1" pass
    > >>> through the same rate? If so, "gpu" works just like "core".
    > >>
    > >> I don't think the zynqmp is capable of any DVFS on the GPU at all, it
    > >> just runs at fixed frequency.
    > >
    > > I see the GPU_REF_CTRL register 13:8 is a divisor, is this for all
    > > "gpu"/"gpu_pp0"/"gpu_pp1" clk rating? If so, can we use it to
    > > dynamically change the GPU clk freq because other SoC also use system
    > > clock to do GPU DVFS, see sun8i-h3.dtsi. If we can't then zynqmp won't
    > > finish
    > > lima_devfreq_init() and get here at all because it does not have an
    > > OPP table.
    > >
    >
    > Jianqiang: Please take a look at this from zynqmp point of view.

    Hi Mads, can you comment on this?

    >
    > Thanks,
    > Michal