Skip to contentSkip to conversation

drm/msm: re-factor devfreq common code

PATCH Superseded Sharat Masetty 5 participants 8 messages May 25, 2018 – May 27, 2018 3 messages not in the archive

Subsystems (4)

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

cover drm/msm: re-factor devfreq common code

This series re-factors the devfreq code a bit in preparation for the upcoming A6x related devfreq changes. The code applies cleanly on 4.17 and has been verified on DB820C.

Sharat Masetty (3):
drm/msm: suspend devfreq on init
drm/msm: move suspend/resume devfreq to their own functions drm/msm: re-factor devfreq code

 drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 15 ++++++++---
 drivers/gpu/drm/msm/msm_gpu.c         | 51 +++++++++++++++++++++++------------
 drivers/gpu/drm/msm/msm_gpu.h         |  6 ++++-
 3 files changed, 50 insertions(+), 22 deletions(-)

--
1.9.1
_______________________________________________
Freedreno mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/freedreno

A message that is not in the archive (sent off-list or not yet received): 1527244246-10519-1-git-send-email-smasetty-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org

#2 lore raw

PATCH 1/3 drm/msm: suspend devfreq on init

Devfreq turns on and starts recommending power level as soon as it is initialized. The GPU is still not powered on by the time the devfreq init happens and this leads to problems on GPU's where register access is needed to get/set power levels. So we start suspended and only restart devfreq when GPU is powered on.

---
 drivers/gpu/drm/msm/msm_gpu.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/gpu/drm/msm/msm_gpu.c +2 -0
index 1c09acf..d7586f2 100644
@@ -104,6 +104,8 @@ static void msm_devfreq_init(struct msm_gpu *gpu)
 		dev_err(&gpu->pdev->dev, "Couldn't initialize GPU devfreq\n");
 		gpu->devfreq.devfreq = NULL;
 	}
+
+	devfreq_suspend_device(gpu->devfreq.devfreq);
 }
 
 static int enable_pwrrail(struct msm_gpu *gpu)
Signature · 6 lines
1.9.1

_______________________________________________
Freedreno mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/freedreno
Reviewed-by on v1 1/3
#3 lore raw

Re: [PATCH 1/3] drm/msm: suspend devfreq on init

Quoted 7 lines · Sharat Masetty

Devfreq turns on and starts recommending power level as soon as it is initialized. The GPU is still not powered on by the time the devfreq init happens and this leads to problems on GPU's where register access is needed to get/set power levels. So we start suspended and only restart devfreq when GPU is powered on.

Quoted 20 lines
---
 drivers/gpu/drm/msm/msm_gpu.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/gpu/drm/msm/msm_gpu.c +2 -0
index 1c09acf..d7586f2 100644
@@ -104,6 +104,8 @@ static void msm_devfreq_init(struct msm_gpu *gpu)
 		dev_err(&gpu->pdev->dev, "Couldn't initialize GPU devfreq\n");
 		gpu->devfreq.devfreq = NULL;
 	}
+
+	devfreq_suspend_device(gpu->devfreq.devfreq);
 }
 
 static int enable_pwrrail(struct msm_gpu *gpu)
Signature · 1 line
1.9.1
Signature · 6 lines
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#4 lore raw

PATCH 2/3 drm/msm: move suspend/resume devfreq to their own functions

In reply to a message not in the archive

This is needed for hardware revisions which do not rely on the generic suspend, resume handlers for power management.

---
 drivers/gpu/drm/msm/msm_gpu.c | 26 ++++++++++++++++++--------
 drivers/gpu/drm/msm/msm_gpu.h |  2 ++
 2 files changed, 20 insertions(+), 8 deletions(-)
drivers/gpu/drm/msm/msm_gpu.c +18 -8
index d7586f2..d8d4fc9 100644
@@ -203,6 +203,22 @@ static int disable_axi(struct msm_gpu *gpu)
 	return 0;
 }
 
+void msm_gpu_resume_devfreq(struct msm_gpu *gpu)
+{
+	if (gpu->devfreq.devfreq) {
+		gpu->devfreq.busy_cycles = 0;
+		gpu->devfreq.time = ktime_get();
+
+		devfreq_resume_device(gpu->devfreq.devfreq);
+	}
+}
+
+void msm_gpu_suspend_devfreq(struct msm_gpu *gpu)
+{
+	if (gpu->devfreq.devfreq)
+		devfreq_suspend_device(gpu->devfreq.devfreq);
+}
+
 int msm_gpu_pm_resume(struct msm_gpu *gpu)
 {
 	int ret;
@@ -221,12 +237,7 @@ int msm_gpu_pm_resume(struct msm_gpu *gpu)
 	if (ret)
 		return ret;
 
-	if (gpu->devfreq.devfreq) {
-		gpu->devfreq.busy_cycles = 0;
-		gpu->devfreq.time = ktime_get();
-
-		devfreq_resume_device(gpu->devfreq.devfreq);
-	}
+	msm_gpu_resume_devfreq(gpu);
 
 	gpu->needs_hw_init = true;
 
@@ -239,8 +250,7 @@ int msm_gpu_pm_suspend(struct msm_gpu *gpu)
 
 	DBG("%s", gpu->name);
 
-	if (gpu->devfreq.devfreq)
-		devfreq_suspend_device(gpu->devfreq.devfreq);
+	msm_gpu_suspend_devfreq(gpu);
 
 	ret = disable_axi(gpu);
 	if (ret)
drivers/gpu/drm/msm/msm_gpu.h +2 -0
index b824117..1876b81 100644
@@ -226,6 +226,8 @@ static inline void gpu_write64(struct msm_gpu *gpu, u32 lo, u32 hi, u64 val)
 
 int msm_gpu_pm_suspend(struct msm_gpu *gpu);
 int msm_gpu_pm_resume(struct msm_gpu *gpu);
+void msm_gpu_resume_devfreq(struct msm_gpu *gpu);
+void msm_gpu_suspend_devfreq(struct msm_gpu *gpu);
 
 int msm_gpu_hw_init(struct msm_gpu *gpu);
 
Signature · 6 lines
1.9.1

_______________________________________________
Freedreno mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/freedreno

A message that is not in the archive (sent off-list or not yet received): 1527244246-10519-3-git-send-email-smasetty-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org

#6 lore raw

Re: [PATCH 2/3] drm/msm: move suspend/resume devfreq to their own functions

Quoted 15 lines · Sharat Masetty

This is needed for hardware revisions which do not rely on the generic suspend, resume handlers for power management.

---
 drivers/gpu/drm/msm/msm_gpu.c | 26 ++++++++++++++++++--------
 drivers/gpu/drm/msm/msm_gpu.h |  2 ++
 2 files changed, 20 insertions(+), 8 deletions(-)
drivers/gpu/drm/msm/msm_gpu.c +0 -0
index d7586f2..d8d4fc9 100644
@@ -203,6 +203,22 @@ static int disable_axi(struct msm_gpu *gpu)
 	return 0;

drivers/gpu/drm/msm/msm_gpu.c static int disable_axi(struct msm_gpu *gpu) · line 213

Quoted 10 lines
drivers/gpu/drm/msm/msm_gpu.c +8 -0
 }
 
+void msm_gpu_resume_devfreq(struct msm_gpu *gpu)
+{
+	if (gpu->devfreq.devfreq) {
+		gpu->devfreq.busy_cycles = 0;
+		gpu->devfreq.time = ktime_get();
+
+		devfreq_resume_device(gpu->devfreq.devfreq);
+	}

I just looked, we don't need the if() check - devfreq_resume_device should be safe. Yeah, we're doing an extra ktime_get() but its not the end of the world.

drivers/gpu/drm/msm/msm_gpu.c static int disable_axi(struct msm_gpu *gpu) · line 219

Quoted 6 lines
drivers/gpu/drm/msm/msm_gpu.c +6 -0
+}
+
+void msm_gpu_suspend_devfreq(struct msm_gpu *gpu)
+{
+	if (gpu->devfreq.devfreq)
+		devfreq_suspend_device(gpu->devfreq.devfreq);

We don't need the if check here either.

Quoted 45 lines
drivers/gpu/drm/msm/msm_gpu.c +4 -8
+}
+
 int msm_gpu_pm_resume(struct msm_gpu *gpu)
 {
 	int ret;
@@ -221,12 +237,7 @@ int msm_gpu_pm_resume(struct msm_gpu *gpu)
 	if (ret)
 		return ret;
 
-	if (gpu->devfreq.devfreq) {
-		gpu->devfreq.busy_cycles = 0;
-		gpu->devfreq.time = ktime_get();
-
-		devfreq_resume_device(gpu->devfreq.devfreq);
-	}
+	msm_gpu_resume_devfreq(gpu);
 
 	gpu->needs_hw_init = true;
 
@@ -239,8 +250,7 @@ int msm_gpu_pm_suspend(struct msm_gpu *gpu)
 
 	DBG("%s", gpu->name);
 
-	if (gpu->devfreq.devfreq)
-		devfreq_suspend_device(gpu->devfreq.devfreq);
+	msm_gpu_suspend_devfreq(gpu);
 
 	ret = disable_axi(gpu);
 	if (ret)
drivers/gpu/drm/msm/msm_gpu.h +2 -0
index b824117..1876b81 100644
@@ -226,6 +226,8 @@ static inline void gpu_write64(struct msm_gpu *gpu, u32 lo, u32 hi, u64 val)
 
 int msm_gpu_pm_suspend(struct msm_gpu *gpu);
 int msm_gpu_pm_resume(struct msm_gpu *gpu);
+void msm_gpu_resume_devfreq(struct msm_gpu *gpu);
+void msm_gpu_suspend_devfreq(struct msm_gpu *gpu);
 
 int msm_gpu_hw_init(struct msm_gpu *gpu);
 
Signature · 1 line
1.9.1
Signature · 6 lines
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project
_______________________________________________
Freedreno mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/freedreno
#7 lore raw

PATCH 3/3 drm/msm: re-factor devfreq code

In reply to Sharat Masetty

devfreq framework requires the drivers to provide busy time estimations. The GPU driver relies on the hardware performance counters for the busy time estimations, but different hardware revisions have counters which can be sourced from different clocks. So the busy time estimation will be target dependent. Additionally on targets where the clocks are completely controlled by the on chip microcontroller, fetching and setting the current GPU frequency will be different. This patch aims to embrace these differences by re-factoring the devfreq code a bit.

---
 drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 15 +++++++++++----
 drivers/gpu/drm/msm/msm_gpu.c         | 23 ++++++++++++++---------
 drivers/gpu/drm/msm/msm_gpu.h         |  4 +++-
 3 files changed, 28 insertions(+), 14 deletions(-)
drivers/gpu/drm/msm/adreno/a5xx_gpu.c +11 -4
index d39400e..a35a840 100644
@@ -1219,12 +1219,19 @@ static struct msm_ringbuffer *a5xx_active_ring(struct msm_gpu *gpu)
 	return a5xx_gpu->cur_ring;
 }
 
-static int a5xx_gpu_busy(struct msm_gpu *gpu, uint64_t *value)
+static u64 a5xx_gpu_busy(struct msm_gpu *gpu)
 {
-	*value = gpu_read64(gpu, REG_A5XX_RBBM_PERFCTR_RBBM_0_LO,
-		REG_A5XX_RBBM_PERFCTR_RBBM_0_HI);
+	u64 busy_cycles, busy_time;
 
-	return 0;
+	busy_cycles = gpu_read64(gpu, REG_A5XX_RBBM_PERFCTR_RBBM_0_LO,
+			REG_A5XX_RBBM_PERFCTR_RBBM_0_HI);
+
+	busy_time = (busy_cycles - gpu->devfreq.busy_cycles) /
+		(clk_get_rate(gpu->core_clk) / 1000000);
+
+	gpu->devfreq.busy_cycles = busy_cycles;
+
+	return busy_time;
 }
 
 static const struct adreno_gpu_funcs funcs = {
drivers/gpu/drm/msm/msm_gpu.c +14 -9
index d8d4fc9..f437ee8 100644
@@ -40,7 +40,11 @@ static int msm_devfreq_target(struct device *dev, unsigned long *freq,
 	if (IS_ERR(opp))
 		return PTR_ERR(opp);
 
-	clk_set_rate(gpu->core_clk, *freq);
+	if (gpu->funcs->gpu_set_freq)
+		gpu->funcs->gpu_set_freq(gpu, (u64)*freq);
+	else
+		clk_set_rate(gpu->core_clk, *freq);
+
 	dev_pm_opp_put(opp);
 
 	return 0;
@@ -50,16 +54,14 @@ static int msm_devfreq_get_dev_status(struct device *dev,
 		struct devfreq_dev_status *status)
 {
 	struct msm_gpu *gpu = platform_get_drvdata(to_platform_device(dev));
-	u64 cycles;
-	u32 freq = ((u32) status->current_frequency) / 1000000;
 	ktime_t time;
 
-	status->current_frequency = (unsigned long) clk_get_rate(gpu->core_clk);
-	gpu->funcs->gpu_busy(gpu, &cycles);
-
-	status->busy_time = ((u32) (cycles - gpu->devfreq.busy_cycles)) / freq;
+	if (gpu->funcs->gpu_get_freq)
+		status->current_frequency = gpu->funcs->gpu_get_freq(gpu);
+	else
+		status->current_frequency = clk_get_rate(gpu->core_clk);
 
-	gpu->devfreq.busy_cycles = cycles;
+	status->busy_time = gpu->funcs->gpu_busy(gpu);
 
 	time = ktime_get();
 	status->total_time = ktime_us_delta(time, gpu->devfreq.time);
@@ -72,7 +74,10 @@ static int msm_devfreq_get_cur_freq(struct device *dev, unsigned long *freq)
 {
 	struct msm_gpu *gpu = platform_get_drvdata(to_platform_device(dev));
 
-	*freq = (unsigned long) clk_get_rate(gpu->core_clk);
+	if (gpu->funcs->gpu_get_freq)
+		*freq = gpu->funcs->gpu_get_freq(gpu);
+	else
+		*freq = clk_get_rate(gpu->core_clk);
 
 	return 0;
 }
drivers/gpu/drm/msm/msm_gpu.h +3 -1
index 1876b81..d3f02e7 100644
@@ -68,7 +68,9 @@ struct msm_gpu_funcs {
 	/* for generation specific debugfs: */
 	int (*debugfs_init)(struct msm_gpu *gpu, struct drm_minor *minor);
 #endif
-	int (*gpu_busy)(struct msm_gpu *gpu, uint64_t *value);
+	u64 (*gpu_busy)(struct msm_gpu *gpu);
+	u64 (*gpu_get_freq)(struct msm_gpu *gpu);
+	int (*gpu_set_freq)(struct msm_gpu *gpu, u64 freq);
 };
 
 struct msm_gpu {
Signature · 6 lines
1.9.1

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#8 lore raw

Re: [PATCH 3/3] drm/msm: re-factor devfreq code

Quoted 84 lines · Sharat Masetty

devfreq framework requires the drivers to provide busy time estimations. The GPU driver relies on the hardware performance counters for the busy time estimations, but different hardware revisions have counters which can be sourced from different clocks. So the busy time estimation will be target dependent. Additionally on targets where the clocks are completely controlled by the on chip microcontroller, fetching and setting the current GPU frequency will be different. This patch aims to embrace these differences by re-factoring the devfreq code a bit.

---
 drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 15 +++++++++++----
 drivers/gpu/drm/msm/msm_gpu.c         | 23 ++++++++++++++---------
 drivers/gpu/drm/msm/msm_gpu.h         |  4 +++-
 3 files changed, 28 insertions(+), 14 deletions(-)
drivers/gpu/drm/msm/adreno/a5xx_gpu.c +11 -4
index d39400e..a35a840 100644
@@ -1219,12 +1219,19 @@ static struct msm_ringbuffer *a5xx_active_ring(struct msm_gpu *gpu)
 	return a5xx_gpu->cur_ring;
 }
 
-static int a5xx_gpu_busy(struct msm_gpu *gpu, uint64_t *value)
+static u64 a5xx_gpu_busy(struct msm_gpu *gpu)
 {
-	*value = gpu_read64(gpu, REG_A5XX_RBBM_PERFCTR_RBBM_0_LO,
-		REG_A5XX_RBBM_PERFCTR_RBBM_0_HI);
+	u64 busy_cycles, busy_time;
 
-	return 0;
+	busy_cycles = gpu_read64(gpu, REG_A5XX_RBBM_PERFCTR_RBBM_0_LO,
+			REG_A5XX_RBBM_PERFCTR_RBBM_0_HI);
+
+	busy_time = (busy_cycles - gpu->devfreq.busy_cycles) /
+		(clk_get_rate(gpu->core_clk) / 1000000);
+
+	gpu->devfreq.busy_cycles = busy_cycles;
+
+	return busy_time;
 }
 
 static const struct adreno_gpu_funcs funcs = {
drivers/gpu/drm/msm/msm_gpu.c +10 -8
index d8d4fc9..f437ee8 100644
@@ -40,7 +40,11 @@ static int msm_devfreq_target(struct device *dev, unsigned long *freq,
 	if (IS_ERR(opp))
 		return PTR_ERR(opp);
 
-	clk_set_rate(gpu->core_clk, *freq);
+	if (gpu->funcs->gpu_set_freq)
+		gpu->funcs->gpu_set_freq(gpu, (u64)*freq);
+	else
+		clk_set_rate(gpu->core_clk, *freq);
+
 	dev_pm_opp_put(opp);
 
 	return 0;
@@ -50,16 +54,14 @@ static int msm_devfreq_get_dev_status(struct device *dev,
 		struct devfreq_dev_status *status)
 {
 	struct msm_gpu *gpu = platform_get_drvdata(to_platform_device(dev));
-	u64 cycles;
-	u32 freq = ((u32) status->current_frequency) / 1000000;
 	ktime_t time;
 
-	status->current_frequency = (unsigned long) clk_get_rate(gpu->core_clk);
-	gpu->funcs->gpu_busy(gpu, &cycles);
-
-	status->busy_time = ((u32) (cycles - gpu->devfreq.busy_cycles)) / freq;
+	if (gpu->funcs->gpu_get_freq)
+		status->current_frequency = gpu->funcs->gpu_get_freq(gpu);
+	else
+		status->current_frequency = clk_get_rate(gpu->core_clk);
 
-	gpu->devfreq.busy_cycles = cycles;
+	status->busy_time = gpu->funcs->gpu_busy(gpu);
 
 	time = ktime_get();
 	status->total_time = ktime_us_delta(time, gpu->devfreq.time);

drivers/gpu/drm/msm/msm_gpu.c static int msm_devfreq_get_cur_freq(struct device *dev, unsigned long *freq) · line 78

Quoted 6 lines
drivers/gpu/drm/msm/msm_gpu.c +2 -1
@@ -72,7 +74,10 @@ static int msm_devfreq_get_cur_freq(struct device *dev, unsigned long *freq)
 {
 	struct msm_gpu *gpu = platform_get_drvdata(to_platform_device(dev));
 
-	*freq = (unsigned long) clk_get_rate(gpu->core_clk);
+	if (gpu->funcs->gpu_get_freq)
+		*freq = gpu->funcs->gpu_get_freq(gpu);

This will get ugly on a 32 bit build. I think that as long as the reset of the subsystem uses unsigned long we might as well too. I can put on my fortune tellers hat and guess we're probably not going to see a 4GB core running on a 32 bit target in the near future.

Quoted 7 lines
drivers/gpu/drm/msm/msm_gpu.c +0 -0
 	return 0;
 }
drivers/gpu/drm/msm/msm_gpu.h +0 -0
index 1876b81..d3f02e7 100644

drivers/gpu/drm/msm/msm_gpu.h struct msm_gpu_funcs { · line 73

Quoted 7 lines
drivers/gpu/drm/msm/msm_gpu.h +3 -1
@@ -68,7 +68,9 @@ struct msm_gpu_funcs {
 	/* for generation specific debugfs: */
 	int (*debugfs_init)(struct msm_gpu *gpu, struct drm_minor *minor);
 #endif
-	int (*gpu_busy)(struct msm_gpu *gpu, uint64_t *value);
+	u64 (*gpu_busy)(struct msm_gpu *gpu);
+	u64 (*gpu_get_freq)(struct msm_gpu *gpu);
+	int (*gpu_set_freq)(struct msm_gpu *gpu, u64 freq);

So go ahead and change both of these to an unsigned long and that should help the casting.

Jordan

Quoted 6 lines
drivers/gpu/drm/msm/msm_gpu.h +0 -0
 };
 
 struct msm_gpu {
Signature · 1 line
1.9.1
Signature · 6 lines
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

A message that is not in the archive (sent off-list or not yet received): 1527244246-10519-4-git-send-email-smasetty-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org

#10 lore raw

Re: [PATCH 3/3] drm/msm: re-factor devfreq code

Thank you for the patch! Yet something to improve:

Hi Sharat,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on robclark/msm-next]
[also build test ERROR on v4.17-rc6]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Sharat-Masetty/drm-msm-re-factor-devfreq-common-code/20180527-191724
base:   git://people.freedesktop.org/~robclark/linux msm-next
config: arm-multi_v7_defconfig (attached as .config)
compiler: arm-linux-gnueabi-gcc (Debian 7.2.0-11) 7.2.0

reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
make.cross ARCH=arm

All errors (new ones prefixed by >>):

drivers/gpu/drm/msm/adreno/a5xx_gpu.o: In function `a5xx_gpu_busy':

>> a5xx_gpu.c:(.text+0xcc): undefined reference to `__aeabi_uldivmod'

---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation

_______________________________________________
Freedreno mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/freedreno