Implement full clockgating for Kepler1 and 2
Subsystems (3)
Lyude Paul
maintainercover Implement full clockgating for Kepler1 and 2
It's here! After a lot of investigation, rewrites, and traces, I present the patch series to implement all known levels of clockgating for Kepler1 and Kepler2 GPUs.
Starting with Fermi GPUs (this is probably present on earlier GPUs as well, but with a far less easy to manage interface), nvidia added two clockgating levels that are handled mostly in firmware (with the exception of course, of the driver initially programming all of the register values containing engine delays and that stuff):
- CG_CTRL - Main register for enabling/disabling clockgating for engines and hw blocks
- BLCG - "Block-level clockgating", a deeper level of clockgating
Starting with kepler2 as well, nvidia also introduced:
- SLCG - "??? clockgating" even deeper level of clockgating
Originally this patchset was going to include work for making this work on Fermi, however on closer investigation it seems that Fermi has one pretty big difference in it's requirements that we don't entirely understand yet. On Fermi the CG_CTRL register for the gr needs to be adjusted during reclocking, while kepler and later generations need no such adjustments. Since this requires more research; the current plan is to leave fermi out of this patchset, and then just rework the code at a later point in time to support Fermi once we understand more.
For the time being however, we put as much code we're sure Fermi will be sharing into gf100 files for the future.
For the time being; this patchset supports every GPU in the kepler1 and kepler2 family. The main kernel config option to enable this is
config=NvPmEnableGating=<level>
Where <level> is the deepest level of powersaving to enable. The levels being:
0. NOCG
1. CG (just CG_CTRL)
2. BLCG
3. SLCG
Additionally, we leave a couple of small TODO comments to mark spots we'll eventually need to add code for the final level of power saving we have yet to implement or understand: ELPG (engine-level powergating).
Here's some very lazily done benchmarks on how much power this saves as well. It should be noted that chances are, these patches save a lot more power then you see here since clockgating works best when the GPU is under load. I have a feeling this will be especially true with SLCG.
GK104 stats:
Idle (1 4k display @ fbcon):
NOCG:
07: 20.53W
0a: 44.12W
0e: 61.23W
0f: 62.23W
CG:
07: 18.95W-19.02W
0a: 34.95W-37.48W
0e: 52.70W-54.32W
0f: 53.53W-55.68W
BLCG:
07: 18.64W-19.02W
0a: 32.68W-34.78W
0e: 49.14W-50.51W
0f: 49.75W-51.73W
GK110 stats:
Idle (1 4k display @ fbcon):
NOCG:
07: 25.68W
0a: 36.16W
0d: 64.71W
0f: 64.99W
CG:
07: 25.68W
0a: 35.58W
0d: 61.54W-61.74W
0f: 61.93W-62.12W
BLCG:
07: 25W-25.2W
0a: 33.85W-34.04W
0d: 59.25W-59.44W
0f: 59.35W-59.54W
SLCG:
07: 25W-25.2W
0a: 33.85W-33.95W (spikes to 34.04W briefly every now and then)
0d: 59.15W-59.35W
0f: 59.35W-59.54WFor the time being, this will be disabled by default in the kernel as we wait to get more widespread testing with this patchset. If you're willing to put up with the potential of instability, please feel free to try this patchset and let us know how well it works for you!
Lyude Paul (4):
drm/nouveau: Add support for basic clockgating on Kepler1 drm/nouveau: Add support for BLCG clockgating for Kepler1 drm/nouveau: Add BLCG clockgating for Kepler2 drm/nouveau: Add SLCG clockgating for Kepler2
drivers/gpu/drm/nouveau/include/nvkm/subdev/fb.h | 1 + .../gpu/drm/nouveau/include/nvkm/subdev/therm.h | 25 +++ drivers/gpu/drm/nouveau/nvkm/engine/device/base.c | 25 +-- drivers/gpu/drm/nouveau/nvkm/engine/gr/gf100.h | 1 + drivers/gpu/drm/nouveau/nvkm/engine/gr/gk104.c | 215 ++++++++++++++++++++- drivers/gpu/drm/nouveau/nvkm/engine/gr/gk104.h | 55 ++++++ drivers/gpu/drm/nouveau/nvkm/engine/gr/gk110.c | 176 +++++++++++++++++ drivers/gpu/drm/nouveau/nvkm/subdev/fb/Kbuild | 1 + drivers/gpu/drm/nouveau/nvkm/subdev/fb/gf100.c | 6 + drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk104.c | 53 +++++ drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk104.h | 35 ++++ drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk110.c | 77 ++++++++ drivers/gpu/drm/nouveau/nvkm/subdev/fb/priv.h | 3 + drivers/gpu/drm/nouveau/nvkm/subdev/therm/Kbuild | 2 + drivers/gpu/drm/nouveau/nvkm/subdev/therm/base.c | 84 +++++++- drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf100.c | 77 ++++++++ drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf100.h | 35 ++++ drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf119.c | 8 +- drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.c | 136 +++++++++++++ drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.h | 56 ++++++ drivers/gpu/drm/nouveau/nvkm/subdev/therm/gt215.c | 2 +- drivers/gpu/drm/nouveau/nvkm/subdev/therm/priv.h | 23 ++- 22 files changed, 1069 insertions(+), 27 deletions(-) create mode 100644 drivers/gpu/drm/nouveau/nvkm/engine/gr/gk104.h create mode 100644 drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk104.h create mode 100644 drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk110.c create mode 100644 drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf100.c create mode 100644 drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf100.h create mode 100644 drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.c create mode 100644 drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.h
Signature · 6 lines
2.14.3 _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Lyude Paul
maintainerRFC 1/4 drm/nouveau: Add support for basic clockgating on Kepler1
This adds support for enabling automatic clockgating on nvidia GPUs for Kepler1, referred to as "CG" throughout the driver. This is one of two powersaving levels that Kepler1 supports.
This introduces two therm helpers for controlling basic clockgating:
nvkm_therm_clkgate_enable() - enables clockgating through CG_CTRL, done after initializing the GPU fully nvkm_therm_clkgate_fini() - prepares clockgating for suspend or driver unload
As well, we add the nouveau kernel config parameter NvPmEnableGating, which can be set to the highest level of clockgating (in this case, we only have CG) the user desires to enable. Since we've only had limited testing on this thus far, we disable this by default.
A lot of this code was originally going to be based off of fermi; however it turns out that while Fermi's the first line of GPUs that introduced this kind of power saving, Fermi requires more fine tuned control of the CG_CTRL registers from the driver while reclocking that we don't entirely understand yet.
For the simple parts we will be sharing with Fermi for certain however, we at least add those into a new subdev/therm/gf100.h header.
--- .../gpu/drm/nouveau/include/nvkm/subdev/therm.h | 10 ++ drivers/gpu/drm/nouveau/nvkm/engine/device/base.c | 17 +-- drivers/gpu/drm/nouveau/nvkm/subdev/therm/Kbuild | 1 + drivers/gpu/drm/nouveau/nvkm/subdev/therm/base.c | 72 +++++++++-- drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf100.h | 35 ++++++ drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf119.c | 8 +- drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.c | 135 +++++++++++++++++++++ drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.h | 56 +++++++++ drivers/gpu/drm/nouveau/nvkm/subdev/therm/priv.h | 15 ++- 9 files changed, 328 insertions(+), 21 deletions(-) create mode 100644 drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf100.h create mode 100644 drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.c create mode 100644 drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.h
drivers/gpu/drm/nouveau/include/nvkm/subdev/therm.h +10 -0
@@ -46,6 +46,11 @@ enum nvkm_therm_attr_type { NVKM_THERM_ATTR_THRS_SHUTDOWN_HYST = 17, }; +enum nvkm_therm_clkgate_level { +NVKM_THERM_CLKGATE_NONE =0 , +NVKM_THERM_CLKGATE_CG ,/* basic clockgating */ +}; +struct nvkm_therm {const struct nvkm_therm_func *func;struct nvkm_subdev subdev;
@@ -85,17 +90,22 @@ struct nvkm_therm {int (*attr_get)(struct nvkm_therm *,enum nvkm_therm_attr_type );int (*attr_set)(struct nvkm_therm *,enum nvkm_therm_attr_type ,int ); + +enum nvkm_therm_clkgate_level clkgate_level; };int nvkm_therm_temp_get (struct nvkm_therm *);int nvkm_therm_fan_sense (struct nvkm_therm *);int nvkm_therm_cstate (struct nvkm_therm *,int ,int ); +void nvkm_therm_clkgate_enable (struct nvkm_therm *); +void nvkm_therm_clkgate_fini (struct nvkm_therm *,bool );int nv40_therm_new (struct nvkm_device *,int ,struct nvkm_therm **);int nv50_therm_new (struct nvkm_device *,int ,struct nvkm_therm **);int g84_therm_new (struct nvkm_device *,int ,struct nvkm_therm **);int gt215_therm_new (struct nvkm_device *,int ,struct nvkm_therm **);int gf119_therm_new (struct nvkm_device *,int ,struct nvkm_therm **); +int gk104_therm_new (struct nvkm_device *,int ,struct nvkm_therm **);int gm107_therm_new (struct nvkm_device *,int ,struct nvkm_therm **);int gm200_therm_new (struct nvkm_device *,int ,struct nvkm_therm **);int gp100_therm_new (struct nvkm_device *,int ,struct nvkm_therm **);
drivers/gpu/drm/nouveau/nvkm/engine/device/base.c +10 -7
@@ -28,6 +28,7 @@#include <core/option.h> #include <subdev/bios.h> +#include <subdev/therm.h> static DEFINE_MUTEX (nv_devices_mutex);static LIST_HEAD (nv_devices);
@@ -1682,7 +1683,7 @@ nve4_chipset = { .mxm = nv50_mxm_new, .pci = gk104_pci_new, .pmu = gk104_pmu_new, - .therm = gf119_therm_new, + .therm = gk104_therm_new, .timer = nv41_timer_new, .top = gk104_top_new, .volt = gk104_volt_new,
@@ -1721,7 +1722,7 @@ nve6_chipset = { .mxm = nv50_mxm_new, .pci = gk104_pci_new, .pmu = gk104_pmu_new, - .therm = gf119_therm_new, + .therm = gk104_therm_new, .timer = nv41_timer_new, .top = gk104_top_new, .volt = gk104_volt_new,
@@ -1760,7 +1761,7 @@ nve7_chipset = { .mxm = nv50_mxm_new, .pci = gk104_pci_new, .pmu = gk104_pmu_new, - .therm = gf119_therm_new, + .therm = gk104_therm_new, .timer = nv41_timer_new, .top = gk104_top_new, .volt = gk104_volt_new,
@@ -1824,7 +1825,7 @@ nvf0_chipset = { .mxm = nv50_mxm_new, .pci = gk104_pci_new, .pmu = gk110_pmu_new, - .therm = gf119_therm_new, + .therm = gk104_therm_new, .timer = nv41_timer_new, .top = gk104_top_new, .volt = gk104_volt_new,
@@ -1862,7 +1863,7 @@ nvf1_chipset = { .mxm = nv50_mxm_new, .pci = gk104_pci_new, .pmu = gk110_pmu_new, - .therm = gf119_therm_new, + .therm = gk104_therm_new, .timer = nv41_timer_new, .top = gk104_top_new, .volt = gk104_volt_new,
@@ -1900,7 +1901,7 @@ nv106_chipset = { .mxm = nv50_mxm_new, .pci = gk104_pci_new, .pmu = gk208_pmu_new, - .therm = gf119_therm_new, + .therm = gk104_therm_new, .timer = nv41_timer_new, .top = gk104_top_new, .volt = gk104_volt_new,
@@ -1938,7 +1939,7 @@ nv108_chipset = { .mxm = nv50_mxm_new, .pci = gk104_pci_new, .pmu = gk208_pmu_new, - .therm = gf119_therm_new, + .therm = gk104_therm_new, .timer = nv41_timer_new, .top = gk104_top_new, .volt = gk104_volt_new,
@@ -2508,6 +2509,7 @@ nvkm_device_fini(struct nvkm_device *device, bool suspend) } } +nvkm_therm_clkgate_fini (device->therm, suspend);if (device->func->fini) device->func->fini (device, suspend);
@@ -2597,6 +2599,7 @@ nvkm_device_init(struct nvkm_device *device) }nvkm_acpi_init (device); +nvkm_therm_clkgate_enable (device->therm); time =ktime_to_us (ktime_get ()) - time;nvdev_trace (device,"init completed in %lldus\n" , time);
drivers/gpu/drm/nouveau/nvkm/subdev/therm/Kbuild +1 -0
@@ -10,6 +10,7 @@ nvkm-y += nvkm/subdev/therm/nv50.o nvkm-y += nvkm/subdev/therm/g84.o nvkm-y += nvkm/subdev/therm/gt215.o nvkm-y += nvkm/subdev/therm/gf119.o +nvkm-y += nvkm/subdev/therm/gk104.o nvkm-y += nvkm/subdev/therm/gm107.o nvkm-y += nvkm/subdev/therm/gm200.o nvkm-y += nvkm/subdev/therm/gp100.o
drivers/gpu/drm/nouveau/nvkm/subdev/therm/base.c +64 -8
102 lines not on this page · raw message
drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf100.h new file +35 -0
@@ -0,0 +1,35 @@ +/* +* Copyright 2018 Red Hat Inc. +* +* Permission is hereby granted, free of charge, to any person obtaining a +* copy of this software and associated documentation files (the "Software"), +* to deal in the Software without restriction, including without limitation +* the rights to use, copy, modify, merge, publish, distribute, sublicense, +* and/or sell copies of the Software, and to permit persons to whom the +* Software is furnished to do so, subject to the following conditions: +* +* The above copyright notice and this permission notice shall be included in +* all copies or substantial portions of the Software. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +* THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR +* OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, +* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR +* OTHER DEALINGS IN THE SOFTWARE. +* +* Authors: Lyude Paul +*/ + +#ifndef __GF100_THERM_H__ +#define __GF100_THERM_H__ + +#include <core/device.h> + +struct gf100_idle_filter { +u32 fecs; +u32 hubmmu; +}; + +#endif
drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf119.c +4 -4
@@ -49,7 +49,7 @@ pwm_info(struct nvkm_therm *therm, int line)return -ENODEV ; } -static int +int gf119_fan_pwm_ctrl (struct nvkm_therm *therm,int line,bool enable) {struct nvkm_device *device = therm->subdev.device;
@@ -63,7 +63,7 @@ gf119_fan_pwm_ctrl(struct nvkm_therm *therm, int line, bool enable)return 0 ; } -static int +int gf119_fan_pwm_get (struct nvkm_therm *therm,int line,u32 *divs,u32 *duty) {struct nvkm_device *device = therm->subdev.device;
@@ -85,7 +85,7 @@ gf119_fan_pwm_get(struct nvkm_therm *therm, int line, u32 *divs, u32 *duty)return -EINVAL ; } -static int +int gf119_fan_pwm_set (struct nvkm_therm *therm,int line,u32 divs,u32 duty) {struct nvkm_device *device = therm->subdev.device;
@@ -102,7 +102,7 @@ gf119_fan_pwm_set(struct nvkm_therm *therm, int line, u32 divs, u32 duty)return 0 ; } -static int +int gf119_fan_pwm_clock (struct nvkm_therm *therm,int line) {struct nvkm_device *device = therm->subdev.device;
drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.c new file +135 -0
135 lines not on this page · raw message
drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.h new file +56 -0
56 lines not on this page · raw message
drivers/gpu/drm/nouveau/nvkm/subdev/therm/priv.h +13 -2
@@ -32,6 +32,8 @@int nvkm_therm_new_ (const struct nvkm_therm_func *,struct nvkm_device *,int index,struct nvkm_therm **); +void nvkm_therm_ctor (struct nvkm_therm *therm,struct nvkm_device *device, +int index,const struct nvkm_therm_func *func);struct nvkm_fan {struct nvkm_therm *parent;
@@ -66,8 +68,6 @@ int nvkm_therm_fan_set(struct nvkm_therm *, bool now, int percent);int nvkm_therm_fan_user_get (struct nvkm_therm *);int nvkm_therm_fan_user_set (struct nvkm_therm *,int percent); -int nvkm_therm_preinit (struct nvkm_therm *); -int nvkm_therm_sensor_init (struct nvkm_therm *);int nvkm_therm_sensor_fini (struct nvkm_therm *,bool suspend);void nvkm_therm_sensor_preinit (struct nvkm_therm *);
@@ -96,6 +96,9 @@ struct nvkm_therm_func {int (*fan_sense)(struct nvkm_therm *);void (*program_alarms)(struct nvkm_therm *); + +void (*clkgate_enable)(struct nvkm_therm *); +void (*clkgate_fini)(struct nvkm_therm *,bool ); };void nv40_therm_intr (struct nvkm_therm *);
@@ -112,8 +115,16 @@ void g84_therm_fini(struct nvkm_therm *);int gt215_therm_fan_sense (struct nvkm_therm *);void g84_therm_init (struct nvkm_therm *); + +int gf119_fan_pwm_ctrl (struct nvkm_therm *,int ,bool ); +int gf119_fan_pwm_get (struct nvkm_therm *,int ,u32 *,u32 *); +int gf119_fan_pwm_set (struct nvkm_therm *,int ,u32 ,u32 ); +int gf119_fan_pwm_clock (struct nvkm_therm *,int );void gf119_therm_init (struct nvkm_therm *); +void gk104_clkgate_enable (struct nvkm_therm *); +void gk104_clkgate_fini (struct nvkm_therm *,bool ); +int nvkm_fanpwm_create (struct nvkm_therm *,struct dcb_gpio_func *);int nvkm_fantog_create (struct nvkm_therm *,struct dcb_gpio_func *);int nvkm_fannil_create (struct nvkm_therm *);
Signature · 6 lines
2.14.3 _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [RFC 1/4] drm/nouveau: Add support for basic clockgating on Kepler1
Quoted 3 lines · Lyude Paul
This adds support for enabling automatic clockgating on nvidia GPUs for Kepler1, referred to as "CG" throughout the driver. This is one of two powersaving levels that Kepler1 supports.
Thanks a lot for all this work! It was long overdue and it is nice to see the project finally getting to an end, after passing into so many hands!
Quoted 11 lines
This introduces two therm helpers for controlling basic clockgating:
nvkm_therm_clkgate_enable()- enables clockgating throughCG_CTRL, done after initializing the GPU fullynvkm_therm_clkgate_fini()- prepares clockgating for suspend or driver unloadAs well, we add the nouveau kernel config parameter NvPmEnableGating, which can be set to the highest level of clockgating (in this case, we only have CG) the user desires to enable. Since we've only had limited testing on this thus far, we disable this by default.
I am not sure I understand the purpose of this level here. As far as I understand, you only have per-engine control whether you want to enable CG or not. What you call BLCG and SLCG levels just mean "don't use the boot values, but rather use our values (taken from nvidia)".
Now, here comes the nasty part: NVIDIA only ever validated the boot values (I guess they are extremely safe ones), and the optimised values (the ones coming from your patch 2, 3, and 4 along with the level 3.
I think introducing a single parameter that controls both CG, PG, and automatic reclocking would be safer. For CG and PG, it would be a all-or-nothing (either boot values, or everything like nvidia).
Quoted 213 lines
A lot of this code was originally going to be based off of fermi; however it turns out that while Fermi's the first line of GPUs that introduced this kind of power saving, Fermi requires more fine tuned control of the
CG_CTRLregisters from the driver while reclocking that we don't entirely understand yet.For the simple parts we will be sharing with Fermi for certain however, we at least add those into a new
subdev/therm/gf100.hheader.Signed-off-by Lyude Paul <[email protected]>--- .../gpu/drm/nouveau/include/nvkm/subdev/therm.h | 10 ++ drivers/gpu/drm/nouveau/nvkm/engine/device/base.c | 17 +-- drivers/gpu/drm/nouveau/nvkm/subdev/therm/Kbuild | 1 + drivers/gpu/drm/nouveau/nvkm/subdev/therm/base.c | 72 +++++++++-- drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf100.h | 35 ++++++ drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf119.c | 8 +- drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.c | 135 +++++++++++++++++++++ drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.h | 56 +++++++++ drivers/gpu/drm/nouveau/nvkm/subdev/therm/priv.h | 15 ++- 9 files changed, 328 insertions(+), 21 deletions(-) create mode 100644 drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf100.h create mode 100644 drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.c create mode 100644 drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.hdrivers/gpu/drm/nouveau/include/nvkm/subdev/therm.h +10
-0@@ -46,6 +46,11 @@ enum nvkm_therm_attr_type { NVKM_THERM_ATTR_THRS_SHUTDOWN_HYST = 17, }; +enum nvkm_therm_clkgate_level { +NVKM_THERM_CLKGATE_NONE =0 , +NVKM_THERM_CLKGATE_CG ,/* basic clockgating */ +}; +struct nvkm_therm {const struct nvkm_therm_func *func;struct nvkm_subdev subdev;@@ -85,17 +90,22 @@ struct nvkm_therm {int (*attr_get)(struct nvkm_therm *,enum nvkm_therm_attr_type );int (*attr_set)(struct nvkm_therm *,enum nvkm_therm_attr_type ,int ); + +enum nvkm_therm_clkgate_level clkgate_level; };int nvkm_therm_temp_get (struct nvkm_therm *);int nvkm_therm_fan_sense (struct nvkm_therm *);int nvkm_therm_cstate (struct nvkm_therm *,int ,int ); +void nvkm_therm_clkgate_enable (struct nvkm_therm *); +void nvkm_therm_clkgate_fini (struct nvkm_therm *,bool );int nv40_therm_new (struct nvkm_device *,int ,struct nvkm_therm **);int nv50_therm_new (struct nvkm_device *,int ,struct nvkm_therm **);int g84_therm_new (struct nvkm_device *,int ,struct nvkm_therm **);int gt215_therm_new (struct nvkm_device *,int ,struct nvkm_therm **);int gf119_therm_new (struct nvkm_device *,int ,struct nvkm_therm **); +int gk104_therm_new (struct nvkm_device *,int ,struct nvkm_therm **);int gm107_therm_new (struct nvkm_device *,int ,struct nvkm_therm **);int gm200_therm_new (struct nvkm_device *,int ,struct nvkm_therm **);int gp100_therm_new (struct nvkm_device *,int ,struct nvkm_therm **);drivers/gpu/drm/nouveau/nvkm/engine/device/base.c +10
-7@@ -28,6 +28,7 @@#include <core/option.h> #include <subdev/bios.h> +#include <subdev/therm.h> static DEFINE_MUTEX (nv_devices_mutex);static LIST_HEAD (nv_devices);@@ -1682,7 +1683,7 @@ nve4_chipset = { .mxm = nv50_mxm_new, .pci = gk104_pci_new, .pmu = gk104_pmu_new, - .therm = gf119_therm_new, + .therm = gk104_therm_new, .timer = nv41_timer_new, .top = gk104_top_new, .volt = gk104_volt_new,@@ -1721,7 +1722,7 @@ nve6_chipset = { .mxm = nv50_mxm_new, .pci = gk104_pci_new, .pmu = gk104_pmu_new, - .therm = gf119_therm_new, + .therm = gk104_therm_new, .timer = nv41_timer_new, .top = gk104_top_new, .volt = gk104_volt_new,@@ -1760,7 +1761,7 @@ nve7_chipset = { .mxm = nv50_mxm_new, .pci = gk104_pci_new, .pmu = gk104_pmu_new, - .therm = gf119_therm_new, + .therm = gk104_therm_new, .timer = nv41_timer_new, .top = gk104_top_new, .volt = gk104_volt_new,@@ -1824,7 +1825,7 @@ nvf0_chipset = { .mxm = nv50_mxm_new, .pci = gk104_pci_new, .pmu = gk110_pmu_new, - .therm = gf119_therm_new, + .therm = gk104_therm_new, .timer = nv41_timer_new, .top = gk104_top_new, .volt = gk104_volt_new,@@ -1862,7 +1863,7 @@ nvf1_chipset = { .mxm = nv50_mxm_new, .pci = gk104_pci_new, .pmu = gk110_pmu_new, - .therm = gf119_therm_new, + .therm = gk104_therm_new, .timer = nv41_timer_new, .top = gk104_top_new, .volt = gk104_volt_new,@@ -1900,7 +1901,7 @@ nv106_chipset = { .mxm = nv50_mxm_new, .pci = gk104_pci_new, .pmu = gk208_pmu_new, - .therm = gf119_therm_new, + .therm = gk104_therm_new, .timer = nv41_timer_new, .top = gk104_top_new, .volt = gk104_volt_new,@@ -1938,7 +1939,7 @@ nv108_chipset = { .mxm = nv50_mxm_new, .pci = gk104_pci_new, .pmu = gk208_pmu_new, - .therm = gf119_therm_new, + .therm = gk104_therm_new, .timer = nv41_timer_new, .top = gk104_top_new, .volt = gk104_volt_new,@@ -2508,6 +2509,7 @@ nvkm_device_fini(struct nvkm_device *device, bool suspend) } } +nvkm_therm_clkgate_fini (device->therm, suspend);if (device->func->fini) device->func->fini (device, suspend);@@ -2597,6 +2599,7 @@ nvkm_device_init(struct nvkm_device *device) }nvkm_acpi_init (device); +nvkm_therm_clkgate_enable (device->therm); time =ktime_to_us (ktime_get ()) - time;nvdev_trace (device,"init completed in %lldus\n" , time);drivers/gpu/drm/nouveau/nvkm/subdev/therm/Kbuild +1
-0@@ -10,6 +10,7 @@ nvkm-y += nvkm/subdev/therm/nv50.o nvkm-y += nvkm/subdev/therm/g84.o nvkm-y += nvkm/subdev/therm/gt215.o nvkm-y += nvkm/subdev/therm/gf119.o +nvkm-y += nvkm/subdev/therm/gk104.o nvkm-y += nvkm/subdev/therm/gm107.o nvkm-y += nvkm/subdev/therm/gm200.o nvkm-y += nvkm/subdev/therm/gp100.odrivers/gpu/drm/nouveau/nvkm/subdev/therm/base.c +30
-0@@ -21,6 +21,7 @@* * Authors: Martin Peres */ +#include <nvkm/core/option.h> #include "priv.h" int @@ -297,6 +298,47 @@ nvkm_therm_attr_set(struct nvkm_therm *therm,return -EINVAL ; } +void +nvkm_therm_clkgate_enable (struct nvkm_therm *therm) +{ +if (!therm->func->clkgate_enable || !therm->clkgate_level) +return ; + +nvkm_debug (&therm->subdev, +"Enabling clock/powergating\n" ); + therm->func->clkgate_enable (therm); +} + +void +nvkm_therm_clkgate_fini (struct nvkm_therm *therm,bool suspend) +{ +if (!therm->func->clkgate_fini || !therm->clkgate_level) +return ; + +nvkm_debug (&therm->subdev, +"Preparing clock/powergating for %s\n" , + suspend ?"suspend" :"fini" ); + therm->func->clkgate_fini (therm, suspend); +} + +static void +nvkm_therm_clkgate_oneinit (struct nvkm_therm *therm) +{ +const char *clkgate_str; + +if (!therm->func->clkgate_enable || !therm->clkgate_level)
drivers/gpu/drm/nouveau/nvkm/subdev/therm/base.c nvkm_therm_attr_set(struct nvkm_therm *therm, · line 339
Quoted 10 lines
drivers/gpu/drm/nouveau/nvkm/subdev/therm/base.c +10
-0+return ; + +switch (therm->clkgate_level) { +case NVKM_THERM_CLKGATE_CG : clkgate_str ="CG" ;break ; +default :BUG (); + } + +nvkm_info (&therm->subdev, +"Clock/powergating enabled up to level %d (%s)\n" , + therm->clkgate_level, clkgate_str);
This message is a bit odd, whether we keep the notion of levels or not.
Can you get rid of the mention of powergating given that this is not part of this patchset?
If you agree with having a single enable bit for CG, then a simple:
"Clockgating status: (boot|optimized)" would work perfectly.
Quoted 180 lines
drivers/gpu/drm/nouveau/nvkm/subdev/therm/base.c +24
-8+} +static void nvkm_therm_intr (struct nvkm_subdev *subdev) {@@ -333,6 +375,7 @@ nvkm_therm_oneinit(struct nvkm_subdev *subdev)nvkm_therm_fan_ctor (therm);nvkm_therm_fan_mode (therm,NVKM_THERM_CTRL_AUTO );nvkm_therm_sensor_preinit (therm); +nvkm_therm_clkgate_oneinit (therm);return 0 ; }@@ -374,15 +417,10 @@ nvkm_therm = { .intr = nvkm_therm_intr, }; -int -nvkm_therm_new_ (const struct nvkm_therm_func *func,struct nvkm_device *device, -int index,struct nvkm_therm **ptherm) +void +nvkm_therm_ctor (struct nvkm_therm *therm,struct nvkm_device *device, +int index,const struct nvkm_therm_func *func) { -struct nvkm_therm *therm; - -if (!(therm = *ptherm =kzalloc (sizeof (*therm),GFP_KERNEL ))) -return -ENOMEM ; -nvkm_subdev_ctor (&nvkm_therm, device, index, &therm->subdev); therm->func = func;@@ -395,5 +433,23 @@ nvkm_therm_new_(const struct nvkm_therm_func *func, struct nvkm_device *device, therm->attr_get = nvkm_therm_attr_get; therm->attr_set = nvkm_therm_attr_set; therm->mode = therm->suspend =-1 ;/* undefined */ + + therm->clkgate_level = +clamp ((int )nvkm_longopt (device->cfgopt, +"NvPmEnableGating" , +NVKM_THERM_CLKGATE_NONE ), +NVKM_THERM_CLKGATE_NONE ,NVKM_THERM_CLKGATE_CG ); +} + +int +nvkm_therm_new_ (const struct nvkm_therm_func *func,struct nvkm_device *device, +int index,struct nvkm_therm **ptherm) +{ +struct nvkm_therm *therm; + +if (!(therm = *ptherm =kzalloc (sizeof (*therm),GFP_KERNEL ))) +return -ENOMEM ; + +nvkm_therm_ctor (therm, device, index, func);return 0 ; }drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf100.h new file +35
-0@@ -0,0 +1,35 @@ +/* +* Copyright 2018 Red Hat Inc. +* +* Permission is hereby granted, free of charge, to any person obtaining a +* copy of this software and associated documentation files (the "Software"), +* to deal in the Software without restriction, including without limitation +* the rights to use, copy, modify, merge, publish, distribute, sublicense, +* and/or sell copies of the Software, and to permit persons to whom the +* Software is furnished to do so, subject to the following conditions: +* +* The above copyright notice and this permission notice shall be included in +* all copies or substantial portions of the Software. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +* THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR +* OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, +* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR +* OTHER DEALINGS IN THE SOFTWARE. +* +* Authors: Lyude Paul +*/ + +#ifndef __GF100_THERM_H__ +#define __GF100_THERM_H__ + +#include <core/device.h> + +struct gf100_idle_filter { +u32 fecs; +u32 hubmmu; +}; + +#endif drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf119.c +4
-4@@ -49,7 +49,7 @@ pwm_info(struct nvkm_therm *therm, int line)return -ENODEV ; } -static int +int gf119_fan_pwm_ctrl (struct nvkm_therm *therm,int line,bool enable) {struct nvkm_device *device = therm->subdev.device;@@ -63,7 +63,7 @@ gf119_fan_pwm_ctrl(struct nvkm_therm *therm, int line, bool enable)return 0 ; } -static int +int gf119_fan_pwm_get (struct nvkm_therm *therm,int line,u32 *divs,u32 *duty) {struct nvkm_device *device = therm->subdev.device;@@ -85,7 +85,7 @@ gf119_fan_pwm_get(struct nvkm_therm *therm, int line, u32 *divs, u32 *duty)return -EINVAL ; } -static int +int gf119_fan_pwm_set (struct nvkm_therm *therm,int line,u32 divs,u32 duty) {struct nvkm_device *device = therm->subdev.device;@@ -102,7 +102,7 @@ gf119_fan_pwm_set(struct nvkm_therm *therm, int line, u32 divs, u32 duty)return 0 ; } -static int +int gf119_fan_pwm_clock (struct nvkm_therm *therm,int line) {struct nvkm_device *device = therm->subdev.device;drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.c new file +37
-0@@ -0,0 +1,135 @@ +/* +* Copyright 2018 Red Hat Inc. +* +* Permission is hereby granted, free of charge, to any person obtaining a +* copy of this software and associated documentation files (the "Software"), +* to deal in the Software without restriction, including without limitation +* the rights to use, copy, modify, merge, publish, distribute, sublicense, +* and/or sell copies of the Software, and to permit persons to whom the +* Software is furnished to do so, subject to the following conditions: +* +* The above copyright notice and this permission notice shall be included in +* all copies or substantial portions of the Software. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +* THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR +* OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, +* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR +* OTHER DEALINGS IN THE SOFTWARE. +* +* Authors: Lyude Paul +*/ +#include <core/device.h> + +#include "priv.h" +#include "gk104.h" + +void +gk104_clkgate_enable (struct nvkm_therm *base) +{ +struct gk104_therm *therm =gk104_therm (base); +struct nvkm_device *dev = therm->base.subdev.device; +const struct gk104_clkgate_engine_info *order = therm->clkgate_order; +int i; + +/* Program ENG_MANT, ENG_FILTER */
drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.c line 47
Quoted 10 lines
drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.c new file +10
-0+for (i =0 ; order[i].engine !=NVKM_SUBDEV_NR ; i++) { +if (!nvkm_device_subdev (dev, order[i].engine)) +continue ; + +nvkm_mask (dev,0x20200 + order[i].offset,0xff00 ,0x4500 ); + } + +/* magic */ +nvkm_wr32 (dev,0x020288 , therm->idle_filter->fecs); +nvkm_wr32 (dev,0x02028c , therm->idle_filter->hubmmu);
All this time, I thought these parameters were for power gating... I also did not expect that clock gating had to be disabled before we could program them.
Great find!
Quoted 69 lines
drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.c new file +69
-0+ +/* Enable clockgating (ENG_CLK = RUN->AUTO) */ +for (i =0 ; order[i].engine !=NVKM_SUBDEV_NR ; i++) { +if (!nvkm_device_subdev (dev, order[i].engine)) +continue ; + +nvkm_mask (dev,0x20200 + order[i].offset,0x00ff ,0x0045 ); + } +} + +void +gk104_clkgate_fini (struct nvkm_therm *base,bool suspend) +{ +struct gk104_therm *therm =gk104_therm (base); +struct nvkm_device *dev = therm->base.subdev.device; +const struct gk104_clkgate_engine_info *order = therm->clkgate_order; +int i; + +/* ENG_CLK = AUTO->RUN, ENG_PWR = RUN->AUTO */ +for (i =0 ; order[i].engine !=NVKM_SUBDEV_NR ; i++) { +if (!nvkm_device_subdev (dev, order[i].engine)) +continue ; + +nvkm_mask (dev,0x20200 + order[i].offset,0xff ,0x54 ); + } +} + +const struct gk104_clkgate_engine_info gk104_clkgate_engine_info[] = { + {NVKM_ENGINE_GR ,0x00 }, + {NVKM_ENGINE_MSPDEC ,0x04 }, + {NVKM_ENGINE_MSPPP ,0x08 }, + {NVKM_ENGINE_MSVLD ,0x0c }, + {NVKM_ENGINE_CE0 ,0x10 }, + {NVKM_ENGINE_CE1 ,0x14 }, + {NVKM_ENGINE_MSENC ,0x18 }, + {NVKM_ENGINE_CE2 ,0x1c }, + {NVKM_SUBDEV_NR ,0 }, +}; + +const struct gf100_idle_filter gk104_idle_filter = { + .fecs =0x00001000 , + .hubmmu =0x00001000 , +}; + +static const struct nvkm_therm_func +gk104_therm_func = { + .init = gf119_therm_init, + .fini = g84_therm_fini, + .pwm_ctrl = gf119_fan_pwm_ctrl, + .pwm_get = gf119_fan_pwm_get, + .pwm_set = gf119_fan_pwm_set, + .pwm_clock = gf119_fan_pwm_clock, + .temp_get = g84_temp_get, + .fan_sense = gt215_therm_fan_sense, + .program_alarms = nvkm_therm_program_alarms_polling, + .clkgate_enable = gk104_clkgate_enable, + .clkgate_fini = gk104_clkgate_fini, +}; + +int +gk104_therm_new_ (const struct nvkm_therm_func *func, +struct nvkm_device *device, +int index, +const struct gk104_clkgate_engine_info *clkgate_order, +const struct gf100_idle_filter *idle_filter, +struct nvkm_therm **ptherm) +{ +struct gk104_therm *therm =kzalloc (sizeof (*therm),GFP_KERNEL ); +
drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.c line 126
Quoted 10 lines
drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.c new file +10
-0+if (!therm) +return -ENOMEM ; + +nvkm_therm_ctor (&therm->base, device, index, func); + *ptherm = &therm->base; + therm->clkgate_order = clkgate_order; + therm->idle_filter = idle_filter; + +return 0 ; +}
Why introduce gk104_therm_new_? I can't find references to it in this patch (outside of the function below) or in the following patches.
As you even export this function, it looks like you used to use this function in an earlier revision of this series.
Aside from all these nitpicks, the approach is quite self contained and I like the following patches. Well done!
Once we settle on the configuration parameter, I can give you my R-b :)
Martin
Quoted 121 lines
drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.c new file +9
-0+ +int +gk104_therm_new (struct nvkm_device *device, +int index,struct nvkm_therm **ptherm) +{ +return gk104_therm_new_ (&gk104_therm_func, device, index, + gk104_clkgate_engine_info, &gk104_idle_filter, + ptherm); +}drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.h new file +56
-0@@ -0,0 +1,56 @@ +/* +* Copyright 2018 Red Hat Inc. +* +* Permission is hereby granted, free of charge, to any person obtaining a +* copy of this software and associated documentation files (the "Software"), +* to deal in the Software without restriction, including without limitation +* the rights to use, copy, modify, merge, publish, distribute, sublicense, +* and/or sell copies of the Software, and to permit persons to whom the +* Software is furnished to do so, subject to the following conditions: +* +* The above copyright notice and this permission notice shall be included in +* all copies or substantial portions of the Software. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +* THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR +* OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, +* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR +* OTHER DEALINGS IN THE SOFTWARE. +* +* Authors: Lyude Paul +*/ + +#ifndef __GK104_THERM_H__ +#define __GK104_THERM_H__ +#define gk104_therm (p) (container_of((p), struct gk104_therm, base)) + +#include <subdev/therm.h> +#include "priv.h" +#include "gf100.h" + +struct gk104_clkgate_engine_info { +enum nvkm_devidx engine; +u8 offset; +}; + +struct gk104_therm { +struct nvkm_therm base; + +const struct gk104_clkgate_engine_info *clkgate_order; +const struct gf100_idle_filter *idle_filter; +}; + +int +gk104_therm_new_ (const struct nvkm_therm_func *func, +struct nvkm_device *device, +int , +const struct gk104_clkgate_engine_info *, +const struct gf100_idle_filter *, +struct nvkm_therm **); + +extern const struct gk104_clkgate_engine_info gk104_clkgate_engine_info[]; +extern const struct gf100_idle_filter gk104_idle_filter; + +#endif drivers/gpu/drm/nouveau/nvkm/subdev/therm/priv.h +13
-2@@ -32,6 +32,8 @@int nvkm_therm_new_ (const struct nvkm_therm_func *,struct nvkm_device *,int index,struct nvkm_therm **); +void nvkm_therm_ctor (struct nvkm_therm *therm,struct nvkm_device *device, +int index,const struct nvkm_therm_func *func);struct nvkm_fan {struct nvkm_therm *parent;@@ -66,8 +68,6 @@ int nvkm_therm_fan_set(struct nvkm_therm *, bool now, int percent);int nvkm_therm_fan_user_get (struct nvkm_therm *);int nvkm_therm_fan_user_set (struct nvkm_therm *,int percent); -int nvkm_therm_preinit (struct nvkm_therm *); -int nvkm_therm_sensor_init (struct nvkm_therm *);int nvkm_therm_sensor_fini (struct nvkm_therm *,bool suspend);void nvkm_therm_sensor_preinit (struct nvkm_therm *);@@ -96,6 +96,9 @@ struct nvkm_therm_func {int (*fan_sense)(struct nvkm_therm *);void (*program_alarms)(struct nvkm_therm *); + +void (*clkgate_enable)(struct nvkm_therm *); +void (*clkgate_fini)(struct nvkm_therm *,bool ); };void nv40_therm_intr (struct nvkm_therm *);@@ -112,8 +115,16 @@ void g84_therm_fini(struct nvkm_therm *);int gt215_therm_fan_sense (struct nvkm_therm *);void g84_therm_init (struct nvkm_therm *); + +int gf119_fan_pwm_ctrl (struct nvkm_therm *,int ,bool ); +int gf119_fan_pwm_get (struct nvkm_therm *,int ,u32 *,u32 *); +int gf119_fan_pwm_set (struct nvkm_therm *,int ,u32 ,u32 ); +int gf119_fan_pwm_clock (struct nvkm_therm *,int );void gf119_therm_init (struct nvkm_therm *); +void gk104_clkgate_enable (struct nvkm_therm *); +void gk104_clkgate_fini (struct nvkm_therm *,bool ); +int nvkm_fanpwm_create (struct nvkm_therm *,struct dcb_gpio_func *);int nvkm_fantog_create (struct nvkm_therm *,struct dcb_gpio_func *);int nvkm_fannil_create (struct nvkm_therm *);
Lyude Paul
maintainerRFC 2/4 drm/nouveau: Add support for BLCG clockgating for Kepler1
In reply to Lyude PaulThis enables the second level of clockgating for Kepler1: BLCG (presumably, this stands for block-level clockgating). This is a deeper level of power savings then just CG_CTRL. Setting it up is as simple as loading some mmio packs, and turning it on and off is still controlled through CG_CTRL.
We also introduce the nvkm_therm_clkgate_init() helper, which handles programming various BLCG register values while initializing the GPU, but before CG_CTRL is enabled.
Enabling BLCG is done with the nouveau config option NvPmEnableGating=2.
As well, this commit shares a lot more code with Fermi since BLCG is mostly the same there as far as we can tell. In the future, it's likely we'll reformat the clkgate_packs for kepler1 so that they share a list of mmio packs with Fermi.
--- .../gpu/drm/nouveau/include/nvkm/subdev/therm.h | 14 ++ drivers/gpu/drm/nouveau/nvkm/engine/gr/gf100.h | 1 + drivers/gpu/drm/nouveau/nvkm/engine/gr/gk104.c | 215 ++++++++++++++++++++- drivers/gpu/drm/nouveau/nvkm/engine/gr/gk104.h | 55 ++++++ drivers/gpu/drm/nouveau/nvkm/subdev/fb/gf100.c | 6 + drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk104.c | 53 +++++ drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk104.h | 35 ++++ drivers/gpu/drm/nouveau/nvkm/subdev/fb/priv.h | 3 + drivers/gpu/drm/nouveau/nvkm/subdev/therm/Kbuild | 1 + drivers/gpu/drm/nouveau/nvkm/subdev/therm/base.c | 15 +- drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf100.c | 77 ++++++++ drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.c | 1 + drivers/gpu/drm/nouveau/nvkm/subdev/therm/gt215.c | 2 +- drivers/gpu/drm/nouveau/nvkm/subdev/therm/priv.h | 8 + 14 files changed, 482 insertions(+), 4 deletions(-) create mode 100644 drivers/gpu/drm/nouveau/nvkm/engine/gr/gk104.h create mode 100644 drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk104.h create mode 100644 drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf100.c
drivers/gpu/drm/nouveau/include/nvkm/subdev/therm.h +14 -0
@@ -49,6 +49,18 @@ enum nvkm_therm_attr_type {enum nvkm_therm_clkgate_level {NVKM_THERM_CLKGATE_NONE =0 ,NVKM_THERM_CLKGATE_CG ,/* basic clockgating */ +NVKM_THERM_CLKGATE_BLCG , +}; + +struct nvkm_therm_clkgate_init { +u32 addr; +u8 count; +u32 data; +}; + +struct nvkm_therm_clkgate_pack { +enum nvkm_therm_clkgate_level level; +const struct nvkm_therm_clkgate_init **init; };struct nvkm_therm {
@@ -97,6 +109,8 @@ struct nvkm_therm {int nvkm_therm_temp_get (struct nvkm_therm *);int nvkm_therm_fan_sense (struct nvkm_therm *);int nvkm_therm_cstate (struct nvkm_therm *,int ,int ); +void nvkm_therm_clkgate_init (struct nvkm_therm *, +const struct nvkm_therm_clkgate_pack *);void nvkm_therm_clkgate_enable (struct nvkm_therm *);void nvkm_therm_clkgate_fini (struct nvkm_therm *,bool );
drivers/gpu/drm/nouveau/nvkm/engine/gr/gf100.h +1 -0
@@ -137,6 +137,7 @@ struct gf100_gr_func {int (*rops)(struct gf100_gr *);int ppc_nr;const struct gf100_grctx_func *grctx; +const struct nvkm_therm_clkgate_pack *clkgate_pack;struct nvkm_sclass sclass[]; };
drivers/gpu/drm/nouveau/nvkm/engine/gr/gk104.c +214 -1
245 lines not on this page · raw message
drivers/gpu/drm/nouveau/nvkm/engine/gr/gk104.h new file +55 -0
55 lines not on this page · raw message
drivers/gpu/drm/nouveau/nvkm/subdev/fb/gf100.c +6 -0
@@ -26,6 +26,7 @@#include <core/memory.h> #include <core/option.h> +#include <subdev/therm.h> void gf100_fb_intr (struct nvkm_fb *base)
@@ -92,6 +93,11 @@ gf100_fb_init(struct nvkm_fb *base)if (fb->r100c10_page)nvkm_wr32 (device,0x100c10 , fb->r100c10 >>8 ); + +if (base->func->clkgate_pack) { +nvkm_therm_clkgate_init (device->therm, + base->func->clkgate_pack); + } } void *
drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk104.c +53 -0
69 lines not on this page · raw message
drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk104.h new file +35 -0
@@ -0,0 +1,35 @@ +/* +* Copyright 2018 Red Hat Inc. +* +* Permission is hereby granted, free of charge, to any person obtaining a +* copy of this software and associated documentation files (the "Software"), +* to deal in the Software without restriction, including without limitation +* the rights to use, copy, modify, merge, publish, distribute, sublicense, +* and/or sell copies of the Software, and to permit persons to whom the +* Software is furnished to do so, subject to the following conditions: +* +* The above copyright notice and this permission notice shall be included in +* all copies or substantial portions of the Software. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +* THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR +* OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, +* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR +* OTHER DEALINGS IN THE SOFTWARE. +* +* Authors: Lyude Paul +*/ + +#ifndef __GK104_FB_H__ +#define __GK104_FB_H__ + +#include <subdev/therm.h> + +extern const struct nvkm_therm_clkgate_init gk104_fb_clkgate_blcg_init_unk_0[]; +extern const struct nvkm_therm_clkgate_init gk104_fb_clkgate_blcg_init_vm_0[]; +extern const struct nvkm_therm_clkgate_init gk104_fb_clkgate_blcg_init_main_0[]; +extern const struct nvkm_therm_clkgate_init gk104_fb_clkgate_blcg_init_bcast_0[]; + +#endif
drivers/gpu/drm/nouveau/nvkm/subdev/fb/priv.h +3 -0
@@ -3,6 +3,7 @@#define __NVKM_FB_PRIV_H__#define nvkm_fb (p) container_of((p), struct nvkm_fb, subdev)#include <subdev/fb.h> +#include <subdev/therm.h> struct nvkm_bios ;struct nvkm_fb_func {
@@ -27,6 +28,7 @@ struct nvkm_fb_func {int (*ram_new)(struct nvkm_fb *,struct nvkm_ram **);u8 default_bigpage; +const struct nvkm_therm_clkgate_pack *clkgate_pack; }; void nvkm_fb_ctor(const struct nvkm_fb_func *, struct nvkm_device *device,
@@ -66,4 +68,5 @@ int gf100_fb_oneinit(struct nvkm_fb *);int gf100_fb_init_page (struct nvkm_fb *);int gm200_fb_init_page (struct nvkm_fb *); +#endif
drivers/gpu/drm/nouveau/nvkm/subdev/therm/Kbuild +1 -0
@@ -9,6 +9,7 @@ nvkm-y += nvkm/subdev/therm/nv40.o nvkm-y += nvkm/subdev/therm/nv50.o nvkm-y += nvkm/subdev/therm/g84.o nvkm-y += nvkm/subdev/therm/gt215.o +nvkm-y += nvkm/subdev/therm/gf100.o nvkm-y += nvkm/subdev/therm/gf119.o nvkm-y += nvkm/subdev/therm/gk104.o nvkm-y += nvkm/subdev/therm/gm107.o
drivers/gpu/drm/nouveau/nvkm/subdev/therm/base.c +13 -2
@@ -330,7 +330,8 @@ nvkm_therm_clkgate_oneinit(struct nvkm_therm *therm)return ;switch (therm->clkgate_level) { -case NVKM_THERM_CLKGATE_CG : clkgate_str ="CG" ;break ; +case NVKM_THERM_CLKGATE_CG : clkgate_str ="CG" ;break ; +case NVKM_THERM_CLKGATE_BLCG : clkgate_str ="BLCG" ;break ;default :BUG (); }
@@ -400,6 +401,16 @@ nvkm_therm_init(struct nvkm_subdev *subdev)return 0 ; } +void +nvkm_therm_clkgate_init (struct nvkm_therm *therm, +const struct nvkm_therm_clkgate_pack *p) +{ +if (!therm->clkgate_level || !therm->func->clkgate_init) +return ; + + therm->func->clkgate_init (therm, p); +} +static void *nvkm_therm_dtor (struct nvkm_subdev *subdev) {
@@ -438,7 +449,7 @@ nvkm_therm_ctor(struct nvkm_therm *therm, struct nvkm_device *device,clamp ((int )nvkm_longopt (device->cfgopt,"NvPmEnableGating" ,NVKM_THERM_CLKGATE_NONE ), -NVKM_THERM_CLKGATE_NONE ,NVKM_THERM_CLKGATE_CG ); +NVKM_THERM_CLKGATE_NONE ,NVKM_THERM_CLKGATE_BLCG ); }int
drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf100.c new file +77 -0
77 lines not on this page · raw message
drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.c +1 -0
@@ -100,6 +100,7 @@ gk104_therm_func = { .temp_get = g84_temp_get, .fan_sense = gt215_therm_fan_sense, .program_alarms = nvkm_therm_program_alarms_polling, + .clkgate_init = gf100_clkgate_init, .clkgate_enable = gk104_clkgate_enable, .clkgate_fini = gk104_clkgate_fini, };
drivers/gpu/drm/nouveau/nvkm/subdev/therm/gt215.c +1 -1
@@ -36,7 +36,7 @@ gt215_therm_fan_sense(struct nvkm_therm *therm)return -ENODEV ; } -static void +void gt215_therm_init (struct nvkm_therm *therm) {struct nvkm_device *device = therm->subdev.device;
drivers/gpu/drm/nouveau/nvkm/subdev/therm/priv.h +8 -0
@@ -97,6 +97,8 @@ struct nvkm_therm_func {void (*program_alarms)(struct nvkm_therm *); +void (*clkgate_init)(struct nvkm_therm *, +const struct nvkm_therm_clkgate_pack *);void (*clkgate_enable)(struct nvkm_therm *);void (*clkgate_fini)(struct nvkm_therm *,bool ); };
@@ -114,6 +116,9 @@ void g84_therm_fini(struct nvkm_therm *);int gt215_therm_fan_sense (struct nvkm_therm *); +void gf100_clkgate_init (struct nvkm_therm *, +const struct nvkm_therm_clkgate_pack *); +void g84_therm_init (struct nvkm_therm *);int gf119_fan_pwm_ctrl (struct nvkm_therm *,int ,bool );
@@ -122,6 +127,9 @@ int gf119_fan_pwm_set(struct nvkm_therm *, int, u32, u32);int gf119_fan_pwm_clock (struct nvkm_therm *,int );void gf119_therm_init (struct nvkm_therm *); +void gt215_therm_init (struct nvkm_therm *therm); + +void gk104_therm_init (struct nvkm_therm *);void gk104_clkgate_enable (struct nvkm_therm *);void gk104_clkgate_fini (struct nvkm_therm *,bool );
Signature · 6 lines
2.14.3 _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Lyude Paul
maintainerRFC 3/4 drm/nouveau: Add BLCG clockgating for Kepler2
In reply to Lyude PaulThis is mostly the same as Kepler1, but we save even more power!
--- drivers/gpu/drm/nouveau/include/nvkm/subdev/fb.h | 1 + drivers/gpu/drm/nouveau/nvkm/engine/device/base.c | 8 +-- drivers/gpu/drm/nouveau/nvkm/engine/gr/gk110.c | 68 ++++++++++++++++++++ drivers/gpu/drm/nouveau/nvkm/subdev/fb/Kbuild | 1 + drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk110.c | 77 +++++++++++++++++++++++ 5 files changed, 151 insertions(+), 4 deletions(-) create mode 100644 drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk110.c
drivers/gpu/drm/nouveau/include/nvkm/subdev/fb.h +1 -0
@@ -75,6 +75,7 @@ int mcp89_fb_new(struct nvkm_device *, int, struct nvkm_fb **);int gf100_fb_new (struct nvkm_device *,int ,struct nvkm_fb **);int gf108_fb_new (struct nvkm_device *,int ,struct nvkm_fb **);int gk104_fb_new (struct nvkm_device *,int ,struct nvkm_fb **); +int gk110_fb_new (struct nvkm_device *,int ,struct nvkm_fb **);int gk20a_fb_new (struct nvkm_device *,int ,struct nvkm_fb **);int gm107_fb_new (struct nvkm_device *,int ,struct nvkm_fb **);int gm200_fb_new (struct nvkm_device *,int ,struct nvkm_fb **);
drivers/gpu/drm/nouveau/nvkm/engine/device/base.c +4 -4
@@ -1812,7 +1812,7 @@ nvf0_chipset = { .bus = gf100_bus_new, .clk = gk104_clk_new, .devinit = gf100_devinit_new, - .fb = gk104_fb_new, + .fb = gk110_fb_new, .fuse = gf100_fuse_new, .gpio = gk104_gpio_new, .i2c = gk104_i2c_new,
@@ -1850,7 +1850,7 @@ nvf1_chipset = { .bus = gf100_bus_new, .clk = gk104_clk_new, .devinit = gf100_devinit_new, - .fb = gk104_fb_new, + .fb = gk110_fb_new, .fuse = gf100_fuse_new, .gpio = gk104_gpio_new, .i2c = gk104_i2c_new,
@@ -1888,7 +1888,7 @@ nv106_chipset = { .bus = gf100_bus_new, .clk = gk104_clk_new, .devinit = gf100_devinit_new, - .fb = gk104_fb_new, + .fb = gk110_fb_new, .fuse = gf100_fuse_new, .gpio = gk104_gpio_new, .i2c = gk104_i2c_new,
@@ -1926,7 +1926,7 @@ nv108_chipset = { .bus = gf100_bus_new, .clk = gk104_clk_new, .devinit = gf100_devinit_new, - .fb = gk104_fb_new, + .fb = gk110_fb_new, .fuse = gf100_fuse_new, .gpio = gk104_gpio_new, .i2c = gk104_i2c_new,
drivers/gpu/drm/nouveau/nvkm/engine/gr/gk110.c +68 -0
86 lines not on this page · raw message
drivers/gpu/drm/nouveau/nvkm/subdev/fb/Kbuild +1 -0
@@ -22,6 +22,7 @@ nvkm-y += nvkm/subdev/fb/mcp89.o nvkm-y += nvkm/subdev/fb/gf100.o nvkm-y += nvkm/subdev/fb/gf108.o nvkm-y += nvkm/subdev/fb/gk104.o +nvkm-y += nvkm/subdev/fb/gk110.o nvkm-y += nvkm/subdev/fb/gk20a.o nvkm-y += nvkm/subdev/fb/gm107.o nvkm-y += nvkm/subdev/fb/gm200.o
drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk110.c new file +77 -0
77 lines not on this page · raw message
Signature · 6 lines
2.14.3 _______________________________________________ Nouveau mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/nouveau
Lyude Paul
maintainerRFC 4/4 drm/nouveau: Add SLCG clockgating for Kepler2
In reply to Lyude PaulThat's right, there's still more power saving to go! This enables the third level of clockgating, SLCG (this stands for... we don't actually know what this stands for yet :\). While the register values look a little different, programming them is exactly the same as BLCG. Additionally, it should be noted that SLCG was added starting with kepler2, previous generations have no SLCG.
SLCG can be enabled with the nouveau config option, NvPmEnableGating=3
--- .../gpu/drm/nouveau/include/nvkm/subdev/therm.h | 1 + drivers/gpu/drm/nouveau/nvkm/engine/gr/gk110.c | 108 +++++++++++++++++++++ drivers/gpu/drm/nouveau/nvkm/subdev/therm/base.c | 3 +- 3 files changed, 111 insertions(+), 1 deletion(-)
drivers/gpu/drm/nouveau/include/nvkm/subdev/therm.h +1 -0
@@ -50,6 +50,7 @@ enum nvkm_therm_clkgate_level {NVKM_THERM_CLKGATE_NONE =0 ,NVKM_THERM_CLKGATE_CG ,/* basic clockgating */ NVKM_THERM_CLKGATE_BLCG , + NVKM_THERM_CLKGATE_SLCG, }; struct nvkm_therm_clkgate_init {
drivers/gpu/drm/nouveau/nvkm/engine/gr/gk110.c +108 -0
120 lines not on this page · raw message
drivers/gpu/drm/nouveau/nvkm/subdev/therm/base.c +2 -1
@@ -332,6 +332,7 @@ nvkm_therm_clkgate_oneinit(struct nvkm_therm *therm)switch (therm->clkgate_level) {case NVKM_THERM_CLKGATE_CG : clkgate_str ="CG" ;break ;case NVKM_THERM_CLKGATE_BLCG : clkgate_str ="BLCG" ;break ; +case NVKM_THERM_CLKGATE_SLCG : clkgate_str ="SLCG" ;break ;default :BUG (); }
@@ -449,7 +450,7 @@ nvkm_therm_ctor(struct nvkm_therm *therm, struct nvkm_device *device,clamp ((int )nvkm_longopt (device->cfgopt,"NvPmEnableGating" ,NVKM_THERM_CLKGATE_NONE ), -NVKM_THERM_CLKGATE_NONE ,NVKM_THERM_CLKGATE_BLCG ); +NVKM_THERM_CLKGATE_NONE ,NVKM_THERM_CLKGATE_SLCG ); }int
Signature · 6 lines
2.14.3 _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
A message that is not in the archive (sent off-list or not yet received): CACY=MfW3b0qXALO1Y7bt=F7jMCfrdgi3qmmzKOXehJK5Xc5MmQ@mail.gmail.com
A message that is not in the archive (sent off-list or not yet received): CACY=MfVkNPsvNWGiy9o9A=YAZciV5k=jCJ4hrNUBocThgA56sQ@mail.gmail.com
Hello Paul
I have a GTX 480 (GF100 I believe) at home in an old machine. Is the rest of the Fermi patch set available somewhere for me to test?
Thank you
Regards Brock
On 16 Jan. 2018 9:08 am, "Lyude Paul" <[email protected]> wrote:
It's here! After a lot of investigation, rewrites, and traces, I present the patch series to implement all known )levels of clockgating for Kepler1 and Kepler2 GPUs.
Starting with Fermi GPUs (this is probably present on earlier GPUs as well, but with a far less easy to manage interface), nvidia added two clockgating levels that are handled mostly in firmware (with the exception of course, of the driver initially programming all of the register values containing engine delays and that stuff):
- CG_CTRL - Main register for enabling/disabling clockgating for engines and hw blocks
- BLCG - "Block-level clockgating", a deeper level of clockgating
Starting with kepler2 as well, nvidia also introduced:
- SLCG - "??? clockgating" even deeper level of clockgating
Originally this patchset was going to include work for making this work on Fermi, however on closer investigation it seems that Fermi has one pretty big difference in it's requirements that we don't entirely understand yet. On Fermi the CG_CTRL register for the gr needs to be adjusted during reclocking, while kepler and later generations need no such adjustments. Since this requires more research; the current plan is to leave fermi out of this patchset, and then just rework the code at a later point in time to support Fermi once we understand more.
For the time being however, we put as much code we're sure Fermi will be sharing into gf100 files for the future.
For the time being; this patchset supports every GPU in the kepler1 and kepler2 family. The main kernel config option to enable this is
config=NvPmEnableGating=<level>
Where <level> is the deepest level of powersaving to enable. The levels being:
0. NOCG
1. CG (just CG_CTRL)
2. BLCG
3. SLCG
Additionally, we leave a couple of small TODO comments to mark spots we'll eventually need to add code for the final level of power saving we have yet to implement or understand: ELPG (engine-level powergating).
Here's some very lazily done benchmarks on how much power this saves as well. It should be noted that chances are, these patches save a lot more power then you see here since clockgating works best when the GPU is under load. I have a feeling this will be especially true with SLCG.
GK104 stats:
Idle (1 4k display @ fbcon):
NOCG:
07: 20.53W
0a: 44.12W
0e: 61.23W
0f: 62.23W
CG:
07: 18.95W-19.02W
0a: 34.95W-37.48W
0e: 52.70W-54.32W
0f: 53.53W-55.68W
BLCG:
07: 18.64W-19.02W
0a: 32.68W-34.78W
0e: 49.14W-50.51W
0f: 49.75W-51.73W
GK110 stats:
Idle (1 4k display @ fbcon):
NOCG:
07: 25.68W
0a: 36.16W
0d: 64.71W
0f: 64.99W
CG:
07: 25.68W
0a: 35.58W
0d: 61.54W-61.74W
0f: 61.93W-62.12W
BLCG:
07: 25W-25.2W
0a: 33.85W-34.04W
0d: 59.25W-59.44W
0f: 59.35W-59.54W
SLCG:
07: 25W-25.2W
0a: 33.85W-33.95W (spikes to 34.04W briefly every now and then)
0d: 59.15W-59.35W
0f: 59.35W-59.54WFor the time being, this will be disabled by default in the kernel as we wait to get more widespread testing with this patchset. If you're willing to put up with the potential of instability, please feel free to try this patchset and let us know how well it works for you!
Lyude Paul (4):
drm/nouveau: Add support for basic clockgating on Kepler1 drm/nouveau: Add support for BLCG clockgating for Kepler1 drm/nouveau: Add BLCG clockgating for Kepler2 drm/nouveau: Add SLCG clockgating for Kepler2
drivers/gpu/drm/nouveau/include/nvkm/subdev/fb.h | 1 + .../gpu/drm/nouveau/include/nvkm/subdev/therm.h | 25 +++ drivers/gpu/drm/nouveau/nvkm/engine/device/base.c | 25 +-- drivers/gpu/drm/nouveau/nvkm/engine/gr/gf100.h | 1 + drivers/gpu/drm/nouveau/nvkm/engine/gr/gk104.c | 215 ++++++++++++++++++++- drivers/gpu/drm/nouveau/nvkm/engine/gr/gk104.h | 55 ++++++ drivers/gpu/drm/nouveau/nvkm/engine/gr/gk110.c | 176 +++++++++++++++++ drivers/gpu/drm/nouveau/nvkm/subdev/fb/Kbuild | 1 + drivers/gpu/drm/nouveau/nvkm/subdev/fb/gf100.c | 6 + drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk104.c | 53 +++++ drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk104.h | 35 ++++ drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk110.c | 77 ++++++++ drivers/gpu/drm/nouveau/nvkm/subdev/fb/priv.h | 3 + drivers/gpu/drm/nouveau/nvkm/subdev/therm/Kbuild | 2 + drivers/gpu/drm/nouveau/nvkm/subdev/therm/base.c | 84 +++++++- drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf100.c | 77 ++++++++ drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf100.h | 35 ++++ drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf119.c | 8 +- drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.c | 136 +++++++++++++ drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.h | 56 ++++++ drivers/gpu/drm/nouveau/nvkm/subdev/therm/gt215.c | 2 +- drivers/gpu/drm/nouveau/nvkm/subdev/therm/priv.h | 23 ++- 22 files changed, 1069 insertions(+), 27 deletions(-) create mode 100644 drivers/gpu/drm/nouveau/nvkm/engine/gr/gk104.h create mode 100644 drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk104.h create mode 100644 drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk110.c create mode 100644 drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf100.c create mode 100644 drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf100.h create mode 100644 drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.c create mode 100644 drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.h
--
2.14.3
_______________________________________________
Nouveau mailing list [email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau
_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau
Lyude Paul
maintainerUnfortunately no. in order to support reclocking on Fermi with powergating (which is important; since you can't test clockgating very well without it) we are going to need to do some more investigation. Unlike Kepler and later generations, Fermi seems to require some extra maintanence of it's clockgating registers (CG_CTRL) during reclocking operations that we don't quite understand yet. We'll figure it out, but I'd like to get out initial support for Kepler first since I'm confident this series is pretty solid and it would be good to get early feedback on this.
Plus, Fermi doesn't have working reclocking in nouveau just yet. I am planning on eventually adding Fermi clockgating support when reclocking gets fixed!
Quoted 148 lines · Brock York
Hello Paul
I have a GTX 480 (GF100 I believe) at home in an old machine. Is the rest of the Fermi patch set available somewhere for me to test?
Thank you
Regards BrockQuoted 137 lines · Lyude Paul
It's here! After a lot of investigation, rewrites, and traces, I present the patch series to implement all known )levels of clockgating for Kepler1 and Kepler2 GPUs.
Starting with Fermi GPUs (this is probably present on earlier GPUs as well, but with a far less easy to manage interface), nvidia added two clockgating levels that are handled mostly in firmware (with the exception of course, of the driver initially programming all of the register values containing engine delays and that stuff):
-CG_CTRL- Main register for enabling/disabling clockgating for engines and hw blocks
- BLCG - "Block-level clockgating", a deeper level of clockgating
Starting with kepler2 as well, nvidia also introduced:
- SLCG - "??? clockgating" even deeper level of clockgatingOriginally this patchset was going to include work for making this work on Fermi, however on closer investigation it seems that Fermi has one pretty big difference in it's requirements that we don't entirely understand yet. On Fermi the
CG_CTRLregister for the gr needs to be adjusted during reclocking, while kepler and later generations need no such adjustments. Since this requires more research; the current plan is to leave fermi out of this patchset, and then just rework the code at a later point in time to support Fermi once we understand more.For the time being however, we put as much code we're sure Fermi will be sharing into gf100 files for the future.
For the time being; this patchset supports every GPU in the kepler1 and kepler2 family. The main kernel config option to enable this is
config=NvPmEnableGating=<level>
Where <level> is the deepest level of powersaving to enable. The levels being:
0. NOCG
1. CG (justCG_CTRL)
2. BLCG
3. SLCGAdditionally, we leave a couple of small TODO comments to mark spots we'll eventually need to add code for the final level of power saving we have yet to implement or understand: ELPG (engine-level powergating).
Here's some very lazily done benchmarks on how much power this saves as well. It should be noted that chances are, these patches save a lot more power then you see here since clockgating works best when the GPU is under load. I have a feeling this will be especially true with SLCG.
GK104 stats:
Idle (1 4k display @ fbcon):NOCG: 07: 20.53W 0a: 44.12W 0e: 61.23W 0f: 62.23W CG: 07: 18.95W-19.02W 0a: 34.95W-37.48W 0e: 52.70W-54.32W 0f: 53.53W-55.68W BLCG: 07: 18.64W-19.02W 0a: 32.68W-34.78W 0e: 49.14W-50.51W 0f: 49.75W-51.73W GK110 stats: Idle (1 4k display @ fbcon): NOCG: 07: 25.68W 0a: 36.16W 0d: 64.71W 0f: 64.99W CG: 07: 25.68W 0a: 35.58W 0d: 61.54W-61.74W 0f: 61.93W-62.12W BLCG: 07: 25W-25.2W 0a: 33.85W-34.04W 0d: 59.25W-59.44W 0f: 59.35W-59.54W SLCG: 07: 25W-25.2W 0a: 33.85W-33.95W (spikes to 34.04W briefly every now and then) 0d: 59.15W-59.35W 0f: 59.35W-59.54WFor the time being, this will be disabled by default in the kernel as we wait to get more widespread testing with this patchset. If you're willing to put up with the potential of instability, please feel free to try this patchset and let us know how well it works for you!
Lyude Paul (4):
drm/nouveau: Add support for basic clockgating on Kepler1 drm/nouveau: Add support for BLCG clockgating for Kepler1 drm/nouveau: Add BLCG clockgating for Kepler2 drm/nouveau: Add SLCG clockgating for Kepler2drivers/gpu/drm/nouveau/include/nvkm/subdev/fb.h | 1 + .../gpu/drm/nouveau/include/nvkm/subdev/therm.h | 25 +++ drivers/gpu/drm/nouveau/nvkm/engine/device/base.c | 25 +-- drivers/gpu/drm/nouveau/nvkm/engine/gr/gf100.h | 1 + drivers/gpu/drm/nouveau/nvkm/engine/gr/gk104.c | 215 ++++++++++++++++++++- drivers/gpu/drm/nouveau/nvkm/engine/gr/gk104.h | 55 ++++++ drivers/gpu/drm/nouveau/nvkm/engine/gr/gk110.c | 176 +++++++++++++++++ drivers/gpu/drm/nouveau/nvkm/subdev/fb/Kbuild | 1 + drivers/gpu/drm/nouveau/nvkm/subdev/fb/gf100.c | 6 + drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk104.c | 53 +++++ drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk104.h | 35 ++++ drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk110.c | 77 ++++++++ drivers/gpu/drm/nouveau/nvkm/subdev/fb/priv.h | 3 + drivers/gpu/drm/nouveau/nvkm/subdev/therm/Kbuild | 2 + drivers/gpu/drm/nouveau/nvkm/subdev/therm/base.c | 84 +++++++- drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf100.c | 77 ++++++++ drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf100.h | 35 ++++ drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf119.c | 8 +- drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.c | 136 +++++++++++++ drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.h | 56 ++++++ drivers/gpu/drm/nouveau/nvkm/subdev/therm/gt215.c | 2 +- drivers/gpu/drm/nouveau/nvkm/subdev/therm/priv.h | 23 ++- 22 files changed, 1069 insertions(+), 27 deletions(-) create mode 100644 drivers/gpu/drm/nouveau/nvkm/engine/gr/gk104.h create mode 100644 drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk104.h create mode 100644 drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk110.c create mode 100644 drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf100.c create mode 100644 drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf100.h create mode 100644 drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.c create mode 100644 drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.h--
2.14.3_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau
Signature · 6 lines
Cheers, Lyude Paul _______________________________________________ Nouveau mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/nouveau
Quoted 4 lines · Lyude Paul
It's here! After a lot of investigation, rewrites, and traces, I present the patch series to implement all known levels of clockgating for Kepler1 and Kepler2 GPUs.
Quoted 10 lines
Starting with Fermi GPUs (this is probably present on earlier GPUs as well, but with a far less easy to manage interface), nvidia added two clockgating levels that are handled mostly in firmware (with the exception of course, of the driver initially programming all of the register values containing engine delays and that stuff):
-CG_CTRL- Main register for enabling/disabling clockgating for engines and hw blocks
- BLCG - "Block-level clockgating", a deeper level of clockgating
Starting with kepler2 as well, nvidia also introduced:
- SLCG - "??? clockgating" even deeper level of clockgating
FWIW, SLCG stands for "second level clock gating".
Cheers,
Mikko