nouveau : fix r570 suspend/resume regression
Subsystems (4)
Dave Airlie
maintainerSince R570 firmware got enabled, a number of GPUs have not being successfully suspend/resuming. Lyude showed it on RTX6000, which I reproduced and wasted a lot of time down various rabbit holes.
There are two required fixes, the first one adds proper sequence numbers to the rpc messages, which fixes a bunch of NOCAT asserts.
Then we have to pass the runtime vs non-runtime state down to the GSP fbsr code. This however requires replacing a bool with an enum which refactors quite a bunch of interfaces unfortunately, but it was the cleanest way to do it.
The final patch hooks up the interface so normal suspend doesn't set the GCOFF flags.
Dave.
Dave Airlie
maintainerPATCH 1/3 nouveau/gsp: use rpc sequence numbers properly.
From: Dave Airlie <[email protected]>
There are two layers of sequence numbers, one at the msg level and one at the rpc level.
570 firmware started asserting on the sequence numbers being in the right order, and we would see nocat records with asserts in them.
Add the rpc level sequence number support.
--- drivers/gpu/drm/nouveau/include/nvkm/subdev/gsp.h | 6 ++++++ drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r535/gsp.c | 4 ++-- drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r535/rpc.c | 6 ++++++ drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/gsp.c | 2 +- 4 files changed, 15 insertions(+), 3 deletions(-)
drivers/gpu/drm/nouveau/include/nvkm/subdev/gsp.h +6 -0
@@ -44,6 +44,9 @@ typedef void (*nvkm_gsp_event_func)(struct nvkm_gsp_event *, void *repv, u32 rep* NVKM_GSP_RPC_REPLY_NOWAIT - If specified, immediately return to the * caller after the GSP RPC command is issued. * +* NVKM_GSP_RPC_REPLY_NOSEQ - If specified, exactly like NOWAIT +* but don't emit RPC sequence number. +* * NVKM_GSP_RPC_REPLY_RECV - If specified, wait and receive the entire GSP * RPC message after the GSP RPC command is issued. *
@@ -53,6 +56,7 @@ typedef void (*nvkm_gsp_event_func)(struct nvkm_gsp_event *, void *repv, u32 rep*/ enum nvkm_gsp_rpc_reply_policy {NVKM_GSP_RPC_REPLY_NOWAIT =0 , +NVKM_GSP_RPC_REPLY_NOSEQ ,NVKM_GSP_RPC_REPLY_RECV ,NVKM_GSP_RPC_REPLY_POLL , };
@@ -242,6 +246,8 @@ struct nvkm_gsp {/* The size of the registry RPC */ size_t registry_rpc_size; +u32 rpc_seq; +#ifdef CONFIG_DEBUG_FS /* * Logging buffers in debugfs. The wrapper objects need to remain
drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r535/gsp.c +2 -2
@@ -704,7 +704,7 @@ r535_gsp_rpc_set_registry(struct nvkm_gsp *gsp)build_registry (gsp, rpc); -return nvkm_gsp_rpc_wr (gsp, rpc,NVKM_GSP_RPC_REPLY_NOWAIT ); +return nvkm_gsp_rpc_wr (gsp, rpc,NVKM_GSP_RPC_REPLY_NOSEQ ); fail:clean_registry (gsp);
@@ -921,7 +921,7 @@ r535_gsp_set_system_info(struct nvkm_gsp *gsp) info->pciConfigMirrorSize = device->pci->func->cfg.size;r535_gsp_acpi_info (gsp, &info->acpiMethodData); -return nvkm_gsp_rpc_wr (gsp, info,NVKM_GSP_RPC_REPLY_NOWAIT ); +return nvkm_gsp_rpc_wr (gsp, info,NVKM_GSP_RPC_REPLY_NOSEQ ); } static int
drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r535/rpc.c +6 -0
@@ -557,6 +557,7 @@ r535_gsp_rpc_handle_reply(struct nvkm_gsp *gsp, u32 fn,switch (policy) {case NVKM_GSP_RPC_REPLY_NOWAIT : +case NVKM_GSP_RPC_REPLY_NOSEQ :break ;case NVKM_GSP_RPC_REPLY_RECV : reply =r535_gsp_msg_recv (gsp, fn, gsp_rpc_len);
@@ -588,6 +589,11 @@ r535_gsp_rpc_send(struct nvkm_gsp *gsp, void *payload, rpc->data, rpc->length - sizeof(*rpc), true); } +if (policy ==NVKM_GSP_RPC_REPLY_NOSEQ ) + rpc->sequence =0 ; +else + rpc->sequence = gsp->rpc_seq++; + ret =r535_gsp_cmdq_push (gsp, rpc);if (ret)return ERR_PTR (ret);
drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/gsp.c +1 -1
@@ -176,7 +176,7 @@ r570_gsp_set_system_info(struct nvkm_gsp *gsp) info->bIsPrimary =video_is_primary_device (device->dev); info->bPreserveVideoMemoryAllocations = false; -return nvkm_gsp_rpc_wr (gsp, info,NVKM_GSP_RPC_REPLY_NOWAIT ); +return nvkm_gsp_rpc_wr (gsp, info,NVKM_GSP_RPC_REPLY_NOSEQ ); } static void
Signature · 1 line
2.52.0
Re: [PATCH 1/3] nouveau/gsp: use rpc sequence numbers properly.
Quoted 11 lines · Dave Airlie
drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r535/gsp.c +0
-0@@ -704,7 +704,7 @@ r535_gsp_rpc_set_registry(struct nvkm_gsp *gsp)
build_registry(gsp, rpc);drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r535/gsp.c +1
-1-return nvkm_gsp_rpc_wr (gsp, rpc,NVKM_GSP_RPC_REPLY_NOWAIT ); +return nvkm_gsp_rpc_wr (gsp, rpc,NVKM_GSP_RPC_REPLY_NOSEQ );
Quoted 3 lines
fail:
clean_registry(gsp);
So are you saying that some RPC commands need to have a sequence number set, and some do not?
Dave Airlie
maintainerRe: [PATCH 1/3] nouveau/gsp: use rpc sequence numbers properly.
Quoted 11 lines · Timur Tabi
Quoted 9 lines · Dave Airlie
drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r535/gsp.c +0
-0@@ -704,7 +704,7 @@ r535_gsp_rpc_set_registry(struct nvkm_gsp *gsp)build_registry (gsp, rpc);
Quoted 7 lines
drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r535/gsp.c r535_gsp_rpc_set_registry(struct nvkm_gsp *gsp) · line 710
Quoted 5 lines
drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r535/gsp.c +1
-1-return nvkm_gsp_rpc_wr (gsp, rpc,NVKM_GSP_RPC_REPLY_NOWAIT ); +return nvkm_gsp_rpc_wr (gsp, rpc,NVKM_GSP_RPC_REPLY_NOSEQ ); fail:clean_registry (gsp);So are you saying that some RPC commands need to have a sequence number set, and some do not?
I'm copying the behaviour of opengpu here,src/kernel/gpu/gsp/kernel_gsp.c
if (pSequence)
vgpu_rpc_message_header_v->sequence = *pSequence = pRpc->sequence++;
else
vgpu_rpc_message_header_v->sequence = 0;src/kernel/vgpu/rpc.c:_issueRpcAsync
doesn't pass pSequence_issueRpcAndWait does pass it.
The SetSystemInfo and SetRegistry are the two async calls in nouveau.
So I'm not saying some RPC commands need to have a sequence number and some don't, that would be up to someone who can access GSP source code, I'm saying that opengpu does this and I'm copying it :-)
Dave.
Re: [PATCH 1/3] nouveau/gsp: use rpc sequence numbers properly.
Quoted 1 line · Dave Airlie
On Thu, 5 Feb 2026 at 07:36, Timur Tabi <[email protected]> wrote:
...
Quoted 13 lines
Quoted 1 line
So are you saying that some RPC commands need to have a sequence number set, and some do not?
I'm copying the behaviour of opengpu here,
src/kernel/gpu/gsp/kernel_gsp.c
if (pSequence)vgpu_rpc_message_header_v->sequence = *pSequence = pRpc->sequence++; else vgpu_rpc_message_header_v->sequence = 0;src/kernel/vgpu/rpc.c:_issueRpcAsync
doesn't pass pSequence_issueRpcAndWaitdoes pass it.
Quoted 6 lines
The SetSystemInfo and SetRegistry are the two async calls in nouveau.
So I'm not saying some RPC commands need to have a sequence number and some don't, that would be up to someone who can access GSP source code, I'm saying that opengpu does this and I'm copying it :-)
lol I love it. If only someone here had access to the GSP code and the GSP team! :)
OK, then, I just had a very enlightening call with one of our GSP RPC experts, in order to learn what the sequence number story really is. The notes below are sort of Nova-centric, but it should apply equally to Nouveau.
Let me Cc Eliot, because he was also fixing up other aspects of GSP RPC calls on Nova, in case this overlaps.
==========================================================
Background
==========
Today there are some loose ends and inconsistencies even in the Open RM + GSP scenario, for how sequence numbers are used. And these are being cleaned up and fixed. In fact, I was even able to request, and receive some nice clean behavior, which will be implemented in GSP soon (we'll get it when we upgrade, likely sometime this year).
Today, there are 2+ sequence number spaces, one for send-receive pairs (command + response) RPC calls, and another for GSP-initiated ("async") messages to the CPU.
The "2+" is because there is an inconsistency (it will be fixed in GSP), leading to the first two very early RPC calls being in yet another unaccounted for number space. These:
NovaCore 0000:01:00.0: GSP RPC: send: seq# 0, function=GSP_SET_SYSTEM_INFO, length=0x3f0 NovaCore 0000:01:00.0: GSP RPC: send: seq# 1, function=SET_REGISTRY, length=0xc5
...are not included in the counting, by GSP.
The GSP finally starts counting up when it gets the first "non-async" (command/response) message, here:
NovaCore 0000:01:00.0: GSP RPC: send: seq# 2, function=GET_GSP_STATIC_INFO, length=0x6c8 NovaCore 0000:01:00.0: GSP RPC: receive: seq# 0, function=Ok(GetGspStaticInfo), length=0x6c8
But even here, it's not what I think we want, because we want the CPU to get back the same seq num that it sent, for command/response pairs. But that's not what actually happens (at least not directly).
So for now, seq numbers on Nova and Nouveau are generally "do what you want, it will work ok". But actually, we will soon be able to use them for
a) debugging aids,
b) detecting missing messages, and
c) recovering from "CPU sent a message, timed out waiting for a response" cases.
==========================================================
Next steps for Nova (and maybe Nouveau, if not already done)
============================================================
a) Change debug output to print the seq number numeric space, which is either "async message from GSP" or "command response from GSP".
b) Put a comment in the code to indicate that GSP_SET_SYSTEM_INFO and SET_REGISTRY do not yet participate in the incrementing seq number system, but will in future GSP versions.
I'll send patches for Nova soon, to do the above.
==========================================================
Examples to illustrate:
=======================
Today's Nova debug logs on Blackwell:
NovaCore 0000:01:00.0: GSP RPC: send: seq# 0, function=GSP_SET_SYSTEM_INFO, length=0x3f0 NovaCore 0000:01:00.0: GSP RPC: send: seq# 1, function=SET_REGISTRY, length=0xc5 NovaCore 0000:01:00.0: GSP RPC: receive: seq# 0, function=Ok(GspLockdownNotice), length=0x51 NovaCore 0000:01:00.0: GSP RPC: receive: seq# 0, function=Ok(GspLockdownNotice), length=0x51 NovaCore 0000:01:00.0: GSP RPC: receive: seq# 0, function=Ok(UcodeLibOsPrint), length=0x68 NovaCore 0000:01:00.0: GSP RPC: receive: seq# 0, function=Ok(UcodeLibOsPrint), length=0x70 NovaCore 0000:01:00.0: GSP RPC: receive: seq# 0, function=Ok(GspPostNoCat), length=0x50c NovaCore 0000:01:00.0: GSP RPC: receive: seq# 0, function=Ok(GspLockdownNotice), length=0x51 ...more of these...
NovaCore 0000:01:00.0: GSP RPC: receive: seq# 0, function=Ok(GspInitDone), length=0x50 NovaCore 0000:01:00.0: GSP RPC: send: seq# 2, function=GET_GSP_STATIC_INFO, length=0x6c8 NovaCore 0000:01:00.0: GSP RPC: receive: seq# 0, function=Ok(GetGspStaticInfo), length=0x6c8
Future (after changing Nova first, and eventually upgrading GSP):
NovaCore 0000:01:00.0: GSP RPC: send: seq# 0, function=GSP_SET_SYSTEM_INFO, length=0x3f0 NovaCore 0000:01:00.0: GSP RPC: send: seq# 1, function=SET_REGISTRY, length=0xc5 NovaCore 0000:01:00.0: GSP RPC: async received: seq# 0, function=Ok(GspLockdownNotice), length=0x51 NovaCore 0000:01:00.0: GSP RPC: async received: seq# 1, function=Ok(GspLockdownNotice), length=0x51 NovaCore 0000:01:00.0: GSP RPC: async received: seq# 2, function=Ok(UcodeLibOsPrint), length=0x68 NovaCore 0000:01:00.0: GSP RPC: async received: seq# 3, function=Ok(UcodeLibOsPrint), length=0x70 NovaCore 0000:01:00.0: GSP RPC: async received: seq# 4, function=Ok(GspPostNoCat), length=0x50c NovaCore 0000:01:00.0: GSP RPC: async received: seq# 5 function=Ok(GspLockdownNotice), length=0x51 ...more of these...
NovaCore 0000:01:00.0: GSP RPC: async received: seq# 12, function=Ok(GspInitDone), length=0x50 NovaCore 0000:01:00.0: GSP RPC: send: seq# 2, function=GET_GSP_STATIC_INFO, length=0x6c8 NovaCore 0000:01:00.0: GSP RPC: response received: seq# 2, function=Ok(GetGspStaticInfo), length=0x6c8
As you can see, the CPU side can then track the two types of messages clearly.
thanks,
Signature · 1 line
John Hubbard
Dave Airlie
maintainerRe: [PATCH 1/3] nouveau/gsp: use rpc sequence numbers properly.
Quoted 86 lines · John Hubbard
Quoted 1 line · Dave Airlie
On Thu, 5 Feb 2026 at 07:36, Timur Tabi <[email protected]> wrote:
...
Quoted 13 lines
Quoted 1 line
So are you saying that some RPC commands need to have a sequence number set, and some do not?
I'm copying the behaviour of opengpu here,
src/kernel/gpu/gsp/kernel_gsp.c
if (pSequence)vgpu_rpc_message_header_v->sequence = *pSequence = pRpc->sequence++; else vgpu_rpc_message_header_v->sequence = 0;src/kernel/vgpu/rpc.c:_issueRpcAsync
doesn't pass pSequence_issueRpcAndWaitdoes pass it.Quoted 6 lines
The SetSystemInfo and SetRegistry are the two async calls in nouveau.
So I'm not saying some RPC commands need to have a sequence number and some don't, that would be up to someone who can access GSP source code, I'm saying that opengpu does this and I'm copying it :-)
lol I love it. If only someone here had access to the GSP code and the GSP team! :)
OK, then, I just had a very enlightening call with one of our GSP RPC experts, in order to learn what the sequence number story really is. The notes below are sort of Nova-centric, but it should apply equally to Nouveau.
Let me Cc Eliot, because he was also fixing up other aspects of GSP RPC calls on Nova, in case this overlaps.
==========================================================
Background
==========Today there are some loose ends and inconsistencies even in the Open RM + GSP scenario, for how sequence numbers are used. And these are being cleaned up and fixed. In fact, I was even able to request, and receive some nice clean behavior, which will be implemented in GSP soon (we'll get it when we upgrade, likely sometime this year).
Today, there are 2+ sequence number spaces, one for send-receive pairs (command + response) RPC calls, and another for GSP-initiated ("async") messages to the CPU.
The "2+" is because there is an inconsistency (it will be fixed in GSP), leading to the first two very early RPC calls being in yet another unaccounted for number space. These:
NovaCore 0000:01:00.0: GSP RPC: send: seq# 0, function=GSP_SET_SYSTEM_INFO, length=0x3f0 NovaCore 0000:01:00.0: GSP RPC: send: seq# 1, function=SET_REGISTRY, length=0xc5
...are not included in the counting, by GSP.
The GSP finally starts counting up when it gets the first "non-async" (command/response) message, here:
NovaCore 0000:01:00.0: GSP RPC: send: seq# 2, function=GET_GSP_STATIC_INFO, length=0x6c8 NovaCore 0000:01:00.0: GSP RPC: receive: seq# 0, function=Ok(GetGspStaticInfo), length=0x6c8
But even here, it's not what I think we want, because we want the CPU to get back the same seq num that it sent, for command/response pairs. But that's not what actually happens (at least not directly).
So for now, seq numbers on Nova and Nouveau are generally "do what you want, it will work ok". But actually, we will soon be able to use them for
a) debugging aids,
b) detecting missing messages, and
c) recovering from "CPU sent a message, timed out waiting for a response" cases.==========================================================
Next steps for Nova (and maybe Nouveau, if not already done)
============================================================a) Change debug output to print the seq number numeric space, which is either "async message from GSP" or "command response from GSP".
Quoted 5 lines
b) Put a comment in the code to indicate that
GSP_SET_SYSTEM_INFOandSET_REGISTRYdo not yet participate in the incrementing seq number system, but will in future GSP versions.I'll send patches for Nova soon, to do the above.
So just FYI nova gets this wrong as well.
There are two sequence numbers:
the one in GSP_MSG_QUEUE_ELEMENT and the one in rpc_message_header_v
The rules for the first seem to be just increment, the rules for the 2nd seems to be along the lines above, two msgs get 0, then start from 0.
Currently we never init the rpc message header sequence to anything but 0.
Dave.
Re: [PATCH 1/3] nouveau/gsp: use rpc sequence numbers properly.
Quoted 96 lines · Dave Airlie
Quoted 85 lines · John Hubbard
Quoted 1 line · Dave Airlie
On Thu, 5 Feb 2026 at 07:36, Timur Tabi <[email protected]> wrote:
...
Quoted 13 lines
Quoted 1 line
So are you saying that some RPC commands need to have a sequence number set, and some do not?
I'm copying the behaviour of opengpu here,
src/kernel/gpu/gsp/kernel_gsp.c
if (pSequence)vgpu_rpc_message_header_v->sequence = *pSequence = pRpc->sequence++; else vgpu_rpc_message_header_v->sequence = 0;src/kernel/vgpu/rpc.c:_issueRpcAsync
doesn't pass pSequence_issueRpcAndWaitdoes pass it.Quoted 6 lines
The SetSystemInfo and SetRegistry are the two async calls in nouveau.
So I'm not saying some RPC commands need to have a sequence number and some don't, that would be up to someone who can access GSP source code, I'm saying that opengpu does this and I'm copying it :-)
lol I love it. If only someone here had access to the GSP code and the GSP team! :)
OK, then, I just had a very enlightening call with one of our GSP RPC experts, in order to learn what the sequence number story really is. The notes below are sort of Nova-centric, but it should apply equally to Nouveau.
Let me Cc Eliot, because he was also fixing up other aspects of GSP RPC calls on Nova, in case this overlaps.
==========================================================
Background
==========Today there are some loose ends and inconsistencies even in the Open RM + GSP scenario, for how sequence numbers are used. And these are being cleaned up and fixed. In fact, I was even able to request, and receive some nice clean behavior, which will be implemented in GSP soon (we'll get it when we upgrade, likely sometime this year).
Today, there are 2+ sequence number spaces, one for send-receive pairs (command + response) RPC calls, and another for GSP-initiated ("async") messages to the CPU.
The "2+" is because there is an inconsistency (it will be fixed in GSP), leading to the first two very early RPC calls being in yet another unaccounted for number space. These:
NovaCore 0000:01:00.0: GSP RPC: send: seq# 0, function=GSP_SET_SYSTEM_INFO, length=0x3f0 NovaCore 0000:01:00.0: GSP RPC: send: seq# 1, function=SET_REGISTRY, length=0xc5
...are not included in the counting, by GSP.
The GSP finally starts counting up when it gets the first "non-async" (command/response) message, here:
NovaCore 0000:01:00.0: GSP RPC: send: seq# 2, function=GET_GSP_STATIC_INFO, length=0x6c8 NovaCore 0000:01:00.0: GSP RPC: receive: seq# 0, function=Ok(GetGspStaticInfo), length=0x6c8
But even here, it's not what I think we want, because we want the CPU to get back the same seq num that it sent, for command/response pairs. But that's not what actually happens (at least not directly).
So for now, seq numbers on Nova and Nouveau are generally "do what you want, it will work ok". But actually, we will soon be able to use them for
a) debugging aids,
b) detecting missing messages, and
c) recovering from "CPU sent a message, timed out waiting for a response" cases.==========================================================
Next steps for Nova (and maybe Nouveau, if not already done)
============================================================a) Change debug output to print the seq number numeric space, which is either "async message from GSP" or "command response from GSP".
Quoted 5 lines
b) Put a comment in the code to indicate that
GSP_SET_SYSTEM_INFOandSET_REGISTRYdo not yet participate in the incrementing seq number system, but will in future GSP versions.I'll send patches for Nova soon, to do the above.
So just FYI nova gets this wrong as well.
There are two sequence numbers:
Quoted 8 lines
the one in
GSP_MSG_QUEUE_ELEMENTand the one inrpc_message_header_vThe rules for the first seem to be just increment, the rules for the 2nd seems to be along the lines above, two msgs get 0, then start from 0.
Currently we never init the rpc message header sequence to anything but 0.
Yes. That's accurate. To be fixed.
thanks,
Signature · 1 line
John Hubbard
Dave Airlie
maintainerPATCH 2/3 nouveau: add a third state to the fini handler.
In reply to Dave AirlieFrom: Dave Airlie <[email protected]>
This is just refactoring to allow the lower layers to distinguish between suspend and runtime suspend.
GSP 570 needs to set a flag with the GPU is going into GCOFF, this flag taken from the opengpu driver is set whenever runtime suspend is enterning GCOFF but not for normal suspend paths.
This just refactors the code, a subsequent patch use the information.
--- drivers/gpu/drm/nouveau/include/nvif/client.h | 2 +- drivers/gpu/drm/nouveau/include/nvif/driver.h | 2 +- .../drm/nouveau/include/nvkm/core/device.h | 3 ++- .../drm/nouveau/include/nvkm/core/engine.h | 2 +- .../drm/nouveau/include/nvkm/core/object.h | 5 +++-- .../drm/nouveau/include/nvkm/core/oproxy.h | 2 +- .../drm/nouveau/include/nvkm/core/subdev.h | 4 ++-- .../nouveau/include/nvkm/core/suspend_state.h | 11 ++++++++++ drivers/gpu/drm/nouveau/nouveau_drm.c | 2 +- drivers/gpu/drm/nouveau/nouveau_nvif.c | 10 +++++++-- drivers/gpu/drm/nouveau/nvif/client.c | 4 ++-- drivers/gpu/drm/nouveau/nvkm/core/engine.c | 4 ++-- drivers/gpu/drm/nouveau/nvkm/core/ioctl.c | 4 ++-- drivers/gpu/drm/nouveau/nvkm/core/object.c | 20 +++++++++++++---- drivers/gpu/drm/nouveau/nvkm/core/oproxy.c | 2 +- drivers/gpu/drm/nouveau/nvkm/core/subdev.c | 18 ++++++++++++--- drivers/gpu/drm/nouveau/nvkm/core/uevent.c | 2 +- .../gpu/drm/nouveau/nvkm/engine/ce/ga100.c | 2 +- drivers/gpu/drm/nouveau/nvkm/engine/ce/priv.h | 2 +- .../gpu/drm/nouveau/nvkm/engine/device/base.c | 22 ++++++++++++++----- .../gpu/drm/nouveau/nvkm/engine/device/pci.c | 4 ++-- .../gpu/drm/nouveau/nvkm/engine/device/priv.h | 2 +- .../gpu/drm/nouveau/nvkm/engine/device/user.c | 2 +- .../gpu/drm/nouveau/nvkm/engine/disp/base.c | 4 ++-- .../gpu/drm/nouveau/nvkm/engine/disp/chan.c | 2 +- drivers/gpu/drm/nouveau/nvkm/engine/falcon.c | 4 ++-- .../gpu/drm/nouveau/nvkm/engine/fifo/base.c | 2 +- .../gpu/drm/nouveau/nvkm/engine/fifo/uchan.c | 6 ++--- drivers/gpu/drm/nouveau/nvkm/engine/gr/base.c | 4 ++-- .../gpu/drm/nouveau/nvkm/engine/gr/gf100.c | 2 +- drivers/gpu/drm/nouveau/nvkm/engine/gr/nv04.c | 2 +- drivers/gpu/drm/nouveau/nvkm/engine/gr/nv10.c | 2 +- drivers/gpu/drm/nouveau/nvkm/engine/gr/nv20.c | 2 +- drivers/gpu/drm/nouveau/nvkm/engine/gr/nv20.h | 2 +- drivers/gpu/drm/nouveau/nvkm/engine/gr/nv40.c | 4 ++-- .../gpu/drm/nouveau/nvkm/engine/mpeg/nv44.c | 2 +- .../gpu/drm/nouveau/nvkm/engine/sec2/base.c | 2 +- drivers/gpu/drm/nouveau/nvkm/engine/xtensa.c | 4 ++-- .../gpu/drm/nouveau/nvkm/subdev/acr/base.c | 2 +- .../gpu/drm/nouveau/nvkm/subdev/bar/base.c | 2 +- .../gpu/drm/nouveau/nvkm/subdev/clk/base.c | 2 +- .../drm/nouveau/nvkm/subdev/devinit/base.c | 4 ++-- .../gpu/drm/nouveau/nvkm/subdev/fault/base.c | 2 +- .../gpu/drm/nouveau/nvkm/subdev/fault/user.c | 2 +- .../gpu/drm/nouveau/nvkm/subdev/gpio/base.c | 2 +- .../gpu/drm/nouveau/nvkm/subdev/gsp/base.c | 2 +- .../gpu/drm/nouveau/nvkm/subdev/gsp/gh100.c | 2 +- .../gpu/drm/nouveau/nvkm/subdev/gsp/priv.h | 8 +++---- .../drm/nouveau/nvkm/subdev/gsp/rm/r535/gsp.c | 2 +- .../gpu/drm/nouveau/nvkm/subdev/gsp/tu102.c | 2 +- .../gpu/drm/nouveau/nvkm/subdev/i2c/base.c | 2 +- .../drm/nouveau/nvkm/subdev/instmem/base.c | 2 +- .../gpu/drm/nouveau/nvkm/subdev/pci/base.c | 2 +- .../gpu/drm/nouveau/nvkm/subdev/pmu/base.c | 2 +- .../gpu/drm/nouveau/nvkm/subdev/therm/base.c | 6 ++--- .../gpu/drm/nouveau/nvkm/subdev/timer/base.c | 2 +- 56 files changed, 139 insertions(+), 84 deletions(-) create mode 100644 drivers/gpu/drm/nouveau/include/nvkm/core/suspend_state.h
drivers/gpu/drm/nouveau/include/nvif/client.h +1 -1
@@ -11,7 +11,7 @@ struct nvif_client {int nvif_client_ctor (struct nvif_client *parent,const char *name,struct nvif_client *);void nvif_client_dtor (struct nvif_client *); -int nvif_client_suspend (struct nvif_client *); +int nvif_client_suspend (struct nvif_client *,bool );int nvif_client_resume (struct nvif_client *);/*XXX*/
drivers/gpu/drm/nouveau/include/nvif/driver.h +1 -1
@@ -8,7 +8,7 @@ struct nvif_driver {const char *name;int (*init)(const char *name,u64 device,const char *cfg,const char *dbg,void **priv); -int (*suspend)(void *priv); +int (*suspend)(void *priv,bool runtime);int (*resume)(void *priv);int (*ioctl)(void *priv,void *data,u32 size,void **hack);void __iomem *(*map)(void *priv,u64 handle,u32 size);
drivers/gpu/drm/nouveau/include/nvkm/core/device.h +2 -1
@@ -2,6 +2,7 @@#ifndef __NVKM_DEVICE_H__#define __NVKM_DEVICE_H__#include <core/oclass.h> +#include <core/suspend_state.h> #include <core/intr.h> enum nvkm_subdev_type ;
@@ -93,7 +94,7 @@ struct nvkm_device_func {void *(*dtor)(struct nvkm_device *);int (*preinit)(struct nvkm_device *);int (*init)(struct nvkm_device *); -void (*fini)(struct nvkm_device *,bool suspend); +void (*fini)(struct nvkm_device *,enum nvkm_suspend_state suspend);int (*irq)(struct nvkm_device *);resource_size_t (*resource_addr)(struct nvkm_device *,enum nvkm_bar_id );resource_size_t (*resource_size)(struct nvkm_device *,enum nvkm_bar_id );
drivers/gpu/drm/nouveau/include/nvkm/core/engine.h +1 -1
@@ -20,7 +20,7 @@ struct nvkm_engine_func {int (*oneinit)(struct nvkm_engine *);int (*info)(struct nvkm_engine *,u64 mthd,u64 *data);int (*init)(struct nvkm_engine *); -int (*fini)(struct nvkm_engine *,bool suspend); +int (*fini)(struct nvkm_engine *,enum nvkm_suspend_state suspend);int (*reset)(struct nvkm_engine *);int (*nonstall)(struct nvkm_engine *);void (*intr)(struct nvkm_engine *);
drivers/gpu/drm/nouveau/include/nvkm/core/object.h +3 -2
@@ -2,6 +2,7 @@#ifndef __NVKM_OBJECT_H__#define __NVKM_OBJECT_H__#include <core/oclass.h> +#include <core/suspend_state.h> struct nvkm_event ;struct nvkm_gpuobj ;struct nvkm_uevent ;
@@ -27,7 +28,7 @@ enum nvkm_object_map {struct nvkm_object_func {void *(*dtor)(struct nvkm_object *);int (*init)(struct nvkm_object *); -int (*fini)(struct nvkm_object *,bool suspend); +int (*fini)(struct nvkm_object *,enum nvkm_suspend_state suspend);int (*mthd)(struct nvkm_object *,u32 mthd,void *data,u32 size);int (*ntfy)(struct nvkm_object *,u32 mthd,struct nvkm_event **); int (*map)(struct nvkm_object *, void *argv, u32 argc,
@@ -49,7 +50,7 @@ int nvkm_object_new(const struct nvkm_oclass *, void *data, u32 size,void nvkm_object_del (struct nvkm_object **);void *nvkm_object_dtor (struct nvkm_object *);int nvkm_object_init (struct nvkm_object *); -int nvkm_object_fini (struct nvkm_object *,bool suspend); +int nvkm_object_fini (struct nvkm_object *,enum nvkm_suspend_state );int nvkm_object_mthd (struct nvkm_object *,u32 mthd,void *data,u32 size);int nvkm_object_ntfy (struct nvkm_object *,u32 mthd,struct nvkm_event **); int nvkm_object_map(struct nvkm_object *, void *argv, u32 argc,
drivers/gpu/drm/nouveau/include/nvkm/core/oproxy.h +1 -1
@@ -13,7 +13,7 @@ struct nvkm_oproxy {struct nvkm_oproxy_func {void (*dtor[2 ])(struct nvkm_oproxy *);int (*init[2 ])(struct nvkm_oproxy *); -int (*fini[2 ])(struct nvkm_oproxy *,bool suspend); +int (*fini[2 ])(struct nvkm_oproxy *,enum nvkm_suspend_state suspend); }; void nvkm_oproxy_ctor(const struct nvkm_oproxy_func *,
drivers/gpu/drm/nouveau/include/nvkm/core/subdev.h +2 -2
@@ -40,7 +40,7 @@ struct nvkm_subdev_func {int (*oneinit)(struct nvkm_subdev *);int (*info)(struct nvkm_subdev *,u64 mthd,u64 *data);int (*init)(struct nvkm_subdev *); -int (*fini)(struct nvkm_subdev *,bool suspend); +int (*fini)(struct nvkm_subdev *,enum nvkm_suspend_state suspend);void (*intr)(struct nvkm_subdev *); };
@@ -65,7 +65,7 @@ void nvkm_subdev_unref(struct nvkm_subdev *);int nvkm_subdev_preinit (struct nvkm_subdev *);int nvkm_subdev_oneinit (struct nvkm_subdev *);int nvkm_subdev_init (struct nvkm_subdev *); -int nvkm_subdev_fini (struct nvkm_subdev *,bool suspend); +int nvkm_subdev_fini (struct nvkm_subdev *,enum nvkm_suspend_state suspend);int nvkm_subdev_info (struct nvkm_subdev *,u64 ,u64 *);void nvkm_subdev_intr (struct nvkm_subdev *);
drivers/gpu/drm/nouveau/include/nvkm/core/suspend_state.h new file +11 -0
@@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: MIT */ +#ifndef __NVKM_SUSPEND_STATE_H__ +#define __NVKM_SUSPEND_STATE_H__ + +enum nvkm_suspend_state { +NVKM_POWEROFF , +NVKM_SUSPEND , +NVKM_RUNTIME_SUSPEND , +}; + +#endif
drivers/gpu/drm/nouveau/nouveau_drm.c +1 -1
@@ -983,7 +983,7 @@ nouveau_do_suspend(struct nouveau_drm *drm, bool runtime) }NV_DEBUG (drm,"suspending object tree...\n" ); - ret =nvif_client_suspend (&drm->_client); + ret =nvif_client_suspend (&drm->_client, runtime);if (ret) goto fail_client;
drivers/gpu/drm/nouveau/nouveau_nvif.c +8 -2
@@ -62,10 +62,16 @@ nvkm_client_resume(void *priv) }static int -nvkm_client_suspend (void *priv) +nvkm_client_suspend (void *priv,bool runtime) {struct nvkm_client *client = priv; -return nvkm_object_fini (&client->object, true); +enum nvkm_suspend_state state; + +if (runtime) + state =NVKM_RUNTIME_SUSPEND ; +else + state =NVKM_SUSPEND ; +return nvkm_object_fini (&client->object, state); } static int
drivers/gpu/drm/nouveau/nvif/client.c +2 -2
@@ -30,9 +30,9 @@#include <nvif/if0000.h> int -nvif_client_suspend (struct nvif_client *client) +nvif_client_suspend (struct nvif_client *client,bool runtime) { -return client->driver->suspend (client->object.priv); +return client->driver->suspend (client->object.priv, runtime); }int
drivers/gpu/drm/nouveau/nvkm/core/engine.c +2 -2
@@ -41,7 +41,7 @@ nvkm_engine_reset(struct nvkm_engine *engine)if (engine->func->reset)return engine->func->reset (engine); -nvkm_subdev_fini (&engine->subdev, false); +nvkm_subdev_fini (&engine->subdev,NVKM_POWEROFF );return nvkm_subdev_init (&engine->subdev); }
@@ -98,7 +98,7 @@ nvkm_engine_info(struct nvkm_subdev *subdev, u64 mthd, u64 *data) }static int -nvkm_engine_fini (struct nvkm_subdev *subdev,bool suspend) +nvkm_engine_fini (struct nvkm_subdev *subdev,enum nvkm_suspend_state suspend) {struct nvkm_engine *engine =nvkm_engine (subdev);if (engine->func->fini)
drivers/gpu/drm/nouveau/nvkm/core/ioctl.c +2 -2
@@ -141,7 +141,7 @@ nvkm_ioctl_new(struct nvkm_client *client, } ret = -EEXIST ; } -nvkm_object_fini (object, false); +nvkm_object_fini (object,NVKM_POWEROFF ); }nvkm_object_del (&object);
@@ -160,7 +160,7 @@ nvkm_ioctl_del(struct nvkm_client *client,nvif_ioctl (object,"delete size %d\n" , size);if (!(ret =nvif_unvers (ret, &data, &size, args->none))) {nvif_ioctl (object,"delete\n" ); -nvkm_object_fini (object, false); +nvkm_object_fini (object,NVKM_POWEROFF );nvkm_object_del (&object); }
drivers/gpu/drm/nouveau/nvkm/core/object.c +16 -4
@@ -142,13 +142,25 @@ nvkm_object_bind(struct nvkm_object *object, struct nvkm_gpuobj *gpuobj, }int -nvkm_object_fini (struct nvkm_object *object,bool suspend) +nvkm_object_fini (struct nvkm_object *object,enum nvkm_suspend_state suspend) { -const char *action = suspend ?"suspend" :"fini" ; +const char *action;struct nvkm_object *child;s64 time;int ret; +switch (suspend) { +case NVKM_POWEROFF : +default : + action ="fini" ; +break ; +case NVKM_SUSPEND : + action ="suspend" ; +break ; +case NVKM_RUNTIME_SUSPEND : + action ="runtime" ; +break ; + }nvif_debug (object,"%s children...\n" , action); time =ktime_to_us (ktime_get ());list_for_each_entry_reverse (child, &object->tree, head) {
@@ -212,11 +224,11 @@ nvkm_object_init(struct nvkm_object *object) fail_child:list_for_each_entry_continue_reverse (child, &object->tree, head) -nvkm_object_fini (child, false); +nvkm_object_fini (child,NVKM_POWEROFF ); fail:nvif_error (object,"init failed with %d\n" , ret);if (object->func->fini) - object->func->fini (object, false); + object->func->fini (object,NVKM_POWEROFF );return ret; }
drivers/gpu/drm/nouveau/nvkm/core/oproxy.c +1 -1
@@ -87,7 +87,7 @@ nvkm_oproxy_uevent(struct nvkm_object *object, void *argv, u32 argc, }static int -nvkm_oproxy_fini (struct nvkm_object *object,bool suspend) +nvkm_oproxy_fini (struct nvkm_object *object,enum nvkm_suspend_state suspend) {struct nvkm_oproxy *oproxy =nvkm_oproxy (object);int ret;
drivers/gpu/drm/nouveau/nvkm/core/subdev.c +15 -3
@@ -51,12 +51,24 @@ nvkm_subdev_info(struct nvkm_subdev *subdev, u64 mthd, u64 *data) }int -nvkm_subdev_fini (struct nvkm_subdev *subdev,bool suspend) +nvkm_subdev_fini (struct nvkm_subdev *subdev,enum nvkm_suspend_state suspend) {struct nvkm_device *device = subdev->device; -const char *action = suspend ?"suspend" : subdev->use.enabled ?"fini" :"reset" ; +const char *action;s64 time; +switch (suspend) { +case NVKM_POWEROFF : +default : + action = subdev->use.enabled ?"fini" :"reset" ; +break ; +case NVKM_SUSPEND : + action ="suspend" ; +break ; +case NVKM_RUNTIME_SUSPEND : + action ="runtime" ; +break ; + }nvkm_trace (subdev,"%s running...\n" , action); time =ktime_to_us (ktime_get ());
@@ -186,7 +198,7 @@ void nvkm_subdev_unref(struct nvkm_subdev *subdev) {if (refcount_dec_and_mutex_lock (&subdev->use.refcount, &subdev->use.mutex)) { -nvkm_subdev_fini (subdev, false); +nvkm_subdev_fini (subdev,NVKM_POWEROFF );mutex_unlock (&subdev->use.mutex); } }
drivers/gpu/drm/nouveau/nvkm/core/uevent.c +1 -1
@@ -73,7 +73,7 @@ nvkm_uevent_mthd(struct nvkm_object *object, u32 mthd, void *argv, u32 argc) }static int -nvkm_uevent_fini (struct nvkm_object *object,bool suspend) +nvkm_uevent_fini (struct nvkm_object *object,enum nvkm_suspend_state suspend) {struct nvkm_uevent *uevent =nvkm_uevent (object);
drivers/gpu/drm/nouveau/nvkm/engine/ce/ga100.c +1 -1
@@ -46,7 +46,7 @@ ga100_ce_nonstall(struct nvkm_engine *engine) }int -ga100_ce_fini (struct nvkm_engine *engine,bool suspend) +ga100_ce_fini (struct nvkm_engine *engine,enum nvkm_suspend_state suspend) {nvkm_inth_block (&engine->subdev.inth);return 0 ;
drivers/gpu/drm/nouveau/nvkm/engine/ce/priv.h +1 -1
@@ -14,7 +14,7 @@ extern const struct nvkm_object_func gv100_ce_cclass;int ga100_ce_oneinit (struct nvkm_engine *);int ga100_ce_init (struct nvkm_engine *); -int ga100_ce_fini (struct nvkm_engine *,bool ); +int ga100_ce_fini (struct nvkm_engine *,enum nvkm_suspend_state );int ga100_ce_nonstall (struct nvkm_engine *);u32 gb202_ce_grce_mask (struct nvkm_device *);
drivers/gpu/drm/nouveau/nvkm/engine/device/base.c +17 -5
@@ -2936,13 +2936,25 @@ nvkm_device_engine(struct nvkm_device *device, int type, int inst) }int -nvkm_device_fini (struct nvkm_device *device,bool suspend) +nvkm_device_fini (struct nvkm_device *device,enum nvkm_suspend_state suspend) { -const char *action = suspend ?"suspend" :"fini" ; +const char *action;struct nvkm_subdev *subdev;int ret;s64 time; +switch (suspend) { +case NVKM_POWEROFF : +default : + action ="fini" ; +break ; +case NVKM_SUSPEND : + action ="suspend" ; +break ; +case NVKM_RUNTIME_SUSPEND : + action ="runtime" ; +break ; + }nvdev_trace (device,"%s running...\n" , action); time =ktime_to_us (ktime_get ());
@@ -3032,7 +3044,7 @@ nvkm_device_init(struct nvkm_device *device)if (ret)return ret; -nvkm_device_fini (device, false); +nvkm_device_fini (device,NVKM_POWEROFF );nvdev_trace (device,"init running...\n" ); time =ktime_to_us (ktime_get ());
@@ -3060,9 +3072,9 @@ nvkm_device_init(struct nvkm_device *device) fail_subdev:list_for_each_entry_from (subdev, &device->subdev, head) -nvkm_subdev_fini (subdev, false); +nvkm_subdev_fini (subdev,NVKM_POWEROFF ); fail: -nvkm_device_fini (device, false); +nvkm_device_fini (device,NVKM_POWEROFF );nvdev_error (device,"init failed with %d\n" , ret);return ret;
drivers/gpu/drm/nouveau/nvkm/engine/device/pci.c +2 -2
@@ -1605,10 +1605,10 @@ nvkm_device_pci_irq(struct nvkm_device *device) }static void -nvkm_device_pci_fini (struct nvkm_device *device,bool suspend) +nvkm_device_pci_fini (struct nvkm_device *device,enum nvkm_suspend_state suspend) {struct nvkm_device_pci *pdev =nvkm_device_pci (device); -if (suspend) { +if (suspend !=NVKM_POWEROFF ) {pci_disable_device (pdev->pdev); pdev->suspend = true; }
drivers/gpu/drm/nouveau/nvkm/engine/device/priv.h +1 -1
@@ -56,5 +56,5 @@ int nvkm_device_ctor(const struct nvkm_device_func *, const char *name, const char *cfg, const char *dbg, struct nvkm_device *);int nvkm_device_init (struct nvkm_device *); -int nvkm_device_fini (struct nvkm_device *,bool suspend); +int nvkm_device_fini (struct nvkm_device *,enum nvkm_suspend_state suspend);#endif
drivers/gpu/drm/nouveau/nvkm/engine/device/user.c +1 -1
@@ -218,7 +218,7 @@ nvkm_udevice_map(struct nvkm_object *object, void *argv, u32 argc, }static int -nvkm_udevice_fini (struct nvkm_object *object,bool suspend) +nvkm_udevice_fini (struct nvkm_object *object,enum nvkm_suspend_state suspend) {struct nvkm_udevice *udev =nvkm_udevice (object);struct nvkm_device *device = udev->device;
drivers/gpu/drm/nouveau/nvkm/engine/disp/base.c +2 -2
@@ -99,13 +99,13 @@ nvkm_disp_intr(struct nvkm_engine *engine) }static int -nvkm_disp_fini (struct nvkm_engine *engine,bool suspend) +nvkm_disp_fini (struct nvkm_engine *engine,enum nvkm_suspend_state suspend) {struct nvkm_disp *disp =nvkm_disp (engine);struct nvkm_outp *outp;if (disp->func->fini) - disp->func->fini (disp, suspend); + disp->func->fini (disp, suspend !=NVKM_POWEROFF );list_for_each_entry (outp, &disp->outps, head) {if (outp->func->fini)
drivers/gpu/drm/nouveau/nvkm/engine/disp/chan.c +1 -1
@@ -128,7 +128,7 @@ nvkm_disp_chan_child_get(struct nvkm_object *object, int index, struct nvkm_ocla }static int -nvkm_disp_chan_fini (struct nvkm_object *object,bool suspend) +nvkm_disp_chan_fini (struct nvkm_object *object,enum nvkm_suspend_state suspend) {struct nvkm_disp_chan *chan =nvkm_disp_chan (object);
drivers/gpu/drm/nouveau/nvkm/engine/falcon.c +2 -2
@@ -93,13 +93,13 @@ nvkm_falcon_intr(struct nvkm_engine *engine) }static int -nvkm_falcon_fini (struct nvkm_engine *engine,bool suspend) +nvkm_falcon_fini (struct nvkm_engine *engine,enum nvkm_suspend_state suspend) {struct nvkm_falcon *falcon =nvkm_falcon (engine);struct nvkm_device *device = falcon->engine.subdev.device;const u32 base = falcon->addr; -if (!suspend) { +if (suspend ==NVKM_POWEROFF ) {nvkm_memory_unref (&falcon->core);if (falcon->external) {vfree (falcon->data.data);
drivers/gpu/drm/nouveau/nvkm/engine/fifo/base.c +1 -1
@@ -122,7 +122,7 @@ nvkm_fifo_class_get(struct nvkm_oclass *oclass, int index, const struct nvkm_dev }static int -nvkm_fifo_fini (struct nvkm_engine *engine,bool suspend) +nvkm_fifo_fini (struct nvkm_engine *engine,enum nvkm_suspend_state suspend) {struct nvkm_fifo *fifo =nvkm_fifo (engine);struct nvkm_runl *runl;
drivers/gpu/drm/nouveau/nvkm/engine/fifo/uchan.c +3 -3
@@ -72,7 +72,7 @@ struct nvkm_uobj { };static int -nvkm_uchan_object_fini_1 (struct nvkm_oproxy *oproxy,bool suspend) +nvkm_uchan_object_fini_1 (struct nvkm_oproxy *oproxy,enum nvkm_suspend_state suspend) {struct nvkm_uobj *uobj =container_of (oproxy,typeof (*uobj), oproxy);struct nvkm_chan *chan = uobj->chan;
@@ -87,7 +87,7 @@ nvkm_uchan_object_fini_1(struct nvkm_oproxy *oproxy, bool suspend)nvkm_chan_cctx_bind (chan, ectx->engn,NULL );if (refcount_dec_and_test (&ectx->uses)) -nvkm_object_fini (ectx->object, false); +nvkm_object_fini (ectx->object,NVKM_POWEROFF );mutex_unlock (&chan->cgrp->mutex); }
@@ -269,7 +269,7 @@ nvkm_uchan_map(struct nvkm_object *object, void *argv, u32 argc, }static int -nvkm_uchan_fini (struct nvkm_object *object,bool suspend) +nvkm_uchan_fini (struct nvkm_object *object,enum nvkm_suspend_state suspend) {struct nvkm_chan *chan =nvkm_uchan (object)->chan;
drivers/gpu/drm/nouveau/nvkm/engine/gr/base.c +2 -2
@@ -168,11 +168,11 @@ nvkm_gr_init(struct nvkm_engine *engine) }static int -nvkm_gr_fini (struct nvkm_engine *engine,bool suspend) +nvkm_gr_fini (struct nvkm_engine *engine,enum nvkm_suspend_state suspend) {struct nvkm_gr *gr =nvkm_gr (engine);if (gr->func->fini) -return gr->func->fini (gr, suspend); +return gr->func->fini (gr, suspend !=NVKM_POWEROFF );return 0 ; }
drivers/gpu/drm/nouveau/nvkm/engine/gr/gf100.c +1 -1
@@ -2330,7 +2330,7 @@ gf100_gr_reset(struct nvkm_gr *base)WARN_ON (gf100_gr_fecs_halt_pipeline (gr)); - subdev->func->fini (subdev, false); + subdev->func->fini (subdev,NVKM_POWEROFF );nvkm_mc_disable (device, subdev->type, subdev->inst);if (gr->func->gpccs.reset) gr->func->gpccs.reset (gr);
drivers/gpu/drm/nouveau/nvkm/engine/gr/nv04.c +1 -1
@@ -1158,7 +1158,7 @@ nv04_gr_chan_dtor(struct nvkm_object *object) }static int -nv04_gr_chan_fini (struct nvkm_object *object,bool suspend) +nv04_gr_chan_fini (struct nvkm_object *object,enum nvkm_suspend_state suspend) {struct nv04_gr_chan *chan =nv04_gr_chan (object);struct nv04_gr *gr = chan->gr;
drivers/gpu/drm/nouveau/nvkm/engine/gr/nv10.c +1 -1
@@ -951,7 +951,7 @@ nv10_gr_context_switch(struct nv10_gr *gr) }static int -nv10_gr_chan_fini (struct nvkm_object *object,bool suspend) +nv10_gr_chan_fini (struct nvkm_object *object,enum nvkm_suspend_state suspend) {struct nv10_gr_chan *chan =nv10_gr_chan (object);struct nv10_gr *gr = chan->gr;
drivers/gpu/drm/nouveau/nvkm/engine/gr/nv20.c +1 -1
@@ -27,7 +27,7 @@ nv20_gr_chan_init(struct nvkm_object *object) }int -nv20_gr_chan_fini (struct nvkm_object *object,bool suspend) +nv20_gr_chan_fini (struct nvkm_object *object,enum nvkm_suspend_state suspend) {struct nv20_gr_chan *chan =nv20_gr_chan (object);struct nv20_gr *gr = chan->gr;
drivers/gpu/drm/nouveau/nvkm/engine/gr/nv20.h +1 -1
@@ -31,5 +31,5 @@ struct nv20_gr_chan {void *nv20_gr_chan_dtor (struct nvkm_object *);int nv20_gr_chan_init (struct nvkm_object *); -int nv20_gr_chan_fini (struct nvkm_object *,bool ); +int nv20_gr_chan_fini (struct nvkm_object *,enum nvkm_suspend_state );#endif
drivers/gpu/drm/nouveau/nvkm/engine/gr/nv40.c +2 -2
@@ -89,7 +89,7 @@ nv40_gr_chan_bind(struct nvkm_object *object, struct nvkm_gpuobj *parent, }static int -nv40_gr_chan_fini (struct nvkm_object *object,bool suspend) +nv40_gr_chan_fini (struct nvkm_object *object,enum nvkm_suspend_state suspend) {struct nv40_gr_chan *chan =nv40_gr_chan (object);struct nv40_gr *gr = chan->gr;
@@ -101,7 +101,7 @@ nv40_gr_chan_fini(struct nvkm_object *object, bool suspend)nvkm_mask (device,0x400720 ,0x00000001 ,0x00000000 );if (nvkm_rd32 (device,0x40032c ) == inst) { -if (suspend) { +if (suspend !=NVKM_POWEROFF ) {nvkm_wr32 (device,0x400720 ,0x00000000 );nvkm_wr32 (device,0x400784 , inst);nvkm_mask (device,0x400310 ,0x00000020 ,0x00000020 );
drivers/gpu/drm/nouveau/nvkm/engine/mpeg/nv44.c +1 -1
@@ -65,7 +65,7 @@ nv44_mpeg_chan_bind(struct nvkm_object *object, struct nvkm_gpuobj *parent, }static int -nv44_mpeg_chan_fini (struct nvkm_object *object,bool suspend) +nv44_mpeg_chan_fini (struct nvkm_object *object,enum nvkm_suspend_state suspend) {struct nv44_mpeg_chan *chan =nv44_mpeg_chan (object);
drivers/gpu/drm/nouveau/nvkm/engine/sec2/base.c +1 -1
@@ -37,7 +37,7 @@ nvkm_sec2_finimsg(void *priv, struct nvfw_falcon_msg *hdr) }static int -nvkm_sec2_fini (struct nvkm_engine *engine,bool suspend) +nvkm_sec2_fini (struct nvkm_engine *engine,enum nvkm_suspend_state suspend) {struct nvkm_sec2 *sec2 =nvkm_sec2 (engine);struct nvkm_subdev *subdev = &sec2->engine.subdev;
drivers/gpu/drm/nouveau/nvkm/engine/xtensa.c +2 -2
@@ -76,7 +76,7 @@ nvkm_xtensa_intr(struct nvkm_engine *engine) }static int -nvkm_xtensa_fini (struct nvkm_engine *engine,bool suspend) +nvkm_xtensa_fini (struct nvkm_engine *engine,enum nvkm_suspend_state suspend) {struct nvkm_xtensa *xtensa =nvkm_xtensa (engine);struct nvkm_device *device = xtensa->engine.subdev.device;
@@ -85,7 +85,7 @@ nvkm_xtensa_fini(struct nvkm_engine *engine, bool suspend)nvkm_wr32 (device, base +0xd84 ,0 );/* INTR_EN */ nvkm_wr32 (device, base +0xd94 ,0 );/* FIFO_CTRL */ -if (!suspend) +if (suspend ==NVKM_POWEROFF )nvkm_memory_unref (&xtensa->gpu_fw);return 0 ; }
drivers/gpu/drm/nouveau/nvkm/subdev/acr/base.c +1 -1
@@ -182,7 +182,7 @@ nvkm_acr_managed_falcon(struct nvkm_device *device, enum nvkm_acr_lsf_id id) }static int -nvkm_acr_fini (struct nvkm_subdev *subdev,bool suspend) +nvkm_acr_fini (struct nvkm_subdev *subdev,enum nvkm_suspend_state suspend) {if (!subdev->use.enabled)return 0 ;
drivers/gpu/drm/nouveau/nvkm/subdev/bar/base.c +1 -1
@@ -90,7 +90,7 @@ nvkm_bar_bar2_init(struct nvkm_device *device) }static int -nvkm_bar_fini (struct nvkm_subdev *subdev,bool suspend) +nvkm_bar_fini (struct nvkm_subdev *subdev,enum nvkm_suspend_state suspend) {struct nvkm_bar *bar =nvkm_bar (subdev);
drivers/gpu/drm/nouveau/nvkm/subdev/clk/base.c +1 -1
@@ -577,7 +577,7 @@ nvkm_clk_read(struct nvkm_clk *clk, enum nv_clk_src src) }static int -nvkm_clk_fini (struct nvkm_subdev *subdev,bool suspend) +nvkm_clk_fini (struct nvkm_subdev *subdev,enum nvkm_suspend_state suspend) {struct nvkm_clk *clk =nvkm_clk (subdev);flush_work (&clk->work);
drivers/gpu/drm/nouveau/nvkm/subdev/devinit/base.c +2 -2
@@ -67,11 +67,11 @@ nvkm_devinit_post(struct nvkm_devinit *init) }static int -nvkm_devinit_fini (struct nvkm_subdev *subdev,bool suspend) +nvkm_devinit_fini (struct nvkm_subdev *subdev,enum nvkm_suspend_state suspend) {struct nvkm_devinit *init =nvkm_devinit (subdev);/* force full reinit on resume */ -if (suspend) +if (suspend !=NVKM_POWEROFF ) init->post = true;return 0 ; }
drivers/gpu/drm/nouveau/nvkm/subdev/fault/base.c +1 -1
@@ -51,7 +51,7 @@ nvkm_fault_intr(struct nvkm_subdev *subdev) }static int -nvkm_fault_fini (struct nvkm_subdev *subdev,bool suspend) +nvkm_fault_fini (struct nvkm_subdev *subdev,enum nvkm_suspend_state suspend) {struct nvkm_fault *fault =nvkm_fault (subdev);if (fault->func->fini)
drivers/gpu/drm/nouveau/nvkm/subdev/fault/user.c +1 -1
@@ -56,7 +56,7 @@ nvkm_ufault_map(struct nvkm_object *object, void *argv, u32 argc, }static int -nvkm_ufault_fini (struct nvkm_object *object,bool suspend) +nvkm_ufault_fini (struct nvkm_object *object,enum nvkm_suspend_state suspend) {struct nvkm_fault_buffer *buffer =nvkm_fault_buffer (object); buffer->fault->func->buffer.fini (buffer);
drivers/gpu/drm/nouveau/nvkm/subdev/gpio/base.c +1 -1
@@ -144,7 +144,7 @@ nvkm_gpio_intr(struct nvkm_subdev *subdev) }static int -nvkm_gpio_fini (struct nvkm_subdev *subdev,bool suspend) +nvkm_gpio_fini (struct nvkm_subdev *subdev,enum nvkm_suspend_state suspend) {struct nvkm_gpio *gpio =nvkm_gpio (subdev);u32 mask = (1ULL << gpio->func->lines) -1 ;
drivers/gpu/drm/nouveau/nvkm/subdev/gsp/base.c +1 -1
@@ -48,7 +48,7 @@ nvkm_gsp_intr_stall(struct nvkm_gsp *gsp, enum nvkm_subdev_type type, int inst) }static int -nvkm_gsp_fini (struct nvkm_subdev *subdev,bool suspend) +nvkm_gsp_fini (struct nvkm_subdev *subdev,enum nvkm_suspend_state suspend) {struct nvkm_gsp *gsp =nvkm_gsp (subdev);
drivers/gpu/drm/nouveau/nvkm/subdev/gsp/gh100.c +1 -1
@@ -17,7 +17,7 @@#include <nvhw/ref/gh100/dev_riscv_pri.h> int -gh100_gsp_fini (struct nvkm_gsp *gsp,bool suspend) +gh100_gsp_fini (struct nvkm_gsp *gsp,enum nvkm_suspend_state suspend) {struct nvkm_falcon *falcon = &gsp->falcon;int ret, time =4000 ;
drivers/gpu/drm/nouveau/nvkm/subdev/gsp/priv.h +4 -4
@@ -59,7 +59,7 @@ struct nvkm_gsp_func {void (*dtor)(struct nvkm_gsp *);int (*oneinit)(struct nvkm_gsp *);int (*init)(struct nvkm_gsp *); -int (*fini)(struct nvkm_gsp *,bool suspend); +int (*fini)(struct nvkm_gsp *,enum nvkm_suspend_state suspend);int (*reset)(struct nvkm_gsp *);struct {
@@ -75,7 +75,7 @@ int tu102_gsp_fwsec_sb_ctor(struct nvkm_gsp *);void tu102_gsp_fwsec_sb_dtor (struct nvkm_gsp *);int tu102_gsp_oneinit (struct nvkm_gsp *);int tu102_gsp_init (struct nvkm_gsp *); -int tu102_gsp_fini (struct nvkm_gsp *,bool suspend); +int tu102_gsp_fini (struct nvkm_gsp *,enum nvkm_suspend_state suspend);int tu102_gsp_reset (struct nvkm_gsp *);u64 tu102_gsp_wpr_heap_size (struct nvkm_gsp *);
@@ -87,12 +87,12 @@ int ga102_gsp_reset(struct nvkm_gsp *);int gh100_gsp_oneinit (struct nvkm_gsp *);int gh100_gsp_init (struct nvkm_gsp *); -int gh100_gsp_fini (struct nvkm_gsp *,bool suspend); +int gh100_gsp_fini (struct nvkm_gsp *,enum nvkm_suspend_state suspend);void r535_gsp_dtor (struct nvkm_gsp *);int r535_gsp_oneinit (struct nvkm_gsp *);int r535_gsp_init (struct nvkm_gsp *); -int r535_gsp_fini (struct nvkm_gsp *,bool suspend); +int r535_gsp_fini (struct nvkm_gsp *,enum nvkm_suspend_state suspend);int nvkm_gsp_new_ (const struct nvkm_gsp_fwif *,struct nvkm_device *,enum nvkm_subdev_type ,int ,struct nvkm_gsp **);
drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r535/gsp.c +1 -1
@@ -1721,7 +1721,7 @@ r535_gsp_sr_data_size(struct nvkm_gsp *gsp) }int -r535_gsp_fini (struct nvkm_gsp *gsp,bool suspend) +r535_gsp_fini (struct nvkm_gsp *gsp,enum nvkm_suspend_state suspend) {struct nvkm_rm *rm = gsp->rm;int ret;
drivers/gpu/drm/nouveau/nvkm/subdev/gsp/tu102.c +1 -1
@@ -161,7 +161,7 @@ tu102_gsp_reset(struct nvkm_gsp *gsp) }int -tu102_gsp_fini (struct nvkm_gsp *gsp,bool suspend) +tu102_gsp_fini (struct nvkm_gsp *gsp,enum nvkm_suspend_state suspend) {u32 mbox0 =0xff , mbox1 =0xff ;int ret;
drivers/gpu/drm/nouveau/nvkm/subdev/i2c/base.c +1 -1
@@ -135,7 +135,7 @@ nvkm_i2c_intr(struct nvkm_subdev *subdev) }static int -nvkm_i2c_fini (struct nvkm_subdev *subdev,bool suspend) +nvkm_i2c_fini (struct nvkm_subdev *subdev,enum nvkm_suspend_state suspend) {struct nvkm_i2c *i2c =nvkm_i2c (subdev);struct nvkm_i2c_pad *pad;
drivers/gpu/drm/nouveau/nvkm/subdev/instmem/base.c +1 -1
@@ -176,7 +176,7 @@ nvkm_instmem_boot(struct nvkm_instmem *imem) }static int -nvkm_instmem_fini (struct nvkm_subdev *subdev,bool suspend) +nvkm_instmem_fini (struct nvkm_subdev *subdev,enum nvkm_suspend_state suspend) {struct nvkm_instmem *imem =nvkm_instmem (subdev);int ret;
drivers/gpu/drm/nouveau/nvkm/subdev/pci/base.c +1 -1
@@ -74,7 +74,7 @@ nvkm_pci_rom_shadow(struct nvkm_pci *pci, bool shadow) }static int -nvkm_pci_fini (struct nvkm_subdev *subdev,bool suspend) +nvkm_pci_fini (struct nvkm_subdev *subdev,enum nvkm_suspend_state suspend) {struct nvkm_pci *pci =nvkm_pci (subdev);
drivers/gpu/drm/nouveau/nvkm/subdev/pmu/base.c +1 -1
@@ -77,7 +77,7 @@ nvkm_pmu_intr(struct nvkm_subdev *subdev) }static int -nvkm_pmu_fini (struct nvkm_subdev *subdev,bool suspend) +nvkm_pmu_fini (struct nvkm_subdev *subdev,enum nvkm_suspend_state suspend) {struct nvkm_pmu *pmu =nvkm_pmu (subdev);
drivers/gpu/drm/nouveau/nvkm/subdev/therm/base.c +3 -3
@@ -341,15 +341,15 @@ nvkm_therm_intr(struct nvkm_subdev *subdev) }static int -nvkm_therm_fini (struct nvkm_subdev *subdev,bool suspend) +nvkm_therm_fini (struct nvkm_subdev *subdev,enum nvkm_suspend_state suspend) {struct nvkm_therm *therm =nvkm_therm (subdev);if (therm->func->fini) therm->func->fini (therm); -nvkm_therm_fan_fini (therm, suspend); -nvkm_therm_sensor_fini (therm, suspend); +nvkm_therm_fan_fini (therm, suspend !=NVKM_POWEROFF ); +nvkm_therm_sensor_fini (therm, suspend !=NVKM_POWEROFF );if (suspend) { therm->suspend = therm->mode;
drivers/gpu/drm/nouveau/nvkm/subdev/timer/base.c +1 -1
@@ -149,7 +149,7 @@ nvkm_timer_intr(struct nvkm_subdev *subdev) }static int -nvkm_timer_fini (struct nvkm_subdev *subdev,bool suspend) +nvkm_timer_fini (struct nvkm_subdev *subdev,enum nvkm_suspend_state suspend) {struct nvkm_timer *tmr =nvkm_timer (subdev); tmr->func->alarm_fini (tmr);
Signature · 1 line
2.52.0
Dave Airlie
maintainerPATCH 3/3 nouveau/gsp: fix suspend/resume regression on r570 firmware
In reply to Dave AirlieFrom: Dave Airlie <[email protected]>
The r570 firmware with certain GPUs (at least RTX6000) needs this flag to reflect the suspend vs runtime PM state of the driver.
This uses that info to set the correct flags to the firmware.
This fixes a regression on RTX6000 and other GPUs since r570 firmware was enabled.
--- drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r535/fbsr.c | 2 +- drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r535/gsp.c | 2 +- drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/fbsr.c | 8 ++++---- drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/rm.h | 2 +- 4 files changed, 7 insertions(+), 7 deletions(-)
drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r535/fbsr.c +1 -1
@@ -208,7 +208,7 @@ r535_fbsr_resume(struct nvkm_gsp *gsp) }static int -r535_fbsr_suspend (struct nvkm_gsp *gsp) +r535_fbsr_suspend (struct nvkm_gsp *gsp,bool runtime) {struct nvkm_subdev *subdev = &gsp->subdev;struct nvkm_device *device = subdev->device;
drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r535/gsp.c +1 -1
@@ -1748,7 +1748,7 @@ r535_gsp_fini(struct nvkm_gsp *gsp, enum nvkm_suspend_state suspend) sr->sysmemAddrOfSuspendResumeData = gsp->sr.radix3.lvl0.addr; sr->sizeOfSuspendResumeData = len; - ret = rm->api->fbsr->suspend (gsp); + ret = rm->api->fbsr->suspend (gsp, suspend ==NVKM_RUNTIME_SUSPEND );if (ret) {nvkm_gsp_mem_dtor (&gsp->sr.meta);nvkm_gsp_radix3_dtor (gsp, &gsp->sr.radix3);
drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/fbsr.c +4 -4
@@ -62,7 +62,7 @@ r570_fbsr_resume(struct nvkm_gsp *gsp) }static int -r570_fbsr_init (struct nvkm_gsp *gsp,struct sg_table *sgt,u64 size) +r570_fbsr_init (struct nvkm_gsp *gsp,struct sg_table *sgt,u64 size,bool runtime) {NV2080_CTRL_INTERNAL_FBSR_INIT_PARAMS *ctrl;struct nvkm_gsp_object memlist;
@@ -81,7 +81,7 @@ r570_fbsr_init(struct nvkm_gsp *gsp, struct sg_table *sgt, u64 size) ctrl->hClient = gsp->internal.client.object.handle; ctrl->hSysMem = memlist.handle; ctrl->sysmemAddrOfSuspendResumeData = gsp->sr.meta.addr; - ctrl->bEnteringGcoffState =1 ; + ctrl->bEnteringGcoffState = runtime ?1 :0 ; ret =nvkm_gsp_rm_ctrl_wr (&gsp->internal.device.subdevice, ctrl);if (ret)
@@ -92,7 +92,7 @@ r570_fbsr_init(struct nvkm_gsp *gsp, struct sg_table *sgt, u64 size) }static int -r570_fbsr_suspend (struct nvkm_gsp *gsp) +r570_fbsr_suspend (struct nvkm_gsp *gsp,bool runtime) {struct nvkm_subdev *subdev = &gsp->subdev;struct nvkm_device *device = subdev->device;
@@ -133,7 +133,7 @@ r570_fbsr_suspend(struct nvkm_gsp *gsp)return ret;/* Initialise FBSR on RM. */ - ret =r570_fbsr_init (gsp, &gsp->sr.fbsr, size); + ret =r570_fbsr_init (gsp, &gsp->sr.fbsr, size, runtime);if (ret) {nvkm_gsp_sg_free (device, &gsp->sr.fbsr);return ret;
drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/rm.h +1 -1
@@ -78,7 +78,7 @@ struct nvkm_rm_api { } *device;const struct nvkm_rm_api_fbsr { -int (*suspend)(struct nvkm_gsp *); +int (*suspend)(struct nvkm_gsp *,bool runtime);void (*resume)(struct nvkm_gsp *); } *fbsr;
Signature · 1 line
2.52.0
For the whole series:
My power bill thanks you :)
Quoted 16 lines · Dave Airlie
Since R570 firmware got enabled, a number of GPUs have not being successfully suspend/resuming. Lyude showed it on RTX6000, which I reproduced and wasted a lot of time down various rabbit holes.
There are two required fixes, the first one adds proper sequence numbers to the rpc messages, which fixes a bunch of NOCAT asserts.
Then we have to pass the runtime vs non-runtime state down to the GSP fbsr code. This however requires replacing a bool with an enum which refactors quite a bunch of interfaces unfortunately, but it was the cleanest way to do it.
The final patch hooks up the interface so normal suspend doesn't set the GCOFF flags.
Dave.
Dave Airlie
maintainerThis is now pushed to drm-misc-fixes.
Dave.
Quoted 26 lines · On Wed, 4 Feb 2026 at 06:08, <[email protected]> wrote:
For the whole series:
Reviewed-by Lyude Paul <[email protected]>Tested-by Lyude Paul <[email protected]>My power bill thanks you :)
Quoted 16 lines · Dave Airlie
Since R570 firmware got enabled, a number of GPUs have not being successfully suspend/resuming. Lyude showed it on RTX6000, which I reproduced and wasted a lot of time down various rabbit holes.
There are two required fixes, the first one adds proper sequence numbers to the rpc messages, which fixes a bunch of NOCAT asserts.
Then we have to pass the runtime vs non-runtime state down to the GSP fbsr code. This however requires replacing a bool with an enum which refactors quite a bunch of interfaces unfortunately, but it was the cleanest way to do it.
The final patch hooks up the interface so normal suspend doesn't set the GCOFF flags.
Dave.