gpu: nova-core: gsp: add locking to Cmdq
Subsystems (4)
v5 cover gpu: nova-core: gsp: add locking to Cmdq
Add locking to Cmdq. This is required e.g. for unloading the driver, which needs to send the UnloadingGuestDriver via the command queue on unbind which may be on a different thread.
We have commands that need a reply and commands that don't. For commands with a reply we want to make sure that they don't get the reply of a different command back. The approach this patch series takes is by making those commands block until they get a response. For now this should be ok, and we expect GSP to be fast anyway.
To do this, we need to know which commands expect a reply and which don't. John's existing series[1] adds IS_ASYNC which solves part of the problem, but we need to know a bit more. So instead, add an associated type called Reply which tells us what the reply is.
An alternative would be to define traits inheriting CommandToGsp, e.g. CommandWithReply and CommandWithoutReply, instead of using the associated type. I implemented the associated type version because it feels more compositional rather than inherity so seemed a bit better to me. But both of these approaches work and are fine, IMO.
In summary, this patch series has three steps:
1. Add the type infrastructure to know what replies are expected for a command and update each caller to explicitly wait for the reply or not.
2. Make Cmdq pinned so we can use Mutex
3. Add a Mutex to protect Cmdq by moving the relevant state to an inner struct.
[1]: https://lore.kernel.org/all/[email protected]/
---
Changes in v5:
- tweak documentation
- rebase on drm-rust-next
- Link to v4: https://lore.kernel.org/r/[email protected]
Changes in v4:
- Change RECEIVE_TIMEOUT from 10s to 5s
- Update NoReply doc comment
- Move CommandToGsp doc comment update
- Fix import formatting in continuation.rs
- Move CmdqInner after Cmdq to reduce diff size
- Link to v3: https://lore.kernel.org/r/[email protected]
Changes in v3:
- Rename send_sync_command/send_async_command to send_command/send_command_no_wait.
- Move dev field into CmdqInner to avoid passing it through method parameters.
- Add RECEIVE_TIMEOUT constant for the 10s receive timeout.
- Link to v2: https://lore.kernel.org/r/[email protected]
Changes in v2:
- Rebase on drm-rust-next
- Link to v1: https://lore.kernel.org/r/[email protected]
---
Eliot Courtney (5):
gpu: nova-core: gsp: fix stale doc comments on command queue methods
gpu: nova-core: gsp: add RECEIVE_TIMEOUT constant for command queue
gpu: nova-core: gsp: add reply/no-reply info to CommandToGsp
gpu: nova-core: gsp: make Cmdq a pinned type
gpu: nova-core: gsp: add mutex locking to Cmdq
drivers/gpu/nova-core/gsp.rs | 5 +- drivers/gpu/nova-core/gsp/boot.rs | 13 +- drivers/gpu/nova-core/gsp/cmdq.rs | 159 +++++++++++++++++++------ drivers/gpu/nova-core/gsp/cmdq/continuation.rs | 8 +- drivers/gpu/nova-core/gsp/commands.rs | 23 ++-- drivers/gpu/nova-core/gsp/sequencer.rs | 4 +- 6 files changed, 151 insertions(+), 61 deletions(-)
---
base-commit: d19ab42867ae7c68be84ed957d95712b7934773f
change-id: 20260225-cmdq-locking-d32928a2c2cf
Best regards,
Signature · 1 line
Eliot Courtney <[email protected]>
PATCH v5 1/5 gpu: nova-core: gsp: fix stale doc comments on command queue methods
Fix some inaccuracies / old doc comments.
--- drivers/gpu/nova-core/gsp/cmdq.rs | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-)
drivers/gpu/nova-core/gsp/cmdq.rs +8 -9
@@ -502,6 +502,7 @@ fn notify_gsp(bar: &Bar0) {/// /// # Errors /// +/// - `EMSGSIZE` if the command exceeds the maximum queue element size. /// - `ETIMEDOUT` if space does not become available within the timeout. /// - `EIO` if the variable payload requested by the command has not been entirely /// written to by its [`CommandToGsp::init_variable_payload`] method.
@@ -682,22 +683,20 @@ fn wait_for_msg(&self, timeout: Delta) -> Result<GspMessage<'_>> {/// Receive a message from the GSP. /// -/// `init` is a closure tasked with processing the message. It receives a reference to the -/// message in the message queue, and a [`SBufferIter`] pointing to its variable-length -/// payload, if any. +/// The expected message type is specified using the `M` generic parameter. If the pending +/// message has a different function code, `ERANGE` is returned and the message is consumed. /// -/// The expected message is specified using the `M` generic parameter. If the pending message -/// is different, `EAGAIN` is returned and the unexpected message is dropped. -/// -/// This design is by no means final, but it is simple and will let us go through GSP -/// initialization. +/// The read pointer is always advanced past the message, regardless of whether it matched. /// /// # Errors /// /// - `ETIMEDOUT` if `timeout` has elapsed before any message becomes available. /// - `EIO` if there was some inconsistency (e.g. message shorter than advertised) on the /// message queue. -/// - `EINVAL` if the function of the message was unrecognized. +/// - `EINVAL` if the function code of the message was not recognized. +/// - `ERANGE` if the message had a recognized but non-matching function code. +/// +/// Error codes returned by [`MessageFromGsp::read`] are propagated as-is. pub (crate )fn receive_msg <M :MessageFromGsp >(&mut self, timeout:Delta ) ->Result <M >where // This allows all error types, including `Infallible`, to be used for `M::InitError`.
Signature · 1 line
2.53.0
PATCH v5 2/5 gpu: nova-core: gsp: add `RECEIVE_TIMEOUT` constant for command queue
In reply to Eliot CourtneyRemove magic numbers and add a default timeout for callers to use.
--- drivers/gpu/nova-core/gsp/cmdq.rs | 3 +++ drivers/gpu/nova-core/gsp/commands.rs | 5 ++--- drivers/gpu/nova-core/gsp/sequencer.rs | 2 +- 3 files changed, 6 insertions(+), 4 deletions(-)
drivers/gpu/nova-core/gsp/cmdq.rs +3 -0
@@ -467,6 +467,9 @@ impl Cmdq {/// Timeout for waiting for space on the command queue. const ALLOCATE_TIMEOUT :Delta =Delta ::from_secs (1 ); +/// Default timeout for receiving a message from the GSP. +pub (super )const RECEIVE_TIMEOUT :Delta =Delta ::from_secs (5 ); +/// Creates a new command queue for `dev`. pub (crate )fn new (dev: &device::Device <device::Bound >) ->Result <Cmdq > {let gsp_mem =DmaGspMem ::new (dev)?;
drivers/gpu/nova-core/gsp/commands.rs +2 -3
@@ -11,7 +11,6 @@ device, pci, prelude::*, - time::Delta, transmute::{AsBytes ,FromBytes ,//
@@ -165,7 +164,7 @@ fn read(/// Waits for GSP initialization to complete. pub (crate )fn wait_gsp_init_done (cmdq: &mut Cmdq ) ->Result {loop { -match cmdq.receive_msg ::<GspInitDone >(Delta ::from_secs (10 )) { +match cmdq.receive_msg ::<GspInitDone >(Cmdq ::RECEIVE_TIMEOUT ) {Ok (_) =>break Ok (()),Err (ERANGE ) =>continue ,Err (e) =>break Err (e),
@@ -235,7 +234,7 @@ pub(crate) fn get_gsp_info(cmdq: &mut Cmdq, bar: &Bar0) -> Result<GetGspStaticIn cmdq.send_command (bar,GetGspStaticInfo )?;loop { -match cmdq.receive_msg ::<GetGspStaticInfoReply >(Delta ::from_secs (5 )) { +match cmdq.receive_msg ::<GetGspStaticInfoReply >(Cmdq ::RECEIVE_TIMEOUT ) {Ok (info) =>return Ok (info),Err (ERANGE ) =>continue ,Err (e) =>return Err (e),
drivers/gpu/nova-core/gsp/sequencer.rs +1 -1
@@ -358,7 +358,7 @@ pub(crate) struct GspSequencerParams<'a> {impl <'a>GspSequencer <'a> {pub (crate )fn run (cmdq: &mut Cmdq , params:GspSequencerParams <'a>) ->Result {let seq_info =loop { -match cmdq.receive_msg ::<GspSequence >(Delta ::from_secs (10 )) { +match cmdq.receive_msg ::<GspSequence >(Cmdq ::RECEIVE_TIMEOUT ) {Ok (seq_info) =>break seq_info,Err (ERANGE ) =>continue ,Err (e) =>return Err (e),
Signature · 1 line
2.53.0
PATCH v5 3/5 gpu: nova-core: gsp: add reply/no-reply info to `CommandToGsp`
In reply to Eliot CourtneyAdd type infrastructure to know what reply is expected from each CommandToGsp. Uses a marker type NoReply which does not implement MessageFromGsp to mark commands which don't expect a response.
Update send_command to wait for a reply and add send_command_no_wait which sends a command that has no reply, without blocking.
This prepares for adding locking to the queue.
--- drivers/gpu/nova-core/gsp/boot.rs | 5 ++- drivers/gpu/nova-core/gsp/cmdq.rs | 62 ++++++++++++++++++++++++-- drivers/gpu/nova-core/gsp/cmdq/continuation.rs | 8 +++- drivers/gpu/nova-core/gsp/commands.rs | 16 +++---- 4 files changed, 75 insertions(+), 16 deletions(-)
drivers/gpu/nova-core/gsp/boot.rs +3 -2
@@ -169,8 +169,9 @@ pub(crate) fn boot(dma_write! (wpr_meta, [0 ]?,GspFwWprMeta ::new (&gsp_fw, &fb_layout)); self.cmdq - .send_command (bar, commands::SetSystemInfo ::new (pdev))?; - self.cmdq.send_command (bar, commands::SetRegistry ::new ())?; + .send_command_no_wait (bar, commands::SetSystemInfo ::new (pdev))?; + self.cmdq + .send_command_no_wait (bar, commands::SetRegistry ::new ())?; gsp_falcon.reset (bar)?;let libos_handle = self.libos.dma_handle ();
drivers/gpu/nova-core/gsp/cmdq.rs +59 -3
88 lines not on this page · raw message
drivers/gpu/nova-core/gsp/cmdq/continuation.rs +7 -1
@@ -6,7 +6,10 @@use kernel::prelude::*; -use super ::CommandToGsp ; +use super ::{ +CommandToGsp , +NoReply ,// +};use crate ::{ gsp::fw::{
@@ -63,6 +66,7 @@ fn new(data: &'a [u8]) -> Self {impl <'a>CommandToGsp for ContinuationRecord <'a> {const FUNCTION :MsgFunction =MsgFunction ::ContinuationRecord ;type Command = (); +type Reply =NoReply ;type InitError =Infallible ;fn init (&self) ->impl Init <Self ::Command ,Self ::InitError > {
@@ -144,6 +148,7 @@ fn new(command: C, payload: KVVec<u8>) -> Self {impl <C :CommandToGsp >CommandToGsp for SplitCommand <C > {const FUNCTION :MsgFunction =C ::FUNCTION ;type Command =C ::Command ; +type Reply =C ::Reply ;type InitError =C ::InitError ;fn init (&self) ->impl Init <Self ::Command ,Self ::InitError > {
@@ -206,6 +211,7 @@ fn new(len: usize) -> Result<Self> {impl CommandToGsp for TestPayload {const FUNCTION :MsgFunction =MsgFunction ::Nop ;type Command =TestHeader ; +type Reply =NoReply ;type InitError =Infallible ;fn init (&self) ->impl Init <Self ::Command ,Self ::InitError > {
drivers/gpu/nova-core/gsp/commands.rs +6 -10
@@ -23,7 +23,8 @@ cmdq::{Cmdq ,CommandToGsp , - MessageFromGsp, // +MessageFromGsp , +NoReply ,// }, fw::{ commands::*,
@@ -48,6 +49,7 @@ pub(crate) fn new(pdev: &'a pci::Device<device::Bound>) -> Self {impl <'a>CommandToGsp for SetSystemInfo <'a> {const FUNCTION :MsgFunction =MsgFunction ::GspSetSystemInfo ;type Command =GspSetSystemInfo ; +type Reply =NoReply ;type InitError =Error ;fn init (&self) ->impl Init <Self ::Command ,Self ::InitError > {
@@ -99,6 +101,7 @@ pub(crate) fn new() -> Self {impl CommandToGsp for SetRegistry {const FUNCTION :MsgFunction =MsgFunction ::SetRegistry ;type Command =PackedRegistryTable ; +type Reply =NoReply ;type InitError =Infallible ;fn init (&self) ->impl Init <Self ::Command ,Self ::InitError > {
@@ -178,6 +181,7 @@ pub(crate) fn wait_gsp_init_done(cmdq: &mut Cmdq) -> Result {impl CommandToGsp for GetGspStaticInfo {const FUNCTION :MsgFunction =MsgFunction ::GetGspStaticInfo ;type Command =GspStaticConfigInfo ; +type Reply =GetGspStaticInfoReply ;type InitError =Infallible ;fn init (&self) ->impl Init <Self ::Command ,Self ::InitError > {
@@ -231,13 +235,5 @@ pub(crate) fn gpu_name(&self) -> core::result::Result<&str, GpuNameError> {/// Send the [`GetGspInfo`] command and awaits for its reply. pub (crate )fn get_gsp_info (cmdq: &mut Cmdq , bar: &Bar0 ) ->Result <GetGspStaticInfoReply > { - cmdq.send_command (bar,GetGspStaticInfo )?; - -loop { -match cmdq.receive_msg ::<GetGspStaticInfoReply >(Cmdq ::RECEIVE_TIMEOUT ) { -Ok (info) =>return Ok (info), -Err (ERANGE ) =>continue , -Err (e) =>return Err (e), - } - } + cmdq.send_command (bar,GetGspStaticInfo ) }
Signature · 1 line
2.53.0
PATCH v5 4/5 gpu: nova-core: gsp: make `Cmdq` a pinned type
In reply to Eliot CourtneyMake Cmdq a pinned type. This is needed to use Mutex, which is needed to add locking to Cmdq.
--- drivers/gpu/nova-core/gsp.rs | 5 +++-- drivers/gpu/nova-core/gsp/cmdq.rs | 9 ++++----- 2 files changed, 7 insertions(+), 7 deletions(-)
drivers/gpu/nova-core/gsp.rs +3 -2
@@ -114,6 +114,7 @@ pub(crate) struct Gsp {/// RM log buffer. logrm: LogBuffer,/// Command queue. +#[pin] pub(crate) cmdq: Cmdq,/// RM arguments. rmargs: CoherentAllocation<GspArgumentsPadded>,
@@ -134,7 +135,7 @@ pub(crate) fn new(pdev: &pci::Device<device::Bound>) -> impl PinInit<Self, Error loginit: LogBuffer::new(dev)?, logintr: LogBuffer::new(dev)?, logrm: LogBuffer::new(dev)?, - cmdq: Cmdq::new(dev)?, + cmdq <- Cmdq::new(dev), rmargs: CoherentAllocation::<GspArgumentsPadded>::alloc_coherent( dev, 1,
@@ -151,7 +152,7 @@ pub(crate) fn new(pdev: &pci::Device<device::Bound>) -> impl PinInit<Self, Error libos, [1]?, LibosMemoryRegionInitArgument::new("LOGINTR", &logintr.0) );dma_write! (libos, [2 ]?,LibosMemoryRegionInitArgument ::new ("LOGRM" , &logrm.0 )); -dma_write! (rmargs, [0 ]?.inner, fw::GspArgumentsCached ::new (cmdq)); +dma_write! (rmargs, [0 ]?.inner, fw::GspArgumentsCached ::new (&cmdq));dma_write! (libos, [3 ]?,LibosMemoryRegionInitArgument ::new ("RMARGS" , rmargs)); }, }))
drivers/gpu/nova-core/gsp/cmdq.rs +4 -5
@@ -446,6 +446,7 @@ struct GspMessage<'a> {/// /// Provides the ability to send commands and receive messages from the GSP using a shared memory /// area. +#[pin_data] pub (crate )struct Cmdq {/// Device this command queue belongs to. dev:ARef <device::Device >,
@@ -479,13 +480,11 @@ impl Cmdq {pub (super )const RECEIVE_TIMEOUT :Delta =Delta ::from_secs (5 );/// Creates a new command queue for `dev`. -pub (crate )fn new (dev: &device::Device <device::Bound >) ->Result <Cmdq > { -let gsp_mem =DmaGspMem ::new (dev)?; - -Ok (Cmdq { +pub (crate )fn new (dev: &device::Device <device::Bound >) ->impl PinInit <Self ,Error > + '_ { +try_pin_init! (Self { + gsp_mem:DmaGspMem ::new (dev)?, dev: dev.into (), seq:0 , - gsp_mem, }) }
Signature · 1 line
2.53.0
PATCH v5 5/5 gpu: nova-core: gsp: add mutex locking to Cmdq
In reply to Eliot CourtneyWrap Cmdq's mutable state in a new struct CmdqInner and wrap that in a Mutex. This lets Cmdq methods take &self instead of &mut self, which lets required commands be sent e.g. while unloading the driver.
The mutex is held over both send and receive in send_command to make sure that it doesn't get the reply of some other command that could have been sent just beforehand.
--- drivers/gpu/nova-core/gsp/boot.rs | 8 +- drivers/gpu/nova-core/gsp/cmdq.rs | 170 +++++++++++++++++++-------------- drivers/gpu/nova-core/gsp/commands.rs | 4 +- drivers/gpu/nova-core/gsp/sequencer.rs | 2 +- 4 files changed, 107 insertions(+), 77 deletions(-)
drivers/gpu/nova-core/gsp/boot.rs +4 -4
@@ -137,7 +137,7 @@ fn run_fwsec_frts(/// /// Upon return, the GSP is up and running, and its runtime object given as return value. pub (crate )fn boot ( -mut self:Pin <&mut Self >, + self:Pin <&mut Self >, pdev: &pci::Device <device::Bound >, bar: &Bar0 , chipset: Chipset,
@@ -223,13 +223,13 @@ pub(crate) fn boot( dev: pdev.as_ref().into(), bar, }; -GspSequencer ::run (&mut self.cmdq, seq_params)?; +GspSequencer ::run (&self.cmdq, seq_params)?;// Wait until GSP is fully initialized. - commands::wait_gsp_init_done (&mut self.cmdq)?; + commands::wait_gsp_init_done (&self.cmdq)?;// Obtain and display basic GPU information. -let info = commands::get_gsp_info (&mut self.cmdq, bar)?; +let info = commands::get_gsp_info (&self.cmdq, bar)?;match info.gpu_name () {Ok (name) =>dev_info! (pdev,"GPU name: {}\n" , name),Err (e) =>dev_warn! (pdev,"GPU name unavailable: {:?}\n" , e),
drivers/gpu/nova-core/gsp/cmdq.rs +100 -70
235 lines not on this page · raw message
drivers/gpu/nova-core/gsp/commands.rs +2 -2
@@ -165,7 +165,7 @@ fn read( }/// Waits for GSP initialization to complete. -pub (crate )fn wait_gsp_init_done (cmdq: &mut Cmdq ) ->Result { +pub (crate )fn wait_gsp_init_done (cmdq: &Cmdq ) ->Result {loop {match cmdq.receive_msg ::<GspInitDone >(Cmdq ::RECEIVE_TIMEOUT ) {Ok (_) =>break Ok (()),
@@ -234,6 +234,6 @@ pub(crate) fn gpu_name(&self) -> core::result::Result<&str, GpuNameError> { }/// Send the [`GetGspInfo`] command and awaits for its reply. -pub (crate )fn get_gsp_info (cmdq: &mut Cmdq , bar: &Bar0 ) ->Result <GetGspStaticInfoReply > { +pub (crate )fn get_gsp_info (cmdq: &Cmdq , bar: &Bar0 ) ->Result <GetGspStaticInfoReply > { cmdq.send_command (bar,GetGspStaticInfo ) }
drivers/gpu/nova-core/gsp/sequencer.rs +1 -1
@@ -356,7 +356,7 @@ pub(crate) struct GspSequencerParams<'a> { }impl <'a>GspSequencer <'a> { -pub (crate )fn run (cmdq: &mut Cmdq , params:GspSequencerParams <'a>) ->Result { +pub (crate )fn run (cmdq: &Cmdq , params:GspSequencerParams <'a>) ->Result {let seq_info =loop {match cmdq.receive_msg ::<GspSequence >(Cmdq ::RECEIVE_TIMEOUT ) {Ok (seq_info) =>break seq_info,
Signature · 1 line
2.53.0
Alexandre Courbot
maintainerRe: [PATCH v5 5/5] gpu: nova-core: gsp: add mutex locking to Cmdq
On Wed Mar 18, 2026 at 1:07 PM JST, Eliot Courtney wrote:
<snip>
drivers/gpu/nova-core/gsp/cmdq.rs fn notify_gsp(bar: &Bar0) { · line 577
Quoted 4 lines
drivers/gpu/nova-core/gsp/cmdq.rs +4
-0+/// Returns the DMA handle of the command queue's shared memory region. +pub (crate )fn dma_handle (&self) ->DmaAddress { + self.inner.lock ().gsp_mem.0 .dma_handle () + }
Just noticed that we now need to lock to get the DMA handle. It's inconsequential in practice but a bit inelegant.
Since the DMA handle never changes, and is only ever needed during initialization, I think I will just insert a patch before this one that adds a pub(super) dma_handle member to Cmdq. That way we only need to obtain the handle at construction time and can get rid of this method, which keeps the public API focused on message handling.
No need to resend, I will apply this patch on top of mine and merge.
Danilo Krummrich
maintainerRe: [PATCH v5 5/5] gpu: nova-core: gsp: add mutex locking to Cmdq
Quoted 10 lines · Alexandre Courbot
On Wed Mar 18, 2026 at 1:07 PM JST, Eliot Courtney wrote:
<snip>drivers/gpu/nova-core/gsp/cmdq.rs fn notify_gsp(bar: &Bar0) { · line 577
Quoted 4 lines
drivers/gpu/nova-core/gsp/cmdq.rs +4
-0+/// Returns the DMA handle of the command queue's shared memory region. +pub (crate )fn dma_handle (&self) ->DmaAddress { + self.inner.lock ().gsp_mem.0 .dma_handle () + }Just noticed that we now need to lock to get the DMA handle. It's inconsequential in practice but a bit inelegant.
Quoted 7 lines
Since the DMA handle never changes, and is only ever needed during initialization, I think I will just insert a patch before this one that adds a
pub(super) dma_handlemember toCmdq. That way we only need to obtain the handle at construction time and can get rid of this method, which keeps the public API focused on message handling.No need to resend, I will apply this patch on top of mine and merge.
It is a minor inconvinience, but it may indicate that dma::Coherent should probaly support locking, i.e. you want to protect the data within the dma::Coherent allocation, not the dma::Coherent object itself.
Alexandre Courbot
maintainerRe: [PATCH v5 5/5] gpu: nova-core: gsp: add mutex locking to Cmdq
Quoted 17 lines · Danilo Krummrich
Quoted 16 lines · Alexandre Courbot
On Wed Mar 18, 2026 at 1:07 PM JST, Eliot Courtney wrote:
<snip>drivers/gpu/nova-core/gsp/cmdq.rs fn notify_gsp(bar: &Bar0) { · line 577
Quoted 4 lines
drivers/gpu/nova-core/gsp/cmdq.rs +4
-0+/// Returns the DMA handle of the command queue's shared memory region. +pub (crate )fn dma_handle (&self) ->DmaAddress { + self.inner.lock ().gsp_mem.0 .dma_handle () + }Just noticed that we now need to lock to get the DMA handle. It's inconsequential in practice but a bit inelegant.
Since the DMA handle never changes, and is only ever needed during initialization, I think I will just insert a patch before this one that adds a
pub(super) dma_handlemember toCmdq. That way we only need to obtain the handle at construction time and can get rid of this method, which keeps the public API focused on message handling.
Quoted 5 lines
Quoted 1 line
No need to resend, I will apply this patch on top of mine and merge.
It is a minor inconvinience, but it may indicate that
dma::Coherentshould probaly support locking, i.e. you want to protect the data within thedma::Coherentallocation, not thedma::Coherentobject itself.
Not quite sure I understand what you mean here - can you elaborate? This sounds to me like you want to add a Mutex to every dma::Coherent, so I am likely misunderstanding. :)
Alexandre Courbot
maintainerRe: [PATCH v5 5/5] gpu: nova-core: gsp: add mutex locking to Cmdq
In reply to Alexandre CourbotQuoted 10 lines · Alexandre Courbot
On Wed Mar 18, 2026 at 1:07 PM JST, Eliot Courtney wrote:
<snip>drivers/gpu/nova-core/gsp/cmdq.rs fn notify_gsp(bar: &Bar0) { · line 577
Quoted 4 lines
drivers/gpu/nova-core/gsp/cmdq.rs +4
-0+/// Returns the DMA handle of the command queue's shared memory region. +pub (crate )fn dma_handle (&self) ->DmaAddress { + self.inner.lock ().gsp_mem.0 .dma_handle () + }Just noticed that we now need to lock to get the DMA handle. It's inconsequential in practice but a bit inelegant.
Quoted 7 lines
Since the DMA handle never changes, and is only ever needed during initialization, I think I will just insert a patch before this one that adds a
pub(super) dma_handlemember toCmdq. That way we only need to obtain the handle at construction time and can get rid of this method, which keeps the public API focused on message handling.No need to resend, I will apply this patch on top of mine and merge.
Actually let's fix that afterwards, what I proposed above would introduce code that hasn't undergone review, and doesn't reduce the number of patches in the series so doesn't bring any benefit. I'll merge the series as-is for now.
Re: [PATCH v5 5/5] gpu: nova-core: gsp: add mutex locking to Cmdq
Quoted 17 lines · Alexandre Courbot
Quoted 16 lines · Alexandre Courbot
On Wed Mar 18, 2026 at 1:07 PM JST, Eliot Courtney wrote:
<snip>drivers/gpu/nova-core/gsp/cmdq.rs fn notify_gsp(bar: &Bar0) { · line 577
Quoted 4 lines
drivers/gpu/nova-core/gsp/cmdq.rs +4
-0+/// Returns the DMA handle of the command queue's shared memory region. +pub (crate )fn dma_handle (&self) ->DmaAddress { + self.inner.lock ().gsp_mem.0 .dma_handle () + }Just noticed that we now need to lock to get the DMA handle. It's inconsequential in practice but a bit inelegant.
Since the DMA handle never changes, and is only ever needed during initialization, I think I will just insert a patch before this one that adds a
pub(super) dma_handlemember toCmdq. That way we only need to obtain the handle at construction time and can get rid of this method, which keeps the public API focused on message handling.
Quoted 6 lines
Quoted 1 line
No need to resend, I will apply this patch on top of mine and merge.
Actually let's fix that afterwards, what I proposed above would introduce code that hasn't undergone review, and doesn't reduce the number of patches in the series so doesn't bring any benefit. I'll merge the series as-is for now.
For what it's worth, this avoided an email from me, to the effect of "there is no such thing as a quick locking fixup, don't do this".
So, good. :)
thanks,
Signature · 1 line
John Hubbard
Re: [PATCH v5 5/5] gpu: nova-core: gsp: add mutex locking to Cmdq
In reply to Alexandre CourbotQuoted 10 lines · Alexandre Courbot
On Wed Mar 18, 2026 at 1:07 PM JST, Eliot Courtney wrote:
<snip>drivers/gpu/nova-core/gsp/cmdq.rs fn notify_gsp(bar: &Bar0) { · line 577
Quoted 4 lines
drivers/gpu/nova-core/gsp/cmdq.rs +4
-0+/// Returns the DMA handle of the command queue's shared memory region. +pub (crate )fn dma_handle (&self) ->DmaAddress { + self.inner.lock ().gsp_mem.0 .dma_handle () + }Just noticed that we now need to lock to get the DMA handle. It's inconsequential in practice but a bit inelegant.
Quoted 7 lines
Since the DMA handle never changes, and is only ever needed during initialization, I think I will just insert a patch before this one that adds a
pub(super) dma_handlemember toCmdq. That way we only need to obtain the handle at construction time and can get rid of this method, which keeps the public API focused on message handling.No need to resend, I will apply this patch on top of mine and merge.
Yerp it's not ideal, but we only call this method in one location, once, when creating the command queue. The locking in this series is very basic and we probably want to improve it later to be more performant, so I think it's simpler and better to just leave this as is for now.
Quoted 29 lines · Eliot Courtney
Add locking to Cmdq. This is required e.g. for unloading the driver, which needs to send the UnloadingGuestDriver via the command queue on unbind which may be on a different thread.
We have commands that need a reply and commands that don't. For commands with a reply we want to make sure that they don't get the reply of a different command back. The approach this patch series takes is by making those commands block until they get a response. For now this should be ok, and we expect GSP to be fast anyway.
To do this, we need to know which commands expect a reply and which don't. John's existing series[1] adds
IS_ASYNCwhich solves part of the problem, but we need to know a bit more. So instead, add an associated type called Reply which tells us what the reply is.An alternative would be to define traits inheriting CommandToGsp, e.g. CommandWithReply and CommandWithoutReply, instead of using the associated type. I implemented the associated type version because it feels more compositional rather than inherity so seemed a bit better to me. But both of these approaches work and are fine, IMO.
In summary, this patch series has three steps:
1. Add the type infrastructure to know what replies are expected for a command and update each caller to explicitly wait for the reply or not.
2. Make Cmdq pinned so we can use Mutex
3. Add a Mutex to protect Cmdq by moving the relevant state to an inner struct.
Quoted 3 lines
[1]: https://lore.kernel.org/all/[email protected]/
Signed-off-by Eliot Courtney <[email protected]>
Merged into drm-rust-next, thanks!