accel/rocket: RK3576 NPU (RKNN) enablement
Subsystems (7)
v17 cover accel/rocket: RK3576 NPU (RKNN) enablement
This series adds RK3576 NPU (RKNN) support to the mainline DRM rocket driver.
Based on: next-20261113
The pmdomain part of v16 9/16 is now a separate series on Ulf's next (v16 10/16 is dropped). 12/12 needs its 2/3 to make the NPU domain take its supply:
https://lore.kernel.org/all/[email protected]/
Changes in v17:
- Dropped v16 7/16, 10/16 and 14/16 (the power-domain reset and the extra domain clocks), following Finley Xiao's (Rockchip) review.
- Following Heiko, the clock indices are now ROCKET_CLK_* constants (1/12 and 9/12).
- The pmdomain changes are now a separate series, linked above.
Testing:
Tested on a Radxa ROCK 4D running Armbian edge 7.3, with the v17 changes backported (this series and the pmdomain series):
- charsiu (https://github.com/gahingwoo/charsiu), an open source LLM runtime that drives the NPU through the rocket UAPI (/dev/accel) without Mesa or the vendor runtime, ran Qwen3-0.6B (Q4_0) 30 times after a warm reboot and 30 times after a cold power-on. Each run starts with the NPU power domain off. All 60 outputs are identical, with no job timeout, IOMMU error or SError.
- Job timeouts forced with a 1 ms timeout, three times in one boot:
after each, the next run gives the same output as before, with no error.
The v15/v16 comparison of the reset order is in 4/12.
Logs: https://github.com/gahingwoo/linux-rk3576-npu/tree/main/board-logs
Known issue: in v16 I saw the IOMMU not recover after repeated resets with a 2 ms job timeout ("MMU_DTE_ADDR is not functioning"). With v17, after three rounds of forced timeouts in one boot, it recovered each time; I have not looked into what changed.
Tool use:
An LLM helped me read code and technical material and write part of the implementation. Some of the code in the series was first drafted by an LLM, and the commit messages and cover letters of earlier versions were mostly written by an LLM.
I reviewed the generated code and tested the changes on real hardware. I also used an LLM to write test scripts and to compare register write traces.
For v17 I wrote the cover letter, commit messages and code comments myself in Chinese first and translated them into English with an LLM.
Tag changes:
- 1/12 changed in v17. Heiko, does your Reviewed-by still apply? If the change goes beyond what you reviewed, please take another look.
- 7/12 keeps Conor's Acked-by. The clock count changed from five to four in v16; Conor, please confirm it still applies.
- Tomeu, the GET_PARAM enum in 10/12 is unchanged (RK3588 = 1, RK3576 = 2). Is that fine?
Igor Paunovic (1):
accel/rocket: request the core clocks by name
Jiaxing Hu (11):
accel/rocket: take the completion register writes under job_lock accel/rocket: wait for a running IRQ handler before resetting a core accel/rocket: let the core suspend after a reset accel/rocket: factor the completion tail out of the IRQ handler
dt-bindings: npu: rockchip: add rockchip,rk3576-rknn-core
dt-bindings: iommu: rockchip: describe the RK3576 NPU MMU accel/rocket: select the per-core clock and reset counts from match
data
accel/rocket: add RK3576 NPU (RKNN) support
accel/rocket: add a GET_PARAM ioctl for the hardware version
arm64: dts: rockchip: add NPU (RKNN) nodes to rk3576
arm64: dts: rockchip: enable the NPU on rk3576-rock-4d
.../bindings/iommu/rockchip,iommu.yaml | 26 ++++++ .../npu/rockchip,rk3588-rknn-core.yaml | 40 ++++++++- .../boot/dts/rockchip/rk3576-rock-4d.dts | 22 +++++ arch/arm64/boot/dts/rockchip/rk3576.dtsi | 70 +++++++++++++++- drivers/accel/rocket/rocket_core.c | 14 +++- drivers/accel/rocket/rocket_core.h | 20 ++++- drivers/accel/rocket/rocket_device.c | 3 +- drivers/accel/rocket/rocket_drv.c | 61 ++++++++++++-- drivers/accel/rocket/rocket_drv.h | 2 + drivers/accel/rocket/rocket_job.c | 84 ++++++++++++++----- include/uapi/drm/rocket_accel.h | 40 +++++++++ 11 files changed, 345 insertions(+), 37 deletions(-)
base-commit: 84abb955b1015477aa11a96e2f3ac1b938b77f48
Signature · 1 line
2.43.0
PATCH v17 1/12 accel/rocket: request the core clocks by name
From: Igor Paunovic <[email protected]>
rocket_core_init() never sets the .id of the clocks, so all four are requested with a NULL name and each one resolves to the first clock in the DT, aclk.
As a result aclk is taken and enabled four times, while hclk, pclk and the NPU compute clock are never enabled by the driver. It only works today because the power-domain driver enables those clocks for it.
Give the four clocks their names, in the order the binding defines: aclk, hclk, npu and pclk.
With that, a DT that lacks any of these clock-names fails probe with -ENOENT instead of running on the wrong clock.
--- drivers/accel/rocket/rocket_core.c | 4 ++++ drivers/accel/rocket/rocket_core.h | 10 +++++++++- 2 files changed, 13 insertions(+), 1 deletion(-)
drivers/accel/rocket/rocket_core.c +4 -0
@@ -28,6 +28,10 @@ int rocket_core_init(struct rocket_core *core)if (err)return dev_err_probe (dev, err,"failed to get resets for core %d\n" , core->index); + core->clks[ROCKET_CLK_ACLK ].id ="aclk" ; + core->clks[ROCKET_CLK_HCLK ].id ="hclk" ; + core->clks[ROCKET_CLK_NPU ].id ="npu" ; + core->clks[ROCKET_CLK_PCLK ].id ="pclk" ; err =devm_clk_bulk_get (dev,ARRAY_SIZE (core->clks), core->clks);if (err)return dev_err_probe (dev, err,"failed to get clocks for core %d\n" , core->index);
drivers/accel/rocket/rocket_core.h +9 -1
@@ -27,6 +27,14 @@#define rocket_core_writel (core, reg, value) \ writel(value, (core)->core_iomem + (REG_CORE_##reg) - REG_CORE_S_STATUS) +enum rocket_clk { +ROCKET_CLK_ACLK , +ROCKET_CLK_HCLK , +ROCKET_CLK_NPU , +ROCKET_CLK_PCLK , +ROCKET_CLK_MAX , +}; +struct rocket_core {struct device *dev;struct rocket_device *rdev;
@@ -36,7 +44,7 @@ struct rocket_core {void __iomem *pc_iomem;void __iomem *cna_iomem;void __iomem *core_iomem; -struct clk_bulk_data clks[4 ]; +struct clk_bulk_data clks[ROCKET_CLK_MAX ];struct reset_control_bulk_data resets[2 ];struct iommu_group *iommu_group;
Signature · 1 line
2.43.0
PATCH v17 2/12 accel/rocket: take the completion register writes under job_lock
In reply to Jiaxing HuThe IRQ handler writes OPERATION_ENABLE and INTERRUPT_CLEAR before it takes job_lock, while rocket_job_hw_submit() sets OPERATION_ENABLE to 1 under that lock.
If the two run at the same time, the 0 written by the IRQ handler can overwrite the 1 written for a new task and stop a task that has just started.
Move both writes into the existing scoped_guard(), so they are under the same lock as the submit.
--- drivers/accel/rocket/rocket_job.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-)
drivers/accel/rocket/rocket_job.c +9 -3
@@ -371,10 +371,15 @@ static void rocket_job_handle_irq(struct rocket_core *core) {pm_runtime_mark_last_busy (core->dev); -rocket_pc_writel (core,OPERATION_ENABLE ,0x0 ); -rocket_pc_writel (core,INTERRUPT_CLEAR ,0x1ffff ); + scoped_guard(mutex, &core->job_lock) { +/* +* Stopping the block belongs under the lock. hw_submit() writes +* OPERATION_ENABLE too, and outside the lock this zero can land +* after that one and stop a task that has only just started. +*/ +rocket_pc_writel (core,OPERATION_ENABLE ,0x0 ); +rocket_pc_writel (core,INTERRUPT_CLEAR ,0x1ffff ); - scoped_guard(mutex, &core->job_lock)if (core->in_flight_job) {if (core->in_flight_job->next_task_idx < core->in_flight_job->task_count) {rocket_job_hw_submit (core, core->in_flight_job);
@@ -386,6 +391,7 @@ static void rocket_job_handle_irq(struct rocket_core *core)pm_runtime_put_autosuspend (core->dev); core->in_flight_job =NULL ; } + } } static void
Signature · 1 line
2.43.0
PATCH v17 3/12 accel/rocket: wait for a running IRQ handler before resetting a core
In reply to Jiaxing Hudrm_sched_stop() does not wait for a threaded IRQ handler that is already running, so call synchronize_irq() after it to make sure the handler has finished before the reset.
Before synchronize_irq(), mask the interrupt and clear the raw status, so that the NPU core cannot report a completion afterwards.
Do this under job_lock, because the submit path sets the interrupt mask under the same lock.
Only touch these registers when the device is powered: check with pm_runtime_get_if_active(), so that a register access with the power domain off cannot raise an SError.
--- drivers/accel/rocket/rocket_job.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
drivers/accel/rocket/rocket_job.c +10 -0
@@ -402,6 +402,16 @@ rocket_reset(struct rocket_core *core, struct drm_sched_job *bad)drm_sched_stop (&core->sched, bad); + scoped_guard(mutex, &core->job_lock) { +if (pm_runtime_get_if_active (core->dev) >0 ) { +rocket_pc_writel (core,INTERRUPT_MASK ,0x0 ); +rocket_pc_writel (core,INTERRUPT_CLEAR ,0x1ffff ); +pm_runtime_put_autosuspend (core->dev); + } + } + +synchronize_irq (core->irq); +/* * Remaining interrupts have been handled, but we might still have * stuck jobs. Let's make sure the PM counters stay balanced by
Signature · 1 line
2.43.0
PATCH v17 4/12 accel/rocket: let the core suspend after a reset
In reply to Jiaxing HuThe reset path drops the runtime PM reference with pm_runtime_put_noidle(), which does not let the device suspend, so the power domain is never switched off and on again.
On the RK3576, powering the domain on resets the NPU bus interface. If that does not happen after a reset, the IOMMU may not respond and the next job after a timeout fails.
Use pm_runtime_put_autosuspend() instead, so the reset path handles runtime PM the same way as a normal job completion.
Also drop the runtime PM reference after rocket_core_reset(), so the NPU core stays powered during the reset.
The earlier test on the ROCK 4D was done on the v15 code: the job after a timeout went from all 128 channels wrong to all 128 correct.
Repeated resets were triggered with a 2 ms job timeout.
v15 (put, then reset): tested twice; both times the board hung within the same second as the first batch of job timeouts, with nothing printed.
v16 (reset, then put): tested four times; the board never hung.
So moving pm_runtime_put_autosuspend() after rocket_core_reset() avoids the board hanging during a reset.
--- drivers/accel/rocket/rocket_job.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)
drivers/accel/rocket/rocket_job.c +7 -3
@@ -397,6 +397,8 @@ static void rocket_job_handle_irq(struct rocket_core *core)static void rocket_reset (struct rocket_core *core,struct drm_sched_job *bad) { +bool put_job_ref; +if (!atomic_read (&core->reset.pending))return ;
@@ -415,11 +417,10 @@ rocket_reset(struct rocket_core *core, struct drm_sched_job *bad)/* * Remaining interrupts have been handled, but we might still have * stuck jobs. Let's make sure the PM counters stay balanced by -* manually calling pm_runtime_put_noidle(). +* dropping the job's reference once the core has been reset. */ scoped_guard(mutex, &core->job_lock) { -if (core->in_flight_job) -pm_runtime_put_noidle (core->dev); + put_job_ref = core->in_flight_job;iommu_detach_group (NULL , core->iommu_group);
@@ -429,6 +430,9 @@ rocket_reset(struct rocket_core *core, struct drm_sched_job *bad)/* Proceed with reset now. */ rocket_core_reset (core); +if (put_job_ref) +pm_runtime_put_autosuspend (core->dev); +/* NPU has been reset, we can clear the reset pending bit. */ atomic_set (&core->reset.pending,0 );
Signature · 1 line
2.43.0
PATCH v17 5/12 accel/rocket: factor the completion tail out of the IRQ handler
In reply to Jiaxing HuMove the part of the IRQ handler that starts the next task or finishes the current job into rocket_job_next_locked(). This makes the code easier to follow and does not change behaviour.
--- drivers/accel/rocket/rocket_job.c | 30 +++++++++++++++++++----------- 1 file changed, 19 insertions(+), 11 deletions(-)
drivers/accel/rocket/rocket_job.c +19 -11
@@ -367,6 +367,24 @@ static struct dma_fence *rocket_job_run(struct drm_sched_job *sched_job)return ERR_PTR (ret); } +static void rocket_job_next_locked (struct rocket_core *core) +{ +lockdep_assert_held (&core->job_lock); + +if (!core->in_flight_job) +return ; + +if (core->in_flight_job->next_task_idx < core->in_flight_job->task_count) { +rocket_job_hw_submit (core, core->in_flight_job); +return ; + } + +iommu_detach_group (NULL ,iommu_group_get (core->dev)); +dma_fence_signal (core->in_flight_job->done_fence); +pm_runtime_put_autosuspend (core->dev); + core->in_flight_job =NULL ; +} +static void rocket_job_handle_irq (struct rocket_core *core) {pm_runtime_mark_last_busy (core->dev);
@@ -380,17 +398,7 @@ static void rocket_job_handle_irq(struct rocket_core *core)rocket_pc_writel (core,OPERATION_ENABLE ,0x0 );rocket_pc_writel (core,INTERRUPT_CLEAR ,0x1ffff ); -if (core->in_flight_job) { -if (core->in_flight_job->next_task_idx < core->in_flight_job->task_count) { -rocket_job_hw_submit (core, core->in_flight_job); -return ; - } - -iommu_detach_group (NULL ,iommu_group_get (core->dev)); -dma_fence_signal (core->in_flight_job->done_fence); -pm_runtime_put_autosuspend (core->dev); - core->in_flight_job =NULL ; - } +rocket_job_next_locked (core); } }
Signature · 1 line
2.43.0
PATCH v17 6/12 dt-bindings: npu: rockchip: add rockchip,rk3576-rknn-core
In reply to Jiaxing HuThe RK3576 and the RK3588 use the same NPU block, with different hardware resources.
The RK3576 has two cores. Each core needs six clocks, two more than the RK3588 (aclk_cbuf and hclk_cbuf), needs only one reset, and has no sram-supply.
Define the requirements of each SoC in allOf. The RK3588 still requires sram-supply.
--- .../npu/rockchip,rk3588-rknn-core.yaml | 40 ++++++++++++++++++- 1 file changed, 38 insertions(+), 2 deletions(-)
Documentation/devicetree/bindings/npu/rockchip,rk3588-rknn-core.yaml +38 -2
@@ -21,6 +21,7 @@ properties: compatible: enum: + - rockchip,rk3576-rknn-core - rockchip,rk3588-rknn-core reg:
@@ -33,14 +34,18 @@ properties: - const: core # Main NPU core processing unit registers clocks: - maxItems: 4 + minItems: 4 + maxItems: 6 clock-names: + minItems: 4 items: - const: aclk - const: hclk - const: npu - const: pclk + - const: aclk_cbuf + - const: hclk_cbuf interrupts: maxItems: 1
@@ -54,9 +59,11 @@ properties: maxItems: 1 resets: + minItems: 1 maxItems: 2 reset-names: + minItems: 1 items: - const: srst_a - const: srst_h
@@ -75,7 +82,36 @@ required: - resets - reset-names - npu-supply - - sram-supply + +allOf: + - if: + properties: + compatible: + contains: + const: rockchip,rk3588-rknn-core + then: + properties: + clocks: + maxItems: 4 + clock-names: + maxItems: 4 + resets: + minItems: 2 + reset-names: + minItems: 2 + required: + - sram-supply + else: + properties: + clocks: + minItems: 6 + clock-names: + minItems: 6 + resets: + maxItems: 1 + reset-names: + maxItems: 1 + sram-supply: false additionalProperties: false
Signature · 1 line
2.43.0
PATCH v17 7/12 dt-bindings: iommu: rockchip: describe the RK3576 NPU MMU
In reply to Jiaxing HuThe RK3576 NPU IOMMU is compatible with rk3568-iommu but needs four
clocks: aclk, iface and the two CBUF clocks.
Measured: with only aclk and iface, the IOMMU reset fails and the NPU does not probe. Without either one of the CBUF clocks it fails as well, and one of those configurations does not even boot.
With all four clocks the IOMMU works, and the npu clock is not needed.
rk_iommu enables every clock listed in the device tree (841363ebb508), so this list decides whether the IOMMU can reset.
--- .../bindings/iommu/rockchip,iommu.yaml | 26 +++++++++++++++++++ 1 file changed, 26 insertions(+)
Documentation/devicetree/bindings/iommu/rockchip,iommu.yaml +26 -0
@@ -26,6 +26,7 @@ properties: - items: - enum: - rockchip,rk3576-iommu + - rockchip,rk3576-npu-iommu - rockchip,rk3588-iommu - const: rockchip,rk3568-iommu
@@ -42,14 +43,20 @@ properties: minItems: 1 clocks: + minItems: 2 items: - description: Core clock - description: Interface clock + - description: Convolution buffer core clock + - description: Convolution buffer interface clock clock-names: + minItems: 2 items: - const: aclk - const: iface + - const: aclk_cbuf + - const: hclk_cbuf "#iommu-cells": const: 0
@@ -72,6 +79,25 @@ required: - clock-names - "#iommu-cells" +allOf: + - if: + properties: + compatible: + contains: + const: rockchip,rk3576-npu-iommu + then: + properties: + clocks: + minItems: 4 + clock-names: + minItems: 4 + else: + properties: + clocks: + maxItems: 2 + clock-names: + maxItems: 2 + additionalProperties: false examples:
Signature · 1 line
2.43.0
PATCH v17 8/12 accel/rocket: select the per-core clock and reset counts from match data
In reply to Jiaxing HuThe RK3576 and the RK3588 use the same NPU block, but with different numbers of clocks and resets.
Take the number of clocks and resets for each core from soc_data in the match data, so the driver uses the right count on each SoC.
The RK3588 keeps four clocks and two resets.
A device without match data fails before any resource is allocated.
--- drivers/accel/rocket/rocket_core.c | 8 ++++---- drivers/accel/rocket/rocket_core.h | 6 ++++++ drivers/accel/rocket/rocket_drv.c | 18 +++++++++++++++--- 3 files changed, 25 insertions(+), 7 deletions(-)
drivers/accel/rocket/rocket_core.c +4 -4
@@ -23,7 +23,7 @@ int rocket_core_init(struct rocket_core *core) core->resets[0 ].id ="srst_a" ; core->resets[1 ].id ="srst_h" ; - err =devm_reset_control_bulk_get_exclusive (&pdev->dev,ARRAY_SIZE (core->resets), + err =devm_reset_control_bulk_get_exclusive (&pdev->dev, core->soc->num_resets, core->resets);if (err)return dev_err_probe (dev, err,"failed to get resets for core %d\n" , core->index);
@@ -32,7 +32,7 @@ int rocket_core_init(struct rocket_core *core) core->clks[ROCKET_CLK_HCLK ].id ="hclk" ; core->clks[ROCKET_CLK_NPU ].id ="npu" ; core->clks[ROCKET_CLK_PCLK ].id ="pclk" ; - err =devm_clk_bulk_get (dev,ARRAY_SIZE (core->clks), core->clks); + err =devm_clk_bulk_get (dev, core->soc->num_clks, core->clks);if (err)return dev_err_probe (dev, err,"failed to get clocks for core %d\n" , core->index);
@@ -106,9 +106,9 @@ void rocket_core_fini(struct rocket_core *core)void rocket_core_reset (struct rocket_core *core) { -reset_control_bulk_assert (ARRAY_SIZE (core->resets), core->resets); +reset_control_bulk_assert (core->soc->num_resets, core->resets);udelay (10 ); -reset_control_bulk_deassert (ARRAY_SIZE (core->resets), core->resets); +reset_control_bulk_deassert (core->soc->num_resets, core->resets); }
drivers/accel/rocket/rocket_core.h +6 -0
@@ -35,9 +35,15 @@ enum rocket_clk { ROCKET_CLK_MAX, }; +struct rocket_soc_data { +unsigned int num_clks; +unsigned int num_resets; +}; +struct rocket_core {struct device *dev;struct rocket_device *rdev; +const struct rocket_soc_data *soc;unsigned int index;int irq;
drivers/accel/rocket/rocket_drv.c +15 -3
@@ -159,8 +159,14 @@ static const struct drm_driver rocket_drm_driver = {static int rocket_probe (struct platform_device *pdev) { +const struct rocket_soc_data *soc =of_device_get_match_data (&pdev->dev);int ret; +if (!soc) { +dev_err (&pdev->dev,"no match data for this device\n" ); +return -ENODEV ; + } +if (rdev ==NULL ) {/* First core probing, initialize DRM device. */ rdev =rocket_device_init (drm_dev, &rocket_drm_driver);
@@ -179,6 +185,7 @@ static int rocket_probe(struct platform_device *pdev) rdev->cores[core].rdev = rdev; rdev->cores[core].dev = &pdev->dev; + rdev->cores[core].soc = soc; rdev->cores[core].index = core; rdev->num_cores++;
@@ -216,8 +223,13 @@ static void rocket_remove(struct platform_device *pdev) } } +static const struct rocket_soc_data rk3588_soc_data = { + .num_clks =4 , + .num_resets =2 , +}; +static const struct of_device_id dt_match[] = { - { .compatible ="rockchip,rk3588-rknn-core" }, + { .compatible ="rockchip,rk3588-rknn-core" , .data = &rk3588_soc_data }, {} };MODULE_DEVICE_TABLE (of, dt_match);
@@ -243,7 +255,7 @@ static int rocket_device_runtime_resume(struct device *dev)if (core <0 )return -ENODEV ; - err =clk_bulk_prepare_enable (ARRAY_SIZE (rdev->cores[core].clks), rdev->cores[core].clks); + err =clk_bulk_prepare_enable (rdev->cores[core].soc->num_clks, rdev->cores[core].clks);if (err) {dev_err (dev,"failed to enable (%d) clocks for core %d\n" , err, core);return err;
@@ -263,7 +275,7 @@ static int rocket_device_runtime_suspend(struct device *dev)if (!rocket_job_is_idle (&rdev->cores[core]))return -EBUSY ; -clk_bulk_disable_unprepare (ARRAY_SIZE (rdev->cores[core].clks), rdev->cores[core].clks); +clk_bulk_disable_unprepare (rdev->cores[core].soc->num_clks, rdev->cores[core].clks);return 0 ; }
Signature · 1 line
2.43.0
PATCH v17 9/12 accel/rocket: add RK3576 NPU (RKNN) support
In reply to Jiaxing HuAdd support for the RK3576 NPU. It has two cores; each core needs six clocks and one reset, with aclk_cbuf and hclk_cbuf on top of what the RK3588 has.
The task number in PC_TASK_CON is 16 bits wide, not 12. BIT16, BIT17 and BIT18 are pp_en, count_clear and last_layer_clear. Chaoyi Chen confirmed this layout on the list:
https://lore.kernel.org/all/[email protected]/
The number of NPU cores is taken from the driver's own match table.
# Conflicts:
# drivers/accel/rocket/rocket_core.c
# drivers/accel/rocket/rocket_core.h
--- drivers/accel/rocket/rocket_core.c | 2 ++ drivers/accel/rocket/rocket_core.h | 3 +++ drivers/accel/rocket/rocket_device.c | 3 ++- drivers/accel/rocket/rocket_drv.c | 14 +++++++++++--- drivers/accel/rocket/rocket_drv.h | 2 ++ drivers/accel/rocket/rocket_job.c | 20 ++++++++++++++++---- 6 files changed, 36 insertions(+), 8 deletions(-)
drivers/accel/rocket/rocket_core.c +2 -0
@@ -32,6 +32,8 @@ int rocket_core_init(struct rocket_core *core) core->clks[ROCKET_CLK_HCLK ].id ="hclk" ; core->clks[ROCKET_CLK_NPU ].id ="npu" ; core->clks[ROCKET_CLK_PCLK ].id ="pclk" ; + core->clks[ROCKET_CLK_ACLK_CBUF ].id ="aclk_cbuf" ; + core->clks[ROCKET_CLK_HCLK_CBUF ].id ="hclk_cbuf" ; err =devm_clk_bulk_get (dev, core->soc->num_clks, core->clks);if (err)return dev_err_probe (dev, err,"failed to get clocks for core %d\n" , core->index);
drivers/accel/rocket/rocket_core.h +3 -0
@@ -32,12 +32,15 @@ enum rocket_clk {ROCKET_CLK_HCLK ,ROCKET_CLK_NPU ,ROCKET_CLK_PCLK , +ROCKET_CLK_ACLK_CBUF , +ROCKET_CLK_HCLK_CBUF , ROCKET_CLK_MAX, };struct rocket_soc_data {unsigned int num_clks;unsigned int num_resets; +bool task_con_16bit; }; struct rocket_core {
drivers/accel/rocket/rocket_device.c +2 -1
@@ -9,6 +9,7 @@#include <linux/of.h> #include "rocket_device.h" +#include "rocket_drv.h" struct rocket_device *rocket_device_init (struct platform_device *pdev,const struct drm_driver *rocket_drm_driver)
@@ -27,7 +28,7 @@ struct rocket_device *rocket_device_init(struct platform_device *pdev, ddev = &rdev->ddev;dev_set_drvdata (dev, rdev); -for_each_compatible_node (core_node,NULL ,"rockchip,rk3588-rknn-core" ) +for_each_matching_node (core_node, rocket_dt_match)if (of_device_is_available (core_node)) num_cores++;
drivers/accel/rocket/rocket_drv.c +11 -3
@@ -226,13 +226,21 @@ static void rocket_remove(struct platform_device *pdev)static const struct rocket_soc_data rk3588_soc_data = { .num_clks =4 , .num_resets =2 , + .task_con_16bit = false, }; -static const struct of_device_id dt_match[] = { +static const struct rocket_soc_data rk3576_soc_data = { + .num_clks =6 , + .num_resets =1 , + .task_con_16bit = true, +}; + +const struct of_device_id rocket_dt_match[] = { { .compatible ="rockchip,rk3588-rknn-core" , .data = &rk3588_soc_data }, + { .compatible ="rockchip,rk3576-rknn-core" , .data = &rk3576_soc_data }, {} }; -MODULE_DEVICE_TABLE (of, dt_match); +MODULE_DEVICE_TABLE (of, rocket_dt_match);static int find_core_for_dev (struct device *dev) {
@@ -291,7 +299,7 @@ static struct platform_driver rocket_driver = { .driver = { .name = "rocket", .pm = pm_ptr(&rocket_pm_ops), - .of_match_table = dt_match, + .of_match_table = rocket_dt_match, }, };
drivers/accel/rocket/rocket_drv.h +2 -0
@@ -6,10 +6,12 @@#include <drm/drm_mm.h> #include <drm/gpu_scheduler.h> +#include <linux/device-id/of.h> #include "rocket_device.h" extern const struct dev_pm_ops rocket_pm_ops; +extern const struct of_device_id rocket_dt_match[];struct rocket_iommu_domain {struct iommu_domain *domain;
drivers/accel/rocket/rocket_job.c +16 -4
@@ -31,6 +31,11 @@/* Bits 3:0 of PC_BASE_ADDRESS hold the mode selection bit and reserved bits */ #define ROCKET_REGCMD_ALIGN 16 +#define RK3576_PC_TASK_CON_TASK_NUMBER (n) ((n) & 0xffff) +#define RK3576_PC_TASK_CON_PP_EN BIT(16) +#define RK3576_PC_TASK_CON_COUNT_CLEAR BIT(17) +#define RK3576_PC_TASK_CON_LAST_LAYER_CLEAR BIT(18) +static struct rocket_job *to_rocket_job (struct drm_sched_job *sched_job) {
@@ -166,10 +171,17 @@ static void rocket_job_hw_submit(struct rocket_core *core, struct rocket_job *jorocket_pc_writel (core,INTERRUPT_MASK ,PC_INTERRUPT_MASK_DPU_0 |PC_INTERRUPT_MASK_DPU_1 );rocket_pc_writel (core,INTERRUPT_CLEAR ,PC_INTERRUPT_CLEAR_DPU_0 |PC_INTERRUPT_CLEAR_DPU_1 ); -rocket_pc_writel (core,TASK_CON ,PC_TASK_CON_RESERVED_0 (1 ) | -PC_TASK_CON_TASK_COUNT_CLEAR (1 ) | -PC_TASK_CON_TASK_NUMBER (1 ) | -PC_TASK_CON_TASK_PP_EN (1 )); +if (core->soc->task_con_16bit) +rocket_pc_writel (core,TASK_CON , +RK3576_PC_TASK_CON_LAST_LAYER_CLEAR | +RK3576_PC_TASK_CON_COUNT_CLEAR | +RK3576_PC_TASK_CON_PP_EN | +RK3576_PC_TASK_CON_TASK_NUMBER (1 )); +else +rocket_pc_writel (core,TASK_CON ,PC_TASK_CON_RESERVED_0 (1 ) | +PC_TASK_CON_TASK_COUNT_CLEAR (1 ) | +PC_TASK_CON_TASK_NUMBER (1 ) | +PC_TASK_CON_TASK_PP_EN (1 ));rocket_pc_writel (core,TASK_DMA_BASE_ADDR ,PC_TASK_DMA_BASE_ADDR_DMA_BASE_ADDR (0x0 ));
Signature · 1 line
2.43.0
PATCH v17 10/12 accel/rocket: add a GET_PARAM ioctl for the hardware version
In reply to Jiaxing HuAdd DRM_IOCTL_ROCKET_GET_PARAM so that userspace can query the NPU hardware version and pick the matching hardware description, such as the SRAM configuration and the register layout.
Add DRM_ROCKET_PARAM_HW_VERSION, which returns a hardware version enum that matches the device tree compatible.
The enum starts at 1 and 0 is not used:
RK3588: 1
RK3576: 2
An unknown parameter or a non-zero reserved field returns -EINVAL.
--- drivers/accel/rocket/rocket_core.h | 1 + drivers/accel/rocket/rocket_drv.c | 29 ++++++++++++++++++++++ include/uapi/drm/rocket_accel.h | 40 ++++++++++++++++++++++++++++++ 3 files changed, 70 insertions(+)
drivers/accel/rocket/rocket_core.h +1 -0
@@ -41,6 +41,7 @@ struct rocket_soc_data {unsigned int num_clks;unsigned int num_resets;bool task_con_16bit; +u32 hw_version; };struct rocket_core {
drivers/accel/rocket/rocket_drv.c +29 -0
@@ -5,6 +5,7 @@#include <drm/drm_drv.h> #include <drm/drm_gem.h> #include <drm/drm_ioctl.h> +#include <drm/drm_print.h> #include <drm/rocket_accel.h> #include <linux/clk.h> #include <linux/err.h>
@@ -129,6 +130,31 @@ rocket_postclose(struct drm_device *dev, struct drm_file *file)module_put (THIS_MODULE ); } +static int +rocket_ioctl_get_param (struct drm_device *dev,void *data,struct drm_file *file) +{ +struct rocket_device *rdev =to_rocket_device (dev); +struct drm_rocket_get_param *args = data; +const struct rocket_soc_data *soc; + +if (args->reserved !=0 ) { +drm_dbg (dev,"Reserved field in drm_rocket_get_param struct should be 0.\n" ); +return -EINVAL ; + } + + soc =READ_ONCE (rdev->cores[0 ].soc); +if (!soc) +return -ENODEV ; + +switch (args->param) { +case DRM_ROCKET_PARAM_HW_VERSION : + args->value = soc->hw_version; +return 0 ; +default : +return -EINVAL ; + } +} +static const struct drm_ioctl_desc rocket_drm_driver_ioctls[] = { #define ROCKET_IOCTL(n, func) \ DRM_IOCTL_DEF_DRV(ROCKET_##n, rocket_ioctl_##func, 0)
@@ -137,6 +163,7 @@ static const struct drm_ioctl_desc rocket_drm_driver_ioctls[] = {ROCKET_IOCTL (SUBMIT , submit),ROCKET_IOCTL (PREP_BO , prep_bo),ROCKET_IOCTL (FINI_BO , fini_bo), +ROCKET_IOCTL (GET_PARAM , get_param), };DEFINE_DRM_ACCEL_FOPS (rocket_accel_driver_fops);
@@ -227,12 +254,14 @@ static const struct rocket_soc_data rk3588_soc_data = { .num_clks = 4, .num_resets = 2, .task_con_16bit = false, + .hw_version = DRM_ROCKET_HW_RK3588, };static const struct rocket_soc_data rk3576_soc_data = { .num_clks =6 , .num_resets =1 , .task_con_16bit = true, + .hw_version =DRM_ROCKET_HW_RK3576 , };const struct of_device_id rocket_dt_match[] = {
include/uapi/drm/rocket_accel.h +40 -0
@@ -15,11 +15,13 @@ extern "C" {#define DRM_ROCKET_SUBMIT 0x01#define DRM_ROCKET_PREP_BO 0x02#define DRM_ROCKET_FINI_BO 0x03 +#define DRM_ROCKET_GET_PARAM 0x04#define DRM_IOCTL_ROCKET_CREATE_BO DRM_IOWR(DRM_COMMAND_BASE + DRM_ROCKET_CREATE_BO, struct drm_rocket_create_bo)#define DRM_IOCTL_ROCKET_SUBMIT DRM_IOW(DRM_COMMAND_BASE + DRM_ROCKET_SUBMIT, struct drm_rocket_submit)#define DRM_IOCTL_ROCKET_PREP_BO DRM_IOW(DRM_COMMAND_BASE + DRM_ROCKET_PREP_BO, struct drm_rocket_prep_bo)#define DRM_IOCTL_ROCKET_FINI_BO DRM_IOW(DRM_COMMAND_BASE + DRM_ROCKET_FINI_BO, struct drm_rocket_fini_bo) +#define DRM_IOCTL_ROCKET_GET_PARAM DRM_IOWR(DRM_COMMAND_BASE + DRM_ROCKET_GET_PARAM, struct drm_rocket_get_param)/** * struct drm_rocket_create_bo - ioctl argument for creating Rocket BOs.
@@ -185,6 +187,44 @@ struct drm_rocket_submit {__u64 reserved; }; +/** +* enum drm_rocket_param - parameters for DRM_IOCTL_ROCKET_GET_PARAM +* @DRM_ROCKET_PARAM_HW_VERSION: an enum drm_rocket_hw_version value. +*/ +enum drm_rocket_param { +DRM_ROCKET_PARAM_HW_VERSION =0 , +}; + +/** +* enum drm_rocket_hw_version - the NPU, one value per DT compatible +* @DRM_ROCKET_HW_RK3588: "rockchip,rk3588-rknn-core" +* @DRM_ROCKET_HW_RK3576: "rockchip,rk3576-rknn-core" +*/ +enum drm_rocket_hw_version { +DRM_ROCKET_HW_RK3588 =1 , +DRM_ROCKET_HW_RK3576 =2 , +}; + +/** +* struct drm_rocket_get_param - ioctl argument for querying a parameter. +*/ +struct drm_rocket_get_param { +/** +* @param: Input: an enum drm_rocket_param value. +*/ +__u32 param; + +/** +* @reserved: Reserved, must be zero. +*/ +__u32 reserved; + +/** +* @value: Output: the value of the parameter. +*/ +__u64 value; +}; +#if defined(__cplusplus) }#endif
Signature · 1 line
2.43.0
PATCH v17 11/12 arm64: dts: rockchip: add NPU (RKNN) nodes to rk3576
In reply to Jiaxing HuAdd the two NPU cores and their two IOMMUs, disabled by default; boards enable them.
Each core is on its own power domain only: core 0 on PD_NPU0, core 1 on PD_NPU1. Measured with both cores running, the results are correct, and the nputop and npu domains above them are powered through the domain hierarchy.
Label PD_NPU so a board can connect the NPU supply. The IOMMUs use four clocks.
Also set CLK_RKNN_DSU0 to 594 MHz. Mainline sets neither its rate nor the NPU voltage. On the ROCK 4D the DSU runs at 786 MHz by default and the rail boots at 750 mV; with both cores running, each pass of the 5400-row test has 11 to 25 wrong words. At 594 MHz the test is all correct. In the vendor OPP table, the 500 MHz and 600 MHz steps both need 725 mV.
--- arch/arm64/boot/dts/rockchip/rk3576.dtsi | 70 +++++++++++++++++++++++- 1 file changed, 69 insertions(+), 1 deletion(-)
arch/arm64/boot/dts/rockchip/rk3576.dtsi +69 -1
@@ -1131,7 +1131,7 @@ power: power-controller { #address-cells = <1>; #size-cells = <0>; - power-domain@RK3576_PD_NPU { + pd_npu: power-domain@RK3576_PD_NPU { reg = <RK3576_PD_NPU>; #power-domain-cells = <1>; #address-cells = <1>;
@@ -1350,6 +1350,74 @@ power-domain@RK3576_PD_VO1 { }; }; + rknn_core_0: npu@27700000 { + compatible = "rockchip,rk3576-rknn-core"; + reg = <0x0 0x27700000 0x0 0x1000>, + <0x0 0x27701000 0x0 0x1000>, + <0x0 0x27703000 0x0 0x1000>; + reg-names = "pc", "cna", "core"; + interrupts = <GIC_SPI 247 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&cru ACLK_RKNN0>, <&cru HCLK_RKNN_ROOT>, + <&cru CLK_RKNN_DSU0>, <&cru PCLK_NPUTOP_ROOT>, + <&cru ACLK_RKNN_CBUF>, <&cru HCLK_RKNN_CBUF>; + clock-names = "aclk", "hclk", "npu", "pclk", + "aclk_cbuf", "hclk_cbuf"; + assigned-clocks = <&cru CLK_RKNN_DSU0>; + assigned-clock-rates = <594000000>; + resets = <&cru SRST_A_RKNN0>; + reset-names = "srst_a"; + power-domains = <&power RK3576_PD_NPU0>; + iommus = <&rknn_mmu_0>; + status = "disabled"; + }; + + rknn_mmu_0: iommu@27702000 { + compatible = "rockchip,rk3576-npu-iommu", "rockchip,rk3568-iommu"; + reg = <0x0 0x27702000 0x0 0x100>, + <0x0 0x27702100 0x0 0x100>; + interrupts = <GIC_SPI 247 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&cru ACLK_RKNN0>, <&cru HCLK_RKNN_ROOT>, + <&cru ACLK_RKNN_CBUF>, <&cru HCLK_RKNN_CBUF>; + clock-names = "aclk", "iface", "aclk_cbuf", "hclk_cbuf"; + #iommu-cells = <0>; + power-domains = <&power RK3576_PD_NPU0>; + status = "disabled"; + }; + + rknn_core_1: npu@27708000 { + compatible = "rockchip,rk3576-rknn-core"; + reg = <0x0 0x27708000 0x0 0x1000>, + <0x0 0x27709000 0x0 0x1000>, + <0x0 0x2770b000 0x0 0x1000>; + reg-names = "pc", "cna", "core"; + interrupts = <GIC_SPI 248 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&cru ACLK_RKNN1>, <&cru HCLK_RKNN_ROOT>, + <&cru CLK_RKNN_DSU0>, <&cru PCLK_NPUTOP_ROOT>, + <&cru ACLK_RKNN_CBUF>, <&cru HCLK_RKNN_CBUF>; + clock-names = "aclk", "hclk", "npu", "pclk", + "aclk_cbuf", "hclk_cbuf"; + assigned-clocks = <&cru CLK_RKNN_DSU0>; + assigned-clock-rates = <594000000>; + resets = <&cru SRST_A_RKNN1>; + reset-names = "srst_a"; + power-domains = <&power RK3576_PD_NPU1>; + iommus = <&rknn_mmu_1>; + status = "disabled"; + }; + + rknn_mmu_1: iommu@2770a000 { + compatible = "rockchip,rk3576-npu-iommu", "rockchip,rk3568-iommu"; + reg = <0x0 0x2770a000 0x0 0x100>, + <0x0 0x2770a100 0x0 0x100>; + interrupts = <GIC_SPI 248 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&cru ACLK_RKNN1>, <&cru HCLK_RKNN_ROOT>, + <&cru ACLK_RKNN_CBUF>, <&cru HCLK_RKNN_CBUF>; + clock-names = "aclk", "iface", "aclk_cbuf", "hclk_cbuf"; + #iommu-cells = <0>; + power-domains = <&power RK3576_PD_NPU1>; + status = "disabled"; + }; + gpu: gpu@27800000 { compatible = "rockchip,rk3576-mali", "arm,mali-bifrost"; reg = <0x0 0x27800000 0x0 0x20000>;
Signature · 1 line
2.43.0
PATCH v17 12/12 arm64: dts: rockchip: enable the NPU on rk3576-rock-4d
In reply to Jiaxing HuEnable both NPU cores and their IOMMUs in the ROCK 4D device tree.
Make vdd_npu_s0 the supply of the NPU power domain, so the NPU rail is switched on and off with the domain.
--- .../boot/dts/rockchip/rk3576-rock-4d.dts | 22 +++++++++++++++++++ 1 file changed, 22 insertions(+)
arch/arm64/boot/dts/rockchip/rk3576-rock-4d.dts +22 -0
@@ -722,6 +722,10 @@ &pcie0 { status = "okay"; }; +&pd_npu { + domain-supply = <&vdd_npu_s0>; +}; + &pinctrl { hdmi { hdmi_tx_on_h: hdmi-tx-on-h {
@@ -779,6 +783,24 @@ wifi_en_h: wifi-en-h { }; }; +&rknn_core_0 { + npu-supply = <&vdd_npu_s0>; + status = "okay"; +}; + +&rknn_mmu_0 { + status = "okay"; +}; + +&rknn_core_1 { + npu-supply = <&vdd_npu_s0>; + status = "okay"; +}; + +&rknn_mmu_1 { + status = "okay"; +}; + &sai6 { status = "okay"; };
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH v17 06/12] dt-bindings: npu: rockchip: add rockchip,rk3576-rknn-core
In reply to Jiaxing HuThank you for your contribution! Sashiko AI review found 1 potential issue to consider: - [Low] dt-bindings: npu: rockchip: unwieldy per-compatible allOf in rknn-core --
Thank you for your contribution! Sashiko AI review found 1 potential issue to consider:
- [Low] dt-bindings: npu: rockchip: unwieldy per-compatible allOf in rknn-core
--
commit a0f34734ea2ccc04a47837d4a01f9cdb7cf55a23
Author: Jiaxing Hu <[email protected]>
dt-bindings: npu: rockchip: add rockchip,rk3576-rknn-core
This commit adds the rockchip,rk3576-rknn-core compatible string to the Rockchip RKNN core binding and defines per-SoC clock, reset, and regulator constraints in an allOf conditional block.
Quoted 71 lines
Documentation/devicetree/bindings/npu/rockchip,rk3588-rknn-core.yaml +30
-2@@ -21,6 +21,7 @@ properties: compatible: enum: + - rockchip,rk3576-rknn-core - rockchip,rk3588-rknn-core reg:@@ -33,14 +34,18 @@ properties: - const: core # Main NPU core processing unit registers clocks: - maxItems: 4 + minItems: 4 + maxItems: 6 clock-names: + minItems: 4 items: - const: aclk - const: hclk - const: npu - const: pclk + - const: aclk_cbuf + - const: hclk_cbuf interrupts: maxItems: 1@@ -54,9 +59,11 @@ properties: maxItems: 1 resets: + minItems: 1 maxItems: 2 reset-names: + minItems: 1 items: - const: srst_a - const: srst_h@@ -75,7 +82,36 @@ required: - resets - reset-names - npu-supply - - sram-supply + +allOf: + - if: + properties: + compatible: + contains: + const: rockchip,rk3588-rknn-core + then: + properties: + clocks: + maxItems: 4 + clock-names: + maxItems: 4 + resets: + minItems: 2 + reset-names: + minItems: 2 + required: + - sram-supply + else: + properties: + clocks:
Documentation/devicetree/bindings/npu/rockchip,rk3588-rknn-core.yaml required: · line 116
Quoted 10 lines
Documentation/devicetree/bindings/npu/rockchip,rk3588-rknn-core.yaml +8
-0+ minItems: 6 + clock-names: + minItems: 6 + resets: + maxItems: 1 + reset-names: + maxItems: 1 + sram-supply: false additionalProperties: false
[Severity: Low]
This isn't a bug, but since rockchip,rk3576-rknn-core and rockchip,rk3588-rknn-core differ across clocks, clock-names, resets, reset-names, and sram-supply, would it be cleaner to split rockchip,rk3576-rknn-core into its own schema document rather than loosening the top-level properties and overriding all of them in allOf?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
[email protected]
AI reviewRe: [PATCH v17 04/12] accel/rocket: let the core suspend after a reset
In reply to Jiaxing HuThank you for your contribution! Sashiko AI review found 2 potential issues to consider: - [High] accel/rocket: asynchronous autosuspend in rocket_reset() skips domain reset - [Medium] accel/rocket…
Thank you for your contribution! Sashiko AI review found 2 potential issues to consider:
- [High] accel/rocket: asynchronous autosuspend in rocket_reset() skips domain reset
- [Medium] accel/rocket: rocket_core_reset() runs without PM ref if job IRQ races
--
commit 8873affc45494653d715f08563b9727e826503f5
Author: Jiaxing Hu <[email protected]>
accel/rocket: let the core suspend after a reset
This commit replaces pm_runtime_put_noidle() with pm_runtime_put_autosuspend() in rocket_reset() and moves the call after rocket_core_reset() so the NPU core stays powered during the reset and can suspend afterward.
Quoted 28 lines
drivers/accel/rocket/rocket_job.c +4
-3@@ -397,6 +397,8 @@ static void rocket_job_handle_irq(struct rocket_core *core)static void rocket_reset (struct rocket_core *core,struct drm_sched_job *bad) { +bool put_job_ref; +if (!atomic_read (&core->reset.pending))return ;@@ -415,11 +417,10 @@ rocket_reset(struct rocket_core *core, struct drm_sched_job *bad)/* * Remaining interrupts have been handled, but we might still have * stuck jobs. Let's make sure the PM counters stay balanced by -* manually calling pm_runtime_put_noidle(). +* dropping the job's reference once the core has been reset. */ scoped_guard(mutex, &core->job_lock) { -if (core->in_flight_job) -pm_runtime_put_noidle (core->dev); + put_job_ref = core->in_flight_job;iommu_detach_group (NULL , core->iommu_group);
drivers/accel/rocket/rocket_job.c rocket_reset(struct rocket_core *core, struct drm_sched_job *bad) · line 431
Quoted 2 lines
drivers/accel/rocket/rocket_job.c +0
-0@@ -429,6 +430,9 @@ rocket_reset(struct rocket_core *core, struct drm_sched_job *bad)/* Proceed with reset now. */ rocket_core_reset (core);
[Severity: Medium]
As reported by Igor Paunovic on v8 [1], can rocket_core_reset() still run without holding a runtime PM reference if the in-flight job completes concurrently with the timeout?
CPU0 (reset work) CPU1 (threaded IRQ)rocket_reset()
drm_sched_stop()
pm_runtime_get_if_active()
...
pm_runtime_put_autosuspend()
rocket_job_handle_irq()
pm_runtime_put_autosuspend(core->dev)
core->in_flight_job = NULL synchronize_irq(core->irq)
put_job_ref = core->in_flight_job (NULL)
rocket_core_reset(core) <-- no runtime PM ref held
rocket_reset() relies on the in-flight job's runtime PM reference to keep core->dev powered across rocket_core_reset(), because the temporary reference from pm_runtime_get_if_active() is dropped before synchronize_irq(). If rocket_job_handle_irq() finishes the last task and clears core->in_flight_job before synchronize_irq() returns, rocket_core_reset() executes with zero runtime PM references held, which allows core->dev to suspend before or during the reset and hang the board.
drivers/accel/rocket/rocket_job.c rocket_reset(struct rocket_core *core, struct drm_sched_job *bad) · line 434
Quoted 3 lines
drivers/accel/rocket/rocket_job.c +2
-0+if (put_job_ref) +pm_runtime_put_autosuspend (core->dev);
[Severity: High]
Does pm_runtime_put_autosuspend() guarantee that the power domain is switched off and on again before the next job runs?
rocket_core_init() configures a 50 ms autosuspend delay on core->dev:
drivers/accel/rocket/rocket_core.c:rocket_core_init() {
...
pm_runtime_use_autosuspend(dev);
...
pm_runtime_set_autosuspend_delay(dev, 50);
}In rocket_reset(), pm_runtime_put_autosuspend(core->dev) only arms the 50 ms timer and leaves core->dev active, and drm_sched_start(&core->sched,
0) is called immediately afterward. When another job is already queued in the scheduler (or submitted within 50 ms), rocket_job_run() calls pm_runtime_resume_and_get(core->dev) while core->dev is still active, canceling the autosuspend timer without ever suspending the device or cycling the power domain. On RK3576, the NPU bus interface is then not reset and the job after the timeout still fails.
[1] https://lore.kernel.org/r/CAEWPSH7tM9WQUJ0H13ghkiHGzDcdycC1syTuO5B8DpC6cUSnfg@mail.gmail.com
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
[email protected]
AI reviewRe: [PATCH v17 11/12] arm64: dts: rockchip: add NPU (RKNN) nodes to rk3576
In reply to Jiaxing HuThank you for your contribution! Sashiko AI review found 1 potential issue to consider: - [High] arm64: dts: rockchip: missing resets in rk3576 NPU power-domain nodes --
Thank you for your contribution! Sashiko AI review found 1 potential issue to consider:
- [High] arm64: dts: rockchip: missing resets in rk3576 NPU power-domain nodes
--
commit 4129f134d43a1e4e19c937117eb7565caa3c233d
Author: Jiaxing Hu <[email protected]>
arm64: dts: rockchip: add NPU (RKNN) nodes to rk3576
This commit adds the two RK3576 NPU core nodes and their IOMMU nodes to rk3576.dtsi, and labels the NPU power domain node.
Quoted 27 lines
arch/arm64/boot/dts/rockchip/rk3576.dtsi +11
-1@@ -1131,7 +1131,7 @@ power: power-controller { #address-cells = <1>; #size-cells = <0>; - power-domain@RK3576_PD_NPU { + pd_npu: power-domain@RK3576_PD_NPU { reg = <RK3576_PD_NPU>; #power-domain-cells = <1>; #address-cells = <1>;@@ -1350,6 +1350,74 @@ power-domain@RK3576_PD_VO1 { }; }; + rknn_core_0: npu@27700000 { + compatible = "rockchip,rk3576-rknn-core"; + reg = <0x0 0x27700000 0x0 0x1000>, + <0x0 0x27701000 0x0 0x1000>, + <0x0 0x27703000 0x0 0x1000>; + reg-names = "pc", "cna", "core"; + interrupts = <GIC_SPI 247 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&cru ACLK_RKNN0>, <&cru HCLK_RKNN_ROOT>, + <&cru CLK_RKNN_DSU0>, <&cru PCLK_NPUTOP_ROOT>, + <&cru ACLK_RKNN_CBUF>, <&cru HCLK_RKNN_CBUF>;
arch/arm64/boot/dts/rockchip/rk3576.dtsi power-domain@RK3576_PD_VO1 { · line 1372
Quoted 10 lines
arch/arm64/boot/dts/rockchip/rk3576.dtsi +10
-0+ clock-names = "aclk", "hclk", "npu", "pclk", + "aclk_cbuf", "hclk_cbuf"; + assigned-clocks = <&cru CLK_RKNN_DSU0>; + assigned-clock-rates = <594000000>; + resets = <&cru SRST_A_RKNN0>; + reset-names = "srst_a"; + power-domains = <&power RK3576_PD_NPU0>; + iommus = <&rknn_mmu_0>; + status = "disabled"; + };
[Severity: High]
Are the power-domain@RK3576_PD_NPU0 and power-domain@RK3576_PD_NPU1 nodes missing their resets properties (such as SRST_A_RKNN0_BIU and SRST_A_RKNN1_BIU)?
40f898ca63cc ("pmdomain: rockchip: cycle an optional power-domain reset on power-on") and commit "accel/rocket: let the core suspend after a reset" rely on rockchip_pd_power() pulsing pd->reset when powering on the RK3576 NPU domains, both to clear the undefined bus interface state before QoS and register accesses and to recover the IOMMU after a job timeout reset. However, none of the RK3576 NPU power-domain nodes in rk3576.dtsi specify a resets property, so pd->reset remains NULL and the bus interface reset is never cycled:
arch/arm64/boot/dts/rockchip/rk3576.dtsi:
power-domain@RK3576_PD_NPU0 {
reg = <RK3576_PD_NPU0>;
clocks = <&cru HCLK_RKNN_ROOT>,
<&cru ACLK_RKNN0>;
pm_qos = <&qos_npu_m0>;
#power-domain-cells = <0>;
};
power-domain@RK3576_PD_NPU1 {
reg = <RK3576_PD_NPU1>;
clocks = <&cru HCLK_RKNN_ROOT>,
<&cru ACLK_RKNN1>;
pm_qos = <&qos_npu_m1>;
#power-domain-cells = <0>;
};Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=11
[email protected]
AI reviewRe: [PATCH v17 03/12] accel/rocket: wait for a running IRQ handler before resetting a core
In reply to Jiaxing HuThank you for your contribution! Sashiko AI review found 3 potential issues to consider: - [High] accel/rocket: raw IRQ status can re-latch before reset in rocket_reset() - [High] accel/rocket: race…
Thank you for your contribution! Sashiko AI review found 3 potential issues to consider:
- [High] accel/rocket: raw IRQ status can re-latch before reset in rocket_reset()
- [High] accel/rocket: race with pm_runtime_force_suspend() in rocket_reset()
- [Low] accel/rocket: missing Fixes trailer for IRQ race fix in rocket_reset()
--
commit 9e5305abf9afea9e1a362f158ce4e7b07f213cce
Author: Jiaxing Hu <[email protected]>
Quoted 12 lines
accel/rocket: wait for a running IRQ handler before resetting a core
drm_sched_stop()does not wait for a threaded IRQ handler that is already running, so callsynchronize_irq()after it to make sure the handler has finished before the reset.Before
synchronize_irq(), mask the interrupt and clear the raw status, so that the NPU core cannot report a completion afterwards.Do this under
job_lock, because the submit path sets the interrupt mask under the same lock.
Quoted 7 lines
Only touch these registers when the device is powered: check with
pm_runtime_get_if_active(), so that a register access with the power domain off cannot raise an SError.
[Severity: Low]
This fixes the IRQ race in rocket_reset() introduced with job submission. Should this include:
Quoted 5 lines
drivers/accel/rocket/rocket_job.c +0
-0
drivers/accel/rocket/rocket_job.c rocket_reset(struct rocket_core *core, struct drm_sched_job *bad) · line 411
Quoted 10 lines
drivers/accel/rocket/rocket_job.c +7
-0@@ -402,6 +402,16 @@ rocket_reset(struct rocket_core *core, struct drm_sched_job *bad)drm_sched_stop (&core->sched, bad); + scoped_guard(mutex, &core->job_lock) { +if (pm_runtime_get_if_active (core->dev) >0 ) { +rocket_pc_writel (core,INTERRUPT_MASK ,0x0 ); +rocket_pc_writel (core,INTERRUPT_CLEAR ,0x1ffff ); +pm_runtime_put_autosuspend (core->dev); + } + }
[Severity: High]
Can pm_runtime_force_suspend() disable the core clocks while rocket_reset() writes INTERRUPT_MASK and INTERRUPT_CLEAR?
core->reset.wq is not WQ_FREEZABLE, and drm_sched_stop() drops sched.credit_count to 0, making rocket_job_is_idle() return true while rocket_reset() is still running. Because pm_runtime_force_suspend() ignores dev->power.usage_count and rocket_device_runtime_suspend() does not take core->job_lock, system suspend can disable the clocks concurrently:
kworker (core->reset.wq) system suspendrocket_reset()
drm_sched_stop()
credit_count -> 0
mutex_lock(&core->job_lock)
pm_runtime_get_if_active() > 0
pm_runtime_force_suspend()
rocket_device_runtime_suspend()
rocket_job_is_idle() -> true
clk_bulk_disable_unprepare() rocket_pc_writel(...)
<-- unclocked MMIO access (SError / bus hang)
In addition, in rocket_core_fini(), an in-flight job holds a runtime PM reference that keeps the core active across pm_runtime_disable() before rocket_job_fini() cancels the reset worker. pm_runtime_get_if_active() then returns -EINVAL and skips masking and clearing interrupts on the active core before synchronize_irq().
drivers/accel/rocket/rocket_job.c rocket_reset(struct rocket_core *core, struct drm_sched_job *bad) · line 413
Quoted 2 lines
drivers/accel/rocket/rocket_job.c +2
-0+ +synchronize_irq (core->irq);
[Severity: High]
Can an active DPU task complete after INTERRUPT_CLEAR is written and re-latch INTERRUPT_RAW_STATUS before rocket_core_reset() stops the hardware?
Unlike rocket_job_handle_irq(), which writes 0x0 to OPERATION_ENABLE before writing INTERRUPT_CLEAR, rocket_reset() leaves the core running across synchronize_irq() and iommu_detach_group() until rocket_core_reset() asserts the reset lines. And because rocket_job_irq_handler() checks the unmasked INTERRUPT_RAW_STATUS register on a shared IRQ line (shared with the IOMMU):
drivers/accel/rocket/rocket_job.c:rocket_job_irq_handler() {
u32 raw_status = rocket_pc_readl(core, INTERRUPT_RAW_STATUS);
...
if (!(raw_status & PC_INTERRUPT_RAW_STATUS_DPU_0 ||
raw_status & PC_INTERRUPT_RAW_STATUS_DPU_1))
return IRQ_NONE;
rocket_pc_writel(core, INTERRUPT_MASK, 0x0);
return IRQ_WAKE_THREAD;
}a task finishing in that window sets PC_INTERRUPT_RAW_STATUS_DPU_0/1 again, and a shared IOMMU interrupt on core->irq can wake rocket_job_irq_handler_thread() after synchronize_irq() has already returned:
kworker (core->reset.wq) <hardirq> / irq threadrocket_reset()
rocket_pc_writel(INTERRUPT_MASK, 0x0) rocket_pc_writel(INTERRUPT_CLEAR, 0x1ffff)
[DPU finishes, re-latches INTERRUPT_RAW_STATUS] synchronize_irq(core->irq)
[shared IOMMU IRQ fires] rocket_job_irq_handler()
reads INTERRUPT_RAW_STATUS returns IRQ_WAKE_THREAD
iommu_detach_group()
rocket_core_reset()
rocket_job_irq_handler_thread()
rocket_job_handle_irq() <-- races with reset
This also detaches the IOMMU group in iommu_detach_group() while OPERATION_ENABLE is still set and the NPU has not yet been reset. Clearing OPERATION_ENABLE before INTERRUPT_CLEAR (or reading the masked INTERRUPT_STATUS register in rocket_job_irq_handler()) would prevent the raw status from re-triggering the IRQ thread.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
Quoted 2 lines · Jiaxing Hu
This series adds RK3576 NPU (RKNN) support to the mainline DRM rocket driver.
I checked the accel/rocket patches for regressions on a ROCK 5B+ (RK3588): MobileNet inference with up to 6 concurrent clients, runtime PM cycling, core unbind/rebind, GET_PARAM (returns 1) and recovery from 93 forced job timeouts all pass, with no warnings.
For patches 2-5 and 8-10:
Thanks,
Sidong
Re: [PATCH v17 03/12] accel/rocket: wait for a running IRQ handler before resetting a core
In reply to Jiaxing HuAm Samstag, 14. November 2026, 04:05:20 Mitteleuropäische Normalzeit schrieb Jiaxing Hu:
Quoted 10 lines
drm_sched_stop()does not wait for a threaded IRQ handler that is already running, so callsynchronize_irq()after it to make sure the handler has finished before the reset.Before
synchronize_irq(), mask the interrupt and clear the raw status, so that the NPU core cannot report a completion afterwards.Do this under
job_lock, because the submit path sets the interrupt mask under the same lock.
Quoted 7 lines
Only touch these registers when the device is powered: check with
pm_runtime_get_if_active(), so that a register access with the power domain off cannot raise an SError.