Add support for Wave6 video codec driver
Subsystems (3)
v8 cover Add support for Wave6 video codec driver
This patch series introduces support for the Chips&Media Wave6 video codec IP, a completely different hardware architecture compared to Wave5.
The wave6 driver is a M2M stateful encoder/decoder driver. It supports various video formats, including H.264 and H.265, for both encoding and decoding.
While other versions of the Wave6 IP may support VP9 decoding and AV1 decoding and encoding those formats are not implemented or validated in this driver at this time.
On NXP i.MX SoCs, the Wave6 hardware exposes one control register region and four interface register regions for one shared video processing engine.
VPU Control region, Manages shared resources such as firmware memory. VPU Core region, Provides encoding and decoding capabilities. The control and interface regions are distinct DMA requesters and can be associated with separate IOMMU stream IDs, allowing DMA isolation between them.
The interface regions are described as child nodes of the VPU node rather than folded into the parent with reg-names/interrupt-names, because each one is a separate DMA requester. This follows the same structure already accepted for the Qualcomm video codec bindings, and was mentioned on the v6 thread.
The firmware tested by this driver has been upstreamed in linux-firmware:
- Path: cnm/wave633c_imx9_codec_fw.bin
This driver has been tested with GStreamer on:
- NXP i.MX95 board
- pre-silicon FPGA environment
Test results for decoder fluster with -j2 option:
- JVT-AVC_V1, Ran 77/135 tests successfully in 24.180 secs - JVT-FR-EXT, Ran 25/69 tests successfully in 11.157 secs - JCT-VC-HEVC_V1, Ran 132/147 tests successfully in 45.534 secs
- All failures are due to unsupported hardware features:
-- 10bit, Resolutions higher than 4K, FMO, MBAFF -- Extended profile, Field encoding and High422 sreams.
Test results for v4l2-compliance:
v4l2-compliance 1.31.0-5386, 64 bits, 64-bit time_t v4l2-compliance SHA: 48316b8a20aa 2025-08-12 12:44:56
Compliance test for wave6-dec device /dev/video0:
fail: v4l2-test-controls.cpp(1204): !have_source_change || !have_eos
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
Total for wave6-dec device /dev/video0: 48, Succeeded: 47, Failed: 1, Warnings: 0
Compliance test for wave6-enc device /dev/video1:
fail: v4l2-test-controls.cpp(1193): node->codec_mask & STATEFUL_ENCODER
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
Total for wave6-enc device /dev/video1: 48, Succeeded: 47, Failed: 1, Warnings: 0
Note: the failures are all related with the eos event.
Changelog:
v8:
- Dropped "media: v4l2-common: Fix P010 format info", now applied upstream
- Split the core, decoder and encoder support into smaller patches so that every patch builds on its own
- Moved the performance profiling code into its own patch
- Built the control and core drivers into one wave6 module and replaced the function pointers between them with direct calls
- Reworked the decoder and encoder drain to follow the stateful codec interface, using the v4l2-mem2mem helpers in the encoder
- Replaced the instance STOP state with an ERROR state set on command failures
- Made the job timeout watchdog per instance
- Fixed runtime PM, probe, teardown and cooling device issues
- Range-checked the frame buffer registration counts
- Set DEFAULT_NUM_TICKS_POC_DIFF to 1 to align with the codec specification
- Added the imx list to MAINTAINERS and updated the copyright year
- Cleaned up the code: helper macros, relaxed register accessors, devm helpers, unused codes and includes, coding style
v7:
- Moved the sram property from imx95-19x19-evk.dts to imx95.dtsi
- Simplified the sram binding property to maxItems: 1
- Fixed the issues reported by the sashiko-bot review of v6:
- Bounded frame buffer and aux buffer counts before indexing
- Added vb2_is_busy() checks to the decoder S_FMT handlers
- Fixed video_device lifetime and driver teardown ordering
- Propagated hardware init, runtime PM and thermal failures upward
- Fixed tracepoint field widths and debugfs handling
- Balanced the VPU reference on the core activation error path
v6:
- Fixed the existing P010 format_info entry
- Moved all encoder parameter validation from the HW layer into the V4L2 layer
- Removed redundant HW-abstraction wrappers from the VPU API layer
- Computed frame stride and sizeimage in the V4L2 layer
- Added explicit instance lifecycle helpers to the core driver
- Enabled the VPU only on the tested imx95-19x19-evk board
RFC v5:
- Move all shared resources to the parent node
- Drop child compatible and use data-only interface child nodes
- Update the VPU driver to create child devices and load the core driver
v4:
- Fixed build issues reported by CI tools
- Updated commit messages to use imperative mood
- Avoided using the same name for both nodes and labels in devicetree
- Removed unused labels from YAML examples
- Added description for child(vpu-core) node
- Added iommus property to both parent(vpu) and child(vpu-core) nodes
- Updated probe() functions to use dev_err_probe() when returning -EPROBE_DEFER
- Added wave6_vpu prefix to trace functions
- Updated HEVC decoder profile control to report MAIN_STILL profile
- Fixed bug in multiple instance creation by pre-allocating work buffer
- Fixed interrupt handling by checking INSTANCE_INFO register and instance list
v3:
- Removed ambiguous SUPPORT_FOLLOWER feature
- Used WARN_ON() for unexpected programming errors
- Split thermal device code into wave6-vpu-thermal.c/h
- Dropped wave6_cooling_disable module parameter
- Replaced mutex_lock() with guard()
- Added lockdep_assert_held() to clarify locking regions
- Removed exported function due to dual-license and used function pointer
- Added documentation and validation for state transitions
- Added documentation for device structures
- Added patch to enable VPU device in imx95 DTS
- Updated DT bindings and driver to align with parent(vpu) and child(vpu-core)
- Replaced magic numbers with mask and offset macros when accessing registers
- Placed goto statements after an empty line
- Printed HW info (e.g. product_code) via dev_dbg() for debugging
- Replaced wave6_vpu_dec_give_command() with dedicated functions
v2:
- Refined DT bindings to better represent the hardware
- Reworked driver to align with the parent(VPU) and child(CTRL, CORE)
- Fixed build issues reported by CI tools (Smatch, Sparse, TRACE)
- Improved commit messages with clearer descriptions
- Added kernel-doc for exported functions
- Removed redundant print statements and unused code
- Reordered patches to prevent build failures
Nas Chung (11):
dt-bindings: media: nxp: Add Wave6 video codec device
media: chips-media: wave6: Add Wave6 control driver
media: chips-media: wave6: Add Wave6 core driver
media: chips-media: wave6: Add Wave6 decoder firmware interface
media: chips-media: wave6: Add Wave6 m2m decoder
media: chips-media: wave6: Add Wave6 encoder firmware interface
media: chips-media: wave6: Add Wave6 m2m encoder
media: chips-media: wave6: Add trace and debugfs
media: chips-media: wave6: Add performance profiling
media: chips-media: wave6: Add Wave6 thermal cooling device
arm64: dts: freescale: imx95: Add video codec node
.../bindings/media/nxp,imx95-vpu.yaml | 162 + MAINTAINERS | 9 + .../boot/dts/freescale/imx95-19x19-evk.dts | 10 + arch/arm64/boot/dts/freescale/imx95.dtsi | 37 + drivers/media/platform/chips-media/Kconfig | 1 + drivers/media/platform/chips-media/Makefile | 1 + .../media/platform/chips-media/wave6/Kconfig | 16 + .../media/platform/chips-media/wave6/Makefile | 15 + .../platform/chips-media/wave6/wave6-hw.c | 1934 ++++++++++++ .../platform/chips-media/wave6/wave6-hw.h | 58 + .../chips-media/wave6/wave6-regdefine.h | 651 ++++ .../platform/chips-media/wave6/wave6-trace.h | 284 ++ .../platform/chips-media/wave6/wave6-vdi.h | 102 + .../chips-media/wave6/wave6-vpu-core.c | 446 +++ .../chips-media/wave6/wave6-vpu-core.h | 130 + .../chips-media/wave6/wave6-vpu-dbg.c | 184 ++ .../chips-media/wave6/wave6-vpu-dbg.h | 14 + .../chips-media/wave6/wave6-vpu-dec.c | 1969 ++++++++++++ .../chips-media/wave6/wave6-vpu-enc.c | 2762 +++++++++++++++++ .../chips-media/wave6/wave6-vpu-thermal.c | 141 + .../chips-media/wave6/wave6-vpu-thermal.h | 24 + .../chips-media/wave6/wave6-vpu-v4l2.c | 500 +++ .../platform/chips-media/wave6/wave6-vpu.c | 824 +++++ .../platform/chips-media/wave6/wave6-vpu.h | 142 + .../platform/chips-media/wave6/wave6-vpuapi.c | 333 ++ .../platform/chips-media/wave6/wave6-vpuapi.h | 1014 ++++++ .../chips-media/wave6/wave6-vpuconfig.h | 72 + .../chips-media/wave6/wave6-vpuerror.h | 252 ++ 28 files changed, 12087 insertions(+) create mode 100644 Documentation/devicetree/bindings/media/nxp,imx95-vpu.yaml create mode 100644 drivers/media/platform/chips-media/wave6/Kconfig create mode 100644 drivers/media/platform/chips-media/wave6/Makefile create mode 100644 drivers/media/platform/chips-media/wave6/wave6-hw.c create mode 100644 drivers/media/platform/chips-media/wave6/wave6-hw.h create mode 100644 drivers/media/platform/chips-media/wave6/wave6-regdefine.h create mode 100644 drivers/media/platform/chips-media/wave6/wave6-trace.h create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vdi.h create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-core.c create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-core.h create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.c create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.h create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.c create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.h create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu.c create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu.h create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpuapi.c create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpuapi.h create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpuerror.h
base-commit: aa0f4c41d4870a18e65772a1817af16427200371
Signature · 1 line
2.41.0.windows.3
PATCH v8 1/11 dt-bindings: media: nxp: Add Wave6 video codec device
Add documentation for the Chips&Media Wave6 video codec on NXP i.MX SoCs.
The hardware contains one control register region and four interface register regions for a shared video processing engine. The control region manages shared resources such as firmware memory, while each interface region has its own MMIO range and interrupt.
The control region and each interface region are distinct DMA requesters and can be associated with separate IOMMU stream IDs. Represent the control region as the parent node and the interface register regions as child nodes to describe these resources.
--- .../bindings/media/nxp,imx95-vpu.yaml | 162 ++++++++++++++++++ MAINTAINERS | 8 + 2 files changed, 170 insertions(+) create mode 100644 Documentation/devicetree/bindings/media/nxp,imx95-vpu.yaml
Documentation/devicetree/bindings/media/nxp,imx95-vpu.yaml new file +162 -0
@@ -0,0 +1,162 @@ +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/media/nxp,imx95-vpu.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Chips&Media Wave6 Series multi-standard codec IP on NXP i.MX SoCs + +maintainers: + - Nas Chung <[email protected]> + - Jackson Lee <[email protected]> + +description: + The Chips&Media Wave6 codec IP is a multi-standard video encoder/decoder. + On NXP i.MX SoCs, the Wave6 codec IP exposes one control register region and + four interface register regions for a shared video processing engine. + The parent node describes the control region, which has its own MMIO range and + manages shared resources such as firmware memory. The child nodes describe the + interface register regions. Each interface region has its own MMIO range and + interrupt. + The control region and the interface regions are distinct DMA requesters. + The control region and each interface region can be associated with separate + IOMMU stream IDs, allowing DMA isolation between them. + +properties: + compatible: + enum: + - nxp,imx95-vpu + + reg: + maxItems: 1 + + clocks: + items: + - description: VPU core clock + - description: VPU associated block clock + + clock-names: + items: + - const: core + - const: vpublk + + power-domains: + items: + - description: Main VPU power domain + - description: Performance power domain + + power-domain-names: + items: + - const: vpu + - const: perf + + memory-region: + maxItems: 1 + + sram: + maxItems: 1 + description: + Used to store reference data, reducing DMA memory bandwidth. + + iommus: + maxItems: 1 + + "#cooling-cells": + const: 2 + + "#address-cells": + const: 2 + + "#size-cells": + const: 2 + + ranges: true + +patternProperties: + "^interface@[0-9a-f]+$": + type: object + description: + An interface register region within the Chips&Media Wave6 codec IP. + Each region has its own MMIO range and interrupt and can be associated + with a separate IOMMU stream ID for DMA isolation. + additionalProperties: false + + properties: + reg: + maxItems: 1 + + interrupts: + maxItems: 1 + + iommus: + maxItems: 1 + + required: + - reg + - interrupts + +required: + - compatible + - reg + - clocks + - clock-names + - power-domains + - power-domain-names + - memory-region + - "#address-cells" + - "#size-cells" + - ranges + +additionalProperties: false + +examples: + - | + #include <dt-bindings/interrupt-controller/arm-gic.h> + #include <dt-bindings/clock/nxp,imx95-clock.h> + + soc { + #address-cells = <2>; + #size-cells = <2>; + + video-codec@4c4c0000 { + compatible = "nxp,imx95-vpu"; + reg = <0x0 0x4c4c0000 0x0 0x10000>; + clocks = <&scmi_clk 115>, + <&vpu_blk_ctrl IMX95_CLK_VPUBLK_WAVE>; + clock-names = "core", "vpublk"; + power-domains = <&scmi_devpd 21>, + <&scmi_perf 10>; + power-domain-names = "vpu", "perf"; + memory-region = <&vpu_boot>; + sram = <&sram1>; + iommus = <&smmu 0x32>; + #cooling-cells = <2>; + #address-cells = <2>; + #size-cells = <2>; + ranges; + + interface@4c480000 { + reg = <0x0 0x4c480000 0x0 0x10000>; + interrupts = <GIC_SPI 299 IRQ_TYPE_LEVEL_HIGH>; + iommus = <&smmu 0x33>; + }; + + interface@4c490000 { + reg = <0x0 0x4c490000 0x0 0x10000>; + interrupts = <GIC_SPI 300 IRQ_TYPE_LEVEL_HIGH>; + iommus = <&smmu 0x34>; + }; + + interface@4c4a0000 { + reg = <0x0 0x4c4a0000 0x0 0x10000>; + interrupts = <GIC_SPI 301 IRQ_TYPE_LEVEL_HIGH>; + iommus = <&smmu 0x35>; + }; + + interface@4c4b0000 { + reg = <0x0 0x4c4b0000 0x0 0x10000>; + interrupts = <GIC_SPI 302 IRQ_TYPE_LEVEL_HIGH>; + iommus = <&smmu 0x36>; + }; + }; + };
MAINTAINERS +8 -0
@@ -30125,6 +30125,14 @@ S: Maintained F: Documentation/devicetree/bindings/media/cnm,wave521c.yaml F: drivers/media/platform/chips-media/wave5/ +WAVE6 VPU CODEC DRIVER +M: Nas Chung <[email protected]> +M: Jackson Lee <[email protected]> +L: [email protected] +L: [email protected] +S: Maintained +F: Documentation/devicetree/bindings/media/nxp,imx95-vpu.yaml + WHISKEYCOVE PMIC GPIO DRIVER M: Kuppuswamy Sathyanarayanan <[email protected]> L: [email protected]
Signature · 1 line
2.41.0.windows.3
PATCH v8 2/11 media: chips-media: wave6: Add Wave6 control driver
In reply to Nas ChungAdd the control driver for the Chips&Media Wave6 video codec IP.
The hardware contains one control register region and four interface register regions for a shared video processing engine.
This driver handles the control region and manages shared resources such as firmware loading, firmware memory allocation, and coordination required by the interface register regions.
It also instantiates and coordinates with the wave6-core child devices for firmware and power state management.
--- MAINTAINERS | 1 + drivers/media/platform/chips-media/Kconfig | 1 + drivers/media/platform/chips-media/Makefile | 1 + .../media/platform/chips-media/wave6/Kconfig | 13 + .../media/platform/chips-media/wave6/Makefile | 4 + .../chips-media/wave6/wave6-regdefine.h | 49 ++ .../platform/chips-media/wave6/wave6-vdi.h | 82 ++ .../platform/chips-media/wave6/wave6-vpu.c | 780 ++++++++++++++++++ .../platform/chips-media/wave6/wave6-vpu.h | 137 +++ .../chips-media/wave6/wave6-vpuconfig.h | 37 + 10 files changed, 1105 insertions(+) create mode 100644 drivers/media/platform/chips-media/wave6/Kconfig create mode 100644 drivers/media/platform/chips-media/wave6/Makefile create mode 100644 drivers/media/platform/chips-media/wave6/wave6-regdefine.h create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vdi.h create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu.c create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu.h create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h
MAINTAINERS +1 -0
@@ -30132,6 +30132,7 @@ L: [email protected] L: [email protected] S: Maintained F: Documentation/devicetree/bindings/media/nxp,imx95-vpu.yaml +F: drivers/media/platform/chips-media/wave6/ WHISKEYCOVE PMIC GPIO DRIVER M: Kuppuswamy Sathyanarayanan <[email protected]>
drivers/media/platform/chips-media/Kconfig +1 -0
@@ -4,3 +4,4 @@ comment "Chips&Media media platform drivers" source "drivers/media/platform/chips-media/coda/Kconfig" source "drivers/media/platform/chips-media/wave5/Kconfig" +source "drivers/media/platform/chips-media/wave6/Kconfig"
drivers/media/platform/chips-media/Makefile +1 -0
@@ -2,3 +2,4 @@ obj-y += coda/ obj-y += wave5/ +obj-y += wave6/
drivers/media/platform/chips-media/wave6/Kconfig new file +13 -0
@@ -0,0 +1,13 @@ +# SPDX-License-Identifier: GPL-2.0 + +config VIDEO_WAVE6_VPU + tristate "Chips&Media Wave6 Codec Driver" + depends on V4L_MEM2MEM_DRIVERS + depends on VIDEO_DEV && OF + depends on ARCH_MXC || COMPILE_TEST + select GENERIC_ALLOCATOR + help + Chips&Media Wave6 stateful codec driver. + The wave6 driver manages shared resources such as firmware memory. + To compile this driver as a module, choose M here: the + module will be called wave6.
drivers/media/platform/chips-media/wave6/Makefile new file +4 -0
@@ -0,0 +1,4 @@ +# SPDX-License-Identifier: GPL-2.0 + +wave6-objs += wave6-vpu.o +obj-$(CONFIG_VIDEO_WAVE6_VPU) += wave6.o
drivers/media/platform/chips-media/wave6/wave6-regdefine.h new file +49 -0
49 lines not on this page · raw message
drivers/media/platform/chips-media/wave6/wave6-vdi.h new file +82 -0
82 lines not on this page · raw message
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +780 -0
780 lines not on this page · raw message
drivers/media/platform/chips-media/wave6/wave6-vpu.h new file +137 -0
137 lines not on this page · raw message
drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h new file +37 -0
@@ -0,0 +1,37 @@ +/* SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) */ +/* +* Wave6 series multi-standard codec IP - product config definitions +* +* Copyright (C) 2026 CHIPS&MEDIA INC +*/ + +#ifndef __WAVE6_VPUCONFIG_H__ +#define __WAVE6_VPUCONFIG_H__ + +#define WAVE617_CODE 0x6170 +#define WAVE627_CODE 0x6270 +#define WAVE633_CODE 0x6330 +#define WAVE637_CODE 0x6370 +#define WAVE663_CODE 0x6630 +#define WAVE677_CODE 0x6670 + +#define W6_VPU_MAX_NUM_CORE 4 +#define MAX_NUM_INSTANCE 32 + +#define W6_VPU_POLL_DELAY_US 10 +#define W6_VPU_POLL_TIMEOUT 300000 + +#define W6_MAX_REMAP_PAGE_SIZE (1024 * 1024) +#define W6_MAX_CODE_BUF_SIZE (4 * 1024 * 1024) +#define W6_CODE_BUF_SIZE (1 * 1024 * 1024) +#define W6_EXTRA_CODE_BUF_SIZE (256 * 1024) +#define W637DEC_WORKBUF_SIZE_FOR_CQ (3 * 512 * 1024) + +static inline bool wave6_is_product_w_series (u32 code) +{ +return code ==WAVE617_CODE || code ==WAVE627_CODE || + code ==WAVE633_CODE || code ==WAVE637_CODE || + code ==WAVE663_CODE || code ==WAVE677_CODE ; +} + +#endif /* __WAVE6_VPUCONFIG_H__ */
Signature · 1 line
2.41.0.windows.3
PATCH v8 3/11 media: chips-media: wave6: Add Wave6 core driver
In reply to Nas ChungAdd the core driver for the Chips&Media Wave6 video codec IP.
The hardware contains one control register region and four interface register regions for a shared video processing engine. The core driver handles the interface register regions, each with its own MMIO range and interrupt, while relying on the control driver for firmware loading and shared resource management.
It configures the V4L2 mem2mem devices and communicates with the Wave6 hardware to perform video processing tasks.
--- .../media/platform/chips-media/wave6/Kconfig | 2 + .../media/platform/chips-media/wave6/Makefile | 5 +- .../platform/chips-media/wave6/wave6-hw.c | 181 ++++++++ .../platform/chips-media/wave6/wave6-hw.h | 22 + .../chips-media/wave6/wave6-regdefine.h | 65 +++ .../platform/chips-media/wave6/wave6-vdi.h | 20 + .../chips-media/wave6/wave6-vpu-core.c | 392 ++++++++++++++++++ .../chips-media/wave6/wave6-vpu-core.h | 106 +++++ .../chips-media/wave6/wave6-vpu-v4l2.c | 384 +++++++++++++++++ .../platform/chips-media/wave6/wave6-vpu.c | 24 +- .../platform/chips-media/wave6/wave6-vpu.h | 2 + .../platform/chips-media/wave6/wave6-vpuapi.h | 381 +++++++++++++++++ .../chips-media/wave6/wave6-vpuconfig.h | 6 + 13 files changed, 1588 insertions(+), 2 deletions(-) create mode 100644 drivers/media/platform/chips-media/wave6/wave6-hw.c create mode 100644 drivers/media/platform/chips-media/wave6/wave6-hw.h create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-core.c create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-core.h create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpuapi.h
drivers/media/platform/chips-media/wave6/Kconfig +2 -0
@@ -5,6 +5,8 @@ config VIDEO_WAVE6_VPU depends on V4L_MEM2MEM_DRIVERS depends on VIDEO_DEV && OF depends on ARCH_MXC || COMPILE_TEST + select VIDEOBUF2_DMA_CONTIG + select V4L2_MEM2MEM_DEV select GENERIC_ALLOCATOR help Chips&Media Wave6 stateful codec driver.
drivers/media/platform/chips-media/wave6/Makefile +4 -1
@@ -1,4 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 -wave6-objs += wave6-vpu.o +wave6-objs += wave6-vpu.o \ + wave6-vpu-core.o \ + wave6-vpu-v4l2.o \ + wave6-hw.o obj-$(CONFIG_VIDEO_WAVE6_VPU) += wave6.o
drivers/media/platform/chips-media/wave6/wave6-hw.c new file +181 -0
181 lines not on this page · raw message
drivers/media/platform/chips-media/wave6/wave6-hw.h new file +22 -0
@@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) */ +/* +* Wave6 series multi-standard codec IP - wave6 backend interface +* +* Copyright (C) 2026 CHIPS&MEDIA INC +*/ + +#ifndef __WAVE6_HW_H__ +#define __WAVE6_HW_H__ + +enum product_id { +PRODUCT_ID_617 , +PRODUCT_ID_627 , +PRODUCT_ID_637 , +PRODUCT_ID_NONE , +}; + +bool wave6_vpu_is_init (struct vpu_core_device *core); +int wave6_vpu_get_version (struct vpu_core_device *core); +void wave6_vpu_enable_interrupt (struct vpu_core_device *core); + +#endif /* __WAVE6_HW_H__ */
drivers/media/platform/chips-media/wave6/wave6-regdefine.h +65 -0
90 lines not on this page · raw message
drivers/media/platform/chips-media/wave6/wave6-vdi.h +20 -0
@@ -13,6 +13,26 @@#include <linux/dma-mapping.h> #include <linux/io.h> +enum endian_mode { +VDI_128BIT_BIG_ENDIAN =0 , +VDI_128BIT_BE_BYTE_SWAP , +VDI_128BIT_BE_WORD_SWAP , +VDI_128BIT_BE_WORD_BYTE_SWAP , +VDI_128BIT_BE_DWORD_SWAP , +VDI_128BIT_BE_DWORD_BYTE_SWAP , +VDI_128BIT_BE_DWORD_WORD_SWAP , +VDI_128BIT_BE_DWORD_WORD_BYTE_SWAP , +VDI_128BIT_LE_DWORD_WORD_BYTE_SWAP , +VDI_128BIT_LE_DWORD_WORD_SWAP , +VDI_128BIT_LE_DWORD_BYTE_SWAP , +VDI_128BIT_LE_DWORD_SWAP , +VDI_128BIT_LE_WORD_BYTE_SWAP , +VDI_128BIT_LE_WORD_SWAP , +VDI_128BIT_LE_BYTE_SWAP , +VDI_128BIT_LITTLE_ENDIAN =15 , +VDI_ENDIAN_MAX +}; +/** * struct vpu_buf - VPU buffer for a coherent DMA buffer * @size: Buffer size
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c new file +392 -0
392 lines not on this page · raw message
drivers/media/platform/chips-media/wave6/wave6-vpu-core.h new file +106 -0
106 lines not on this page · raw message
drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c new file +384 -0
384 lines not on this page · raw message
drivers/media/platform/chips-media/wave6/wave6-vpu.c +23 -1
@@ -773,7 +773,29 @@ static struct platform_driver wave6_vpu_driver = { .remove = wave6_vpu_remove, }; -module_platform_driver (wave6_vpu_driver); +static int __init wave6_vpu_init (void ) +{ +int ret; + + ret =wave6_vpu_core_register_driver (); +if (ret) +return ret; + + ret =platform_driver_register (&wave6_vpu_driver); +if (ret) +wave6_vpu_core_unregister_driver (); + +return ret; +} +module_init (wave6_vpu_init); + +static void __exit wave6_vpu_exit(void) +{ +platform_driver_unregister (&wave6_vpu_driver); +wave6_vpu_core_unregister_driver (); +} +module_exit (wave6_vpu_exit); +MODULE_DESCRIPTION ("chips&media Wave6 VPU driver" );MODULE_AUTHOR ("CHIPS&MEDIA INC" );MODULE_FIRMWARE ("cnm/wave633c_imx9_codec_fw.bin" );
drivers/media/platform/chips-media/wave6/wave6-vpu.h +2 -0
@@ -133,5 +133,7 @@ int wave6_vpu_get(struct wave6_vpu_device *vpu, void __iomem *core_base);void wave6_vpu_put (struct wave6_vpu_device *vpu,void __iomem *core_base);void wave6_vpu_require_work_buffer (struct wave6_vpu_device *vpu,void __iomem *core_base); +int wave6_vpu_core_register_driver (void ); +void wave6_vpu_core_unregister_driver (void );#endif /* __WAVE6_VPU_H__ */
drivers/media/platform/chips-media/wave6/wave6-vpuapi.h new file +381 -0
381 lines not on this page · raw message
drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h +6 -0
@@ -18,13 +18,19 @@#define W6_VPU_MAX_NUM_CORE 4#define MAX_NUM_INSTANCE 32 +#define W6_MAX_PIC_STRIDE (4096 * 4) +#define W6_PIC_STRIDE_ALIGNMENT 32 +#define W6_VPU_POLL_DELAY_US 10#define W6_VPU_POLL_TIMEOUT 300000 +#define W6_BOOT_WAIT_TIMEOUT 10000 +#define W6_VPU_TIMEOUT 6000#define W6_MAX_REMAP_PAGE_SIZE (1024 * 1024)#define W6_MAX_CODE_BUF_SIZE (4 * 1024 * 1024)#define W6_CODE_BUF_SIZE (1 * 1024 * 1024)#define W6_EXTRA_CODE_BUF_SIZE (256 * 1024) +#define W6_TEMPBUF_SIZE (3 * 1024 * 1024)#define W637DEC_WORKBUF_SIZE_FOR_CQ (3 * 512 * 1024)static inline bool wave6_is_product_w_series (u32 code)
Signature · 1 line
2.41.0.windows.3
PATCH v8 4/11 media: chips-media: wave6: Add Wave6 decoder firmware interface
In reply to Nas ChungAdd an interface layer to manage hardware register configuration and communication with the Chips&Media Wave6 video codec IP for video decoding.
The interface provides low-level helper functions used by the Wave6 core driver to implement video decoding operations. It handles command submission to the firmware via MMIO registers, and waits for a response by polling the firmware busy flag.
The decoder commands cover instance creation, flush and destruction, sequence initialization, frame buffer and display buffer registration, and picture decoding.
Sequence initialization and picture decoding complete asynchronously:
the interface interrupt reports completion and a query command reads back the result.
--- .../media/platform/chips-media/wave6/Makefile | 1 + .../platform/chips-media/wave6/wave6-hw.c | 753 ++++++++++++++++++ .../platform/chips-media/wave6/wave6-hw.h | 27 + .../chips-media/wave6/wave6-regdefine.h | 219 +++++ .../platform/chips-media/wave6/wave6-vpuapi.c | 202 +++++ .../platform/chips-media/wave6/wave6-vpuapi.h | 168 ++++ .../chips-media/wave6/wave6-vpuconfig.h | 5 + .../chips-media/wave6/wave6-vpuerror.h | 252 ++++++ 8 files changed, 1627 insertions(+) create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpuapi.c create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpuerror.h
drivers/media/platform/chips-media/wave6/Makefile +1 -0
@@ -3,5 +3,6 @@ wave6-objs += wave6-vpu.o \ wave6-vpu-core.o \ wave6-vpu-v4l2.o \ + wave6-vpuapi.o \ wave6-hw.o obj-$(CONFIG_VIDEO_WAVE6_VPU) += wave6.o
drivers/media/platform/chips-media/wave6/wave6-hw.c +753 -0
762 lines not on this page · raw message
drivers/media/platform/chips-media/wave6/wave6-hw.h +27 -0
@@ -15,8 +15,35 @@ enum product_id { PRODUCT_ID_NONE, }; +#define COMMAND_QUEUE_DEPTH 1 +#define SINGLE_CORE_IDC 1 +#define SINGLE_CORE 1 + +#define MAX_SET_FB_GROUP_SIZE 16 + +#define DECODE_ALL_TEMPORAL_LAYERS 0 +#define DECODE_ALL_SPATIAL_LAYERS 0 + +#define PIXEL_ORDER_INCREASING 1 + +#define WTL_RIGHT_JUSTIFIED 0 +#define WTL_LEFT_JUSTIFIED 1 +#define WTL_PIXEL_8BIT 0 +#define WTL_PIXEL_16BIT 1 +#define WTL_PIXEL_32BIT 2 + +#define H265_NAL_UNIT_TYPE_IDR_W_RADL 19 +#define H265_NAL_UNIT_TYPE_IDR_N_LP 20 + +#define H264_NAL_UNIT_TYPE_IDR_PICTURE 5 +bool wave6_vpu_is_init (struct vpu_core_device *core);int wave6_vpu_get_version (struct vpu_core_device *core);void wave6_vpu_enable_interrupt (struct vpu_core_device *core); +int wave6_vpu_build_up_dec_param (struct vpu_instance *inst,struct dec_open_param *param); +int wave6_vpu_dec_get_seq_info (struct vpu_instance *inst,struct dec_seq_info *info); +int wave6_vpu_dec_fini_seq (struct vpu_instance *inst,u32 *fail_res); +dma_addr_t wave6_vpu_dec_get_rd_ptr (struct vpu_instance *inst); +#endif /* __WAVE6_HW_H__ */
drivers/media/platform/chips-media/wave6/wave6-regdefine.h +219 -0
229 lines not on this page · raw message
drivers/media/platform/chips-media/wave6/wave6-vpuapi.c new file +202 -0
202 lines not on this page · raw message
drivers/media/platform/chips-media/wave6/wave6-vpuapi.h +168 -0
214 lines not on this page · raw message
drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h +5 -0
@@ -25,12 +25,17 @@#define W6_VPU_POLL_TIMEOUT 300000#define W6_BOOT_WAIT_TIMEOUT 10000#define W6_VPU_TIMEOUT 6000 +#define W6_VPU_TIMEOUT_CYCLE_COUNT (8000000 * 4 * 4) + +#define HOST_ENDIAN VDI_128BIT_LITTLE_ENDIAN +#define VPU_STREAM_ENDIAN HOST_ENDIAN#define W6_MAX_REMAP_PAGE_SIZE (1024 * 1024)#define W6_MAX_CODE_BUF_SIZE (4 * 1024 * 1024)#define W6_CODE_BUF_SIZE (1 * 1024 * 1024)#define W6_EXTRA_CODE_BUF_SIZE (256 * 1024)#define W6_TEMPBUF_SIZE (3 * 1024 * 1024) +#define W637DEC_WORKBUF_SIZE (2 * 512 * 1024)#define W637DEC_WORKBUF_SIZE_FOR_CQ (3 * 512 * 1024)static inline bool wave6_is_product_w_series (u32 code)
drivers/media/platform/chips-media/wave6/wave6-vpuerror.h new file +252 -0
252 lines not on this page · raw message
Signature · 1 line
2.41.0.windows.3
PATCH v8 5/11 media: chips-media: wave6: Add Wave6 m2m decoder
In reply to Nas ChungAdd the V4L2 m2m stateful decoder for the Chips&Media Wave6 video codec IP.
The decoder supports the following features and formats:
- Supported codecs: AVC, HEVC (8-bit streams only)
- Supported raw (CAPTURE) formats:
* V4L2_PIX_FMT_YUV420/NV12/NV21
* V4L2_PIX_FMT_YUV420M/NV12M/NV21M
- Supports scale down
- Supports dynamic resolution change
While other versions of the Wave6 IP may support VP9 and AV1 decoding, those formats are not implemented or validated in this driver at this time.
--- .../media/platform/chips-media/wave6/Kconfig | 3 +- .../media/platform/chips-media/wave6/Makefile | 1 + .../chips-media/wave6/wave6-vpu-core.c | 21 + .../chips-media/wave6/wave6-vpu-core.h | 2 + .../chips-media/wave6/wave6-vpu-dec.c | 1935 +++++++++++++++++ .../platform/chips-media/wave6/wave6-vpuapi.h | 2 + .../chips-media/wave6/wave6-vpuconfig.h | 10 + 7 files changed, 1973 insertions(+), 1 deletion(-) create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c
drivers/media/platform/chips-media/wave6/Kconfig +2 -1
@@ -10,6 +10,7 @@ config VIDEO_WAVE6_VPU select GENERIC_ALLOCATOR help Chips&Media Wave6 stateful codec driver. - The wave6 driver manages shared resources such as firmware memory. + It provides decoding capabilities for H.264, HEVC, + and other video formats. To compile this driver as a module, choose M here: the module will be called wave6.
drivers/media/platform/chips-media/wave6/Makefile +1 -0
@@ -4,5 +4,6 @@ wave6-objs += wave6-vpu.o \ wave6-vpu-core.o \ wave6-vpu-v4l2.o \ wave6-vpuapi.o \ + wave6-vpu-dec.o \ wave6-hw.o obj-$(CONFIG_VIDEO_WAVE6_VPU) += wave6.o
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c +21 -0
@@ -181,6 +181,11 @@ static void wave6_vpu_core_free_temp_vbuf(void *data)wave6_vdi_free_dma (&core->temp_vbuf); } +static void wave6_vpu_core_unregister_dec (void *data) +{ +wave6_vpu_dec_unregister_device (data); +} +static int wave6_vpu_core_probe (struct platform_device *pdev) {struct vpu_core_device *core;
@@ -278,6 +283,22 @@ static int wave6_vpu_core_probe(struct platform_device *pdev)if (ret)return ret; +if (core->res->codec_types &WAVE6_IS_DEC ) { + ret =wave6_vpu_dec_register_device (core); +if (ret) { +dev_err (&pdev->dev, +"failed to register video_dev_dec: %d\n" , ret); +return ret; + } + + ret =devm_add_action_or_reset (&pdev->dev, wave6_vpu_core_unregister_dec, core); +if (ret) +return ret; + } + +dev_dbg (&pdev->dev,"Added wave6 driver with caps %s\n" , + core->res->codec_types &WAVE6_IS_DEC ?"'DECODE'" :"" ); +return 0 ; }
drivers/media/platform/chips-media/wave6/wave6-vpu-core.h +2 -0
@@ -92,6 +92,8 @@ const char *wave6_vpu_instance_state_name(enum vpu_instance_state state);void wave6_vpu_set_instance_state (struct vpu_instance *inst,enum vpu_instance_state state);int wave6_vpu_wait_interrupt (struct vpu_instance *inst,unsigned int timeout); +int wave6_vpu_dec_register_device (struct vpu_core_device *core); +void wave6_vpu_dec_unregister_device (struct vpu_core_device *core);void wave6_vpu_device_run_timeout (struct work_struct *work);void wave6_vpu_finish_job (struct vpu_instance *inst);int wave6_vpu_init_m2m_dev (struct vpu_core_device *core);
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c new file +1935 -0
1,935 lines not on this page · raw message
drivers/media/platform/chips-media/wave6/wave6-vpuapi.h +2 -0
@@ -376,6 +376,7 @@ struct vpu_irq {* @m2m_dev: V4L2 mem2mem device * @instances: List of VPU instances * @inst_lock: Spinlock protecting instance list +* @video_dev_dec: Video device node for decoder * @dev_lock: Mutex protecting video_device, src, dst vb2_queue * @hw_lock: Mutex protecting register access * @attr: Hardware attributes retrieved after boot
@@ -397,6 +398,7 @@ struct vpu_core_device {struct v4l2_m2m_dev *m2m_dev;struct list_head instances;spinlock_t inst_lock;/* Protects instance list */ +struct video_device *video_dev_dec;struct mutex dev_lock;/* Protects video_device, src, dst vb2_queue */ struct mutex hw_lock;/* Protects register access */ struct vpu_attr attr;
drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h +10 -0
@@ -20,6 +20,15 @@#define W6_MAX_PIC_STRIDE (4096 * 4)#define W6_PIC_STRIDE_ALIGNMENT 32 +#define W6_FBC_BUF_ALIGNMENT 32 +#define W6_DEC_BUF_ALIGNMENT 32 +#define W6_DEF_DEC_PIC_WIDTH 720 +#define W6_DEF_DEC_PIC_HEIGHT 480 +#define W6_MIN_DEC_PIC_WIDTH 64 +#define W6_MIN_DEC_PIC_HEIGHT 64 +#define W6_MAX_DEC_PIC_WIDTH 4096 +#define W6_MAX_DEC_PIC_HEIGHT 4096 +#define W6_DEC_PIC_SIZE_STEP 1#define W6_VPU_POLL_DELAY_US 10#define W6_VPU_POLL_TIMEOUT 300000
@@ -29,6 +38,7 @@#define HOST_ENDIAN VDI_128BIT_LITTLE_ENDIAN#define VPU_STREAM_ENDIAN HOST_ENDIAN +#define VPU_SOURCE_ENDIAN HOST_ENDIAN#define W6_MAX_REMAP_PAGE_SIZE (1024 * 1024)#define W6_MAX_CODE_BUF_SIZE (4 * 1024 * 1024)
Signature · 1 line
2.41.0.windows.3
PATCH v8 6/11 media: chips-media: wave6: Add Wave6 encoder firmware interface
In reply to Nas ChungAdd an interface layer to manage hardware register configuration and communication with the Chips&Media Wave6 video codec IP for video encoding.
The interface provides low-level helper functions used by the Wave6 core driver to implement video encoding operations. It handles command submission to the firmware via MMIO registers, and waits for a response by polling the firmware busy flag.
The encoder commands cover instance creation and destruction, parameter setting and change, frame buffer registration, and picture encoding.
Parameter setting and picture encoding complete asynchronously:
the interface interrupt reports completion and a query command reads back the result.
--- .../platform/chips-media/wave6/wave6-hw.c | 940 +++++++++++++++++- .../platform/chips-media/wave6/wave6-hw.h | 7 + .../chips-media/wave6/wave6-regdefine.h | 312 ++++++ .../chips-media/wave6/wave6-vpu-core.c | 3 +- .../platform/chips-media/wave6/wave6-vpuapi.c | 131 +++ .../platform/chips-media/wave6/wave6-vpuapi.h | 428 ++++++++ .../chips-media/wave6/wave6-vpuconfig.h | 14 + 7 files changed, 1833 insertions(+), 2 deletions(-)
drivers/media/platform/chips-media/wave6/wave6-hw.c +939 -1
@@ -162,7 +162,9 @@ void wave6_vpu_enable_interrupt(struct vpu_core_device *core)lockdep_assert_held (&core->hw_lock); - data =BIT (W6_INT_BIT_INIT_SEQ ); + data =BIT (W6_INT_BIT_ENC_SET_PARAM ); + data |=BIT (W6_INT_BIT_ENC_PIC ); + data |=BIT (W6_INT_BIT_INIT_SEQ ); data |=BIT (W6_INT_BIT_DEC_PIC ); data |=BIT (W6_INT_BIT_BSBUF_ERROR ); data |=BIT (W6_INT_BIT_REQ_WORK_BUF );
@@ -932,3 +934,939 @@ int wave6_vpu_dec_flush_instance(struct vpu_instance *inst)return 0 ; } + +struct enc_cmd_set_param_reg { +u32 enable; +u32 src_size; +u32 custom_map_endian; +u32 sps; +u32 pps; +u32 gop; +u32 intra; +u32 conf_win0; +u32 conf_win1; +u32 rdo; +u32 slice; +u32 intra_refresh; +u32 intra_qp; +u32 rc_frame_rate; +u32 rc_target_rate; +u32 rc; +u32 hvs; +u32 rc_max_bitrate; +u32 rc_vbv_buffer_size; +u32 inter_qp; +u32 rot; +u32 num_units_in_tick; +u32 time_scale; +u32 num_ticks_poc_diff_one; +u32 max_intra_pic_bit; +u32 max_inter_pic_bit; +u32 bg; +u32 non_vcl_param; +u32 vui_rbsp_addr; +u32 hrd_rbsp_addr; +u32 qround_offset; +u32 quant1; +u32 quant2; +u32 custom_gop; +u32 custom_gop_pic[MAX_CUSTOM_GOP_NUM ]; +u32 tile_param; +u32 custom_lambda[MAX_CUSTOM_LAMBDA_NUM ]; +u32 temp_layer_qp[MAX_NUM_CHANGEABLE_TEMP_LAYER ]; +u32 scaler_size; +u32 scaler; +u32 color; +u32 sar; +u32 sar_extended; +}; + +struct enc_cmd_change_param_reg { +u32 enable; +u32 rc_target_rate; +}; + +int wave6_vpu_build_up_enc_param (struct device *dev,struct vpu_instance *inst, +struct enc_open_param *param) +{ +struct enc_info *p_enc_info = &inst->codec_info->enc_info; +u32 reg_val; +int ret; + +lockdep_assert_held (&inst->dev->hw_lock); + + p_enc_info->seq_info_obtained = false; + p_enc_info->sec_axi.use_enc_rdo = true; + p_enc_info->sec_axi.use_enc_lf = true; + p_enc_info->mir_dir = param->mir_dir; + p_enc_info->rot_angle = param->rot_angle; + +vpu_write_reg_relaxed (inst->dev,W6_CMD_CREATE_INST_TEMP_BASE , + param->inst_buffer.temp_base); +vpu_write_reg_relaxed (inst->dev,W6_CMD_CREATE_INST_TEMP_SIZE , + param->inst_buffer.temp_size); +vpu_write_reg_relaxed (inst->dev,W6_CMD_CREATE_INST_AR_TABLE_BASE , + param->inst_buffer.ar_base); +vpu_write_reg_relaxed (inst->dev,W6_CMD_CREATE_INST_BS_PARAM ,VPU_STREAM_ENDIAN ); +vpu_write_reg_relaxed (inst->dev,W6_CMD_CREATE_INST_SRC_OPT ,0 ); +vpu_write_reg_relaxed (inst->dev,W6_CMD_CREATE_INST_ADDR_EXT , param->ext_addr_vcpu); + + reg_val =CREATE_INST_CORE_INFO_CQ_DEPTH (COMMAND_QUEUE_DEPTH ) | +CREATE_INST_CORE_INFO_CORE_IDC (SINGLE_CORE_IDC ) | +CREATE_INST_CORE_INFO_CORE_NUM (SINGLE_CORE ); +vpu_write_reg_relaxed (inst->dev,W6_CMD_CREATE_INST_CORE_INFO , reg_val); + + reg_val =CREATE_INST_PRIORITY_SECURITY_FLAG (param->is_secure_inst) | +CREATE_INST_PRIORITY_VALUE (param->inst_priority); +vpu_write_reg_relaxed (inst->dev,W6_CMD_CREATE_INST_PRIORITY , reg_val); +vpu_write_reg_relaxed (inst->dev,W6_CMD_CREATE_INST_TIMEOUT_CYCLE_COUNT , +W6_VPU_TIMEOUT_CYCLE_COUNT ); + +wave6_send_command (inst->dev,0 , inst->std,W6_CMD_CREATE_INSTANCE ); + ret =wave6_wait_vpu_busy (inst->dev,W6_VPU_BUSY_STATUS ); +if (ret) { +dev_err (inst->dev->dev,"%s: timeout\n" , __func__); +return ret; + } + +if (!vpu_read_reg_relaxed (inst->dev,W6_RET_SUCCESS )) { +u32 reason_code =vpu_read_reg_relaxed (inst->dev,W6_RET_FAIL_REASON ); + +wave6_print_reg_err (inst->dev, reason_code); +return -EIO ; + } + + inst->id =vpu_read_reg_relaxed (inst->dev,W6_RET_INSTANCE_ID ); + +return 0 ; +} + +static int wave6_set_enc_crop_info (u32 codec,struct enc_codec_param *param, +u32 rot_mir,int width,int height) +{ +int aligned_height = (codec ==W_HEVC_ENC ) ?ALIGN (height,32 ) :ALIGN (height,16 ); +int aligned_width = (codec ==W_HEVC_ENC ) ?ALIGN (width,32 ) :ALIGN (width,16 ); +int crop_right, crop_left, crop_top, crop_bot; +int pad_right, pad_bot; + +if (codec ==W_HEVC_ENC && + (!rot_mir || + rot_mir == (SET_PARAM_ROT_MIRH |SET_PARAM_ROT_MIRV |SET_PARAM_ROT_180 ))) +return 0 ; + + pad_right = aligned_width - width; + pad_bot = aligned_height - height; + +if (param->conf_win.right >0 ) + crop_right = param->conf_win.right + pad_right; +else + crop_right = pad_right; + +if (param->conf_win.bottom >0 ) + crop_bot = param->conf_win.bottom + pad_bot; +else + crop_bot = pad_bot; + + crop_top = param->conf_win.top; + crop_left = param->conf_win.left; + +if (rot_mir ==SET_PARAM_ROT_90 || + rot_mir == (SET_PARAM_ROT_MIRH |SET_PARAM_ROT_MIRV |SET_PARAM_ROT_270 )) { + param->conf_win.top = crop_right; + param->conf_win.left = crop_top; + param->conf_win.bottom = crop_left; + param->conf_win.right = crop_bot; + }else if (rot_mir ==SET_PARAM_ROT_180 || + rot_mir == (SET_PARAM_ROT_MIRH |SET_PARAM_ROT_MIRV )) { + param->conf_win.top = crop_bot; + param->conf_win.left = crop_right; + param->conf_win.bottom = crop_top; + param->conf_win.right = crop_left; + }else if (rot_mir ==SET_PARAM_ROT_270 || + rot_mir == (SET_PARAM_ROT_MIRH |SET_PARAM_ROT_MIRV |SET_PARAM_ROT_90 )) { + param->conf_win.top = crop_left; + param->conf_win.left = crop_bot; + param->conf_win.bottom = crop_right; + param->conf_win.right = crop_top; + }else if (rot_mir ==SET_PARAM_ROT_MIRV || + rot_mir == (SET_PARAM_ROT_MIRH |SET_PARAM_ROT_180 )) { + param->conf_win.top = crop_bot; + param->conf_win.left = crop_left; + param->conf_win.bottom = crop_top; + param->conf_win.right = crop_right; + }else if (rot_mir ==SET_PARAM_ROT_MIRH || + rot_mir == (SET_PARAM_ROT_MIRV |SET_PARAM_ROT_180 )) { + param->conf_win.top = crop_top; + param->conf_win.left = crop_right; + param->conf_win.bottom = crop_bot; + param->conf_win.right = crop_left; + }else if (rot_mir == (SET_PARAM_ROT_MIRV |SET_PARAM_ROT_90 ) || + rot_mir == (SET_PARAM_ROT_MIRH |SET_PARAM_ROT_270 )) { + param->conf_win.top = crop_left; + param->conf_win.left = crop_top; + param->conf_win.bottom = crop_right; + param->conf_win.right = crop_bot; + }else if (rot_mir == (SET_PARAM_ROT_MIRV |SET_PARAM_ROT_270 ) || + rot_mir == (SET_PARAM_ROT_MIRH |SET_PARAM_ROT_90 )) { + param->conf_win.top = crop_right; + param->conf_win.left = crop_bot; + param->conf_win.bottom = crop_left; + param->conf_win.right = crop_top; + }else { + param->conf_win.top = crop_top; + param->conf_win.left = crop_left; + param->conf_win.bottom = crop_bot; + param->conf_win.right = crop_right; + } + +return 0 ; +} + +static void wave6_update_enc_info (struct enc_info *p_enc_info) +{ +struct enc_open_param op = p_enc_info->open_param; + + p_enc_info->width = op.pic_width; + p_enc_info->height = op.pic_height; + p_enc_info->c_fmt_idc =get_chroma_format_idc (op.out_fmt); +} + +static u32 wave6_enc_rot_mir (enum rotation_angle rot_angle, +enum mirror_direction mir_dir) +{ +u32 rot_mir =0 ; + +switch (rot_angle) { +case ROT_0 : +break ; +case ROT_90 : + rot_mir |=SET_PARAM_ROT_90 ; +break ; +case ROT_180 : + rot_mir |=SET_PARAM_ROT_180 ; +break ; +case ROT_270 : + rot_mir |=SET_PARAM_ROT_270 ; +break ; + } + +switch (mir_dir) { +case MIR_NONE : +break ; +case MIR_VER : + rot_mir |=SET_PARAM_ROT_MIRV ; +break ; +case MIR_HOR : + rot_mir |=SET_PARAM_ROT_MIRH ; +break ; +case MIR_HOR_VER : + rot_mir |=SET_PARAM_ROT_MIRH |SET_PARAM_ROT_MIRV ; +break ; + } + +return rot_mir; +} + +static void wave6_gen_set_param_reg_common (struct enc_info *p_enc_info,enum codec_std std, +struct enc_cmd_set_param_reg *reg) +{ +struct enc_open_param *p_open_param = &p_enc_info->open_param; +struct enc_codec_param *p_param = &p_open_param->codec_param; +u32 rot_mir; + + rot_mir =wave6_enc_rot_mir (p_enc_info->rot_angle, p_enc_info->mir_dir); +wave6_set_enc_crop_info (std, p_param, rot_mir, p_enc_info->width, p_enc_info->height); + + reg->custom_map_endian =VPU_USER_DATA_ENDIAN ; +if (rot_mir) + reg->rot = rot_mir |SET_PARAM_ROT_ENABLE ; + reg->src_size =SET_PARAM_SRC_SIZE_HEIGHT (p_enc_info->height) | +SET_PARAM_SRC_SIZE_WIDTH (p_enc_info->width); + reg->conf_win0 =SET_PARAM_CONF_WIN0_BOTTOM (p_param->conf_win.bottom) | +SET_PARAM_CONF_WIN0_TOP (p_param->conf_win.top); + reg->conf_win1 =SET_PARAM_CONF_WIN1_RIGHT (p_param->conf_win.right) | +SET_PARAM_CONF_WIN1_LEFT (p_param->conf_win.left); + reg->gop =SET_PARAM_GOP_TEMP_LAYER_CNT (p_param->temp_layer_cnt) | +SET_PARAM_GOP_TEMP_LAYER3_QP_ENABLE (p_param->temp_layer[3 ].change_qp) | +SET_PARAM_GOP_TEMP_LAYER2_QP_ENABLE (p_param->temp_layer[2 ].change_qp) | +SET_PARAM_GOP_TEMP_LAYER1_QP_ENABLE (p_param->temp_layer[1 ].change_qp) | +SET_PARAM_GOP_TEMP_LAYER0_QP_ENABLE (p_param->temp_layer[0 ].change_qp) | +SET_PARAM_GOP_PRESET_IDX (p_param->gop_preset_idx); +for (int i =0 ; i <MAX_NUM_CHANGEABLE_TEMP_LAYER ; i++) { + reg->temp_layer_qp[i] =SET_PARAM_TEMP_LAYER_QP_B (p_param->temp_layer[i].qp_b) | +SET_PARAM_TEMP_LAYER_QP_P (p_param->temp_layer[i].qp_p) | +SET_PARAM_TEMP_LAYER_QP_I (p_param->temp_layer[i].qp_i); + } + reg->intra_refresh =SET_PARAM_INTRA_REFRESH_ARGUMENT (p_param->intra_refresh_arg) | +SET_PARAM_INTRA_REFRESH_MODE (p_param->intra_refresh_mode); + reg->intra_qp =SET_PARAM_INTRA_QP_MAX (p_param->max_qp_i) | +SET_PARAM_INTRA_QP_MIN (p_param->min_qp_i); + reg->inter_qp =SET_PARAM_INTER_QP_MAX_B (p_param->max_qp_b) | +SET_PARAM_INTER_QP_MIN_B (p_param->min_qp_b) | +SET_PARAM_INTER_QP_MAX_P (p_param->max_qp_p) | +SET_PARAM_INTER_QP_MIN_P (p_param->min_qp_p); + reg->rc_frame_rate = p_param->frame_rate; + reg->rc_target_rate = p_param->bitrate; + reg->rc_max_bitrate = p_param->max_bitrate; + reg->rc_vbv_buffer_size = p_param->cpb_size; + reg->rc =SET_PARAM_RC_UPDATE_SPEED (p_param->rc_update_speed) | +SET_PARAM_RC_INITIAL_LEVEL (p_param->rc_initial_level) | +SET_PARAM_RC_INITIAL_QP (p_param->rc_initial_qp) | +SET_PARAM_RC_MODE (p_param->rc_mode) | +SET_PARAM_RC_PIC_LEVEL_MAX_DELTA_QP (p_param->pic_rc_max_dqp) | +SET_PARAM_RC_VBV_OVERFLOW_DROP_FRAME (p_param->en_skip_frame) | +SET_PARAM_RC_CU_LEVEL_ENABLE (p_param->en_cu_level_rate_control) | +SET_PARAM_RC_ENABLE (p_param->en_rate_control); + reg->hvs =SET_PARAM_HVS_MAX_DELTA_QP (p_param->max_delta_qp) | +SET_PARAM_HVS_QP_SCALE (p_param->hvs_qp_scale_div2); + reg->num_units_in_tick = p_param->num_units_in_tick; + reg->time_scale = p_param->time_scale; + reg->num_ticks_poc_diff_one = p_param->num_ticks_poc_diff_one; + reg->max_intra_pic_bit = p_param->max_intra_pic_bit; + reg->max_inter_pic_bit = p_param->max_inter_pic_bit; + reg->bg =SET_PARAM_BG_DELTA_QP (p_param->bg_delta_qp) | +SET_PARAM_BG_THRESHOLD_MEAN_DIFF (p_param->bg_th_mean_diff) | +SET_PARAM_BG_THRESHOLD_MAX_DIFF (p_param->bg_th_diff) | +SET_PARAM_BG_ENABLE (p_param->en_bg_detect); + reg->qround_offset =SET_PARAM_QROUND_OFFSET_INTER (p_param->qround_inter) | +SET_PARAM_QROUND_OFFSET_INTRA (p_param->qround_intra); + reg->custom_gop = p_param->gop_param.size; +for (int i =0 ; i < p_param->gop_param.size; i++) { +struct custom_gop_pic_param pic = p_param->gop_param.pic[i]; + + reg->custom_gop_pic[i] =SET_PARAM_CUSTOM_GOP_PIC_TEMP_ID (pic.temporal_id) | +SET_PARAM_CUSTOM_GOP_PIC_REF1_POC (pic.ref_poc_l1) | +SET_PARAM_CUSTOM_GOP_PIC_REF0_POC (pic.ref_poc_l0) | +SET_PARAM_CUSTOM_GOP_PIC_MULTI_REF_P (pic.multi_ref_p) | +SET_PARAM_CUSTOM_GOP_PIC_QP (pic.pic_qp) | +SET_PARAM_CUSTOM_GOP_PIC_POC_OFFSET (pic.poc_offset) | +SET_PARAM_CUSTOM_GOP_PIC_TYPE (pic.pic_type); + } +for (int i =0 ; i <MAX_CUSTOM_LAMBDA_NUM ; i++) { + reg->custom_lambda[i] =SET_PARAM_CUSTOM_LAMBDA_SSD (p_param->lambda_ssd[i]) | +SET_PARAM_CUSTOM_LAMBDA_SAD (p_param->lambda_sad[i]); + } + reg->scaler_size =SET_PARAM_SCALER_SIZE_HEIGHT (p_enc_info->height) | +SET_PARAM_SCALER_SIZE_WIDTH (p_enc_info->width); + reg->scaler =SET_PARAM_SCALER_COEF_MODE (p_enc_info->scaler.coef_mode) | +SET_PARAM_SCALER_ENABLE (p_enc_info->scaler.enable); + reg->color =SET_PARAM_COLOR_RANGE (p_param->color.color_range) | +SET_PARAM_COLOR_MATRIX_COEFF (p_param->color.matrix_coefficients) | +SET_PARAM_COLOR_TRANS_CHAR (p_param->color.transfer_characteristics) | +SET_PARAM_COLOR_PRIMARIES (p_param->color.color_primaries) | +SET_PARAM_COLOR_DESCRIPTION_PRESENT_ON ; + reg->sar =SET_PARAM_SAR_ASPECT_RATIO_IDC (p_param->sar.idc) | +SET_PARAM_SAR_ASPECT_RATIO_ENABLE (p_param->sar.enable); + reg->sar_extended =SET_PARAM_SAR_EXTENDED_HEIGHT (p_param->sar.height) | +SET_PARAM_SAR_EXTENDED_WIDTH (p_param->sar.width); +} + +static void wave6_gen_set_param_reg_hevc (struct enc_info *p_enc_info, +struct enc_cmd_set_param_reg *reg) +{ +struct enc_open_param *p_open_param = &p_enc_info->open_param; +struct enc_codec_param *p_param = &p_open_param->codec_param; + + reg->sps =SET_PARAM_SPS_DEFAULT_SCALING_LIST (p_param->en_scaling_list) | +SET_PARAM_SPS_STILL_PICTURE (p_param->en_still_picture) | +SET_PARAM_SPS_AUTO_LEVEL_ADJUSTING_ON | +SET_PARAM_SPS_STRONG_INTRA_SMOOTHING (p_param->en_intra_smooth) | +SET_PARAM_SPS_INTRA_TRANSFORM_SKIP_ON | +SET_PARAM_SPS_SAMPLE_ADAPTIVE_OFFSET (p_param->en_sao) | +SET_PARAM_SPS_TEMPORAL_MVP (p_param->en_temporal_mvp) | +SET_PARAM_SPS_LONGTERM_REFERENCE (p_param->en_longterm) | +SET_PARAM_SPS_C_FMT_IDC (p_enc_info->c_fmt_idc) | +SET_PARAM_SPS_BIT_DEPTH (p_param->internal_bit_depth) | +SET_PARAM_SPS_TIER (p_param->tier) | +SET_PARAM_SPS_LEVEL (p_param->level) | +SET_PARAM_SPS_PROFILE (p_param->profile); + reg->pps =SET_PARAM_PPS_CR_QP_OFFSET (p_param->cr_qp_offset) | +SET_PARAM_PPS_CB_QP_OFFSET (p_param->cb_qp_offset) | +SET_PARAM_PPS_TC_OFFSET_DIV2 (p_param->tc_offset_div2) | +SET_PARAM_PPS_BETA_OFFSET_DIV2 (p_param->beta_offset_div2) | +SET_PARAM_PPS_DEBLOCKING_FILTER (!p_param->en_dbk) | +SET_PARAM_PPS_LF_SLICE_BOUNDARY (p_param->en_lf_slice_boundary) | +SET_PARAM_PPS_CONST_INTRA_PREDICTION (p_param->en_const_intra_pred); + reg->intra =SET_PARAM_INTRA_PERIOD (p_param->intra_period) | +SET_PARAM_INTRA_HEADER_MODE (p_param->forced_idr_header) | +SET_PARAM_INTRA_QP (p_param->qp) | +SET_PARAM_INTRA_REFRESH_TYPE (p_param->decoding_refresh_type); + reg->rdo =SET_PARAM_RDO_CUSTOM_LAMBDA (p_param->en_custom_lambda) | +SET_PARAM_RDO_ME_SEARCH_CENTER_ON | +SET_PARAM_RDO_QROUND_OFFSET (p_param->en_qround_offset) | +SET_PARAM_RDO_ADAPTIVE_ROUND_ON | +SET_PARAM_RDO_HVS_QP (p_param->en_hvs_qp); + reg->slice =SET_PARAM_SLICE_ARGUMENT (p_param->slice_arg) | +SET_PARAM_SLICE_MODE (p_param->slice_mode); + reg->quant2 =SET_PARAM_QUANT2_LAMBDA_DQP_INTER (p_param->lambda_dqp_inter) | +SET_PARAM_QUANT2_LAMBDA_DQP_INTRA (p_param->lambda_dqp_intra); + reg->non_vcl_param =SET_PARAM_NON_VCL_PARAM_ENCODE_VUI ; +} + +static void wave6_gen_set_param_reg_avc (struct enc_info *p_enc_info, +struct enc_cmd_set_param_reg *reg) +{ +struct enc_open_param *p_open_param = &p_enc_info->open_param; +struct enc_codec_param *p_param = &p_open_param->codec_param; + + reg->sps =SET_PARAM_SPS_DEFAULT_SCALING_LIST (p_param->en_scaling_list) | +SET_PARAM_SPS_AUTO_LEVEL_ADJUSTING_ON | +SET_PARAM_SPS_LONGTERM_REFERENCE (p_param->en_longterm) | +SET_PARAM_SPS_C_FMT_IDC (p_enc_info->c_fmt_idc) | +SET_PARAM_SPS_BIT_DEPTH (p_param->internal_bit_depth) | +SET_PARAM_SPS_LEVEL (p_param->level) | +SET_PARAM_SPS_PROFILE (p_param->profile); + reg->pps =SET_PARAM_PPS_ENTROPY_CODING_MODE (p_param->en_cabac) | +SET_PARAM_PPS_TRANSFORM8X8 (p_param->en_transform8x8) | +SET_PARAM_PPS_CR_QP_OFFSET (p_param->cr_qp_offset) | +SET_PARAM_PPS_CB_QP_OFFSET (p_param->cb_qp_offset) | +SET_PARAM_PPS_TC_OFFSET_DIV2 (p_param->tc_offset_div2) | +SET_PARAM_PPS_BETA_OFFSET_DIV2 (p_param->beta_offset_div2) | +SET_PARAM_PPS_DEBLOCKING_FILTER (!p_param->en_dbk) | +SET_PARAM_PPS_LF_SLICE_BOUNDARY (p_param->en_lf_slice_boundary) | +SET_PARAM_PPS_CONST_INTRA_PREDICTION (p_param->en_const_intra_pred); + reg->intra =SET_PARAM_INTRA_HEADER_MODE_AVC (p_param->forced_idr_header) | +SET_PARAM_INTRA_IDR_PERIOD_AVC (p_param->idr_period) | +SET_PARAM_INTRA_PERIOD_AVC (p_param->intra_period) | +SET_PARAM_INTRA_QP_AVC (p_param->qp); + reg->rdo =SET_PARAM_RDO_CUSTOM_LAMBDA (p_param->en_custom_lambda) | +SET_PARAM_RDO_QROUND_OFFSET (p_param->en_qround_offset) | +SET_PARAM_RDO_ADAPTIVE_ROUND_ON | +SET_PARAM_RDO_HVS_QP (p_param->en_hvs_qp); + reg->slice =SET_PARAM_SLICE_ARGUMENT (p_param->slice_arg) | +SET_PARAM_SLICE_MODE (p_param->slice_mode); + reg->quant2 =SET_PARAM_QUANT2_LAMBDA_DQP_INTER (p_param->lambda_dqp_inter) | +SET_PARAM_QUANT2_LAMBDA_DQP_INTRA (p_param->lambda_dqp_intra); + reg->non_vcl_param =SET_PARAM_NON_VCL_PARAM_ENCODE_VUI ; +} + +static void wave6_gen_change_param_reg_common (struct vpu_instance *inst, +struct enc_info *p_enc_info, +struct enc_cmd_change_param_reg *reg) +{ +if (p_enc_info->open_param.codec_param.bitrate != inst->enc_ctrls.bitrate) { + reg->enable |=SET_PARAM_ENABLE_RC_TARGET_RATE ; + reg->rc_target_rate = inst->enc_ctrls.bitrate; + p_enc_info->open_param.codec_param.bitrate = inst->enc_ctrls.bitrate; + } +} + +int wave6_vpu_enc_issue_seq_init (struct vpu_instance *inst) +{ +struct enc_cmd_set_param_reg reg = {}; +struct enc_info *p_enc_info; +int ret; + +guard (mutex)(&inst->dev->hw_lock); + + p_enc_info = &inst->codec_info->enc_info; + +wave6_update_enc_info (p_enc_info); + +wave6_gen_set_param_reg_common (p_enc_info, inst->std, ®); +if (inst->std ==W_HEVC_ENC ) +wave6_gen_set_param_reg_hevc (p_enc_info, ®); +else if (inst->std ==W_AVC_ENC ) +wave6_gen_set_param_reg_avc (p_enc_info, ®); + +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_OPTION ,W6_SET_PARAM_OPT_COMMON ); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_ENABLE , reg.enable); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_SRC_SIZE , reg.src_size); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_CUSTOM_MAP_ENDIAN , + reg.custom_map_endian); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_SPS , reg.sps); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_PPS , reg.pps); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_GOP , reg.gop); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_INTRA , reg.intra); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_CONF_WIN0 , reg.conf_win0); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_CONF_WIN1 , reg.conf_win1); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_RDO , reg.rdo); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_SLICE , reg.slice); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_INTRA_REFRESH , reg.intra_refresh); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_INTRA_QP , reg.intra_qp); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_RC_FRAME_RATE , reg.rc_frame_rate); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_RC_TARGET_RATE , reg.rc_target_rate); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_RC , reg.rc); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_HVS , reg.hvs); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_RC_MAX_BITRATE , reg.rc_max_bitrate); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_RC_VBV_BUFFER_SIZE , + reg.rc_vbv_buffer_size); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_INTER_QP , reg.inter_qp); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_ROT , reg.rot); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_NUM_UNITS_IN_TICK , + reg.num_units_in_tick); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_TIME_SCALE , reg.time_scale); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_NUM_TICKS_POC_DIFF_ONE , + reg.num_ticks_poc_diff_one); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_MAX_INTRA_PIC_BIT , + reg.max_intra_pic_bit); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_MAX_INTER_PIC_BIT , + reg.max_inter_pic_bit); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_BG , reg.bg); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_NON_VCL_PARAM , reg.non_vcl_param); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_VUI_RBSP_ADDR , reg.vui_rbsp_addr); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_HRD_RBSP_ADDR , reg.hrd_rbsp_addr); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_QROUND_OFFSET , reg.qround_offset); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_QUANT1 , reg.quant1); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_QUANT2 , reg.quant2); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_CUSTOM_GOP , reg.custom_gop); +for (int i =0 ; i <MAX_CUSTOM_GOP_NUM ; i++) +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_CUSTOM_GOP_PIC (i), + reg.custom_gop_pic[i]); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_TILE_PARAM , reg.tile_param); +for (int i =0 ; i <MAX_CUSTOM_LAMBDA_NUM ; i++) +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_CUSTOM_LAMBDA (i), + reg.custom_lambda[i]); +for (int i =0 ; i <MAX_NUM_CHANGEABLE_TEMP_LAYER ; i++) +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_TEMP_LAYER_QP (i), + reg.temp_layer_qp[i]); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_SCALER_SIZE , reg.scaler_size); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_SCALER , reg.scaler); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_COLOR , reg.color); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_SAR , reg.sar); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_SAR_EXTENDED , reg.sar_extended); + +wave6_send_command (inst->dev, inst->id, inst->std,W6_CMD_ENC_SET_PARAM ); + ret =wave6_wait_vpu_busy (inst->dev,W6_VPU_BUSY_STATUS ); +if (ret) { +dev_err (inst->dev->dev,"%s: timeout\n" , __func__); +return ret; + } + +if (!vpu_read_reg_relaxed (inst->dev,W6_RET_SUCCESS )) { +u32 reason_code =vpu_read_reg_relaxed (inst->dev,W6_RET_FAIL_REASON ); + +wave6_print_reg_err (inst->dev, reason_code); +return -EIO ; + } + +return 0 ; +} + +int wave6_vpu_enc_get_seq_info (struct vpu_instance *inst,struct enc_seq_info *info) +{ +int ret; + +lockdep_assert_held (&inst->dev->hw_lock); + + ret =wave6_send_query (inst->dev, inst->id, inst->std,W6_QUERY_OPT_GET_RESULT ); +if (ret) +return ret; + +if (vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_ENCODING_SUCCESS ) !=1 ) { + info->err_reason =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_ERR_INFO ); + ret = -EIO ; + }else { + info->warn_info =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_WARN_INFO ); + } + + info->min_frame_buffer_count =vpu_read_reg_relaxed (inst->dev, +W6_RET_ENC_NUM_REQUIRED_FBC_FB ); + info->min_src_frame_count =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_MIN_SRC_BUF_NUM ); + info->max_latency_pictures =vpu_read_reg_relaxed (inst->dev, +W6_RET_ENC_PIC_MAX_LATENCY_PICTURES ); + info->req_mv_buffer_count =vpu_read_reg_relaxed (inst->dev, +W6_RET_ENC_NUM_REQUIRED_COL_BUF ); + +return ret; +} + +int wave6_vpu_enc_issue_seq_change (struct vpu_instance *inst,bool *changed) +{ +struct enc_cmd_change_param_reg reg = {}; +struct enc_info *p_enc_info; +int ret; + +guard (mutex)(&inst->dev->hw_lock); + + p_enc_info = &inst->codec_info->enc_info; + +wave6_gen_change_param_reg_common (inst, p_enc_info, ®); + +if (!reg.enable) +return 0 ; + +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_OPTION , +W6_SET_PARAM_OPT_CHANGE_PARAM ); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_ENABLE , reg.enable); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_SET_PARAM_RC_TARGET_RATE , reg.rc_target_rate); + +wave6_send_command (inst->dev, inst->id, inst->std,W6_CMD_ENC_SET_PARAM ); + ret =wave6_wait_vpu_busy (inst->dev,W6_VPU_BUSY_STATUS ); +if (ret) { +dev_warn (inst->dev->dev,"enc set param timed out\n" ); +return ret; + } + +if (!vpu_read_reg_relaxed (inst->dev,W6_RET_SUCCESS )) { +u32 reason_code =vpu_read_reg_relaxed (inst->dev,W6_RET_FAIL_REASON ); + +wave6_print_reg_err (inst->dev, reason_code); +return -EIO ; + } + + *changed = true; + +return 0 ; +} + +int wave6_vpu_enc_register_frame_buffer (struct vpu_instance *inst) +{ +struct enc_info *p_enc_info; +struct frame_buffer *fb_arr; +dma_addr_t fbc_cr_tbl_addr; +u32 fbc_count, mv_count; +int ret; + +guard (mutex)(&inst->dev->hw_lock); + + p_enc_info = &inst->codec_info->enc_info; + + fb_arr = &inst->frame_buf[0 ]; + fbc_count = p_enc_info->seq_info.min_frame_buffer_count; + mv_count = p_enc_info->seq_info.req_mv_buffer_count; +if (!fbc_count || fbc_count >MAX_SET_FB_GROUP_SIZE || + !mv_count || mv_count >MAX_SET_FB_GROUP_SIZE ) +return -EINVAL ; + +if (!wave6_check_aux_vbuf (inst,AUX_BUF_FBC_Y_TBL , fbc_count) || + !wave6_check_aux_vbuf (inst,AUX_BUF_FBC_C_TBL , fbc_count) || + !wave6_check_aux_vbuf (inst,AUX_BUF_SUB_SAMPLE , fbc_count) || + !wave6_check_aux_vbuf (inst,AUX_BUF_MV_COL , mv_count)) +return -EINVAL ; + +vpu_write_reg_relaxed (inst->dev,W6_CMD_SET_FB_OPTION , +SET_FB_OPTION_END |SET_FB_OPTION_START ); +vpu_write_reg_relaxed (inst->dev,W6_CMD_SET_FB_PIC_INFO , +SET_FB_PIC_INFO_STRIDE (fb_arr[0 ].stride)); +vpu_write_reg_relaxed (inst->dev,W6_CMD_SET_FB_PIC_SIZE , +SET_FB_PIC_SIZE_WIDTH (fb_arr[0 ].stride) | +SET_FB_PIC_SIZE_HEIGHT (fb_arr[0 ].height)); +vpu_write_reg_relaxed (inst->dev,W6_CMD_SET_FB_NUM , +SET_FB_NUM_FBC_END_IDX (fbc_count -1 ) | +SET_FB_NUM_MV_COL_END_IDX (mv_count -1 )); +vpu_write_reg_relaxed (inst->dev,W6_CMD_SET_FB_FBC_STRIDE , +SET_FB_FBC_STRIDE_L (fb_arr[0 ].stride) | +SET_FB_FBC_STRIDE_C (fb_arr[0 ].stride_c)); +vpu_write_reg_relaxed (inst->dev,W6_CMD_SET_FB_DEFAULT_CDF ,0 ); + +for (u32 idx =0 ; idx < fbc_count; idx++) { +vpu_write_reg_relaxed (inst->dev,W6_CMD_SET_FB_FBC_Y (idx), fb_arr[idx].buf_y); +vpu_write_reg_relaxed (inst->dev,W6_CMD_SET_FB_FBC_C (idx), fb_arr[idx].buf_cb); +vpu_write_reg_relaxed (inst->dev,W6_CMD_SET_FB_FBC_CR (idx), fb_arr[idx].buf_cr); +vpu_write_reg_relaxed (inst->dev,W6_CMD_SET_FB_FBC_Y_OFFSET (idx), + inst->aux_vbuf[AUX_BUF_FBC_Y_TBL ][idx].daddr); +vpu_write_reg_relaxed (inst->dev,W6_CMD_SET_FB_FBC_C_OFFSET (idx), + inst->aux_vbuf[AUX_BUF_FBC_C_TBL ][idx].daddr); + fbc_cr_tbl_addr = inst->aux_vbuf[AUX_BUF_FBC_C_TBL ][idx].daddr + + (inst->aux_vbuf[AUX_BUF_FBC_C_TBL ][idx].size >>1 ); +vpu_write_reg_relaxed (inst->dev,W6_CMD_SET_FB_FBC_CR_OFFSET (idx), + fbc_cr_tbl_addr); +vpu_write_reg_relaxed (inst->dev,W6_CMD_SET_FB_SUB_SAMPLED (idx), + inst->aux_vbuf[AUX_BUF_SUB_SAMPLE ][idx].daddr); + } +for (u32 idx =0 ; idx < mv_count; idx++) +vpu_write_reg_relaxed (inst->dev,W6_CMD_SET_FB_MV_COL (idx), + inst->aux_vbuf[AUX_BUF_MV_COL ][idx].daddr); + +wave6_send_command (inst->dev, inst->id, inst->std,W6_CMD_SET_FB ); + ret =wave6_wait_vpu_busy (inst->dev,W6_VPU_BUSY_STATUS ); +if (ret) { +dev_err (inst->dev->dev,"%s: timeout\n" , __func__); +return ret; + } + +if (!vpu_read_reg_relaxed (inst->dev,W6_RET_SUCCESS )) +return -EIO ; + +return 0 ; +} + +struct enc_cmd_enc_pic_reg { +u32 bs_start; +u32 bs_size; +u32 bs_option; +u32 sec_axi; +u32 report; +u32 mv_histo0; +u32 mv_histo1; +u32 custom_map_param; +u32 custom_map_addr; +u32 src_pic_idx; +u32 src_addr_y; +u32 src_addr_u; +u32 src_addr_v; +u32 src_stride; +u32 src_fmt; +u32 src_axi_sel; +u32 code_option; +u32 param; +u32 longterm_pic; +u32 prefix_sei_nal_addr; +u32 prefix_sei_info; +u32 suffix_sei_nal_addr; +u32 suffix_sei_info; +u32 timestamp_low; +u32 timestamp_high; +u32 csc_coeff[MAX_CSC_COEFF_NUM ]; +}; + +static u32 wave6_enc_src_fmt (enum frame_buffer_format fmt) +{ +bool is_csc_format = false; +bool is_packed = false; +bool is_24bit = false; +bool is_10bit = false; +bool is_ayuv = false; + +switch (fmt) { +case FORMAT_YUYV : + is_packed = true; +break ; +case FORMAT_420_P10_16BIT_MSB : + is_10bit = true; +break ; +case FORMAT_RGB_24BIT_PACKED : + is_24bit = true; + is_csc_format = true; +break ; +case FORMAT_RGB_32BIT_PACKED : + is_csc_format = true; +break ; +case FORMAT_RGB_P10_32BIT_PACKED : + is_csc_format = true; + is_10bit = true; +break ; +case FORMAT_YUV444_24BIT_PACKED : + is_24bit = true; + is_ayuv = true; + is_csc_format = true; +break ; +case FORMAT_YUV444_24BIT : + is_ayuv = true; +break ; +default : +break ; + } + +return ENC_PIC_SRC_FMT_C_FMT_IDC (get_chroma_format_idc (fmt)) | +ENC_PIC_SRC_FMT_CSC_24BIT (is_24bit) | +ENC_PIC_SRC_FMT_CSC_AYUV (is_ayuv) | +ENC_PIC_SRC_FMT_CSC_ENABLE (is_csc_format) | +ENC_PIC_SRC_FMT_10BIT_ENABLE (is_10bit) | +ENC_PIC_SRC_FMT_YUV422_PACKED (is_packed); +} + +static void wave6_gen_enc_pic_reg (struct enc_info *p_enc_info,bool cbcr_interleave,bool nv21, +struct enc_param *opt,struct enc_cmd_enc_pic_reg *reg) +{ +struct enc_open_param open = p_enc_info->open_param; +struct enc_codec_param param = open.codec_param; +u32 stride_c = opt->source_frame->stride_c; +u32 c_fmt_idc; + + c_fmt_idc =get_chroma_format_idc (open.src_fmt); +if ((c_fmt_idc ==C_FMT_IDC_YUV422 || c_fmt_idc ==C_FMT_IDC_YUV444 ) && + p_enc_info->c_fmt_idc ==C_FMT_IDC_YUV420 ) + stride_c *=2 ; + + reg->src_fmt =wave6_enc_src_fmt (open.src_fmt) | +ENC_PIC_SRC_FMT_CSC_FMT_ORDER (opt->csc.fmt_order) | +ENC_PIC_SRC_FMT_ENDIAN (open.source_endian) | +ENC_PIC_SRC_FMT_CBCR_ORDER (nv21) | +ENC_PIC_SRC_FMT_2PLANE (cbcr_interleave); + reg->bs_start = opt->pic_stream_buffer_addr; + reg->bs_size = opt->pic_stream_buffer_size; + reg->sec_axi =ENC_PIC_SEC_AXI_RDO_ENABLE (p_enc_info->sec_axi.use_enc_rdo) | +ENC_PIC_SEC_AXI_LF_ENABLE (p_enc_info->sec_axi.use_enc_lf); + reg->report =ENC_PIC_REPORT_MV_HISTO_ENABLE (param.en_report_mv_histo); + reg->mv_histo0 =ENC_PIC_MV_HISTO0_THRESHOLD0 (param.mv_histo_th0) | +ENC_PIC_MV_HISTO0_THRESHOLD1 (param.mv_histo_th1); + reg->mv_histo1 =ENC_PIC_MV_HISTO1_THRESHOLD2 (param.mv_histo_th2) | +ENC_PIC_MV_HISTO1_THRESHOLD3 (param.mv_histo_th3); + reg->src_pic_idx = opt->src_idx; + reg->src_addr_y = opt->source_frame->buf_y; + reg->src_addr_u = opt->source_frame->buf_cb; + reg->src_addr_v = opt->source_frame->buf_cr; + reg->src_stride =ENC_PIC_SRC_STRIDE_L (opt->source_frame->stride) | +ENC_PIC_SRC_STRIDE_C (stride_c); + reg->src_axi_sel =ENC_PIC_SRC_AXI_SEL_PRP_PORT ; + reg->code_option =ENC_PIC_CODE_OPTION_ENCODE_VCL | +ENC_PIC_CODE_OPTION_ENCODE_HEADER ; + reg->param =ENC_PIC_PARAM_INTRA_4X4 (param.intra_4x4) | +ENC_PIC_PARAM_FORCE_PIC_TYPE (opt->force_pic_type) | +ENC_PIC_PARAM_FORCE_PIC_TYPE_ENABLE (opt->force_pic) | +ENC_PIC_PARAM_FORCE_PIC_QP_B (opt->force_pic_qp_b) | +ENC_PIC_PARAM_FORCE_PIC_QP_P (opt->force_pic_qp_p) | +ENC_PIC_PARAM_FORCE_PIC_QP_I (opt->force_pic_qp_i) | +ENC_PIC_PARAM_FORCE_PIC_QP_ENABLE (opt->force_pic_qp) | +ENC_PIC_PARAM_PIC_SKIP_FLAG (opt->skip_picture); + reg->timestamp_low =lower_32_bits (opt->timestamp); + reg->timestamp_high =upper_32_bits (opt->timestamp); + reg->csc_coeff[0 ] =ENC_PIC_CSC_COEFF0_RY (opt->csc.coef_ry) | +ENC_PIC_CSC_COEFF0_GY (opt->csc.coef_gy) | +ENC_PIC_CSC_COEFF0_BY (opt->csc.coef_by); + reg->csc_coeff[1 ] =ENC_PIC_CSC_COEFF1_RCB (opt->csc.coef_rcb) | +ENC_PIC_CSC_COEFF1_GCB (opt->csc.coef_gcb) | +ENC_PIC_CSC_COEFF1_BCB (opt->csc.coef_bcb); + reg->csc_coeff[2 ] =ENC_PIC_CSC_COEFF2_RCR (opt->csc.coef_rcr) | +ENC_PIC_CSC_COEFF2_GCR (opt->csc.coef_gcr) | +ENC_PIC_CSC_COEFF2_BCR (opt->csc.coef_bcr); + reg->csc_coeff[3 ] =ENC_PIC_CSC_COEFF3_OFFSET_Y (opt->csc.offset_y) | +ENC_PIC_CSC_COEFF3_OFFSET_CB (opt->csc.offset_cb) | +ENC_PIC_CSC_COEFF3_OFFSET_CR (opt->csc.offset_cr); +} + +int wave6_vpu_enc_start_one_frame (struct vpu_instance *inst,struct enc_param *param, +u32 *fail_res) +{ +struct enc_cmd_enc_pic_reg reg = {}; +struct enc_info *p_enc_info; +int ret; + +guard (mutex)(&inst->dev->hw_lock); + + p_enc_info = &inst->codec_info->enc_info; + +wave6_gen_enc_pic_reg (p_enc_info, inst->cbcr_interleave, + inst->nv21, param, ®); + +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_PIC_BS_START , reg.bs_start); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_PIC_BS_SIZE , reg.bs_size); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_PIC_BS_OPTION , reg.bs_option); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_PIC_SEC_AXI , reg.sec_axi); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_PIC_REPORT , reg.report); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_PIC_MV_HISTO0 , reg.mv_histo0); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_PIC_MV_HISTO1 , reg.mv_histo1); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_PIC_CUSTOM_MAP_PARAM , reg.custom_map_param); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_PIC_CUSTOM_MAP_ADDR , reg.custom_map_addr); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_PIC_SRC_PIC_IDX , reg.src_pic_idx); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_PIC_SRC_ADDR_Y , reg.src_addr_y); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_PIC_SRC_ADDR_U , reg.src_addr_u); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_PIC_SRC_ADDR_V , reg.src_addr_v); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_PIC_SRC_STRIDE , reg.src_stride); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_PIC_SRC_FMT , reg.src_fmt); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_PIC_SRC_AXI_SEL , reg.src_axi_sel); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_PIC_CODE_OPTION , reg.code_option); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_PIC_PARAM , reg.param); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_PIC_LONGTERM_PIC , reg.longterm_pic); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_PIC_PREFIX_SEI_NAL_ADDR , + reg.prefix_sei_nal_addr); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_PIC_PREFIX_SEI_INFO , reg.prefix_sei_info); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_PIC_SUFFIX_SEI_NAL_ADDR , + reg.suffix_sei_nal_addr); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_PIC_SUFFIX_SEI_INFO , reg.suffix_sei_info); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_PIC_TIMESTAMP_LOW , reg.timestamp_low); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_PIC_TIMESTAMP_HIGH , reg.timestamp_high); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_PIC_CSC_COEFF0 , reg.csc_coeff[0 ]); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_PIC_CSC_COEFF1 , reg.csc_coeff[1 ]); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_PIC_CSC_COEFF2 , reg.csc_coeff[2 ]); +vpu_write_reg_relaxed (inst->dev,W6_CMD_ENC_PIC_CSC_COEFF3 , reg.csc_coeff[3 ]); + +wave6_send_command (inst->dev, inst->id, inst->std,W6_CMD_ENC_PIC ); + ret =wave6_wait_vpu_busy (inst->dev,W6_VPU_BUSY_STATUS ); +if (ret) { +dev_err (inst->dev->dev,"%s: timeout\n" , __func__); +return -ETIMEDOUT ; + } + +if (!vpu_read_reg_relaxed (inst->dev,W6_RET_SUCCESS )) { + *fail_res =vpu_read_reg_relaxed (inst->dev,W6_RET_FAIL_REASON ); +wave6_print_reg_err (inst->dev, *fail_res); +return -EIO ; + } + +return 0 ; +} + +int wave6_vpu_enc_get_output_info (struct vpu_instance *inst,struct enc_output_info *info) +{ +u32 reg_val; +int ret; + +if (WARN_ON (!info)) +return -EINVAL ; + +guard (mutex)(&inst->dev->hw_lock); + + ret =wave6_send_query (inst->dev, inst->id, inst->std,W6_QUERY_OPT_GET_RESULT ); +if (ret) +return ret; + + info->encoding_success =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_ENCODING_SUCCESS ); +if (!info->encoding_success) + info->error_reason =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_ERR_INFO ); +else + info->warn_info =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_WARN_INFO ); + + info->enc_pic_cnt =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_PIC_NUM ); + info->pic_type =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_PIC_TYPE ); + info->enc_vcl_nut =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_VCL_NUT ); + info->non_ref_pic =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_PIC_NON_REF_PIC_FLAG ); + info->num_of_slices =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_PIC_SLICE_NUM ); + info->pic_skipped =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_PIC_SKIP ); + info->num_of_intra =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_PIC_NUM_INTRA ); + info->num_of_merge =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_PIC_NUM_MERGE ); + info->num_of_skip_block =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_PIC_NUM_SKIP ); + info->avg_ctu_qp =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_PIC_AVG_CTU_QP ); + info->enc_pic_byte =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_PIC_BYTE ); + info->enc_gop_pic_idx =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_GOP_PIC_IDX ); + info->enc_pic_poc =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_PIC_POC ); + info->enc_src_idx =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_USED_SRC_IDX ); + info->wr_ptr =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_WR_PTR ); + info->rd_ptr =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_RD_PTR ); + info->bitstream_buffer =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_RD_PTR ); + info->pic_distortion_low =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_PIC_DIST_LOW ); + info->pic_distortion_high =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_PIC_DIST_HIGH ); + info->mv_histo.cnt0 =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_HISTO_CNT0 ); + info->mv_histo.cnt1 =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_HISTO_CNT1 ); + info->mv_histo.cnt2 =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_HISTO_CNT2 ); + info->mv_histo.cnt3 =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_HISTO_CNT3 ); + info->mv_histo.cnt4 =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_HISTO_CNT4 ); + info->fme_sum.lower_x0 =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_SUM_ME0_X_DIR_LOWER ); + info->fme_sum.higher_x0 =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_SUM_ME0_X_DIR_HIGHER ); + info->fme_sum.lower_y0 =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_SUM_ME0_Y_DIR_LOWER ); + info->fme_sum.higher_y0 =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_SUM_ME0_Y_DIR_HIGHER ); + info->fme_sum.lower_x1 =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_SUM_ME1_X_DIR_LOWER ); + info->fme_sum.higher_x1 =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_SUM_ME1_X_DIR_HIGHER ); + info->fme_sum.lower_y1 =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_SUM_ME1_Y_DIR_LOWER ); + info->fme_sum.higher_y1 =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_SUM_ME1_Y_DIR_HIGHER ); + info->src_y_addr =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_SRC_Y_ADDR ); + info->custom_map_addr =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_CUSTOM_MAP_ADDR ); + info->prefix_sei_nal_addr =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_PREFIX_SEI_NAL_ADDR ); + info->suffix_sei_nal_addr =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_SUFFIX_SEI_NAL_ADDR ); + + info->recon_frame_index =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_PIC_IDX ); +if (info->recon_frame_index ==RECON_IDX_FLAG_HEADER_ONLY ) + info->bitstream_size = info->enc_pic_byte; +else if (info->recon_frame_index <0 ) + info->bitstream_size =0 ; +else + info->bitstream_size = info->enc_pic_byte; + + reg_val =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_TIMESTAMP_LOW ); + info->timestamp =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_TIMESTAMP_HIGH ); + info->timestamp = (info->timestamp <<32 ) | reg_val; + +return 0 ; +} + +int wave6_vpu_enc_fini_seq (struct vpu_instance *inst,u32 *fail_res) +{ +int ret; + +lockdep_assert_held (&inst->dev->hw_lock); + +wave6_send_command (inst->dev, inst->id, inst->std,W6_CMD_DESTROY_INSTANCE ); + ret =wave6_wait_vpu_busy (inst->dev,W6_VPU_BUSY_STATUS ); +if (ret) +return -ETIMEDOUT ; + +if (!vpu_read_reg_relaxed (inst->dev,W6_RET_SUCCESS )) { + *fail_res =vpu_read_reg_relaxed (inst->dev,W6_RET_FAIL_REASON ); +wave6_print_reg_err (inst->dev, *fail_res); +return -EIO ; + } + +return 0 ; +}
drivers/media/platform/chips-media/wave6/wave6-hw.h +7 -0
@@ -32,6 +32,8 @@ enum product_id {#define WTL_PIXEL_16BIT 1#define WTL_PIXEL_32BIT 2 +#define MAX_CSC_COEFF_NUM 4 +#define H265_NAL_UNIT_TYPE_IDR_W_RADL 19#define H265_NAL_UNIT_TYPE_IDR_N_LP 20
@@ -46,4 +48,9 @@ int wave6_vpu_dec_get_seq_info(struct vpu_instance *inst, struct dec_seq_info *iint wave6_vpu_dec_fini_seq (struct vpu_instance *inst,u32 *fail_res);dma_addr_t wave6_vpu_dec_get_rd_ptr (struct vpu_instance *inst); +int wave6_vpu_build_up_enc_param (struct device *dev,struct vpu_instance *inst, +struct enc_open_param *param); +int wave6_vpu_enc_get_seq_info (struct vpu_instance *inst,struct enc_seq_info *info); +int wave6_vpu_enc_fini_seq (struct vpu_instance *inst,u32 *fail_res); +#endif /* __WAVE6_HW_H__ */
drivers/media/platform/chips-media/wave6/wave6-regdefine.h +312 -0
364 lines not on this page · raw message
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c +2 -1
@@ -92,7 +92,8 @@ static irqreturn_t wave6_vpu_core_irq_thread(int irq, void *dev_id)if (!inst)continue ; -if (irq_info.status &BIT (W6_INT_BIT_INIT_SEQ )) { +if ((irq_info.status &BIT (W6_INT_BIT_INIT_SEQ )) || + (irq_info.status &BIT (W6_INT_BIT_ENC_SET_PARAM ))) {complete (&inst->irq_done);continue ; }
drivers/media/platform/chips-media/wave6/wave6-vpuapi.c +131 -0
134 lines not on this page · raw message
drivers/media/platform/chips-media/wave6/wave6-vpuapi.h +428 -0
539 lines not on this page · raw message
drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h +14 -0
@@ -30,6 +30,17 @@#define W6_MAX_DEC_PIC_HEIGHT 4096#define W6_DEC_PIC_SIZE_STEP 1 +#define W6_DEF_ENC_PIC_WIDTH 416 +#define W6_DEF_ENC_PIC_HEIGHT 240 +#define W6_MIN_ENC_PIC_WIDTH 256 +#define W6_MIN_ENC_PIC_HEIGHT 128 +#define W6_MAX_ENC_PIC_WIDTH 4096 +#define W6_MAX_ENC_PIC_HEIGHT 4096 +#define W6_ENC_PIC_SIZE_STEP 8 +#define W6_ENC_CROP_X_POS_STEP 32 +#define W6_ENC_CROP_Y_POS_STEP 8 +#define W6_ENC_CROP_STEP 2 +#define W6_VPU_POLL_DELAY_US 10#define W6_VPU_POLL_TIMEOUT 300000#define W6_BOOT_WAIT_TIMEOUT 10000
@@ -38,13 +49,16 @@#define HOST_ENDIAN VDI_128BIT_LITTLE_ENDIAN#define VPU_STREAM_ENDIAN HOST_ENDIAN +#define VPU_USER_DATA_ENDIAN HOST_ENDIAN#define VPU_SOURCE_ENDIAN HOST_ENDIAN#define W6_MAX_REMAP_PAGE_SIZE (1024 * 1024) +#define W6_ARBUF_SIZE (1024)#define W6_MAX_CODE_BUF_SIZE (4 * 1024 * 1024)#define W6_CODE_BUF_SIZE (1 * 1024 * 1024)#define W6_EXTRA_CODE_BUF_SIZE (256 * 1024)#define W6_TEMPBUF_SIZE (3 * 1024 * 1024) +#define W627ENC_WORKBUF_SIZE (512 * 1024)#define W637DEC_WORKBUF_SIZE (2 * 512 * 1024)#define W637DEC_WORKBUF_SIZE_FOR_CQ (3 * 512 * 1024)
Signature · 1 line
2.41.0.windows.3
PATCH v8 7/11 media: chips-media: wave6: Add Wave6 m2m encoder
In reply to Nas ChungAdd the V4L2 m2m stateful encoder for the Chips&Media Wave6 video codec IP.
The encoder supports the following features and formats:
- Supported codecs: AVC, HEVC (8-bit streams only)
- Supported raw (OUTPUT) formats:
* V4L2_PIX_FMT_YUV420/NV12/NV21
* V4L2_PIX_FMT_YUV420M/NV12M/NV21M
* V4L2_PIX_FMT_YUV422P/NV16/NV61
* V4L2_PIX_FMT_YUV422M/NV16M/NV61M
* V4L2_PIX_FMT_YUYV
* V4L2_PIX_FMT_YUV24/NV24/NV42
* V4L2_PIX_FMT_P010
* V4L2_PIX_FMT_RGB24
* V4L2_PIX_FMT_ARGB32/XRGB32/RGBA32/RGBX32
* V4L2_PIX_FMT_ARGB2101010
- Although 10-bit raw formats are accepted (P010, ARGB2101010), the encoded stream is limited to 8-bit
- Supports crop
- Supports rotation and horizontal/vertical flip, except for YUV 4:4:4 and RGB input
- Supports the V4L2 encoder controls
While other versions of the Wave6 IP may support AV1 encoding, it is not implemented or validated in this driver at this time.
--- .../media/platform/chips-media/wave6/Kconfig | 2 +- .../media/platform/chips-media/wave6/Makefile | 1 + .../chips-media/wave6/wave6-vpu-core.c | 20 +- .../chips-media/wave6/wave6-vpu-core.h | 12 + .../chips-media/wave6/wave6-vpu-enc.c | 2732 +++++++++++++++++ .../chips-media/wave6/wave6-vpu-v4l2.c | 22 + .../platform/chips-media/wave6/wave6-vpuapi.h | 2 + 7 files changed, 2789 insertions(+), 2 deletions(-) create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c
drivers/media/platform/chips-media/wave6/Kconfig +1 -1
@@ -10,7 +10,7 @@ config VIDEO_WAVE6_VPU select GENERIC_ALLOCATOR help Chips&Media Wave6 stateful codec driver. - It provides decoding capabilities for H.264, HEVC, + It provides encoding and decoding capabilities for H.264, HEVC, and other video formats. To compile this driver as a module, choose M here: the module will be called wave6.
drivers/media/platform/chips-media/wave6/Makefile +1 -0
@@ -5,5 +5,6 @@ wave6-objs += wave6-vpu.o \ wave6-vpu-v4l2.o \ wave6-vpuapi.o \ wave6-vpu-dec.o \ + wave6-vpu-enc.o \ wave6-hw.o obj-$(CONFIG_VIDEO_WAVE6_VPU) += wave6.o
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c +19 -1
@@ -187,6 +187,11 @@ static void wave6_vpu_core_unregister_dec(void *data)wave6_vpu_dec_unregister_device (data); } +static void wave6_vpu_core_unregister_enc (void *data) +{ +wave6_vpu_enc_unregister_device (data); +} +static int wave6_vpu_core_probe (struct platform_device *pdev) {struct vpu_core_device *core;
@@ -296,8 +301,21 @@ static int wave6_vpu_core_probe(struct platform_device *pdev)if (ret)return ret; } +if (core->res->codec_types &WAVE6_IS_ENC ) { + ret =wave6_vpu_enc_register_device (core); +if (ret) { +dev_err (&pdev->dev, +"failed to register video_dev_enc: %d\n" , ret); +return ret; + } + + ret =devm_add_action_or_reset (&pdev->dev, wave6_vpu_core_unregister_enc, core); +if (ret) +return ret; + } -dev_dbg (&pdev->dev,"Added wave6 driver with caps %s\n" , +dev_dbg (&pdev->dev,"Added wave6 driver with caps %s %s\n" , + core->res->codec_types &WAVE6_IS_ENC ?"'ENCODE'" :"" , core->res->codec_types &WAVE6_IS_DEC ?"'DECODE'" :"" );return 0 ;
drivers/media/platform/chips-media/wave6/wave6-vpu-core.h +12 -0
@@ -27,6 +27,12 @@ struct vpu_buffer {bool consumed;bool used;bool error; +bool force_key_frame; +bool force_frame_qp; +u32 force_i_frame_qp; +u32 force_p_frame_qp; +u32 force_b_frame_qp; +u32 average_qp; };enum vpu_fmt_type {
@@ -36,6 +42,9 @@ enum vpu_fmt_type {#define VPU_FMT_FLAG_CBCR_INTERLEAVED BIT(0)#define VPU_FMT_FLAG_CRCB_ORDER BIT(1) +#define VPU_FMT_FLAG_10BIT BIT(2) +#define VPU_FMT_FLAG_RGB BIT(3) +#define VPU_FMT_FLAG_SUPPORT_ROT_MIR BIT(4)struct vpu_format {unsigned int v4l2_pix_fmt;
@@ -46,6 +55,7 @@ struct vpu_format {unsigned int num_planes;enum frame_buffer_format fb_fmt;enum endian_mode endian; +enum csc_format_order csc_fmt_order;unsigned int flags; };
@@ -94,6 +104,8 @@ void wave6_vpu_set_instance_state(struct vpu_instance *inst,int wave6_vpu_wait_interrupt (struct vpu_instance *inst,unsigned int timeout);int wave6_vpu_dec_register_device (struct vpu_core_device *core);void wave6_vpu_dec_unregister_device (struct vpu_core_device *core); +int wave6_vpu_enc_register_device (struct vpu_core_device *core); +void wave6_vpu_enc_unregister_device (struct vpu_core_device *core);void wave6_vpu_device_run_timeout (struct work_struct *work);void wave6_vpu_finish_job (struct vpu_instance *inst);int wave6_vpu_init_m2m_dev (struct vpu_core_device *core);
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c new file +2732 -0
2,732 lines not on this page · raw message
drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c +22 -0
@@ -131,6 +131,12 @@ static bool wave6_vpu_valid_instance_transition(struct vpu_instance *inst, next == VPU_INST_STATE_INIT_SEQ || next == VPU_INST_STATE_SEEK)return true; + }else { +/* to NONE: instance destroyed */ +/* to INIT_SEQ: encoder parameter configured */ +if (next ==VPU_INST_STATE_NONE || + next ==VPU_INST_STATE_INIT_SEQ ) +return true; }break ;case VPU_INST_STATE_INIT_SEQ :
@@ -142,6 +148,12 @@ static bool wave6_vpu_valid_instance_transition(struct vpu_instance *inst, next == VPU_INST_STATE_PIC_RUN || next == VPU_INST_STATE_SEEK)return true; + }else { +/* to NONE: instance destroyed */ +/* to PIC_RUN: frame buffers registered */ +if (next ==VPU_INST_STATE_NONE || + next ==VPU_INST_STATE_PIC_RUN ) +return true; }break ;case VPU_INST_STATE_PIC_RUN :
@@ -155,6 +167,12 @@ static bool wave6_vpu_valid_instance_transition(struct vpu_instance *inst, next == VPU_INST_STATE_SEEK || next == VPU_INST_STATE_ERROR)return true; + }else { +/* to NONE: instance destroyed */ +/* to ERROR: encode command failed, encode frame failed */ +if (next ==VPU_INST_STATE_NONE || + next ==VPU_INST_STATE_ERROR ) +return true; }break ;case VPU_INST_STATE_SEEK :
@@ -177,6 +195,10 @@ static bool wave6_vpu_valid_instance_transition(struct vpu_instance *inst,if (next ==VPU_INST_STATE_NONE || next ==VPU_INST_STATE_SEEK )return true; + }else { +/* to NONE: instance destroyed */ +if (next ==VPU_INST_STATE_NONE ) +return true; }break ; }
drivers/media/platform/chips-media/wave6/wave6-vpuapi.h +2 -0
@@ -784,6 +784,7 @@ struct vpu_irq {* @instances: List of VPU instances * @inst_lock: Spinlock protecting instance list * @video_dev_dec: Video device node for decoder +* @video_dev_enc: Video device node for encoder * @dev_lock: Mutex protecting video_device, src, dst vb2_queue * @hw_lock: Mutex protecting register access * @attr: Hardware attributes retrieved after boot
@@ -806,6 +807,7 @@ struct vpu_core_device {struct list_head instances;spinlock_t inst_lock;/* Protects instance list */ struct video_device *video_dev_dec; +struct video_device *video_dev_enc;struct mutex dev_lock;/* Protects video_device, src, dst vb2_queue */ struct mutex hw_lock;/* Protects register access */ struct vpu_attr attr;
Signature · 1 line
2.41.0.windows.3
PATCH v8 9/11 media: chips-media: wave6: Add performance profiling
In reply to Nas ChungAdd per-instance performance profiling to help identify whether a throughput problem comes from the firmware or from the host side.
Record per-frame timestamps and the firmware-reported processing time, and report the frame rates and latencies in the per-instance debugfs file.
--- .../platform/chips-media/wave6/wave6-hw.c | 28 ++++++ .../platform/chips-media/wave6/wave6-hw.h | 2 + .../chips-media/wave6/wave6-regdefine.h | 6 ++ .../chips-media/wave6/wave6-vpu-core.h | 10 ++ .../chips-media/wave6/wave6-vpu-dbg.c | 36 ++++++++ .../chips-media/wave6/wave6-vpu-dec.c | 15 +++ .../chips-media/wave6/wave6-vpu-enc.c | 13 +++ .../chips-media/wave6/wave6-vpu-v4l2.c | 91 +++++++++++++++++++ .../platform/chips-media/wave6/wave6-vpuapi.h | 28 ++++++ 9 files changed, 229 insertions(+)
drivers/media/platform/chips-media/wave6/wave6-hw.c +28 -0
@@ -872,6 +872,20 @@ int wave6_vpu_dec_get_output_info(struct vpu_instance *inst, struct dec_output_i info->timestamp =vpu_read_reg_relaxed (inst->dev,W6_RET_DEC_TIMESTAMP_HIGH ); info->timestamp = (info->timestamp <<32 ) | reg_val; + info->cycle.host_cmd_s =vpu_read_reg_relaxed (inst->dev,W6_RET_CQ_IN_TICK ); + info->cycle.host_cmd_e =vpu_read_reg_relaxed (inst->dev,W6_RET_RQ_OUT_TICK ); + info->cycle.proc_s =vpu_read_reg_relaxed (inst->dev,W6_RET_HW_RUN_TICK ); + info->cycle.proc_e =vpu_read_reg_relaxed (inst->dev,W6_RET_HW_DONE_TICK ); + info->cycle.vpu_s =vpu_read_reg_relaxed (inst->dev,W6_RET_FW_RUN_TICK ); + info->cycle.vpu_e =vpu_read_reg_relaxed (inst->dev,W6_RET_FW_DONE_TICK ); + info->cycle.frame_cycle = (info->cycle.vpu_e - info->cycle.host_cmd_s) * +CYCLE_PER_TICK_VALUE ; + info->cycle.proc_cycle = (info->cycle.proc_e - info->cycle.proc_s) * +CYCLE_PER_TICK_VALUE ; + info->cycle.vpu_cycle = ((info->cycle.vpu_e - info->cycle.vpu_s) - + (info->cycle.proc_e - info->cycle.proc_s)) * +CYCLE_PER_TICK_VALUE ; +if (decoded_idx >=0 && decoded_idx <WAVE6_MAX_FBS ) p_dec_info->dec_out_info[decoded_idx].decoded_poc = info->decoded_poc;
@@ -1882,6 +1896,20 @@ int wave6_vpu_enc_get_output_info(struct vpu_instance *inst, struct enc_output_i info->timestamp =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_TIMESTAMP_HIGH ); info->timestamp = (info->timestamp <<32 ) | reg_val; + info->cycle.host_cmd_s =vpu_read_reg_relaxed (inst->dev,W6_RET_CQ_IN_TICK ); + info->cycle.host_cmd_e =vpu_read_reg_relaxed (inst->dev,W6_RET_RQ_OUT_TICK ); + info->cycle.proc_s =vpu_read_reg_relaxed (inst->dev,W6_RET_HW_RUN_TICK ); + info->cycle.proc_e =vpu_read_reg_relaxed (inst->dev,W6_RET_HW_DONE_TICK ); + info->cycle.vpu_s =vpu_read_reg_relaxed (inst->dev,W6_RET_FW_RUN_TICK ); + info->cycle.vpu_e =vpu_read_reg_relaxed (inst->dev,W6_RET_FW_DONE_TICK ); + info->cycle.frame_cycle = (info->cycle.vpu_e - info->cycle.host_cmd_s) * +CYCLE_PER_TICK_VALUE ; + info->cycle.proc_cycle = (info->cycle.proc_e - info->cycle.proc_s) * +CYCLE_PER_TICK_VALUE ; + info->cycle.vpu_cycle = ((info->cycle.vpu_e - info->cycle.vpu_s) - + (info->cycle.proc_e - info->cycle.proc_s)) * +CYCLE_PER_TICK_VALUE ; +return 0 ; }
drivers/media/platform/chips-media/wave6/wave6-hw.h +2 -0
@@ -34,6 +34,8 @@ enum product_id {#define MAX_CSC_COEFF_NUM 4 +#define CYCLE_PER_TICK_VALUE 256 +#define H265_NAL_UNIT_TYPE_IDR_W_RADL 19#define H265_NAL_UNIT_TYPE_IDR_N_LP 20
drivers/media/platform/chips-media/wave6/wave6-regdefine.h +6 -0
@@ -110,6 +110,12 @@ enum wave6_interrupt_bit {#define W6_RET_INT_INSTANCE_INFO (W6_REG_BASE + 0x21C)#define INT_INSTANCE_INFO_CLEAR 0#define W6_RET_INSTANCE_ID (W6_REG_BASE + 0x220) +#define W6_RET_CQ_IN_TICK (W6_REG_BASE + 0x23C) +#define W6_RET_FW_RUN_TICK (W6_REG_BASE + 0x240) +#define W6_RET_HW_RUN_TICK (W6_REG_BASE + 0x244) +#define W6_RET_HW_DONE_TICK (W6_REG_BASE + 0x248) +#define W6_RET_FW_DONE_TICK (W6_REG_BASE + 0x24C) +#define W6_RET_RQ_OUT_TICK (W6_REG_BASE + 0x250)/* COMMON - QUERY : GET_VPU_INFO */ #define W6_RET_FW_VERSION (W6_REG_BASE + 0x300)
drivers/media/platform/chips-media/wave6/wave6-vpu-core.h +10 -0
@@ -32,6 +32,11 @@ struct vpu_buffer {u32 force_i_frame_qp;u32 force_p_frame_qp;u32 force_b_frame_qp; +ktime_t ts_input; +ktime_t ts_start; +ktime_t ts_finish; +ktime_t ts_output; +u64 hw_time;u32 average_qp; };
@@ -101,6 +106,7 @@ enum codec_std wave6_to_codec_std(enum vpu_instance_type type, unsigned int v4l2const char *wave6_vpu_instance_state_name (enum vpu_instance_state state);void wave6_vpu_set_instance_state (struct vpu_instance *inst,enum vpu_instance_state state); +u64 wave6_vpu_cycle_to_ns (struct vpu_core_device *core,u64 cycle);int wave6_vpu_wait_interrupt (struct vpu_instance *inst,unsigned int timeout);int wave6_vpu_dec_register_device (struct vpu_core_device *core);void wave6_vpu_dec_unregister_device (struct vpu_core_device *core);
@@ -108,6 +114,10 @@ int wave6_vpu_enc_register_device(struct vpu_core_device *core);void wave6_vpu_enc_unregister_device (struct vpu_core_device *core);void wave6_vpu_device_run_timeout (struct work_struct *work);void wave6_vpu_finish_job (struct vpu_instance *inst); +void wave6_vpu_record_performance_timestamps (struct vpu_instance *inst); +void wave6_vpu_handle_performance (struct vpu_instance *inst, +struct vpu_buffer *vpu_buf); +void wave6_vpu_reset_performance (struct vpu_instance *inst);int wave6_vpu_init_m2m_dev (struct vpu_core_device *core);void wave6_vpu_release_m2m_dev (struct vpu_core_device *core); int wave6_vpu_subscribe_event(struct v4l2_fh *fh,
drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.c +36 -0
@@ -15,7 +15,13 @@static int wave6_vpu_dbg_instance (struct seq_file *s,void *data) {struct vpu_instance *inst = s->private; +struct vpu_performance_info *perf = &inst->performance;struct vb2_queue *vq; +s64 tmp; +s64 fps; +s64 duration; +s64 ms[3 ]; +s32 ns[3 ];if (!inst->v4l2_fh.m2m_ctx)return 0 ;
@@ -77,6 +83,36 @@ static int wave6_vpu_dbg_instance(struct seq_file *s, void *data) inst->error_buf_num, v4l2_m2m_has_stopped(inst->v4l2_fh.m2m_ctx)); +seq_puts (s,"fps" ); + tmp =MSEC_PER_SEC * inst->processed_buf_num; +if (perf->ts_last > perf->ts_first +NSEC_PER_MSEC ) { + duration =ktime_ms_delta (perf->ts_last, perf->ts_first); + fps =DIV64_U64_ROUND_CLOSEST (tmp, duration); +seq_printf (s," actual: %lld;" , fps); + } + duration =div_u64 (perf->total_sw_time,NSEC_PER_MSEC ); +if (duration >0 ) { + fps =DIV64_U64_ROUND_CLOSEST (tmp, duration); +seq_printf (s," sw: %lld;" , fps); + } + duration =div_u64 (perf->total_hw_time,NSEC_PER_MSEC ); +if (duration >0 ) { + fps =DIV64_U64_ROUND_CLOSEST (tmp, duration); +seq_printf (s," hw: %lld" , fps); + } +seq_putc (s,'\n' ); + + ms[0 ] =div_s64_rem (perf->latency_first,NSEC_PER_MSEC , &ns[0 ]); + ms[1 ] =div_s64_rem (perf->latency_max,NSEC_PER_MSEC , &ns[1 ]); + ms[2 ] =div_s64_rem (ktime_sub (perf->ts_first, perf->ts_start),NSEC_PER_MSEC , &ns[2 ]); +seq_printf (s,"latency(ms) first: %lld.%06d, max %lld.%06d, setup %lld.%06d\n" , + ms[0 ], ns[0 ], ms[1 ], ns[1 ], ms[2 ], ns[2 ]); + + ms[0 ] =div_s64_rem (perf->min_process_time,NSEC_PER_MSEC , &ns[0 ]); + ms[1 ] =div_s64_rem (perf->max_process_time,NSEC_PER_MSEC , &ns[1 ]); +seq_printf (s,"process frame time(ms) min: %lld.%06d, max %lld.%06d\n" , + ms[0 ], ns[0 ], ms[1 ], ns[1 ]); +if (inst->type ==VPU_INST_TYPE_DEC ) {seq_printf (s,"%s order\n" , inst->disp_mode ==DISP_MODE_DISP_ORDER ?"display" :"decode" );
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c +15 -0
@@ -181,6 +181,7 @@ static void wave6_handle_bitstream_buffer(struct vpu_instance *inst,return ; } + vpu_buf->ts_start =ktime_get_raw (); vpu_buf->consumed = true;wave6_vpu_dec_set_rd_ptr (inst, rd_ptr, true);
@@ -304,6 +305,8 @@ static int wave6_vpu_dec_prepare_fb(struct vpu_instance *inst)wave6_vpu_set_instance_state (inst,VPU_INST_STATE_PIC_RUN ); +wave6_vpu_record_performance_timestamps (inst); +return 0 ; error:
@@ -606,6 +609,10 @@ static void wave6_handle_decoded_frame(struct vpu_instance *inst, dst_vpu_buf->used = true;if (state ==VB2_BUF_STATE_ERROR ) dst_vpu_buf->error = true; + dst_vpu_buf->ts_input = vpu_buf->ts_input; + dst_vpu_buf->ts_start = vpu_buf->ts_start; + dst_vpu_buf->ts_finish =ktime_get_raw (); + dst_vpu_buf->hw_time =wave6_vpu_cycle_to_ns (inst->dev, info->cycle.frame_cycle); }if (state ==VB2_BUF_STATE_ERROR ) {
@@ -658,6 +665,9 @@ static void wave6_handle_display_frame(struct vpu_instance *inst,vb2_set_plane_payload (&dst_buf->vb2_buf, i, inst->dst_fmt.plane_fmt[i].sizeimage); + vpu_buf->ts_output =ktime_get_raw (); +wave6_vpu_handle_performance (inst, vpu_buf); +if (vpu_buf->error) state =VB2_BUF_STATE_ERROR ; dst_buf->sequence = inst->sequence++;
@@ -1404,6 +1414,8 @@ static int wave6_vpu_dec_create_instance(struct vpu_instance *inst)memset (&open_param,0 ,sizeof (struct dec_open_param )); +wave6_vpu_record_performance_timestamps (inst); +wave6_vpu_core_activate (inst->dev); ret =pm_runtime_resume_and_get (inst->dev->dev);if (ret) {
@@ -1567,6 +1579,7 @@ static int wave6_vpu_dec_prepare_decode(struct vpu_instance *inst)static void wave6_vpu_dec_buf_queue_src (struct vb2_buffer *vb) {struct vb2_v4l2_buffer *vbuf =to_vb2_v4l2_buffer (vb); +struct vpu_buffer *vpu_buf =wave6_to_vpu_buf (vbuf);struct vpu_instance *inst =vb2_get_drv_priv (vb->vb2_queue); dev_dbg(inst->dev->dev, "type %4d index %4d size[0] %4ld size[1] : %4ld | size[2] : %4ld\n",
@@ -1574,6 +1587,7 @@ static void wave6_vpu_dec_buf_queue_src(struct vb2_buffer *vb) vb2_plane_size(&vbuf->vb2_buf, 1), vb2_plane_size(&vbuf->vb2_buf, 2)); vbuf->sequence = inst->queued_src_buf_num++; + vpu_buf->ts_input =ktime_get_raw ();v4l2_m2m_buf_queue (inst->v4l2_fh.m2m_ctx, vbuf); }
@@ -1670,6 +1684,7 @@ static void wave6_vpu_dec_stop_streaming(struct vb2_queue *q) goto exit;if (V4L2_TYPE_IS_OUTPUT (q->type)) { +wave6_vpu_reset_performance (inst); inst->queued_src_buf_num =0 ; inst->processed_buf_num =0 ; inst->error_buf_num =0 ;
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c +13 -0
@@ -791,6 +791,7 @@ static int wave6_vpu_enc_start_encode(struct vpu_instance *inst) src_vbuf =wave6_to_vpu_buf (src_buf); src_vbuf->consumed = true; + src_vbuf->ts_start =ktime_get_raw ();if (inst->src_fmt.num_planes ==1 ) { frame_buf.buf_y =wave6_get_dma_addr (src_buf,0 ); frame_buf.buf_cb = frame_buf.buf_y + luma_size;
@@ -887,6 +888,12 @@ static void wave6_handle_encoded_frame(struct vpu_instance *inst, dst_vpu_buf =wave6_to_vpu_buf (dst_buf); dst_vpu_buf->average_qp = info->avg_ctu_qp; + dst_vpu_buf->ts_input = vpu_buf->ts_input; + dst_vpu_buf->ts_start = vpu_buf->ts_start; + dst_vpu_buf->ts_finish =ktime_get_raw (); + dst_vpu_buf->hw_time =wave6_vpu_cycle_to_ns (inst->dev, info->cycle.frame_cycle); + dst_vpu_buf->ts_output =ktime_get_raw (); +wave6_vpu_handle_performance (inst, dst_vpu_buf);if (v4l2_m2m_is_last_draining_src_buf (inst->v4l2_fh.m2m_ctx, src_buf)) { dst_buf->flags |=V4L2_BUF_FLAG_LAST ;
@@ -2023,6 +2030,8 @@ static int wave6_vpu_enc_create_instance(struct vpu_instance *inst)memset (&open_param,0 ,sizeof (struct enc_open_param )); +wave6_vpu_record_performance_timestamps (inst); +wave6_vpu_core_activate (inst->dev); ret =pm_runtime_resume_and_get (inst->dev->dev);if (ret) {
@@ -2179,6 +2188,8 @@ static int wave6_vpu_enc_prepare_fb(struct vpu_instance *inst)wave6_vpu_set_instance_state (inst,VPU_INST_STATE_PIC_RUN ); +wave6_vpu_record_performance_timestamps (inst); +return 0 ; error:
@@ -2267,6 +2278,7 @@ static void wave6_vpu_enc_buf_queue(struct vb2_buffer *vb)if (V4L2_TYPE_IS_OUTPUT (vb->type)) { vbuf->sequence = inst->queued_src_buf_num++; + vpu_buf->ts_input =ktime_get_raw (); vpu_buf->force_key_frame = inst->enc_ctrls.force_key_frame; inst->enc_ctrls.force_key_frame = false; vpu_buf->force_frame_qp = !inst->enc_ctrls.frame_rc_enable;
@@ -2363,6 +2375,7 @@ static void wave6_vpu_enc_stop_streaming(struct vb2_queue *q) goto exit;if (V4L2_TYPE_IS_OUTPUT (q->type)) { +wave6_vpu_reset_performance (inst); inst->queued_src_buf_num =0 ; inst->processed_buf_num =0 ; inst->error_buf_num =0 ;
drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c +91 -0
109 lines not on this page · raw message
drivers/media/platform/chips-media/wave6/wave6-vpuapi.h +28 -0
@@ -339,6 +339,18 @@ struct instance_buffer {dma_addr_t ar_base; }; +struct report_cycle { +u32 host_cmd_s; +u32 host_cmd_e; +u32 proc_s; +u32 proc_e; +u32 vpu_s; +u32 vpu_e; +u32 frame_cycle; +u32 proc_cycle; +u32 vpu_cycle; +}; +struct color_param {u32 color_range;u32 matrix_coefficients;
@@ -423,6 +435,7 @@ struct dec_output_info {int error_reason;int warn_info;unsigned int sequence_no; +struct report_cycle cycle;dma_addr_t release_disp_frame_addr[WAVE6_MAX_FBS ];dma_addr_t disp_frame_addr[WAVE6_MAX_FBS ];u64 timestamp;
@@ -671,6 +684,7 @@ struct enc_output_info {bool encoding_success;struct enc_report_fme_sum fme_sum;struct enc_report_mv_histo mv_histo; +struct report_cycle cycle;u64 timestamp;dma_addr_t src_y_addr;dma_addr_t custom_map_addr;
@@ -839,6 +853,18 @@ struct vpu_instance_ops {void (*finish_process)(struct vpu_instance *inst,bool error); }; +struct vpu_performance_info { +ktime_t ts_start; +ktime_t ts_first; +ktime_t ts_last; +s64 latency_first; +s64 latency_max; +s64 min_process_time; +s64 max_process_time; +u64 total_sw_time; +u64 total_hw_time; +}; +/** * struct vpu_instance - VPU instance structure * @list: List head for VPU core device instance list
@@ -883,6 +909,7 @@ struct vpu_instance_ops {* @error_recovery: Error recovery flag * @disp_mode: Display mode for decoder V4L2 control * @enc_ctrls: Encoder V4L2 control and streaming parameters +* @performance: Performance information * @debugfs: Debugfs entry */ struct vpu_instance {
@@ -943,6 +970,7 @@ struct vpu_instance {enum display_mode disp_mode;struct enc_controls enc_ctrls; +struct vpu_performance_info performance;struct dentry *debugfs; };
Signature · 1 line
2.41.0.windows.3
PATCH v8 8/11 media: chips-media: wave6: Add trace and debugfs
In reply to Nas ChungAdd debugfs entries and trace events to provide detailed debugging information.
Help diagnose issues and improve debugging capabilities for the Wave6 core driver.
Print a readable reason for the known firmware system errors instead of only the raw fail reason code.
--- .../media/platform/chips-media/wave6/Makefile | 4 + .../platform/chips-media/wave6/wave6-hw.c | 36 ++- .../platform/chips-media/wave6/wave6-trace.h | 284 ++++++++++++++++++ .../chips-media/wave6/wave6-vpu-core.c | 14 + .../chips-media/wave6/wave6-vpu-dbg.c | 148 +++++++++ .../chips-media/wave6/wave6-vpu-dbg.h | 14 + .../chips-media/wave6/wave6-vpu-dec.c | 25 +- .../chips-media/wave6/wave6-vpu-enc.c | 27 +- .../chips-media/wave6/wave6-vpu-v4l2.c | 3 + .../platform/chips-media/wave6/wave6-vpuapi.h | 5 + 10 files changed, 551 insertions(+), 9 deletions(-) create mode 100644 drivers/media/platform/chips-media/wave6/wave6-trace.h create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.c create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.h
drivers/media/platform/chips-media/wave6/Makefile +4 -0
@@ -1,8 +1,12 @@ # SPDX-License-Identifier: GPL-2.0 +# tell define_trace.h where to find the trace header +CFLAGS_wave6-vpu-core.o := -I$(src) + wave6-objs += wave6-vpu.o \ wave6-vpu-core.o \ wave6-vpu-v4l2.o \ + wave6-vpu-dbg.o \ wave6-vpuapi.o \ wave6-vpu-dec.o \ wave6-vpu-enc.o \
drivers/media/platform/chips-media/wave6/wave6-hw.c +35 -1
@@ -10,6 +10,7 @@#include "wave6-vpu-core.h" #include "wave6-hw.h" #include "wave6-regdefine.h" +#include "wave6-trace.h" #define wave6_print_reg_err (core, reason) \ _wave6_print_reg_err((core), (reason), __func__)
@@ -17,6 +18,7 @@void wave6_vpu_writel (struct vpu_core_device *core,u32 addr,u32 data) {wave6_vdi_writel (core->reg_base, addr, data); +trace_wave6_vpu_writel (core->dev, addr, data); }u32 wave6_vpu_readl (struct vpu_core_device *core,u32 addr)
@@ -24,6 +26,7 @@ u32 wave6_vpu_readl(struct vpu_core_device *core, u32 addr)u32 data; data =wave6_vdi_readl (core->reg_base, addr); +trace_wave6_vpu_readl (core->dev, addr, data);return data; }
@@ -31,6 +34,7 @@ u32 wave6_vpu_readl(struct vpu_core_device *core, u32 addr)void wave6_vpu_writel_relaxed (struct vpu_core_device *core,u32 addr,u32 data) {wave6_vdi_writel_relaxed (core->reg_base, addr, data); +trace_wave6_vpu_writel (core->dev, addr, data); }u32 wave6_vpu_readl_relaxed (struct vpu_core_device *core,u32 addr)
@@ -38,6 +42,7 @@ u32 wave6_vpu_readl_relaxed(struct vpu_core_device *core, u32 addr)u32 data; data =wave6_vdi_readl_relaxed (core->reg_base, addr); +trace_wave6_vpu_readl (core->dev, addr, data);return data; }
@@ -45,7 +50,34 @@ u32 wave6_vpu_readl_relaxed(struct vpu_core_device *core, u32 addr)static void _wave6_print_reg_err (struct vpu_core_device *core,u32 fail_reason,const char *func) { -dev_err (core->dev,"%s: failure: 0x%x\n" , func, fail_reason); +struct device *dev = core->dev; + +switch (fail_reason) { +case WAVE6_SYSERR_QUEUEING_FAIL : +dev_err (dev,"%s: queueing failure 0x%x\n" , func, fail_reason); +break ; +case WAVE6_SYSERR_RESULT_NOT_READY : +dev_err (dev,"%s: result not ready 0x%x\n" , func, fail_reason); +break ; +case WAVE6_SYSERR_ACCESS_VIOLATION_HW : +dev_err (dev,"%s: access violation 0x%x\n" , func, fail_reason); +break ; +case WAVE6_SYSERR_WATCHDOG_TIMEOUT : +dev_err (dev,"%s: watchdog timeout 0x%x\n" , func, fail_reason); +break ; +case WAVE6_SYSERR_BUS_ERROR : +dev_err (dev,"%s: bus error 0x%x\n" , func, fail_reason); +break ; +case WAVE6_SYSERR_DOUBLE_FAULT : +dev_err (dev,"%s: double fault 0x%x\n" , func, fail_reason); +break ; +case WAVE6_SYSERR_VPU_STILL_RUNNING : +dev_err (dev,"%s: still running 0x%x\n" , func, fail_reason); +break ; +default : +dev_err (dev,"%s: failure: 0x%x\n" , func, fail_reason); +break ; + } }static void wave6_dec_set_display_buffer (struct vpu_instance *inst,struct frame_buffer fb)
@@ -218,6 +250,8 @@ static void wave6_send_command(struct vpu_core_device *core, u32 id, u32 std, u3vpu_write_reg_relaxed (core,W6_VPU_BUSY_STATUS ,BUSY_STATUS_SET );vpu_write_reg_relaxed (core,W6_COMMAND , cmd);vpu_write_reg (core,W6_VPU_HOST_INT_REQ ,HOST_INT_REQ_ON ); + +trace_wave6_vpu_send_command (core, id, std, cmd); } static int wave6_send_query(struct vpu_core_device *core, u32 id, u32 std,
drivers/media/platform/chips-media/wave6/wave6-trace.h new file +284 -0
284 lines not on this page · raw message
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c +14 -0
@@ -10,6 +10,7 @@#include <linux/clk.h> #include <linux/interrupt.h> #include <linux/pm_runtime.h> +#include <linux/debugfs.h> #include <linux/iopoll.h> #include "wave6-vpu-core.h"
@@ -17,6 +18,11 @@#include "wave6-vpuconfig.h" #include "wave6-hw.h" +#define CREATE_TRACE_POINTS +#include "wave6-trace.h" + +#define WAVE6_VPU_DEBUGFS_DIR "wave6" +static irqreturn_t wave6_vpu_core_irq (int irq,void *dev_id) {struct vpu_core_device *core = dev_id;
@@ -32,6 +38,8 @@ static irqreturn_t wave6_vpu_core_irq(int irq, void *dev_id)vpu_write_reg (core,W6_VPU_VINT_REASON_CLEAR , irq_info.status);vpu_write_reg (core,W6_VPU_VINT_CLEAR ,VINT_CLEAR ); +trace_wave6_vpu_irq (core, irq_info.status, irq_info.inst_idc); +if (irq_info.status &BIT (W6_INT_BIT_REQ_WORK_BUF )) {wave6_vpu_require_work_buffer (core->vpu, core->reg_base);
@@ -285,6 +293,12 @@ static int wave6_vpu_core_probe(struct platform_device *pdev)if (ret)return ret; + core->debugfs =debugfs_lookup (WAVE6_VPU_DEBUGFS_DIR ,NULL ); +if (!IS_ERR_OR_NULL (core->debugfs)) +dput (core->debugfs); +else + core->debugfs =debugfs_create_dir (WAVE6_VPU_DEBUGFS_DIR ,NULL ); + ret =devm_pm_runtime_enable (&pdev->dev);if (ret)return ret;
drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.c new file +148 -0
148 lines not on this page · raw message
drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.h new file +14 -0
@@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) */ +/* +* Wave6 series multi-standard codec IP - debug interface +* +* Copyright (C) 2026 CHIPS&MEDIA INC +*/ + +#ifndef __WAVE6_VPU_DBG_H__ +#define __WAVE6_VPU_DBG_H__ + +int wave6_vpu_create_dbgfs_file (struct vpu_instance *inst); +void wave6_vpu_remove_dbgfs_file (struct vpu_instance *inst); + +#endif /* __WAVE6_VPU_DBG_H__ */
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c +22 -3
@@ -8,6 +8,8 @@#include <linux/pm_runtime.h> #include "wave6-vpu-core.h" +#include "wave6-vpu-dbg.h" +#include "wave6-trace.h" #define VPU_DEC_DEV_NAME "C&M Wave6 VPU decoder"#define VPU_DEC_DRV_NAME "wave6-dec"
@@ -135,13 +137,16 @@ static void wave6_vpu_dec_release_fb(struct vpu_instance *inst) } } -static void wave6_vpu_dec_destroy_instance (struct vpu_instance *inst) +static void wave6_vpu_dec_destroy_instance (struct vpu_instance *inst,bool remove_dbgfs) {u32 fail_res =0 ;int ret;wave6_vpu_disable_instance (inst); +if (remove_dbgfs) +wave6_vpu_remove_dbgfs_file (inst); + ret =wave6_vpu_dec_close (inst, &fail_res);if (ret) { dev_err(inst->dev->dev, "failed destroy instance: %d (%d)\n",
@@ -190,6 +195,8 @@ static void wave6_handle_bitstream_buffer(struct vpu_instance *inst,wave6_vpu_dec_set_rd_ptr (inst, wr, true); } +trace_wave6_vpu_dec_pic (inst, src_buf ? src_buf->vb2_buf.index :-1 , src_size); + ret =wave6_vpu_dec_update_bitstream_buffer (inst, src_size);if (ret) { dev_dbg(inst->dev->dev, "%s: Update bitstream buffer fail %d\n",
@@ -769,6 +776,8 @@ static void wave6_vpu_dec_handle_source_change(struct vpu_instance *inst) info->pic_width, info->pic_height, true); +trace_wave6_vpu_source_change (inst, info); +v4l2_event_queue_fh (&inst->v4l2_fh, &vpu_event_src_ch); }
@@ -795,6 +804,8 @@ static void wave6_vpu_dec_finish_decode(struct vpu_instance *inst, bool error)if (ret) goto finish_decode; +trace_wave6_vpu_dec_done (inst, &info); +dev_dbg (inst->dev->dev,"dec %d dis %d noti_flag %d stream_end %d\n" , info.frame_decoded, info.frame_display, info.notification_flags, info.stream_end);
@@ -1353,6 +1364,8 @@ static int wave6_vpu_dec_s_ctrl(struct v4l2_ctrl *ctrl) {struct vpu_instance *inst =wave6_ctrl_to_vpu_inst (ctrl); +trace_wave6_vpu_s_ctrl (inst, ctrl); +dev_dbg (inst->dev->dev,"%s: name %s value %d\n" , __func__, ctrl->name, ctrl->val);
@@ -1419,6 +1432,7 @@ static int wave6_vpu_dec_create_instance(struct vpu_instance *inst) scoped_guard(spinlock, &inst->dev->inst_lock)list_add_tail (&inst->list, &inst->dev->instances);wave6_vpu_enable_instance (inst); +wave6_vpu_create_dbgfs_file (inst);wave6_vpu_set_instance_state (inst,VPU_INST_STATE_OPEN );return 0 ;
@@ -1471,7 +1485,7 @@ static int wave6_vpu_dec_queue_setup(struct vb2_queue *q, unsigned int *num_buff }if (V4L2_TYPE_IS_OUTPUT (q->type) && inst->state ==VPU_INST_STATE_SEEK ) -wave6_vpu_dec_destroy_instance (inst); +wave6_vpu_dec_destroy_instance (inst, true);return 0 ; }
@@ -1604,6 +1618,8 @@ static int wave6_vpu_dec_start_streaming(struct vb2_queue *q, unsigned int countstruct v4l2_pix_format_mplane *fmt;int ret =0 ; +trace_wave6_vpu_start_streaming (inst,V4L2_TYPE_IS_OUTPUT (q->type)); +if (V4L2_TYPE_IS_OUTPUT (q->type)) { fmt = &inst->src_fmt;if (inst->state ==VPU_INST_STATE_NONE ) {
@@ -1644,6 +1660,8 @@ static void wave6_vpu_dec_stop_streaming(struct vb2_queue *q)struct vpu_instance *inst =vb2_get_drv_priv (q);struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx; +trace_wave6_vpu_stop_streaming (inst,V4L2_TYPE_IS_OUTPUT (q->type)); +dev_dbg (inst->dev->dev,"[%d] %s, input %d, decode %d error %d\n" , inst->id,V4L2_TYPE_IS_OUTPUT (q->type) ?"output" :"capture" , inst->queued_src_buf_num, inst->processed_buf_num, inst->error_buf_num);
@@ -1876,10 +1894,11 @@ static int wave6_vpu_dec_release(struct file *filp) {struct vpu_instance *inst =wave6_file_to_vpu_inst (filp); +wave6_vpu_remove_dbgfs_file (inst);v4l2_m2m_ctx_release (inst->v4l2_fh.m2m_ctx);if (inst->state !=VPU_INST_STATE_NONE ) -wave6_vpu_dec_destroy_instance (inst); +wave6_vpu_dec_destroy_instance (inst, false);v4l2_ctrl_handler_free (&inst->v4l2_ctrl_hdl);v4l2_fh_del (&inst->v4l2_fh, filp);
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c +22 -5
@@ -9,6 +9,8 @@#include <linux/pm_runtime.h> #include "wave6-vpu-core.h" +#include "wave6-vpu-dbg.h" +#include "wave6-trace.h" #define VPU_ENC_DEV_NAME "C&M Wave6 VPU encoder"#define VPU_ENC_DRV_NAME "wave6-enc"
@@ -366,13 +368,16 @@ static void wave6_vpu_enc_release_fb(struct vpu_instance *inst) } } -static void wave6_vpu_enc_destroy_instance (struct vpu_instance *inst) +static void wave6_vpu_enc_destroy_instance (struct vpu_instance *inst,bool remove_dbgfs) {u32 fail_res =0 ;int ret;wave6_vpu_disable_instance (inst); +if (remove_dbgfs) +wave6_vpu_remove_dbgfs_file (inst); + ret =wave6_vpu_enc_close (inst, &fail_res);if (ret) { dev_err(inst->dev->dev, "failed destroy instance: %d (%d)\n",
@@ -817,6 +822,8 @@ static int wave6_vpu_enc_start_encode(struct vpu_instance *inst) }wave6_set_csc (inst, &pic_param); +trace_wave6_vpu_enc_pic (inst, &pic_param); + ret =wave6_vpu_enc_start_one_frame (inst, &pic_param, &fail_res);if (ret) {dev_err (inst->dev->dev,"[%d] %s: fail %d\n" , inst->id, __func__, ret);
@@ -931,6 +938,8 @@ static void wave6_vpu_enc_finish_encode(struct vpu_instance *inst, bool error) goto finish_encode; } +trace_wave6_vpu_enc_done (inst, &info); +if (info.enc_src_idx >=0 && info.recon_frame_index >=0 )wave6_handle_encoded_frame (inst, &info);
@@ -1426,6 +1435,8 @@ static int wave6_vpu_enc_s_ctrl(struct v4l2_ctrl *ctrl)struct vpu_instance *inst =wave6_ctrl_to_vpu_inst (ctrl);struct enc_controls *p = &inst->enc_ctrls; +trace_wave6_vpu_s_ctrl (inst, ctrl); +dev_dbg (inst->dev->dev,"%s: name %s value %d\n" , __func__, ctrl->name, ctrl->val);
@@ -2040,6 +2051,7 @@ static int wave6_vpu_enc_create_instance(struct vpu_instance *inst) scoped_guard(spinlock, &inst->dev->inst_lock)list_add_tail (&inst->list, &inst->dev->instances);wave6_vpu_enable_instance (inst); +wave6_vpu_create_dbgfs_file (inst);wave6_vpu_set_instance_state (inst,VPU_INST_STATE_OPEN );return 0 ;
@@ -2304,6 +2316,8 @@ static int wave6_vpu_enc_start_streaming(struct vb2_queue *q, unsigned int countstruct vpu_instance *inst =vb2_get_drv_priv (q);int ret =0 ; +trace_wave6_vpu_start_streaming (inst,V4L2_TYPE_IS_OUTPUT (q->type)); +v4l2_m2m_update_start_streaming_state (inst->v4l2_fh.m2m_ctx, q);if (inst->state ==VPU_INST_STATE_NONE ) {
@@ -2315,7 +2329,7 @@ static int wave6_vpu_enc_start_streaming(struct vb2_queue *q, unsigned int countif (inst->state ==VPU_INST_STATE_OPEN ) { ret =wave6_vpu_enc_initialize_instance (inst);if (ret) { -wave6_vpu_enc_destroy_instance (inst); +wave6_vpu_enc_destroy_instance (inst, true); goto exit; } }
@@ -2323,7 +2337,7 @@ static int wave6_vpu_enc_start_streaming(struct vb2_queue *q, unsigned int countif (inst->state ==VPU_INST_STATE_INIT_SEQ ) { ret =wave6_vpu_enc_prepare_fb (inst);if (ret) { -wave6_vpu_enc_destroy_instance (inst); +wave6_vpu_enc_destroy_instance (inst, true); goto exit; } }
@@ -2339,6 +2353,8 @@ static void wave6_vpu_enc_stop_streaming(struct vb2_queue *q) {struct vpu_instance *inst =vb2_get_drv_priv (q); +trace_wave6_vpu_stop_streaming (inst,V4L2_TYPE_IS_OUTPUT (q->type)); +dev_dbg (inst->dev->dev,"[%d] %s, input %d, encode %d\n" , inst->id,V4L2_TYPE_IS_OUTPUT (q->type) ?"output" :"capture" , inst->queued_src_buf_num, inst->sequence);
@@ -2356,7 +2372,7 @@ static void wave6_vpu_enc_stop_streaming(struct vb2_queue *q) }if (V4L2_TYPE_IS_CAPTURE (q->type) && inst->state !=VPU_INST_STATE_NONE ) -wave6_vpu_enc_destroy_instance (inst); +wave6_vpu_enc_destroy_instance (inst, true); exit:wave6_vpu_return_buffers (inst, q->type,VB2_BUF_STATE_ERROR );
@@ -2673,10 +2689,11 @@ static int wave6_vpu_enc_release(struct file *filp) {struct vpu_instance *inst =wave6_file_to_vpu_inst (filp); +wave6_vpu_remove_dbgfs_file (inst);v4l2_m2m_ctx_release (inst->v4l2_fh.m2m_ctx);if (inst->state !=VPU_INST_STATE_NONE ) -wave6_vpu_enc_destroy_instance (inst); +wave6_vpu_enc_destroy_instance (inst, false);v4l2_ctrl_handler_free (&inst->v4l2_ctrl_hdl);v4l2_fh_del (&inst->v4l2_fh, filp);
drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c +3 -0
@@ -7,6 +7,7 @@#include "wave6-vpu.h" #include "wave6-vpu-core.h" +#include "wave6-trace.h" void wave6_update_pix_fmt(struct v4l2_pix_format_mplane *pix_mp, unsigned int width,
@@ -217,6 +218,8 @@ void wave6_vpu_set_instance_state(struct vpu_instance *inst,if (!wave6_vpu_valid_instance_transition (inst, state))return ; +trace_wave6_vpu_set_state (inst, state); + dev_dbg(inst->dev->dev, "[%d] %s -> %s\n", inst->id, wave6_vpu_instance_state_name(inst->state),
drivers/media/platform/chips-media/wave6/wave6-vpuapi.h +5 -0
@@ -796,6 +796,7 @@ struct vpu_irq {* @irq_buffer: Buffer used by @irq_fifo * @irq: Interrupt number * @res: Device compatible data +* @debugfs: Debugfs entry * @active_count: Number of decoder/encoder instances using the core * @vpu_held: Core holds a reference taken by wave6_vpu_get() */
@@ -825,6 +826,7 @@ struct vpu_core_device {int irq;const struct wave6_vpu_core_resource *res; +struct dentry *debugfs;atomic_t active_count;bool vpu_held; };
@@ -881,6 +883,7 @@ struct vpu_instance_ops {* @error_recovery: Error recovery flag * @disp_mode: Display mode for decoder V4L2 control * @enc_ctrls: Encoder V4L2 control and streaming parameters +* @debugfs: Debugfs entry */ struct vpu_instance {struct list_head list;
@@ -939,6 +942,8 @@ struct vpu_instance {enum display_mode disp_mode;struct enc_controls enc_ctrls; + +struct dentry *debugfs; };void wave6_vpu_writel (struct vpu_core_device *core,u32 addr,u32 data);
Signature · 1 line
2.41.0.windows.3
PATCH v8 10/11 media: chips-media: wave6: Add Wave6 thermal cooling device
In reply to Nas ChungAdd a thermal cooling device for the Wave6 VPU, so the thermal framework can lower the VPU performance state when the SoC gets hot.
The control driver registers the cooling device on its device tree node. The cooling states are built from the OPP table of the "perf" power domain.
--- .../media/platform/chips-media/wave6/Makefile | 1 + .../chips-media/wave6/wave6-vpu-thermal.c | 141 ++++++++++++++++++ .../chips-media/wave6/wave6-vpu-thermal.h | 24 +++ .../platform/chips-media/wave6/wave6-vpu.c | 22 +++ .../platform/chips-media/wave6/wave6-vpu.h | 3 + 5 files changed, 191 insertions(+) create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.c create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.h
drivers/media/platform/chips-media/wave6/Makefile +1 -0
@@ -4,6 +4,7 @@ CFLAGS_wave6-vpu-core.o := -I$(src) wave6-objs += wave6-vpu.o \ + wave6-vpu-thermal.o \ wave6-vpu-core.o \ wave6-vpu-v4l2.o \ wave6-vpu-dbg.o \
drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.c new file +141 -0
141 lines not on this page · raw message
drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.h new file +24 -0
@@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) */ +/* +* Wave6 series multi-standard codec IP - wave6 thermal cooling interface +* +* Copyright (C) 2026 CHIPS&MEDIA INC +* +*/ + +#ifndef __WAVE6_VPU_THERMAL_H__ +#define __WAVE6_VPU_THERMAL_H__ + +#include <linux/thermal.h> + +struct vpu_thermal_cooling { +struct device *dev; +int thermal_event; +int thermal_max; +struct thermal_cooling_device *cooling; +unsigned int *level_table; +}; + +int wave6_vpu_cooling_init (struct device *dev,struct vpu_thermal_cooling *thermal); + +#endif /* __WAVE6_VPU_THERMAL_H__ */
drivers/media/platform/chips-media/wave6/wave6-vpu.c +22 -0
@@ -613,6 +613,21 @@ static int wave6_vpu_attach_pm_domains(struct wave6_vpu_device *vpu)return 0 ; } +static struct device *wave6_vpu_get_performance_dev (struct wave6_vpu_device *vpu) +{ +int index; + +if (!vpu->num_pm_domains) +return vpu->dev; + + index =of_property_match_string (vpu->dev->of_node, +"power-domain-names" ,"perf" ); +if (index <0 || index >= vpu->num_pm_domains) +return NULL ; + +return vpu->pd_list->pd_devs[index]; +} +static int wave6_vpu_probe (struct platform_device *pdev) {struct device_node *np;
@@ -696,6 +711,13 @@ static int wave6_vpu_probe(struct platform_device *pdev) } } + vpu->thermal.dev =wave6_vpu_get_performance_dev (vpu); +if (vpu->thermal.dev) { + ret =wave6_vpu_cooling_init (vpu->dev, &vpu->thermal); +if (ret) +dev_err (&pdev->dev,"failed to initialize thermal cooling, %d\n" , ret); + } +wave6_vpu_allocate_work_buffers (vpu);pm_runtime_enable (&pdev->dev);
drivers/media/platform/chips-media/wave6/wave6-vpu.h +3 -0
@@ -12,6 +12,7 @@#include <linux/device.h> #include "wave6-vpuconfig.h" +#include "wave6-vpu-thermal.h" #include "wave6-vdi.h" #define WAVE6_VPU_PLATFORM_DRIVER_NAME "wave6-vpu"
@@ -93,6 +94,7 @@ struct wave6_vpu_core_resource {* @work_buffers: Array of work buffers * @work_buffers_alloc: Number of allocated work buffers * @work_buffers_avail: Number of available work buffers +* @thermal: Thermal cooling device * @core_count: Number of VPU cores holding a wave6_vpu_get() reference * @num_pm_domains: Number of attached power domains * @pd_list: List of attached power domains
@@ -121,6 +123,7 @@ struct wave6_vpu_device {u32 work_buffers_alloc;u32 work_buffers_avail; +struct vpu_thermal_cooling thermal;atomic_t core_count;int num_pm_domains;
Signature · 1 line
2.41.0.windows.3
PATCH v8 11/11 arm64: dts: freescale: imx95: Add video codec node
In reply to Nas ChungAdd the Chips and Media wave633 video codec node on IMX95 SoCs.
--- .../boot/dts/freescale/imx95-19x19-evk.dts | 10 +++++ arch/arm64/boot/dts/freescale/imx95.dtsi | 37 +++++++++++++++++++ 2 files changed, 47 insertions(+)
arch/arm64/boot/dts/freescale/imx95-19x19-evk.dts +10 -0
@@ -143,6 +143,11 @@ vdev0buffer: vdev0buffer@88020000 { reg = <0 0x88020000 0 0x100000>; no-map; }; + + vpu_boot: memory@a0000000 { + reg = <0 0xa0000000 0 0x100000>; + no-map; + }; }; flexcan1_phy: can-phy0 {
@@ -1225,3 +1230,8 @@ &tpm6 { pinctrl-0 = <&pinctrl_tpm6>; status = "okay"; }; + +&vpu { + memory-region = <&vpu_boot>; + status = "okay"; +};
arch/arm64/boot/dts/freescale/imx95.dtsi +37 -0
@@ -2139,6 +2139,43 @@ vpu_blk_ctrl: clock-controller@4c410000 { assigned-clock-rates = <133333333>, <667000000>, <500000000>; }; + vpu: video-codec@4c4c0000 { + compatible = "nxp,imx95-vpu"; + reg = <0x0 0x4c4c0000 0x0 0x10000>; + clocks = <&scmi_clk IMX95_CLK_VPU>, + <&vpu_blk_ctrl IMX95_CLK_VPUBLK_WAVE>; + clock-names = "core", "vpublk"; + power-domains = <&scmi_devpd IMX95_PD_VPU>, + <&scmi_perf IMX95_PERF_VPU>; + power-domain-names = "vpu", "perf"; + sram = <&sram1>; + #cooling-cells = <2>; + #address-cells = <2>; + #size-cells = <2>; + ranges; + status = "disabled"; + + interface@4c480000 { + reg = <0x0 0x4c480000 0x0 0x10000>; + interrupts = <GIC_SPI 299 IRQ_TYPE_LEVEL_HIGH>; + }; + + interface@4c490000 { + reg = <0x0 0x4c490000 0x0 0x10000>; + interrupts = <GIC_SPI 300 IRQ_TYPE_LEVEL_HIGH>; + }; + + interface@4c4a0000 { + reg = <0x0 0x4c4a0000 0x0 0x10000>; + interrupts = <GIC_SPI 301 IRQ_TYPE_LEVEL_HIGH>; + }; + + interface@4c4b0000 { + reg = <0x0 0x4c4b0000 0x0 0x10000>; + interrupts = <GIC_SPI 302 IRQ_TYPE_LEVEL_HIGH>; + }; + }; + jpegdec: jpegdec@4c500000 { compatible = "nxp,imx95-jpgdec", "nxp,imx8qxp-jpgdec"; reg = <0x0 0x4C500000 0x0 0x00050000>;
Signature · 1 line
2.41.0.windows.3
[email protected]
AI reviewRe: [PATCH v8 06/11] media: chips-media: wave6: Add Wave6 encoder firmware interface
In reply to Nas ChungThank you for your contribution! Sashiko AI review found 1 potential issue to consider: - [Medium] media: wave6: avoid large struct copies on stack in wave6_gen_enc_pic_reg() --
Thank you for your contribution! Sashiko AI review found 1 potential issue to consider:
- [Medium] media: wave6: avoid large struct copies on stack in wave6_gen_enc_pic_reg()
--
commit 0c77c50267143f16a1d2daf6f5d24a657e85b50e
Author: Nas Chung <[email protected]>
media: chips-media: wave6: Add Wave6 encoder firmware interface
This commit adds the firmware interface and MMIO register configuration helpers for Wave6 video encoding operations.
Quoted 4 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c +0
-0
[ ... ]
Quoted 8 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c +8
-0+static void wave6_update_enc_info (struct enc_info *p_enc_info) +{ +struct enc_open_param op = p_enc_info->open_param; + + p_enc_info->width = op.pic_width; + p_enc_info->height = op.pic_height; + p_enc_info->c_fmt_idc =get_chroma_format_idc (op.out_fmt); +}
[ ... ]
Quoted 7 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c +7
-0+static void wave6_gen_enc_pic_reg (struct enc_info *p_enc_info,bool cbcr_interleave,bool nv21, +struct enc_param *opt,struct enc_cmd_enc_pic_reg *reg) +{ +struct enc_open_param open = p_enc_info->open_param; +struct enc_codec_param param = open.codec_param; +u32 stride_c = opt->source_frame->stride_c; +u32 c_fmt_idc;
[ ... ]
drivers/media/platform/chips-media/wave6/wave6-hw.c int wave6_vpu_dec_flush_instance(struct vpu_instance *inst) · line 1732
Quoted 13 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c +13
-0+int wave6_vpu_enc_start_one_frame (struct vpu_instance *inst,struct enc_param *param, +u32 *fail_res) +{ +struct enc_cmd_enc_pic_reg reg = {}; +struct enc_info *p_enc_info; +int ret; + +guard (mutex)(&inst->dev->hw_lock); + + p_enc_info = &inst->codec_info->enc_info; + +wave6_gen_enc_pic_reg (p_enc_info, inst->cbcr_interleave, + inst->nv21, param, ®);
[Severity: Medium]
Should wave6_gen_enc_pic_reg() and wave6_update_enc_info() take pointers to p_enc_info->open_param and its codec_param instead of copying them by value onto the stack?
struct enc_open_param is around 824 bytes and struct enc_codec_param is around 772 bytes. When wave6_gen_enc_pic_reg() is inlined into wave6_vpu_enc_start_one_frame() alongside struct enc_cmd_enc_pic_reg, the local structures alone consume roughly 1,700 bytes of stack space (and over 1,220 bytes in wave6_vpu_enc_issue_seq_init() via wave6_update_enc_info()), exceeding the 1,280-byte CONFIG_FRAME_WARN threshold on 32-bit builds.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
[email protected]
AI reviewRe: [PATCH v8 08/11] media: chips-media: wave6: Add trace and debugfs
In reply to Nas ChungThank you for your contribution! Sashiko AI review found 1 potential issue to consider: - [Medium] media: wave6: debugfs directory leak in wave6_vpu_core_probe() on teardown --
Thank you for your contribution! Sashiko AI review found 1 potential issue to consider:
- [Medium] media: wave6: debugfs directory leak in wave6_vpu_core_probe() on teardown
--
commit 5ece419bea0b582239e058aa46da23a997a50915
Author: Nas Chung <[email protected]>
media: chips-media: wave6: Add trace and debugfs
This commit adds tracepoints, per-instance debugfs entries, and readable firmware error messages to the Wave6 VPU driver.
Quoted 4 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c +0
-0
[ ... ]
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c static int wave6_vpu_core_probe(struct platform_device *pdev) · line 304
Quoted 13 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c +6
-0@@ -285,6 +293,12 @@ static int wave6_vpu_core_probe(struct platform_device *pdev)if (ret)return ret; + core->debugfs =debugfs_lookup (WAVE6_VPU_DEBUGFS_DIR ,NULL ); +if (!IS_ERR_OR_NULL (core->debugfs)) +dput (core->debugfs); +else + core->debugfs =debugfs_create_dir (WAVE6_VPU_DEBUGFS_DIR ,NULL ); + ret =devm_pm_runtime_enable (&pdev->dev);if (ret)return ret;
[Severity: Medium]
Does this leak the wave6 debugfs directory on probe failure or module unload? Every other resource in wave6_vpu_core_probe() registers a devm_add_action_or_reset() cleanup callback because wave6_vpu_core_driver has no .remove callback, and wave6_vpu_remove_dbgfs_file() only removes the per-instance file, so nothing ever calls debugfs_remove() on core->debugfs.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
[email protected]
AI reviewRe: [PATCH v8 09/11] media: chips-media: wave6: Add performance profiling
In reply to Nas ChungThank you for your contribution! Sashiko AI review found 5 potential issues to consider: - [Medium] media: wave6: performance timestamps not re-recorded on OUTPUT stream restart - [Medium] media…
Thank you for your contribution! Sashiko AI review found 5 potential issues to consider:
- [Medium] media: wave6: performance timestamps not re-recorded on OUTPUT stream restart
- [Medium] media: wave6: skipped performance reset in wave6_vpu_enc_stop_streaming()
- [Medium] media: wave6: wave6_vpu_cycle_to_ns() queries empty core->clks instead of vpu
- [Medium] media: wave6: unlocked TOCTOU race and div-by-zero in wave6_vpu_dbg_instance()
- [Low] media: wave6: negative setup latency in wave6_vpu_dbg_instance() before PIC_RUN
--
commit bc504ce1eb7a7d90813e57f32d70181940592318
Author: Nas Chung <[email protected]>
media: chips-media: wave6: Add performance profiling
This commit adds per-instance performance profiling to record per-frame timestamps and firmware cycle counts, reporting frame rates and latencies in debugfs.
Quoted 4 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.c +0
-0
[ ... ]
drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.c static int wave6_vpu_dbg_instance(struct seq_file *s, void *data) · line 92
Quoted 11 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.c +7
-0@@ -77,6 +83,36 @@ static int wave6_vpu_dbg_instance(struct seq_file *s, void *data) inst->error_buf_num, v4l2_m2m_has_stopped(inst->v4l2_fh.m2m_ctx)); +seq_puts (s,"fps" ); + tmp =MSEC_PER_SEC * inst->processed_buf_num; +if (perf->ts_last > perf->ts_first +NSEC_PER_MSEC ) { + duration =ktime_ms_delta (perf->ts_last, perf->ts_first); + fps =DIV64_U64_ROUND_CLOSEST (tmp, duration); +seq_printf (s," actual: %lld;" , fps); + }
[Severity: Medium]
Can a concurrent OUTPUT queue stop_streaming() cause a divide-by-zero in DIV64_U64_ROUND_CLOSEST(tmp, duration)?
wave6_vpu_dbg_instance() reads inst->performance without locking while the debugfs file is active:
CPU0 CPU1wave6_vpu_dbg_instance()
if (perf->ts_last >
perf->ts_first + NSEC_PER_MSEC)
wave6_vpu_dec_stop_streaming()
wave6_vpu_reset_performance()
memset(&inst->performance, 0, ...)
duration = ktime_ms_delta(
perf->ts_last, perf->ts_first)
DIV64_U64_ROUND_CLOSEST(
tmp, duration) <-- divide-by-zero when duration == 0Unlike wave6_vpu_reset_performance(), this path does not check duration > 0 before dividing, and if only perf->ts_last is seen zeroed, duration becomes negative and is cast to a large u64 divisor.
Quoted 7 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.c +7
-0+ duration =div_u64 (perf->total_sw_time,NSEC_PER_MSEC ); +if (duration >0 ) { + fps =DIV64_U64_ROUND_CLOSEST (tmp, duration); +seq_printf (s," sw: %lld;" , fps); + } + duration =div_u64 (perf->total_hw_time,NSEC_PER_MSEC ); +if (duration >0 ) {
drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.c static int wave6_vpu_dbg_instance(struct seq_file *s, void *data) · line 109
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.c +10
-0+ fps =DIV64_U64_ROUND_CLOSEST (tmp, duration); +seq_printf (s," hw: %lld" , fps); + } +seq_putc (s,'\n' ); + + ms[0 ] =div_s64_rem (perf->latency_first,NSEC_PER_MSEC , &ns[0 ]); + ms[1 ] =div_s64_rem (perf->latency_max,NSEC_PER_MSEC , &ns[1 ]); + ms[2 ] =div_s64_rem (ktime_sub (perf->ts_first, perf->ts_start),NSEC_PER_MSEC , &ns[2 ]); +seq_printf (s,"latency(ms) first: %lld.%06d, max %lld.%06d, setup %lld.%06d\n" , + ms[0 ], ns[0 ], ms[1 ], ns[1 ], ms[2 ], ns[2 ]);
[Severity: Low]
What happens to the setup latency output if the debugfs file is read before the instance reaches VPU_INST_STATE_PIC_RUN?
perf->ts_start is set when the instance is created (before wave6_vpu_create_dbgfs_file() exposes the debugfs entry in VPU_INST_STATE_OPEN), whereas perf->ts_first remains 0 until wave6_vpu_dec_prepare_fb() or wave6_vpu_enc_prepare_fb() transitions the instance to VPU_INST_STATE_PIC_RUN. Reading debugfs in VPU_INST_STATE_OPEN or VPU_INST_STATE_INIT_SEQ passes -perf->ts_start to div_s64_rem(), producing a negative quotient ms[2] and a negative remainder ns[2] formatted with %lld.%06d (e.g. "setup -12345.-678900").
[ ... ]
Quoted 4 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c +0
-0
[ ... ]
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c static void wave6_vpu_dec_stop_streaming(struct vb2_queue *q) · line 1690
Quoted 8 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c +1
-0@@ -1670,6 +1684,7 @@ static void wave6_vpu_dec_stop_streaming(struct vb2_queue *q) goto exit;if (V4L2_TYPE_IS_OUTPUT (q->type)) { +wave6_vpu_reset_performance (inst); inst->queued_src_buf_num =0 ; inst->processed_buf_num =0 ; inst->error_buf_num =0 ;
[Severity: Medium]
Does zeroing inst->performance on OUTPUT stop_streaming() leave perf->ts_start and perf->ts_first at 0 when OUTPUT streaming is restarted?
During a decoder seek (VIDIOC_STREAMOFF followed by VIDIOC_STREAMON on the OUTPUT queue), wave6_vpu_dec_stop_streaming() clears inst->performance and moves inst->state to VPU_INST_STATE_SEEK. When OUTPUT streaming restarts, wave6_vpu_dec_start_streaming() restores inst->state directly to inst->state_in_seek without calling wave6_vpu_record_performance_timestamps():
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c:wave6_vpu_dec_start_streaming() {
...
if (V4L2_TYPE_IS_OUTPUT(q->type)) {
fmt = &inst->src_fmt;
if (inst->state == VPU_INST_STATE_NONE) {
ret = wave6_vpu_dec_create_instance(inst);
...
}
if (inst->state == VPU_INST_STATE_SEEK)
wave6_vpu_set_instance_state(inst, inst->state_in_seek);
}
...
}The same happens in the encoder when the OUTPUT queue is stopped and restarted while CAPTURE remains streaming in VPU_INST_STATE_PIC_RUN. Because perf->ts_first stays 0 while perf->ts_last updates on each decoded or encoded frame, ktime_ms_delta(perf->ts_last, perf->ts_first) in debugfs and wave6_vpu_reset_performance() divides by total system uptime since boot.
Quoted 4 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c +0
-0
[ ... ]
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c static void wave6_vpu_enc_stop_streaming(struct vb2_queue *q) · line 2381
Quoted 8 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c +1
-0@@ -2363,6 +2375,7 @@ static void wave6_vpu_enc_stop_streaming(struct vb2_queue *q) goto exit;if (V4L2_TYPE_IS_OUTPUT (q->type)) { +wave6_vpu_reset_performance (inst); inst->queued_src_buf_num =0 ; inst->processed_buf_num =0 ; inst->error_buf_num =0 ;
[Severity: Medium]
Is wave6_vpu_reset_performance() skipped whenever the encoder CAPTURE queue stops streaming before the OUTPUT queue?
In wave6_vpu_enc_stop_streaming(), stopping the CAPTURE queue calls wave6_vpu_enc_destroy_instance(inst, true), which sets inst->state to VPU_INST_STATE_NONE. When wave6_vpu_enc_stop_streaming() is then called for the OUTPUT queue (for example via v4l2_m2m_ctx_release() on file close, which always stops CAPTURE before OUTPUT, or via VIDIOC_STREAMOFF on CAPTURE first), the early check:
if (inst->state == VPU_INST_STATE_NONE) goto exit;
bypasses wave6_vpu_reset_performance(inst) and the counter resets. This skips logging the performance summary and leaves stale metrics and non-zero ts_start / ts_first values in inst->performance if streaming is restarted on the same file descriptor.
Quoted 4 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c +0
-0
[ ... ]
Quoted 7 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c +3
-0@@ -228,6 +231,20 @@ void wave6_vpu_set_instance_state(struct vpu_instance *inst, inst->state = state; } +u64 wave6_vpu_cycle_to_ns (struct vpu_core_device *core,u64 cycle) +{ +unsigned long rate;
drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c void wave6_vpu_set_instance_state(struct vpu_instance *inst, · line 246
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c +10
-0+ +if (!core->num_clks || !core->clks) +return 0 ; + + rate =clk_get_rate (core->clks[0 ].clk); +if (!rate) +return 0 ; + +return div64_ul (cycle *NSEC_PER_SEC , rate); +}
[Severity: Medium]
Does wave6_vpu_cycle_to_ns() always return 0 because the clocks belong to core->vpu rather than core?
In the Device Tree binding (nxp,imx95-vpu.yaml), the clocks property is on the parent VPU node and is acquired into vpu->clks by wave6_vpu_probe(), whereas the child interface nodes used to create the wave6-vpu-core devices have no clocks property. As a result, core->num_clks is 0, per-frame hw_time and perf->total_hw_time remain 0, and hardware FPS is never reported.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9
[email protected]
AI reviewRe: [PATCH v8 10/11] media: chips-media: wave6: Add Wave6 thermal cooling device
In reply to Nas ChungThank you for your contribution! Sashiko AI review found 1 potential issue to consider: - [Medium] media: chips-media: wave6: UAF of thermal.dev due to devm teardown order --
Thank you for your contribution! Sashiko AI review found 1 potential issue to consider:
- [Medium] media: chips-media: wave6: UAF of thermal.dev due to devm teardown order
--
commit 4476163b22eacaa7f1b0467ae8902df8f1de4240
Author: Nas Chung <[email protected]>
media: chips-media: wave6: Add Wave6 thermal cooling device
This commit adds a thermal cooling device for the Wave6 VPU that adjusts the "perf" power domain's performance state based on its OPP table.
Quoted 27 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c +15
-0@@ -613,6 +613,21 @@ static int wave6_vpu_attach_pm_domains(struct wave6_vpu_device *vpu)return 0 ; } +static struct device *wave6_vpu_get_performance_dev (struct wave6_vpu_device *vpu) +{ +int index; + +if (!vpu->num_pm_domains) +return vpu->dev; + + index =of_property_match_string (vpu->dev->of_node, +"power-domain-names" ,"perf" ); +if (index <0 || index >= vpu->num_pm_domains) +return NULL ; + +return vpu->pd_list->pd_devs[index]; +} +static int wave6_vpu_probe (struct platform_device *pdev) {struct device_node *np;
drivers/media/platform/chips-media/wave6/wave6-vpu.c static int wave6_vpu_probe(struct platform_device *pdev) · line 719
Quoted 9 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c +6
-0@@ -696,6 +711,13 @@ static int wave6_vpu_probe(struct platform_device *pdev) } } + vpu->thermal.dev =wave6_vpu_get_performance_dev (vpu); +if (vpu->thermal.dev) { + ret =wave6_vpu_cooling_init (vpu->dev, &vpu->thermal); +if (ret) +dev_err (&pdev->dev,"failed to initialize thermal cooling, %d\n" , ret); + }
[Severity: Medium]
Can registering the cooling device via devm on vpu->dev cause a use-after-free of vpu->thermal.dev during driver teardown?
When multiple power domains are attached via dev_pm_domain_attach_list(), vpu->thermal.dev points to the virtual domain device in vpu->pd_list->pd_devs[index], while wave6_vpu_cooling_init() registers vpu->thermal.cooling via devm_thermal_of_child_cooling_device_register() on vpu->dev. Because wave6_vpu_remove() (and the wave6_vpu_probe() error path) calls wave6_vpu_detach_pm_domains() before devres cleans up vpu->dev, the cooling device remains registered after the virtual domain device is detached and freed:
CPU0 CPU1wave6_vpu_remove()
wave6_vpu_detach_pm_domains()
dev_pm_domain_detach_list()
/* unregisters and frees
pd_devs[index] */
__thermal_cdev_update()
wave6_vpu_cooling_set_cur_state()
wave6_vpu_thermal_cooling_update()
dev_pm_genpd_set_performance_state(thermal->dev, ...)
<-- use-after-freedevres_release_all()
thermal_cooling_device_unregister()
Attaching the power domains with devm_pm_domain_attach_list() before wave6_vpu_cooling_init(), or unregistering the cooling device explicitly before wave6_vpu_detach_pm_domains(), would keep the teardown order consistent.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=10
[email protected]
AI reviewRe: [PATCH v8 04/11] media: chips-media: wave6: Add Wave6 decoder firmware interface
In reply to Nas ChungThank you for your contribution! Sashiko AI review found 3 potential issues to consider: - [Low] media: wave6: unset disp_pic_width/height in wave6_vpu_dec_get_output_info() - [Low] media: wave6…
Thank you for your contribution! Sashiko AI review found 3 potential issues to consider:
- [Low] media: wave6: unset disp_pic_width/height in wave6_vpu_dec_get_output_info()
- [Low] media: wave6: invalid W6_RET_DEC_BS_RD_PTR read in wave6_vpu_dec_get_rd_ptr()
- [Low] media: wave6: ctu_size 2-bit bitfield truncates DEC_PIC_TYPE_CTU_SIZE to 0
--
commit 4430346b0188df2bacf518d95646056af3905289
Author: Nas Chung <[email protected]>
media: chips-media: wave6: Add Wave6 decoder firmware interface
This commit adds the firmware and register interface layer for the Chips&Media Wave6 video decoder, covering instance management, sequence initialization, buffer registration, and picture decoding.
Quoted 4 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c +0
-0
[ ... ]
Quoted 1 line
drivers/media/platform/chips-media/wave6/wave6-hw.c +0
-0@@ -179,3 +276,659 @@ int wave6_vpu_get_version(struct vpu_core_device *core)
[ ... ]
Quoted 2 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c +2
-0+int wave6_vpu_dec_get_output_info (struct vpu_instance *inst,struct dec_output_info *info) +{
[ ... ]
Quoted 18 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c +18
-0+ reg_val =vpu_read_reg_relaxed (inst->dev,W6_RET_DEC_PIC_SIZE ); + info->dec_pic_width =DEC_PIC_SIZE_WIDTH (reg_val); + info->dec_pic_height =DEC_PIC_SIZE_HEIGHT (reg_val); + + info->rd_ptr =wave6_vpu_dec_get_rd_ptr (inst); + info->wr_ptr = p_dec_info->stream_wr_ptr; + info->byte_pos_frame_start =vpu_read_reg_relaxed (inst->dev,W6_RET_DEC_AU_START_POS ); + info->byte_pos_frame_end =vpu_read_reg_relaxed (inst->dev,W6_RET_DEC_AU_END_POS ); + info->stream_end =vpu_read_reg_relaxed (inst->dev,W6_RET_DEC_STREAM_END ); + info->notification_flags =vpu_read_reg_relaxed (inst->dev,W6_RET_DEC_NOTIFICATION ); + info->last_frame_in_au =vpu_read_reg_relaxed (inst->dev,W6_RET_DEC_LAST_FRAME_FLAG ); + + reg_val =vpu_read_reg_relaxed (inst->dev,W6_RET_DEC_TIMESTAMP_LOW ); + info->timestamp =vpu_read_reg_relaxed (inst->dev,W6_RET_DEC_TIMESTAMP_HIGH ); + info->timestamp = (info->timestamp <<32 ) | reg_val; + +if (decoded_idx >=0 && decoded_idx <WAVE6_MAX_FBS ) + p_dec_info->dec_out_info[decoded_idx].decoded_poc = info->decoded_poc;
drivers/media/platform/chips-media/wave6/wave6-hw.c int wave6_vpu_get_version(struct vpu_core_device *core) · line 850
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c +10
-0+ +if (disp_idx >=0 && disp_idx <WAVE6_MAX_FBS ) { + info->display_poc = p_dec_info->dec_out_info[disp_idx].decoded_poc; + info->disp_pic_width = p_dec_info->dec_out_info[disp_idx].dec_pic_width; + info->disp_pic_height = p_dec_info->dec_out_info[disp_idx].dec_pic_height; + } + + info->sequence_no = p_dec_info->seq_info.sequence_no; +if (decoded_idx >=0 && decoded_idx <WAVE6_MAX_FBS ) + p_dec_info->dec_out_info[decoded_idx] = *info;
[Severity: Low]
When a frame is decoded and displayed in the same step (disp_idx == decoded_idx), such as on the initial IDR frame or with zero reorder delay, does reading disp_pic_width and disp_pic_height from p_dec_info->dec_out_info[disp_idx] before p_dec_info->dec_out_info[decoded_idx] is updated leave them set to 0 or stale dimensions from an earlier frame?
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-vpuapi.c new file +0
-0@@ -0,0 +1,202 @@
[ ... ]
Quoted 19 lines
drivers/media/platform/chips-media/wave6/wave6-vpuapi.c new file +19
-0+int wave6_vpu_dec_complete_seq_init (struct vpu_instance *inst, +struct dec_seq_info *info) +{ +struct dec_info *p_dec_info = &inst->codec_info->dec_info; +int ret; + +guard (mutex)(&inst->dev->hw_lock); + + ret =wave6_vpu_dec_get_seq_info (inst, info); +if (!ret) + p_dec_info->seq_info_obtained = true; + + info->rd_ptr =wave6_vpu_dec_get_rd_ptr (inst); + info->wr_ptr = p_dec_info->stream_wr_ptr; + + p_dec_info->seq_info = *info; + +return ret; +}
[ ... ]
Quoted 8 lines
drivers/media/platform/chips-media/wave6/wave6-vpuapi.c new file +8
-0+void wave6_vpu_dec_get_bitstream_buffer (struct vpu_instance *inst, +dma_addr_t *p_rd_ptr, +dma_addr_t *p_wr_ptr) +{ +struct dec_info *p_dec_info = &inst->codec_info->dec_info; +dma_addr_t rd_ptr; +dma_addr_t wr_ptr; +
drivers/media/platform/chips-media/wave6/wave6-vpuapi.c line 153
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpuapi.c new file +10
-0+guard (mutex)(&inst->dev->hw_lock); + + rd_ptr =wave6_vpu_dec_get_rd_ptr (inst); + wr_ptr = p_dec_info->stream_wr_ptr; + +if (p_rd_ptr) + *p_rd_ptr = rd_ptr; +if (p_wr_ptr) + *p_wr_ptr = wr_ptr; +}
[Severity: Low]
Can wave6_vpu_dec_get_rd_ptr() return an unrelated register value when called outside a W6_QUERY_OPT_GET_RESULT query?
W6_RET_DEC_BS_RD_PTR (offset 0x30C) shares its register offset with command registers such as W6_CMD_DEC_SET_DISP_Y_BASE and W6_CMD_DEC_PIC_SEC_AXI, and wave6_vpu_dec_get_rd_ptr() reads it directly without issuing a query command:
drivers/media/platform/chips-media/wave6/wave6-hw.c:wave6_vpu_dec_get_rd_ptr() {
lockdep_assert_held(&inst->dev->hw_lock);
return vpu_read_reg(inst->dev, W6_RET_DEC_BS_RD_PTR);
}Calling wave6_vpu_dec_get_rd_ptr() in wave6_vpu_dec_get_bitstream_buffer(), or in wave6_vpu_dec_complete_seq_init() when wave6_vpu_dec_get_seq_info() fails (which also overwrites p_dec_info->seq_info on error), reads whatever value was last written to 0x30C by a previous command.
Quoted 4 lines
drivers/media/platform/chips-media/wave6/wave6-vpuapi.h +0
-0
[ ... ]
Quoted 1 line
drivers/media/platform/chips-media/wave6/wave6-vpuapi.h +0
-0@@ -208,12 +248,116 @@ struct instance_buffer {
[ ... ]
Quoted 1 line
+struct
dec_output_info{
[ ... ]
drivers/media/platform/chips-media/wave6/wave6-vpuapi.h struct instance_buffer { · line 339
Quoted 3 lines
drivers/media/platform/chips-media/wave6/wave6-vpuapi.h +3
-0+ u32 release_disp_frame_num: 5; + u32 disp_frame_num: 5; + u32 ctu_size: 2;
[Severity: Low]
Does the 2-bit ctu_size bitfield truncate DEC_PIC_TYPE_CTU_SIZE() to 0?
wave6_vpu_dec_get_output_info() assigns info->ctu_size from DEC_PIC_TYPE_CTU_SIZE(reg_val), which evaluates to (16 << FIELD_GET(GENMASK(11, 10), (x))) and produces 16, 32, 64, or 128, none of which fit in a 2-bit bitfield.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
[email protected]
AI reviewRe: [PATCH v8 02/11] media: chips-media: wave6: Add Wave6 control driver
In reply to Nas ChungThank you for your contribution! Sashiko AI review found 8 potential issues to consider: - [High] media: wave6: vpu->lock held across core device register and unregister - [High] media: wave6: data…
Thank you for your contribution! Sashiko AI review found 8 potential issues to consider:
- [High] media: wave6: vpu->lock held across core device register and unregister
- [High] media: wave6: data race on work_buffers_avail in wave6_vpu_require_work_buffer()
- [Medium] media: wave6: stuck PREPARE state on work buf error in wave6_vpu_init_vpu()
- [Medium] media: wave6: incompatible phys_addr_t pointer in wave6_vpu_probe()
- [Medium] media: wave6: wave6_vpu_probe() succeeds without required memory-region
- [Medium] media: wave6: wrong product code value for WAVE677_CODE
- [Low] media: wave6: off-by-one of_irq_get() error check in wave6_vpu_create_cores()
- [Low] media: wave6: mixed guard(mutex) and goto cleanup in wave6_vpu_get()
--
commit 600333b152222fe3a820d242ccd857c34cf7d58c
Author: Nas Chung <[email protected]>
media: chips-media: wave6: Add Wave6 control driver
This commit adds the control driver for the Chips&Media Wave6 video codec IP to manage firmware loading, shared memory resources, and child core devices.
Quoted 5 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +0
-0
[ ... ]
Quoted 37 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +37
-0+static int wave6_vpu_init_vpu (struct wave6_vpu_device *vpu, +void __iomem *core_base) +{ +int ret; + +lockdep_assert_held (&vpu->lock); + +/* try init directly as firmware is running */ +if (wave6_vdi_readl (core_base,W6_VPU_VCPU_CUR_PC )) + goto init_done; + +wave6_vpu_set_state (vpu,WAVE6_VPU_STATE_PREPARE ); + +wave6_vpu_remap_code_buf (vpu); + +wave6_vdi_writel (core_base,W6_VPU_BUSY_STATUS ,BUSY_STATUS_SET ); +wave6_vdi_writel (core_base,W6_CMD_INIT_VPU_SEC_AXI_BASE_CORE0 , + vpu->sram_buf.dma_addr); +wave6_vdi_writel (core_base,W6_CMD_INIT_VPU_SEC_AXI_SIZE_CORE0 , + vpu->sram_buf.size); +wave6_vdi_writel (vpu->reg_base,W6_COMMAND_GB ,W6_CMD_INIT_VPU ); +wave6_vdi_writel (vpu->reg_base,W6_VPU_REMAP_CORE_START_GB , +REMAP_CORE_START_ON ); + + ret =wave6_vpu_wait_busy (core_base); +if (ret) { +dev_err (vpu->dev,"init vpu timeout\n" ); +wave6_vpu_set_state (vpu,WAVE6_VPU_STATE_OFF ); +return -EINVAL ; + } + + ret =wave6_vpu_check_result (core_base); +if (ret) { +dev_err (vpu->dev,"init vpu fail, reason 0x%x\n" , ret); +wave6_vpu_set_state (vpu,WAVE6_VPU_STATE_OFF ); +return -EIO ; + }
drivers/media/platform/chips-media/wave6/wave6-vpu.c line 278
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +10
-0+ +init_done: + ret =wave6_vpu_init_work_buf (vpu, core_base); +if (ret) +return ret; + +wave6_vpu_set_state (vpu,WAVE6_VPU_STATE_ON ); + +return 0 ; +}
[Severity: Medium]
Does returning on wave6_vpu_init_work_buf() failure leave vpu->state stuck in WAVE6_VPU_STATE_PREPARE?
Unlike the timeout and error paths above init_done, this path does not reset vpu->state to WAVE6_VPU_STATE_OFF. wave6_vpu_get() then powers down the device and drops core_count to 0, and every later wave6_vpu_get() -> wave6_vpu_try_boot() sees vpu->state == WAVE6_VPU_STATE_PREPARE and returns 0 without booting the VPU or initializing work buffers.
[ ... ]
Quoted 17 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +17
-0+int wave6_vpu_get (struct wave6_vpu_device *vpu,void __iomem *core_base) +{ +int ret; + +guard (mutex)(&vpu->lock); + +/* Only the first core executes boot; others return */ +if (atomic_inc_return (&vpu->core_count) >1 ) +return 0 ; + + ret =pm_runtime_resume_and_get (vpu->dev); +if (ret) + goto error_pm; + + ret =wave6_vpu_try_boot (vpu, core_base); +if (ret) + goto error_boot;
drivers/media/platform/chips-media/wave6/wave6-vpu.c line 410
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +10
-0+ +return 0 ; + +error_boot: +pm_runtime_put_sync (vpu->dev); +error_pm: +atomic_dec (&vpu->core_count); + +return ret; +}
[Severity: Low]
This isn't a bug, but wave6_vpu_get() mixes guard(mutex) with goto-based error unwinding in the same function, contrary to the Cleanup API conventions in include/linux/cleanup.h.
[ ... ]
Quoted 14 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +14
-0+void wave6_vpu_require_work_buffer (struct wave6_vpu_device *vpu, +void __iomem *core_base) +{ +struct vpu_buf *vb; +u32 size; + + size =wave6_vdi_readl (core_base,W6_CMD_SET_WORK_BUF_SIZE ); +if (!size) +return ; + +if (WARN_ON (size >W637DEC_WORKBUF_SIZE_FOR_CQ )) + goto exit; + +if (WARN_ON (vpu->work_buffers_avail ==0 ))
drivers/media/platform/chips-media/wave6/wave6-vpu.c line 448
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +10
-0+ goto exit; + + vpu->work_buffers_avail--; + vb = &vpu->work_buffers[vpu->work_buffers_avail]; + +wave6_vdi_writel (core_base,W6_CMD_SET_WORK_BUF_ADDR , vb->daddr); + +exit: +wave6_vdi_writel (core_base,W6_CMD_SET_WORK_BUF_SIZE ,SET_WORK_BUF_SIZE_ACK ); +}
[Severity: High]
Can concurrent calls to wave6_vpu_require_work_buffer() across VPU cores race on vpu->work_buffers_avail?
In commit "media: chips-media: wave6: Add Wave6 core driver", each child core device calls wave6_vpu_require_work_buffer() directly from its wave6_vpu_core_irq() hardirq handler without holding any shared lock:
CPU0 (<hardirq> core 0) CPU1 (<hardirq> core 1)wave6_vpu_core_irq() wave6_vpu_core_irq()
wave6_vpu_require_work_buffer() wave6_vpu_require_work_buffer()
if (work_buffers_avail == 0) if (work_buffers_avail == 0)
work_buffers_avail--; work_buffers_avail--;
vb = &work_buffers[avail]; vb = &work_buffers[avail];<-- duplicate buffer or underflow
Two cores can be assigned the same work_buffers[] entry, or when vpu->work_buffers_avail is 1, both cores can pass the zero check and underflow the u32 counter to 0xffffffff, reading vpu->work_buffers[] out of bounds.
Also, when jumping to exit on error, should W6_CMD_SET_WORK_BUF_ADDR be cleared before writing SET_WORK_BUF_SIZE_ACK so the firmware does not consume a stale address?
Quoted 17 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +17
-0+static int wave6_vpu_create_cores (struct wave6_vpu_device *vpu) +{ +struct device_node *child; +int num_cores =0 ; + +for_each_available_child_of_node (vpu->dev->of_node, child) { +struct platform_device_info info = {}; +struct platform_device *pdev; +struct resource res[2 ]; +int irq; + +if (num_cores >=W6_VPU_MAX_NUM_CORE ) { +of_node_put (child); +break ; + } + +if (of_address_to_resource (child,0 , &res[0 ])) {
drivers/media/platform/chips-media/wave6/wave6-vpu.c line 476
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +10
-0+dev_warn (vpu->dev,"%pOF: missing reg property\n" , child); +continue ; + } + + irq =of_irq_get (child,0 ); +if (irq <0 ) { +dev_warn (vpu->dev,"%pOF: missing interrupts property\n" , child); +continue ; + } + res[1 ] =DEFINE_RES_IRQ (irq);
[Severity: Low]
Should the error check for of_irq_get() be "if (irq <= 0)"?
of_irq_get() returns 0 when IRQ mapping fails, which would register the child platform device with an invalid IRQ 0 resource.
[ ... ]
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +6
-0+static void wave6_vpu_release (struct wave6_vpu_device *vpu) +{ +guard (mutex)(&vpu->lock); + +wave6_vpu_destroy_cores (vpu); +wave6_vpu_free_work_buffers (vpu);
[ ... ]
Quoted 13 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +13
-0+} + +static void wave6_vpu_init_firmware (struct wave6_vpu_device *vpu, +const struct firmware *fw) +{ +guard (mutex)(&vpu->lock); + +if (!fw || !fw->data) { +dev_err (vpu->dev,"No firmware.\n" ); +return ; + } + +if (fw->size +W6_EXTRA_CODE_BUF_SIZE >wave6_vpu_get_code_buf_size (vpu)) {
drivers/media/platform/chips-media/wave6/wave6-vpu.c line 559
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +10
-0+dev_err (vpu->dev,"firmware size (%zd > %zd) is too big\n" , + fw->size, vpu->code_buf.size); +return ; + } + +memcpy (vpu->code_buf.vaddr, fw->data, fw->size); + +if (!wave6_vpu_create_cores (vpu)) +dev_err (vpu->dev,"Failed to create VPU cores\n" ); +}
[Severity: High]
Does holding vpu->lock across wave6_vpu_create_cores() and wave6_vpu_destroy_cores() invert the lock hierarchy against child core device locks and runtime PM transitions?
wave6_vpu_create_cores() -> platform_device_register_full() and wave6_vpu_destroy_cores() -> platform_device_unregister() acquire the child core device's device_lock (core_dev->mutex) while vpu->lock is held, and platform_device_unregister() -> __device_release_driver() also runs pm_runtime_get_sync(core_dev) and pm_runtime_put_sync(core_dev). In commit "media: chips-media: wave6: Add Wave6 core driver", the child's runtime PM and system sleep callbacks call wave6_vpu_get() and wave6_vpu_put(), which acquire vpu->lock under core_dev->mutex or during a core_dev runtime PM transition:
CPU0 CPU1
---- ----lock(&vpu->lock)
lock(&core_dev->mutex)
lock(&core_dev->mutex)
lock(&vpu->lock)*** DEADLOCK ***
CPU0: wave6_vpu_init_firmware() -> wave6_vpu_create_cores() ->
platform_device_register_full()
CPU1: device_suspend() -> wave6_vpu_core_suspend() ->
pm_runtime_force_suspend() -> wave6_vpu_core_runtime_suspend() ->
wave6_vpu_put()During teardown, if core_dev is in RPM_RESUMING or RPM_SUSPENDING waiting for vpu->lock in wave6_vpu_get() or wave6_vpu_put() when wave6_vpu_release() runs, __device_release_driver()'s pm_runtime_get_sync(core_dev) will also wait on core_dev->power.wait_queue while holding vpu->lock.
[ ... ]
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +6
-0+ np =of_parse_phandle (pdev->dev.of_node,"memory-region" ,0 ); +if (np) { +struct resource mem; + + ret =of_address_to_resource (np,0 , &mem); +of_node_put (np);
drivers/media/platform/chips-media/wave6/wave6-vpu.c line 674
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +10
-0+if (!ret) { + vpu->code_buf.phys_addr = mem.start; + vpu->code_buf.size =resource_size (&mem); + ret =wave6_vpu_init_code_buf (vpu); +if (ret) + goto error_detach_pm_domains; + }else { +dev_warn (&pdev->dev,"memory-region is not available.\n" ); + } + }
[Severity: Medium]
Should wave6_vpu_probe() fail and jump to error_detach_pm_domains when "memory-region" is missing or of_address_to_resource() fails?
Without it, vpu->code_buf.size remains 0 while wave6_vpu_probe() continues, allocates 48 MiB of coherent DMA work buffers and SRAM, and returns 0, only for wave6_vpu_init_firmware() to unconditionally fail its size check against wave6_vpu_get_code_buf_size(vpu) and leave those resources allocated without creating any core devices.
drivers/media/platform/chips-media/wave6/wave6-vpu.c line 681
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +6
-0+ vpu->sram_pool =of_gen_pool_get (pdev->dev.of_node,"sram" ,0 ); +if (vpu->sram_pool) { + vpu->sram_buf.size = vpu->res->sram_size; + vpu->sram_buf.vaddr =gen_pool_dma_alloc (vpu->sram_pool, + vpu->sram_buf.size, + &vpu->sram_buf.phys_addr);
[Severity: Medium]
Does passing &vpu->sram_buf.phys_addr to gen_pool_dma_alloc() fail to build when phys_addr_t and dma_addr_t have different widths?
struct wave6_vpu_dma_buf declares phys_addr as phys_addr_t, whereas gen_pool_dma_alloc() takes a dma_addr_t * third argument, which triggers -Werror=incompatible-pointer-types on configs where CONFIG_PHYS_ADDR_T_64BIT and CONFIG_ARCH_DMA_ADDR_T_64BIT differ (exposed by COMPILE_TEST).
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h new file +0
-0@@ -0,0 +1,37 @@
[ ... ]
drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h line 16
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h new file +6
-0+#define WAVE617_CODE 0x6170 +#define WAVE627_CODE 0x6270 +#define WAVE633_CODE 0x6330 +#define WAVE637_CODE 0x6370 +#define WAVE663_CODE 0x6630 +#define WAVE677_CODE 0x6670
[Severity: Medium]
Should WAVE677_CODE be 0x6770 instead of 0x6670?
All other Wave6 product codes encode the three-digit model number in bits [15:4], so wave6_is_product_w_series() in wave6_vpu_try_boot() would reject a Wave677 reporting 0x6770 with -EINVAL.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
[email protected]
AI reviewRe: [PATCH v8 05/11] media: chips-media: wave6: Add Wave6 m2m decoder
In reply to Nas ChungThank you for your contribution! Sashiko AI review found 13 potential issues to consider: - [High] media: wave6: infinite job loop on failure in wave6_vpu_dec_seek_header() - [High] media: wave6…
Thank you for your contribution! Sashiko AI review found 13 potential issues to consider:
- [High] media: wave6: infinite job loop on failure in wave6_vpu_dec_seek_header()
- [High] media: wave6: missing CAPTURE buffer size check after format update
- [High] media: wave6: use-after-free of m2m_ctx in wave6_vpu_dec_release()
- [Medium] media: wave6: unhandled query failure in wave6_vpu_dec_finish_decode()
- [Medium] media: wave6: fbc_buf_registered not reset in wave6_vpu_dec_release_fb()
- [Medium] media: wave6: zero pixelformat matched in wave6_find_vpu_fmt()
- [Medium] media: wave6: skipped FBC setup in wave6_vpu_dec_start_streaming() after seek
- [Medium] media: wave6: duplicate display buffer entries in p_dec_info->disp_buf[]
- [Medium] media: wave6: infinite job rescheduling loop on DEC_NOTI_FLAG_NO_FB
- [Medium] media: wave6: race between wave6_vpu_dec_finish_decode() and timeout work
- [Medium] media: wave6: unlocked cap_q_ctx.rdy_queue traversal races with queue ops
- [Medium] media: wave6: unlocked state updates and early event in source change
- [Medium] media: wave6: ignored buffer registration errors in prepare_decode()
--
commit 2d9ead557a0ff6a4de5fd71e8b36d2fcff95e3a1
Author: Nas Chung <[email protected]>
media: chips-media: wave6: Add Wave6 m2m decoder
This commit adds the V4L2 m2m stateful decoder driver for the Chips&Media Wave6 video codec IP, supporting H.264 and HEVC 8-bit decoding, downscaling, and dynamic resolution changes.
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c new file +0
-0@@ -0,0 +1,1935 @@
[ ... ]
Quoted 19 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c new file +19
-0+static const struct vpu_format wave6_vpu_dec_fmt_list[2 ][6 ] = { + [VPU_FMT_TYPE_CODEC ] = { + { + .v4l2_pix_fmt =V4L2_PIX_FMT_HEVC , + .max_width =W6_MAX_DEC_PIC_WIDTH , + .min_width =W6_MIN_DEC_PIC_WIDTH , + .max_height =W6_MAX_DEC_PIC_HEIGHT , + .min_height =W6_MIN_DEC_PIC_HEIGHT , + .num_planes =1 , + }, + { + .v4l2_pix_fmt =V4L2_PIX_FMT_H264 , + .max_width =W6_MAX_DEC_PIC_WIDTH , + .min_width =W6_MIN_DEC_PIC_WIDTH , + .max_height =W6_MAX_DEC_PIC_HEIGHT , + .min_height =W6_MIN_DEC_PIC_HEIGHT , + .num_planes =1 , + }, + },
[ ... ]
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c line 113
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c new file +10
-0+static const struct vpu_format *wave6_find_vpu_fmt (unsigned int v4l2_pix_fmt, +enum vpu_fmt_type type) +{ +for (int i =0 ; i <ARRAY_SIZE (wave6_vpu_dec_fmt_list[type]); i++) { +if (wave6_vpu_dec_fmt_list[type][i].v4l2_pix_fmt == v4l2_pix_fmt) +return &wave6_vpu_dec_fmt_list[type][i]; + } + +return NULL ; +}
[Severity: Medium]
Can wave6_find_vpu_fmt() match an unpopulated entry in wave6_vpu_dec_fmt_list[VPU_FMT_TYPE_CODEC] when v4l2_pix_fmt is 0?
VPU_FMT_TYPE_CODEC only populates 2 of its 6 slots, leaving slots 2..5 zero-initialized. Passing pixelformat == 0 to VIDIOC_S_FMT on OUTPUT matches slot 2 instead of falling back to inst->src_fmt, which sets inst->src_fmt.num_planes and plane sizes to 0 and triggers WARN_ON() in vb2_core_reqbufs() on a subsequent VIDIOC_REQBUFS.
[ ... ]
Quoted 22 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c new file +22
-0+static void wave6_vpu_dec_release_fb (struct vpu_instance *inst) +{ +for (int i =0 ; i <WAVE6_MAX_FBS ; i++) { +wave6_vdi_free_dma (&inst->frame_vbuf[i]); +memset (&inst->frame_buf[i],0 ,sizeof (struct frame_buffer )); +wave6_vdi_free_dma (&inst->aux_vbuf[AUX_BUF_FBC_Y_TBL ][i]); +wave6_vdi_free_dma (&inst->aux_vbuf[AUX_BUF_FBC_C_TBL ][i]); +wave6_vdi_free_dma (&inst->aux_vbuf[AUX_BUF_MV_COL ][i]); + } +} + +static void wave6_vpu_dec_destroy_instance (struct vpu_instance *inst) +{ +u32 fail_res =0 ; +int ret; + +wave6_vpu_disable_instance (inst); + + ret =wave6_vpu_dec_close (inst, &fail_res); +if (ret) { +dev_err (inst->dev->dev,"failed destroy instance: %d (%d)\n" , + ret, fail_res);
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c line 158
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c new file +10
-0+ } + + scoped_guard(spinlock, &inst->dev->inst_lock) +list_del_init (&inst->list); +wave6_vpu_dec_release_fb (inst); +wave6_vpu_set_instance_state (inst,VPU_INST_STATE_NONE ); + +pm_runtime_put_sync (inst->dev->dev); +wave6_vpu_core_deactivate (inst->dev); +}
[Severity: Medium]
Should wave6_vpu_dec_release_fb() or wave6_vpu_dec_destroy_instance() reset inst->fbc_buf_registered to false?
When wave6_vpu_dec_queue_setup() destroys the instance in VPU_INST_STATE_SEEK and a new instance is created on the next OUTPUT STREAMON, inst->fbc_buf_registered remains true from the destroyed instance even though all FBC buffers were freed. If V4L2_DEC_CMD_STOP is then issued before a sequence header is parsed, wave6_vpu_dec_prepare_decode() skips the !inst->fbc_buf_registered early drain exit and issues W6_CMD_INIT_SEQ on an empty bitstream.
[ ... ]
Quoted 4 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c new file +4
-0+static void wave6_vpu_dec_handle_dst_buffer (struct vpu_instance *inst) +{ +struct vb2_v4l2_buffer *dst_buf; +struct v4l2_m2m_buffer *v4l2_m2m_buf;
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c line 321
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c new file +10
-0+struct vpu_buffer *vpu_buf; +u32 fb_stride = inst->dst_fmt.plane_fmt[0 ].bytesperline; +u32 luma_size = fb_stride * inst->dst_fmt.height; +u32 chroma_size = (fb_stride /2 ) * (inst->dst_fmt.height /2 ); +struct frame_buffer disp_buffer = { }; +struct dec_seq_info *seq_info = &inst->codec_info->dec_info.seq_info; +int consumed_num =wave6_vpu_get_consumed_fb_num (inst); +int ret; + +v4l2_m2m_for_each_dst_buf (inst->v4l2_fh.m2m_ctx, v4l2_m2m_buf) {
[Severity: Medium]
What serializes v4l2_m2m_for_each_dst_buf() against concurrent list modifications on cap_q_ctx.rdy_queue?
v4l2_m2m_for_each_dst_buf() iterates cap_q_ctx.rdy_queue without holding cap_q_ctx.rdy_spinlock:
kworker CPU0 (VIDIOC_QBUF)wave6_vpu_device_run()
wave6_vpu_dec_prepare_decode()
wave6_vpu_dec_handle_dst_buffer()
v4l2_m2m_for_each_dst_buf()
wave6_vpu_dec_buf_queue_dst()
guard(mutex)(&inst->lock)
v4l2_m2m_buf_queue()
list_add_tail(&b->list, ...)
/* unlocked list traversal */Similarly, wave6_vpu_core_irq_thread() traverses cap_q_ctx.rdy_queue unlocked in wave6_get_dst_buf_by_addr() and calls v4l2_m2m_dst_buf_remove_by_buf(), which can race with wave6_vpu_dec_buf_queue_dst() or with v4l2_m2m_for_each_dst_buf() in wave6_vpu_dec_cmd_start() and corrupt the list traversal.
Quoted 28 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c new file +28
-0+ dst_buf = &v4l2_m2m_buf->vb; + vpu_buf =wave6_to_vpu_buf (dst_buf); + +if (vpu_buf->consumed) +continue ; + +if (consumed_num >=WAVE6_MAX_FBS ) +break ; + +if (inst->dst_fmt.num_planes ==1 ) { + disp_buffer.buf_y =wave6_get_dma_addr (dst_buf,0 ); + disp_buffer.buf_cb = disp_buffer.buf_y + luma_size; + disp_buffer.buf_cr = disp_buffer.buf_cb + chroma_size; + }else if (inst->dst_fmt.num_planes ==2 ) { + disp_buffer.buf_y =wave6_get_dma_addr (dst_buf,0 ); + disp_buffer.buf_cb =wave6_get_dma_addr (dst_buf,1 ); + disp_buffer.buf_cr = disp_buffer.buf_cb + chroma_size; + }else if (inst->dst_fmt.num_planes ==3 ) { + disp_buffer.buf_y =wave6_get_dma_addr (dst_buf,0 ); + disp_buffer.buf_cb =wave6_get_dma_addr (dst_buf,1 ); + disp_buffer.buf_cr =wave6_get_dma_addr (dst_buf,2 ); + } + disp_buffer.height = inst->src_fmt.height; + disp_buffer.stride = fb_stride; + disp_buffer.map_type =LINEAR_FRAME_MAP ; + disp_buffer.luma_bit_depth = seq_info->luma_bit_depth; + disp_buffer.chroma_bit_depth = seq_info->chroma_bit_depth; + disp_buffer.c_fmt_idc = seq_info->c_fmt_idc;
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c line 359
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c new file +10
-0+ + ret =wave6_vpu_dec_register_display_buffer (inst, disp_buffer); +if (ret) { +dev_err (inst->dev->dev,"fail register display buffer %d" , ret); +break ; + } + + vpu_buf->consumed = true; + consumed_num++; + }
[Severity: Medium]
Does re-queuing a displayed CAPTURE buffer insert duplicate entries into p_dec_info->disp_buf[] until all WAVE6_MAX_FBS slots are exhausted?
When a displayed buffer is dequeued and re-queued via VIDIOC_QBUF, wave6_vpu_dec_buf_queue() resets vpu_buf->consumed = false, so wave6_vpu_dec_handle_dst_buffer() calls wave6_vpu_dec_register_display_buffer() again. In commit "media:
chips-media: wave6: Add Wave6 decoder firmware interface", wave6_dec_set_display_buffer() always writes to the first empty !disp_buf[i].buf_y slot without checking if fb.buf_y is already present, and wave6_vpu_dec_get_output_info() only removes buffers on W6_RET_DEC_RELEASE_IDC, not W6_RET_DEC_DISP_IDC:
drivers/media/platform/chips-media/wave6/wave6-hw.c:wave6_dec_set_display_buffer() {
for (int i = 0; i < WAVE6_MAX_FBS; i++) {
if (!p_dec_info->disp_buf[i].buf_y) {
p_dec_info->disp_buf[i] = fb;
break;
}
}
}After 31 displayed frames are re-queued, p_dec_info->disp_buf[] is full of duplicates and cannot record any new buffer address.
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c line 368
Quoted 9 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c new file +9
-0+ +if (!inst->fbc_buf_registered) { + ret =wave6_vpu_dec_register_frame_buffer (inst); +if (ret) +dev_err (inst->dev->dev,"register frame buffer fail %d\n" , ret); +else + inst->fbc_buf_registered = true; + } +}
[Severity: Medium]
Should wave6_vpu_dec_handle_dst_buffer() propagate registration errors to wave6_vpu_dec_prepare_decode()?
If wave6_vpu_dec_register_frame_buffer() or wave6_vpu_dec_register_display_buffer() fails, the error is logged and dropped, and wave6_vpu_dec_prepare_decode() still returns 0. wave6_vpu_device_run() then arms inst->task_timer and calls wave6_vpu_dec_start_decode() without registered FBC or display buffers.
[ ... ]
Quoted 23 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c new file +23
-0+static void wave6_vpu_dec_handle_source_change (struct vpu_instance *inst) +{ +static const struct v4l2_event vpu_event_src_ch = { + .type =V4L2_EVENT_SOURCE_CHANGE , + .u.src_change.changes =V4L2_EVENT_SRC_CH_RESOLUTION , + }; +struct dec_seq_info *info = &inst->codec_info->dec_info.seq_info; + +dev_dbg (inst->dev->dev,"pic size %dx%d profile %d, min_fb_cnt : %d | min_disp_cnt : %d\n" , + info->pic_width, info->pic_height, + info->profile, info->min_frame_buffer_count, info->frame_buf_delay); + +wave6_vpu_dec_retry_one_frame (inst); +wave6_vpu_dec_reset_frame_buffer_info (inst); + inst->fbc_buf_registered = false; + +wave6_vpu_set_instance_state (inst,VPU_INST_STATE_INIT_SEQ ); + + inst->scaler_info.enable = false; + inst->crop.left = info->pic_crop_rect.left; + inst->crop.top = info->pic_crop_rect.top; + inst->crop.width = info->pic_crop_rect.right - inst->crop.left; + inst->crop.height = info->pic_crop_rect.bottom - inst->crop.top;
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c line 773
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c new file +10
-0+ +wave6_update_v4l2_ctrls (inst, info); +wave6_update_color_info (inst, info); +wave6_update_pix_fmt (&inst->src_fmt, info->pic_width, info->pic_height); +wave6_update_pix_fmt_cap (&inst->dst_fmt, + info->pic_width, info->pic_height, + true); + +v4l2_event_queue_fh (&inst->v4l2_fh, &vpu_event_src_ch); +}
[Severity: Medium]
Can queuing V4L2_EVENT_SOURCE_CHANGE inside wave6_vpu_dec_handle_source_change() wake userspace before the caller finishes marking the last frame?
CPU0 (irq_thread / kworker) CPU1 (userspace)wave6_vpu_dec_seek_header()
wave6_vpu_dec_handle_source_change()
/* updates formats unlocked */
v4l2_event_queue_fh(...)
/* wakes on SOURCE_CHANGE event */
VIDIOC_STREAMON(CAPTURE)
VIDIOC_QBUF(CAPTURE) if (vb2_is_streaming(dst_vq))
wave6_handle_last_frame() <-- tags new buffer with LAST
In addition, wave6_vpu_dec_handle_source_change() modifies inst->state, inst->crop, inst->src_fmt, and inst->dst_fmt without holding core->dev_lock, so a concurrent VIDIOC_G_FMT or VIDIOC_TRY_FMT can observe partially updated format fields (such as bytesperline temporarily set to 0). Queuing the event after wave6_handle_last_frame() and serializing the format update would avoid this race.
[ ... ]
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c line 796
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c new file +10
-0+static void wave6_vpu_dec_finish_decode (struct vpu_instance *inst,bool error) +{ +struct dec_output_info info; +int ret; + +memset (&info,0 ,sizeof (info)); + + ret =wave6_vpu_dec_get_output_info (inst, &info); +if (ret) + goto finish_decode;
[Severity: Medium]
What synchronizes wave6_vpu_dec_finish_decode() against inst->task_timer (wave6_vpu_device_run_timeout())?
wave6_vpu_dec_finish_decode() does not cancel inst->task_timer on entry; wave6_vpu_finish_job() only calls non-synchronous cancel_delayed_work() at the very end:
CPU0 (irq_thread) CPU1 (kworker)wave6_vpu_dec_finish_decode() wave6_vpu_device_run_timeout()
wave6_handle_decoded_frame()
src_buf = v4l2_m2m_next_src_buf()
src_buf = v4l2_m2m_src_buf_remove()
v4l2_m2m_buf_done(src_buf, ...)
wave6_vpu_dec_src_buf_done()
v4l2_m2m_src_buf_remove_by_buf()
v4l2_m2m_buf_done(src_buf) <-- double remove and buf_doneIf the timeout fires while the completion interrupt is being handled, both paths remove and complete the same source and destination buffers and both call v4l2_m2m_job_finish().
[Severity: Medium]
If wave6_vpu_dec_get_output_info() fails here, does jumping to finish_decode leave the consumed head OUTPUT buffer on the ready queue?
Unlike wave6_vpu_dec_start_decode(), this error path neither removes and completes the head src_buf with VB2_BUF_STATE_ERROR nor transitions inst->state to VPU_INST_STATE_ERROR, so v4l2_m2m_job_finish() can schedule the next job with the already-consumed buffer still at the head of the queue.
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c line 805
Quoted 9 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c new file +9
-0+ +dev_dbg (inst->dev->dev,"dec %d dis %d noti_flag %d stream_end %d\n" , + info.frame_decoded, info.frame_display, + info.notification_flags, info.stream_end); + +if (info.notification_flags &DEC_NOTI_FLAG_NO_FB ) { +wave6_vpu_dec_retry_one_frame (inst); + goto finish_decode; + }
[Severity: Medium]
Does DEC_NOTI_FLAG_NO_FB cause an immediate job rescheduling busy loop?
In wave6_vpu_dec_handle_dst_buffer(), queued CAPTURE buffers have vpu_buf->consumed set to true while vpu_buf->used remains false until wave6_handle_decoded_frame(). When DEC_NOTI_FLAG_NO_FB jumps to finish_decode without setting vpu_buf->used, wave6_vpu_check_fb_available() in commit "media: chips-media: wave6: Add Wave6 core driver" still sees !vpu_buf->used and reports the job as ready:
drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c:wave6_vpu_check_fb_available() {
v4l2_m2m_for_each_dst_buf(inst->v4l2_fh.m2m_ctx, v4l2_m2m_buf) {
vb2_v4l2_buf = &v4l2_m2m_buf->vb;
vpu_buf = wave6_to_vpu_buf(vb2_v4l2_buf);
if (!vpu_buf->used)
return true;
}
}v4l2_m2m_job_finish() then immediately re-runs the job without waiting for a new CAPTURE buffer, and wave6_vpu_dec_handle_dst_buffer() skips all existing CAPTURE buffers because their vpu_buf->consumed is already true.
[ ... ]
Quoted 2 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c new file +2
-0+static int wave6_vpu_dec_s_fmt_out (struct file *file,void *fh,struct v4l2_format *f) +{
[ ... ]
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c line 1100
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c new file +6
-0+wave6_update_pix_fmt_cap (&inst->dst_fmt, + pix_mp->width, pix_mp->height, + true); + +return 0 ; +}
[Severity: High]
Can updating inst->dst_fmt in wave6_vpu_dec_s_fmt_out() or wave6_vpu_dec_handle_source_change() allow undersized CAPTURE buffers to be registered for DMA?
If userspace allocates and queues (or prepares via VIDIOC_PREPARE_BUF) CAPTURE buffers at the default 720x480 format before CAPTURE STREAMON, wave6_vpu_dec_buf_prepare() validates their sizes against 720x480 and marks them prepared. If inst->dst_fmt is then increased (up to 4096x4096) via VIDIOC_S_FMT on OUTPUT or sequence header parsing, VIDIOC_STREAMON on CAPTURE calls wave6_vpu_dec_start_streaming() -> wave6_vpu_dec_prepare_fb() without re-checking the prepared buffers' plane sizes against the new inst->dst_fmt.plane_fmt[i].sizeimage (unlike wave6_vpu_dec_cmd_start()). wave6_vpu_dec_handle_dst_buffer() then registers those smaller DMA buffers with the hardware using the new stride and height, causing out-of-bounds DMA writes.
[ ... ]
Quoted 57 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c new file +57
-0+static int wave6_vpu_dec_seek_header (struct vpu_instance *inst) +{ +struct dec_seq_info seq_info; +int ret; + +memset (&seq_info,0 ,sizeof (struct dec_seq_info )); + + ret =wave6_vpu_dec_issue_seq_init (inst); +if (ret) { +dev_err (inst->dev->dev,"failed wave6_vpu_dec_issue_seq_init %d\n" , ret); +return ret; + } + + ret =wave6_vpu_wait_interrupt (inst,W6_VPU_TIMEOUT ); +if (ret) { +dev_err (inst->dev->dev,"failed to call vpu_wait_interrupt()\n" ); +return ret; + } + + ret =wave6_vpu_dec_complete_seq_init (inst, &seq_info); +if (ret) { +dev_err (inst->dev->dev,"vpu_dec_complete_seq_init: %d, reason : 0x%x\n" , + ret, seq_info.err_reason); +if ((seq_info.err_reason &WAVE6_SYSERR_NOT_SUPPORT ) || + (seq_info.err_reason &WAVE6_SYSERR_NOT_SUPPORT_PROFILE )) { + ret = -EINVAL ; + }else if (seq_info.err_reason ==HEVC_ETCERR_INIT_SEQ_SPS_NOT_FOUND || + seq_info.err_reason ==AVC_ETCERR_INIT_SEQ_SPS_NOT_FOUND ) { +wave6_handle_skipped_frame (inst); + ret =0 ; + } + }else { +wave6_vpu_dec_handle_source_change (inst); + inst->v4l2_fh.m2m_ctx->ignore_cap_streaming = false; +v4l2_m2m_set_dst_buffered (inst->v4l2_fh.m2m_ctx, false); +if (vb2_is_streaming (v4l2_m2m_get_dst_vq (inst->v4l2_fh.m2m_ctx))) { + scoped_guard(mutex, &inst->lock) +wave6_handle_last_frame (inst,NULL ); + } + } + +return ret; +} + +static int wave6_vpu_dec_prepare_decode (struct vpu_instance *inst) +{ +bool stream_end; + +scoped_guard (mutex, &inst->lock) + stream_end = inst->drain && !inst->drain_src_buf_num; + +if (stream_end && !inst->fbc_buf_registered) { +wave6_vpu_dec_finish_drain (inst); +return -EAGAIN ; + } + +wave6_handle_bitstream_buffer (inst, stream_end);
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c line 1545
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c new file +10
-0+ +if (inst->state <VPU_INST_STATE_PIC_RUN ) { +if (inst->state ==VPU_INST_STATE_OPEN && +wave6_vpu_dec_seek_header (inst)) { +vb2_queue_error (v4l2_m2m_get_src_vq (inst->v4l2_fh.m2m_ctx)); +vb2_queue_error (v4l2_m2m_get_dst_vq (inst->v4l2_fh.m2m_ctx)); + } + +return -EAGAIN ; + }
[Severity: High]
Does a failure in wave6_vpu_dec_seek_header() trigger an infinite m2m job loop?
When wave6_vpu_dec_seek_header() fails with any error other than SPS_NOT_FOUND (such as WAVE6_SYSERR_NOT_SUPPORT_PROFILE on a 10-bit stream or a timeout), the consumed head OUTPUT buffer is not removed from the ready queue and inst->state remains VPU_INST_STATE_OPEN. Calling vb2_queue_error() only sets q->error without stopping streaming, and wave6_vpu_job_ready() returns 1 whenever inst->state == VPU_INST_STATE_OPEN. When wave6_vpu_dec_prepare_decode() returns -EAGAIN, wave6_vpu_device_run() calls v4l2_m2m_job_finish(), which immediately re-queues and re-runs the job in an unbounded loop on the same buffer.
[ ... ]
Quoted 15 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c new file +15
-0+static int wave6_vpu_dec_start_streaming (struct vb2_queue *q,unsigned int count) +{ +struct vpu_instance *inst =vb2_get_drv_priv (q); +struct v4l2_pix_format_mplane *fmt; +int ret =0 ; + +if (V4L2_TYPE_IS_OUTPUT (q->type)) { + fmt = &inst->src_fmt; +if (inst->state ==VPU_INST_STATE_NONE ) { + ret =wave6_vpu_dec_create_instance (inst); +if (!ret) { + inst->v4l2_fh.m2m_ctx->ignore_cap_streaming = true; +v4l2_m2m_set_dst_buffered (inst->v4l2_fh.m2m_ctx, true); + } + }
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c line 1625
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c new file +10
-0+if (inst->state ==VPU_INST_STATE_SEEK ) +wave6_vpu_set_instance_state (inst, inst->state_in_seek); + }else { + fmt = &inst->dst_fmt; +if (inst->state ==VPU_INST_STATE_INIT_SEQ ) { + ret =wave6_vpu_dec_prepare_fb (inst); +if (ret) + goto exit; + } + }
[Severity: Medium]
Can wave6_vpu_dec_prepare_fb() be skipped if CAPTURE streaming starts while inst->state is VPU_INST_STATE_SEEK?
If OUTPUT streaming is stopped while in VPU_INST_STATE_INIT_SEQ, wave6_vpu_dec_stop_streaming() saves inst->state_in_seek = VPU_INST_STATE_INIT_SEQ and sets inst->state = VPU_INST_STATE_SEEK. If userspace then starts CAPTURE streaming before restarting OUTPUT streaming, CAPTURE start_streaming sees inst->state == VPU_INST_STATE_SEEK and skips wave6_vpu_dec_prepare_fb(), and OUTPUT start_streaming only restores inst->state to VPU_INST_STATE_INIT_SEQ without calling wave6_vpu_dec_prepare_fb(), leaving decoding permanently stalled in VPU_INST_STATE_INIT_SEQ.
Also, in wave6_vpu_dec_try_fmt_cap(), checking inst->state >= VPU_INST_STATE_INIT_SEQ evaluates to true in VPU_INST_STATE_SEEK even when inst->state_in_seek was VPU_INST_STATE_OPEN.
[ ... ]
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c line 1882
Quoted 8 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c new file +8
-0+static int wave6_vpu_dec_release (struct file *filp) +{ +struct vpu_instance *inst =wave6_file_to_vpu_inst (filp); + +v4l2_m2m_ctx_release (inst->v4l2_fh.m2m_ctx); + +if (inst->state !=VPU_INST_STATE_NONE ) +wave6_vpu_dec_destroy_instance (inst);
[Severity: High]
Can releasing inst->v4l2_fh.m2m_ctx before wave6_vpu_dec_destroy_instance() cause a use-after-free on m2m_ctx?
v4l2_m2m_ctx_release() waits only for TRANS_RUNNING to clear and then frees m2m_ctx, whereas wave6_vpu_disable_instance() (which sets inst->enable = false, calls synchronize_irq(), and calls cancel_delayed_work_sync() on inst->task_timer) only runs afterward inside wave6_vpu_dec_destroy_instance():
CPU0 CPU1 (irq_thread / kworker)wave6_vpu_dec_release()
wave6_vpu_device_run_timeout()
v4l2_m2m_job_finish()
/* clears TRANS_RUNNING */
v4l2_m2m_ctx_release()
kfree(m2m_ctx)
wave6_vpu_core_irq_thread()
wave6_vpu_dec_finish_decode()
...
v4l2_m2m_next_src_buf(m2m_ctx)
<-- use-after-free wave6_vpu_dec_destroy_instance()
wave6_vpu_disable_instance()
A similar race happens if wave6_vpu_finish_job() calls non-synchronous cancel_delayed_work(&inst->task_timer) and clears TRANS_RUNNING while wave6_vpu_device_run_timeout() is already running on another CPU. Calling wave6_vpu_disable_instance() before v4l2_m2m_ctx_release() would ensure the IRQ thread and timeout work cannot touch m2m_ctx after it is freed.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
[email protected]
AI reviewRe: [PATCH v8 03/11] media: chips-media: wave6: Add Wave6 core driver
In reply to Nas ChungThank you for your contribution! Sashiko AI review found 7 potential issues to consider: - [High] media: wave6: data race in wave6_vpu_require_work_buffer() across cores - [High] media: wave6…
Thank you for your contribution! Sashiko AI review found 7 potential issues to consider:
- [High] media: wave6: data race in wave6_vpu_require_work_buffer() across cores
- [High] media: wave6: dropped interrupts in wave6_vpu_core_irq() and irq_thread
- [High] media: wave6: unlocked rdy_queue traversal in wave6_vpu_check_fb_available()
- [Medium] media: wave6: skipped VPU init and active_count race in runtime_resume
- [Medium] media: wave6: race and missing error state in wave6_vpu_device_run_timeout()
- [Medium] media: wave6: stale codec sizeimage in wave6_update_pix_fmt()
- [Medium] media: wave6: vpu->lock inversion and clock leak on core removal
--
commit 203cdebd14b0043756ee7b19b462b06f101581ec
Author: Nas Chung <[email protected]>
media: chips-media: wave6: Add Wave6 core driver
This commit adds the core platform driver for the Chips&Media Wave6 video codec IP to manage the per-core interface register regions, interrupts, runtime PM, and V4L2 mem2mem helpers.
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c new file +0
-0@@ -0,0 +1,392 @@
[ ... ]
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c new file +10
-0+static irqreturn_t wave6_vpu_core_irq (int irq,void *dev_id) +{ +struct vpu_core_device *core = dev_id; +struct vpu_irq irq_info; + +if (!vpu_read_reg (core,W6_VPU_VPU_INT_STS )) +return IRQ_NONE ; + + irq_info.status =vpu_read_reg (core,W6_VPU_VINT_REASON ); + irq_info.inst_idc =vpu_read_reg (core,W6_RET_INT_INSTANCE_INFO );
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c line 39
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c new file +10
-0+ +vpu_write_reg (core,W6_RET_INT_INSTANCE_INFO ,INT_INSTANCE_INFO_CLEAR ); +vpu_write_reg (core,W6_VPU_VINT_REASON_CLEAR , irq_info.status); +vpu_write_reg (core,W6_VPU_VINT_CLEAR ,VINT_CLEAR ); + +if (irq_info.status &BIT (W6_INT_BIT_REQ_WORK_BUF )) { +wave6_vpu_require_work_buffer (core->vpu, core->reg_base); + +return IRQ_HANDLED ; + }
[Severity: High]
Can concurrent W6_INT_BIT_REQ_WORK_BUF interrupts on different cores race on core->vpu->work_buffers_avail in wave6_vpu_require_work_buffer()?
Each core has its own interrupt line, but all cores share the parent core->vpu:
drivers/media/platform/chips-media/wave6/wave6-vpu.c:wave6_vpu_require_work_buffer() {
...
if (WARN_ON(vpu->work_buffers_avail == 0))
goto exit;
vpu->work_buffers_avail--;
vb = &vpu->work_buffers[vpu->work_buffers_avail];
...
}CPU0 (core 0 hardirq) CPU1 (core 1 hardirq)wave6_vpu_core_irq() wave6_vpu_core_irq()
wave6_vpu_require_work_buffer() wave6_vpu_require_work_buffer()
work_buffers_avail == 1 work_buffers_avail == 1
vpu->work_buffers_avail-- (0)
vpu->work_buffers_avail-- (U32_MAX)
&vpu->work_buffers[U32_MAX] <-- OOB readEven when work_buffers_avail > 1, two cores can read the same decremented index and assign the same DMA work buffer to two codec instances.
[ ... ]
Quoted 14 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c new file +14
-0+static struct vpu_instance *wave6_vpu_core_get_instance (struct vpu_core_device *core, +u32 inst_idc) +{ +struct vpu_instance *inst; + +guard (spinlock)(&core->inst_lock); + +list_for_each_entry (inst, &core->instances, list) { +if ((BIT (inst->id) & inst_idc) && inst->enable) +return inst; + } + +return NULL ; +}
[ ... ]
Quoted 20 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c new file +20
-0+static irqreturn_t wave6_vpu_core_irq_thread (int irq,void *dev_id) +{ +struct vpu_core_device *core = dev_id; +struct vpu_instance *inst; +struct vpu_irq irq_info; + +while (kfifo_len (&core->irq_fifo)) { +bool error = false; + +if (!kfifo_out (&core->irq_fifo, &irq_info,sizeof (struct vpu_irq ))) +break ; + + inst =wave6_vpu_core_get_instance (core, irq_info.inst_idc); +if (!inst) +continue ; + +if (irq_info.status &BIT (W6_INT_BIT_INIT_SEQ )) { +complete (&inst->irq_done); +continue ; + }
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c line 108
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c new file +10
-0+ +if (irq_info.status &BIT (W6_INT_BIT_BSBUF_ERROR )) + error = true; + +if (inst->ops && inst->ops->finish_process) + inst->ops->finish_process (inst, error); + } + +return IRQ_HANDLED ; +}
[Severity: High]
Can wave6_vpu_core_irq() and wave6_vpu_core_irq_thread() drop pending interrupts when multiple status or instance bits are set at once?
In wave6_vpu_core_irq(), W6_RET_INT_INSTANCE_INFO and W6_VPU_VINT_REASON_CLEAR are cleared for the entire irq_info.status mask before the BIT(W6_INT_BIT_REQ_WORK_BUF) check. If W6_INT_BIT_REQ_WORK_BUF is set alongside another bit such as W6_INT_BIT_DEC_PIC or W6_INT_BIT_INIT_SEQ, returning IRQ_HANDLED early without queuing the remaining bits to core->irq_fifo loses that completion interrupt.
In addition, irq_info.inst_idc is a bitmask of instance IDs, and W6_RET_INT_INSTANCE_INFO has already been cleared to 0 in the hardirq handler. Because wave6_vpu_core_irq_thread() calls wave6_vpu_core_get_instance() only once per FIFO entry and returns the first matching instance, any other instances with bits set in irq_info.inst_idc never receive their completion callback and will time out.
[ ... ]
Quoted 17 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c new file +17
-0+void wave6_vpu_core_activate (struct vpu_core_device *core) +{ +atomic_inc (&core->active_count); +} + +void wave6_vpu_core_deactivate (struct vpu_core_device *core) +{ +atomic_dec (&core->active_count); +} + +static int wave6_vpu_core_wait_activated (struct vpu_core_device *core) +{ +if (!atomic_read (&core->active_count)) +return 0 ; + +return wave6_vpu_core_check_state (core); +}
[ ... ]
Quoted 13 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c new file +13
-0+ ret =devm_pm_runtime_enable (&pdev->dev); +if (ret) +return ret; + +return 0 ; +} + +static int wave6_vpu_core_runtime_suspend (struct device *dev) +{ +struct vpu_core_device *core =dev_get_drvdata (dev); + +if (WARN_ON (!core)) +return -ENODEV ;
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c line 299
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c new file +10
-0+ +if (core->vpu_held) { +wave6_vpu_put (core->vpu, core->reg_base); + core->vpu_held = false; + } + +clk_bulk_disable_unprepare (core->num_clks, core->clks); + +return 0 ; +}
[Severity: Medium]
Can unregistering child core devices under vpu->lock in wave6_vpu_release() deadlock with wave6_vpu_core_runtime_suspend() or leak prepared clocks on teardown?
drivers/media/platform/chips-media/wave6/wave6-vpu.c:wave6_vpu_release() {
guard(mutex)(&vpu->lock);
wave6_vpu_destroy_cores(vpu);
...
}wave6_vpu_destroy_cores() calls platform_device_unregister() while holding vpu->lock, which waits for child runtime PM transitions and barriers, whereas wave6_vpu_core_runtime_suspend() and wave6_vpu_core_runtime_resume() call wave6_vpu_put() and wave6_vpu_get(), which acquire vpu->lock:
CPU0 CPU1
---- ----lock(&vpu->lock)
/* runtime PM transition on core */
/* waits for core runtime PM */
lock(&vpu->lock)*** DEADLOCK ***
CPU0: wave6_vpu_release() -> wave6_vpu_destroy_cores() -> platform_device_unregister()
CPU1: rpm_suspend() -> wave6_vpu_core_runtime_suspend() -> wave6_vpu_put()
Furthermore, wave6_vpu_core_driver has no remove callback or devm action to suspend a runtime-active core before devm_clk_bulk_get_all() releases core->clks. If a core is runtime-active at unbind (for example via sysfs power/control set to "on"), core->clks are never disabled or unprepared and any held core->vpu_held reference is leaked.
Quoted 16 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c new file +16
-0+ +static int wave6_vpu_core_runtime_resume (struct device *dev) +{ +struct vpu_core_device *core =dev_get_drvdata (dev); +int ret; + +if (WARN_ON (!core)) +return -ENODEV ; + + ret =clk_bulk_prepare_enable (core->num_clks, core->clks); +if (ret) { +dev_err (dev,"failed to enable clocks: %d\n" , ret); +return ret; + } + +if (atomic_read (&core->active_count)) {
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c line 325
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c new file +10
-0+ ret =wave6_vpu_get (core->vpu, core->reg_base); +if (ret) + goto error_get_vpu; + + core->vpu_held = true; + } + + ret =wave6_vpu_core_wait_activated (core); +if (ret) + goto error_activate;
[Severity: Medium]
What happens if wave6_vpu_core_runtime_resume() runs while core->active_count is 0?
If the core device is runtime-resumed before wave6_vpu_core_activate() is called (for instance when runtime PM is forced on via sysfs power/control), wave6_vpu_core_runtime_resume() skips both wave6_vpu_get() and wave6_vpu_core_check_state(), leaving core->vpu_held false. When an instance later calls wave6_vpu_core_activate() followed by pm_runtime_resume_and_get(), runtime PM sees the device is already RPM_ACTIVE and returns 0 without running wave6_vpu_core_runtime_resume(), so the parent VPU is never powered on or initialized and core interrupts are never enabled.
Also, because wave6_vpu_core_runtime_resume() and wave6_vpu_core_wait_activated() read core->active_count twice without synchronization (rather than checking core->vpu_held in wave6_vpu_core_wait_activated()), an intervening wave6_vpu_core_activate() can cause wave6_vpu_core_check_state() to poll VPU registers when wave6_vpu_get() was skipped.
Quoted 27 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c new file +21
-0@@ -0,0 +1,384 @@ +// SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) +/* +* Wave6 series multi-standard codec IP - v4l2 driver helper interface +* +* Copyright (C) 2026 CHIPS&MEDIA INC +*/ + +#include "wave6-vpu.h" +#include "wave6-vpu-core.h" + +void wave6_update_pix_fmt (struct v4l2_pix_format_mplane *pix_mp, +unsigned int width, +unsigned int height) +{ +const struct v4l2_format_info *fmt_info; +unsigned int stride_y; + + pix_mp->width = width; + pix_mp->height = height; + pix_mp->flags =0 ; + pix_mp->field =V4L2_FIELD_NONE ;
drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c line 31
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c new file +10
-0+memset (pix_mp->reserved,0 ,sizeof (pix_mp->reserved)); + + fmt_info =v4l2_format_info (pix_mp->pixelformat); +if (!fmt_info) { + pix_mp->plane_fmt[0 ].bytesperline =0 ; +if (!pix_mp->plane_fmt[0 ].sizeimage) + pix_mp->plane_fmt[0 ].sizeimage = width * height; + +return ; + }
[Severity: Medium]
Does preserving a non-zero pix_mp->plane_fmt[0].sizeimage for compressed formats leave sizeimage stale when the resolution is increased?
For codec formats (where v4l2_format_info() returns NULL), if pix_mp->plane_fmt[0].sizeimage was already initialized to a default resolution's size (or populated by VIDIOC_G_FMT) and wave6_update_pix_fmt() is then called for a larger resolution without zeroing sizeimage first, plane_fmt[0].sizeimage remains clamped to the old smaller resolution's buffer size.
[ ... ]
Quoted 25 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c new file +25
-0+static bool wave6_vpu_check_fb_available (struct vpu_instance *inst) +{ +struct vb2_v4l2_buffer *vb2_v4l2_buf; +struct v4l2_m2m_buffer *v4l2_m2m_buf; +struct vpu_buffer *vpu_buf; + +v4l2_m2m_for_each_dst_buf (inst->v4l2_fh.m2m_ctx, v4l2_m2m_buf) { + vb2_v4l2_buf = &v4l2_m2m_buf->vb; + vpu_buf =wave6_to_vpu_buf (vb2_v4l2_buf); + +if (!vpu_buf->used) +return true; + } + +return false; +} + +static int wave6_vpu_job_ready (void *priv) +{ +struct vpu_instance *inst = priv; + +dev_dbg (inst->dev->dev,"[%d]%s: state %d\n" , + inst->id, __func__, inst->state); + +if (inst->state ==VPU_INST_STATE_OPEN )
drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c line 310
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c new file +10
-0+return 1 ; +if (inst->state <VPU_INST_STATE_PIC_RUN ) +return 0 ; +if (inst->state ==VPU_INST_STATE_ERROR ) +return 0 ; +if (!wave6_vpu_check_fb_available (inst)) +return 0 ; + +return 1 ; +}
[Severity: High]
Isn't m2m_ctx->cap_q_ctx.rdy_queue traversed without holding cap_q_ctx.rdy_spinlock or core->dev_lock when wave6_vpu_check_fb_available() is called from wave6_vpu_finish_job()?
v4l2_m2m_for_each_dst_buf() expands to list_for_each_entry() over m2m_ctx->cap_q_ctx.rdy_queue without taking cap_q_ctx.rdy_spinlock. When wave6_vpu_finish_job() runs from wave6_vpu_core_irq_thread(), core->dev_lock is not held, and v4l2_m2m_job_finish() only takes m2m_dev->job_spinlock before calling wave6_vpu_job_ready(). Meanwhile, VIDIOC_QBUF on another CPU holds core->dev_lock and cap_q_ctx.rdy_spinlock while mutating cap_q_ctx.rdy_queue in v4l2_m2m_buf_queue():
CPU0 (irq thread) CPU1 (VIDIOC_QBUF)wave6_vpu_finish_job() v4l2_m2m_qbuf()
v4l2_m2m_job_finish() v4l2_m2m_buf_queue()
lock(&m2m_dev->job_spinlock) lock(&cap_q_ctx.rdy_spinlock)
wave6_vpu_job_ready() list_add_tail(..., &rdy_queue)
wave6_vpu_check_fb_available()
v4l2_m2m_for_each_dst_buf() <-- concurrent list traversal/mutation
Quoted 43 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c new file +43
-0+ +void wave6_vpu_device_run_timeout (struct work_struct *work) +{ +struct vpu_instance *inst =container_of (to_delayed_work (work), + struct vpu_instance, task_timer); +struct vb2_v4l2_buffer *src_buf =NULL ; +struct vb2_v4l2_buffer *dst_buf =NULL ; + +dev_err (inst->dev->dev,"[%d] sequence %d timeout\n" , inst->id, inst->sequence); + src_buf =v4l2_m2m_src_buf_remove (inst->v4l2_fh.m2m_ctx); +if (src_buf) { +v4l2_m2m_buf_done (src_buf,VB2_BUF_STATE_ERROR ); +if (inst->type ==VPU_INST_TYPE_DEC ) + inst->processed_buf_num++; + inst->error_buf_num++; + } + + dst_buf =v4l2_m2m_dst_buf_remove (inst->v4l2_fh.m2m_ctx); +if (dst_buf) +v4l2_m2m_buf_done (dst_buf,VB2_BUF_STATE_ERROR ); + +vb2_queue_error (v4l2_m2m_get_src_vq (inst->v4l2_fh.m2m_ctx)); +vb2_queue_error (v4l2_m2m_get_dst_vq (inst->v4l2_fh.m2m_ctx)); + +v4l2_m2m_job_finish (inst->dev->m2m_dev, inst->v4l2_fh.m2m_ctx); +} + +static void wave6_vpu_device_run (void *priv) +{ +struct vpu_instance *inst = priv; +int ret; + +dev_dbg (inst->dev->dev,"[%d]%s: state %d\n" , + inst->id, __func__, inst->state); + + ret = inst->ops->prepare_process (inst); +if (ret) { +v4l2_m2m_job_finish (inst->dev->m2m_dev, inst->v4l2_fh.m2m_ctx); +return ; + } + +schedule_delayed_work (&inst->task_timer,msecs_to_jiffies (W6_VPU_TIMEOUT )); +
drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c line 363
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c new file +10
-0+ ret = inst->ops->start_process (inst); +if (ret) +wave6_vpu_finish_job (inst); +} + +void wave6_vpu_finish_job (struct vpu_instance *inst) +{ +cancel_delayed_work (&inst->task_timer); +v4l2_m2m_job_finish (inst->dev->m2m_dev, inst->v4l2_fh.m2m_ctx); +}
[Severity: Medium]
Can wave6_vpu_device_run_timeout() race with normal job completion in finish_process() and wave6_vpu_finish_job()?
wave6_vpu_finish_job() uses non-synchronous cancel_delayed_work() and does not check its return value. If wave6_vpu_device_run_timeout() has already started running when finish_process() executes on the IRQ thread, nothing serializes them:
CPU0 (kworker) CPU1 (irq thread)wave6_vpu_device_run_timeout() wave6_vpu_core_irq_thread()
inst->ops->finish_process()
v4l2_m2m_src_buf_remove() ...
inst->error_buf_num++ wave6_vpu_finish_job()
cancel_delayed_work() -> false
v4l2_m2m_job_finish() v4l2_m2m_job_finish() <-- double v4l2_m2m_job_finish()
Both contexts concurrently remove buffers, update inst->processed_buf_num and inst->error_buf_num, and call v4l2_m2m_job_finish().
In addition, wave6_vpu_device_run_timeout() does not transition inst->state to VPU_INST_STATE_ERROR before calling v4l2_m2m_job_finish(). Because v4l2_m2m_job_finish() immediately checks wave6_vpu_job_ready() to schedule the next queued job, and neither __v4l2_m2m_try_queue() nor wave6_vpu_job_ready() checks vq->error, another queued job will be started on the timed-out instance.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
[email protected]
AI reviewRe: [PATCH v8 07/11] media: chips-media: wave6: Add Wave6 m2m encoder
In reply to Nas ChungThank you for your contribution! Sashiko AI review found 12 potential issues to consider: - [High] media: wave6: buffer leak and m2m job hang in wave6_vpu_enc_finish_encode() - [High] media: wave6…
Thank you for your contribution! Sashiko AI review found 12 potential issues to consider:
- [High] media: wave6: buffer leak and m2m job hang in wave6_vpu_enc_finish_encode()
- [High] media: wave6: unvalidated CAPTURE size and stale sizeimage in dst_fmt
- [High] media: wave6: asymmetric instance lifecycle in start and stop_streaming()
- [High] media: wave6: missing busy check and zero crop return in s_selection()
- [High] media: wave6: unlocked m2m rdy_queue traversal in encoder buffer lookup
- [High] media: wave6: race between wave6_vpu_enc_finish_encode() and timeout work
- [Medium] media: wave6: zero CSC coefficients in wave6_set_csc() for XV601 and XV709
- [Medium] media: wave6: HEVC refresh controls overwritten in wave6_set_enc_hevc_param()
- [Medium] media: wave6: skipped CAPTURE validation in wave6_vpu_enc_buf_prepare()
- [Medium] media: wave6: zero-padded format table match in wave6_find_vpu_fmt()
- [Medium] media: wave6: wave6_find_vpu_fmt() matches zeroed format table entries
- [Low] media: wave6: unupdated timeperframe in wave6_vpu_enc_s_parm() --
commit d914baad0e743b67d2e3cd7e1bd636ba7a5de85f
Author: Nas Chung <[email protected]>
media: chips-media: wave6: Add Wave6 m2m encoder
This commit adds the V4L2 m2m stateful encoder driver for the Chips&Media Wave6 video codec IP, supporting H.264 and HEVC encoding.
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c new file +0
-0@@ -0,0 +1,2732 @@
[ ... ]
Quoted 19 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c new file +19
-0+static const struct vpu_format wave6_vpu_enc_fmt_list[2 ][23 ] = { + [VPU_FMT_TYPE_CODEC ] = { + { + .v4l2_pix_fmt =V4L2_PIX_FMT_HEVC , + .max_width =W6_MAX_ENC_PIC_WIDTH , + .min_width =W6_MIN_ENC_PIC_WIDTH , + .max_height =W6_MAX_ENC_PIC_HEIGHT , + .min_height =W6_MIN_ENC_PIC_HEIGHT , + .num_planes =1 , + }, + { + .v4l2_pix_fmt =V4L2_PIX_FMT_H264 , + .max_width =W6_MAX_ENC_PIC_WIDTH , + .min_width =W6_MIN_ENC_PIC_WIDTH , + .max_height =W6_MAX_ENC_PIC_HEIGHT , + .min_height =W6_MIN_ENC_PIC_HEIGHT , + .num_planes =1 , + }, + },
[ ... ]
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c line 329
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c new file +10
-0+static const struct vpu_format *wave6_find_vpu_fmt (unsigned int v4l2_pix_fmt, +enum vpu_fmt_type type) +{ +for (int i =0 ; i <ARRAY_SIZE (wave6_vpu_enc_fmt_list[type]); i++) { +if (wave6_vpu_enc_fmt_list[type][i].v4l2_pix_fmt == v4l2_pix_fmt) +return &wave6_vpu_enc_fmt_list[type][i]; + } + +return NULL ; +}
[Severity: Medium]
Can wave6_find_vpu_fmt() match the unpopulated trailing entries in wave6_vpu_enc_fmt_list[VPU_FMT_TYPE_CODEC] when v4l2_pix_fmt is 0?
Because wave6_vpu_enc_fmt_list is sized to 23 elements for both sub-arrays while VPU_FMT_TYPE_CODEC only populates 2 entries, the remaining 21 entries are zero-initialized. Unlike wave6_find_vpu_fmt_by_idx(), wave6_find_vpu_fmt() does not reject a zero pixelformat, so passing pixelformat 0 to VIDIOC_TRY_FMT on the capture queue matches a zeroed entry and sets pix_mp->pixelformat and pix_mp->num_planes to 0 instead of falling back to inst->dst_fmt.
[ ... ]
Quoted 29 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c new file +29
-0+static struct vb2_v4l2_buffer *wave6_get_valid_src_buf (struct vpu_instance *inst) +{ +struct vb2_v4l2_buffer *vb2_v4l2_buf; +struct v4l2_m2m_buffer *v4l2_m2m_buf; +struct vpu_buffer *vpu_buf =NULL ; + +v4l2_m2m_for_each_src_buf (inst->v4l2_fh.m2m_ctx, v4l2_m2m_buf) { + vb2_v4l2_buf = &v4l2_m2m_buf->vb; + vpu_buf =wave6_to_vpu_buf (vb2_v4l2_buf); + +if (!vpu_buf->consumed) { +dev_dbg (inst->dev->dev,"no consumed src idx : %d\n" , + vb2_v4l2_buf->vb2_buf.index); +return vb2_v4l2_buf; + } + } + +return NULL ; +} + +static struct vb2_v4l2_buffer *wave6_get_valid_dst_buf (struct vpu_instance *inst) +{ +struct vb2_v4l2_buffer *vb2_v4l2_buf; +struct v4l2_m2m_buffer *v4l2_m2m_buf; +struct vpu_buffer *vpu_buf; + +v4l2_m2m_for_each_dst_buf (inst->v4l2_fh.m2m_ctx, v4l2_m2m_buf) { + vb2_v4l2_buf = &v4l2_m2m_buf->vb; + vpu_buf =wave6_to_vpu_buf (vb2_v4l2_buf);
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c line 430
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c new file +10
-0+ +if (!vpu_buf->consumed) { +dev_dbg (inst->dev->dev,"no consumed dst idx : %d\n" , + vb2_v4l2_buf->vb2_buf.index); +return vb2_v4l2_buf; + } + } + +return NULL ; +}
[Severity: High]
Does iterating the M2M ready queues in wave6_get_valid_src_buf(), wave6_get_valid_dst_buf(), and wave6_get_dst_buf_by_addr() race with v4l2_m2m_buf_queue()?
v4l2_m2m_for_each_src_buf() and v4l2_m2m_for_each_dst_buf() expand to plain list_for_each_entry() over out_q_ctx.rdy_queue and cap_q_ctx.rdy_queue without taking q_ctx->rdy_spinlock:
CPU0 (v4l2_m2m_device_run_work) CPU1 (VIDIOC_QBUF)wave6_vpu_device_run() wave6_vpu_enc_buf_queue()
wave6_vpu_enc_start_encode() v4l2_m2m_buf_queue()
wave6_get_valid_src_buf() spin_lock(&q_ctx->rdy_spinlock)
v4l2_m2m_for_each_src_buf() list_add_tail(&b->list,
&q_ctx->rdy_queue)
/* unlocked list traversal <-- data race */When wave6_vpu_device_run() runs from the M2M job workqueue, neither inst->dev->dev_lock nor rdy_spinlock is held. Likewise, when wave6_handle_encoded_frame() calls wave6_get_dst_buf_by_addr() from the IRQ thread, it holds only inst->lock while VIDIOC_QBUF mutates rdy_queue under inst->dev->dev_lock and rdy_spinlock.
Quoted 18 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c new file +18
-0+ +static void wave6_set_csc (struct vpu_instance *inst,struct enc_param *pic_param) +{ +const struct vpu_format *vpu_fmt; +bool is_10bit; + + vpu_fmt =wave6_find_vpu_fmt (inst->src_fmt.pixelformat,VPU_FMT_TYPE_RAW ); +if (!vpu_fmt) +return ; + +if (!(vpu_fmt->flags &VPU_FMT_FLAG_RGB )) +return ; + + is_10bit = !!(vpu_fmt->flags &VPU_FMT_FLAG_10BIT ); + pic_param->csc.fmt_order = vpu_fmt->csc_fmt_order; + +if (inst->ycbcr_enc ==V4L2_YCBCR_ENC_DEFAULT || + inst->ycbcr_enc ==V4L2_YCBCR_ENC_601 ) {
[ ... ]
Quoted 32 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c new file +32
-0+ }else if (inst->ycbcr_enc ==V4L2_YCBCR_ENC_XV601 ) { +if (inst->quantization ==V4L2_QUANTIZATION_LIM_RANGE ) { +/* +* Y 0.2558(R) 0.5021(G) 0.0975(B) +* Cb -0.1476(R) -0.2899(G) 0.4375(B) +* Cr 0.4375(R) -0.3664(G) -0.0711(B) +*/ + pic_param->csc.coef_ry =0x083 ; + pic_param->csc.coef_gy =0x101 ; + pic_param->csc.coef_by =0x032 ; + pic_param->csc.coef_rcb =0xffffffb4 ; + pic_param->csc.coef_gcb =0xffffff6c ; + pic_param->csc.coef_bcb =0x0e0 ; + pic_param->csc.coef_rcr =0x0e0 ; + pic_param->csc.coef_gcr =0xffffff44 ; + pic_param->csc.coef_bcr =0xffffffdc ; + pic_param->csc.offset_y = (is_10bit) ?0x40 :0x10 ; + pic_param->csc.offset_cb =0x0 ; + pic_param->csc.offset_cr =0x0 ; + } + }else if (inst->ycbcr_enc ==V4L2_YCBCR_ENC_XV709 ) { +if (inst->quantization ==V4L2_QUANTIZATION_LIM_RANGE ) { +/* +* Y 0.1819(R) 0.6118(G) 0.0618(B) +* Cb -0.1003(R) -0.3372(G) 0.4375(B) +* Cr 0.4375(R) -0.3974(G) -0.0401(B) +*/ + pic_param->csc.coef_ry =0x05d ; + pic_param->csc.coef_gy =0x139 ; + pic_param->csc.coef_by =0x020 ; + pic_param->csc.coef_rcb =0xffffffcd ; + pic_param->csc.coef_gcb =0xffffff53 ;
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c line 626
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c new file +10
-0+ pic_param->csc.coef_bcb =0x0e0 ; + pic_param->csc.coef_rcr =0x0e0 ; + pic_param->csc.coef_gcr =0xffffff35 ; + pic_param->csc.coef_bcr =0xffffffeb ; + pic_param->csc.offset_y = (is_10bit) ?0x40 :0x10 ; + pic_param->csc.offset_cb =0x0 ; + pic_param->csc.offset_cr =0x0 ; + } + } +}
[Severity: Medium]
Does wave6_set_csc() leave pic_param->csc zeroed when encoding RGB formats with V4L2_YCBCR_ENC_XV601 or V4L2_YCBCR_ENC_XV709 and default quantization?
inst->quantization defaults to V4L2_QUANTIZATION_DEFAULT, and wave6_vpu_enc_try_fmt_out() only converts V4L2_QUANTIZATION_FULL_RANGE to V4L2_QUANTIZATION_LIM_RANGE for XV601 and XV709. Because these two branches require inst->quantization == V4L2_QUANTIZATION_LIM_RANGE, all CSC coefficients remain 0 when quantization is V4L2_QUANTIZATION_DEFAULT, producing all-zero YCbCr output. In addition, V4L2_YCBCR_ENC_DEFAULT is always treated as BT.601 here instead of being resolved via V4L2_MAP_YCBCR_ENC_DEFAULT(inst->colorspace) as in to_matrix_coeffs().
[ ... ]
Quoted 2 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c new file +2
-0+static int wave6_vpu_enc_start_encode (struct vpu_instance *inst) +{
[ ... ]
Quoted 16 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c new file +16
-0+ src_buf =wave6_get_valid_src_buf (inst); + dst_buf =wave6_get_valid_dst_buf (inst); + +if (!src_buf || !dst_buf) { +dev_dbg (inst->dev->dev,"no valid src or dst buf\n" ); +return 0 ; + } + + dst_vbuf =wave6_to_vpu_buf (dst_buf); + dst_vbuf->consumed = true; + dst_vbuf->used = true; + pic_param.pic_stream_buffer_addr =wave6_get_dma_addr (dst_buf,0 ); + pic_param.pic_stream_buffer_size =vb2_plane_size (&dst_buf->vb2_buf,0 ); + + src_vbuf =wave6_to_vpu_buf (src_buf); + src_vbuf->consumed = true;
[ ... ]
Quoted 32 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c new file +32
-0+static void wave6_handle_encoded_frame (struct vpu_instance *inst, +struct enc_output_info *info) +{ +struct vb2_v4l2_buffer *src_buf; +struct vb2_v4l2_buffer *dst_buf; +struct vpu_buffer *vpu_buf; +struct vpu_buffer *dst_vpu_buf; +enum vb2_buffer_state state; + +guard (mutex)(&inst->lock); + + state = info->encoding_success ?VB2_BUF_STATE_DONE :VB2_BUF_STATE_ERROR ; + + dst_buf =wave6_get_dst_buf_by_addr (inst, info->bitstream_buffer); +if (!dst_buf) { +dev_err (inst->dev->dev,"[%d] encoder can't find dst buffer\n" , inst->id); +return ; + } + + src_buf =v4l2_m2m_src_buf_remove_by_idx (inst->v4l2_fh.m2m_ctx, + info->enc_src_idx); +if (!src_buf) { +dev_err (inst->dev->dev,"[%d] encoder can't find src buffer\n" , inst->id); +return ; + } + + vpu_buf =wave6_to_vpu_buf (src_buf); +if (!vpu_buf->consumed) { +dev_err (inst->dev->dev,"[%d] src buffer is not consumed\n" , inst->id); +v4l2_m2m_buf_done (src_buf,VB2_BUF_STATE_ERROR ); +return ; + }
[ ... ]
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c line 910
Quoted 9 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c new file +9
-0+v4l2_m2m_dst_buf_remove_by_buf (inst->v4l2_fh.m2m_ctx, dst_buf); +if (state ==VB2_BUF_STATE_ERROR ) { +dev_dbg (inst->dev->dev,"[%d] error frame %d\n" , inst->id, inst->sequence); + inst->error_recovery = true; + inst->error_buf_num++; + } +v4l2_m2m_buf_done (dst_buf, state); + inst->processed_buf_num++; +}
[Severity: High]
Can wave6_handle_encoded_frame() race with wave6_vpu_device_run_timeout() and corrupt the destination queue list or complete dst_buf twice?
wave6_vpu_enc_finish_encode() runs wave6_handle_encoded_frame() before cancelling inst->task_timer in wave6_vpu_finish_job(), and wave6_vpu_finish_job() only uses asynchronous cancel_delayed_work() while wave6_vpu_device_run_timeout() does not take inst->lock:
CPU0 (wave6_vpu_core_irq_thread) CPU1 (wave6_vpu_device_run_timeout) wave6_vpu_enc_finish_encode()
wave6_handle_encoded_frame()
mutex_lock(&inst->lock)
dst_buf = wave6_get_dst_buf_by_addr()
dst_buf = v4l2_m2m_dst_buf_remove()
v4l2_m2m_buf_done(dst_buf,
VB2_BUF_STATE_ERROR)
v4l2_m2m_dst_buf_remove_by_buf(..., dst_buf)
list_del(&dst_buf->list) <-- double list_del
v4l2_m2m_buf_done(dst_buf, state)
<-- double buffer completionQuoted 19 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c new file +19
-0+ +static void wave6_vpu_enc_finish_encode (struct vpu_instance *inst,bool error) +{ +int ret; +struct enc_output_info info; + +if (error) { +vb2_queue_error (v4l2_m2m_get_src_vq (inst->v4l2_fh.m2m_ctx)); +vb2_queue_error (v4l2_m2m_get_dst_vq (inst->v4l2_fh.m2m_ctx)); + +wave6_vpu_set_instance_state (inst,VPU_INST_STATE_ERROR ); + + goto finish_encode; + } + +memset (&info,0 ,sizeof (info)); + + ret =wave6_vpu_enc_get_output_info (inst, &info); +if (ret) {
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c line 939
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c new file +10
-0+dev_dbg (inst->dev->dev,"vpu_enc_get_output_info fail %d\n" , ret); + goto finish_encode; + } + +if (info.enc_src_idx >=0 && info.recon_frame_index >=0 ) +wave6_handle_encoded_frame (inst, &info); + +finish_encode: +wave6_vpu_finish_job (inst); +}
[Severity: High]
Are the in-flight source and destination buffers stranded on the M2M ready queues when a frame is skipped or an encode error occurs?
wave6_vpu_enc_start_encode() marks src_vbuf->consumed = true and dst_vbuf->consumed = true while leaving both buffers on m2m_ctx until wave6_handle_encoded_frame() removes and completes them. When a frame is skipped due to V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_BUF_LIMIT (so info.recon_frame_index < 0), or when error is true or wave6_vpu_enc_get_output_info() fails, wave6_handle_encoded_frame() is not called; similarly, its early returns if src_buf is missing or unconsumed leave dst_buf unremoved. Those buffers remain on the ready queues with consumed = true and are skipped forever by wave6_get_valid_src_buf() and wave6_get_valid_dst_buf().
Furthermore, when wave6_vpu_enc_start_encode() subsequently finds no unconsumed buffer (!src_buf || !dst_buf), it returns 0 without starting a hardware command, so wave6_vpu_device_run() leaves the M2M job running with inst->task_timer armed until the 6-second watchdog timeout fires and kills both queues.
[ ... ]
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c line 962
Quoted 13 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c new file +13
-0+static int wave6_vpu_enc_enum_framesizes (struct file *f,void *fh,struct v4l2_frmsizeenum *fsize) +{ +const struct vpu_format *vpu_fmt; + +if (fsize->index) +return -EINVAL ; + + vpu_fmt =wave6_find_vpu_fmt (fsize->pixel_format,VPU_FMT_TYPE_CODEC ); +if (!vpu_fmt) { + vpu_fmt =wave6_find_vpu_fmt (fsize->pixel_format,VPU_FMT_TYPE_RAW ); +if (!vpu_fmt) +return -EINVAL ; + }
[Severity: Medium]
Does wave6_vpu_enc_enum_framesizes() succeed with 0x0 stepwise bounds when fsize->pixel_format is 0?
Because wave6_find_vpu_fmt(0, VPU_FMT_TYPE_CODEC) returns a pointer to the first zero-padded entry in wave6_vpu_enc_fmt_list[VPU_FMT_TYPE_CODEC], VIDIOC_ENUM_FRAMESIZES returns 0 with min_width, max_width, min_height, and max_height all set to 0 instead of returning -EINVAL.
[ ... ]
Quoted 18 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c new file +18
-0+static int wave6_vpu_enc_try_fmt_cap (struct file *file,void *fh,struct v4l2_format *f) +{ +struct vpu_instance *inst =wave6_file_to_vpu_inst (file); +struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp; +const struct vpu_format *vpu_fmt; +int width, height; + +dev_dbg (inst->dev->dev,"%s: 4cc %d w %d h %d plane %d colorspace %d\n" , + __func__, pix_mp->pixelformat, pix_mp->width, pix_mp->height, + pix_mp->num_planes, pix_mp->colorspace); + +if (!V4L2_TYPE_IS_CAPTURE (f->type)) +return -EINVAL ; + + vpu_fmt =wave6_find_vpu_fmt (pix_mp->pixelformat,VPU_FMT_TYPE_CODEC ); +if (!vpu_fmt) { + width = inst->dst_fmt.width; + height = inst->dst_fmt.height;
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c line 1019
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c new file +10
-0+ pix_mp->pixelformat = inst->dst_fmt.pixelformat; + pix_mp->num_planes = inst->dst_fmt.num_planes; + }else { + width = pix_mp->width; + height = pix_mp->height; + pix_mp->pixelformat = vpu_fmt->v4l2_pix_fmt; + pix_mp->num_planes = vpu_fmt->num_planes; + } + +wave6_update_pix_fmt (pix_mp, width, height);
[Severity: High]
Can wave6_vpu_enc_try_fmt_cap() set plane_fmt[0].sizeimage to 0, and do wave6_vpu_enc_s_fmt_out() and wave6_vpu_enc_s_selection() leave inst->dst_fmt.plane_fmt[0].sizeimage stuck at the initial 416x240 size?
In wave6_update_pix_fmt(), sizeimage for coded formats is only recalculated when plane_fmt[0].sizeimage is 0:
drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c:wave6_update_pix_fmt() {
...
if (!fmt_info) {
pix_mp->plane_fmt[0].bytesperline = 0;
if (!pix_mp->plane_fmt[0].sizeimage)
pix_mp->plane_fmt[0].sizeimage = width * height;
return;
}
...
}When pix_mp->pixelformat is valid in wave6_vpu_enc_try_fmt_cap(), width and height are taken directly from userspace without clamping or using inst->dst_fmt. Passing width = 0 or height = 0 with sizeimage = 0 sets inst->dst_fmt.plane_fmt[0].sizeimage to 0 in wave6_vpu_enc_s_fmt_cap(), which triggers WARN_ON(!plane_sizes[i]) in vb2_core_reqbufs().
Conversely, because wave6_vpu_open_enc() initializes inst->dst_fmt to 416x240 (sizeimage = 99840) and neither wave6_vpu_enc_s_fmt_out() nor wave6_vpu_enc_s_selection() clears inst->dst_fmt.plane_fmt[0].sizeimage before calling wave6_update_pix_fmt(&inst->dst_fmt, ...), raising the output resolution leaves capture bitstream buffers undersized at 99840 bytes.
[ ... ]
Quoted 37 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c new file +37
-0+static int wave6_vpu_enc_s_selection (struct file *file,void *fh,struct v4l2_selection *s) +{ +struct vpu_instance *inst =wave6_file_to_vpu_inst (file); +u32 max_crop_w, max_crop_h; + +if (!V4L2_TYPE_IS_OUTPUT (s->type)) +return -EINVAL ; + +if (s->target !=V4L2_SEL_TGT_CROP ) +return -EINVAL ; + +if (!(s->flags & (V4L2_SEL_FLAG_GE |V4L2_SEL_FLAG_LE ))) + s->flags |=V4L2_SEL_FLAG_LE ; + +if (s->flags &V4L2_SEL_FLAG_GE ) { + s->r.left =round_up (s->r.left,W6_ENC_CROP_STEP ); + s->r.top =round_up (s->r.top,W6_ENC_CROP_STEP ); + s->r.width =round_up (s->r.width,W6_ENC_CROP_STEP ); + s->r.height =round_up (s->r.height,W6_ENC_CROP_STEP ); + } +if (s->flags &V4L2_SEL_FLAG_LE ) { + s->r.left =round_down (s->r.left,W6_ENC_CROP_STEP ); + s->r.top =round_down (s->r.top,W6_ENC_CROP_STEP ); + s->r.width =round_down (s->r.width,W6_ENC_CROP_STEP ); + s->r.height =round_down (s->r.height,W6_ENC_CROP_STEP ); + } + + s->r.left =clamp_t (s32, s->r.left,0 , + inst->src_fmt.width -W6_MIN_ENC_PIC_WIDTH ); + s->r.top =clamp_t (s32, s->r.top,0 , + inst->src_fmt.height -W6_MIN_ENC_PIC_HEIGHT ); + + max_crop_w = inst->src_fmt.width - s->r.left; + max_crop_h = inst->src_fmt.height - s->r.top; + +if (!s->r.width || !s->r.height) +return 0 ;
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c line 1300
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c new file +10
-0+if (max_crop_w <W6_MIN_ENC_PIC_WIDTH ) +return 0 ; +if (max_crop_h <W6_MIN_ENC_PIC_HEIGHT ) +return 0 ; + + s->r.width =clamp (s->r.width,W6_MIN_ENC_PIC_WIDTH , max_crop_w); + s->r.height =clamp (s->r.height,W6_MIN_ENC_PIC_HEIGHT , max_crop_h); + +wave6_update_pix_fmt (&inst->dst_fmt, s->r.width, s->r.height); +wave6_update_crop_info (inst, s->r.left, s->r.top, s->r.width, s->r.height);
[Severity: High]
Can wave6_vpu_enc_s_selection() modify inst->codec_rect while streaming, and does it return a 0-sized rectangle when width or height is 0?
wave6_vpu_enc_s_selection() does not check vb2_is_busy() on the queues or whether inst->state != VPU_INST_STATE_NONE. Because hardware picture dimensions are configured once at streamon while wave6_update_frame_buf_addr() adds inst->codec_rect.top and inst->codec_rect.left to the source frame DMA addresses on every frame, changing the crop offset mid-stream shifts the DMA base addresses while the VPU still reads the original frame dimensions past the end of the source buffer.
In addition, returning 0 early when !s->r.width || !s->r.height bypasses the clamp() calls right below and reports an invalid 0-sized crop rectangle back to userspace.
[ ... ]
Quoted 11 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c new file +11
-0+static int wave6_vpu_enc_s_parm (struct file *file,void *fh,struct v4l2_streamparm *a) +{ +struct vpu_instance *inst =wave6_file_to_vpu_inst (file); +struct enc_controls *ctrls = &inst->enc_ctrls; + +dev_dbg (inst->dev->dev,"%s: type %d\n" , __func__, a->type); + +if (!V4L2_TYPE_IS_OUTPUT (a->type)) +return -EINVAL ; + + a->parm.output.capability =V4L2_CAP_TIMEPERFRAME ;
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c line 1377
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c new file +10
-0+if (a->parm.output.timeperframe.denominator && + a->parm.output.timeperframe.numerator) { + ctrls->frame_rate =DIV_ROUND_CLOSEST (a->parm.output.timeperframe.denominator, + a->parm.output.timeperframe.numerator); + ctrls->frame_rate =clamp (ctrls->frame_rate, +MIN_FRAME_RATE ,MAX_FRAME_RATE ); + }else { + a->parm.output.timeperframe.numerator =1 ; + a->parm.output.timeperframe.denominator = ctrls->frame_rate; + }
[Severity: Low]
Should wave6_vpu_enc_s_parm() update a->parm.output.timeperframe after computing and clamping ctrls->frame_rate?
When numerator and denominator are non-zero, the driver rounds and clamps ctrls->frame_rate but leaves the caller's unadjusted values in a->parm.output.timeperframe instead of returning 1 / ctrls->frame_rate.
[ ... ]
Quoted 3 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c new file +3
-0+static void wave6_set_enc_hevc_param (struct enc_codec_param *output, +struct hevc_enc_controls *ctrls) +{
[ ... ]
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c new file +6
-0+switch (ctrls->refresh_type) { +case V4L2_MPEG_VIDEO_HEVC_REFRESH_NONE : + output->decoding_refresh_type =DEC_REFRESH_TYPE_NON_IRAP ; +break ; +case V4L2_MPEG_VIDEO_HEVC_REFRESH_IDR : + output->decoding_refresh_type =DEC_REFRESH_TYPE_IDR ;
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c line 1894
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c new file +10
-0+break ; +default : +break ; + } + output->intra_period = ctrls->refresh_period; +if (output->idr_period) { + output->decoding_refresh_type =DEC_REFRESH_TYPE_IDR ; + output->intra_period = output->idr_period; + output->idr_period =0 ; + }
[Severity: Medium]
Does checking if (output->idr_period) here overwrite the HEVC refresh type and period whenever V4L2_CID_MPEG_VIDEO_GOP_SIZE is non-zero?
wave6_set_enc_open_param() sets output->idr_period = ctrls->gop_size (which defaults to 30) before calling wave6_set_enc_hevc_param(). Checking if (output->idr_period) instead of if (!output->intra_period) clobbers ctrls->refresh_period and ctrls->refresh_type unless GOP size is explicitly set to 0.
[ ... ]
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c line 2225
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c new file +6
-0+static int wave6_vpu_enc_buf_prepare (struct vb2_buffer *vb) +{ +struct vpu_instance *inst =vb2_get_drv_priv (vb->vb2_queue); + +if (V4L2_TYPE_IS_CAPTURE (vb->type)) +return 0 ;
[Severity: Medium]
Should wave6_vpu_enc_buf_prepare() validate the capture plane size against inst->dst_fmt.plane_fmt[0].sizeimage and reset the plane payload to 0?
Returning early for capture buffers allows undersized imported DMABUFs to be queued without error, and a re-queued capture buffer that is later completed as an empty EOS buffer via v4l2_m2m_last_buffer_done() when V4L2_ENC_CMD_STOP or output streamoff runs retains its stale bytesused payload from the previous frame.
[ ... ]
Quoted 53 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c new file +53
-0+static int wave6_vpu_enc_start_streaming (struct vb2_queue *q,unsigned int count) +{ +struct vpu_instance *inst =vb2_get_drv_priv (q); +int ret =0 ; + +v4l2_m2m_update_start_streaming_state (inst->v4l2_fh.m2m_ctx, q); + +if (inst->state ==VPU_INST_STATE_NONE ) { + ret =wave6_vpu_enc_create_instance (inst); +if (ret) + goto exit; + } + +if (inst->state ==VPU_INST_STATE_OPEN ) { + ret =wave6_vpu_enc_initialize_instance (inst); +if (ret) { +wave6_vpu_enc_destroy_instance (inst); + goto exit; + } + } + +if (inst->state ==VPU_INST_STATE_INIT_SEQ ) { + ret =wave6_vpu_enc_prepare_fb (inst); +if (ret) { +wave6_vpu_enc_destroy_instance (inst); + goto exit; + } + } + +exit: +if (ret) +wave6_vpu_return_buffers (inst, q->type,VB2_BUF_STATE_QUEUED ); + +return ret; +} + +static void wave6_vpu_enc_stop_streaming (struct vb2_queue *q) +{ +struct vpu_instance *inst =vb2_get_drv_priv (q); + +dev_dbg (inst->dev->dev,"[%d] %s, input %d, encode %d\n" , + inst->id,V4L2_TYPE_IS_OUTPUT (q->type) ?"output" :"capture" , + inst->queued_src_buf_num, inst->sequence); + +if (inst->state ==VPU_INST_STATE_NONE ) + goto exit; + +if (V4L2_TYPE_IS_OUTPUT (q->type)) { + inst->queued_src_buf_num =0 ; + inst->processed_buf_num =0 ; + inst->error_buf_num =0 ; + inst->sequence =0 ; + }else {
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c line 2364
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c new file +10
-0+ inst->queued_dst_buf_num =0 ; + } + +if (V4L2_TYPE_IS_CAPTURE (q->type) && inst->state !=VPU_INST_STATE_NONE ) +wave6_vpu_enc_destroy_instance (inst); + +exit: +wave6_vpu_return_buffers (inst, q->type,VB2_BUF_STATE_ERROR ); +v4l2_m2m_update_stop_streaming_state (inst->v4l2_fh.m2m_ctx, q); +}
[Severity: High]
Can starting streaming on one queue before configuring the other leave the hardware instance and FBC reference buffers out of sync with the configured formats?
wave6_vpu_enc_start_streaming() creates the hardware instance and allocates FBC buffers sized to inst->codec_rect on whichever queue starts streaming first, while wave6_vpu_enc_s_fmt_out() and wave6_vpu_enc_s_fmt_cap() only check vb2_is_busy() on their own queue. If userspace starts capture streaming first (or stops only the output queue, which does not destroy the instance) and then changes the output resolution or capture codec format, the second STREAMON sees inst->state == VPU_INST_STATE_PIC_RUN and skips reinitializing the hardware instance and FBC buffers.
In addition, if capture stops streaming before output, wave6_vpu_enc_stop_streaming() destroys the instance and sets inst->state to VPU_INST_STATE_NONE, so stopping the output queue afterwards takes the early goto exit and skips resetting queued_src_buf_num, processed_buf_num, error_buf_num, and sequence.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
[email protected]
AI reviewRe: [PATCH v8 11/11] arm64: dts: freescale: imx95: Add video codec node
In reply to Nas ChungThank you for your contribution! Sashiko AI review found 8 potential issues to consider: - [High] media: chips-media: wave6: UAF of m2m_ctx during release and job finish - [High] media: chips-media…
Thank you for your contribution! Sashiko AI review found 8 potential issues to consider:
- [High] media: chips-media: wave6: UAF of m2m_ctx during release and job finish
- [High] media: chips-media: wave6: unlocked work_buffers_avail access in hard IRQ
- [High] media: chips-media: wave6: asymmetric encoder instance streaming lifecycle
- [High] media: chips-media: wave6: infinite job loop on wave6_vpu_dec_seek_header()
- [Medium] media: chips-media: wave6: PM domains detached before devm cooling teardown
- [Medium] media: chips-media: wave6: stale fbc_buf_registered after destroy_instance
- [Medium] media: chips-media: wave6: wave6_vpu_get() skipped if core already active
- [Low] media: chips-media: wave6: debugfs dir leaked in wave6_vpu_core_probe()
--
commit aa070a21021b515d839ff256d8cb72ca0721cb00
Author: Nas Chung <[email protected]>
arm64: dts: freescale: imx95: Add video codec node
This commit adds the Chips&Media Wave633 video codec node to the i.MX95 SoC device tree and enables it with a reserved boot memory region on the i.MX95 19x19 EVK board.
Quoted 9 lines
arch/arm64/boot/dts/freescale/imx95.dtsi +1
-0@@ -2139,6 +2139,43 @@ vpu_blk_ctrl: clock-controller@4c410000 { assigned-clock-rates = <133333333>, <667000000>, <500000000>; }; + vpu: video-codec@4c4c0000 {
arch/arm64/boot/dts/freescale/imx95.dtsi vpu_blk_ctrl: clock-controller@4c410000 { · line 2152
Quoted 10 lines
arch/arm64/boot/dts/freescale/imx95.dtsi +10
-0+ compatible = "nxp,imx95-vpu"; + reg = <0x0 0x4c4c0000 0x0 0x10000>; + clocks = <&scmi_clk IMX95_CLK_VPU>, + <&vpu_blk_ctrl IMX95_CLK_VPUBLK_WAVE>; + clock-names = "core", "vpublk"; + power-domains = <&scmi_devpd IMX95_PD_VPU>, + <&scmi_perf IMX95_PERF_VPU>; + power-domain-names = "vpu", "perf"; + sram = <&sram1>; + #cooling-cells = <2>;
[Severity: Medium]
With both "vpu" and "perf" power domains and cooling cells enabled here, can wave6_vpu_remove() (or probe error unwind in wave6_vpu_probe()) cause a use-after-free of vpu->thermal.dev in the wave6 driver introduced earlier in this series (commit "media: chips-media: wave6: Add Wave6 control driver" and commit "media: chips-media: wave6: Add Wave6 thermal cooling device")?
wave6_vpu_attach_pm_domains() attaches the domains with the non-managed dev_pm_domain_attach_list(), and wave6_vpu_get_performance_dev() stores vpu->pd_list->pd_devs[index] in vpu->thermal.dev. wave6_vpu_cooling_init() then registers the cooling device via devm_thermal_of_child_cooling_device_register() on vpu->dev:
drivers/media/platform/chips-media/wave6/wave6-vpu.c:wave6_vpu_remove() {
wait_for_completion(&vpu->fw_done);
wave6_vpu_release(vpu);
pm_runtime_disable(vpu->dev);
wave6_vpu_detach_pm_domains(vpu);
}Because wave6_vpu_detach_pm_domains() calls dev_pm_domain_detach_list() before the driver core runs devres_release_all() to unregister the devm cooling device, vpu->thermal.dev is unregistered and freed while wave6_vpu_cooling_set_cur_state() can still call dev_pm_genpd_set_performance_state(thermal->dev, level).
Quoted 14 lines
arch/arm64/boot/dts/freescale/imx95.dtsi +14
-0+ #address-cells = <2>; + #size-cells = <2>; + ranges; + status = "disabled"; + + interface@4c480000 { + reg = <0x0 0x4c480000 0x0 0x10000>; + interrupts = <GIC_SPI 299 IRQ_TYPE_LEVEL_HIGH>; + }; + + interface@4c490000 { + reg = <0x0 0x4c490000 0x0 0x10000>; + interrupts = <GIC_SPI 300 IRQ_TYPE_LEVEL_HIGH>; + };
arch/arm64/boot/dts/freescale/imx95.dtsi vpu_blk_ctrl: clock-controller@4c410000 { · line 2176
Quoted 10 lines
arch/arm64/boot/dts/freescale/imx95.dtsi +10
-0+ + interface@4c4a0000 { + reg = <0x0 0x4c4a0000 0x0 0x10000>; + interrupts = <GIC_SPI 301 IRQ_TYPE_LEVEL_HIGH>; + }; + + interface@4c4b0000 { + reg = <0x0 0x4c4b0000 0x0 0x10000>; + interrupts = <GIC_SPI 302 IRQ_TYPE_LEVEL_HIGH>; + };
[Severity: High]
Since the four core interface nodes have separate GIC SPI interrupts, can concurrent W6_INT_BIT_REQ_WORK_BUF interrupts race on vpu->work_buffers_avail in wave6_vpu_require_work_buffer() (from commit "media: chips-media: wave6: Add Wave6 control driver")?
drivers/media/platform/chips-media/wave6/wave6-vpu.c:wave6_vpu_require_work_buffer() {
...
if (WARN_ON(vpu->work_buffers_avail == 0))
goto exit;
vpu->work_buffers_avail--;
vb = &vpu->work_buffers[vpu->work_buffers_avail];
...
}Each core's hard IRQ handler wave6_vpu_core_irq() calls wave6_vpu_require_work_buffer(core->vpu, core->reg_base) on the shared parent vpu without a lock:
CPU0 (<hardirq> SPI 299) CPU1 (<hardirq> SPI 300)wave6_vpu_core_irq() wave6_vpu_core_irq()
wave6_vpu_require_work_buffer() wave6_vpu_require_work_buffer()
work_buffers_avail == 1 work_buffers_avail == 1
work_buffers_avail-- (0) work_buffers_avail-- (0xffffffff)
vpu->work_buffers[0xffffffff]
<-- out-of-bounds readEven when work_buffers_avail > 1, two cores racing on the decrement can receive the same DMA work buffer.
[Severity: High]
In the core, decoder and encoder drivers instantiated for these interface nodes (commit "media: chips-media: wave6: Add Wave6 core driver", commit "media: chips-media: wave6: Add Wave6 m2m decoder" and commit "media:
chips-media: wave6: Add Wave6 m2m encoder"), can wave6_vpu_dec_release(), wave6_vpu_enc_release() and wave6_vpu_finish_job() cause a use-after-free of inst->v4l2_fh.m2m_ctx?
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c:wave6_vpu_dec_release() {
wave6_vpu_remove_dbgfs_file(inst);
v4l2_m2m_ctx_release(inst->v4l2_fh.m2m_ctx);
if (inst->state != VPU_INST_STATE_NONE)
wave6_vpu_dec_destroy_instance(inst, false);
...
}drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c:wave6_vpu_finish_job() {
cancel_delayed_work(&inst->task_timer);
v4l2_m2m_job_finish(inst->dev->m2m_dev, inst->v4l2_fh.m2m_ctx);
}wave6_vpu_dec_stop_streaming() does not destroy the instance (nor does wave6_vpu_enc_stop_streaming() when only the OUTPUT queue was streaming), so v4l2_m2m_ctx_release() frees m2m_ctx before destroy_instance() calls wave6_vpu_disable_instance() to clear inst->enable, synchronize_irq() and cancel_delayed_work_sync(&inst->task_timer):
CPU0 CPU1wave6_vpu_dec_release()
v4l2_m2m_ctx_release(m2m_ctx)
kfree(m2m_ctx)
wave6_vpu_core_irq_thread()
inst->ops->finish_process()
...
v4l2_m2m_job_finish(..., m2m_ctx)
<-- use-after-free wave6_vpu_dec_destroy_instance()
wave6_vpu_disable_instance()
In addition, wave6_vpu_finish_job() uses cancel_delayed_work() rather than cancel_delayed_work_sync(), allowing an already-running wave6_vpu_device_run_timeout() work item to access m2m_ctx and call v4l2_m2m_job_finish() concurrently with frame completion.
[Severity: High]
In commit "media: chips-media: wave6: Add Wave6 m2m encoder", can starting and stopping only the encoder OUTPUT queue leave the instance in VPU_INST_STATE_PIC_RUN with stale format parameters and undersized FBC DMA buffers?
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c:wave6_vpu_enc_start_streaming() {
...
if (inst->state == VPU_INST_STATE_NONE) {
ret = wave6_vpu_enc_create_instance(inst);
if (ret)
goto exit;
}
...
if (inst->state == VPU_INST_STATE_INIT_SEQ) {
ret = wave6_vpu_enc_prepare_fb(inst);
...
}drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c:wave6_vpu_enc_stop_streaming() {
...
if (V4L2_TYPE_IS_CAPTURE(q->type) && inst->state != VPU_INST_STATE_NONE)
wave6_vpu_enc_destroy_instance(inst, true);
...
}wave6_vpu_enc_start_streaming() creates the firmware instance and allocates FBC buffers on the first queue that starts streaming (even if q->type is OUTPUT), transitioning inst->state to VPU_INST_STATE_PIC_RUN. Because wave6_vpu_enc_stop_streaming() only destroys the instance when stopping the CAPTURE queue, stopping OUTPUT leaves inst->state at VPU_INST_STATE_PIC_RUN while neither queue is streaming; a subsequent VIDIOC_S_FMT to a larger resolution followed by VIDIOC_STREAMON skips instance recreation and FBC buffer reallocation.
[Severity: High]
In commit "media: chips-media: wave6: Add Wave6 m2m decoder", does a failure in wave6_vpu_dec_seek_header() trigger an infinite workqueue loop in wave6_vpu_device_run()?
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c:wave6_vpu_dec_prepare_decode() {
...
wave6_handle_bitstream_buffer(inst, stream_end);
if (inst->state < VPU_INST_STATE_PIC_RUN) {
if (inst->state == VPU_INST_STATE_OPEN &&
wave6_vpu_dec_seek_header(inst)) {
vb2_queue_error(v4l2_m2m_get_src_vq(inst->v4l2_fh.m2m_ctx));
vb2_queue_error(v4l2_m2m_get_dst_vq(inst->v4l2_fh.m2m_ctx));
}
return -EAGAIN;
}
...
}When wave6_vpu_dec_seek_header() fails (for example on an unsupported profile or timeout), the source buffer remains queued on m2m_ctx and inst->state stays VPU_INST_STATE_OPEN. Because wave6_vpu_job_ready() unconditionally returns 1 when inst->state == VPU_INST_STATE_OPEN, calling v4l2_m2m_job_finish() in wave6_vpu_device_run() immediately reschedules wave6_vpu_device_run() in an endless loop.
[Severity: Medium]
In commit "media: chips-media: wave6: Add Wave6 m2m decoder", does wave6_vpu_dec_destroy_instance() leave inst->fbc_buf_registered set to true when an instance is destroyed and recreated during a seek?
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c:wave6_vpu_dec_destroy_instance() {
...
wave6_vpu_dec_release_fb(inst);
wave6_vpu_set_instance_state(inst, VPU_INST_STATE_NONE);
...
}When wave6_vpu_dec_queue_setup() calls wave6_vpu_dec_destroy_instance() in VPU_INST_STATE_SEEK, the FBC buffers are freed and the firmware instance is closed, but inst->fbc_buf_registered is not reset to false on the reused struct vpu_instance. If streaming restarts and a drain occurs before new FBC buffers are registered, wave6_vpu_dec_prepare_decode() sees the stale inst->fbc_buf_registered flag and skips wave6_vpu_dec_finish_drain().
[Severity: Medium]
In commit "media: chips-media: wave6: Add Wave6 core driver", can wave6_vpu_get() be skipped if a core device is already runtime-active when an instance is created?
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c:wave6_vpu_core_runtime_resume() {
...
if (atomic_read(&core->active_count)) {
ret = wave6_vpu_get(core->vpu, core->reg_base);
if (ret)
goto error_get_vpu;
core->vpu_held = true;
}
...
}If the core device is runtime-resumed while core->active_count is 0 (for example via sysfs power/control), wave6_vpu_core_runtime_resume() finishes with core->vpu_held == false. When wave6_vpu_dec_create_instance() or wave6_vpu_enc_create_instance() later increments active_count and calls pm_runtime_resume_and_get(inst->dev->dev), runtime PM sees the core device is already RPM_ACTIVE and returns 0 without calling wave6_vpu_core_runtime_resume(), leaving the parent VPU device unpowered.
[Severity: Low]
In commit "media: chips-media: wave6: Add trace and debugfs", does wave6_vpu_core_probe() leak the top-level "wave6" debugfs directory?
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c:wave6_vpu_core_probe() {
...
core->debugfs = debugfs_lookup(WAVE6_VPU_DEBUGFS_DIR, NULL);
if (!IS_ERR_OR_NULL(core->debugfs))
dput(core->debugfs);
else
core->debugfs = debugfs_create_dir(WAVE6_VPU_DEBUGFS_DIR, NULL);
...
}Nothing removes this directory on probe failure, device removal, or module unload.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=11
Re: [PATCH v8 02/11] media: chips-media: wave6: Add Wave6 control driver
In reply to [email protected]Quoted 5 lines · On Thu, Nov 19, 2026 at 09:26:25AM +0000, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 8 potential issues to consider:
- [High] media: wave6:vpu->lockheld across core device register and unregister
- [High] media: wave6: data race onwork_buffers_availin
Quoted 8 lines
wave6_vpu_require_work_buffer()
- [Medium] media: wave6: stuck PREPARE state on work buf error inwave6_vpu_init_vpu()
- [Medium] media: wave6: incompatiblephys_addr_tpointer inwave6_vpu_probe()
- [Medium] media: wave6:wave6_vpu_probe()succeeds without required memory-region
- [Medium] media: wave6: wrong product code value forWAVE677_CODE
Please fix above issues
Frank
Quoted 338 lines
- [Low] media: wave6: off-by-one
of_irq_get()error check inwave6_vpu_create_cores()
- [Low] media: wave6: mixed guard(mutex) and goto cleanup inwave6_vpu_get()
--commit
600333b152222fe3a820d242ccd857c34cf7d58c
Author: Nas Chung <[email protected]>media: chips-media: wave6: Add Wave6 control driver
This commit adds the control driver for the Chips&Media Wave6 video codec IP to manage firmware loading, shared memory resources, and child core devices.
Quoted 5 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +0
-0[ ... ]
Quoted 37 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +37
-0+static int wave6_vpu_init_vpu (struct wave6_vpu_device *vpu, +void __iomem *core_base) +{ +int ret; + +lockdep_assert_held (&vpu->lock); + +/* try init directly as firmware is running */ +if (wave6_vdi_readl (core_base,W6_VPU_VCPU_CUR_PC )) + goto init_done; + +wave6_vpu_set_state (vpu,WAVE6_VPU_STATE_PREPARE ); + +wave6_vpu_remap_code_buf (vpu); + +wave6_vdi_writel (core_base,W6_VPU_BUSY_STATUS ,BUSY_STATUS_SET ); +wave6_vdi_writel (core_base,W6_CMD_INIT_VPU_SEC_AXI_BASE_CORE0 , + vpu->sram_buf.dma_addr); +wave6_vdi_writel (core_base,W6_CMD_INIT_VPU_SEC_AXI_SIZE_CORE0 , + vpu->sram_buf.size); +wave6_vdi_writel (vpu->reg_base,W6_COMMAND_GB ,W6_CMD_INIT_VPU ); +wave6_vdi_writel (vpu->reg_base,W6_VPU_REMAP_CORE_START_GB , +REMAP_CORE_START_ON ); + + ret =wave6_vpu_wait_busy (core_base); +if (ret) { +dev_err (vpu->dev,"init vpu timeout\n" ); +wave6_vpu_set_state (vpu,WAVE6_VPU_STATE_OFF ); +return -EINVAL ; + } + + ret =wave6_vpu_check_result (core_base); +if (ret) { +dev_err (vpu->dev,"init vpu fail, reason 0x%x\n" , ret); +wave6_vpu_set_state (vpu,WAVE6_VPU_STATE_OFF ); +return -EIO ; + }drivers/media/platform/chips-media/wave6/wave6-vpu.c line 278
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +10
-0+ +init_done: + ret =wave6_vpu_init_work_buf (vpu, core_base); +if (ret) +return ret; + +wave6_vpu_set_state (vpu,WAVE6_VPU_STATE_ON ); + +return 0 ; +}[Severity: Medium]
Does returning onwave6_vpu_init_work_buf()failure leavevpu->statestuck inWAVE6_VPU_STATE_PREPARE?Unlike the timeout and error paths above
init_done, this path does not resetvpu->statetoWAVE6_VPU_STATE_OFF.wave6_vpu_get()then powers down the device and dropscore_countto 0, and every laterwave6_vpu_get()->wave6_vpu_try_boot()seesvpu->state==WAVE6_VPU_STATE_PREPAREand returns 0 without booting the VPU or initializing work buffers.[ ... ]
Quoted 17 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +17
-0+int wave6_vpu_get (struct wave6_vpu_device *vpu,void __iomem *core_base) +{ +int ret; + +guard (mutex)(&vpu->lock); + +/* Only the first core executes boot; others return */ +if (atomic_inc_return (&vpu->core_count) >1 ) +return 0 ; + + ret =pm_runtime_resume_and_get (vpu->dev); +if (ret) + goto error_pm; + + ret =wave6_vpu_try_boot (vpu, core_base); +if (ret) + goto error_boot;drivers/media/platform/chips-media/wave6/wave6-vpu.c line 410
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +10
-0+ +return 0 ; + +error_boot: +pm_runtime_put_sync (vpu->dev); +error_pm: +atomic_dec (&vpu->core_count); + +return ret; +}[Severity: Low]
This isn't a bug, butwave6_vpu_get()mixes guard(mutex) with goto-based error unwinding in the same function, contrary to the Cleanup API conventions ininclude/linux/cleanup.h.[ ... ]
Quoted 14 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +14
-0+void wave6_vpu_require_work_buffer (struct wave6_vpu_device *vpu, +void __iomem *core_base) +{ +struct vpu_buf *vb; +u32 size; + + size =wave6_vdi_readl (core_base,W6_CMD_SET_WORK_BUF_SIZE ); +if (!size) +return ; + +if (WARN_ON (size >W637DEC_WORKBUF_SIZE_FOR_CQ )) + goto exit; + +if (WARN_ON (vpu->work_buffers_avail ==0 ))drivers/media/platform/chips-media/wave6/wave6-vpu.c line 448
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +10
-0+ goto exit; + + vpu->work_buffers_avail--; + vb = &vpu->work_buffers[vpu->work_buffers_avail]; + +wave6_vdi_writel (core_base,W6_CMD_SET_WORK_BUF_ADDR , vb->daddr); + +exit: +wave6_vdi_writel (core_base,W6_CMD_SET_WORK_BUF_SIZE ,SET_WORK_BUF_SIZE_ACK ); +}[Severity: High]
Can concurrent calls towave6_vpu_require_work_buffer()across VPU cores race onvpu->work_buffers_avail?In commit "media: chips-media: wave6: Add Wave6 core driver", each child core device calls
wave6_vpu_require_work_buffer()directly from itswave6_vpu_core_irq()hardirq handler without holding any shared lock:CPU0 (<hardirq> core 0) CPU1 (<hardirq> core 1)
wave6_vpu_core_irq()wave6_vpu_core_irq()
wave6_vpu_require_work_buffer()wave6_vpu_require_work_buffer()if (work_buffers_avail == 0) if (work_buffers_avail == 0) work_buffers_avail--; work_buffers_avail--; vb = &work_buffers[avail]; vb = &work_buffers[avail];<-- duplicate buffer or underflow
Two cores can be assigned the same work_buffers[] entry, or when
vpu->work_buffers_availis 1, both cores can pass the zero check and underflow the u32 counter to0xffffffff, reading vpu->work_buffers[] out of bounds.Also, when jumping to exit on error, should
W6_CMD_SET_WORK_BUF_ADDRbe cleared before writingSET_WORK_BUF_SIZE_ACKso the firmware does not consume a stale address?Quoted 17 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +17
-0+static int wave6_vpu_create_cores (struct wave6_vpu_device *vpu) +{ +struct device_node *child; +int num_cores =0 ; + +for_each_available_child_of_node (vpu->dev->of_node, child) { +struct platform_device_info info = {}; +struct platform_device *pdev; +struct resource res[2 ]; +int irq; + +if (num_cores >=W6_VPU_MAX_NUM_CORE ) { +of_node_put (child); +break ; + } + +if (of_address_to_resource (child,0 , &res[0 ])) {drivers/media/platform/chips-media/wave6/wave6-vpu.c line 476
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +10
-0+dev_warn (vpu->dev,"%pOF: missing reg property\n" , child); +continue ; + } + + irq =of_irq_get (child,0 ); +if (irq <0 ) { +dev_warn (vpu->dev,"%pOF: missing interrupts property\n" , child); +continue ; + } + res[1 ] =DEFINE_RES_IRQ (irq);[Severity: Low]
Should the error check forof_irq_get()be "if (irq <= 0)"?
of_irq_get()returns 0 when IRQ mapping fails, which would register the child platform device with an invalid IRQ 0 resource.[ ... ]
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +6
-0+static void wave6_vpu_release (struct wave6_vpu_device *vpu) +{ +guard (mutex)(&vpu->lock); + +wave6_vpu_destroy_cores (vpu); +wave6_vpu_free_work_buffers (vpu);[ ... ]
Quoted 13 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +13
-0+} + +static void wave6_vpu_init_firmware (struct wave6_vpu_device *vpu, +const struct firmware *fw) +{ +guard (mutex)(&vpu->lock); + +if (!fw || !fw->data) { +dev_err (vpu->dev,"No firmware.\n" ); +return ; + } + +if (fw->size +W6_EXTRA_CODE_BUF_SIZE >wave6_vpu_get_code_buf_size (vpu)) {drivers/media/platform/chips-media/wave6/wave6-vpu.c line 559
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +10
-0+dev_err (vpu->dev,"firmware size (%zd > %zd) is too big\n" , + fw->size, vpu->code_buf.size); +return ; + } + +memcpy (vpu->code_buf.vaddr, fw->data, fw->size); + +if (!wave6_vpu_create_cores (vpu)) +dev_err (vpu->dev,"Failed to create VPU cores\n" ); +}[Severity: High]
Does holdingvpu->lockacrosswave6_vpu_create_cores()andwave6_vpu_destroy_cores()invert the lock hierarchy against child core device locks and runtime PM transitions?
wave6_vpu_create_cores()->platform_device_register_full()andwave6_vpu_destroy_cores()->platform_device_unregister()acquire the child core device'sdevice_lock(core_dev->mutex) whilevpu->lockis held, andplatform_device_unregister()->__device_release_driver()also runspm_runtime_get_sync(core_dev)andpm_runtime_put_sync(core_dev). In commit "media: chips-media: wave6: Add Wave6 core driver", the child's runtime PM and system sleep callbacks callwave6_vpu_get()andwave6_vpu_put(), which acquirevpu->lockundercore_dev->mutexor during acore_devruntime PM transition:CPU0 CPU1
---- ----lock(&vpu->lock)lock(&core_dev->mutex) lock(&core_dev->mutex) lock(&vpu->lock)*** DEADLOCK ***
CPU0:
wave6_vpu_init_firmware()->wave6_vpu_create_cores()->
platform_device_register_full()
CPU1:device_suspend()->wave6_vpu_core_suspend()->pm_runtime_force_suspend() -> wave6_vpu_core_runtime_suspend() -> wave6_vpu_put()During teardown, if
core_devis inRPM_RESUMINGorRPM_SUSPENDINGwaiting forvpu->lockinwave6_vpu_get()orwave6_vpu_put()whenwave6_vpu_release()runs, __device_release_driver()'spm_runtime_get_sync(core_dev)will also wait oncore_dev->power.wait_queuewhile holdingvpu->lock.[ ... ]
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +6
-0+ np =of_parse_phandle (pdev->dev.of_node,"memory-region" ,0 ); +if (np) { +struct resource mem; + + ret =of_address_to_resource (np,0 , &mem); +of_node_put (np);drivers/media/platform/chips-media/wave6/wave6-vpu.c line 674
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +10
-0+if (!ret) { + vpu->code_buf.phys_addr = mem.start; + vpu->code_buf.size =resource_size (&mem); + ret =wave6_vpu_init_code_buf (vpu); +if (ret) + goto error_detach_pm_domains; + }else { +dev_warn (&pdev->dev,"memory-region is not available.\n" ); + } + }[Severity: Medium]
Shouldwave6_vpu_probe()fail and jump toerror_detach_pm_domainswhen "memory-region" is missing orof_address_to_resource()fails?Without it,
vpu->code_buf.sizeremains 0 whilewave6_vpu_probe()continues, allocates 48 MiB of coherent DMA work buffers and SRAM, and returns 0, only forwave6_vpu_init_firmware()to unconditionally fail its size check againstwave6_vpu_get_code_buf_size(vpu)and leave those resources allocated without creating any core devices.drivers/media/platform/chips-media/wave6/wave6-vpu.c line 681
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +6
-0+ vpu->sram_pool =of_gen_pool_get (pdev->dev.of_node,"sram" ,0 ); +if (vpu->sram_pool) { + vpu->sram_buf.size = vpu->res->sram_size; + vpu->sram_buf.vaddr =gen_pool_dma_alloc (vpu->sram_pool, + vpu->sram_buf.size, + &vpu->sram_buf.phys_addr);[Severity: Medium]
Does passing&vpu->sram_buf.phys_addrtogen_pool_dma_alloc()fail to build whenphys_addr_tanddma_addr_thave different widths?struct
wave6_vpu_dma_bufdeclaresphys_addrasphys_addr_t, whereasgen_pool_dma_alloc()takes adma_addr_t* third argument, which triggers -Werror=incompatible-pointer-types on configs whereCONFIG_PHYS_ADDR_T_64BITandCONFIG_ARCH_DMA_ADDR_T_64BITdiffer (exposed byCOMPILE_TEST).Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h new file +0
-0@@ -0,0 +1,37 @@[ ... ]
drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h line 16
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h new file +6
-0+#define WAVE617_CODE 0x6170 +#define WAVE627_CODE 0x6270 +#define WAVE633_CODE 0x6330 +#define WAVE637_CODE 0x6370 +#define WAVE663_CODE 0x6630 +#define WAVE677_CODE 0x6670[Severity: Medium]
ShouldWAVE677_CODEbe0x6770instead of0x6670?All other Wave6 product codes encode the three-digit model number in bits [15:4], so
wave6_is_product_w_series()inwave6_vpu_try_boot()would reject a Wave677 reporting0x6770with-EINVAL.--
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Re: [PATCH v8 02/11] media: chips-media: wave6: Add Wave6 control driver
In reply to Nas ChungQuoted 16 lines · Nas Chung
Add the control driver for the Chips&Media Wave6 video codec IP.
The hardware contains one control register region and four interface register regions for a shared video processing engine.
This driver handles the control region and manages shared resources such as firmware loading, firmware memory allocation, and coordination required by the interface register regions.
It also instantiates and coordinates with the
wave6-corechild devices for firmware and power state management.Tested-by Ming Qian <[email protected]>Tested-by Marek Vasut <[email protected]>Signed-off-by Nas Chung <[email protected]>---
...
drivers/media/platform/chips-media/wave6/wave6-vdi.h line 56
Quoted 5 lines
drivers/media/platform/chips-media/wave6/wave6-vdi.h new file +5
-0+ +static inline int wave6_vdi_alloc_dma (struct device *dev,struct vpu_buf *vb) +{ +void *vaddr; +dma_addr_t daddr;
Nit try keep revise christmas order, check others
dma_addr_t daddr;
void *vaddr;
Quoted 15 lines
drivers/media/platform/chips-media/wave6/wave6-vdi.h new file +15
-0+ +if (!vb || !vb->size) +return -EINVAL ; + + vaddr =dma_alloc_coherent (dev, vb->size, &daddr,GFP_KERNEL ); +if (!vaddr) +return -ENOMEM ; + + vb->vaddr = vaddr; + vb->daddr = daddr; + vb->dev = dev; + +return 0 ; +} +
...
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +6
-0+ +#include <linux/kernel.h> +#include <linux/module.h> +#include <linux/platform_device.h> +#include <linux/clk.h> +#include <linux/of.h>
drivers/media/platform/chips-media/wave6/wave6-vpu.c line 22
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +10
-0+#include <linux/of_address.h> +#include <linux/property.h> +#include <linux/of_irq.h> +#include <linux/firmware.h> +#include <linux/interrupt.h> +#include <linux/pm_runtime.h> +#include <linux/pm_domain.h> +#include <linux/dma-mapping.h> +#include <linux/iopoll.h> +#include <linux/genalloc.h>
order by alpabet.
Quoted 11 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +11
-0+ +#include "wave6-vpuconfig.h" +#include "wave6-regdefine.h" +#include "wave6-vpu.h" + +static const struct wave6_vpu_resource wave633c_data = { + .fw_name ="cnm/wave633c_imx9_codec_fw.bin" , +/* For HEVC, AVC, 4096x4096, 8bit */ + .sram_size =0x14800 , +}; +
...
drivers/media/platform/chips-media/wave6/wave6-vpu.c line 126
Quoted 4 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +4
-0+ +static u32 wave6_vpu_get_code_buf_size (struct wave6_vpu_device *vpu) +{ +return min_t (u32, vpu->code_buf.size,W6_MAX_CODE_BUF_SIZE );
now use min()
Quoted 2 lines
(quoted diff) +2
-0+} +
...
drivers/media/platform/chips-media/wave6/wave6-vpu.c line 627
Quoted 13 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +13
-0+ +static int wave6_vpu_probe (struct platform_device *pdev) +{ +struct device_node *np; +struct wave6_vpu_device *vpu; +const struct wave6_vpu_resource *res; +int ret; + + ret =dma_set_mask_and_coherent (&pdev->dev,DMA_BIT_MASK (32 )); +if (ret <0 ) { +dev_err (&pdev->dev,"failed to set DMA mask: %d\n" , ret); +return ret; + }
Needn't check when mask > 32, if check,
return dev_err_probe()
Quoted 5 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +5
-0+ + res =device_get_match_data (&pdev->dev); +if (!res) +return -ENODEV ; +
...
drivers/media/platform/chips-media/wave6/wave6-vpu.c line 701
Quoted 4 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +4
-0+ +wave6_vpu_allocate_work_buffers (vpu); + +pm_runtime_enable (&pdev->dev);
devm_pm_runtime_enable();
you have not enable clock here, so you need select PM at Kconfig, otherwise if !CONFIG_PM, other register access will be hang because stub do nothing of runtime pm when !CONFIG_PM
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +10
-0+ +init_completion (&vpu->fw_done); + ret =firmware_request_nowait_nowarn (THIS_MODULE , + vpu->res->fw_name, + &pdev->dev, +GFP_KERNEL , + vpu, + wave6_vpu_load_firmware); +if (ret) { +dev_err (&pdev->dev,"request firmware fail, ret = %d\n" , ret);
drivers/media/platform/chips-media/wave6/wave6-vpu.c line 721
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +10
-0+ goto error; + } + +return 0 ; + +error: +wave6_vpu_release (vpu); +pm_runtime_disable (&pdev->dev); +error_detach_pm_domains: +wave6_vpu_detach_pm_domains (vpu);
use devm_add_action_or_reset() to avoid goto in probe()
Quoted 4 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +4
-0+ +return ret; +} +
...
Quoted 20 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +6
-0+ +module_platform_driver (wave6_vpu_driver); +MODULE_DESCRIPTION ("chips&media Wave6 VPU driver" ); +MODULE_AUTHOR ("CHIPS&MEDIA INC" ); +MODULE_FIRMWARE ("cnm/wave633c_imx9_codec_fw.bin" ); +MODULE_LICENSE ("Dual BSD/GPL" );drivers/media/platform/chips-media/wave6/wave6-vpu.h new file +8
-0@@ -0,0 +1,137 @@ +/* SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) */ +/* +* Wave6 series multi-standard codec IP - wave6 driver +* +* Copyright (C) 2026 CHIPS&MEDIA INC +*/ + +#ifndef __WAVE6_VPU_H__
drivers/media/platform/chips-media/wave6/wave6-vpu.h line 18
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.h new file +10
-0+#define __WAVE6_VPU_H__ + +#include <linux/completion.h> +#include <linux/device.h> + +#include "wave6-vpuconfig.h" +#include "wave6-vdi.h" + +#define WAVE6_VPU_PLATFORM_DRIVER_NAME "wave6-vpu" +#define WAVE6_VPU_CORE_PLATFORM_DRIVER_NAME "wave6-vpu-core"
If only use once, needn't define macro.
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.h new file +10
-0+ +struct wave6_vpu_device ; + +/** +* enum wave6_vpu_state - VPU states +* @WAVE6_VPU_STATE_OFF: VPU is powered off +* @WAVE6_VPU_STATE_PREPARE: VPU is booting +* @WAVE6_VPU_STATE_ON: VPU is running +* @WAVE6_VPU_STATE_SLEEP: VPU is in a sleep mode +*/
...
drivers/media/platform/chips-media/wave6/wave6-vpu.h
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.h new file +6
-0+ +#define W6_VPU_MAX_NUM_CORE 4 +#define MAX_NUM_INSTANCE 32 + +#define W6_VPU_POLL_DELAY_US 10 +#define W6_VPU_POLL_TIMEOUT 300000
Need unit surfix
Frank
Quoted 19 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.h new file +15
-1+ +#define W6_MAX_REMAP_PAGE_SIZE (1024 * 1024) +#define W6_MAX_CODE_BUF_SIZE (4 * 1024 * 1024) +#define W6_CODE_BUF_SIZE (1 * 1024 * 1024) +#define W6_EXTRA_CODE_BUF_SIZE (256 * 1024) +#define W637DEC_WORKBUF_SIZE_FOR_CQ (3 * 512 * 1024) + +static inline bool wave6_is_product_w_series (u32 code) +{ +return code ==WAVE617_CODE || code ==WAVE627_CODE || + code ==WAVE633_CODE || code ==WAVE637_CODE || + code ==WAVE663_CODE || code ==WAVE677_CODE ; +} + +#endif /* __WAVE6_VPUCONFIG_H__ */ --2.41.0.windows.3
Re: [PATCH v8 03/11] media: chips-media: wave6: Add Wave6 core driver
In reply to Nas ChungQuoted 36 lines · Nas Chung
Add the core driver for the Chips&Media Wave6 video codec IP.
The hardware contains one control register region and four interface register regions for a shared video processing engine. The core driver handles the interface register regions, each with its own MMIO range and interrupt, while relying on the control driver for firmware loading and shared resource management.
It configures the V4L2 mem2mem devices and communicates with the Wave6 hardware to perform video processing tasks.
Tested-by Ming Qian <[email protected]>Tested-by Marek Vasut <[email protected]>Signed-off-by Nas Chung <[email protected]>--- .../media/platform/chips-media/wave6/Kconfig | 2 + .../media/platform/chips-media/wave6/Makefile | 5 +- .../platform/chips-media/wave6/wave6-hw.c | 181 ++++++++ .../platform/chips-media/wave6/wave6-hw.h | 22 + .../chips-media/wave6/wave6-regdefine.h | 65 +++ .../platform/chips-media/wave6/wave6-vdi.h | 20 + .../chips-media/wave6/wave6-vpu-core.c | 392 ++++++++++++++++++ .../chips-media/wave6/wave6-vpu-core.h | 106 +++++ .../chips-media/wave6/wave6-vpu-v4l2.c | 384 +++++++++++++++++ .../platform/chips-media/wave6/wave6-vpu.c | 24 +- .../platform/chips-media/wave6/wave6-vpu.h | 2 + .../platform/chips-media/wave6/wave6-vpuapi.h | 381 +++++++++++++++++ .../chips-media/wave6/wave6-vpuconfig.h | 6 + 13 files changed, 1588 insertions(+), 2 deletions(-) create mode 100644 drivers/media/platform/chips-media/wave6/wave6-hw.c create mode 100644 drivers/media/platform/chips-media/wave6/wave6-hw.h create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-core.c create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-core.h create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpuapi.h
...
drivers/media/platform/chips-media/wave6/wave6-hw.c line 28
Quoted 8 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c new file +8
-0+ +u32 wave6_vpu_readl (struct vpu_core_device *core,u32 addr) +{ +u32 data; + + data =wave6_vdi_readl (core->reg_base, addr); + +return data;
return wave6_vdi_readl(core->reg_base, addr);
drivers/media/platform/chips-media/wave6/wave6-hw.c line 40
Quoted 12 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c new file +12
-0+} + +void wave6_vpu_writel_relaxed (struct vpu_core_device *core,u32 addr,u32 data) +{ +wave6_vdi_writel_relaxed (core->reg_base, addr, data); +} + +u32 wave6_vpu_readl_relaxed (struct vpu_core_device *core,u32 addr) +{ +u32 data; + + data =wave6_vdi_readl_relaxed (core->reg_base, addr);
return wave6_vdi_readl_relaxed(core->reg_base, addr);
Quoted 4 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c new file +4
-0+ +return data; +} +
...
Quoted 8 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c new file +8
-0+static u32 wave6_vpu_get_product_id (struct vpu_core_device *core) +{ +u32 val; + +lockdep_assert_held (&core->hw_lock); + + val =vpu_read_reg_relaxed (core,W6_VPU_RET_PRODUCT_CODE ); +
drivers/media/platform/chips-media/wave6/wave6-hw.c line 99
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c new file +10
-0+switch (val) { +case WAVE617_CODE : +return PRODUCT_ID_617 ; +case WAVE627_CODE : +return PRODUCT_ID_627 ; +case WAVE633_CODE : +case WAVE637_CODE : +case WAVE663_CODE : +case WAVE677_CODE : +return PRODUCT_ID_637 ;
what is use for? why not direct use WAVE*_CODE?
Quoted 8 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c new file +8
-0+default : +dev_err (core->dev,"Invalid product (%x)\n" , val); +break ; + } + +return PRODUCT_ID_NONE ; +} +
...
Quoted 7 lines
drivers/media/platform/chips-media/wave6/wave6-hw.h new file +2
-0+ +#endif /* __WAVE6_HW_H__ */ drivers/media/platform/chips-media/wave6/wave6-regdefine.h +0
-0
drivers/media/platform/chips-media/wave6/wave6-regdefine.h enum wave6_command { · line 23
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-regdefine.h +7
-0@@ -14,14 +14,51 @@ enum wave6_command {W6_CMD_INIT_VPU =0x0001 ,W6_CMD_WAKEUP_VPU =0x0002 ,W6_CMD_SLEEP_VPU =0x0004 , +W6_CMD_CREATE_INSTANCE =0x0008 , +W6_CMD_FLUSH_INSTANCE =0x0010 , +W6_CMD_DESTROY_INSTANCE =0x0020 , +W6_CMD_INIT_SEQ =0x0040 , +W6_CMD_SET_FB =0x0080 , +W6_CMD_DEC_PIC =0x0100 , + W6_CMD_DEC_SET_DISP = 0x0400,
BIT()?
Quoted 4 lines
drivers/media/platform/chips-media/wave6/wave6-regdefine.h +3
-0W6_CMD_INIT_WORK_BUF =0x1000 , + W6_CMD_QUERY = 0x4000, +}; +
...
drivers/media/platform/chips-media/wave6/wave6-vdi.h line 16
Quoted 11 lines
#endif /*
__WAVE6_REGDEFINE_H__*/drivers/media/platform/chips-media/wave6/wave6-vdi.h +1
-0@@ -13,6 +13,26 @@#include <linux/dma-mapping.h> #include <linux/io.h> +enum endian_mode {
name is too general and easy happen name collision.
Quoted 31 lines
drivers/media/platform/chips-media/wave6/wave6-vdi.h +19
-0+VDI_128BIT_BIG_ENDIAN =0 , +VDI_128BIT_BE_BYTE_SWAP , +VDI_128BIT_BE_WORD_SWAP , +VDI_128BIT_BE_WORD_BYTE_SWAP , +VDI_128BIT_BE_DWORD_SWAP , +VDI_128BIT_BE_DWORD_BYTE_SWAP , +VDI_128BIT_BE_DWORD_WORD_SWAP , +VDI_128BIT_BE_DWORD_WORD_BYTE_SWAP , +VDI_128BIT_LE_DWORD_WORD_BYTE_SWAP , +VDI_128BIT_LE_DWORD_WORD_SWAP , +VDI_128BIT_LE_DWORD_BYTE_SWAP , +VDI_128BIT_LE_DWORD_SWAP , +VDI_128BIT_LE_WORD_BYTE_SWAP , +VDI_128BIT_LE_WORD_SWAP , +VDI_128BIT_LE_BYTE_SWAP , +VDI_128BIT_LITTLE_ENDIAN =15 , +VDI_ENDIAN_MAX +}; +/** * struct vpu_buf - VPU buffer for a coherent DMA buffer * @size: Buffer size drivers/media/platform/chips-media/wave6/wave6-vpu-core.c new file +3
-0@@ -0,0 +1,392 @@ +// SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) +/* +* Wave6 series multi-standard codec IP - wave6 core driver
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c line 13
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c new file +10
-0+* +* Copyright (C) 2026 CHIPS&MEDIA INC +*/ + +#include <linux/kernel.h> +#include <linux/platform_device.h> +#include <linux/clk.h> +#include <linux/interrupt.h> +#include <linux/pm_runtime.h> +#include <linux/iopoll.h>
sort
Quoted 8 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c new file +8
-0+ +#include "wave6-vpu-core.h" +#include "wave6-regdefine.h" +#include "wave6-vpuconfig.h" +#include "wave6-hw.h" + +static irqreturn_t wave6_vpu_core_irq (int irq,void *dev_id) +{
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c line 31
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c new file +10
-0+struct vpu_core_device *core = dev_id; +struct vpu_irq irq_info; + +if (!vpu_read_reg (core,W6_VPU_VPU_INT_STS )) +return IRQ_NONE ; + + irq_info.status =vpu_read_reg (core,W6_VPU_VINT_REASON ); + irq_info.inst_idc =vpu_read_reg (core,W6_RET_INT_INSTANCE_INFO ); + +vpu_write_reg (core,W6_RET_INT_INSTANCE_INFO ,INT_INSTANCE_INFO_CLEAR );
suppose only already set irq, irq_info.inst_idc, like below code?
Quoted 18 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c new file +18
-0+vpu_write_reg (core,W6_VPU_VINT_REASON_CLEAR , irq_info.status); +vpu_write_reg (core,W6_VPU_VINT_CLEAR ,VINT_CLEAR ); + +if (irq_info.status &BIT (W6_INT_BIT_REQ_WORK_BUF )) { +wave6_vpu_require_work_buffer (core->vpu, core->reg_base); + +return IRQ_HANDLED ; + } + +if (kfifo_in (&core->irq_fifo, &irq_info,sizeof (struct vpu_irq )) != +sizeof (struct vpu_irq )) +dev_err (core->dev,"IRQ overflow status 0x%x idc 0x%x\n" , + irq_info.status, irq_info.inst_idc); + +return IRQ_WAKE_THREAD ; +} + + core->irq =platform_get_irq (pdev,0 );
...
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c line 245
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c new file +10
-0+if (core->irq <0 ) +return core->irq; + + ret =devm_request_threaded_irq (&pdev->dev, core->irq, + wave6_vpu_core_irq, + wave6_vpu_core_irq_thread, +0 ,"vpu_irq" , core); +if (ret) { +dev_err (&pdev->dev,"failed to request IRQ: %d\n" , ret); +return ret;
return dev_err_probe();
Quoted 8 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c new file +8
-0+ } + + ret =v4l2_device_register (&pdev->dev, &core->v4l2_dev); +if (ret) { +dev_err (&pdev->dev,"failed to register v4l2_dev: %d\n" , ret); +return ret; + } +
...
Quoted 11 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c new file +6
-0+void wave6_vpu_writel (struct vpu_core_device *core,u32 addr,u32 data); +void wave6_vpu_writel_relaxed (struct vpu_core_device *core,u32 addr,u32 data); +u32 wave6_vpu_readl (struct vpu_core_device *core,u32 addr); +u32 wave6_vpu_readl_relaxed (struct vpu_core_device *core,u32 addr); + +#endif /* __WAVE6_VPUAPI_H__ */ drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h +0
-0
drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h line 27
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h +5
-0@@ -18,13 +18,19 @@#define W6_VPU_MAX_NUM_CORE 4#define MAX_NUM_INSTANCE 32 +#define W6_MAX_PIC_STRIDE (4096 * 4) +#define W6_PIC_STRIDE_ALIGNMENT 32 +#define W6_VPU_POLL_DELAY_US 10#define W6_VPU_POLL_TIMEOUT 300000 +#define W6_BOOT_WAIT_TIMEOUT 10000 +#define W6_VPU_TIMEOUT 6000
Need unit
Frank
Quoted 13 lines
drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h +1
-1#define W6_MAX_REMAP_PAGE_SIZE (1024 * 1024)#define W6_MAX_CODE_BUF_SIZE (4 * 1024 * 1024)#define W6_CODE_BUF_SIZE (1 * 1024 * 1024)#define W6_EXTRA_CODE_BUF_SIZE (256 * 1024) +#define W6_TEMPBUF_SIZE (3 * 1024 * 1024)#define W637DEC_WORKBUF_SIZE_FOR_CQ (3 * 512 * 1024)static inline bool wave6_is_product_w_series (u32 code) --2.41.0.windows.3
Re: [PATCH v8 04/11] media: chips-media: wave6: Add Wave6 decoder firmware interface
In reply to Nas ChungQuoted 21 lines · Nas Chung
Add an interface layer to manage hardware register configuration and communication with the Chips&Media Wave6 video codec IP for video decoding.
The interface provides low-level helper functions used by the Wave6 core driver to implement video decoding operations. It handles command submission to the firmware via MMIO registers, and waits for a response by polling the firmware busy flag.
The decoder commands cover instance creation, flush and destruction, sequence initialization, frame buffer and display buffer registration, and picture decoding.
Sequence initialization and picture decoding complete asynchronously:
the interface interrupt reports completion and a query command reads back the result.Tested-by Ming Qian <[email protected]>Tested-by Marek Vasut <[email protected]>Signed-off-by Nas Chung <[email protected]>---
...
Quoted 22 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c +22
-0+int wave6_vpu_dec_issue_seq_init (struct vpu_instance *inst) +{ +struct dec_info *p_dec_info; +u32 reg_val; +int ret; + +guard (mutex)(&inst->dev->hw_lock); + + p_dec_info = &inst->codec_info->dec_info; + +vpu_write_reg_relaxed (inst->dev,W6_CMD_DEC_INIT_SEQ_BS_RD_PTR , p_dec_info->stream_rd_ptr); +vpu_write_reg_relaxed (inst->dev,W6_CMD_DEC_INIT_SEQ_BS_WR_PTR , p_dec_info->stream_wr_ptr); + + reg_val =DEC_PIC_BS_OPTION_STREAM_END (p_dec_info->stream_end) | +DEC_PIC_BS_OPTION_EXPLICIT_END_ON ; +vpu_write_reg_relaxed (inst->dev,W6_CMD_DEC_INIT_SEQ_BS_OPTION , reg_val); + reg_val =INIT_SEQ_OPTION_MODE (W6_INIT_SEQ_OPT_NORMAL ); +vpu_write_reg_relaxed (inst->dev,W6_CMD_DEC_INIT_SEQ_OPTION , reg_val); + +wave6_send_command (inst->dev, inst->id, inst->std,W6_CMD_INIT_SEQ ); + ret =wave6_wait_vpu_busy (inst->dev,W6_VPU_BUSY_STATUS ); +if (ret) {
drivers/media/platform/chips-media/wave6/wave6-hw.c int wave6_vpu_get_version(struct vpu_core_device *core) · line 371
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c +10
-0+dev_err (inst->dev->dev,"%s: timeout\n" , __func__); +return ret; + } + +if (!vpu_read_reg_relaxed (inst->dev,W6_RET_SUCCESS )) { +u32 reason_code =vpu_read_reg_relaxed (inst->dev,W6_RET_FAIL_REASON ); + +wave6_print_reg_err (inst->dev, reason_code); +return -EIO ; + }
I see many place use the same pattern. Can you move to help function to macro like
#define wave6_send_command_sync(...) \ wave6_send_command() ... wave6_wait_vpu_busy() ... ...
Frank
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c +3
-0+ +return 0 ; +}2.41.0.windows.3
Re: [PATCH v8 09/11] media: chips-media: wave6: Add performance profiling
In reply to Nas ChungQuoted 2 lines · Nas Chung
Add per-instance performance profiling to help identify whether a throughput problem comes from the firmware or from the host side.
why not register as perf interface?
Frank
Quoted 514 lines
Record per-frame timestamps and the firmware-reported processing time, and report the frame rates and latencies in the per-instance debugfs file.
Tested-by Ming Qian <[email protected]>Tested-by Marek Vasut <[email protected]>Signed-off-by Nas Chung <[email protected]>--- .../platform/chips-media/wave6/wave6-hw.c | 28 ++++++ .../platform/chips-media/wave6/wave6-hw.h | 2 + .../chips-media/wave6/wave6-regdefine.h | 6 ++ .../chips-media/wave6/wave6-vpu-core.h | 10 ++ .../chips-media/wave6/wave6-vpu-dbg.c | 36 ++++++++ .../chips-media/wave6/wave6-vpu-dec.c | 15 +++ .../chips-media/wave6/wave6-vpu-enc.c | 13 +++ .../chips-media/wave6/wave6-vpu-v4l2.c | 91 +++++++++++++++++++ .../platform/chips-media/wave6/wave6-vpuapi.h | 28 ++++++ 9 files changed, 229 insertions(+)drivers/media/platform/chips-media/wave6/wave6-hw.c +28
-0@@ -872,6 +872,20 @@ int wave6_vpu_dec_get_output_info(struct vpu_instance *inst, struct dec_output_i info->timestamp =vpu_read_reg_relaxed (inst->dev,W6_RET_DEC_TIMESTAMP_HIGH ); info->timestamp = (info->timestamp <<32 ) | reg_val; + info->cycle.host_cmd_s =vpu_read_reg_relaxed (inst->dev,W6_RET_CQ_IN_TICK ); + info->cycle.host_cmd_e =vpu_read_reg_relaxed (inst->dev,W6_RET_RQ_OUT_TICK ); + info->cycle.proc_s =vpu_read_reg_relaxed (inst->dev,W6_RET_HW_RUN_TICK ); + info->cycle.proc_e =vpu_read_reg_relaxed (inst->dev,W6_RET_HW_DONE_TICK ); + info->cycle.vpu_s =vpu_read_reg_relaxed (inst->dev,W6_RET_FW_RUN_TICK ); + info->cycle.vpu_e =vpu_read_reg_relaxed (inst->dev,W6_RET_FW_DONE_TICK ); + info->cycle.frame_cycle = (info->cycle.vpu_e - info->cycle.host_cmd_s) * +CYCLE_PER_TICK_VALUE ; + info->cycle.proc_cycle = (info->cycle.proc_e - info->cycle.proc_s) * +CYCLE_PER_TICK_VALUE ; + info->cycle.vpu_cycle = ((info->cycle.vpu_e - info->cycle.vpu_s) - + (info->cycle.proc_e - info->cycle.proc_s)) * +CYCLE_PER_TICK_VALUE ; +if (decoded_idx >=0 && decoded_idx <WAVE6_MAX_FBS ) p_dec_info->dec_out_info[decoded_idx].decoded_poc = info->decoded_poc;@@ -1882,6 +1896,20 @@ int wave6_vpu_enc_get_output_info(struct vpu_instance *inst, struct enc_output_i info->timestamp =vpu_read_reg_relaxed (inst->dev,W6_RET_ENC_TIMESTAMP_HIGH ); info->timestamp = (info->timestamp <<32 ) | reg_val; + info->cycle.host_cmd_s =vpu_read_reg_relaxed (inst->dev,W6_RET_CQ_IN_TICK ); + info->cycle.host_cmd_e =vpu_read_reg_relaxed (inst->dev,W6_RET_RQ_OUT_TICK ); + info->cycle.proc_s =vpu_read_reg_relaxed (inst->dev,W6_RET_HW_RUN_TICK ); + info->cycle.proc_e =vpu_read_reg_relaxed (inst->dev,W6_RET_HW_DONE_TICK ); + info->cycle.vpu_s =vpu_read_reg_relaxed (inst->dev,W6_RET_FW_RUN_TICK ); + info->cycle.vpu_e =vpu_read_reg_relaxed (inst->dev,W6_RET_FW_DONE_TICK ); + info->cycle.frame_cycle = (info->cycle.vpu_e - info->cycle.host_cmd_s) * +CYCLE_PER_TICK_VALUE ; + info->cycle.proc_cycle = (info->cycle.proc_e - info->cycle.proc_s) * +CYCLE_PER_TICK_VALUE ; + info->cycle.vpu_cycle = ((info->cycle.vpu_e - info->cycle.vpu_s) - + (info->cycle.proc_e - info->cycle.proc_s)) * +CYCLE_PER_TICK_VALUE ; +return 0 ; }drivers/media/platform/chips-media/wave6/wave6-hw.h +2
-0@@ -34,6 +34,8 @@ enum product_id {#define MAX_CSC_COEFF_NUM 4 +#define CYCLE_PER_TICK_VALUE 256 +#define H265_NAL_UNIT_TYPE_IDR_W_RADL 19#define H265_NAL_UNIT_TYPE_IDR_N_LP 20drivers/media/platform/chips-media/wave6/wave6-regdefine.h +6
-0@@ -110,6 +110,12 @@ enum wave6_interrupt_bit {#define W6_RET_INT_INSTANCE_INFO (W6_REG_BASE + 0x21C)#define INT_INSTANCE_INFO_CLEAR 0#define W6_RET_INSTANCE_ID (W6_REG_BASE + 0x220) +#define W6_RET_CQ_IN_TICK (W6_REG_BASE + 0x23C) +#define W6_RET_FW_RUN_TICK (W6_REG_BASE + 0x240) +#define W6_RET_HW_RUN_TICK (W6_REG_BASE + 0x244) +#define W6_RET_HW_DONE_TICK (W6_REG_BASE + 0x248) +#define W6_RET_FW_DONE_TICK (W6_REG_BASE + 0x24C) +#define W6_RET_RQ_OUT_TICK (W6_REG_BASE + 0x250)/* COMMON - QUERY : GET_VPU_INFO */ #define W6_RET_FW_VERSION (W6_REG_BASE + 0x300)drivers/media/platform/chips-media/wave6/wave6-vpu-core.h +10
-0@@ -32,6 +32,11 @@ struct vpu_buffer {u32 force_i_frame_qp;u32 force_p_frame_qp;u32 force_b_frame_qp; +ktime_t ts_input; +ktime_t ts_start; +ktime_t ts_finish; +ktime_t ts_output; +u64 hw_time;u32 average_qp; };@@ -101,6 +106,7 @@ enum codec_std wave6_to_codec_std(enum vpu_instance_type type, unsigned int v4l2const char *wave6_vpu_instance_state_name (enum vpu_instance_state state);void wave6_vpu_set_instance_state (struct vpu_instance *inst,enum vpu_instance_state state); +u64 wave6_vpu_cycle_to_ns (struct vpu_core_device *core,u64 cycle);int wave6_vpu_wait_interrupt (struct vpu_instance *inst,unsigned int timeout);int wave6_vpu_dec_register_device (struct vpu_core_device *core);void wave6_vpu_dec_unregister_device (struct vpu_core_device *core);@@ -108,6 +114,10 @@ int wave6_vpu_enc_register_device(struct vpu_core_device *core);void wave6_vpu_enc_unregister_device (struct vpu_core_device *core);void wave6_vpu_device_run_timeout (struct work_struct *work);void wave6_vpu_finish_job (struct vpu_instance *inst); +void wave6_vpu_record_performance_timestamps (struct vpu_instance *inst); +void wave6_vpu_handle_performance (struct vpu_instance *inst, +struct vpu_buffer *vpu_buf); +void wave6_vpu_reset_performance (struct vpu_instance *inst);int wave6_vpu_init_m2m_dev (struct vpu_core_device *core);void wave6_vpu_release_m2m_dev (struct vpu_core_device *core); int wave6_vpu_subscribe_event(struct v4l2_fh *fh,drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.c +36
-0@@ -15,7 +15,13 @@static int wave6_vpu_dbg_instance (struct seq_file *s,void *data) {struct vpu_instance *inst = s->private; +struct vpu_performance_info *perf = &inst->performance;struct vb2_queue *vq; +s64 tmp; +s64 fps; +s64 duration; +s64 ms[3 ]; +s32 ns[3 ];if (!inst->v4l2_fh.m2m_ctx)return 0 ;@@ -77,6 +83,36 @@ static int wave6_vpu_dbg_instance(struct seq_file *s, void *data) inst->error_buf_num, v4l2_m2m_has_stopped(inst->v4l2_fh.m2m_ctx)); +seq_puts (s,"fps" ); + tmp =MSEC_PER_SEC * inst->processed_buf_num; +if (perf->ts_last > perf->ts_first +NSEC_PER_MSEC ) { + duration =ktime_ms_delta (perf->ts_last, perf->ts_first); + fps =DIV64_U64_ROUND_CLOSEST (tmp, duration); +seq_printf (s," actual: %lld;" , fps); + } + duration =div_u64 (perf->total_sw_time,NSEC_PER_MSEC ); +if (duration >0 ) { + fps =DIV64_U64_ROUND_CLOSEST (tmp, duration); +seq_printf (s," sw: %lld;" , fps); + } + duration =div_u64 (perf->total_hw_time,NSEC_PER_MSEC ); +if (duration >0 ) { + fps =DIV64_U64_ROUND_CLOSEST (tmp, duration); +seq_printf (s," hw: %lld" , fps); + } +seq_putc (s,'\n' ); + + ms[0 ] =div_s64_rem (perf->latency_first,NSEC_PER_MSEC , &ns[0 ]); + ms[1 ] =div_s64_rem (perf->latency_max,NSEC_PER_MSEC , &ns[1 ]); + ms[2 ] =div_s64_rem (ktime_sub (perf->ts_first, perf->ts_start),NSEC_PER_MSEC , &ns[2 ]); +seq_printf (s,"latency(ms) first: %lld.%06d, max %lld.%06d, setup %lld.%06d\n" , + ms[0 ], ns[0 ], ms[1 ], ns[1 ], ms[2 ], ns[2 ]); + + ms[0 ] =div_s64_rem (perf->min_process_time,NSEC_PER_MSEC , &ns[0 ]); + ms[1 ] =div_s64_rem (perf->max_process_time,NSEC_PER_MSEC , &ns[1 ]); +seq_printf (s,"process frame time(ms) min: %lld.%06d, max %lld.%06d\n" , + ms[0 ], ns[0 ], ms[1 ], ns[1 ]); +if (inst->type ==VPU_INST_TYPE_DEC ) {seq_printf (s,"%s order\n" , inst->disp_mode ==DISP_MODE_DISP_ORDER ?"display" :"decode" );drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c +15
-0@@ -181,6 +181,7 @@ static void wave6_handle_bitstream_buffer(struct vpu_instance *inst,return ; } + vpu_buf->ts_start =ktime_get_raw (); vpu_buf->consumed = true;wave6_vpu_dec_set_rd_ptr (inst, rd_ptr, true);@@ -304,6 +305,8 @@ static int wave6_vpu_dec_prepare_fb(struct vpu_instance *inst)wave6_vpu_set_instance_state (inst,VPU_INST_STATE_PIC_RUN ); +wave6_vpu_record_performance_timestamps (inst); +return 0 ; error:@@ -606,6 +609,10 @@ static void wave6_handle_decoded_frame(struct vpu_instance *inst, dst_vpu_buf->used = true;if (state ==VB2_BUF_STATE_ERROR ) dst_vpu_buf->error = true; + dst_vpu_buf->ts_input = vpu_buf->ts_input; + dst_vpu_buf->ts_start = vpu_buf->ts_start; + dst_vpu_buf->ts_finish =ktime_get_raw (); + dst_vpu_buf->hw_time =wave6_vpu_cycle_to_ns (inst->dev, info->cycle.frame_cycle); }if (state ==VB2_BUF_STATE_ERROR ) {@@ -658,6 +665,9 @@ static void wave6_handle_display_frame(struct vpu_instance *inst,vb2_set_plane_payload (&dst_buf->vb2_buf, i, inst->dst_fmt.plane_fmt[i].sizeimage); + vpu_buf->ts_output =ktime_get_raw (); +wave6_vpu_handle_performance (inst, vpu_buf); +if (vpu_buf->error) state =VB2_BUF_STATE_ERROR ; dst_buf->sequence = inst->sequence++;@@ -1404,6 +1414,8 @@ static int wave6_vpu_dec_create_instance(struct vpu_instance *inst)memset (&open_param,0 ,sizeof (struct dec_open_param )); +wave6_vpu_record_performance_timestamps (inst); +wave6_vpu_core_activate (inst->dev); ret =pm_runtime_resume_and_get (inst->dev->dev);if (ret) {@@ -1567,6 +1579,7 @@ static int wave6_vpu_dec_prepare_decode(struct vpu_instance *inst)static void wave6_vpu_dec_buf_queue_src (struct vb2_buffer *vb) {struct vb2_v4l2_buffer *vbuf =to_vb2_v4l2_buffer (vb); +struct vpu_buffer *vpu_buf =wave6_to_vpu_buf (vbuf);struct vpu_instance *inst =vb2_get_drv_priv (vb->vb2_queue); dev_dbg(inst->dev->dev, "type %4d index %4d size[0] %4ld size[1] : %4ld | size[2] : %4ld\n",@@ -1574,6 +1587,7 @@ static void wave6_vpu_dec_buf_queue_src(struct vb2_buffer *vb) vb2_plane_size(&vbuf->vb2_buf, 1), vb2_plane_size(&vbuf->vb2_buf, 2)); vbuf->sequence = inst->queued_src_buf_num++; + vpu_buf->ts_input =ktime_get_raw ();v4l2_m2m_buf_queue (inst->v4l2_fh.m2m_ctx, vbuf); }@@ -1670,6 +1684,7 @@ static void wave6_vpu_dec_stop_streaming(struct vb2_queue *q) goto exit;if (V4L2_TYPE_IS_OUTPUT (q->type)) { +wave6_vpu_reset_performance (inst); inst->queued_src_buf_num =0 ; inst->processed_buf_num =0 ; inst->error_buf_num =0 ;drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c +13
-0@@ -791,6 +791,7 @@ static int wave6_vpu_enc_start_encode(struct vpu_instance *inst) src_vbuf =wave6_to_vpu_buf (src_buf); src_vbuf->consumed = true; + src_vbuf->ts_start =ktime_get_raw ();if (inst->src_fmt.num_planes ==1 ) { frame_buf.buf_y =wave6_get_dma_addr (src_buf,0 ); frame_buf.buf_cb = frame_buf.buf_y + luma_size;@@ -887,6 +888,12 @@ static void wave6_handle_encoded_frame(struct vpu_instance *inst, dst_vpu_buf =wave6_to_vpu_buf (dst_buf); dst_vpu_buf->average_qp = info->avg_ctu_qp; + dst_vpu_buf->ts_input = vpu_buf->ts_input; + dst_vpu_buf->ts_start = vpu_buf->ts_start; + dst_vpu_buf->ts_finish =ktime_get_raw (); + dst_vpu_buf->hw_time =wave6_vpu_cycle_to_ns (inst->dev, info->cycle.frame_cycle); + dst_vpu_buf->ts_output =ktime_get_raw (); +wave6_vpu_handle_performance (inst, dst_vpu_buf);if (v4l2_m2m_is_last_draining_src_buf (inst->v4l2_fh.m2m_ctx, src_buf)) { dst_buf->flags |=V4L2_BUF_FLAG_LAST ;@@ -2023,6 +2030,8 @@ static int wave6_vpu_enc_create_instance(struct vpu_instance *inst)memset (&open_param,0 ,sizeof (struct enc_open_param )); +wave6_vpu_record_performance_timestamps (inst); +wave6_vpu_core_activate (inst->dev); ret =pm_runtime_resume_and_get (inst->dev->dev);if (ret) {@@ -2179,6 +2188,8 @@ static int wave6_vpu_enc_prepare_fb(struct vpu_instance *inst)wave6_vpu_set_instance_state (inst,VPU_INST_STATE_PIC_RUN ); +wave6_vpu_record_performance_timestamps (inst); +return 0 ; error:@@ -2267,6 +2278,7 @@ static void wave6_vpu_enc_buf_queue(struct vb2_buffer *vb)if (V4L2_TYPE_IS_OUTPUT (vb->type)) { vbuf->sequence = inst->queued_src_buf_num++; + vpu_buf->ts_input =ktime_get_raw (); vpu_buf->force_key_frame = inst->enc_ctrls.force_key_frame; inst->enc_ctrls.force_key_frame = false; vpu_buf->force_frame_qp = !inst->enc_ctrls.frame_rc_enable;@@ -2363,6 +2375,7 @@ static void wave6_vpu_enc_stop_streaming(struct vb2_queue *q) goto exit;if (V4L2_TYPE_IS_OUTPUT (q->type)) { +wave6_vpu_reset_performance (inst); inst->queued_src_buf_num =0 ; inst->processed_buf_num =0 ; inst->error_buf_num =0 ;drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c +91
-0@@ -5,6 +5,9 @@* Copyright (C) 2026 CHIPS&MEDIA INC */ +#include <linux/clk.h> +#include <linux/math64.h> +#include "wave6-vpu.h" #include "wave6-vpu-core.h" #include "wave6-trace.h" @@ -228,6 +231,20 @@ void wave6_vpu_set_instance_state(struct vpu_instance *inst, inst->state = state; } +u64 wave6_vpu_cycle_to_ns (struct vpu_core_device *core,u64 cycle) +{ +unsigned long rate; + +if (!core->num_clks || !core->clks) +return 0 ; + + rate =clk_get_rate (core->clks[0 ].clk); +if (!rate) +return 0 ; + +return div64_ul (cycle *NSEC_PER_SEC , rate); +} +int wave6_vpu_wait_interrupt (struct vpu_instance *inst,unsigned int timeout) {int ret;@@ -387,6 +404,80 @@ void wave6_vpu_finish_job(struct vpu_instance *inst)v4l2_m2m_job_finish (inst->dev->m2m_dev, inst->v4l2_fh.m2m_ctx); } +void wave6_vpu_record_performance_timestamps (struct vpu_instance *inst) +{ +if (WARN_ON (!inst)) +return ; + +if (inst->state ==VPU_INST_STATE_NONE && + !inst->performance.ts_start) + inst->performance.ts_start =ktime_get_raw (); + +if (inst->state ==VPU_INST_STATE_PIC_RUN && + !inst->performance.ts_first) + inst->performance.ts_first =ktime_get_raw (); +} + +void wave6_vpu_handle_performance (struct vpu_instance *inst,struct vpu_buffer *vpu_buf) +{ +s64 latency, time_spent; + +if (WARN_ON (!inst || !vpu_buf)) +return ; + + inst->performance.ts_last = vpu_buf->ts_output; + + latency = vpu_buf->ts_output - vpu_buf->ts_input; + time_spent = vpu_buf->ts_finish - vpu_buf->ts_start; + +if (!inst->performance.latency_first) + inst->performance.latency_first = latency; + inst->performance.latency_max =max_t (s64, latency, inst->performance.latency_max); + +if (!inst->performance.min_process_time) + inst->performance.min_process_time = time_spent; +else if (inst->performance.min_process_time > time_spent) + inst->performance.min_process_time = time_spent; + +if (inst->performance.max_process_time < time_spent) + inst->performance.max_process_time = time_spent; + + inst->performance.total_sw_time += time_spent; + inst->performance.total_hw_time += vpu_buf->hw_time; +} + +void wave6_vpu_reset_performance (struct vpu_instance *inst) +{ +if (WARN_ON (!inst)) +return ; + +if (inst->processed_buf_num) { +s64 tmp; +s64 fps_act =0 , fps_sw =0 , fps_hw =0 ; +s64 duration; +s64 latency_ms; +s32 latency_ns; +struct vpu_performance_info *perf = &inst->performance; + + tmp =MSEC_PER_SEC * inst->processed_buf_num; + duration =ktime_ms_delta (perf->ts_last, perf->ts_first); +if (duration >0 ) + fps_act =DIV64_U64_ROUND_CLOSEST (tmp, duration); + duration =div_u64 (perf->total_sw_time,NSEC_PER_MSEC ); +if (duration >0 ) + fps_sw =DIV64_U64_ROUND_CLOSEST (tmp, duration); + duration =div_u64 (perf->total_hw_time,NSEC_PER_MSEC ); +if (duration) + fps_hw =DIV64_U64_ROUND_CLOSEST (tmp, duration); + latency_ms =div_s64_rem (perf->latency_first,NSEC_PER_MSEC , &latency_ns); +dev_dbg (inst->dev->dev, +"[%d] fps actual: %lld, sw: %lld, hw: %lld, latency(ms) %lld.%06d\n" , + inst->id, fps_act, fps_sw, fps_hw, latency_ms, latency_ns); + } + +memset (&inst->performance,0 ,sizeof (inst->performance)); +} +static const struct v4l2_m2m_ops wave6_vpu_m2m_ops = { .device_run = wave6_vpu_device_run, .job_ready = wave6_vpu_job_ready,drivers/media/platform/chips-media/wave6/wave6-vpuapi.h +28
-1@@ -339,6 +339,18 @@ struct instance_buffer {dma_addr_t ar_base; }; +struct report_cycle { +u32 host_cmd_s; +u32 host_cmd_e; +u32 proc_s; +u32 proc_e; +u32 vpu_s; +u32 vpu_e; +u32 frame_cycle; +u32 proc_cycle; +u32 vpu_cycle; +}; +struct color_param {u32 color_range;u32 matrix_coefficients;@@ -423,6 +435,7 @@ struct dec_output_info {int error_reason;int warn_info;unsigned int sequence_no; +struct report_cycle cycle;dma_addr_t release_disp_frame_addr[WAVE6_MAX_FBS ];dma_addr_t disp_frame_addr[WAVE6_MAX_FBS ];u64 timestamp;@@ -671,6 +684,7 @@ struct enc_output_info {bool encoding_success;struct enc_report_fme_sum fme_sum;struct enc_report_mv_histo mv_histo; +struct report_cycle cycle;u64 timestamp;dma_addr_t src_y_addr;dma_addr_t custom_map_addr;@@ -839,6 +853,18 @@ struct vpu_instance_ops {void (*finish_process)(struct vpu_instance *inst,bool error); }; +struct vpu_performance_info { +ktime_t ts_start; +ktime_t ts_first; +ktime_t ts_last; +s64 latency_first; +s64 latency_max; +s64 min_process_time; +s64 max_process_time; +u64 total_sw_time; +u64 total_hw_time; +}; +/** * struct vpu_instance - VPU instance structure * @list: List head for VPU core device instance list @@ -883,6 +909,7 @@ struct vpu_instance_ops {* @error_recovery: Error recovery flag * @disp_mode: Display mode for decoder V4L2 control * @enc_ctrls: Encoder V4L2 control and streaming parameters +* @performance: Performance information * @debugfs: Debugfs entry */ struct vpu_instance {@@ -943,6 +970,7 @@ struct vpu_instance {enum display_mode disp_mode;struct enc_controls enc_ctrls; +struct vpu_performance_info performance;struct dentry *debugfs; }; --2.41.0.windows.3
Re: [PATCH v8 10/11] media: chips-media: wave6: Add Wave6 thermal cooling device
In reply to Nas ChungQuoted 19 lines · Nas Chung
Add a thermal cooling device for the Wave6 VPU, so the thermal framework can lower the VPU performance state when the SoC gets hot.
The control driver registers the cooling device on its device tree node. The cooling states are built from the OPP table of the "perf" power domain.
Tested-by Ming Qian <[email protected]>Tested-by Marek Vasut <[email protected]>Signed-off-by Nas Chung <[email protected]>--- .../media/platform/chips-media/wave6/Makefile | 1 + .../chips-media/wave6/wave6-vpu-thermal.c | 141 ++++++++++++++++++ .../chips-media/wave6/wave6-vpu-thermal.h | 24 +++ .../platform/chips-media/wave6/wave6-vpu.c | 22 +++ .../platform/chips-media/wave6/wave6-vpu.h | 3 + 5 files changed, 191 insertions(+) create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.c create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.h
...
drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.c line 123
Quoted 3 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.c new file +3
-0+ +if (!thermal->thermal_max) + goto error;
nothing clean up at error label, just return ret
drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.c line 134
Quoted 11 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.c new file +11
-0+ + thermal->thermal_event =0 ; + thermal->cooling =devm_thermal_of_child_cooling_device_register (dev, + dev->of_node, +dev_name (thermal->dev), + thermal, + &wave6_cooling_ops); +if (IS_ERR (thermal->cooling)) { +dev_err (thermal->dev,"register cooling device failed\n" ); + ret =PTR_ERR (thermal->cooling); + goto error;
return dev_err_probe()
Frank
Re: [PATCH v8 11/11] arm64: dts: freescale: imx95: Add video codec node
In reply to Nas ChungQuoted 6 lines · Nas Chung
Add the Chips and Media wave633 video codec node on IMX95 SoCs.
Signed-off-by Nas Chung <[email protected]>--- .../boot/dts/freescale/imx95-19x19-evk.dts | 10 +++++ arch/arm64/boot/dts/freescale/imx95.dtsi | 37 +++++++++++++++++++
Please split two patch, one patch touch soc dtsi, the another touch board.
Frank
Quoted 79 lines
2 files changed, 47 insertions(+)arch/arm64/boot/dts/freescale/imx95-19x19-evk.dts +10
-0@@ -143,6 +143,11 @@ vdev0buffer: vdev0buffer@88020000 { reg = <0 0x88020000 0 0x100000>; no-map; }; + + vpu_boot: memory@a0000000 { + reg = <0 0xa0000000 0 0x100000>; + no-map; + }; }; flexcan1_phy: can-phy0 {@@ -1225,3 +1230,8 @@ &tpm6 { pinctrl-0 = <&pinctrl_tpm6>; status = "okay"; }; + +&vpu { + memory-region = <&vpu_boot>; + status = "okay"; +};arch/arm64/boot/dts/freescale/imx95.dtsi +37
-1@@ -2139,6 +2139,43 @@ vpu_blk_ctrl: clock-controller@4c410000 { assigned-clock-rates = <133333333>, <667000000>, <500000000>; }; + vpu: video-codec@4c4c0000 { + compatible = "nxp,imx95-vpu"; + reg = <0x0 0x4c4c0000 0x0 0x10000>; + clocks = <&scmi_clk IMX95_CLK_VPU>, + <&vpu_blk_ctrl IMX95_CLK_VPUBLK_WAVE>; + clock-names = "core", "vpublk"; + power-domains = <&scmi_devpd IMX95_PD_VPU>, + <&scmi_perf IMX95_PERF_VPU>; + power-domain-names = "vpu", "perf"; + sram = <&sram1>; + #cooling-cells = <2>; + #address-cells = <2>; + #size-cells = <2>; + ranges; + status = "disabled"; + + interface@4c480000 { + reg = <0x0 0x4c480000 0x0 0x10000>; + interrupts = <GIC_SPI 299 IRQ_TYPE_LEVEL_HIGH>; + }; + + interface@4c490000 { + reg = <0x0 0x4c490000 0x0 0x10000>; + interrupts = <GIC_SPI 300 IRQ_TYPE_LEVEL_HIGH>; + }; + + interface@4c4a0000 { + reg = <0x0 0x4c4a0000 0x0 0x10000>; + interrupts = <GIC_SPI 301 IRQ_TYPE_LEVEL_HIGH>; + }; + + interface@4c4b0000 { + reg = <0x0 0x4c4b0000 0x0 0x10000>; + interrupts = <GIC_SPI 302 IRQ_TYPE_LEVEL_HIGH>; + }; + }; + jpegdec: jpegdec@4c500000 { compatible = "nxp,imx95-jpgdec", "nxp,imx8qxp-jpgdec"; reg = <0x0 0x4C500000 0x0 0x00050000>; --2.41.0.windows.3
RE: [PATCH v8 11/11] arm64: dts: freescale: imx95: Add video codec node
Hi, Frank.
Quoted 16 lines
-----Original Message-----
From: Frank Li <[email protected]>
Sent: Friday, November 20, 2026 6:05 AM
To: Nas Chung <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; jackson.lee <[email protected]>; lafley.kim <[email protected]>
Subject: Re: [PATCH v8 11/11] arm64: dts: freescale: imx95: Add video codec nodeOn Thu, Nov 19, 2026 at 06:00:05PM +0900, Nas Chung wrote:
Quoted 8 lines
Quoted 6 lines
Add the Chips and Media wave633 video codec node on IMX95 SoCs.
Signed-off-by Nas Chung <[email protected]>--- .../boot/dts/freescale/imx95-19x19-evk.dts | 10 +++++ arch/arm64/boot/dts/freescale/imx95.dtsi | 37 +++++++++++++++++++Please split two patch, one patch touch soc dtsi, the another touch board.
OK.
Thanks.
Nas.
Quoted 85 lines
Frank
Review
Quoted 3 lines
2 files changed, 47 insertions(+)(quoted diff) +0
-0
b/arch/arm64/boot/dts/freescale/imx95-19x19-evk.dtsReview
Quoted 25 lines
index
47ed0ef67611..a15d5b471708100644arch/arm64/boot/dts/freescale/imx95-19x19-evk.dts +10
-0@@ -143,6 +143,11 @@ vdev0buffer: vdev0buffer@88020000 { reg = <0 0x88020000 0 0x100000>; no-map; }; + + vpu_boot: memory@a0000000 { + reg = <0 0xa0000000 0 0x100000>; + no-map; + }; }; flexcan1_phy: can-phy0 {@@ -1225,3 +1230,8 @@ &tpm6 { pinctrl-0 = <&pinctrl_tpm6>; status = "okay"; }; + +&vpu { + memory-region = <&vpu_boot>; + status = "okay"; +};(quoted diff) +0
-0
b/arch/arm64/boot/dts/freescale/imx95.dtsiarch/arm64/boot/dts/freescale/imx95.dtsi vpu_blk_ctrl: clock-controller@4c410000 { · line 2139
Quoted 5 lines
index
d20c2498b9aa..b6323e527ab7100644arch/arm64/boot/dts/freescale/imx95.dtsi +0
-0@@ -2139,6 +2139,43 @@ vpu_blk_ctrl: clock-controller@4c410000 { assigned-clock-rates = <133333333>, <667000000>,<500000000>;
Quoted 46 lines
arch/arm64/boot/dts/freescale/imx95.dtsi +37
-1}; + vpu: video-codec@4c4c0000 { + compatible = "nxp,imx95-vpu"; + reg = <0x0 0x4c4c0000 0x0 0x10000>; + clocks = <&scmi_clk IMX95_CLK_VPU>, + <&vpu_blk_ctrl IMX95_CLK_VPUBLK_WAVE>; + clock-names = "core", "vpublk"; + power-domains = <&scmi_devpd IMX95_PD_VPU>, + <&scmi_perf IMX95_PERF_VPU>; + power-domain-names = "vpu", "perf"; + sram = <&sram1>; + #cooling-cells = <2>; + #address-cells = <2>; + #size-cells = <2>; + ranges; + status = "disabled"; + + interface@4c480000 { + reg = <0x0 0x4c480000 0x0 0x10000>; + interrupts = <GIC_SPI 299 IRQ_TYPE_LEVEL_HIGH>; + }; + + interface@4c490000 { + reg = <0x0 0x4c490000 0x0 0x10000>; + interrupts = <GIC_SPI 300 IRQ_TYPE_LEVEL_HIGH>; + }; + + interface@4c4a0000 { + reg = <0x0 0x4c4a0000 0x0 0x10000>; + interrupts = <GIC_SPI 301 IRQ_TYPE_LEVEL_HIGH>; + }; + + interface@4c4b0000 { + reg = <0x0 0x4c4b0000 0x0 0x10000>; + interrupts = <GIC_SPI 302 IRQ_TYPE_LEVEL_HIGH>; + }; + }; + jpegdec: jpegdec@4c500000 { compatible = "nxp,imx95-jpgdec", "nxp,imx8qxp-jpgdec"; reg = <0x0 0x4C500000 0x0 0x00050000>; --2.41.0.windows.3
RE: [PATCH v8 10/11] media: chips-media: wave6: Add Wave6 thermal cooling device
In reply to Frank LiHi, Frank.
Quoted 35 lines
-----Original Message-----
From: Frank Li <[email protected]>
Sent: Friday, November 20, 2026 6:04 AM
To: Nas Chung <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; jackson.lee <[email protected]>; lafley.kim <[email protected]>
Subject: Re: [PATCH v8 10/11] media: chips-media: wave6: Add Wave6 thermal cooling deviceQuoted 10 lines · Nas Chung
Add a thermal cooling device for the Wave6 VPU, so the thermal framework can lower the VPU performance state when the SoC gets hot.
The control driver registers the cooling device on its device tree node. The cooling states are built from the OPP table of the "perf" power domain.
Tested-by Ming Qian <[email protected]>Tested-by Marek Vasut <[email protected]>Signed-off-by Nas Chung <[email protected]>Quoted 8 lines
--- .../media/platform/chips-media/wave6/Makefile | 1 + .../chips-media/wave6/wave6-vpu-thermal.c | 141 ++++++++++++++++++ .../chips-media/wave6/wave6-vpu-thermal.h | 24 +++ .../platform/chips-media/wave6/wave6-vpu.c | 22 +++ .../platform/chips-media/wave6/wave6-vpu.h | 3 + 5 files changed, 191 insertions(+) create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-
thermal.c
Quoted 8 lines
Quoted 1 line
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-
thermal.h
...drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.c line 123
Quoted 3 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.c new file +3
-0+ +if (!thermal->thermal_max) + goto error;nothing clean up at error label, just return ret
Agreed.
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.c line 136
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.c new file +5
-0+ + thermal->thermal_event =0 ; + thermal->cooling = devm_thermal_of_child_cooling_device_register(dev, + dev->of_node, +
dev_name(thermal->dev),Review
Quoted 2 lines
(quoted diff) +2
-0+ thermal, +
&wave6_cooling_ops);
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.c line 134
Quoted 4 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.c new file +4
-0+if (IS_ERR (thermal->cooling)) { +dev_err (thermal->dev,"register cooling device failed\n" ); + ret =PTR_ERR (thermal->cooling); + goto error;return
dev_err_probe()
I'll address this in v9.
Thanks.
Nas.
Quoted 2 lines
Frank
RE: [PATCH v8 04/11] media: chips-media: wave6: Add Wave6 decoder firmware interface
In reply to Frank LiHi, Frank.
Quoted 79 lines
-----Original Message-----
From: Frank Li <[email protected]>
Sent: Friday, November 20, 2026 5:38 AM
To: Nas Chung <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; jackson.lee <[email protected]>; lafley.kim <[email protected]>
Subject: Re: [PATCH v8 04/11] media: chips-media: wave6: Add Wave6 decoder firmware interfaceQuoted 21 lines · Nas Chung
Add an interface layer to manage hardware register configuration and communication with the Chips&Media Wave6 video codec IP for video decoding.
The interface provides low-level helper functions used by the Wave6 core driver to implement video decoding operations. It handles command submission to the firmware via MMIO registers, and waits for a response by polling the firmware busy flag.
The decoder commands cover instance creation, flush and destruction, sequence initialization, frame buffer and display buffer registration, and picture decoding.
Sequence initialization and picture decoding complete asynchronously:
the interface interrupt reports completion and a query command reads back the result.Tested-by Ming Qian <[email protected]>Tested-by Marek Vasut <[email protected]>Signed-off-by Nas Chung <[email protected]>---
...
drivers/media/platform/chips-media/wave6/wave6-hw.c int wave6_vpu_get_version(struct vpu_core_device *core)
Quoted 11 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c +11
-0+int wave6_vpu_dec_issue_seq_init (struct vpu_instance *inst) +{ +struct dec_info *p_dec_info; +u32 reg_val; +int ret; + +guard (mutex)(&inst->dev->hw_lock); + + p_dec_info = &inst->codec_info->dec_info; + + vpu_write_reg_relaxed(inst->dev, W6_CMD_DEC_INIT_SEQ_BS_RD_PTR,
p_dec_info->stream_rd_ptr);Review
Quoted 1 line
(quoted diff) +1
-0+ vpu_write_reg_relaxed(inst->dev, W6_CMD_DEC_INIT_SEQ_BS_WR_PTR,
p_dec_info->stream_wr_ptr);drivers/media/platform/chips-media/wave6/wave6-hw.c int wave6_vpu_get_version(struct vpu_core_device *core)
Quoted 4 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c +4
-0+ + reg_val =DEC_PIC_BS_OPTION_STREAM_END (p_dec_info->stream_end) | +DEC_PIC_BS_OPTION_EXPLICIT_END_ON ; + vpu_write_reg_relaxed(inst->dev, W6_CMD_DEC_INIT_SEQ_BS_OPTION,
reg_val);Review
Quoted 2 lines
(quoted diff) +2
-0+ reg_val = INIT_SEQ_OPTION_MODE(W6_INIT_SEQ_OPT_NORMAL); + vpu_write_reg_relaxed(inst->dev, W6_CMD_DEC_INIT_SEQ_OPTION,
reg_val);drivers/media/platform/chips-media/wave6/wave6-hw.c int wave6_vpu_get_version(struct vpu_core_device *core)
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c +10
-0+ +wave6_send_command (inst->dev, inst->id, inst->std,W6_CMD_INIT_SEQ ); + ret =wave6_wait_vpu_busy (inst->dev,W6_VPU_BUSY_STATUS ); +if (ret) { +dev_err (inst->dev->dev,"%s: timeout\n" , __func__); +return ret; + } + +if (!vpu_read_reg_relaxed (inst->dev,W6_RET_SUCCESS )) { + u32 reason_code = vpu_read_reg_relaxed(inst->dev,
W6_RET_FAIL_REASON);drivers/media/platform/chips-media/wave6/wave6-hw.c int wave6_vpu_get_version(struct vpu_core_device *core) · line 333
Quoted 4 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c +4
-0+ +wave6_print_reg_err (inst->dev, reason_code); +return -EIO ; + }I see many place use the same pattern. Can you move to help function to macro like
Quoted 5 lines
#define wave6_send_command_sync(...) \ wave6_send_command() ... wave6_wait_vpu_busy() ... ...
Right. I will move this common pattern into a single helper function in v9.
Thanks.
Nas.
Quoted 9 lines
Frank
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c +3
-0+ +return 0 ; +}2.41.0.windows.3
RE: [PATCH v8 03/11] media: chips-media: wave6: Add Wave6 core driver
In reply to Frank LiHi, Frank.
Quoted 57 lines
-----Original Message-----
From: Frank Li <[email protected]>
Sent: Friday, November 20, 2026 5:28 AM
To: Nas Chung <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; jackson.lee <[email protected]>; lafley.kim <[email protected]>
Subject: Re: [PATCH v8 03/11] media: chips-media: wave6: Add Wave6 core driverQuoted 24 lines · Nas Chung
Add the core driver for the Chips&Media Wave6 video codec IP.
The hardware contains one control register region and four interface register regions for a shared video processing engine. The core driver handles the interface register regions, each with its own MMIO range and interrupt, while relying on the control driver for firmware loading and shared resource management.
It configures the V4L2 mem2mem devices and communicates with the Wave6 hardware to perform video processing tasks.
Tested-by Ming Qian <[email protected]>Tested-by Marek Vasut <[email protected]>Signed-off-by Nas Chung <[email protected]>--- .../media/platform/chips-media/wave6/Kconfig | 2 + .../media/platform/chips-media/wave6/Makefile | 5 +- .../platform/chips-media/wave6/wave6-hw.c | 181 ++++++++ .../platform/chips-media/wave6/wave6-hw.h | 22 + .../chips-media/wave6/wave6-regdefine.h | 65 +++ .../platform/chips-media/wave6/wave6-vdi.h | 20 + .../chips-media/wave6/wave6-vpu-core.c | 392 ++++++++++++++++++ .../chips-media/wave6/wave6-vpu-core.h | 106 +++++ .../chips-media/wave6/wave6-vpu-v4l2.c | 384 +++++++++++++++++Quoted 8 lines
.../platform/chips-media/wave6/wave6-vpu.c | 24 +- .../platform/chips-media/wave6/wave6-vpu.h | 2 + .../platform/chips-media/wave6/wave6-vpuapi.h | 381 +++++++++++++++++ .../chips-media/wave6/wave6-vpuconfig.h | 6 + 13 files changed, 1588 insertions(+), 2 deletions(-) create mode 100644 drivers/media/platform/chips-media/wave6/wave6-hw.c create mode 100644 drivers/media/platform/chips-media/wave6/wave6-hw.h create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-
core.cQuoted 1 line
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-
core.hQuoted 1 line
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-
v4l2.cQuoted 1 line
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-
vpuapi.hQuoted 1 line
...
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c line 28
Quoted 8 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c new file +8
-0+ +u32 wave6_vpu_readl (struct vpu_core_device *core,u32 addr) +{ +u32 data; + + data =wave6_vdi_readl (core->reg_base, addr); + +return data;return
wave6_vdi_readl(core->reg_base, addr);
OK.
Quoted 5 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c
Quoted 4 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c new file +3
-0+} + +void wave6_vpu_writel_relaxed(struct vpu_core_device *core, u32 addr,u32 data)
Quoted 11 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c line 40
Quoted 9 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c new file +9
-0+{ +wave6_vdi_writel_relaxed (core->reg_base, addr, data); +} + +u32 wave6_vpu_readl_relaxed (struct vpu_core_device *core,u32 addr) +{ +u32 data; + + data =wave6_vdi_readl_relaxed (core->reg_base, addr);return
wave6_vdi_readl_relaxed(core->reg_base, addr);
OK.
Quoted 18 lines
Quoted 5 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c new file +4
-0+ +return data; +} +...
Quoted 12 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c new file +12
-0+static u32 wave6_vpu_get_product_id (struct vpu_core_device *core) +{ +u32 val; + +lockdep_assert_held (&core->hw_lock); + + val =vpu_read_reg_relaxed (core,W6_VPU_RET_PRODUCT_CODE ); + +switch (val) { +case WAVE617_CODE : +return PRODUCT_ID_617 ; +case WAVE627_CODE :
Quoted 8 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c line 99
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c new file +6
-0+return PRODUCT_ID_627 ; +case WAVE633_CODE : +case WAVE637_CODE : +case WAVE663_CODE : +case WAVE677_CODE : +return PRODUCT_ID_637 ;what is use for? why not direct use WAVE*_CODE?
It is unused. I'll remove enum product_id.
Quoted 14 lines
Quoted 9 lines
drivers/media/platform/chips-media/wave6/wave6-hw.c new file +8
-0+default : +dev_err (core->dev,"Invalid product (%x)\n" , val); +break ; + } + +return PRODUCT_ID_NONE ; +} +...
Review
Quoted 3 lines
drivers/media/platform/chips-media/wave6/wave6-hw.h new file +2
-0+ +#endif /* __WAVE6_HW_H__ */ (quoted diff) +0
-0
b/drivers/media/platform/chips-media/wave6/wave6-regdefine.h
Quoted 16 lines
Quoted 4 lines
index
e879a9743cb3..aa9b972c82d1100644drivers/media/platform/chips-media/wave6/wave6-regdefine.h enum wave6_command { · line 23
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-regdefine.h +7
-0@@ -14,14 +14,51 @@ enum wave6_command {W6_CMD_INIT_VPU =0x0001 ,W6_CMD_WAKEUP_VPU =0x0002 ,W6_CMD_SLEEP_VPU =0x0004 , +W6_CMD_CREATE_INSTANCE =0x0008 , +W6_CMD_FLUSH_INSTANCE =0x0010 , +W6_CMD_DESTROY_INSTANCE =0x0020 , +W6_CMD_INIT_SEQ =0x0040 , +W6_CMD_SET_FB =0x0080 , +W6_CMD_DEC_PIC =0x0100 , + W6_CMD_DEC_SET_DISP = 0x0400,
BIT()?
OK.
Quoted 17 lines
Quoted 5 lines
drivers/media/platform/chips-media/wave6/wave6-regdefine.h +3
-0W6_CMD_INIT_WORK_BUF =0x1000 , + W6_CMD_QUERY = 0x4000, +}; +...
Review
Quoted 3 lines
#endif /*
__WAVE6_REGDEFINE_H__*/(quoted diff) +0
-0
b/drivers/media/platform/chips-media/wave6/wave6-vdi.hQuoted 7 lines
index
69ebf0ebcb13..67704fdbe27d100644drivers/media/platform/chips-media/wave6/wave6-vdi.h +0
-0@@ -13,6 +13,26 @@#include <linux/dma-mapping.h> #include <linux/io.h>
Quoted 3 lines
drivers/media/platform/chips-media/wave6/wave6-vdi.h line 16
Quoted 1 line
drivers/media/platform/chips-media/wave6/wave6-vdi.h +1
-0+enum endian_mode {name is too general and easy happen name collision.
I'll rename it to enum wave6_endian_mode.
Quoted 37 lines
Review
Quoted 24 lines
drivers/media/platform/chips-media/wave6/wave6-vdi.h +19
-0+VDI_128BIT_BIG_ENDIAN =0 , +VDI_128BIT_BE_BYTE_SWAP , +VDI_128BIT_BE_WORD_SWAP , +VDI_128BIT_BE_WORD_BYTE_SWAP , +VDI_128BIT_BE_DWORD_SWAP , +VDI_128BIT_BE_DWORD_BYTE_SWAP , +VDI_128BIT_BE_DWORD_WORD_SWAP , +VDI_128BIT_BE_DWORD_WORD_BYTE_SWAP , +VDI_128BIT_LE_DWORD_WORD_BYTE_SWAP , +VDI_128BIT_LE_DWORD_WORD_SWAP , +VDI_128BIT_LE_DWORD_BYTE_SWAP , +VDI_128BIT_LE_DWORD_SWAP , +VDI_128BIT_LE_WORD_BYTE_SWAP , +VDI_128BIT_LE_WORD_SWAP , +VDI_128BIT_LE_BYTE_SWAP , +VDI_128BIT_LITTLE_ENDIAN =15 , +VDI_ENDIAN_MAX +}; +/** * struct vpu_buf - VPU buffer for a coherent DMA buffer * @size: Buffer size (quoted diff) +0
-0
b/drivers/media/platform/chips-media/wave6/wave6-vpu-core.cQuoted 12 lines
new file mode 100644
index 000000000000..67c6666579d9drivers/media/platform/chips-media/wave6/wave6-vpu-core.c new file +7
-0@@ -0,0 +1,392 @@ +// SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) +/* +* Wave6 series multi-standard codec IP - wave6 core driver +* +* Copyright (C) 2026 CHIPS&MEDIA INC +*/ +
Quoted 8 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c line 13
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c new file +6
-0+#include <linux/kernel.h> +#include <linux/platform_device.h> +#include <linux/clk.h> +#include <linux/interrupt.h> +#include <linux/pm_runtime.h> +#include <linux/iopoll.h> sort
OK.
Quoted 21 lines
Quoted 9 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c new file +8
-0+ +#include "wave6-vpu-core.h" +#include "wave6-regdefine.h" +#include "wave6-vpuconfig.h" +#include "wave6-hw.h" + +static irqreturn_t wave6_vpu_core_irq (int irq,void *dev_id) +{drivers/media/platform/chips-media/wave6/wave6-vpu-core.c
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c new file +10
-0+struct vpu_core_device *core = dev_id; +struct vpu_irq irq_info; + +if (!vpu_read_reg (core,W6_VPU_VPU_INT_STS )) +return IRQ_NONE ; + + irq_info.status =vpu_read_reg (core,W6_VPU_VINT_REASON ); + irq_info.inst_idc =vpu_read_reg (core,W6_RET_INT_INSTANCE_INFO ); + + vpu_write_reg(core, W6_RET_INT_INSTANCE_INFO,
INT_INSTANCE_INFO_CLEAR);
Quoted 1 line
suppose only already set irq, irq_info.inst_idc, like below code?
This is a plain register, not a write-1-to-clear one. The firmware does not write the next instance bit until the driver clears W6_VPU_VINT_REASON, and the driver clears this register first. So writing 0 does not lose any bit.
Quoted 22 lines
Quoted 19 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c new file +18
-0+vpu_write_reg (core,W6_VPU_VINT_REASON_CLEAR , irq_info.status); +vpu_write_reg (core,W6_VPU_VINT_CLEAR ,VINT_CLEAR ); + +if (irq_info.status &BIT (W6_INT_BIT_REQ_WORK_BUF )) { +wave6_vpu_require_work_buffer (core->vpu, core->reg_base); + +return IRQ_HANDLED ; + } + +if (kfifo_in (&core->irq_fifo, &irq_info,sizeof (struct vpu_irq )) != +sizeof (struct vpu_irq )) +dev_err (core->dev,"IRQ overflow status 0x%x idc 0x%x\n" , + irq_info.status, irq_info.inst_idc); + +return IRQ_WAKE_THREAD ; +} + + core->irq =platform_get_irq (pdev,0 );...
Quoted 12 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c line 245
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c new file +10
-0+if (core->irq <0 ) +return core->irq; + + ret =devm_request_threaded_irq (&pdev->dev, core->irq, + wave6_vpu_core_irq, + wave6_vpu_core_irq_thread, +0 ,"vpu_irq" , core); +if (ret) { +dev_err (&pdev->dev,"failed to request IRQ: %d\n" , ret); +return ret;return
dev_err_probe();
OK.
Quoted 26 lines
Quoted 9 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c new file +8
-0+ } + + ret =v4l2_device_register (&pdev->dev, &core->v4l2_dev); +if (ret) { +dev_err (&pdev->dev,"failed to register v4l2_dev: %d\n" , ret); +return ret; + } +...
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c
Quoted 2 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c new file +2
-0+void wave6_vpu_writel (struct vpu_core_device *core,u32 addr,u32 data); +void wave6_vpu_writel_relaxed(struct vpu_core_device *core, u32 addr,u32 data);
Review
Quoted 5 lines
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c new file +4
-0+u32 wave6_vpu_readl (struct vpu_core_device *core,u32 addr); +u32 wave6_vpu_readl_relaxed (struct vpu_core_device *core,u32 addr); + +#endif /* __WAVE6_VPUAPI_H__ */ (quoted diff) +0
-0
b/drivers/media/platform/chips-media/wave6/wave6-vpuconfig.hQuoted 7 lines
index
1d4e016c2a58..fccb854a8a8c100644drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h +0
-0@@ -18,13 +18,19 @@#define W6_VPU_MAX_NUM_CORE 4#define MAX_NUM_INSTANCE 32
Quoted 9 lines
drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h line 27
Quoted 7 lines
drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h +5
-0+#define W6_MAX_PIC_STRIDE (4096 * 4) +#define W6_PIC_STRIDE_ALIGNMENT 32 +#define W6_VPU_POLL_DELAY_US 10#define W6_VPU_POLL_TIMEOUT 300000 +#define W6_BOOT_WAIT_TIMEOUT 10000 +#define W6_VPU_TIMEOUT 6000Need unit
I'll add _MS/_US suffixes.
Thanks.
Nas.
Quoted 15 lines
Frank
Quoted 13 lines
drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h +1
-1#define W6_MAX_REMAP_PAGE_SIZE (1024 * 1024)#define W6_MAX_CODE_BUF_SIZE (4 * 1024 * 1024)#define W6_CODE_BUF_SIZE (1 * 1024 * 1024)#define W6_EXTRA_CODE_BUF_SIZE (256 * 1024) +#define W6_TEMPBUF_SIZE (3 * 1024 * 1024)#define W637DEC_WORKBUF_SIZE_FOR_CQ (3 * 512 * 1024)static inline bool wave6_is_product_w_series (u32 code) --2.41.0.windows.3
RE: [PATCH v8 02/11] media: chips-media: wave6: Add Wave6 control driver
In reply to Frank LiHi, Frank.
Quoted 37 lines
-----Original Message-----
From: Frank Li <[email protected]>
Sent: Friday, November 20, 2026 5:08 AM
To: Nas Chung <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; jackson.lee <[email protected]>; lafley.kim <[email protected]>
Subject: Re: [PATCH v8 02/11] media: chips-media: wave6: Add Wave6 control driverQuoted 7 lines · Nas Chung
Add the control driver for the Chips&Media Wave6 video codec IP.
The hardware contains one control register region and four interface register regions for a shared video processing engine.
This driver handles the control region and manages shared resources such as firmware loading, firmware memory allocation, and coordination
required
Quoted 9 lines
by the interface register regions.
It also instantiates and coordinates with the
wave6-corechild devices for firmware and power state management.Tested-by Ming Qian <[email protected]>Tested-by Marek Vasut <[email protected]>Signed-off-by Nas Chung <[email protected]>---
...
Review
Quoted 2 lines
(quoted diff) +2
-0+ +static inline int wave6_vdi_alloc_dma(struct device *dev, struct vpu_buf*vb)
Quoted 8 lines
drivers/media/platform/chips-media/wave6/wave6-vdi.h line 56
Quoted 3 lines
drivers/media/platform/chips-media/wave6/wave6-vdi.h new file +3
-0+{ +void *vaddr; +dma_addr_t daddr;Nit try keep revise christmas order, check others
dma_addr_tdaddr;
void *vaddr;
OK, I'll fix it across the driver.
Quoted 27 lines
Quoted 16 lines
drivers/media/platform/chips-media/wave6/wave6-vdi.h new file +15
-0+ +if (!vb || !vb->size) +return -EINVAL ; + + vaddr =dma_alloc_coherent (dev, vb->size, &daddr,GFP_KERNEL ); +if (!vaddr) +return -ENOMEM ; + + vb->vaddr = vaddr; + vb->daddr = daddr; + vb->dev = dev; + +return 0 ; +} +...
Quoted 10 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +10
-0+ +#include <linux/kernel.h> +#include <linux/module.h> +#include <linux/platform_device.h> +#include <linux/clk.h> +#include <linux/of.h> +#include <linux/of_address.h> +#include <linux/property.h> +#include <linux/of_irq.h> +#include <linux/firmware.h>
Quoted 8 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c line 22
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +6
-0+#include <linux/interrupt.h> +#include <linux/pm_runtime.h> +#include <linux/pm_domain.h> +#include <linux/dma-mapping.h> +#include <linux/iopoll.h> +#include <linux/genalloc.h> order by alpabet.
OK.
Quoted 13 lines
Quoted 12 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +11
-0+ +#include "wave6-vpuconfig.h" +#include "wave6-regdefine.h" +#include "wave6-vpu.h" + +static const struct wave6_vpu_resource wave633c_data = { + .fw_name ="cnm/wave633c_imx9_codec_fw.bin" , +/* For HEVC, AVC, 4096x4096, 8bit */ + .sram_size =0x14800 , +}; +...
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c line 126
Quoted 4 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +4
-0+ +static u32 wave6_vpu_get_code_buf_size (struct wave6_vpu_device *vpu) +{ +return min_t (u32, vpu->code_buf.size,W6_MAX_CODE_BUF_SIZE );now use
min()
OK.
Quoted 20 lines
Quoted 3 lines
(quoted diff) +2
-0+} +...
drivers/media/platform/chips-media/wave6/wave6-vpu.c line 627
Quoted 13 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +13
-0+ +static int wave6_vpu_probe (struct platform_device *pdev) +{ +struct device_node *np; +struct wave6_vpu_device *vpu; +const struct wave6_vpu_resource *res; +int ret; + + ret =dma_set_mask_and_coherent (&pdev->dev,DMA_BIT_MASK (32 )); +if (ret <0 ) { +dev_err (&pdev->dev,"failed to set DMA mask: %d\n" , ret); +return ret; + }Needn't check when mask > 32, if check,
Quoted 1 line
return
dev_err_probe()
The mask is exactly 32 bits, so I'll keep the check and use dev_err_probe().
Quoted 14 lines
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +5
-0+ + res =device_get_match_data (&pdev->dev); +if (!res) +return -ENODEV ; +...
drivers/media/platform/chips-media/wave6/wave6-vpu.c line 701
Quoted 4 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +4
-0+ +wave6_vpu_allocate_work_buffers (vpu); + +pm_runtime_enable (&pdev->dev);
devm_pm_runtime_enable();
Quoted 3 lines
you have not enable clock here, so you need select PM at
Kconfig, otherwise if !CONFIG_PM, other register access will be hang because stub do nothing of runtime pm when !CONFIG_PM
OK. I'll add "depends on PM", as other drivers in drivers/media/platform do.
Quoted 15 lines
Quoted 15 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +14
-0+ +init_completion (&vpu->fw_done); + ret =firmware_request_nowait_nowarn (THIS_MODULE , + vpu->res->fw_name, + &pdev->dev, +GFP_KERNEL , + vpu, + wave6_vpu_load_firmware); +if (ret) { +dev_err (&pdev->dev,"request firmware fail, ret = %d\n" , ret); + goto error; + } + +return 0 ;
Quoted 8 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c line 721
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +6
-0+ +error: +wave6_vpu_release (vpu); +pm_runtime_disable (&pdev->dev); +error_detach_pm_domains: +wave6_vpu_detach_pm_domains (vpu);use
devm_add_action_or_reset()to avoid goto inprobe()
OK.
Quoted 31 lines
Quoted 5 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +4
-0+ +return ret; +} +...
Review
Quoted 7 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.c new file +6
-0+ +module_platform_driver (wave6_vpu_driver); +MODULE_DESCRIPTION ("chips&media Wave6 VPU driver" ); +MODULE_AUTHOR ("CHIPS&MEDIA INC" ); +MODULE_FIRMWARE ("cnm/wave633c_imx9_codec_fw.bin" ); +MODULE_LICENSE ("Dual BSD/GPL" );(quoted diff) +0
-0
b/drivers/media/platform/chips-media/wave6/wave6-vpu.hQuoted 17 lines
new file mode 100644
index 000000000000..96d67e931e71drivers/media/platform/chips-media/wave6/wave6-vpu.h new file +12
-0@@ -0,0 +1,137 @@ +/* SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) */ +/* +* Wave6 series multi-standard codec IP - wave6 driver +* +* Copyright (C) 2026 CHIPS&MEDIA INC +*/ + +#ifndef __WAVE6_VPU_H__ +#define __WAVE6_VPU_H__ + +#include <linux/completion.h> +#include <linux/device.h>
Quoted 8 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.h line 18
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.h new file +6
-0+ +#include "wave6-vpuconfig.h" +#include "wave6-vdi.h" + +#define WAVE6_VPU_PLATFORM_DRIVER_NAME "wave6-vpu" +#define WAVE6_VPU_CORE_PLATFORM_DRIVER_NAME "wave6-vpu-core"If only use once, needn't define macro.
I'll drop WAVE6_VPU_PLATFORM_DRIVER_NAME. The core name is used by both the child device and the core driver, so it stays.
Quoted 12 lines
Quoted 11 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.h new file +10
-0+ +struct wave6_vpu_device ; + +/** +* enum wave6_vpu_state - VPU states +* @WAVE6_VPU_STATE_OFF: VPU is powered off +* @WAVE6_VPU_STATE_PREPARE: VPU is booting +* @WAVE6_VPU_STATE_ON: VPU is running +* @WAVE6_VPU_STATE_SLEEP: VPU is in a sleep mode +*/ ...
Quoted 8 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.h
Quoted 6 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.h new file +6
-0+ +#define W6_VPU_MAX_NUM_CORE 4 +#define MAX_NUM_INSTANCE 32 + +#define W6_VPU_POLL_DELAY_US 10 +#define W6_VPU_POLL_TIMEOUT 300000Need unit surfix
OK.
Thanks.
Nas.
Quoted 22 lines
Frank
Quoted 19 lines
drivers/media/platform/chips-media/wave6/wave6-vpu.h new file +15
-1+ +#define W6_MAX_REMAP_PAGE_SIZE (1024 * 1024) +#define W6_MAX_CODE_BUF_SIZE (4 * 1024 * 1024) +#define W6_CODE_BUF_SIZE (1 * 1024 * 1024) +#define W6_EXTRA_CODE_BUF_SIZE (256 * 1024) +#define W637DEC_WORKBUF_SIZE_FOR_CQ (3 * 512 * 1024) + +static inline bool wave6_is_product_w_series (u32 code) +{ +return code ==WAVE617_CODE || code ==WAVE627_CODE || + code ==WAVE633_CODE || code ==WAVE637_CODE || + code ==WAVE663_CODE || code ==WAVE677_CODE ; +} + +#endif /* __WAVE6_VPUCONFIG_H__ */ --2.41.0.windows.3