Performance improvement of decoder
Subsystems (2)
Jackson.lee
maintainerv6 cover Performance improvement of decoder
From: Jackson Lee <[email protected]>
v4l2-compliance results:
========================
v4l2-compliance 1.28.1-5233, 64 bits, 64-bit time_t
Buffer ioctls:
warn: v4l2-test-buffers.cpp(693): VIDIOC_CREATE_BUFS not supported
warn: v4l2-test-buffers.cpp(693): VIDIOC_CREATE_BUFS not supported
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
test CREATE_BUFS maximum buffers: OK
test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)
Total for wave5-dec device /dev/video0: 46, Succeeded: 46, Failed: 0, Warnings: 2 Total for wave5-enc device /dev/video1: 46, Succeeded: 46, Failed: 0, Warnings: 0
Fluster test results:
=====================
Running test suite JCT-VC-HEVC_V1 with decoder GStreamer-H.265-V4L2-Gst1.0 Using 3 parallel job(s)
Ran 133/147 tests successfully in 37.043 secs
(1 test fails because of not supporting to parse multi frames, 1 test fails because of a missing frame and slight corruption,
2 tests fail because of sizes which are incompatible with the IP, 11 tests fail because of unsupported 10 bit format)
Running test suite JVT-AVC_V1 with decoder GStreamer-H.264-V4L2-Gst1.0 Using 3 parallel job(s)
Ran 78/135 tests successfully in 41.820 secs
(57 fail because the hardware is unable to decode MBAFF / FMO / Field / Extended profile streams.)
Running test suite JVT-FR-EXT with decoder GStreamer-H.264-V4L2-Gst1.0 Using 3 parallel job(s)
Ran 25/69 tests successfully in 12.485 secs
(44 fail because the hardware does not support field encoded and 422 encoded stream)
Seek test
=====================
1. gst-play-1.0 seek.264
2. this will use waylandsink since gst-play-1.0 uses playbin. if you don't want to hook up display,
you can run gst-play-1.0 seek.264 --videosink=fakevideosink instead
3. Let pipeline run for 2-3 seconds
4. press SPACE key to pause
5. press 0 to reset press SPACE to start again
gst-play-1.0 seek.264 --videosink=fakevideosink Press 'k' to see a list of keyboard shortcuts. Now playing /root/seek.264
Redistribute latency...
Redistribute latency...
Redistribute latency...
Redistribute latency...
Redistribute latency...aused
0:00:09.9 / 0:00:09.9
Reached end of play list.
Sequence Change test
=====================
gst-launch-1.0 filesrc location=./drc.h264 ! h264parse ! v4l2h264dec ! filesink location=./h264_output_420.yuv Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Redistribute latency...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
Got EOS from element "pipeline0".
Execution ended after 0:00:00.100759170
Setting pipeline to NULL ...
Freeing pipeline ...
root@j721s2-evm:~#
Change since v5:
================
* For [PATCH v4 4/4] media: chips-media: wave5: Improve performance of decoder
- reduce high cpu usage while playback adaptive streaming from streaming server
* For [PATCH v4 1/4] media: chips-media: wave5: Fix SError of kernel panic when closed
- fix kernel panic when printing a lot of log messages
Change since v4:
=================
* For [PATCH v5 4/4] media: chips-media: wave5: Improve performance of decoder
- fix the error which the Media CI rebot reported
* For [PATCH v5 2/4] media: chips-media: wave5: Fix Null reference while testing fluster
- fix the error which the Media CI rebot reported
Change since v3:
==================
* For [PATCH v4 4/4] media: chips-media: wave5: Improve performance of decoder
- fix crash and dead lock while testing seek
* For [PATCH v4 3/4] media: chips-media: wave5: Add WARN_ON to check if dec_output_info is NULL
- update commit message
* For [PATCH v4 2/4] media: chips-media: wave5: Fix Null reference while testing fluster
- add thread irq logic
* For [PATCH v4 1/4] media: chips-media: wave5: Fix SError of kernel panic when closed
- add Reviewed-by tag
Change since v2:
==================
* For [PATCH v3 4/4] media: chips-media: wave5: Improve performance of decoder
- squash v2's #3~#6 to #4 patch of v3
Change since v1:
===================
* For [PATCH v2 2/7] media: chips-media: wave5: Improve performance of decoder
- change log to dbg level
Change since v0:
===================
* For [PATCH v1 2/7] media: chips-media: wave5: Improve performance of decoder
- separates the previous patch to a few patches
* For [PATCH v1 3/7] media: chips-media: wave5: Fix not to be closed
- separated from the previous patch of performance improvement of decoder
* For [PATCH v1 4/7] media: chips-media: wave5: Use spinlock whenever state is changed
- separated from the previous patch of performance improvement of decoder
* For [PATCH v1 5/7] media: chips-media: wave5: Fix not to free resources normally when instance was destroyed
- separated from the previous patch of performance improvement of decoder
* For [PATCH v1 7/7] media: chips-media: wave5: Fix SError of kernel panic when closed
- separated from the previous patch of performance improvement of decoder
Jackson Lee (4):
media: chips-media: wave5: Fix SError of kernel panic when closed
media: chips-media: wave5: Fix Null reference while testing fluster
media: chips-media: wave5: Add WARN_ON to check if dec_output_info is
NULL
media: chips-media: wave5: Improve performance of decoder
.../platform/chips-media/wave5/wave5-helper.c | 28 ++- .../platform/chips-media/wave5/wave5-helper.h | 1 + .../platform/chips-media/wave5/wave5-hw.c | 2 +- .../chips-media/wave5/wave5-vpu-dec.c | 177 +++++++++++++----- .../chips-media/wave5/wave5-vpu-enc.c | 8 +- .../platform/chips-media/wave5/wave5-vpu.c | 98 ++++++++-- .../platform/chips-media/wave5/wave5-vpu.h | 2 +- .../platform/chips-media/wave5/wave5-vpuapi.c | 68 ++++--- .../platform/chips-media/wave5/wave5-vpuapi.h | 13 ++ .../chips-media/wave5/wave5-vpuconfig.h | 1 + 10 files changed, 307 insertions(+), 91 deletions(-)
Signature · 1 line
2.43.0
Jackson.lee
maintainerPATCH v6 1/4 media: chips-media: wave5: Fix SError of kernel panic when closed
From: Jackson Lee <[email protected]>
SError of kernel panic rarely happened while testing fluster. The root cause was to enter suspend mode because timeout of autosuspend delay happened.
[ 48.834439] SError Interrupt on CPU0, code 0x00000000bf000000 -- SError [ 48.834455] CPU: 0 UID: 0 PID: 1067 Comm: v4l2h265dec0:sr Not tainted 6.12.9-gc9e21a1ebd75-dirty #7 [ 48.834461] Hardware name: ti Texas Instruments J721S2 EVM/Texas Instruments J721S2 EVM, BIOS 2025.01-00345-gbaf3aaa8ecfa 01/01/2025 [ 48.834464] pstate: 20000005 (nzCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 48.834468] pc : wave5_dec_clr_disp_flag+0x40/0x80 [wave5] [ 48.834488] lr : wave5_dec_clr_disp_flag+0x40/0x80 [wave5] [ 48.834495] sp : ffff8000856e3a30 [ 48.834497] x29: ffff8000856e3a30 x28: ffff0008093f6010 x27: ffff000809158130 [ 48.834504] x26: 0000000000000000 x25: ffff00080b625000 x24: ffff000804a9ba80 [ 48.834509] x23: ffff000802343028 x22: ffff000809158150 x21: ffff000802218000 [ 48.834513] x20: ffff0008093f6000 x19: ffff0008093f6000 x18: 0000000000000000 [ 48.834518] x17: 0000000000000000 x16: 0000000000000000 x15: 0000ffff74009618 [ 48.834523] x14: 000000010000000c x13: 0000000000000000 x12: 0000000000000000 [ 48.834527] x11: ffffffffffffffff x10: ffffffffffffffff x9 : ffff000802343028 [ 48.834532] x8 : ffff00080b6252a0 x7 : 0000000000000038 x6 : 0000000000000000 [ 48.834536] x5 : ffff00080b625060 x4 : 0000000000000000 x3 : 0000000000000000 [ 48.834541] x2 : 0000000000000000 x1 : ffff800084bf0118 x0 : ffff800084bf0000 [ 48.834547] Kernel panic - not syncing: Asynchronous SError Interrupt [ 48.834549] CPU: 0 UID: 0 PID: 1067 Comm: v4l2h265dec0:sr Not tainted 6.12.9-gc9e21a1ebd75-dirty #7 [ 48.834554] Hardware name: ti Texas Instruments J721S2 EVM/Texas Instruments J721S2 EVM, BIOS 2025.01-00345-gbaf3aaa8ecfa 01/01/2025 [ 48.834556] Call trace: [ 48.834559] dump_backtrace+0x94/0xec [ 48.834574] show_stack+0x18/0x24 [ 48.834579] dump_stack_lvl+0x38/0x90 [ 48.834585] dump_stack+0x18/0x24 [ 48.834588] panic+0x35c/0x3e0 [ 48.834592] nmi_panic+0x40/0x8c [ 48.834595] arm64_serror_panic+0x64/0x70 [ 48.834598] do_serror+0x3c/0x78 [ 48.834601] el1h_64_error_handler+0x34/0x4c [ 48.834605] el1h_64_error+0x64/0x68 [ 48.834608] wave5_dec_clr_disp_flag+0x40/0x80 [wave5] [ 48.834615] wave5_vpu_dec_clr_disp_flag+0x54/0x80 [wave5] [ 48.834622] wave5_vpu_dec_buf_queue+0x19c/0x1a0 [wave5] [ 48.834628] __enqueue_in_driver+0x3c/0x74 [videobuf2_common] [ 48.834639] vb2_core_qbuf+0x508/0x61c [videobuf2_common] [ 48.834646] vb2_qbuf+0xa4/0x168 [videobuf2_v4l2] [ 48.834656] v4l2_m2m_qbuf+0x80/0x238 [v4l2_mem2mem] [ 48.834666] v4l2_m2m_ioctl_qbuf+0x18/0x24 [v4l2_mem2mem] [ 48.834673] v4l_qbuf+0x48/0x5c [videodev] [ 48.834704] __video_do_ioctl+0x180/0x3f0 [videodev] [ 48.834725] video_usercopy+0x2ec/0x68c [videodev] [ 48.834745] video_ioctl2+0x18/0x24 [videodev] [ 48.834766] v4l2_ioctl+0x40/0x60 [videodev] [ 48.834786] __arm64_sys_ioctl+0xa8/0xec [ 48.834793] invoke_syscall+0x44/0x100 [ 48.834800] el0_svc_common.constprop.0+0xc0/0xe0 [ 48.834804] do_el0_svc+0x1c/0x28 [ 48.834809] el0_svc+0x30/0xd0 [ 48.834813] el0t_64_sync_handler+0xc0/0xc4 [ 48.834816] el0t_64_sync+0x190/0x194 [ 48.834820] SMP: stopping secondary CPUs [ 48.834831] Kernel Offset: disabled [ 48.834833] CPU features: 0x08,00002002,80200000,4200421b [ 48.834837] Memory Limit: none [ 49.161404] ---[ end Kernel panic - not syncing: Asynchronous SError Interrupt ]---
--- .../platform/chips-media/wave5/wave5-vpu-dec.c | 5 ++--- .../platform/chips-media/wave5/wave5-vpu-enc.c | 3 --- .../media/platform/chips-media/wave5/wave5-vpu.c | 2 +- .../platform/chips-media/wave5/wave5-vpuapi.c | 15 --------------- 4 files changed, 3 insertions(+), 22 deletions(-)
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -3
@@ -1243,6 +1243,7 @@ static void wave5_vpu_dec_buf_queue_dst(struct vb2_buffer *vb)struct vpu_instance *inst =vb2_get_drv_priv (vb->vb2_queue);struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx; +pm_runtime_resume_and_get (inst->dev->dev); vbuf->sequence = inst->queued_dst_buf_num++;if (inst->state ==VPU_INST_STATE_PIC_RUN ) {
@@ -1275,6 +1276,7 @@ static void wave5_vpu_dec_buf_queue_dst(struct vb2_buffer *vb) }else {v4l2_m2m_buf_queue (m2m_ctx, vbuf); } +pm_runtime_put_autosuspend (inst->dev->dev); }static void wave5_vpu_dec_buf_queue (struct vb2_buffer *vb)
@@ -1825,9 +1827,6 @@ static int wave5_vpu_open_dec(struct file *filp)if (ret) goto cleanup_inst; -if (list_empty (&dev->instances)) -pm_runtime_use_autosuspend (inst->dev->dev); -list_add_tail (&inst->list, &dev->instances);mutex_unlock (&dev->dev_lock);
drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c +0 -3
@@ -1768,9 +1768,6 @@ static int wave5_vpu_open_enc(struct file *filp)if (ret) goto cleanup_inst; -if (list_empty (&dev->instances)) -pm_runtime_use_autosuspend (inst->dev->dev); -list_add_tail (&inst->list, &dev->instances);mutex_unlock (&dev->dev_lock);
drivers/media/platform/chips-media/wave5/wave5-vpu.c +1 -1
@@ -322,7 +322,7 @@ static int wave5_vpu_probe(struct platform_device *pdev)dev_info (&pdev->dev,"Product Code: 0x%x\n" , dev->product_code);dev_info (&pdev->dev,"Firmware Revision: %u\n" , fw_revision); -pm_runtime_set_autosuspend_delay (&pdev->dev,100 ); +pm_runtime_set_autosuspend_delay (&pdev->dev,500 );pm_runtime_use_autosuspend (&pdev->dev);pm_runtime_enable (&pdev->dev);wave5_vpu_sleep_wake (&pdev->dev, true,NULL ,0 );
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -15
@@ -207,8 +207,6 @@ int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res)int retry =0 ;struct vpu_device *vpu_dev = inst->dev;int i; -int inst_count =0 ; -struct vpu_instance *inst_elm; *fail_res =0 ;if (!inst->codec_info)
@@ -250,11 +248,6 @@ int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res)wave5_vdi_free_dma_memory (vpu_dev, &p_dec_info->vb_task); -list_for_each_entry (inst_elm, &vpu_dev->instances, list) - inst_count++; -if (inst_count ==1 ) -pm_runtime_dont_use_autosuspend (vpu_dev->dev); - unlock_and_return:mutex_unlock (&vpu_dev->hw_lock);pm_runtime_put_sync (inst->dev->dev);
@@ -720,8 +713,6 @@ int wave5_vpu_enc_close(struct vpu_instance *inst, u32 *fail_res)int ret;int retry =0 ;struct vpu_device *vpu_dev = inst->dev; -int inst_count =0 ; -struct vpu_instance *inst_elm; *fail_res =0 ;if (!inst->codec_info)
@@ -764,12 +755,6 @@ int wave5_vpu_enc_close(struct vpu_instance *inst, u32 *fail_res) }wave5_vdi_free_dma_memory (vpu_dev, &p_enc_info->vb_task); - -list_for_each_entry (inst_elm, &vpu_dev->instances, list) - inst_count++; -if (inst_count ==1 ) -pm_runtime_dont_use_autosuspend (vpu_dev->dev); -mutex_unlock (&vpu_dev->hw_lock);pm_runtime_put_sync (inst->dev->dev);
Signature · 1 line
2.43.0
Jackson.lee
maintainerPATCH v6 2/4 media: chips-media: wave5: Fix Null reference while testing fluster
In reply to Jackson.leeFrom: Jackson Lee <[email protected]>
When multi instances are created/destroyed, many interrupts happens and structures for decoder are removed.
"struct vpu_instance" this structure is shared for all flow in the decoder, so if the structure is not protected by lock, Null dereference could happens sometimes.
IRQ Handler was spilt to two phases and Lock was added as well.
--- .../platform/chips-media/wave5/wave5-helper.c | 28 +++++- .../platform/chips-media/wave5/wave5-helper.h | 1 + .../chips-media/wave5/wave5-vpu-dec.c | 5 + .../chips-media/wave5/wave5-vpu-enc.c | 5 + .../platform/chips-media/wave5/wave5-vpu.c | 96 +++++++++++++++++-- .../platform/chips-media/wave5/wave5-vpuapi.h | 6 ++ 6 files changed, 130 insertions(+), 11 deletions(-)
drivers/media/platform/chips-media/wave5/wave5-helper.c +27 -1
@@ -27,6 +27,11 @@ const char *state_to_str(enum vpu_instance_state state) } } +int wave5_kfifo_alloc (struct vpu_instance *inst) +{ +return kfifo_alloc (&inst->irq_status,16 *sizeof (int ),GFP_KERNEL ); +} +void wave5_cleanup_instance (struct vpu_instance *inst,struct file *filp) {int i;
@@ -49,7 +54,7 @@ void wave5_cleanup_instance(struct vpu_instance *inst, struct file *filp)v4l2_fh_del (&inst->v4l2_fh, filp);v4l2_fh_exit (&inst->v4l2_fh); } -list_del_init (&inst->list); +kfifo_free (&inst->irq_status);ida_free (&inst->dev->inst_ida, inst->id);kfree (inst->codec_info);kfree (inst);
@@ -61,8 +66,29 @@ int wave5_vpu_release_device(struct file *filp, {struct vpu_instance *inst =file_to_vpu_inst (filp);int ret =0 ; +unsigned long flags;v4l2_m2m_ctx_release (inst->v4l2_fh.m2m_ctx); +/* +* To prevent Null reference exception, the existing irq handler were +* separated to two modules. +* One is to queue interrupt reason into the irq handler, +* the other is irq_thread to call the wave5_vpu_dec_finish_decode +* to get decoded frame. +* The list of instances should be protected between all flow of the +* decoding process, but to protect the list in the irq_handler, spin lock +* should be used, and mutex should be used in the irq_thread because spin lock +* is not able to be used because mutex is already being used +* in the wave5_vpu_dec_finish_decode. +* So the spin lock and mutex were used to protect the list in the release function. +*/ + ret =mutex_lock_interruptible (&inst->dev->irq_lock); +if (ret) +return ret; +spin_lock_irqsave (&inst->dev->irq_spinlock, flags); +list_del_init (&inst->list); +spin_unlock_irqrestore (&inst->dev->irq_spinlock, flags); +mutex_unlock (&inst->dev->irq_lock);if (inst->state !=VPU_INST_STATE_NONE ) {u32 fail_res;
drivers/media/platform/chips-media/wave5/wave5-helper.h +1 -0
@@ -33,4 +33,5 @@ void wave5_update_pix_fmt(struct v4l2_pix_format_mplane *pix_mp,unsigned int width, unsigned int height, const struct v4l2_frmsize_stepwise *frmsize); +int wave5_kfifo_alloc (struct vpu_instance *inst);#endif
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5 -0
@@ -1808,6 +1808,11 @@ static int wave5_vpu_open_dec(struct file *filp) inst->xfer_func =V4L2_XFER_FUNC_DEFAULT ;init_completion (&inst->irq_done); + ret =wave5_kfifo_alloc (inst); +if (ret) { +dev_err (inst->dev->dev,"failed to allocate fifo\n" ); + goto cleanup_inst; + } inst->id =ida_alloc (&inst->dev->inst_ida,GFP_KERNEL );if (inst->id <0 ) {
drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c +5 -0
@@ -1754,6 +1754,11 @@ static int wave5_vpu_open_enc(struct file *filp) inst->frame_rate =30 ;init_completion (&inst->irq_done); + ret =wave5_kfifo_alloc (inst); +if (ret) { +dev_err (inst->dev->dev,"failed to allocate fifo\n" ); + goto cleanup_inst; + } inst->id =ida_alloc (&inst->dev->inst_ida,GFP_KERNEL );if (inst->id <0 ) {
drivers/media/platform/chips-media/wave5/wave5-vpu.c +86 -10
170 lines not on this page · raw message
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +6 -0
@@ -8,6 +8,7 @@#ifndef VPUAPI_H_INCLUDED #define VPUAPI_H_INCLUDED +#include <linux/kfifo.h> #include <linux/idr.h> #include <linux/genalloc.h> #include <media/v4l2-device.h>
@@ -747,6 +748,7 @@ struct vpu_device {struct video_device *video_dev_enc;struct mutex dev_lock;/* lock for the src, dst v4l2 queues */ struct mutex hw_lock;/* lock hw configurations */ +struct mutex irq_lock;int irq;enum product_id product;struct vpu_attr attr;
@@ -764,7 +766,10 @@ struct vpu_device {struct kthread_worker *worker;int vpu_poll_interval;int num_clks; +struct task_struct *irq_thread; +struct semaphore irq_sem;/* signal to irq_thread when interrupt happens*/ struct reset_control *resets; +spinlock_t irq_spinlock;/* protect instances list */ };struct vpu_instance ;
@@ -788,6 +793,7 @@ struct vpu_instance {enum v4l2_ycbcr_encoding ycbcr_enc;enum v4l2_quantization quantization; +struct kfifo irq_status;enum vpu_instance_state state;enum vpu_instance_type type;const struct vpu_instance_ops *ops;
Signature · 1 line
2.43.0
Jackson.lee
maintainerPATCH v6 3/4 media: chips-media: wave5: Add WARN_ON to check if dec_output_info is NULL
In reply to Jackson.leeFrom: Jackson Lee <[email protected]>
The dec_output_info should not be a null pointer, WARN_ON around it to indicates a driver issue.
--- drivers/media/platform/chips-media/wave5/wave5-vpuapi.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +1 -1
@@ -485,7 +485,7 @@ int wave5_vpu_dec_get_output_info(struct vpu_instance *inst, struct dec_output_istruct vpu_device *vpu_dev = inst->dev;struct dec_output_info *disp_info; -if (!info) +if (WARN_ON (!info))return -EINVAL ; p_dec_info = &inst->codec_info->dec_info;
Signature · 1 line
2.43.0
Jackson.lee
maintainerPATCH v6 4/4 media: chips-media: wave5: Improve performance of decoder
In reply to Jackson.leeFrom: Jackson Lee <[email protected]>
The current decoding method was to wait until each frame was decoded after feeding a bitstream. As a result, performance was low and Wave5 could not achieve max pixel processing rate.
Update driver to use an asynchronous approach for decoding and feeding a bitstream in order to achieve full capabilities of the device.
WAVE5 supports command-queueing to maximize performance by pipelining internal commands and by hiding wait cycle taken to receive a command from Host processor.
Instead of waiting for each command to be executed before sending the next command, Host processor just places all the commands in the command-queue and goes on doing other things while the commands in the queue are processed by VPU.
While Host processor handles its own tasks, it can receive VPU interrupt request (IRQ).
In this case, host processor can simply exit interrupt service routine (ISR) without accessing to host interface to read the result of the command reported by VPU.
After host processor completed its tasks, host processor can read the command result when host processor needs the reports and does response processing.
To achieve this goal, the device_run() calls v4l2_m2m_job_finish so that next command can be sent to VPU continuously, if there is any result, then irq is triggered and gets decoded frames and returns them to upper layer.
Theses processes work independently each other without waiting a decoded frame.
--- .../platform/chips-media/wave5/wave5-hw.c | 2 +- .../chips-media/wave5/wave5-vpu-dec.c | 165 +++++++++++++----- .../platform/chips-media/wave5/wave5-vpu.h | 2 +- .../platform/chips-media/wave5/wave5-vpuapi.c | 53 ++++-- .../platform/chips-media/wave5/wave5-vpuapi.h | 7 + .../chips-media/wave5/wave5-vpuconfig.h | 1 + 6 files changed, 173 insertions(+), 57 deletions(-)
drivers/media/platform/chips-media/wave5/wave5-hw.c +1 -1
@@ -102,7 +102,7 @@ static void _wave5_print_reg_err(struct vpu_device *vpu_dev, u32 reg_fail_reasondev_dbg (dev,"%s: queueing failure: 0x%x\n" , func, reg_val);break ;case WAVE5_SYSERR_RESULT_NOT_READY : -dev_err (dev,"%s: result not ready: 0x%x\n" , func, reg_fail_reason); +dev_dbg (dev,"%s: result not ready: 0x%x\n" , func, reg_fail_reason);break ;case WAVE5_SYSERR_ACCESS_VIOLATION_HW :dev_err (dev,"%s: access violation: 0x%x\n" , func, reg_fail_reason);
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +124 -41
323 lines not on this page · raw message
drivers/media/platform/chips-media/wave5/wave5-vpu.h +1 -1
@@ -22,8 +22,8 @@struct vpu_src_buffer {struct v4l2_m2m_buffer v4l2_m2m_buf; -struct list_head list;bool consumed; +struct list_head list; };struct vpu_dst_buffer {
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +39 -14
105 lines not on this page · raw message
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +7 -0
@@ -818,6 +818,13 @@ struct vpu_instance {bool cbcr_interleave;bool nv21;bool eos; +bool sent_eos;/* check if EOS is sent to application */ +bool retry;/* retry to feed bitstream if failure reason is WAVE5_SYSERR_QUEUEING_FAIL*/ +int queuing_num;/* count of bitstream queued */ +int queued_count;/* the number of bitstream buffers queued from the buf_queue */ +struct mutex feed_lock;/* lock for feeding bitstream buffers */ +bool queuing_fail;/* if there is the queuing failure */ +bool empty_queue;struct vpu_buf bitstream_vbuf;dma_addr_t last_rd_ptr;size_t remaining_consumed_bytes;
drivers/media/platform/chips-media/wave5/wave5-vpuconfig.h +1 -0
@@ -59,6 +59,7 @@// application specific configuration #define VPU_ENC_TIMEOUT 60000#define VPU_DEC_TIMEOUT 60000 +#define VPU_DEC_STOP_TIMEOUT 10// for WAVE encoder #define USE_SRC_PRP_AXI 0
Signature · 1 line
2.43.0
Re: [PATCH v6 4/4] media: chips-media: wave5: Improve performance of decoder
Le vendredi 14 novembre 2025 à 10:21 +0900, Jackson.lee a écrit :
Quoted 118 lines
From: Jackson Lee <[email protected]>
The current decoding method was to wait until each frame was decoded after feeding a bitstream. As a result, performance was low and Wave5 could not achieve max pixel processing rate.
Update driver to use an asynchronous approach for decoding and feeding a bitstream in order to achieve full capabilities of the device.
WAVE5 supports command-queueing to maximize performance by pipelining internal commands and by hiding wait cycle taken to receive a command from Host processor.
Instead of waiting for each command to be executed before sending the next command, Host processor just places all the commands in the command-queue and goes on doing other things while the commands in the queue are processed by VPU.
While Host processor handles its own tasks, it can receive VPU interrupt request (IRQ).
In this case, host processor can simply exit interrupt service routine (ISR) without accessing to host interface to read the result of the command reported by VPU.
After host processor completed its tasks, host processor can read the command result when host processor needs the reports and does response processing.To achieve this goal, the
device_run()callsv4l2_m2m_job_finishso that next command can be sent to VPU continuously, if there is any result, then irq is triggered and gets decoded frames and returns them to upper layer.
Theses processes work independently each other without waiting a decoded frame.Signed-off-by Jackson Lee <[email protected]>Signed-off-by Nas Chung <[email protected]>---
.../platform/chips-media/wave5/wave5-hw.c | 2 +- .../chips-media/wave5/wave5-vpu-dec.c | 165 +++++++++++++----- .../platform/chips-media/wave5/wave5-vpu.h | 2 +-.../platform/chips-media/wave5/wave5-vpuapi.c| 53 ++++--.../platform/chips-media/wave5/wave5-vpuapi.h| 7 + .../chips-media/wave5/wave5-vpuconfig.h | 1 +6 files changed, 173 insertions(+), 57 deletions(-)drivers/media/platform/chips-media/wave5/wave5-hw.c +0
-0@@ -102,7 +102,7 @@ static void _wave5_print_reg_err(struct vpu_device *vpu_dev, u32 reg_fail_reason dev_dbg(dev, "%s: queueing failure: 0x%x\n", func, reg_val); break;case
WAVE5_SYSERR_RESULT_NOT_READY:drivers/media/platform/chips-media/wave5/wave5-hw.c +1
-1-dev_err (dev,"%s: result not ready: 0x%x\n" , func, reg_fail_reason); +dev_dbg (dev,"%s: result not ready: 0x%x\n" , func, reg_fail_reason);break;
caseWAVE5_SYSERR_ACCESS_VIOLATION_HW:
dev_err(dev, "%s: access violation: 0x%x\n", func, reg_fail_reason);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -268,6 +268,7 @@ static void send_eos_event(struct vpu_instance *inst)
v4l2_event_queue_fh(&inst->v4l2_fh, &vpu_event_eos);inst->eos= false;drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ inst->sent_eos = true;}
static inthandle_dynamic_resolution_change(struct vpu_instance *inst)drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1@@ -347,13 +348,12 @@ static void wave5_vpu_dec_finish_decode(struct vpu_instance *inst) struct vb2_v4l2_buffer *dec_buf = NULL; struct vb2_v4l2_buffer *disp_buf = NULL; struct vb2_queue *dst_vq = v4l2_m2m_get_dst_vq(m2m_ctx); -struct queue_status_info q_status;
dev_dbg(inst->dev->dev, "%s: Fetch output info from firmware.", __func__);
ret =wave5_vpu_dec_get_output_info(inst, &dec_info);
if (ret) {drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-1-dev_warn (inst->dev->dev,"%s: could not get output info." , __func__); +dev_dbg (inst->dev->dev,"%s: could not get output info." , __func__);
v4l2_m2m_job_finish(inst->v4l2_m2m_dev, m2m_ctx); return;
}drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -442,18 +442,17 @@ static void wave5_vpu_dec_finish_decode(struct vpu_instance *inst) spin_unlock_irqrestore(&inst->state_spinlock, flags); }drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +8
-11-/* -* During a resolution change and while draining, the firmware may flush -* the reorder queue regardless of having a matching decoding operation -* pending. Only terminate the job if there are no more IRQ coming. -*/ -wave5_vpu_dec_give_command (inst,DEC_GET_QUEUE_STATUS , &q_status); -if (q_status.report_queue_count ==0 && - (q_status.instance_queue_count == 0 || dec_info.sequence_changed)) { -dev_dbg (inst->dev->dev,"%s: finishing job.\n" , __func__); -pm_runtime_put_autosuspend (inst->dev->dev); -v4l2_m2m_job_finish (inst->v4l2_m2m_dev, m2m_ctx); +if (inst->sent_eos && + v4l2_m2m_get_curr_priv(inst->v4l2_m2m_dev)) { +struct queue_status_info q_status; + +wave5_vpu_dec_give_command (inst,DEC_GET_QUEUE_STATUS , &q_status); +if (q_status.report_queue_count ==0 && + q_status.instance_queue_count == 0) +v4l2_m2m_job_finish (inst->v4l2_m2m_dev, m2m_ctx);}
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0+ + inst->queuing_fail = false;}
static intwave5_vpu_dec_querycap(struct file *file, void *fh, struct v4l2_capability *cap)drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -1142,11 +1141,31 @@ static int write_to_ringbuffer(struct vpu_instance *inst, void *buffer, size_t b return 0; }
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int write_to_ringbuffer(struct vpu_instance *inst, void *buffer, size_t b · line 1150
Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7
-0+static struct vpu_src_buffer *inst_src_buf_remove (struct vpu_instance *inst) +{ +struct vpu_src_buffer *b; + +if (list_empty (&inst->avail_src_bufs)) +return NULL ; + inst->queued_count--;
The only use of this counter can be replaced with list_empty().
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int write_to_ringbuffer(struct vpu_instance *inst, void *buffer, size_t b · line 1155
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+ b = list_first_entry(&inst->avail_src_bufs, struct vpu_src_buffer, list); +list_del (&b->list); + b->list.prev =NULL ; + b->list.next =NULL ; +INIT_LIST_HEAD (&b->list);
These 3 lines above are simply:
list_del_init(&b->list);
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int write_to_ringbuffer(struct vpu_instance *inst, void *buffer, size_t b · line 1160
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+if (inst->queued_count ==0 ) { + inst->avail_src_bufs.prev =NULL ; + inst->avail_src_bufs.next =NULL ; +INIT_LIST_HEAD (&inst->avail_src_bufs); + }
I don't think this is needed, once empty that list_empty() condition should be met again.
Quoted 149 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3
-0+return b; +} +static int
fill_ringbuffer(struct vpu_instance *inst){
structv4l2_m2m_ctx*m2m_ctx=inst->v4l2_fh.m2m_ctx;drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-2-struct v4l2_m2m_buffer *buf, *n; -int ret; +struct vpu_src_buffer *vpu_buf; +int ret =0 ;
if (m2m_ctx->last_src_buf) {
structvpu_src_buffer*vpu_buf=wave5_to_vpu_src_buf(m2m_ctx->last_src_buf);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -1157,9 +1176,8 @@ static int fill_ringbuffer(struct vpu_instance *inst) } }drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-3-v4l2_m2m_for_each_src_buf_safe (m2m_ctx, buf, n) { -struct vb2_v4l2_buffer *vbuf = &buf->vb; -struct vpu_src_buffer *vpu_buf =wave5_to_vpu_src_buf (vbuf); +while ((vpu_buf =inst_src_buf_remove (inst)) !=NULL ) { +struct vb2_v4l2_buffer *vbuf = &vpu_buf->v4l2_m2m_buf.vb;struct
vpu_buf*ring_buffer=&inst->bitstream_vbuf;size_tsrc_size=vb2_get_plane_payload(&vbuf->vb2_buf, 0); void*src_buf=vb2_plane_vaddr(&vbuf->vb2_buf, 0);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3
-0@@ -1219,9 +1237,12 @@ static int fill_ringbuffer(struct vpu_instance *inst) dev_dbg(inst->dev->dev, "last src buffer written to the ring buffer\n"); break; } + + inst->queuing_num++; +break ; }drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-1-return 0 ; +return ret;}
static voidwave5_vpu_dec_buf_queue_src(struct vb2_buffer *vb)drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -1233,7 +1254,8 @@ static void wave5_vpu_dec_buf_queue_src(struct vb2_buffer *vb)
vpu_buf->consumed= false;vbuf->sequence= inst->queued_src_buf_num++;drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-1- +list_add_tail (&vpu_buf->list, &inst->avail_src_bufs); + inst->queued_count++;
v4l2_m2m_buf_queue(m2m_ctx, vbuf);
}drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -1288,10 +1310,16 @@ static void wave5_vpu_dec_buf_queue(struct vb2_buffer *vb) __func__, vb->type, vb->index, vb2_plane_size(&vbuf->vb2_buf, 0), vb2_plane_size(&vbuf->vb2_buf, 1), vb2_plane_size(&vbuf->vb2_buf, 2));drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-1-if (vb->type ==V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE ) +if (vb->type ==V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE ) { +mutex_lock (&inst->feed_lock);
wave5_vpu_dec_buf_queue_src(vb);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-1-else if (vb->type ==V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE ) + +if (inst->empty_queue) + inst->empty_queue = false; +mutex_unlock (&inst->feed_lock); + }else if (vb->type ==V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE ) {
wave5_vpu_dec_buf_queue_dst(vb);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ }}
static intwave5_vpu_dec_allocate_ring_buffer(struct vpu_instance *inst)drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +11
-0@@ -1385,6 +1413,17 @@ static int streamoff_output(struct vb2_queue *q) dma_addr_t new_rd_ptr; struct dec_output_info dec_info; unsigned int i; +struct vpu_src_buffer *vpu_buf, *tmp; + + inst->retry = false; + inst->queuing_num =0 ; + inst->queued_count =0 ; +mutex_lock (&inst->feed_lock); +list_for_each_entry_safe (vpu_buf, tmp, &inst->avail_src_bufs, list) { + vpu_buf->consumed = false; +list_del (&vpu_buf->list); + } +mutex_unlock (&inst->feed_lock);
for (i = 0; i <v4l2_m2m_num_dst_bufs_ready(m2m_ctx); i++) { ret =wave5_vpu_dec_set_disp_flag(inst, i);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0@@ -1470,6 +1509,7 @@ static void wave5_vpu_dec_stop_streaming(struct vb2_queue *q) { struct vpu_instance *inst = vb2_get_drv_priv(q); struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx; + bool check_cmd = TRUE;
dev_dbg(inst->dev->dev, "%s: type: %u\n", __func__, q->type);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -1480,11 +1520,10 @@ static void wave5_vpu_dec_stop_streaming(struct vb2_queue *q) struct dec_output_info dec_output_info;
wave5_vpu_dec_give_command(inst, DEC_GET_QUEUE_STATUS, &q_status);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +4
-5- -if (q_status.report_queue_count ==0 ) -break ; - -if (wave5_vpu_wait_interrupt (inst,VPU_DEC_TIMEOUT ) <0 ) +if ((inst->state ==VPU_INST_STATE_STOP || + inst->state == VPU_INST_STATE_INIT_SEQ || + q_status.instance_queue_count == 0) && + q_status.report_queue_count ==0 )break;
if (wave5_vpu_dec_get_output_info(inst, &dec_output_info))drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -1498,6 +1537,8 @@ static void wave5_vpu_dec_stop_streaming(struct vb2_queue *q)else
streamoff_capture(q);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0+ inst->empty_queue = false; + inst->sent_eos = false;
pm_runtime_put_autosuspend(inst->dev->dev); }drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0@@ -1576,13 +1617,24 @@ static void wave5_vpu_dec_device_run(void *priv) struct queue_status_info q_status; u32 fail_res = 0; int ret = 0; +unsigned long flags;
dev_dbg(inst->dev->dev, "%s: Fill the ring buffer with new bitstream data", __func__);pm_runtime_resume_and_get(inst->dev->dev);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +14
-4- ret =fill_ringbuffer (inst); -if (ret) { -dev_warn (inst->dev->dev,"Filling ring buffer failed\n" ); - goto finish_job_and_return; +if (!inst->retry) { +mutex_lock (&inst->feed_lock); + ret =fill_ringbuffer (inst); +mutex_unlock (&inst->feed_lock); +if (ret <0 ) { +dev_warn (inst->dev->dev,"Filling ring buffer failed\n" ); + goto finish_job_and_return; + }else if (!inst->eos && + inst->queuing_num ==0 && + inst->state ==VPU_INST_STATE_PIC_RUN ) { +dev_dbg (inst->dev->dev,"%s: no bitstream for feeding, so skip " , __func__); + inst->empty_queue = true; + goto finish_job_and_return; + }}
switch (inst->state) {
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_device_run(void *priv) · line 1662
Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0@@ -1607,7 +1659,9 @@ static void wave5_vpu_dec_device_run(void *priv) } spin_unlock_irqrestore(&inst->state_spinlock, flags); } else { +spin_lock_irqsave (&inst->state_spinlock, flags);
Move the locking inside the set_state function.
cheers,
Nicolas
Quoted 277 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0switch_state(inst, VPU_INST_STATE_INIT_SEQ); +spin_unlock_irqrestore (&inst->state_spinlock, flags); }
break;drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +4
-1@@ -1618,8 +1672,9 @@ static void wave5_vpu_dec_device_run(void *priv)* we had a chance to switch, which leads to an invalid state * change. */ +spin_lock_irqsave (&inst->state_spinlock, flags); switch_state(inst, VPU_INST_STATE_PIC_RUN); - +spin_unlock_irqrestore (&inst->state_spinlock, flags); /** During DRC, the picture decoding remains pending, so just leave the job * active until this decode operation completes. @@ -1633,12 +1688,14 @@ static void wave5_vpu_dec_device_run(void *priv) ret = wave5_prepare_fb(inst); if (ret) { dev_warn(inst->dev->dev, "Framebuffer preparation, fail: %d\n", ret); +spin_lock_irqsave (&inst->state_spinlock, flags); switch_state(inst, VPU_INST_STATE_STOP); +spin_unlock_irqrestore (&inst->state_spinlock, flags); break; }
if (q_status.instance_queue_count) {drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-1-dev_dbg (inst->dev->dev,"%s: leave with active job" , __func__); +v4l2_m2m_job_finish (inst->v4l2_m2m_dev, m2m_ctx);return;
}drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +13
-6@@ -1649,25 +1706,42 @@ static void wave5_vpu_dec_device_run(void *priv) dev_err(inst->dev->dev, "Frame decoding on m2m context (%p), fail: %d (result: %d)\n", m2m_ctx, ret, fail_res); -break ; + goto finish_job_and_return; + } + +if (fail_res ==WAVE5_SYSERR_QUEUEING_FAIL ) { + inst->retry = true; + inst->queuing_fail = true; + }else { + inst->retry = false; +if (!inst->eos) + inst->queuing_num--; } -/* Return so that we leave this job active */ -dev_dbg (inst->dev->dev,"%s: leave with active job" , __func__); -return ; -default : -WARN (1 ,"Execution of a job in state %s illegal.\n" ,state_to_str (inst->state)); break; +default : +dev_dbg (inst->dev->dev,"Execution of a job in state %s illegal.\n" , +state_to_str (inst->state)); }
finish_job_and_return:
dev_dbg(inst->dev->dev, "%s: leave and finish job", __func__);pm_runtime_put_autosuspend(inst->dev->dev);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +10
-1-v4l2_m2m_job_finish (inst->v4l2_m2m_dev, m2m_ctx); +/* +* After receiving CMD_STOP, there is no input, but we have to run device_run +* to send DEC_PIC command until display index == -1, so job_finish was always +* called in the device_run to archive it, the logic was very wasteful +* in power and CPU time. +* If EOS is passed, device_run will not call job_finish no more, it is called +* only if HW is idle status in order to reduce overhead. +*/ +if (!inst->sent_eos) +v4l2_m2m_job_finish (inst->v4l2_m2m_dev, m2m_ctx);}
static voidwave5_vpu_dec_job_abort(void *priv)
{
structvpu_instance*inst = priv;drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;int ret;
ret =switch_state(inst, VPU_INST_STATE_STOP);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0@@ -1678,6 +1752,8 @@ static void wave5_vpu_dec_job_abort(void *priv) if (ret) dev_warn(inst->dev->dev, "Setting EOS for the bitstream, fail: %d\n", ret); + +v4l2_m2m_job_finish (inst->v4l2_m2m_dev, m2m_ctx); }
static intwave5_vpu_dec_job_ready(void *priv)drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +6
-1@@ -1713,10 +1789,15 @@ static int wave5_vpu_dec_job_ready(void *priv) "No capture buffer ready to decode!\n"); break; } else if (!wave5_is_draining_or_eos(inst) && - !v4l2_m2m_num_src_bufs_ready(m2m_ctx)) { + (!v4l2_m2m_num_src_bufs_ready(m2m_ctx) || + inst->empty_queue)) { dev_dbg(inst->dev->dev, "No bitstream data to decode!\n"); break; + }else if (inst->state ==VPU_INST_STATE_PIC_RUN && + !wave5_is_draining_or_eos(inst) && + inst->queuing_fail) { +break ; } ret = 1; break;@@ -1753,6 +1834,8 @@ static int wave5_vpu_open_dec(struct file *filp) inst->ops = &wave5_vpu_dec_inst_ops;
spin_lock_init(&inst->state_spinlock);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0+mutex_init (&inst->feed_lock); +INIT_LIST_HEAD (&inst->avail_src_bufs);
inst->codec_info=kzalloc(sizeof(*inst->codec_info), GFP_KERNEL); if (!inst->codec_info)drivers/media/platform/chips-media/wave5/wave5-vpu.h +0
-0@@ -22,8 +22,8 @@
structvpu_src_buffer{
structv4l2_m2m_bufferv4l2_m2m_buf;drivers/media/platform/chips-media/wave5/wave5-vpu.h +0
-1-struct list_head list;bool consumed;
drivers/media/platform/chips-media/wave5/wave5-vpu.h +1
-0+struct list_head list;};
structvpu_dst_buffer{drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +1
-0@@ -52,11 +52,12 @@ int wave5_vpu_init_with_bitcode(struct device *dev, u8 *bitcode, size_t size) int wave5_vpu_flush_instance(struct vpu_instance *inst) { int ret = 0; +int mutex_ret =0 ; int retry = 0;drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +3
-3- ret =mutex_lock_interruptible (&inst->dev->hw_lock); -if (ret) -return ret; + mutex_ret =mutex_lock_interruptible (&inst->dev->hw_lock); +if (mutex_ret) +return mutex_ret;do {
/*
* Repeat the FLUSH command until the firmware reports that thedrivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-0@@ -80,11 +81,16 @@ int wave5_vpu_flush_instance(struct vpu_instance *inst)
mutex_unlock(&inst->dev->hw_lock);wave5_vpu_dec_get_output_info(inst, &dec_info);drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +5
-4- ret =mutex_lock_interruptible (&inst->dev->hw_lock); -if (ret) -return ret; -if (dec_info.index_frame_display >0 ) + mutex_ret =mutex_lock_interruptible (&inst->dev->hw_lock); +if (mutex_ret) +return mutex_ret; +if (dec_info.index_frame_display >=0 ) { +mutex_unlock (&inst->dev->hw_lock);
wave5_vpu_dec_set_disp_flag(inst,ds dec_info.index_frame_display);drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +4
-0+ mutex_ret =mutex_lock_interruptible (&inst->dev->hw_lock); +if (mutex_ret) +return mutex_ret; + }}
} while (ret != 0);
mutex_unlock(&inst->dev->hw_lock);drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +2
-0@@ -207,6 +213,8 @@ int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res) int retry = 0; struct vpu_device *vpu_dev = inst->dev; int i; +struct dec_output_info dec_info; +int ret_mutex;
*fail_res= 0;
if (!inst->codec_info)drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-0@@ -214,10 +222,10 @@ int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res)
pm_runtime_resume_and_get(inst->dev->dev);drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +2
-2- ret =mutex_lock_interruptible (&vpu_dev->hw_lock); -if (ret) { + ret_mutex =mutex_lock_interruptible (&vpu_dev->hw_lock); +if (ret_mutex) {
pm_runtime_put_sync(inst->dev->dev);drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +1
-1-return ret; +return ret_mutex;}
do {drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-0@@ -227,11 +235,26 @@ int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res) goto unlock_and_return; }drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +9
-2-if (*fail_res ==WAVE5_SYSERR_VPU_STILL_RUNNING && - retry++ >= MAX_FIRMWARE_CALL_RETRY) { +if (ret ==0 ) +break ; + +if (*fail_res !=WAVE5_SYSERR_VPU_STILL_RUNNING ) { +dev_warn (inst->dev->dev,"dec_finish_seq timed out\n" ); + goto unlock_and_return; + } + +if (retry++ >=MAX_FIRMWARE_CALL_RETRY ) {ret =
-ETIMEDOUT;
gotounlock_and_return;
}drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +8
-0+ +mutex_unlock (&vpu_dev->hw_lock); +wave5_vpu_dec_get_output_info (inst, &dec_info); + ret_mutex =mutex_lock_interruptible (&vpu_dev->hw_lock); +if (ret_mutex) { +pm_runtime_put_sync (inst->dev->dev); +return ret_mutex; + }} while (ret != 0);
dev_dbg(inst->dev->dev, "%s: dec_finish_seq complete\n", __func__);drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-0@@ -248,6 +271,8 @@ int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res)
wave5_vdi_free_dma_memory(vpu_dev, &p_dec_info->vb_task);drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +2
-0+mutex_destroy (&inst->feed_lock); +unlock_and_return:
mutex_unlock(&vpu_dev->hw_lock);
pm_runtime_put_sync(inst->dev->dev);drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +1
-1@@ -460,11 +485,11 @@ int wave5_vpu_dec_set_rd_ptr(struct vpu_instance *inst, dma_addr_t addr, int upd dma_addr_t wave5_vpu_dec_get_rd_ptr(struct vpu_instance *inst) { int ret; -dma_addr_t rd_ptr; +dma_addr_t rd_ptr =0 ;
ret =mutex_lock_interruptible(&inst->dev->hw_lock); if (ret)drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +1
-1-return ret; +return rd_ptr;
rd_ptr=wave5_dec_get_rd_ptr(inst);drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +7
-0@@ -818,6 +818,13 @@ struct vpu_instance { bool cbcr_interleave; bool nv21; bool eos; +bool sent_eos;/* check if EOS is sent to application */ +bool retry;/* retry to feed bitstream if failure reason is WAVE5_SYSERR_QUEUEING_FAIL*/ +int queuing_num;/* count of bitstream queued */ +int queued_count;/* the number of bitstream buffers queued from the buf_queue */ +struct mutex feed_lock;/* lock for feeding bitstream buffers */ +bool queuing_fail;/* if there is the queuing failure */ +bool empty_queue; struct vpu_buf bitstream_vbuf; dma_addr_t last_rd_ptr; size_t remaining_consumed_bytes;drivers/media/platform/chips-media/wave5/wave5-vpuconfig.h +1
-0@@ -59,6 +59,7 @@ // application specific configuration #define VPU_ENC_TIMEOUT 60000 #define VPU_DEC_TIMEOUT 60000 +#define VPU_DEC_STOP_TIMEOUT 10
// for WAVE encoder
#define USE_SRC_PRP_AXI 0
jackson.lee
maintainerRE: [PATCH v6 4/4] media: chips-media: wave5: Improve performance of decoder
Hi Nicoas
Quoted 29 lines
Quoted 4 lines
}
static int
wave5_vpu_dec_querycap(struct file *file, void *fh, struct v4l2_capability *cap)@@ -1142,11 +1141,31 @@ static intwrite_to_ringbuffer(struct
vpu_instance*inst, void *buffer,size_tbdrivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int write_to_ringbuffer(struct vpu_instance *inst, void *buffer, size_t b · line 1150
Quoted 10 lines
return 0;
}drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7
-0+static struct vpu_src_buffer *inst_src_buf_remove (struct vpu_instance +*inst) { +struct vpu_src_buffer *b; + +if (list_empty (&inst->avail_src_bufs)) +return NULL ; + inst->queued_count--;The only use of this counter can be replaced with
list_empty().Review
Quoted 1 line
(quoted diff) +1
-0+ b = list_first_entry(&inst->avail_src_bufs, struct vpu_src_buffer,list);
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int write_to_ringbuffer(struct vpu_instance *inst, void *buffer, size_t b · line 1155
Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +4
-0+list_del (&b->list); + b->list.prev =NULL ; + b->list.next =NULL ; +INIT_LIST_HEAD (&b->list);These 3 lines above are simply:
list_del_init(&b->list);
Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int write_to_ringbuffer(struct vpu_instance *inst, void *buffer, size_t b · line 1160
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+if (inst->queued_count ==0 ) { + inst->avail_src_bufs.prev =NULL ; + inst->avail_src_bufs.next =NULL ; +INIT_LIST_HEAD (&inst->avail_src_bufs); + }I don't think this is needed, once empty that
list_empty()condition should be met again.
I will do
Quoted 176 lines
Review static int fill_ringbuffer(struct vpu_instance
Quoted 16 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3
-0+return b; +} +static int
fill_ringbuffer(struct vpu_instance *inst){
structv4l2_m2m_ctx*m2m_ctx=inst->v4l2_fh.m2m_ctx;drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-2-struct v4l2_m2m_buffer *buf, *n; -int ret; +struct vpu_src_buffer *vpu_buf; +int ret =0 ;if (m2m_ctx->last_src_buf) {
structvpu_src_buffer*vpu_buf=wave5_to_vpu_src_buf(m2m_ctx->last_src_buf);(quoted diff) +0
-0@@ -1157,9 +1176,8 @@ static int fill_ringbuffer(struct vpu_instance*inst)
Quoted 11 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-3} } -v4l2_m2m_for_each_src_buf_safe (m2m_ctx, buf, n) { -struct vb2_v4l2_buffer *vbuf = &buf->vb; -struct vpu_src_buffer *vpu_buf =wave5_to_vpu_src_buf (vbuf); +while ((vpu_buf =inst_src_buf_remove (inst)) !=NULL ) { +struct vb2_v4l2_buffer *vbuf = &vpu_buf->v4l2_m2m_buf.vb; struct vpu_buf *ring_buffer = &inst->bitstream_vbuf; size_t src_size = vb2_get_plane_payload(&vbuf->vb2_buf, 0);void
*src_buf=vb2_plane_vaddr(&vbuf->vb2_buf, 0); @@ -1219,9
Quoted 2 lines
+1237,12 @@ static int
fill_ringbuffer(struct vpu_instance *inst)dev_dbg(inst->dev->dev, "last src buffer written tothe ring buffer\n");
Review static void wave5_vpu_dec_buf_queue(struct
Quoted 24 lines
break;
}drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3
-0+ + inst->queuing_num++; +break ;}
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-1-return 0 ; +return ret;}
static void
wave5_vpu_dec_buf_queue_src(struct vb2_buffer *vb)@@ -1233,7 +1254,8 @@ static voidwave5_vpu_dec_buf_queue_src(struct vb2_buffer *vb)
vpu_buf->consumed= false;vbuf->sequence= inst->queued_src_buf_num++;drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-1- +list_add_tail (&vpu_buf->list, &inst->avail_src_bufs); + inst->queued_count++;
v4l2_m2m_buf_queue(m2m_ctx, vbuf);
}(quoted diff) +0
-0@@ -1288,10 +1310,16 @@ static void wave5_vpu_dec_buf_queue(struct
vb2_buffer*vb)Review static void wave5_vpu_dec_buf_queue(struct · line 1310
Quoted 1 line
(quoted diff) +0
-0__func__, vb->type, vb->index, vb2_plane_size(&vbuf->vb2_buf,0),
Quoted 18 lines
vb2_plane_size(&vbuf->vb2_buf, 1), vb2_plane_size(&vbuf-vb2_buf,
2));drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-1-if (vb->type ==V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE ) +if (vb->type ==V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE ) { +mutex_lock (&inst->feed_lock);
wave5_vpu_dec_buf_queue_src(vb);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-1-else if (vb->type ==V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE ) + +if (inst->empty_queue) + inst->empty_queue = false; +mutex_unlock (&inst->feed_lock); + }else if (vb->type ==V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE ) {
wave5_vpu_dec_buf_queue_dst(vb);(quoted diff) +1
-0+ }}
Quoted 2 lines
static int
wave5_vpu_dec_allocate_ring_buffer(struct vpu_instance *inst)@@ -1385,6 +1413,17 @@ static int streamoff_output(struct
vb2_queue*q)drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int streamoff_output(struct vb2_queue *q)
Quoted 10 lines
dma_addr_tnew_rd_ptr;
structdec_output_infodec_info;
unsigned int i;drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7
-0+struct vpu_src_buffer *vpu_buf, *tmp; + + inst->retry = false; + inst->queuing_num =0 ; + inst->queued_count =0 ; +mutex_lock (&inst->feed_lock); +list_for_each_entry_safe (vpu_buf, tmp, &inst->avail_src_bufs, list){
Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +4
-0+ vpu_buf->consumed = false; +list_del (&vpu_buf->list); + } +mutex_unlock (&inst->feed_lock);for (i = 0; i <
v4l2_m2m_num_dst_bufs_ready(m2m_ctx); i++) { ret =wave5_vpu_dec_set_disp_flag(inst, i); @@ -1470,6+1509,7 @@
Quoted 9 lines
static void
wave5_vpu_dec_stop_streaming(struct vb2_queue *q){
structvpu_instance*inst =vb2_get_drv_priv(q); structv4l2_m2m_ctx*m2m_ctx=inst->v4l2_fh.m2m_ctx; +
boolcheck_cmd= TRUE;
dev_dbg(inst->dev->dev, "%s: type: %u\n", __func__, q->type); @@ -1480,11 +1520,10 @@ static void wave5_vpu_dec_stop_streaming(struct
vb2_queue*q)Quoted 3 lines
struct
dec_output_infodec_output_info;wave5_vpu_dec_give_command(inst,
DEC_GET_QUEUE_STATUS,
&q_status);Quoted 11 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +4
-5- -if (q_status.report_queue_count ==0 ) -break ; - -if (wave5_vpu_wait_interrupt (inst,VPU_DEC_TIMEOUT ) <0 ) +if ((inst->state ==VPU_INST_STATE_STOP || + inst->state == VPU_INST_STATE_INIT_SEQ || + q_status.instance_queue_count == 0) && + q_status.report_queue_count ==0 )break;
Quoted 1 line
if (
wave5_vpu_dec_get_output_info(inst, &dec_output_info))@@
Quoted 1 line
-1498,6 +1537,8 @@ static void wave5_vpu_dec_stop_streaming(struct
vb2_queue*q)Quoted 14 lines
else
streamoff_capture(q);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0+ inst->empty_queue = false; + inst->sent_eos = false;
pm_runtime_put_autosuspend(inst->dev->dev); }drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0@@ -1576,13 +1617,24 @@ static void wave5_vpu_dec_device_run(void *priv) struct queue_status_info q_status; u32 fail_res = 0; int ret = 0; +unsigned long flags;Quoted 1 line
dev_dbg(inst->dev->dev, "%s: Fill the ring buffer with new
bitstream data",
__func__);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_device_run(void *priv)
Quoted 11 lines
pm_runtime_resume_and_get(inst->dev->dev);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +6
-4- ret =fill_ringbuffer (inst); -if (ret) { -dev_warn (inst->dev->dev,"Filling ring buffer failed\n" ); - goto finish_job_and_return; +if (!inst->retry) { +mutex_lock (&inst->feed_lock); + ret =fill_ringbuffer (inst); +mutex_unlock (&inst->feed_lock); +if (ret <0 ) { + dev_warn(inst->dev->dev, "Filling ring bufferfailed\n");
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_device_run(void *priv)
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+ goto finish_job_and_return; + }else if (!inst->eos && + inst->queuing_num ==0 && + inst->state ==VPU_INST_STATE_PIC_RUN ) { + dev_dbg(inst->dev->dev, "%s: no bitstream for feeding,so skip ",
__func__);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_device_run(void *priv) · line 1662
Quoted 11 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3
-0+ inst->empty_queue = true; + goto finish_job_and_return; + }}
switch (
inst->state) {drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0@@ -1607,7 +1659,9 @@ static void wave5_vpu_dec_device_run(void *priv) } spin_unlock_irqrestore(&inst->state_spinlock, flags); } else { +spin_lock_irqsave (&inst->state_spinlock, flags);Move the locking inside the
set_statefunction.
Quoted 2 lines
cheers,
Nicolas
I think the locking should not be move into the set_state function(switch_state).
Because the send_eos_event, handle_dynamic_resolution_change and flag_last_buffer_done already have the lockdep_assert_held(&inst->state_spinlock); inside those function, So to concisify the above code, even if the locking is moved into switch_statue, the locking should be called again outside.
Thanks
Jackson
Re: [PATCH v6 4/4] media: chips-media: wave5: Improve performance of decoder
Hi,
Le lundi 17 novembre 2025 à 02:07 +0000, jackson.lee a écrit :
Quoted 8 lines
Quoted 7 lines
Review
Quoted 2 lines
} else {
(quoted diff) +1
-0+ spin_lock_irqsave(&inst->state_spinlock, flags);Move the locking inside the
set_statefunction.cheers,
Nicolas
Quoted 7 lines
I think the locking should not be move into the
set_statefunction(switch_state).
Because thesend_eos_event,handle_dynamic_resolution_changeandflag_last_buffer_donealready have the lockdep_assert_held(&inst-Quoted 1 line
state_spinlock); inside those function,So to concisify the above code, even if the locking is moved into
switch_statue, the locking should be called again outside.
My suggestion is to try and reduce code redundancy, another method commonly used is to have a small version of the function that takes the locking, this you can turn 3 lines into one in many places.
Nicolas