Performance improvement of decoder
Subsystems (2)
Jackson.lee
maintainercover Performance improvement of decoder
From: Jackson Lee <[email protected]>
The wave5 codec driver is a stateful encoder/decoder. The following patches is for improving decoder performance.
v4l2-compliance results:
========================
v4l2-compliance 1.28.1-5233, 64 bits, 64-bit time_t
Buffer ioctls:
warn: v4l2-test-buffers.cpp(693): VIDIOC_CREATE_BUFS not supported
warn: v4l2-test-buffers.cpp(693): VIDIOC_CREATE_BUFS not supported
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
test CREATE_BUFS maximum buffers: OK
test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)
Total for wave5-dec device /dev/video0: 46, Succeeded: 46, Failed: 0, Warnings: 2 Total for wave5-enc device /dev/video1: 46, Succeeded: 46, Failed: 0, Warnings: 0
Fluster test results:
=====================
Running test suite JCT-VC-HEVC_V1 with decoder GStreamer-H.265-V4L2-Gst1.0 Using 1 parallel job(s)
Ran 133/147 tests successfully in 68.517 secs
(1 test fails because of not supporting to parse multi frames, 1 test fails because of a missing frame and slight corruption,
2 tests fail because of sizes which are incompatible with the IP, 11 tests fail because of unsupported 10 bit format)
Running test suite JVT-AVC_V1 with decoder GStreamer-H.264-V4L2-Gst1.0 Using 1 parallel job(s)
Ran 78/135 tests successfully in 35.015 secs
(57 fail because the hardware is unable to decode MBAFF / FMO / Field / Extended profile streams.)
Jackson Lee (3):
media: chips-media: wave5: Improve performance of decoder
media: chips-media: wave5: Reduce high CPU load
media: chips-media: wave5: Fix Null reference while testing fluster
.../platform/chips-media/wave5/wave5-helper.c | 10 +- .../chips-media/wave5/wave5-vpu-dec.c | 361 ++++++++++-------- .../chips-media/wave5/wave5-vpu-enc.c | 8 +- .../platform/chips-media/wave5/wave5-vpu.c | 69 +++- .../platform/chips-media/wave5/wave5-vpuapi.c | 25 +- .../platform/chips-media/wave5/wave5-vpuapi.h | 12 +- 6 files changed, 307 insertions(+), 178 deletions(-)
Signature · 1 line
2.43.0
Jackson.lee
maintainerPATCH RESEND 1/3 media: chips-media: wave5: Improve performance of decoder
From: Jackson Lee <[email protected]>
The existing way for decoding frames was to wait until each frame was decoded after feeding a bitstream. As a result, performance was low and Wave5 could not achieve max pixel processing rate.
Update driver to use an asynchronous approach for decoding and feeding a bitstream in order to achieve full capabilities of the device.
--- .../chips-media/wave5/wave5-vpu-dec.c | 344 ++++++++++-------- .../chips-media/wave5/wave5-vpu-enc.c | 3 - .../platform/chips-media/wave5/wave5-vpuapi.c | 25 +- .../platform/chips-media/wave5/wave5-vpuapi.h | 5 +- 4 files changed, 211 insertions(+), 166 deletions(-)
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +196 -148
@@ -230,12 +230,147 @@ static int start_decode(struct vpu_instance *inst, u32 *fail_res)switch_state (inst,VPU_INST_STATE_STOP );dev_dbg (inst->dev->dev,"%s: pic run failed / finish job" , __func__); -v4l2_m2m_job_finish (inst->v4l2_m2m_dev, m2m_ctx); }return ret; } +static int write_to_ringbuffer (struct vpu_instance *inst,void *buffer,size_t buffer_size, +struct vpu_buf *ring_buffer,dma_addr_t wr_ptr) +{ +size_t size; +size_t offset = wr_ptr - ring_buffer->daddr; +int ret; + +if (wr_ptr + buffer_size > ring_buffer->daddr + ring_buffer->size) { + size = ring_buffer->daddr + ring_buffer->size - wr_ptr; + ret =wave5_vdi_write_memory (inst->dev, ring_buffer, offset, (u8 *)buffer, size); +if (ret <0 ) +return ret; + + ret =wave5_vdi_write_memory (inst->dev, ring_buffer,0 , (u8 *)buffer + size, + buffer_size - size); +if (ret <0 ) +return ret; + }else { + ret =wave5_vdi_write_memory (inst->dev, ring_buffer, offset, (u8 *)buffer, + buffer_size); +if (ret <0 ) +return ret; + } + +return 0 ; +} + +static int fill_ringbuffer (struct vpu_instance *inst) +{ +struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx; +struct vpu_src_buffer *vpu_buf; +int ret =0 ; + +if (m2m_ctx->last_src_buf) { +struct vpu_src_buffer *vpu_buf =wave5_to_vpu_src_buf (m2m_ctx->last_src_buf); + +if (vpu_buf->consumed) { +dev_dbg (inst->dev->dev,"last src buffer already written\n" ); +return 0 ; + } + } + +list_for_each_entry (vpu_buf, &inst->avail_src_bufs, list) { +struct vb2_v4l2_buffer *vbuf = &vpu_buf->v4l2_m2m_buf.vb; +struct vpu_buf *ring_buffer = &inst->bitstream_vbuf; +size_t src_size =vb2_get_plane_payload (&vbuf->vb2_buf,0 ); +void *src_buf =vb2_plane_vaddr (&vbuf->vb2_buf,0 ); +dma_addr_t rd_ptr =0 ; +dma_addr_t wr_ptr =0 ; +size_t remain_size =0 ; + +if (vpu_buf->consumed) { +dev_dbg (inst->dev->dev,"already copied src buf (%u) to the ring buffer\n" , + vbuf->vb2_buf.index); +continue ; + } + +if (!src_buf) { +dev_dbg (inst->dev->dev, +"%s: Acquiring kernel pointer to src buf (%u), fail\n" , + __func__, vbuf->vb2_buf.index); +break ; + } + + ret =wave5_vpu_dec_get_bitstream_buffer (inst, &rd_ptr, &wr_ptr, &remain_size); +if (ret) { +/* Unable to acquire the mutex */ +dev_err (inst->dev->dev,"Getting the bitstream buffer, fail: %d\n" , + ret); +return ret; + } + +dev_dbg (inst->dev->dev,"%s: rd_ptr %pad wr_ptr %pad" , __func__, &rd_ptr, &wr_ptr); + +if (remain_size < src_size) { +dev_dbg (inst->dev->dev, +"%s: remaining size: %zu < source size: %zu for src buf (%u)\n" , + __func__, remain_size, src_size, vbuf->vb2_buf.index); +break ; + } + + ret =write_to_ringbuffer (inst, src_buf, src_size, ring_buffer, wr_ptr); +if (ret) { +dev_err (inst->dev->dev,"Write src buf (%u) to ring buffer, fail: %d\n" , + vbuf->vb2_buf.index, ret); +return ret; + } + + ret =wave5_vpu_dec_update_bitstream_buffer (inst, src_size); +if (ret) { +dev_dbg (inst->dev->dev, +"update_bitstream_buffer fail: %d for src buf (%u)\n" , + ret, vbuf->vb2_buf.index); +break ; + } + + vpu_buf->consumed = true; + +/* Don't write buffers passed the last one while draining. */ +if (v4l2_m2m_is_last_draining_src_buf (m2m_ctx, vbuf)) { +dev_dbg (inst->dev->dev,"last src buffer written to the ring buffer\n" ); +break ; + } + + inst->queuing_num++; +list_del_init (&vpu_buf->list); +break ; + } + +return ret; +} + +static void wave5_vpu_dec_feed_remaining (struct vpu_instance *inst) +{ +int ret =0 ; +struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx; +u32 fail_res =0 ; + +mutex_lock (&inst->feed_lock); + ret =fill_ringbuffer (inst); +mutex_unlock (&inst->feed_lock); +if (ret) { +dev_warn (inst->dev->dev,"Filling ring buffer failed\n" ); +return ; + } + + ret =start_decode (inst, &fail_res); +if (ret) { +dev_err (inst->dev->dev, +"Frame decoding on m2m context (%p), fail: %d (result: %d)\n" , + m2m_ctx, ret, fail_res); + } + +v4l2_m2m_job_finish (inst->v4l2_m2m_dev, m2m_ctx); +} +static void flag_last_buffer_done (struct vpu_instance *inst) {struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
@@ -347,7 +482,6 @@ static void wave5_vpu_dec_finish_decode(struct vpu_instance *inst)struct vb2_v4l2_buffer *dec_buf =NULL ;struct vb2_v4l2_buffer *disp_buf =NULL ;struct vb2_queue *dst_vq =v4l2_m2m_get_dst_vq (m2m_ctx); -struct queue_status_info q_status;dev_dbg (inst->dev->dev,"%s: Fetch output info from firmware." , __func__);
@@ -441,20 +575,6 @@ static void wave5_vpu_dec_finish_decode(struct vpu_instance *inst) }spin_unlock_irqrestore (&inst->state_spinlock, flags); } - -/* -* During a resolution change and while draining, the firmware may flush -* the reorder queue regardless of having a matching decoding operation -* pending. Only terminate the job if there are no more IRQ coming. -*/ -wave5_vpu_dec_give_command (inst,DEC_GET_QUEUE_STATUS , &q_status); -if (q_status.report_queue_count ==0 && - (q_status.instance_queue_count ==0 || dec_info.sequence_changed)) { -dev_dbg (inst->dev->dev,"%s: finishing job.\n" , __func__); -pm_runtime_mark_last_busy (inst->dev->dev); -pm_runtime_put_autosuspend (inst->dev->dev); -v4l2_m2m_job_finish (inst->v4l2_m2m_dev, m2m_ctx); - } }static int wave5_vpu_dec_querycap (struct file *file,void *fh,struct v4l2_capability *cap)
@@ -794,11 +914,21 @@ static int wave5_vpu_dec_stop(struct vpu_instance *inst) }if (inst->state !=VPU_INST_STATE_NONE ) { +struct vb2_v4l2_buffer *vbuf; +struct vpu_src_buffer *vpu_buf; +/* * Temporarily release the state_spinlock so that subsequent * calls do not block on a mutex while inside this spinlock. */ spin_unlock_irqrestore (&inst->state_spinlock, flags); + vbuf =v4l2_m2m_last_src_buf (m2m_ctx); +if (vbuf) { + vpu_buf =wave5_to_vpu_src_buf (vbuf); +if (!vpu_buf->consumed) +wave5_vpu_dec_feed_remaining (inst); + } + ret =wave5_vpu_dec_set_eos_on_firmware (inst);if (ret)return ret;
@@ -1116,115 +1246,6 @@ static int wave5_prepare_fb(struct vpu_instance *inst)return 0 ; } -static int write_to_ringbuffer (struct vpu_instance *inst,void *buffer,size_t buffer_size, -struct vpu_buf *ring_buffer,dma_addr_t wr_ptr) -{ -size_t size; -size_t offset = wr_ptr - ring_buffer->daddr; -int ret; - -if (wr_ptr + buffer_size > ring_buffer->daddr + ring_buffer->size) { - size = ring_buffer->daddr + ring_buffer->size - wr_ptr; - ret =wave5_vdi_write_memory (inst->dev, ring_buffer, offset, (u8 *)buffer, size); -if (ret <0 ) -return ret; - - ret =wave5_vdi_write_memory (inst->dev, ring_buffer,0 , (u8 *)buffer + size, - buffer_size - size); -if (ret <0 ) -return ret; - }else { - ret =wave5_vdi_write_memory (inst->dev, ring_buffer, offset, (u8 *)buffer, - buffer_size); -if (ret <0 ) -return ret; - } - -return 0 ; -} - -static int fill_ringbuffer (struct vpu_instance *inst) -{ -struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx; -struct v4l2_m2m_buffer *buf, *n; -int ret; - -if (m2m_ctx->last_src_buf) { -struct vpu_src_buffer *vpu_buf =wave5_to_vpu_src_buf (m2m_ctx->last_src_buf); - -if (vpu_buf->consumed) { -dev_dbg (inst->dev->dev,"last src buffer already written\n" ); -return 0 ; - } - } - -v4l2_m2m_for_each_src_buf_safe (m2m_ctx, buf, n) { -struct vb2_v4l2_buffer *vbuf = &buf->vb; -struct vpu_src_buffer *vpu_buf =wave5_to_vpu_src_buf (vbuf); -struct vpu_buf *ring_buffer = &inst->bitstream_vbuf; -size_t src_size =vb2_get_plane_payload (&vbuf->vb2_buf,0 ); -void *src_buf =vb2_plane_vaddr (&vbuf->vb2_buf,0 ); -dma_addr_t rd_ptr =0 ; -dma_addr_t wr_ptr =0 ; -size_t remain_size =0 ; - -if (vpu_buf->consumed) { -dev_dbg (inst->dev->dev,"already copied src buf (%u) to the ring buffer\n" , - vbuf->vb2_buf.index); -continue ; - } - -if (!src_buf) { -dev_dbg (inst->dev->dev, -"%s: Acquiring kernel pointer to src buf (%u), fail\n" , - __func__, vbuf->vb2_buf.index); -break ; - } - - ret =wave5_vpu_dec_get_bitstream_buffer (inst, &rd_ptr, &wr_ptr, &remain_size); -if (ret) { -/* Unable to acquire the mutex */ -dev_err (inst->dev->dev,"Getting the bitstream buffer, fail: %d\n" , - ret); -return ret; - } - -dev_dbg (inst->dev->dev,"%s: rd_ptr %pad wr_ptr %pad" , __func__, &rd_ptr, &wr_ptr); - -if (remain_size < src_size) { -dev_dbg (inst->dev->dev, -"%s: remaining size: %zu < source size: %zu for src buf (%u)\n" , - __func__, remain_size, src_size, vbuf->vb2_buf.index); -break ; - } - - ret =write_to_ringbuffer (inst, src_buf, src_size, ring_buffer, wr_ptr); -if (ret) { -dev_err (inst->dev->dev,"Write src buf (%u) to ring buffer, fail: %d\n" , - vbuf->vb2_buf.index, ret); -return ret; - } - - ret =wave5_vpu_dec_update_bitstream_buffer (inst, src_size); -if (ret) { -dev_dbg (inst->dev->dev, -"update_bitstream_buffer fail: %d for src buf (%u)\n" , - ret, vbuf->vb2_buf.index); -break ; - } - - vpu_buf->consumed = true; - -/* Don't write buffers passed the last one while draining. */ -if (v4l2_m2m_is_last_draining_src_buf (m2m_ctx, vbuf)) { -dev_dbg (inst->dev->dev,"last src buffer written to the ring buffer\n" ); -break ; - } - } - -return 0 ; -} -static void wave5_vpu_dec_buf_queue_src (struct vb2_buffer *vb) {struct vpu_instance *inst =vb2_get_drv_priv (vb->vb2_queue);
@@ -1236,6 +1257,11 @@ static void wave5_vpu_dec_buf_queue_src(struct vb2_buffer *vb) vbuf->sequence = inst->queued_src_buf_num++;v4l2_m2m_buf_queue (m2m_ctx, vbuf); + +INIT_LIST_HEAD (&vpu_buf->list); +mutex_lock (&inst->feed_lock); +list_add_tail (&vpu_buf->list, &inst->avail_src_bufs); +mutex_unlock (&inst->feed_lock); }static void wave5_vpu_dec_buf_queue_dst (struct vb2_buffer *vb)
@@ -1362,7 +1388,6 @@ static int wave5_vpu_dec_start_streaming(struct vb2_queue *q, unsigned int count goto return_buffers; } } - }pm_runtime_mark_last_busy (inst->dev->dev);pm_runtime_put_autosuspend (inst->dev->dev);
@@ -1385,6 +1410,13 @@ static int streamoff_output(struct vb2_queue *q)dma_addr_t new_rd_ptr;struct dec_output_info dec_info;unsigned int i; +struct vpu_src_buffer *vpu_buf, *tmp; + + inst->retry = false; + inst->queuing_num =0 ; + +list_for_each_entry_safe (vpu_buf, tmp, &inst->avail_src_bufs, list) +list_del_init (&vpu_buf->list);for (i =0 ; i <v4l2_m2m_num_dst_bufs_ready (m2m_ctx); i++) { ret =wave5_vpu_dec_set_disp_flag (inst, i);
@@ -1481,10 +1513,8 @@ static void wave5_vpu_dec_stop_streaming(struct vb2_queue *q)wave5_vpu_dec_give_command (inst,DEC_GET_QUEUE_STATUS , &q_status); -if (q_status.report_queue_count ==0 ) -break ; - -if (wave5_vpu_wait_interrupt (inst,VPU_DEC_TIMEOUT ) <0 ) +if ((inst->state ==VPU_INST_STATE_STOP || q_status.instance_queue_count ==0 ) && + q_status.report_queue_count ==0 )break ;if (wave5_vpu_dec_get_output_info (inst, &dec_output_info))
@@ -1577,13 +1607,23 @@ static void wave5_vpu_dec_device_run(void *priv)struct queue_status_info q_status;u32 fail_res =0 ;int ret =0 ; +unsigned long flags;dev_dbg (inst->dev->dev,"%s: Fill the ring buffer with new bitstream data" , __func__);pm_runtime_resume_and_get (inst->dev->dev); - ret =fill_ringbuffer (inst); -if (ret) { -dev_warn (inst->dev->dev,"Filling ring buffer failed\n" ); - goto finish_job_and_return; +if (!inst->retry) { +mutex_lock (&inst->feed_lock); + ret =fill_ringbuffer (inst); +mutex_unlock (&inst->feed_lock); +if (ret <0 ) { +dev_warn (inst->dev->dev,"Filling ring buffer failed\n" ); + goto finish_job_and_return; + }else if (!inst->eos && + inst->queuing_num ==0 && + inst->state ==VPU_INST_STATE_PIC_RUN ) { +dev_dbg (inst->dev->dev,"%s: no bitstream for feeding, so skip " , __func__); + goto finish_job_and_return; + } }switch (inst->state) {
@@ -1619,7 +1659,9 @@ static void wave5_vpu_dec_device_run(void *priv)* we had a chance to switch, which leads to an invalid state * change. */ +spin_lock_irqsave (&inst->state_spinlock, flags);switch_state (inst,VPU_INST_STATE_PIC_RUN ); +spin_unlock_irqrestore (&inst->state_spinlock, flags);/* * During DRC, the picture decoding remains pending, so just leave the job
@@ -1634,12 +1676,14 @@ static void wave5_vpu_dec_device_run(void *priv) ret =wave5_prepare_fb (inst);if (ret) {dev_warn (inst->dev->dev,"Framebuffer preparation, fail: %d\n" , ret); +spin_lock_irqsave (&inst->state_spinlock, flags);switch_state (inst,VPU_INST_STATE_STOP ); +spin_unlock_irqrestore (&inst->state_spinlock, flags);break ; }if (q_status.instance_queue_count) { -dev_dbg (inst->dev->dev,"%s: leave with active job" , __func__); +v4l2_m2m_job_finish (inst->v4l2_m2m_dev, m2m_ctx);return ; }
@@ -1650,14 +1694,22 @@ static void wave5_vpu_dec_device_run(void *priv)dev_err (inst->dev->dev,"Frame decoding on m2m context (%p), fail: %d (result: %d)\n" , m2m_ctx, ret, fail_res); -break ; + goto finish_job_and_return; + } + +if (fail_res ==WAVE5_SYSERR_QUEUEING_FAIL ) { + inst->retry = true; + }else { + inst->retry = false; +if (!inst->eos) + inst->queuing_num--; } -/* Return so that we leave this job active */ -dev_dbg (inst->dev->dev,"%s: leave with active job" , __func__); -return ; -default : -WARN (1 ,"Execution of a job in state %s illegal.\n" ,state_to_str (inst->state));break ; +default : +if (!v4l2_m2m_has_stopped (m2m_ctx)) +WARN (1 ,"Execution of a job in state %s illegal.\n" , +state_to_str (inst->state)); +return ; } finish_job_and_return:
@@ -1676,10 +1728,7 @@ static void wave5_vpu_dec_job_abort(void *priv)if (ret)return ; - ret =wave5_vpu_dec_set_eos_on_firmware (inst); -if (ret) -dev_warn (inst->dev->dev, -"Setting EOS for the bitstream, fail: %d\n" , ret); +v4l2_m2m_job_finish (inst->v4l2_m2m_dev, inst->v4l2_fh.m2m_ctx); }static int wave5_vpu_dec_job_ready (void *priv)
@@ -1755,6 +1804,8 @@ static int wave5_vpu_open_dec(struct file *filp) inst->ops = &wave5_vpu_dec_inst_ops;spin_lock_init (&inst->state_spinlock); +mutex_init (&inst->feed_lock); +INIT_LIST_HEAD (&inst->avail_src_bufs); inst->codec_info =kzalloc (sizeof (*inst->codec_info),GFP_KERNEL );if (!inst->codec_info)
@@ -1830,9 +1881,6 @@ static int wave5_vpu_open_dec(struct file *filp)if (ret) goto cleanup_inst; -if (list_empty (&dev->instances)) -pm_runtime_use_autosuspend (inst->dev->dev); -list_add_tail (&inst->list, &dev->instances);mutex_unlock (&dev->dev_lock);
drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c +0 -3
@@ -1774,9 +1774,6 @@ static int wave5_vpu_open_enc(struct file *filp)if (ret) goto cleanup_inst; -if (list_empty (&dev->instances)) -pm_runtime_use_autosuspend (inst->dev->dev); -list_add_tail (&inst->list, &dev->instances);mutex_unlock (&dev->dev_lock);
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +11 -14
@@ -207,8 +207,6 @@ int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res)int retry =0 ;struct vpu_device *vpu_dev = inst->dev;int i; -int inst_count =0 ; -struct vpu_instance *inst_elm; *fail_res =0 ;if (!inst->codec_info)
@@ -233,6 +231,16 @@ int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res) retry++ >= MAX_FIRMWARE_CALL_RETRY) { ret = -ETIMEDOUT ; goto unlock_and_return; + } else if (*fail_res == WAVE5_SYSERR_VPU_STILL_RUNNING) { +struct dec_output_info dec_info; + +mutex_unlock (&vpu_dev->hw_lock); +wave5_vpu_dec_get_output_info (inst, &dec_info); + ret =mutex_lock_interruptible (&vpu_dev->hw_lock); +if (ret) { +pm_runtime_put_sync (inst->dev->dev); +return ret; + } } }while (ret !=0 );
@@ -249,11 +257,7 @@ int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res) }wave5_vdi_free_dma_memory (vpu_dev, &p_dec_info->vb_task); - -list_for_each_entry (inst_elm, &vpu_dev->instances, list) - inst_count++; -if (inst_count ==1 ) -pm_runtime_dont_use_autosuspend (vpu_dev->dev); +mutex_destroy (&inst->feed_lock); unlock_and_return:mutex_unlock (&vpu_dev->hw_lock);
@@ -720,8 +724,6 @@ int wave5_vpu_enc_close(struct vpu_instance *inst, u32 *fail_res)int ret;int retry =0 ;struct vpu_device *vpu_dev = inst->dev; -int inst_count =0 ; -struct vpu_instance *inst_elm; *fail_res =0 ;if (!inst->codec_info)
@@ -765,11 +767,6 @@ int wave5_vpu_enc_close(struct vpu_instance *inst, u32 *fail_res)wave5_vdi_free_dma_memory (vpu_dev, &p_enc_info->vb_task); -list_for_each_entry (inst_elm, &vpu_dev->instances, list) - inst_count++; -if (inst_count ==1 ) -pm_runtime_dont_use_autosuspend (vpu_dev->dev); -mutex_unlock (&vpu_dev->hw_lock);pm_runtime_put_sync (inst->dev->dev);
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +4 -1
@@ -163,7 +163,7 @@ enum set_param_option {#define BUFFER_MARGIN 4096 -#define MAX_FIRMWARE_CALL_RETRY 10 +#define MAX_FIRMWARE_CALL_RETRY 30#define VDI_LITTLE_ENDIAN 0x0
@@ -812,6 +812,9 @@ struct vpu_instance {bool cbcr_interleave;bool nv21;bool eos; +bool retry; +int queuing_num; +struct mutex feed_lock;/* lock for feeding bitstream buffers */ struct vpu_buf bitstream_vbuf;dma_addr_t last_rd_ptr;size_t remaining_consumed_bytes;
Signature · 1 line
2.43.0
Re: [RESEND PATCH v0 1/3] media: chips-media: wave5: Improve performance of decoder
Le mercredi 19 mars 2025 à 12:50 +0900, Jackson.lee a écrit :
Quoted 3 lines
From: Jackson Lee <[email protected]>
The existing way for decoding frames was to wait until each frame was
Suggestion:
The current decoding method was to...
Quoted 4 lines
decoded after feeding a bitstream. As a result, performance was low and Wave5 could not achieve max pixel processing rate.
Update driver to use an asynchronous approach for decoding and feeding a
Update the driver ..
Quoted 1 line
bitstream in order to achieve full capabilities of the device.
This second part lacks the technical design choices needed for your reviewer ;-P I did read the patch once, and am under the impression that your approach is to not finish the job anymore.
Quoted 24 lines
Signed-off-by Jackson Lee <[email protected]>Signed-off-by Nas Chung <[email protected]>---
.../chips-media/wave5/wave5-vpu-dec.c | 344 ++++++++++-------- .../chips-media/wave5/wave5-vpu-enc.c | 3 -.../platform/chips-media/wave5/wave5-vpuapi.c| 25 +-.../platform/chips-media/wave5/wave5-vpuapi.h| 5 +-4 files changed, 211 insertions(+), 166 deletions(-)drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -230,12 +230,147 @@ static int start_decode(struct vpu_instance *inst, u32 *fail_res) switch_state(inst, VPU_INST_STATE_STOP);
dev_dbg(inst->dev->dev, "%s: pic run failed / finish job", __func__);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1-v4l2_m2m_job_finish (inst->v4l2_m2m_dev, m2m_ctx);}
return ret;
}
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 240
Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3
-0+static int write_to_ringbuffer (struct vpu_instance *inst,void *buffer,size_t buffer_size, + struct vpu_buf *ring_buffer, dma_addr_t wr_ptr) +{
It seems that from now on this needs to be called with the feed_lock, so add:
lockdep_assert_held(&inst->feed_lock);
Quoted 13 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +13
-0+size_t size; +size_t offset = wr_ptr - ring_buffer->daddr; +int ret; + +if (wr_ptr + buffer_size > ring_buffer->daddr + ring_buffer->size) { + size = ring_buffer->daddr + ring_buffer->size - wr_ptr; + ret =wave5_vdi_write_memory (inst->dev, ring_buffer, offset, (u8 *)buffer, size); +if (ret <0 ) +return ret; + + ret =wave5_vdi_write_memory (inst->dev, ring_buffer,0 , (u8 *)buffer + size, + buffer_size - size); +if (ret <0 )
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 263
Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +10
-0+return ret; + }else { + ret =wave5_vdi_write_memory (inst->dev, ring_buffer, offset, (u8 *)buffer, + buffer_size); +if (ret <0 ) +return ret; + } + +return 0 ; +}
I have a pretty strong preference in seeing code being moved around in seperate patch documented as "non functional changes". This way I know I don't have to review again that code.
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 266
Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3
-0+ +static int fill_ringbuffer (struct vpu_instance *inst) +{
Same, I don't want to have to do back and fourth to check if you changed anything. Also, after the refactoring, the lockdeup assert can be added in the new patch, which makes the introduction of the new lock very obvious.
Quoted 28 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +28
-0+struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx; +struct vpu_src_buffer *vpu_buf; +int ret =0 ; + + if (m2m_ctx->last_src_buf) { +struct vpu_src_buffer *vpu_buf =wave5_to_vpu_src_buf (m2m_ctx->last_src_buf); + +if (vpu_buf->consumed) { +dev_dbg (inst->dev->dev,"last src buffer already written\n" ); +return 0 ; + } + } + +list_for_each_entry (vpu_buf, &inst->avail_src_bufs, list) { +struct vb2_v4l2_buffer *vbuf = &vpu_buf->v4l2_m2m_buf.vb; +struct vpu_buf *ring_buffer = &inst->bitstream_vbuf; +size_t src_size =vb2_get_plane_payload (&vbuf->vb2_buf,0 ); +void *src_buf =vb2_plane_vaddr (&vbuf->vb2_buf,0 ); +dma_addr_t rd_ptr =0 ; +dma_addr_t wr_ptr =0 ; +size_t remain_size =0 ; + +if (vpu_buf->consumed) { +dev_dbg (inst->dev->dev,"already copied src buf (%u) to the ring buffer\n" , + vbuf->vb2_buf.index); +continue ; + } +
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 304
Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +10
-0+if (!src_buf) { +dev_dbg (inst->dev->dev, +"%s: Acquiring kernel pointer to src buf (%u), fail\n" , + __func__, vbuf->vb2_buf.index); +break ; + } + + ret =wave5_vpu_dec_get_bitstream_buffer (inst, &rd_ptr, &wr_ptr, &remain_size); +if (ret) { +/* Unable to acquire the mutex */
Might want to fix that comment in a separate patch, perhaps even remove that ?
Quoted 47 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +47
-0+dev_err (inst->dev->dev,"Getting the bitstream buffer, fail: %d\n" , + ret); +return ret; + } + +dev_dbg (inst->dev->dev,"%s: rd_ptr %pad wr_ptr %pad" , __func__, &rd_ptr, &wr_ptr); + +if (remain_size < src_size) { +dev_dbg (inst->dev->dev, +"%s: remaining size: %zu < source size: %zu for src buf (%u)\n" , + __func__, remain_size, src_size, vbuf->vb2_buf.index); +break ; + } + + ret =write_to_ringbuffer (inst, src_buf, src_size, ring_buffer, wr_ptr); +if (ret) { +dev_err (inst->dev->dev,"Write src buf (%u) to ring buffer, fail: %d\n" , + vbuf->vb2_buf.index, ret); +return ret; + } + + ret =wave5_vpu_dec_update_bitstream_buffer (inst, src_size); +if (ret) { +dev_dbg (inst->dev->dev, +"update_bitstream_buffer fail: %d for src buf (%u)\n" , + ret, vbuf->vb2_buf.index); +break ; + } + + vpu_buf->consumed = true; + +/* Don't write buffers passed the last one while draining. */ +if (v4l2_m2m_is_last_draining_src_buf (m2m_ctx, vbuf)) { +dev_dbg (inst->dev->dev,"last src buffer written to the ring buffer\n" ); +break ; + } + + inst->queuing_num++; +list_del_init (&vpu_buf->list); +break ; + } + +return ret; +} + +static void wave5_vpu_dec_feed_remaining (struct vpu_instance *inst) +{
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 361
Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +10
-0+int ret =0 ; +struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx; +u32 fail_res =0 ; + +mutex_lock (&inst->feed_lock); + ret =fill_ringbuffer (inst); +mutex_unlock (&inst->feed_lock); +if (ret) { +dev_warn (inst->dev->dev,"Filling ring buffer failed\n" ); +return ;
I don't think the error handling is correct. What will happen with the job ?
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 364
Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3
-0+ } + + ret =start_decode (inst, &fail_res);
I'm a little worried seeing this called outside of device_run() context.
Quoted 31 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +9
-0+if (ret) { +dev_err (inst->dev->dev, +"Frame decoding on m2m context (%p), fail: %d (result: %d)\n" , + m2m_ctx, ret, fail_res); + } + +v4l2_m2m_job_finish (inst->v4l2_m2m_dev, m2m_ctx); +} +static void
flag_last_buffer_done(struct vpu_instance *inst){
structv4l2_m2m_ctx*m2m_ctx=inst->v4l2_fh.m2m_ctx;drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1@@ -347,7 +482,6 @@ static void wave5_vpu_dec_finish_decode(struct vpu_instance *inst) struct vb2_v4l2_buffer *dec_buf = NULL; struct vb2_v4l2_buffer *disp_buf = NULL; struct vb2_queue *dst_vq = v4l2_m2m_get_dst_vq(m2m_ctx); -struct queue_status_info q_status;
dev_dbg(inst->dev->dev, "%s: Fetch output info from firmware.", __func__);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-4@@ -441,20 +575,6 @@ static void wave5_vpu_dec_finish_decode(struct vpu_instance *inst) } spin_unlock_irqrestore(&inst->state_spinlock, flags); } - -/* -* During a resolution change and while draining, the firmware may flush -* the reorder queue regardless of having a matching decoding operation
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_finish_decode(struct vpu_instance *inst) · line 457
Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-10-* pending. Only terminate the job if there are no more IRQ coming. -*/ -wave5_vpu_dec_give_command (inst,DEC_GET_QUEUE_STATUS , &q_status); -if (q_status.report_queue_count ==0 && - (q_status.instance_queue_count == 0 || dec_info.sequence_changed)) { -dev_dbg (inst->dev->dev,"%s: finishing job.\n" , __func__); -pm_runtime_mark_last_busy (inst->dev->dev); -pm_runtime_put_autosuspend (inst->dev->dev); -v4l2_m2m_job_finish (inst->v4l2_m2m_dev, m2m_ctx); - }
This removal is why I believe some seek stress testing are needed. I'll try and find some tests file I crafted last year.
Quoted 15 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0}
static intwave5_vpu_dec_querycap(struct file *file, void *fh, struct v4l2_capability *cap)drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -794,11 +914,21 @@ static int wave5_vpu_dec_stop(struct vpu_instance *inst) }
if (inst->state!=VPU_INST_STATE_NONE) {drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3
-0+struct vb2_v4l2_buffer *vbuf; +struct vpu_src_buffer *vpu_buf; +/*
* Temporarily release thestate_spinlockso that subsequent
* calls do not block on a mutex while inside this spinlock. */spin_unlock_irqrestore(&inst->state_spinlock, flags);
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_vpu_dec_stop(struct vpu_instance *inst) · line 931
Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7
-0+ vbuf =v4l2_m2m_last_src_buf (m2m_ctx); +if (vbuf) { + vpu_buf =wave5_to_vpu_src_buf (vbuf); +if (!vpu_buf->consumed) +wave5_vpu_dec_feed_remaining (inst); + } +
Do you really need to add all these checks ? Shouldn't you just "feed_remaining()" and move on ? I also don't think its a good idea to call v4l2_m2m_job_finish() in code that can run concurrently with the device_run() function. This should only happen if you are certain the VPU instance is stopped already.
Quoted 123 lines
ret =
wave5_vpu_dec_set_eos_on_firmware(inst); if (ret)
return ret;drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -1116,115 +1246,6 @@ static int wave5_prepare_fb(struct vpu_instance *inst) return 0; }drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-109-static int write_to_ringbuffer (struct vpu_instance *inst,void *buffer,size_t buffer_size, - struct vpu_buf *ring_buffer, dma_addr_t wr_ptr) -{ -size_t size; -size_t offset = wr_ptr - ring_buffer->daddr; -int ret; - -if (wr_ptr + buffer_size > ring_buffer->daddr + ring_buffer->size) { - size = ring_buffer->daddr + ring_buffer->size - wr_ptr; - ret =wave5_vdi_write_memory (inst->dev, ring_buffer, offset, (u8 *)buffer, size); -if (ret <0 ) -return ret; - - ret =wave5_vdi_write_memory (inst->dev, ring_buffer,0 , (u8 *)buffer + size, - buffer_size - size); -if (ret <0 ) -return ret; - }else { - ret =wave5_vdi_write_memory (inst->dev, ring_buffer, offset, (u8 *)buffer, - buffer_size); -if (ret <0 ) -return ret; - } - -return 0 ; -} - -static int fill_ringbuffer (struct vpu_instance *inst) -{ -struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx; -struct v4l2_m2m_buffer *buf, *n; -int ret; - - if (m2m_ctx->last_src_buf) { -struct vpu_src_buffer *vpu_buf =wave5_to_vpu_src_buf (m2m_ctx->last_src_buf); - -if (vpu_buf->consumed) { -dev_dbg (inst->dev->dev,"last src buffer already written\n" ); -return 0 ; - } - } - -v4l2_m2m_for_each_src_buf_safe (m2m_ctx, buf, n) { -struct vb2_v4l2_buffer *vbuf = &buf->vb; -struct vpu_src_buffer *vpu_buf =wave5_to_vpu_src_buf (vbuf); -struct vpu_buf *ring_buffer = &inst->bitstream_vbuf; -size_t src_size =vb2_get_plane_payload (&vbuf->vb2_buf,0 ); -void *src_buf =vb2_plane_vaddr (&vbuf->vb2_buf,0 ); -dma_addr_t rd_ptr =0 ; -dma_addr_t wr_ptr =0 ; -size_t remain_size =0 ; - -if (vpu_buf->consumed) { -dev_dbg (inst->dev->dev,"already copied src buf (%u) to the ring buffer\n" , - vbuf->vb2_buf.index); -continue ; - } - -if (!src_buf) { -dev_dbg (inst->dev->dev, -"%s: Acquiring kernel pointer to src buf (%u), fail\n" , - __func__, vbuf->vb2_buf.index); -break ; - } - - ret =wave5_vpu_dec_get_bitstream_buffer (inst, &rd_ptr, &wr_ptr, &remain_size); -if (ret) { -/* Unable to acquire the mutex */ -dev_err (inst->dev->dev,"Getting the bitstream buffer, fail: %d\n" , - ret); -return ret; - } - -dev_dbg (inst->dev->dev,"%s: rd_ptr %pad wr_ptr %pad" , __func__, &rd_ptr, &wr_ptr); - -if (remain_size < src_size) { -dev_dbg (inst->dev->dev, -"%s: remaining size: %zu < source size: %zu for src buf (%u)\n" , - __func__, remain_size, src_size, vbuf->vb2_buf.index); -break ; - } - - ret =write_to_ringbuffer (inst, src_buf, src_size, ring_buffer, wr_ptr); -if (ret) { -dev_err (inst->dev->dev,"Write src buf (%u) to ring buffer, fail: %d\n" , - vbuf->vb2_buf.index, ret); -return ret; - } - - ret =wave5_vpu_dec_update_bitstream_buffer (inst, src_size); -if (ret) { -dev_dbg (inst->dev->dev, -"update_bitstream_buffer fail: %d for src buf (%u)\n" , - ret, vbuf->vb2_buf.index); -break ; - } - - vpu_buf->consumed = true; - -/* Don't write buffers passed the last one while draining. */ -if (v4l2_m2m_is_last_draining_src_buf (m2m_ctx, vbuf)) { -dev_dbg (inst->dev->dev,"last src buffer written to the ring buffer\n" ); -break ; - } - } - -return 0 ; -} -static void
wave5_vpu_dec_buf_queue_src(struct vb2_buffer *vb){
structvpu_instance*inst =vb2_get_drv_priv(vb->vb2_queue);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -1236,6 +1257,11 @@ static void wave5_vpu_dec_buf_queue_src(struct vb2_buffer *vb) vbuf->sequence = inst->queued_src_buf_num++;
v4l2_m2m_buf_queue(m2m_ctx, vbuf);
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_buf_queue_src(struct vb2_buffer *vb) · line 1264
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+ +INIT_LIST_HEAD (&vpu_buf->list); +mutex_lock (&inst->feed_lock); +list_add_tail (&vpu_buf->list, &inst->avail_src_bufs); +mutex_unlock (&inst->feed_lock);
This lack documentation, you didn't even mention the introduction of a new src buf list in the commit message.
Quoted 29 lines
}
static voidwave5_vpu_dec_buf_queue_dst(struct vb2_buffer *vb)drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7
-1@@ -1362,7 +1388,6 @@ static int wave5_vpu_dec_start_streaming(struct vb2_queue *q, unsigned int count goto return_buffers; } } - } pm_runtime_mark_last_busy(inst->dev->dev); pm_runtime_put_autosuspend(inst->dev->dev);@@ -1385,6 +1410,13 @@ static int streamoff_output(struct vb2_queue *q) dma_addr_t new_rd_ptr; struct dec_output_info dec_info; unsigned int i; +struct vpu_src_buffer *vpu_buf, *tmp; + + inst->retry = false; + inst->queuing_num =0 ; + +list_for_each_entry_safe (vpu_buf, tmp, &inst->avail_src_bufs, list) +list_del_init (&vpu_buf->list);
for (i = 0; i <v4l2_m2m_num_dst_bufs_ready(m2m_ctx); i++) { ret =wave5_vpu_dec_set_disp_flag(inst, i);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -1481,10 +1513,8 @@ static void wave5_vpu_dec_stop_streaming(struct vb2_queue *q)
wave5_vpu_dec_give_command(inst, DEC_GET_QUEUE_STATUS, &q_status);
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_stop_streaming(struct vb2_queue *q) · line 1516
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-4-if (q_status.report_queue_count ==0 ) -break ; - -if (wave5_vpu_wait_interrupt (inst,VPU_DEC_TIMEOUT ) <0 ) + if ((inst->state == VPU_INST_STATE_STOP || q_status.instance_queue_count == 0) &&
State should be read with appropriate locking.
Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ q_status.report_queue_count == 0)break;
This loop have nothing to wait on anymore. It just actively pool on the HW. I noticed your third patch, but if this is what you address there, squash it somehow, this would break bisects.
The goal of that code was to ensure that this code will never run concurrently to finish_decode() once out of that while loop. I don't think you break that, but that will cause possibly high CPU load.
Quoted 17 lines
if (wave5_vpu_dec_get_output_info(inst, &dec_output_info))drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0@@ -1577,13 +1607,23 @@ static void wave5_vpu_dec_device_run(void *priv) struct queue_status_info q_status; u32 fail_res = 0; int ret = 0; +unsigned long flags;
dev_dbg(inst->dev->dev, "%s: Fill the ring buffer with new bitstream data", __func__);pm_runtime_resume_and_get(inst->dev->dev);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3
-4- ret =fill_ringbuffer (inst); -if (ret) { -dev_warn (inst->dev->dev,"Filling ring buffer failed\n" ); - goto finish_job_and_return; +if (!inst->retry) { +mutex_lock (&inst->feed_lock); + ret =fill_ringbuffer (inst);
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_device_run(void *priv) · line 1626
Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +10
-0+mutex_unlock (&inst->feed_lock); +if (ret <0 ) { +dev_warn (inst->dev->dev,"Filling ring buffer failed\n" ); + goto finish_job_and_return; + }else if (!inst->eos && + inst->queuing_num == 0 && + inst->state == VPU_INST_STATE_PIC_RUN) { +dev_dbg (inst->dev->dev,"%s: no bitstream for feeding, so skip " , __func__); + goto finish_job_and_return; + }
There is quite some overlap between that local src buffer queue and all the code that depends on v4l2_m2m_num_src_bufs_ready(). Are you sure you aren't re-implementing something ? Also, this code belong to the PIC_RUN case in the below switch.
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_device_run(void *priv) · line 1664
Quoted 10 lines
}
switch (inst->state) {drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0@@ -1619,7 +1659,9 @@ static void wave5_vpu_dec_device_run(void *priv)* we had a chance to switch, which leads to an invalid state * change. */ +spin_lock_irqsave (&inst->state_spinlock, flags); switch_state(inst, VPU_INST_STATE_PIC_RUN); +spin_unlock_irqrestore (&inst->state_spinlock, flags);
Should be its own patch, and fixed everywhere. Add missing lockdep assert (if there is a spinlock version of it).
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_device_run(void *priv) · line 1681
Quoted 10 lines
/*
* During DRC, the picture decoding remains pending, so just leave the jobdrivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0@@ -1634,12 +1676,14 @@ static void wave5_vpu_dec_device_run(void *priv) ret = wave5_prepare_fb(inst); if (ret) { dev_warn(inst->dev->dev, "Framebuffer preparation, fail: %d\n", ret); +spin_lock_irqsave (&inst->state_spinlock, flags); switch_state(inst, VPU_INST_STATE_STOP); +spin_unlock_irqrestore (&inst->state_spinlock, flags);
Same, this is separate fix.
Quoted 100 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0break; }
if (q_status.instance_queue_count) {drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-1-dev_dbg (inst->dev->dev,"%s: leave with active job" , __func__); +v4l2_m2m_job_finish (inst->v4l2_m2m_dev, m2m_ctx);return;
}drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +14
-6@@ -1650,14 +1694,22 @@ static void wave5_vpu_dec_device_run(void *priv) dev_err(inst->dev->dev, "Frame decoding on m2m context (%p), fail: %d (result: %d)\n", m2m_ctx, ret, fail_res); -break ; + goto finish_job_and_return; + } + +if (fail_res ==WAVE5_SYSERR_QUEUEING_FAIL ) { + inst->retry = true; + }else { + inst->retry = false; +if (!inst->eos) + inst->queuing_num--; } -/* Return so that we leave this job active */ -dev_dbg (inst->dev->dev,"%s: leave with active job" , __func__); -return ; -default : -WARN (1 ,"Execution of a job in state %s illegal.\n" ,state_to_str (inst->state)); break; +default : +if (!v4l2_m2m_has_stopped (m2m_ctx)) +WARN (1 ,"Execution of a job in state %s illegal.\n" , + state_to_str(inst->state)); +return ; }
finish_job_and_return:drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -1676,10 +1728,7 @@ static void wave5_vpu_dec_job_abort(void *priv) if (ret) return;drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-4- ret =wave5_vpu_dec_set_eos_on_firmware (inst); -if (ret) -dev_warn (inst->dev->dev, -"Setting EOS for the bitstream, fail: %d\n" , ret); +v4l2_m2m_job_finish (inst->v4l2_m2m_dev, inst->v4l2_fh.m2m_ctx);}
static intwave5_vpu_dec_job_ready(void *priv)drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -1755,6 +1804,8 @@ static int wave5_vpu_open_dec(struct file *filp) inst->ops = &wave5_vpu_dec_inst_ops;
spin_lock_init(&inst->state_spinlock);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0+mutex_init (&inst->feed_lock); +INIT_LIST_HEAD (&inst->avail_src_bufs);
inst->codec_info=kzalloc(sizeof(*inst->codec_info), GFP_KERNEL); if (!inst->codec_info)drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -1830,9 +1881,6 @@ static int wave5_vpu_open_dec(struct file *filp) if (ret) goto cleanup_inst;drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-3-if (list_empty (&dev->instances)) -pm_runtime_use_autosuspend (inst->dev->dev); -
list_add_tail(&inst->list, &dev->instances);
mutex_unlock(&dev->dev_lock);drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c +0
-0@@ -1774,9 +1774,6 @@ static int wave5_vpu_open_enc(struct file *filp) if (ret) goto cleanup_inst;drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c +0
-3-if (list_empty (&dev->instances)) -pm_runtime_use_autosuspend (inst->dev->dev); -
list_add_tail(&inst->list, &dev->instances);
mutex_unlock(&dev->dev_lock);drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-2@@ -207,8 +207,6 @@ int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res) int retry = 0; struct vpu_device *vpu_dev = inst->dev; int i; -int inst_count =0 ; -struct vpu_instance *inst_elm;
*fail_res= 0;
if (!inst->codec_info)drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-0@@ -233,6 +231,16 @@ int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res) retry++ >= MAX_FIRMWARE_CALL_RETRY) { ret = -ETIMEDOUT; goto unlock_and_return;
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res) · line 243
Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +10
-0+ }else if (*fail_res ==WAVE5_SYSERR_VPU_STILL_RUNNING ) { +struct dec_output_info dec_info; + +mutex_unlock (&vpu_dev->hw_lock); +wave5_vpu_dec_get_output_info (inst, &dec_info); + ret =mutex_lock_interruptible (&vpu_dev->hw_lock); +if (ret) { +pm_runtime_put_sync (inst->dev->dev); +return ret; + }
Please try and simplify all the branches. Something like:
if (ret == 0)
break;
if (*fail_res != WAVE5_SYSERR_VPU_STILL_RUNNING) {
dev_warn(inst->dev->dev, "dec_finish_seq timed out\n");
goto unlock_and_return;
}
if (retry++ >= MAX_FIRMWARE_CALL_RETRY) {
ret = -ETIMEDOUT;
goto unlock_and_return;
}
mutex_unlock(&vpu_dev->hw_lock);
wave5_vpu_dec_get_output_info(inst, &dec_info);
ret = mutex_lock_interruptible(&vpu_dev->hw_lock);
if (ret) {
pm_runtime_put_sync(inst->dev->dev);
return ret;
}My impression on that change is that it is independent and could be introduced separately.
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res) · line 256
Quoted 12 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-0} } while (ret != 0);drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-0@@ -249,11 +257,7 @@ int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res) }
wave5_vdi_free_dma_memory(vpu_dev, &p_dec_info->vb_task);drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-5- -list_for_each_entry (inst_elm, &vpu_dev->instances, list) - inst_count++; -if (inst_count ==1 ) -pm_runtime_dont_use_autosuspend (vpu_dev->dev);
This seems like some other fix, that should be in its own commit.
Quoted 17 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +1
-0+mutex_destroy (&inst->feed_lock);
unlock_and_return:
mutex_unlock(&vpu_dev->hw_lock);drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-2@@ -720,8 +724,6 @@ int wave5_vpu_enc_close(struct vpu_instance *inst, u32 *fail_res) int ret; int retry = 0; struct vpu_device *vpu_dev = inst->dev; -int inst_count =0 ; -struct vpu_instance *inst_elm;
*fail_res= 0;
if (!inst->codec_info)drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-0@@ -765,11 +767,6 @@ int wave5_vpu_enc_close(struct vpu_instance *inst, u32 *fail_res)
wave5_vdi_free_dma_memory(vpu_dev, &p_enc_info->vb_task);
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_enc_close(struct vpu_instance *inst, u32 *fail_res) · line 772
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-5-list_for_each_entry (inst_elm, &vpu_dev->instances, list) - inst_count++; -if (inst_count ==1 ) -pm_runtime_dont_use_autosuspend (vpu_dev->dev); -
Same.
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h enum set_param_option { · line 166
Quoted 13 lines
mutex_unlock(&vpu_dev->hw_lock);
pm_runtime_put_sync(inst->dev->dev);drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +0
-0@@ -163,7 +163,7 @@ enum set_param_option {
#defineBUFFER_MARGIN4096drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +1
-1-#define MAX_FIRMWARE_CALL_RETRY 10 +#define MAX_FIRMWARE_CALL_RETRY 30
I thought 10s was already very long, am I right this means it can get stuck for 30s now ?
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h struct vpu_instance { · line 816
Quoted 9 lines
#defineVDI_LITTLE_ENDIAN0x0drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +2
-0@@ -812,6 +812,9 @@ struct vpu_instance { bool cbcr_interleave; bool nv21; bool eos; +bool retry; +int queuing_num;
Some comment would be helpful. What does "retry" stand for, and how is queing_num different from v4l2_m2m_num_src_bufs_ready() ?
Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +1
-0+struct mutex feed_lock;/* lock for feeding bitstream buffers */ struct vpu_buf bitstream_vbuf; dma_addr_t last_rd_ptr; size_t remaining_consumed_bytes;
Signature · 2 lines
Nicolas Dufresne Principal Engineer at Collabora
jackson.lee
maintainerRE: [RESEND PATCH v0 1/3] media: chips-media: wave5: Improve performance of decoder
Hi Nicolas
Thanks for your review.
Can you check my comment for your review?
Quoted 63 lines
-----Original Message-----
From: Nicolas Dufresne <[email protected]>
Sent: Thursday, April 3, 2025 3:03 AM
To: jackson.lee <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; lafley.kim <[email protected]>; [email protected]; [email protected]; Nas Chung <[email protected]>
Subject: Re: [RESEND PATCH v0 1/3] media: chips-media: wave5: Improve performance of decoderLe mercredi 19 mars 2025 à 12:50 +0900, Jackson.lee a écrit :
Quoted 3 lines
From: Jackson Lee <[email protected]>
The existing way for decoding frames was to wait until each frame was
Suggestion:
The current decoding method was to...Quoted 5 lines
decoded after feeding a bitstream. As a result, performance was low and Wave5 could not achieve max pixel processing rate.
Update driver to use an asynchronous approach for decoding and feeding a
Update the driver ..
Quoted 1 line
bitstream in order to achieve full capabilities of the device.
This second part lacks the technical design choices needed for your reviewer ;-P I did read the patch once, and am under the impression that your approach is to not finish the job anymore.
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst,
Quoted 17 lines
Signed-off-by Jackson Lee <[email protected]>Signed-off-by Nas Chung <[email protected]>---
.../chips-media/wave5/wave5-vpu-dec.c | 344
++++++++++--------
.../chips-media/wave5/wave5-vpu-enc.c | 3 -.../platform/chips-media/wave5/wave5-vpuapi.c| 25 +-.../platform/chips-media/wave5/wave5-vpuapi.h| 5 +-4 files changed, 211 insertions(+), 166 deletions(-)drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -230,12 +230,147 @@ static int start_decode(struct vpu_instance *inst,u32
*fail_res)drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)
Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0switch_state(inst, VPU_INST_STATE_STOP); dev_dbg(inst->dev->dev, "%s: pic run failed / finish job",
__func__);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)
Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-1-v4l2_m2m_job_finish (inst->v4l2_m2m_dev, m2m_ctx); } return ret; } +static int write_to_ringbuffer(struct vpu_instance *inst, void *buffer,
size_tbuffer_size,
Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 239
Quoted 1 line
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ struct vpu_buf *ring_buffer, dma_addr_t wr_ptr){
It seems that from now on this needs to be called with the
feed_lock, so add:lockdep_assert_held(&inst->feed_lock);
The "feed_lock" mutex is for protecting "inst->avail_src_bufs" this list, so the feed_lock should be used when "static int fill_ringbuffer(struct vpu_instance *inst)" is called, not the write_to_ringbuffer. So I think the feed_lock should not be added in the write_to_ringbuffer function.
Quoted 20 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+size_t size; +size_t offset = wr_ptr - ring_buffer->daddr; +int ret; + +if (wr_ptr + buffer_size > ring_buffer->daddr + ring_buffer->size){
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, · line 246
Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0+ size = ring_buffer->daddr + ring_buffer->size - wr_ptr; + ret = wave5_vdi_write_memory(inst->dev, ring_buffer, offset,(u8 *)buffer, size);
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)
Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +4
-0+if (ret <0 ) +return ret; + + ret = wave5_vdi_write_memory(inst->dev, ring_buffer, 0, (u8*)buffer + size,
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+ buffer_size - size); +if (ret <0 ) +return ret; + }else { + ret = wave5_vdi_write_memory(inst->dev, ring_buffer, offset,(u8 *)buffer,
Quoted 11 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 263
Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7
-0+ buffer_size); +if (ret <0 ) +return ret; + } + +return 0 ; +}I have a pretty strong preference in seeing code being moved around in seperate patch documented as "non functional changes". This way I know I don't have to review again that code.
I will do it.
Quoted 49 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, · line 264
Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0+ +static int fill_ringbuffer (struct vpu_instance *inst) {Same, I don't want to have to do back and fourth to check if you changed anything. Also, after the refactoring, the lockdeup assert can be added in the new patch, which makes the introduction of the new lock very obvious.
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)
Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +10
-0+struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx; +struct vpu_src_buffer *vpu_buf; +int ret =0 ; + + if (m2m_ctx->last_src_buf) { +struct vpu_src_buffer *vpu_buf = +wave5_to_vpu_src_buf (m2m_ctx->last_src_buf); + +if (vpu_buf->consumed) { + dev_dbg(inst->dev->dev, "last src buffer alreadywritten\n");
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+return 0 ; + } + } + +list_for_each_entry (vpu_buf, &inst->avail_src_bufs, list) {drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)
Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +10
-0+struct vb2_v4l2_buffer *vbuf = &vpu_buf->v4l2_m2m_buf.vb; +struct vpu_buf *ring_buffer = &inst->bitstream_vbuf; +size_t src_size =vb2_get_plane_payload (&vbuf->vb2_buf,0 ); +void *src_buf =vb2_plane_vaddr (&vbuf->vb2_buf,0 ); +dma_addr_t rd_ptr =0 ; +dma_addr_t wr_ptr =0 ; +size_t remain_size =0 ; + +if (vpu_buf->consumed) { + dev_dbg(inst->dev->dev, "already copied src buf (%u)to the ring buffer\n",
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)
Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7
-0+ vbuf->vb2_buf.index); +continue ; + } + +if (!src_buf) { + dev_dbg(inst->dev->dev, + "%s: Acquiring kernel pointer to src buf (%u),fail\n",
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+ __func__, vbuf->vb2_buf.index); +break ; + } + + ret = wave5_vpu_dec_get_bitstream_buffer(inst, &rd_ptr,
&wr_ptr,&remain_size);
Quoted 6 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 304
Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0+if (ret) { +/* Unable to acquire the mutex */ Might want to fix that comment in a separate patch, perhaps even remove that ?
Do you want to remove the comment in a separate patch ?
Quoted 149 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, · line 304
Quoted 1 line
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ dev_err(inst->dev->dev, "Getting the bitstream buffer,fail: %d\n",
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+ ret); +return ret; + } + + dev_dbg(inst->dev->dev, "%s: rd_ptr %pad wr_ptr %pad",
__func__,drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+&rd_ptr, &wr_ptr); + +if (remain_size < src_size) { + dev_dbg(inst->dev->dev, + "%s: remaining size: %zu < source size: %zu forsrc buf (%u)\n",
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)
Quoted 6 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+ __func__, remain_size, src_size, vbuf- vb2_buf.index); +break ; + } + + ret = write_to_ringbuffer(inst, src_buf, src_size,
ring_buffer,wr_ptr);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, · line 322
Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0+if (ret) { + dev_err(inst->dev->dev, "Write src buf (%u) to ringbuffer, fail: %d\n",
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)
Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +8
-0+ vbuf->vb2_buf.index, ret); +return ret; + } + + ret =wave5_vpu_dec_update_bitstream_buffer (inst, src_size); +if (ret) { + dev_dbg(inst->dev->dev, + "update_bitstream_buffer fail: %d for src buf(%u)\n",
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)
Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7
-0+ ret, vbuf->vb2_buf.index); +break ; + } + + vpu_buf->consumed = true; + +/* Don't write buffers passed the last one while draining. */
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, · line 339
Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0+if (v4l2_m2m_is_last_draining_src_buf (m2m_ctx, vbuf)) { + dev_dbg(inst->dev->dev, "last src buffer written tothe ring buffer\n");
Quoted 12 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +12
-0+break ; + } + + inst->queuing_num++; +list_del_init (&vpu_buf->list); +break ; + } + +return ret; +} + +static void wave5_vpu_dec_feed_remaining (struct vpu_instance *inst) {drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 361
Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +10
-0+int ret =0 ; +struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx; +u32 fail_res =0 ; + +mutex_lock (&inst->feed_lock); + ret =fill_ringbuffer (inst); +mutex_unlock (&inst->feed_lock); +if (ret) { +dev_warn (inst->dev->dev,"Filling ring buffer failed\n" ); +return ;I don't think the error handling is correct. What will happen with the job ?
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 364
Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3
-0+ } + + ret =start_decode (inst, &fail_res);I'm a little worried seeing this called outside of
device_run()context.drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)
Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3
-0+if (ret) { + dev_err(inst->dev->dev, + "Frame decoding on m2m context (%p), fail: %d(result: %d)\n",
Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +6
-0+ m2m_ctx, ret, fail_res); + } + +v4l2_m2m_job_finish (inst->v4l2_m2m_dev, m2m_ctx); +} +Quoted 4 lines
static void
flag_last_buffer_done(struct vpu_instance *inst){
structv4l2_m2m_ctx*m2m_ctx=inst->v4l2_fh.m2m_ctx; @@ -347,7 +482,6 @@ static void wave5_vpu_dec_finish_decode(structvpu_instance*inst)
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_finish_decode(struct
Quoted 9 lines
struct
vb2_v4l2_buffer*dec_buf= NULL;
structvb2_v4l2_buffer*disp_buf= NULL; structvb2_queue*dst_vq=v4l2_m2m_get_dst_vq(m2m_ctx);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1-struct queue_status_info q_status;
dev_dbg(inst->dev->dev, "%s: Fetch output info from firmware.", __func__);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -441,20 +575,6 @@ static void wave5_vpu_dec_finish_decode(struct
vpu_instance*inst)drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_finish_decode(struct vpu_instance *inst)
Quoted 6 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-3} spin_unlock_irqrestore(&inst->state_spinlock, flags); } - -/* -* During a resolution change and while draining, the firmware may flush
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_finish_decode(struct · line 447
Quoted 1 line
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1-* the reorder queue regardless of having a matching decoding operation
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_finish_decode(struct vpu_instance *inst)
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-5-* pending. Only terminate the job if there are no more IRQ coming. -*/ -wave5_vpu_dec_give_command (inst,DEC_GET_QUEUE_STATUS , &q_status); - if (q_status.report_queue_count == 0 && - (q_status.instance_queue_count == 0 ||dec_info.sequence_changed)) {
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_finish_decode(struct vpu_instance *inst) · line 457
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-5-dev_dbg (inst->dev->dev,"%s: finishing job.\n" , __func__); -pm_runtime_mark_last_busy (inst->dev->dev); -pm_runtime_put_autosuspend (inst->dev->dev); -v4l2_m2m_job_finish (inst->v4l2_m2m_dev, m2m_ctx); - }This removal is why I believe some seek stress testing are needed. I'll try and find some tests file I crafted last year.
Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0}static int
wave5_vpu_dec_querycap(struct file *file, void *fh, struct v4l2_capability *cap)@@ -794,11 +914,21 @@ static int
wave5_vpu_dec_stop(struct vpu_instance *inst)Quoted 16 lines
}
if (
inst->state!=VPU_INST_STATE_NONE) {drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3
-0+struct vb2_v4l2_buffer *vbuf; +struct vpu_src_buffer *vpu_buf; +/*
* Temporarily release thestate_spinlockso that subsequent
* calls do not block on a mutex while inside this spinlock. */spin_unlock_irqrestore(&inst->state_spinlock, flags);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+ vbuf =v4l2_m2m_last_src_buf (m2m_ctx); +if (vbuf) { + vpu_buf =wave5_to_vpu_src_buf (vbuf); +if (!vpu_buf->consumed) +wave5_vpu_dec_feed_remaining (inst);
Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0+ } +Do you really need to add all these checks ? Shouldn't you just "
feed_remaining()" and move on ? I also don't think its a good idea to callv4l2_m2m_job_finish()in code that can run concurrently with thedevice_run()function. This should only happen if you are certain the VPU instance is stopped already.
While testing fluster, stop command is sometimes called before the last bitstream queued is handled. The stop command sets EOS before handling the last bitstream, so sometimes mismatch happens. I think the code should be add in the wave5_vpu_dec_stop.
Quoted 201 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance
Quoted 5 lines
ret =
wave5_vpu_dec_set_eos_on_firmware(inst); if (ret)
return ret;drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -1116,115 +1246,6 @@ static int wave5_prepare_fb(struct vpu_instance*inst)
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)
Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1return 0; } -static int write_to_ringbuffer(struct vpu_instance *inst, void *buffer,
size_tbuffer_size,drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)
Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-7- struct vpu_buf *ring_buffer, dma_addr_t wr_ptr) -{ -size_t size; -size_t offset = wr_ptr - ring_buffer->daddr; -int ret; - -if (wr_ptr + buffer_size > ring_buffer->daddr + ring_buffer->size){
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance · line 1128
Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-2- size = ring_buffer->daddr + ring_buffer->size - wr_ptr; - ret = wave5_vdi_write_memory(inst->dev, ring_buffer, offset,(u8 *)buffer, size);
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)
Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-4-if (ret <0 ) -return ret; - - ret = wave5_vdi_write_memory(inst->dev, ring_buffer, 0, (u8*)buffer + size,
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-5- buffer_size - size); -if (ret <0 ) -return ret; - }else { - ret = wave5_vdi_write_memory(inst->dev, ring_buffer, offset,(u8 *)buffer,
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-5- buffer_size); -if (ret <0 ) -return ret; - } -drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)
Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-10-return 0 ; -} - -static int fill_ringbuffer(struct vpu_instance *inst) -{ -struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx; -struct v4l2_m2m_buffer *buf, *n; -int ret; - - if (m2m_ctx->last_src_buf) { - struct vpu_src_buffer *vpu_buf =
wave5_to_vpu_src_buf(m2m_ctx->last_src_buf);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)
Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-3- -if (vpu_buf->consumed) { - dev_dbg(inst->dev->dev, "last src buffer alreadywritten\n");
Quoted 6 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-6-return 0 ; - } - } - -v4l2_m2m_for_each_src_buf_safe (m2m_ctx, buf, n) { -struct vb2_v4l2_buffer *vbuf = &buf->vb;drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)
Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-10-struct vpu_src_buffer *vpu_buf =wave5_to_vpu_src_buf (vbuf); -struct vpu_buf *ring_buffer = &inst->bitstream_vbuf; -size_t src_size =vb2_get_plane_payload (&vbuf->vb2_buf,0 ); -void *src_buf =vb2_plane_vaddr (&vbuf->vb2_buf,0 ); -dma_addr_t rd_ptr =0 ; -dma_addr_t wr_ptr =0 ; -size_t remain_size =0 ; - -if (vpu_buf->consumed) { - dev_dbg(inst->dev->dev, "already copied src buf (%u)to the ring buffer\n",
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)
Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-7- vbuf->vb2_buf.index); -continue ; - } - -if (!src_buf) { - dev_dbg(inst->dev->dev, - "%s: Acquiring kernel pointer to src buf (%u),fail\n",
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-5- __func__, vbuf->vb2_buf.index); -break ; - } - - ret = wave5_vpu_dec_get_bitstream_buffer(inst, &rd_ptr,
&wr_ptr,&remain_size);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)
Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-3-if (ret) { -/* Unable to acquire the mutex */ - dev_err(inst->dev->dev, "Getting the bitstream buffer,fail: %d\n",
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-5- ret); -return ret; - } - - dev_dbg(inst->dev->dev, "%s: rd_ptr %pad wr_ptr %pad",
__func__,&rd_ptr,&wr_ptr);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)
Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-4- -if (remain_size < src_size) { - dev_dbg(inst->dev->dev, - "%s: remaining size: %zu < source size: %zu forsrc buf (%u)\n",
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)
Quoted 6 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-5- __func__, remain_size, src_size, vbuf- vb2_buf.index); -break ; - } - - ret = write_to_ringbuffer(inst, src_buf, src_size,
ring_buffer,wr_ptr);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance · line 1203
Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-2-if (ret) { - dev_err(inst->dev->dev, "Write src buf (%u) to ringbuffer, fail: %d\n",
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)
Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-8- vbuf->vb2_buf.index, ret); -return ret; - } - - ret =wave5_vpu_dec_update_bitstream_buffer (inst, src_size); -if (ret) { - dev_dbg(inst->dev->dev, - "update_bitstream_buffer fail: %d for src buf(%u)\n",
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)
Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-7- ret, vbuf->vb2_buf.index); -break ; - } - - vpu_buf->consumed = true; - -/* Don't write buffers passed the last one while draining. */
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance · line 1220
Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-2-if (v4l2_m2m_is_last_draining_src_buf (m2m_ctx, vbuf)) { - dev_dbg(inst->dev->dev, "last src buffer written tothe ring buffer\n");
Quoted 11 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-7-break ; - } - } - -return 0 ; -} - static void wave5_vpu_dec_buf_queue_src(struct vb2_buffer *vb) {struct
vpu_instance*inst =vb2_get_drv_priv(vb->vb2_queue); @@ -1236,6 +1257,11 @@ static void wave5_vpu_dec_buf_queue_src(struct
vb2_buffer*vb)drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_buf_queue_src(struct vb2_buffer *vb) · line 1264
Quoted 8 lines
vbuf->sequence= inst->queued_src_buf_num++;
v4l2_m2m_buf_queue(m2m_ctx, vbuf);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+ +INIT_LIST_HEAD (&vpu_buf->list); +mutex_lock (&inst->feed_lock); +list_add_tail (&vpu_buf->list, &inst->avail_src_bufs); +mutex_unlock (&inst->feed_lock);This lack documentation, you didn't even mention the introduction of a new src buf list in the commit message.
Quoted 4 lines
}
static void
wave5_vpu_dec_buf_queue_dst(struct vb2_buffer *vb)@@ -1362,7 +1388,6 @@ static int wave5_vpu_dec_start_streaming(struct
vb2_queue*q, unsigned int countQuoted 20 lines
goto
return_buffers; }
}
-
}pm_runtime_mark_last_busy(inst->dev->dev);pm_runtime_put_autosuspend(inst->dev->dev);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7
-0@@ -1385,6 +1410,13 @@ static int streamoff_output(struct vb2_queue *q) dma_addr_t new_rd_ptr; struct dec_output_info dec_info; unsigned int i; +struct vpu_src_buffer *vpu_buf, *tmp; + + inst->retry = false; + inst->queuing_num =0 ; + +list_for_each_entry_safe (vpu_buf, tmp, &inst->avail_src_bufs, list) +list_del_init (&vpu_buf->list); for (i = 0; i < v4l2_m2m_num_dst_bufs_ready(m2m_ctx); i++) {Quoted 1 line
ret =
wave5_vpu_dec_set_disp_flag(inst, i); @@ -1481,10+1513,8 @@
Quoted 3 lines
static void
wave5_vpu_dec_stop_streaming(struct vb2_queue *q)wave5_vpu_dec_give_command(inst,
DEC_GET_QUEUE_STATUS,
&q_status);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_stop_streaming(struct vb2_queue *q)
Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-4-if (q_status.report_queue_count ==0 ) -break ; - -if (wave5_vpu_wait_interrupt (inst,VPU_DEC_TIMEOUT ) <0 ) + if ((inst->state == VPU_INST_STATE_STOP || +q_status.instance_queue_count == 0) &&State should be read with appropriate locking.
Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ q_status.report_queue_count == 0)break;
This loop have nothing to wait on anymore. It just actively pool on the HW. I noticed your third patch, but if this is what you address there, squash it somehow, this would break bisects.
Quoted 4 lines
The goal of that code was to ensure that this code will never run concurrently to
finish_decode()once out of that while loop. I don't think you break that, but that will cause possibly high CPU load.
The code is not related with the third patch, there was sometimes HW interrupt misses since changing to asynchronous, so an infinite wait happened when closed So I removed it, but I think there is no high CPU load because it runs very short time.
Quoted 29 lines
Quoted 2 lines
if (
wave5_vpu_dec_get_output_info(inst, &dec_output_info))@@
Quoted 7 lines
-1577,13 +1607,23 @@ static void
wave5_vpu_dec_device_run(void *priv)structqueue_status_infoq_status;
u32fail_res= 0;
int ret = 0;drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+unsigned long flags;dev_dbg(inst->dev->dev, "%s: Fill the ring buffer with new
bitstream data",
__func__);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_device_run(void *priv)
Quoted 11 lines
pm_runtime_resume_and_get(inst->dev->dev);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +6
-4- ret =fill_ringbuffer (inst); -if (ret) { -dev_warn (inst->dev->dev,"Filling ring buffer failed\n" ); - goto finish_job_and_return; +if (!inst->retry) { +mutex_lock (&inst->feed_lock); + ret =fill_ringbuffer (inst); +mutex_unlock (&inst->feed_lock); +if (ret <0 ) { + dev_warn(inst->dev->dev, "Filling ring bufferfailed\n");
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_device_run(void *priv)
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+ goto finish_job_and_return; + }else if (!inst->eos && + inst->queuing_num == 0 && + inst->state == VPU_INST_STATE_PIC_RUN) { + dev_dbg(inst->dev->dev, "%s: no bitstream for feeding,so skip ",
__func__);
Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0+ goto finish_job_and_return; + }There is quite some overlap between that local src buffer queue and all the code that depends on
v4l2_m2m_num_src_bufs_ready(). Are you sure you aren't re-implementing something ? Also, this code belong to thePIC_RUNcase in the below switch.
After src buffer was queued, v4l2_m2m_num_src_bufs_ready() always returns true until src buffer was removed in the finish_decode since get_result. So checking if there is buffer or not in the src bufs list was needed in the device_run.
Quoted 251 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_device_run(void *priv) · line 1664
Quoted 10 lines
}
switch (
inst->state) {drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0@@ -1619,7 +1659,9 @@ static void wave5_vpu_dec_device_run(void *priv)* we had a chance to switch, which leads to an invalid state * change. */ +spin_lock_irqsave (&inst->state_spinlock, flags); switch_state(inst, VPU_INST_STATE_PIC_RUN); +spin_unlock_irqrestore (&inst->state_spinlock, flags);Should be its own patch, and fixed everywhere. Add missing lockdep assert (if there is a spinlock version of it).
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_device_run(void *priv) · line 1666
Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0/** During DRC, the picture decoding remains pending, so just leave
Quoted 1 line
the job @@ -1634,12 +1676,14 @@ static void
wave5_vpu_dec_device_run(void *priv)Quoted 3 lines
ret =
wave5_prepare_fb(inst);
if (ret) {
dev_warn(inst->dev->dev, "Framebuffer preparation,fail: %d\n",
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
Quoted 4 lines
ret);
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+spin_lock_irqsave (&inst->state_spinlock, flags);
switch_state(inst, VPU_INST_STATE_STOP);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+spin_unlock_irqrestore (&inst->state_spinlock, flags);Same, this is separate fix.
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
Quoted 5 lines
break;
}if (q_status.instance_queue_count) {
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1- dev_dbg(inst->dev->dev, "%s: leave with active job",
__func__);Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+v4l2_m2m_job_finish (inst->v4l2_m2m_dev, m2m_ctx);return;
}drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -1650,14 +1694,22 @@ static void wave5_vpu_dec_device_run(void *priv) dev_err(inst->dev->dev,"Frame decoding on m2m context (%p), fail: %d
(result: %d)\n",
Quoted 12 lines
m2m_ctx, ret,fail_res);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +9
-1-break ; + goto finish_job_and_return; + } + +if (fail_res ==WAVE5_SYSERR_QUEUEING_FAIL ) { + inst->retry = true; + }else { + inst->retry = false; +if (!inst->eos) + inst->queuing_num--;}
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-2-/* Return so that we leave this job active */ - dev_dbg(inst->dev->dev, "%s: leave with active job",
__func__);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_device_run(void *priv)
Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-3-return ; -default : - WARN(1, "Execution of a job in state %s illegal.\n",
state_to_str(inst->state));drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance *inst,
Quoted 61 lines
break;
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+default : +if (!v4l2_m2m_has_stopped (m2m_ctx)) +WARN (1 ,"Execution of a job in state %s illegal.\n" , + state_to_str(inst->state)); +return ;}
finish_job_and_return:
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-4@@ -1676,10 +1728,7 @@ static void wave5_vpu_dec_job_abort(void *priv) if (ret) return; - ret =wave5_vpu_dec_set_eos_on_firmware (inst); -if (ret) -dev_warn (inst->dev->dev, -"Setting EOS for the bitstream, fail: %d\n" , ret); +v4l2_m2m_job_finish (inst->v4l2_m2m_dev, inst->v4l2_fh.m2m_ctx); }static int
wave5_vpu_dec_job_ready(void *priv)@@ -1755,6 +1804,8 @@ static intwave5_vpu_open_dec(struct file *filp)
inst->ops=&wave5_vpu_dec_inst_ops;
spin_lock_init(&inst->state_spinlock);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0+mutex_init (&inst->feed_lock); +INIT_LIST_HEAD (&inst->avail_src_bufs);
inst->codec_info=kzalloc(sizeof(*inst->codec_info), GFP_KERNEL); if (!inst->codec_info)drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-3@@ -1830,9 +1881,6 @@ static int wave5_vpu_open_dec(struct file *filp) if (ret) goto cleanup_inst; -if (list_empty (&dev->instances)) -pm_runtime_use_autosuspend (inst->dev->dev); - list_add_tail(&inst->list, &dev->instances); mutex_unlock(&dev->dev_lock);drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c +0
-3@@ -1774,9 +1774,6 @@ static int wave5_vpu_open_enc(struct file *filp) if (ret) goto cleanup_inst; -if (list_empty (&dev->instances)) -pm_runtime_use_autosuspend (inst->dev->dev); - list_add_tail(&inst->list, &dev->instances); mutex_unlock(&dev->dev_lock);drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-0@@ -207,8 +207,6 @@ int wave5_vpu_dec_close(struct vpu_instance *inst,u32
*fail_res)drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance *inst,
Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-2int retry = 0; struct vpu_device *vpu_dev = inst->dev; int i; -int inst_count =0 ; -struct vpu_instance *inst_elm; *fail_res = 0; if (!inst->codec_info)@@ -233,6 +231,16 @@ int wave5_vpu_dec_close(struct vpu_instance *inst,u32
*fail_res)drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res) · line 243
Quoted 13 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +10
-0retry++ >= MAX_FIRMWARE_CALL_RETRY) { ret = -ETIMEDOUT; goto unlock_and_return; + } else if (*fail_res == WAVE5_SYSERR_VPU_STILL_RUNNING) { +struct dec_output_info dec_info; + +mutex_unlock (&vpu_dev->hw_lock); +wave5_vpu_dec_get_output_info (inst, &dec_info); + ret =mutex_lock_interruptible (&vpu_dev->hw_lock); +if (ret) { +pm_runtime_put_sync (inst->dev->dev); +return ret; + }Please try and simplify all the branches. Something like:
if (ret == 0) break; if (*fail_res != WAVE5_SYSERR_VPU_STILL_RUNNING) { dev_warn(inst->dev->dev, "dec_finish_seq timed out\n"); goto unlock_and_return; } if (retry++ >= MAX_FIRMWARE_CALL_RETRY) { ret = -ETIMEDOUT; goto unlock_and_return; } mutex_unlock(&vpu_dev->hw_lock); wave5_vpu_dec_get_output_info(inst, &dec_info); ret = mutex_lock_interruptible(&vpu_dev->hw_lock); if (ret) { pm_runtime_put_sync(inst->dev->dev); return ret; }My impression on that change is that it is independent and could be introduced separately.
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance *inst,
Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-0} } while (ret != 0);@@ -249,11 +257,7 @@ int wave5_vpu_dec_close(struct vpu_instance *inst,u32
*fail_res)drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res) · line 256
Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-5} wave5_vdi_free_dma_memory(vpu_dev, &p_dec_info->vb_task); - -list_for_each_entry (inst_elm, &vpu_dev->instances, list) - inst_count++; -if (inst_count ==1 ) -pm_runtime_dont_use_autosuspend (vpu_dev->dev);This seems like some other fix, that should be in its own commit.
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_enc_close(struct vpu_instance *inst,
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +1
-0+mutex_destroy (&inst->feed_lock);unlock_and_return:
mutex_unlock(&vpu_dev->hw_lock);drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-0@@ -720,8 +724,6 @@ int wave5_vpu_enc_close(struct vpu_instance *inst,u32
*fail_res)Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-2int ret; int retry = 0; struct vpu_device *vpu_dev = inst->dev; -int inst_count =0 ; -struct vpu_instance *inst_elm; *fail_res = 0; if (!inst->codec_info)drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_enc_close(struct vpu_instance · line 773
Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-5@@ -765,11 +767,6 @@ int wave5_vpu_enc_close(struct vpu_instance *inst, u32 *fail_res) wave5_vdi_free_dma_memory(vpu_dev, &p_enc_info->vb_task); -list_for_each_entry (inst_elm, &vpu_dev->instances, list) - inst_count++; -if (inst_count ==1 ) -pm_runtime_dont_use_autosuspend (vpu_dev->dev); -Same.
Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-0mutex_unlock(&vpu_dev->hw_lock); pm_runtime_put_sync(inst->dev->dev);drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +0
-0drivers/media/platform/chips-media/wave5/wave5-vpuapi.h enum set_param_option { · line 166
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +1
-1@@ -163,7 +163,7 @@ enum set_param_option { #define BUFFER_MARGIN 4096 -#define MAX_FIRMWARE_CALL_RETRY 10 +#define MAX_FIRMWARE_CALL_RETRY 30I thought 10s was already very long, am I right this means it can get stuck for 30s now ?
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h struct vpu_instance { · line 816
Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +2
-0#define VDI_LITTLE_ENDIAN 0x0@@ -812,6 +812,9 @@ struct vpu_instance { bool cbcr_interleave; bool nv21; bool eos; +bool retry; +int queuing_num;Some comment would be helpful. What does "retry" stand for, and how is
queing_numdifferent fromv4l2_m2m_num_src_bufs_ready()?
Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h struct vpu_instance { · line 820
Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +1
-0+struct mutex feed_lock;/* lock for feeding bitstream buffers */ struct vpu_buf bitstream_vbuf; dma_addr_t last_rd_ptr; size_t remaining_consumed_bytes;--
Nicolas Dufresne
Principal Engineer at Collabora
Thanks
Jackson
Re: [RESEND PATCH v0 1/3] media: chips-media: wave5: Improve performance of decoder
Hi,
Le vendredi 04 avril 2025 à 04:16 +0000, jackson.lee a écrit :
Quoted 93 lines
Hi Nicolas
Thanks for your review.
Can you check my comment for your review?Quoted 79 lines
-----Original Message-----
From: Nicolas Dufresne <[email protected]>
Sent: Thursday, April 3, 2025 3:03 AM
To: jackson.lee <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; lafley.kim
<[email protected]>; [email protected]; [email protected]; Nas
Chung <[email protected]>
Subject: Re: [RESEND PATCH v0 1/3] media: chips-media: wave5:
Improve
performance of decoderLe mercredi 19 mars 2025 à 12:50 +0900, Jackson.lee a écrit :
Quoted 4 lines
From: Jackson Lee <[email protected]>
The existing way for decoding frames was to wait until each frame was
Suggestion:
The current decoding method was to...Quoted 7 lines
decoded after feeding a bitstream. As a result, performance was low
and Wave5 could not achieve max pixel processing rate.Update driver to use an asynchronous approach for decoding and feeding
aUpdate the driver ..
Quoted 1 line
bitstream in order to achieve full capabilities of the device.
This second part lacks the technical design choices needed for your reviewer ;-P I did read the patch once, and am under the impression that
your approach is to not finish the job anymore.Quoted 18 lines
Signed-off-by Jackson Lee <[email protected]>Signed-off-by Nas Chung <[email protected]>---
.../chips-media/wave5/wave5-vpu-dec.c | 344
++++++++++--------
.../chips-media/wave5/wave5-vpu-enc.c | 3 -.../platform/chips-media/wave5/wave5-vpuapi.c| 25 +-.../platform/chips-media/wave5/wave5-vpuapi.h| 5 +-4 files changed, 211 insertions(+), 166 deletions(-)(quoted diff) +0
-0
dec.cb/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.cindexfd71f0c43ac3..cc47da509703100644drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct · line 230
Quoted 1 line
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -230,12 +230,147 @@ static int start_decode(struct vpu_instance *inst,u32
*fail_res)Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0switch_state(inst, VPU_INST_STATE_STOP);dev_dbg(inst->dev->dev, "%s: pic run failed / finish job",
__func__);Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1- v4l2_m2m_job_finish(inst->v4l2_m2m_dev,
m2m_ctx);
}return ret;
}+static int write_to_ringbuffer(struct
vpu_instance*inst, void *buffer,
size_tbuffer_size,Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ struct vpu_buf *ring_buffer,
dma_addr_twr_ptr){
Quoted 6 lines
It seems that from now on this needs to be called with the
feed_lock, so
add:lockdep_assert_held(&inst->feed_lock);
Quoted 6 lines
The "
feed_lock" mutex is for protecting "inst->avail_src_bufs" this list, so thefeed_lockshould be used when "static intfill_ringbuffer(struct vpu_instance *inst)" is called, not thewrite_to_ringbuffer.
So I think thefeed_lockshould not be added in thewrite_to_ringbufferfunction.
This is called from "fill_ringbuffer()" and both calls to that seems to take that lock, my comment was just to make it explicit with lockdep. This is called from CMD_STOP and from device_run() which can be concurrent. I think the code effectively protecting the ring buffer state with.
+ mutex_lock(&inst->feed_lock);
+ ret = fill_ringbuffer(inst);
+ mutex_unlock(&inst->feed_lock);
note that there is bunch of random locking too in the abstraction, this driver will need some cleanup in the long run imho.
Quoted 114 lines
Quoted 28 lines
Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+size_t size; +size_t offset = wr_ptr - ring_buffer->daddr; +int ret; + + if (wr_ptr + buffer_size > ring_buffer->daddr +
ring_buffer->size){
Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ size = ring_buffer->daddr + ring_buffer->size -
wr_ptr;drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ ret = wave5_vdi_write_memory(inst->dev,
ring_buffer, offset,(u8 *)buffer, size);
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +4
-0+if (ret <0 ) +return ret; + + ret = wave5_vdi_write_memory(inst->dev,
ring_buffer, 0, (u8*)buffer + size,
Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ buffer_size -size);
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +4
-0+if (ret <0 ) +return ret; + }else { + ret = wave5_vdi_write_memory(inst->dev,
ring_buffer, offset,(u8 *)buffer,
Quoted 13 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 263
Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7
-0+ buffer_size); +if (ret <0 ) +return ret; + } + +return 0 ; +}I have a pretty strong preference in seeing code being moved around in
seperate patch documented as "non functional changes". This way I know I
don't have to review again that code.I will do it.
Quoted 61 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0+ +static int fill_ringbuffer (struct vpu_instance *inst) {Same, I don't want to have to do back and fourth to check if you changed
anything. Also, after the refactoring, the lockdeup assert can be added in
the new patch, which makes the introduction of the new lock very obvious.Quoted 11 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +10
-0+struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx; +struct vpu_src_buffer *vpu_buf; +int ret =0 ; + + if (m2m_ctx->last_src_buf) { +struct vpu_src_buffer *vpu_buf = +wave5_to_vpu_src_buf (m2m_ctx->last_src_buf); + +if (vpu_buf->consumed) { + dev_dbg(inst->dev->dev, "last src bufferalready
written\n");
Quoted 20 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+return 0 ; + } + } + + list_for_each_entry(vpu_buf, &inst->avail_src_bufs,list) {
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ struct vb2_v4l2_buffer *vbuf = &vpu_buf-Quoted 1 line
v4l2_m2m_buf.vb;
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ struct vpu_buf *ring_buffer = &inst-Quoted 1 line
bitstream_vbuf;drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ size_t src_size = vb2_get_plane_payload(&vbuf-Quoted 1 line
vb2_buf, 0);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ void *src_buf = vb2_plane_vaddr(&vbuf->vb2_buf,0);
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +6
-0+dma_addr_t rd_ptr =0 ; +dma_addr_t wr_ptr =0 ; +size_t remain_size =0 ; + +if (vpu_buf->consumed) { + dev_dbg(inst->dev->dev, "already copiedQuoted 1 line
src buf (%u)
to the ring buffer\n",
Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7
-0+ vbuf->vb2_buf.index); +continue ; + } + +if (!src_buf) { + dev_dbg(inst->dev->dev, + "%s: Acquiring kernel pointer tosrc buf (%u),
fail\n",
Quoted 6 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+ __func__, vbuf->vb2_buf.index); +break ; + } + + ret = wave5_vpu_dec_get_bitstream_buffer(inst,
&rd_ptr,
&wr_ptr,&remain_size);Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 304
Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0+if (ret) { +/* Unable to acquire the mutex */ Might want to fix that comment in a separate patch, perhaps even remove
that ?
Quoted 1 line
Do you want to remove the comment in a separate patch ?
Yes please.
Quoted 198 lines
Quoted 188 lines
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ dev_err(inst->dev->dev, "Getting thebitstream buffer,
fail: %d\n",
Quoted 6 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+ ret); +return ret; + } + + dev_dbg(inst->dev->dev, "%s: rd_ptr %pad wr_ptr%pad",
__func__,Quoted 6 lines
+&rd_ptr,
&wr_ptr);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +4
-0+ +if (remain_size < src_size) { + dev_dbg(inst->dev->dev, + "%s: remaining size: %zu <source size: %zu for
src buf (%u)\n",
Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ __func__, remain_size, src_size,vbuf-
vb2_buf.index);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +4
-0+break ; + } + + ret = write_to_ringbuffer(inst, src_buf,
src_size,
ring_buffer,wr_ptr);Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0+if (ret) { + dev_err(inst->dev->dev, "Write src buf(%u) to ring
buffer, fail: %d\n",
Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+ vbuf->vb2_buf.index, ret); +return ret; + } + + ret =
wave5_vpu_dec_update_bitstream_buffer(inst, src_size);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3
-0+if (ret) { + dev_dbg(inst->dev->dev, + "update_bitstream_buffer fail:%d for src buf
(%u)\n",
Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7
-0+ ret, vbuf->vb2_buf.index); +break ; + } + + vpu_buf->consumed = true; + +/* Don't write buffers passed the last one while draining.
*/
Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ if (v4l2_m2m_is_last_draining_src_buf(m2m_ctx,vbuf)) {
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ dev_dbg(inst->dev->dev, "last src bufferwritten to
the ring buffer\n");
Quoted 23 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +12
-0+break ; + } + + inst->queuing_num++; +list_del_init (&vpu_buf->list); +break ; + } + +return ret; +} + +static void wave5_vpu_dec_feed_remaining(struct vpu_instance*inst) {
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +9
-0+int ret =0 ; +struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx; +u32 fail_res =0 ; + +mutex_lock (&inst->feed_lock); + ret =fill_ringbuffer (inst); +mutex_unlock (&inst->feed_lock); +if (ret) { + dev_warn(inst->dev->dev, "Filling ring bufferfailed\n");
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
Quoted 1 line
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+return ;I don't think the error handling is correct. What will happen with the
job ?drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 364
Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3
-0+ } + + ret =start_decode (inst, &fail_res);I'm a little worried seeing this called outside of
device_run()context.Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3
-0+if (ret) { + dev_err(inst->dev->dev, + "Frame decoding on m2m context (%p),fail: %d
(result: %d)\n",
Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +6
-0+ m2m_ctx, ret, fail_res); + } + +v4l2_m2m_job_finish (inst->v4l2_m2m_dev, m2m_ctx); +} +Quoted 6 lines
static void
flag_last_buffer_done(struct vpu_instance *inst){
structv4l2_m2m_ctx*m2m_ctx=inst->v4l2_fh.m2m_ctx; @@ -347,7
+482,6 @@ static void wave5_vpu_dec_finish_decode(structvpu_instance*inst)
Quoted 11 lines
struct
vb2_v4l2_buffer*dec_buf= NULL;
structvb2_v4l2_buffer*disp_buf= NULL; structvb2_queue*dst_vq=v4l2_m2m_get_dst_vq(m2m_ctx);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1-struct queue_status_info q_status;dev_dbg(inst->dev->dev, "%s: Fetch output info from firmware.",
__func__);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -441,20 +575,6 @@ static voidwave5_vpu_dec_finish_decode(struct
vpu_instance*inst)Quoted 8 lines
}
spin_unlock_irqrestore(&inst->state_spinlock, flags);
}drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-3- -/* -* During a resolution change and while draining, the firmware may
flush
Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1-* the reorder queue regardless of having a matching decoding
operation
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1-* pending. Only terminate the job if there are no more IRQ coming.
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-2-*/ - wave5_vpu_dec_give_command(inst, DEC_GET_QUEUE_STATUS,
&q_status);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-2- if (q_status.report_queue_count == 0 && - (q_status.instance_queue_count == 0 ||dec_info.sequence_changed)) {
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1- dev_dbg(inst->dev->dev, "%s: finishing job.\n",
__func__);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-3-pm_runtime_mark_last_busy (inst->dev->dev); -pm_runtime_put_autosuspend (inst->dev->dev); - v4l2_m2m_job_finish(inst->v4l2_m2m_dev,
m2m_ctx);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1- }This removal is why I believe some seek stress testing are needed. I'll
try and find some tests file I crafted last year.Quoted 5 lines
}
static int wave5_vpu_dec_querycap(struct file *file, void *fh, struct
v4l2_capability*cap) @@ -794,11 +914,21 @@ static int
wave5_vpu_dec_stop(struct vpu_instance *inst)Quoted 20 lines
}
if (
inst->state!=VPU_INST_STATE_NONE) {drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3
-0+struct vb2_v4l2_buffer *vbuf; +struct vpu_src_buffer *vpu_buf; +/*
* Temporarily release thestate_spinlockso that subsequent
* calls do not block on a mutex while inside this spinlock.
*/spin_unlock_irqrestore(&inst->state_spinlock, flags);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+ vbuf =v4l2_m2m_last_src_buf (m2m_ctx); +if (vbuf) { + vpu_buf =wave5_to_vpu_src_buf (vbuf); +if (!vpu_buf->consumed) + wave5_vpu_dec_feed_remaining(inst);
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0+ } +Quoted 9 lines
Do you really need to add all these checks ? Shouldn't you just "
feed_remaining()" and move on ? I also don't think its a good idea to
callv4l2_m2m_job_finish()in code that can run concurrently with thedevice_run()function. This should only happen if you are certain the VPU
instance is stopped already.
Quoted 5 lines
While testing fluster, stop command is sometimes called before the last bitstream queued is handled.
The stop command sets EOS before handling the last bitstream, so sometimes mismatch happens.
I think the code should be add in thewave5_vpu_dec_stop.
This would be a regression, we took good care of handling before and after case correctly.
Quoted 251 lines
Quoted 244 lines
Quoted 8 lines
ret =
wave5_vpu_dec_set_eos_on_firmware(inst); if (ret)
return ret;drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -1116,115 +1246,6 @@ static int wave5_prepare_fb(struct
vpu_instance*inst)
Quoted 5 lines
return 0;
}-static int write_to_ringbuffer(struct
vpu_instance*inst, void *buffer,
size_tbuffer_size,Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1- struct vpu_buf *ring_buffer,
dma_addr_twr_ptr)drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-6-{ - size_t size; - size_t offset = wr_ptr - ring_buffer->daddr; - int ret; - - if (wr_ptr + buffer_size > ring_buffer->daddr +
ring_buffer->size){
Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1- size = ring_buffer->daddr + ring_buffer->size -
wr_ptr;drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1- ret = wave5_vdi_write_memory(inst->dev,
ring_buffer, offset,(u8 *)buffer, size);
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-4-if (ret <0 ) -return ret; - - ret = wave5_vdi_write_memory(inst->dev,
ring_buffer, 0, (u8*)buffer + size,
Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1- buffer_size -size);
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-4-if (ret <0 ) -return ret; - }else { - ret = wave5_vdi_write_memory(inst->dev,
ring_buffer, offset,(u8 *)buffer,
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-5- buffer_size); -if (ret <0 ) -return ret; - } -drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)
Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-10-return 0 ; -} - -static int fill_ringbuffer(struct vpu_instance *inst) -{ -struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx; -struct v4l2_m2m_buffer *buf, *n; -int ret; - - if (m2m_ctx->last_src_buf) { - struct vpu_src_buffer *vpu_buf =
wave5_to_vpu_src_buf(m2m_ctx->last_src_buf);Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-3- -if (vpu_buf->consumed) { - dev_dbg(inst->dev->dev, "last src bufferalready
written\n");
Quoted 20 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-7-return 0 ; - } - } - -v4l2_m2m_for_each_src_buf_safe (m2m_ctx, buf, n) { -struct vb2_v4l2_buffer *vbuf = &buf->vb; - struct vpu_src_buffer *vpu_buf =
wave5_to_vpu_src_buf(vbuf);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1- struct vpu_buf *ring_buffer = &inst-Quoted 1 line
bitstream_vbuf;drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1- size_t src_size = vb2_get_plane_payload(&vbuf-Quoted 1 line
vb2_buf, 0);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1- void *src_buf = vb2_plane_vaddr(&vbuf->vb2_buf,0);
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-6-dma_addr_t rd_ptr =0 ; -dma_addr_t wr_ptr =0 ; -size_t remain_size =0 ; - -if (vpu_buf->consumed) { - dev_dbg(inst->dev->dev, "already copiedQuoted 1 line
src buf (%u)
to the ring buffer\n",
Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-7- vbuf->vb2_buf.index); -continue ; - } - -if (!src_buf) { - dev_dbg(inst->dev->dev, - "%s: Acquiring kernel pointer tosrc buf (%u),
fail\n",
Quoted 6 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-5- __func__, vbuf->vb2_buf.index); -break ; - } - - ret = wave5_vpu_dec_get_bitstream_buffer(inst,
&rd_ptr,
&wr_ptr,&remain_size);Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-3-if (ret) { -/* Unable to acquire the mutex */ - dev_err(inst->dev->dev, "Getting thebitstream buffer,
fail: %d\n",
Quoted 6 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-5- ret); -return ret; - } - - dev_dbg(inst->dev->dev, "%s: rd_ptr %pad wr_ptr%pad",
__func__,&rd_ptr,&wr_ptr);Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-4- -if (remain_size < src_size) { - dev_dbg(inst->dev->dev, - "%s: remaining size: %zu <source size: %zu for
src buf (%u)\n",
Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1- __func__, remain_size, src_size,vbuf-
vb2_buf.index);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-4-break ; - } - - ret = write_to_ringbuffer(inst, src_buf,
src_size,
ring_buffer,wr_ptr);Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-2-if (ret) { - dev_err(inst->dev->dev, "Write src buf(%u) to ring
buffer, fail: %d\n",
Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-5- vbuf->vb2_buf.index, ret); -return ret; - } - - ret =
wave5_vpu_dec_update_bitstream_buffer(inst, src_size);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-3-if (ret) { - dev_dbg(inst->dev->dev, - "update_bitstream_buffer fail:%d for src buf
(%u)\n",
Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-7- ret, vbuf->vb2_buf.index); -break ; - } - - vpu_buf->consumed = true; - -/* Don't write buffers passed the last one while draining.
*/
Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1- if (v4l2_m2m_is_last_draining_src_buf(m2m_ctx,vbuf)) {
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1- dev_dbg(inst->dev->dev, "last src bufferwritten to
the ring buffer\n");
Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-7-break ; - } - } - -return 0 ; -} -static void
wave5_vpu_dec_buf_queue_src(struct vb2_buffer *vb){
structvpu_instance*inst = vb2_get_drv_priv(vb-Quoted 3 lines
Quoted 1 line
vb2_queue); @@-1236,6 +1257,11 @@ static void
wave5_vpu_dec_buf_queue_src(struct
vb2_buffer*vb)drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_buf_queue_src(struct vb2_buffer *vb) · line 1264
Quoted 8 lines
vbuf->sequence= inst->queued_src_buf_num++;
v4l2_m2m_buf_queue(m2m_ctx, vbuf);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+ +INIT_LIST_HEAD (&vpu_buf->list); +mutex_lock (&inst->feed_lock); +list_add_tail (&vpu_buf->list, &inst->avail_src_bufs); +mutex_unlock (&inst->feed_lock);This lack documentation, you didn't even mention the introduction of a new
src buf list in the commit message.Quoted 6 lines
}
static void
wave5_vpu_dec_buf_queue_dst(struct vb2_buffer *vb)@@
-1362,7 +1388,6 @@ static int wave5_vpu_dec_start_streaming(struct
vb2_queue*q, unsigned int countQuoted 21 lines
goto
return_buffers; }
}
-
}pm_runtime_mark_last_busy(inst->dev->dev);pm_runtime_put_autosuspend(inst->dev->dev);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7
-0@@ -1385,6 +1410,13 @@ static int streamoff_output(struct vb2_queue *q) dma_addr_t new_rd_ptr; struct dec_output_info dec_info; unsigned int i; +struct vpu_src_buffer *vpu_buf, *tmp; + + inst->retry = false; + inst->queuing_num =0 ; + +list_for_each_entry_safe (vpu_buf, tmp, &inst- >avail_src_bufs, list) +list_del_init (&vpu_buf->list);Quoted 4 lines
for (i = 0; i <
v4l2_m2m_num_dst_bufs_ready(m2m_ctx); i++) {
ret =wave5_vpu_dec_set_disp_flag(inst, i); @@ - 1481,10+1513,8 @@
Quoted 4 lines
static void
wave5_vpu_dec_stop_streaming(struct vb2_queue *q)wave5_vpu_dec_give_command(inst,
DEC_GET_QUEUE_STATUS,
&q_status);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-4-if (q_status.report_queue_count ==0 ) -break ; - - if (wave5_vpu_wait_interrupt(inst,
VPU_DEC_TIMEOUT) < 0)drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0+ if ((inst->state == VPU_INST_STATE_STOP || +q_status.instance_queue_count == 0) &&State should be read with appropriate locking.
Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ q_status.report_queue_count == 0)break;
This loop have nothing to wait on anymore. It just actively pool on the HW.
I noticed your third patch, but if this is what you address there, squash
it somehow, this would break bisects.Quoted 5 lines
The goal of that code was to ensure that this code will never run concurrently to
finish_decode()once out of that while loop. I don't think
you break that, but that will cause possibly high CPU load.
Quoted 5 lines
The code is not related with the third patch, there was sometimes HW interrupt misses since changing to asynchronous, so an infinite wait happened when closed
So I removed it, but I think there is no high CPU load because it runs very short time.
If the firmware get stuck, Linux can spin, this cannot be done that way I'm sorry. That its most of the time spinning only few times does not seems like valid justification. We need a limit, we need some preemption, etc.
Quoted 50 lines
Quoted 37 lines
Quoted 4 lines
if (
wave5_vpu_dec_get_output_info(inst, &dec_output_info))@@
Quoted 9 lines
-1577,13 +1607,23 @@ static void
wave5_vpu_dec_device_run(void *priv)
structqueue_status_infoq_status;
u32fail_res= 0;
int ret = 0;drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+unsigned long flags;dev_dbg(inst->dev->dev, "%s: Fill the ring buffer with new
bitstream data",
__func__);Quoted 12 lines
pm_runtime_resume_and_get(inst->dev->dev);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-3- ret =fill_ringbuffer (inst); -if (ret) { - dev_warn(inst->dev->dev, "Filling ring bufferfailed\n");
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +6
-1- goto finish_job_and_return; +if (!inst->retry) { +mutex_lock (&inst->feed_lock); + ret =fill_ringbuffer (inst); +mutex_unlock (&inst->feed_lock); +if (ret <0 ) { + dev_warn(inst->dev->dev, "Filling ringQuoted 1 line
buffer
failed\n");
Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +4
-0+ goto finish_job_and_return; + } else if (!inst->eos && + inst->queuing_num == 0 && + inst->state ==
VPU_INST_STATE_PIC_RUN) {drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ dev_dbg(inst->dev->dev, "%s: nobitstream for feeding,
so skip ",
__func__);Quoted 11 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0+ goto finish_job_and_return; + }There is quite some overlap between that local src buffer queue and all
the code that depends onv4l2_m2m_num_src_bufs_ready(). Are you sure you
aren't re-implementing something ? Also, this code belong to thePIC_RUN
case in the below switch.
Quoted 5 lines
After src buffer was queued,
v4l2_m2m_num_src_bufs_ready()always returns true until src buffer was removed in thefinish_decodesinceget_result.
So checking if there is buffer or not in the src bufs list was needed in thedevice_run.
Ack.
Quoted 308 lines
Quoted 296 lines
Quoted 15 lines
}
switch (
inst->state) {drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -1619,7 +1659,9 @@ static void wave5_vpu_dec_device_run(void *priv) * we had a chance to switch, which leads to aninvalid state
* change.
*/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+spin_lock_irqsave (&inst->state_spinlock, flags);
switch_state(inst, VPU_INST_STATE_PIC_RUN);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ spin_unlock_irqrestore(&inst->state_spinlock,Quoted 1 line
flags);
Should be its own patch, and fixed everywhere. Add missing lockdep assert
(if there is a spinlock version of it).Quoted 4 lines
/*
* During DRC, the picture decoding remains pending, so justleave
Quoted 1 line
the job @@ -1634,12 +1676,14 @@ static void
wave5_vpu_dec_device_run(void *priv)Quoted 4 lines
ret =
wave5_prepare_fb(inst);
if (ret) {
dev_warn(inst->dev->dev, "Framebuffer preparation,fail: %d\n",
Quoted 6 lines
ret);
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ spin_lock_irqsave(&inst->state_spinlock,flags);
switch_state(inst, VPU_INST_STATE_STOP);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ spin_unlock_irqrestore(&inst-Quoted 1 line
state_spinlock, flags);Same, this is separate fix.
Quoted 6 lines
break;
}if (q_status.instance_queue_count) {
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1- dev_dbg(inst->dev->dev, "%s: leave withactive job",
__func__);Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ v4l2_m2m_job_finish(inst->v4l2_m2m_dev,
m2m_ctx);
return;
}drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -1650,14 +1694,22 @@ static void wave5_vpu_dec_device_run(void *priv) dev_err(inst->dev->dev,"Frame decoding on m2m context (%p), fail: %d
(result: %d)\n",
Quoted 14 lines
m2m_ctx, ret,fail_res);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +9
-1-break ; + goto finish_job_and_return; + } + +if (fail_res ==WAVE5_SYSERR_QUEUEING_FAIL ) { + inst->retry = true; + }else { + inst->retry = false; +if (!inst->eos) + inst->queuing_num--;}
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-2-/* Return so that we leave this job active */ - dev_dbg(inst->dev->dev, "%s: leave with activeQuoted 1 line
job",
__func__);Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-3-return ; -default : - WARN(1, "Execution of a job in state %sillegal.\n",
state_to_str(inst->state));Quoted 71 lines
break;
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3
-0+default : +if (!v4l2_m2m_has_stopped (m2m_ctx)) + WARN(1, "Execution of a job in state %sillegal.\n",
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0+ state_to_str(inst->state)); +return ;}
finish_job_and_return:
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-4@@ -1676,10 +1728,7 @@ static void wave5_vpu_dec_job_abort(void *priv) if (ret) return; - ret =wave5_vpu_dec_set_eos_on_firmware (inst); -if (ret) -dev_warn (inst->dev->dev, - "Setting EOS for the bitstream, fail: %d\n", ret); + v4l2_m2m_job_finish(inst->v4l2_m2m_dev, inst- >v4l2_fh.m2m_ctx);}
static int
wave5_vpu_dec_job_ready(void *priv)@@ -1755,6 +1804,8 @@
static intwave5_vpu_open_dec(struct file *filp)inst->ops=&wave5_vpu_dec_inst_ops;
spin_lock_init(&inst->state_spinlock);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0+mutex_init (&inst->feed_lock); +INIT_LIST_HEAD (&inst->avail_src_bufs);
inst->codec_info=kzalloc(sizeof(*inst->codec_info), GFP_KERNEL);
if (!inst->codec_info)drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-3@@ -1830,9 +1881,6 @@ static int wave5_vpu_open_dec(struct file *filp) if (ret) goto cleanup_inst; -if (list_empty (&dev->instances)) -pm_runtime_use_autosuspend (inst->dev->dev); - list_add_tail(&inst->list, &dev->instances);
mutex_unlock(&dev->dev_lock);(quoted diff) +0
-0
enc.cb/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.cindex1e5fc5f8b856..cf20f774ed1b100644drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c +0
-3@@ -1774,9 +1774,6 @@ static int wave5_vpu_open_enc(struct file *filp) if (ret) goto cleanup_inst; -if (list_empty (&dev->instances)) -pm_runtime_use_autosuspend (inst->dev->dev); - list_add_tail(&inst->list, &dev->instances);
mutex_unlock(&dev->dev_lock);(quoted diff) +0
-0
vpuapi.cb/drivers/media/platform/chips-media/wave5/wave5-vpuapi.cindexe5e879a13e8b..c1737fac6edd100644drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance · line 207
Quoted 1 line
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-0@@ -207,8 +207,6 @@ int wave5_vpu_dec_close(struct vpu_instance *inst,u32
*fail_res)drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance · line 231
Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-2int retry = 0; struct vpu_device *vpu_dev = inst->dev; int i; -int inst_count =0 ; -struct vpu_instance *inst_elm; *fail_res = 0;if (!inst->codec_info)
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-0@@ -233,6 +231,16 @@ int wave5_vpu_dec_close(struct vpu_instance *inst,u32
*fail_res)Quoted 15 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +6
-0retry++ >= MAX_FIRMWARE_CALL_RETRY) { ret = -ETIMEDOUT; goto unlock_and_return; + } else if (*fail_res ==WAVE5_SYSERR_VPU_STILL_RUNNING ) { +struct dec_output_info dec_info; + +mutex_unlock (&vpu_dev->hw_lock); +wave5_vpu_dec_get_output_info (inst, &dec_info); + ret = mutex_lock_interruptible(&vpu_dev-Quoted 1 line
hw_lock);drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +2
-0+if (ret) { + pm_runtime_put_sync(inst->dev-Quoted 1 line
dev);
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c
Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +2
-0+return ret; + }Please try and simplify all the branches. Something like:
if (ret == 0) break; if (*fail_res != WAVE5_SYSERR_VPU_STILL_RUNNING) { dev_warn(inst->dev->dev, "dec_finish_seq timed out\n"); goto unlock_and_return; } if (retry++ >= MAX_FIRMWARE_CALL_RETRY) { ret = -ETIMEDOUT; goto unlock_and_return; } mutex_unlock(&vpu_dev->hw_lock); wave5_vpu_dec_get_output_info(inst,
&dec_info);
ret = mutex_lock_interruptible(&vpu_dev-Quoted 1 line
hw_lock);if (ret) { pm_runtime_put_sync(inst->dev-Quoted 1 line
dev);
return ret; }My impression on that change is that it is independent and could be introduced separately.
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance · line 257
Quoted 5 lines
}
} while (ret != 0);drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-0@@ -249,11 +257,7 @@ int wave5_vpu_dec_close(struct vpu_instance *inst,u32
*fail_res)drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res) · line 256
Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-0}wave5_vdi_free_dma_memory(vpu_dev, &p_dec_info-
Quoted 1 line
vb_task);drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-5- -list_for_each_entry (inst_elm, &vpu_dev->instances, list) - inst_count++; -if (inst_count ==1 ) -pm_runtime_dont_use_autosuspend (vpu_dev->dev);This seems like some other fix, that should be in its own commit.
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_enc_close(struct vpu_instance · line 724
Quoted 6 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +1
-0+mutex_destroy (&inst->feed_lock);unlock_and_return:
mutex_unlock(&vpu_dev->hw_lock);drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-0@@ -720,8 +724,6 @@ int wave5_vpu_enc_close(struct vpu_instance *inst,u32
*fail_res)Quoted 14 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-2int ret; int retry = 0; struct vpu_device *vpu_dev = inst->dev; -int inst_count =0 ; -struct vpu_instance *inst_elm; *fail_res = 0;if (!inst->codec_info)
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-0@@ -765,11 +767,6 @@ int wave5_vpu_enc_close(struct vpu_instance *inst, u32 *fail_res)wave5_vdi_free_dma_memory(vpu_dev, &p_enc_info-
Quoted 1 line
vb_task);drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_enc_close(struct vpu_instance *inst, u32 *fail_res) · line 772
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-5-list_for_each_entry (inst_elm, &vpu_dev->instances, list) - inst_count++; -if (inst_count ==1 ) -pm_runtime_dont_use_autosuspend (vpu_dev->dev); -Same.
Quoted 10 lines
mutex_unlock(&vpu_dev->hw_lock);
pm_runtime_put_sync(inst->dev->dev);(quoted diff) +0
-0
vpuapi.hb/drivers/media/platform/chips-media/wave5/wave5-vpuapi.hindex45615c15beca..6ca1ddc67c64100644drivers/media/platform/chips-media/wave5/wave5-vpuapi.h enum set_param_option { · line 166
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +1
-1@@ -163,7 +163,7 @@ enum set_param_option { #define BUFFER_MARGIN 4096 -#define MAX_FIRMWARE_CALL_RETRY 10 +#define MAX_FIRMWARE_CALL_RETRY 30I thought 10s was already very long, am I right this means it can get
stuck for 30s now ?drivers/media/platform/chips-media/wave5/wave5-vpuapi.h struct vpu_instance { · line 816
Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +2
-0#define VDI_LITTLE_ENDIAN 0x0@@ -812,6 +812,9 @@ struct vpu_instance { bool cbcr_interleave; bool nv21; bool eos; +bool retry; +int queuing_num;Some comment would be helpful. What does "retry" stand for, and how is
queing_numdifferent fromv4l2_m2m_num_src_bufs_ready()?Quoted 9 lines
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +1
-0+struct mutex feed_lock;/* lock for feeding bitstream buffers */
structvpu_bufbitstream_vbuf;
dma_addr_tlast_rd_ptr;
size_tremaining_consumed_bytes;--
Nicolas Dufresne
Principal Engineer at CollaboraThanks
Jackson
Signature · 2 lines
Nicolas Dufresne Principal Engineer at Collabora
jackson.lee
maintainerRE: [RESEND PATCH v0 1/3] media: chips-media: wave5: Improve performance of decoder
Hi Nicolas
Quoted 441 lines
-----Original Message-----
From: Nicolas Dufresne <[email protected]>
Sent: Saturday, April 5, 2025 1:00 AM
To: jackson.lee <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; lafley.kim <[email protected]>; [email protected]; [email protected]; Nas Chung <[email protected]>
Subject: Re: [RESEND PATCH v0 1/3] media: chips-media: wave5: Improve performance of decoderHi,
Le vendredi 04 avril 2025 à 04:16 +0000, jackson.lee a écrit :
Quoted 89 lines
Hi Nicolas
Thanks for your review.
Can you check my comment for your review?Quoted 75 lines
-----Original Message-----
From: Nicolas Dufresne <[email protected]>
Sent: Thursday, April 3, 2025 3:03 AM
To: jackson.lee <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; lafley.kim <[email protected]>; [email protected]; [email protected]; Nas Chung <[email protected]>
Subject: Re: [RESEND PATCH v0 1/3] media: chips-media: wave5:
Improve
performance of decoderLe mercredi 19 mars 2025 à 12:50 +0900, Jackson.lee a écrit :
Quoted 4 lines
From: Jackson Lee <[email protected]>
The existing way for decoding frames was to wait until each frame was
Suggestion:
The current decoding method was to...Quoted 5 lines
decoded after feeding a bitstream. As a result, performance was low and Wave5 could not achieve max pixel processing rate.
Update driver to use an asynchronous approach for decoding and feeding a
Update the driver ..
Quoted 1 line
bitstream in order to achieve full capabilities of the device.
This second part lacks the technical design choices needed for your reviewer ;-P I did read the patch once, and am under the impression that
your approach is to not finish the job anymore.Quoted 18 lines
Signed-off-by Jackson Lee <[email protected]>Signed-off-by Nas Chung <[email protected]>---
.../chips-media/wave5/wave5-vpu-dec.c | 344
++++++++++--------
.../chips-media/wave5/wave5-vpu-enc.c | 3 -.../platform/chips-media/wave5/wave5-vpuapi.c| 25 +-.../platform/chips-media/wave5/wave5-vpuapi.h| 5 +-4 files changed, 211 insertions(+), 166 deletions(-)(quoted diff) +0
-0
dec.cb/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.cindexfd71f0c43ac3..cc47da509703100644drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct · line 230
Quoted 1 line
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -230,12 +230,147 @@ static int start_decode(struct vpu_instance *inst,u32
*fail_res)Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0switch_state(inst, VPU_INST_STATE_STOP);dev_dbg(inst->dev->dev, "%s: pic run failed / finish job",
__func__);Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1- v4l2_m2m_job_finish(inst->v4l2_m2m_dev,
m2m_ctx);
}return ret;
}+static int write_to_ringbuffer(struct
vpu_instance*inst, void *buffer,
size_tbuffer_size,Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ struct vpu_buf *ring_buffer,
dma_addr_twr_ptr){
Quoted 6 lines
It seems that from now on this needs to be called with the
feed_lock, so
add:lockdep_assert_held(&inst->feed_lock);Quoted 6 lines
The "
feed_lock" mutex is for protecting "inst->avail_src_bufs" this list, so thefeed_lockshould be used when "static intfill_ringbuffer(struct vpu_instance *inst)" is called, not thewrite_to_ringbuffer.
So I think thefeed_lockshould not be added in thewrite_to_ringbufferfunction.This is called from "
fill_ringbuffer()" and both calls to that seems to take that lock, my comment was just to make it explicit with lockdep. This is called fromCMD_STOPand fromdevice_run()which can be concurrent. I think the code effectively protecting the ring buffer state with.drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3
-0+mutex_lock (&inst->feed_lock); + ret =fill_ringbuffer (inst); +mutex_unlock (&inst->feed_lock);note that there is bunch of random locking too in the abstraction, this driver will need some cleanup in the long run imho.
Quoted 114 lines
Quoted 28 lines
Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+size_t size; +size_t offset = wr_ptr - ring_buffer->daddr; +int ret; + + if (wr_ptr + buffer_size > ring_buffer->daddr +
ring_buffer->size){
Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ size = ring_buffer->daddr + ring_buffer->size -
wr_ptr;drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ ret = wave5_vdi_write_memory(inst->dev,
ring_buffer, offset,(u8 *)buffer, size);
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +4
-0+if (ret <0 ) +return ret; + + ret = wave5_vdi_write_memory(inst->dev,
ring_buffer, 0, (u8*)buffer + size,
Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ buffer_size -size);
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +4
-0+if (ret <0 ) +return ret; + }else { + ret = wave5_vdi_write_memory(inst->dev,
ring_buffer, offset,(u8 *)buffer,
Quoted 13 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 263
Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7
-0+ buffer_size); +if (ret <0 ) +return ret; + } + +return 0 ; +}I have a pretty strong preference in seeing code being moved around in
seperate patch documented as "non functional changes". This way I know I
don't have to review again that code.I will do it.
Quoted 61 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0+ +static int fill_ringbuffer (struct vpu_instance *inst) {Same, I don't want to have to do back and fourth to check if you changed
anything. Also, after the refactoring, the lockdeup assert can be added in
the new patch, which makes the introduction of the new lock very obvious.Quoted 11 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +10
-0+struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx; +struct vpu_src_buffer *vpu_buf; +int ret =0 ; + + if (m2m_ctx->last_src_buf) { +struct vpu_src_buffer *vpu_buf = +wave5_to_vpu_src_buf (m2m_ctx->last_src_buf); + +if (vpu_buf->consumed) { + dev_dbg(inst->dev->dev, "last src bufferalready
written\n");
Quoted 20 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+return 0 ; + } + } + + list_for_each_entry(vpu_buf, &inst->avail_src_bufs,list) {
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ struct vb2_v4l2_buffer *vbuf = &vpu_buf-Quoted 1 line
v4l2_m2m_buf.vb;
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ struct vpu_buf *ring_buffer = &inst-Quoted 1 line
bitstream_vbuf;drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ size_t src_size = vb2_get_plane_payload(&vbuf-Quoted 1 line
vb2_buf, 0);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ void *src_buf = vb2_plane_vaddr(&vbuf->vb2_buf,0);
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +6
-0+dma_addr_t rd_ptr =0 ; +dma_addr_t wr_ptr =0 ; +size_t remain_size =0 ; + +if (vpu_buf->consumed) { + dev_dbg(inst->dev->dev, "already copiedQuoted 1 line
src buf (%u)
to the ring buffer\n",
Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7
-0+ vbuf->vb2_buf.index); +continue ; + } + +if (!src_buf) { + dev_dbg(inst->dev->dev, + "%s: Acquiring kernel pointer tosrc buf (%u),
fail\n",
Quoted 6 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+ __func__, vbuf->vb2_buf.index); +break ; + } + + ret = wave5_vpu_dec_get_bitstream_buffer(inst,
&rd_ptr,
&wr_ptr,&remain_size);Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 304
Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0+if (ret) { +/* Unable to acquire the mutex */ Might want to fix that comment in a separate patch, perhaps even remove
that ?Quoted 1 line
Do you want to remove the comment in a separate patch ?
Yes please.
Quoted 198 lines
Quoted 197 lines
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ dev_err(inst->dev->dev, "Getting thebitstream buffer,
fail: %d\n",
Quoted 6 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+ ret); +return ret; + } + + dev_dbg(inst->dev->dev, "%s: rd_ptr %pad wr_ptr%pad",
__func__,Quoted 6 lines
+&rd_ptr,
&wr_ptr);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +4
-0+ +if (remain_size < src_size) { + dev_dbg(inst->dev->dev, + "%s: remaining size: %zu <source size: %zu for
src buf (%u)\n",
Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ __func__, remain_size, src_size,vbuf-
vb2_buf.index);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +4
-0+break ; + } + + ret = write_to_ringbuffer(inst, src_buf,
src_size,
ring_buffer,wr_ptr);Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0+if (ret) { + dev_err(inst->dev->dev, "Write src buf(%u) to ring
buffer, fail: %d\n",
Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+ vbuf->vb2_buf.index, ret); +return ret; + } + + ret =
wave5_vpu_dec_update_bitstream_buffer(inst, src_size);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3
-0+if (ret) { + dev_dbg(inst->dev->dev, + "update_bitstream_buffer fail:%d for src buf
(%u)\n",
Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7
-0+ ret, vbuf->vb2_buf.index); +break ; + } + + vpu_buf->consumed = true; + +/* Don't write buffers passed the last one while draining.
*/
Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ if (v4l2_m2m_is_last_draining_src_buf(m2m_ctx,vbuf)) {
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ dev_dbg(inst->dev->dev, "last src bufferwritten to
the ring buffer\n");
Quoted 23 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +12
-0+break ; + } + + inst->queuing_num++; +list_del_init (&vpu_buf->list); +break ; + } + +return ret; +} + +static void wave5_vpu_dec_feed_remaining(struct vpu_instance*inst) {
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +9
-0+int ret =0 ; +struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx; +u32 fail_res =0 ; + +mutex_lock (&inst->feed_lock); + ret =fill_ringbuffer (inst); +mutex_unlock (&inst->feed_lock); +if (ret) { + dev_warn(inst->dev->dev, "Filling ring bufferfailed\n");
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
Quoted 1 line
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+return ;I don't think the error handling is correct. What will happen with the
job ?drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 364
Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3
-0+ } + + ret =start_decode (inst, &fail_res);I'm a little worried seeing this called outside of
device_run()context.Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3
-0+if (ret) { + dev_err(inst->dev->dev, + "Frame decoding on m2m context (%p),fail: %d
(result: %d)\n",
Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +6
-0+ m2m_ctx, ret, fail_res); + } + +v4l2_m2m_job_finish (inst->v4l2_m2m_dev, m2m_ctx); +} +Quoted 6 lines
static void
flag_last_buffer_done(struct vpu_instance *inst){
structv4l2_m2m_ctx*m2m_ctx=inst->v4l2_fh.m2m_ctx; @@ -347,7
+482,6 @@ static void wave5_vpu_dec_finish_decode(structvpu_instance*inst)
Quoted 11 lines
struct
vb2_v4l2_buffer*dec_buf= NULL;
structvb2_v4l2_buffer*disp_buf= NULL; structvb2_queue*dst_vq=v4l2_m2m_get_dst_vq(m2m_ctx);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1-struct queue_status_info q_status;dev_dbg(inst->dev->dev, "%s: Fetch output info from firmware.",
__func__);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -441,20 +575,6 @@ static voidwave5_vpu_dec_finish_decode(struct
vpu_instance*inst)Quoted 8 lines
}
spin_unlock_irqrestore(&inst->state_spinlock, flags);
}drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-3- -/* -* During a resolution change and while draining, the firmware may
flush
Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1-* the reorder queue regardless of having a matching decoding
operation
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1-* pending. Only terminate the job if there are no more IRQ coming.
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-2-*/ - wave5_vpu_dec_give_command(inst, DEC_GET_QUEUE_STATUS,
&q_status);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-2- if (q_status.report_queue_count == 0 && - (q_status.instance_queue_count == 0 ||dec_info.sequence_changed)) {
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1- dev_dbg(inst->dev->dev, "%s: finishing job.\n",
__func__);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-3-pm_runtime_mark_last_busy (inst->dev->dev); -pm_runtime_put_autosuspend (inst->dev->dev); - v4l2_m2m_job_finish(inst->v4l2_m2m_dev,
m2m_ctx);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1- }This removal is why I believe some seek stress testing are needed. I'll
try and find some tests file I crafted last year.Quoted 5 lines
}
static int wave5_vpu_dec_querycap(struct file *file, void *fh, struct
v4l2_capability*cap) @@ -794,11 +914,21 @@ static int
wave5_vpu_dec_stop(struct vpu_instance *inst)Quoted 20 lines
}
if (
inst->state!=VPU_INST_STATE_NONE) {drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3
-0+struct vb2_v4l2_buffer *vbuf; +struct vpu_src_buffer *vpu_buf; +/*
* Temporarily release thestate_spinlockso that subsequent
* calls do not block on a mutex while inside this spinlock.
*/spin_unlock_irqrestore(&inst->state_spinlock, flags);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+ vbuf =v4l2_m2m_last_src_buf (m2m_ctx); +if (vbuf) { + vpu_buf =wave5_to_vpu_src_buf (vbuf); +if (!vpu_buf->consumed) + wave5_vpu_dec_feed_remaining(inst);
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0+ } +Do you really need to add all these checks ? Shouldn't you just "
feed_remaining()" and move on ? I also don't think its a good idea to
callv4l2_m2m_job_finish()in code that can run concurrently with thedevice_run()function. This should only happen if you are certain the VPU
instance is stopped already.
Quoted 8 lines
Quoted 5 lines
While testing fluster, stop command is sometimes called before the last bitstream queued is handled.
The stop command sets EOS before handling the last bitstream, so sometimes mismatch happens.
I think the code should be add in thewave5_vpu_dec_stop.This would be a regression, we took good care of handling before and after case correctly.
I don't think this is a regression issue, but I will check it.
Quoted 632 lines
Quoted 251 lines
Quoted 244 lines
Quoted 8 lines
ret =
wave5_vpu_dec_set_eos_on_firmware(inst); if (ret)
return ret;drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -1116,115 +1246,6 @@ static int wave5_prepare_fb(struct
vpu_instance*inst)
Quoted 5 lines
return 0;
}-static int write_to_ringbuffer(struct
vpu_instance*inst, void *buffer,
size_tbuffer_size,Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1- struct vpu_buf *ring_buffer,
dma_addr_twr_ptr)drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-6-{ - size_t size; - size_t offset = wr_ptr - ring_buffer->daddr; - int ret; - - if (wr_ptr + buffer_size > ring_buffer->daddr +
ring_buffer->size){
Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1- size = ring_buffer->daddr + ring_buffer->size -
wr_ptr;drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1- ret = wave5_vdi_write_memory(inst->dev,
ring_buffer, offset,(u8 *)buffer, size);
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-4-if (ret <0 ) -return ret; - - ret = wave5_vdi_write_memory(inst->dev,
ring_buffer, 0, (u8*)buffer + size,
Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1- buffer_size -size);
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-4-if (ret <0 ) -return ret; - }else { - ret = wave5_vdi_write_memory(inst->dev,
ring_buffer, offset,(u8 *)buffer,
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-5- buffer_size); -if (ret <0 ) -return ret; - } -drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)
Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-10-return 0 ; -} - -static int fill_ringbuffer(struct vpu_instance *inst) -{ -struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx; -struct v4l2_m2m_buffer *buf, *n; -int ret; - - if (m2m_ctx->last_src_buf) { - struct vpu_src_buffer *vpu_buf =
wave5_to_vpu_src_buf(m2m_ctx->last_src_buf);Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-3- -if (vpu_buf->consumed) { - dev_dbg(inst->dev->dev, "last src bufferalready
written\n");
Quoted 20 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-7-return 0 ; - } - } - -v4l2_m2m_for_each_src_buf_safe (m2m_ctx, buf, n) { -struct vb2_v4l2_buffer *vbuf = &buf->vb; - struct vpu_src_buffer *vpu_buf =
wave5_to_vpu_src_buf(vbuf);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1- struct vpu_buf *ring_buffer = &inst-Quoted 1 line
bitstream_vbuf;drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1- size_t src_size = vb2_get_plane_payload(&vbuf-Quoted 1 line
vb2_buf, 0);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1- void *src_buf = vb2_plane_vaddr(&vbuf->vb2_buf,0);
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-6-dma_addr_t rd_ptr =0 ; -dma_addr_t wr_ptr =0 ; -size_t remain_size =0 ; - -if (vpu_buf->consumed) { - dev_dbg(inst->dev->dev, "already copiedQuoted 1 line
src buf (%u)
to the ring buffer\n",
Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-7- vbuf->vb2_buf.index); -continue ; - } - -if (!src_buf) { - dev_dbg(inst->dev->dev, - "%s: Acquiring kernel pointer tosrc buf (%u),
fail\n",
Quoted 6 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-5- __func__, vbuf->vb2_buf.index); -break ; - } - - ret = wave5_vpu_dec_get_bitstream_buffer(inst,
&rd_ptr,
&wr_ptr,&remain_size);Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-3-if (ret) { -/* Unable to acquire the mutex */ - dev_err(inst->dev->dev, "Getting thebitstream buffer,
fail: %d\n",
Quoted 6 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-5- ret); -return ret; - } - - dev_dbg(inst->dev->dev, "%s: rd_ptr %pad wr_ptr%pad",
__func__,&rd_ptr,&wr_ptr);Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-4- -if (remain_size < src_size) { - dev_dbg(inst->dev->dev, - "%s: remaining size: %zu <source size: %zu for
src buf (%u)\n",
Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1- __func__, remain_size, src_size,vbuf-
vb2_buf.index);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-4-break ; - } - - ret = write_to_ringbuffer(inst, src_buf,
src_size,
ring_buffer,wr_ptr);Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-2-if (ret) { - dev_err(inst->dev->dev, "Write src buf(%u) to ring
buffer, fail: %d\n",
Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-5- vbuf->vb2_buf.index, ret); -return ret; - } - - ret =
wave5_vpu_dec_update_bitstream_buffer(inst, src_size);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-3-if (ret) { - dev_dbg(inst->dev->dev, - "update_bitstream_buffer fail:%d for src buf
(%u)\n",
Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-7- ret, vbuf->vb2_buf.index); -break ; - } - - vpu_buf->consumed = true; - -/* Don't write buffers passed the last one while draining.
*/
Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1- if (v4l2_m2m_is_last_draining_src_buf(m2m_ctx,vbuf)) {
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1- dev_dbg(inst->dev->dev, "last src bufferwritten to
the ring buffer\n");
Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-7-break ; - } - } - -return 0 ; -} -static void
wave5_vpu_dec_buf_queue_src(struct vb2_buffer *vb){
structvpu_instance*inst = vb2_get_drv_priv(vb-Quoted 3 lines
Quoted 1 line
vb2_queue); @@-1236,6 +1257,11 @@ static void
wave5_vpu_dec_buf_queue_src(struct
vb2_buffer*vb)drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_buf_queue_src(struct vb2_buffer *vb) · line 1264
Quoted 8 lines
vbuf->sequence= inst->queued_src_buf_num++;
v4l2_m2m_buf_queue(m2m_ctx, vbuf);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5
-0+ +INIT_LIST_HEAD (&vpu_buf->list); +mutex_lock (&inst->feed_lock); +list_add_tail (&vpu_buf->list, &inst->avail_src_bufs); +mutex_unlock (&inst->feed_lock);This lack documentation, you didn't even mention the introduction of a new
src buf list in the commit message.Quoted 6 lines
}
static void
wave5_vpu_dec_buf_queue_dst(struct vb2_buffer *vb)@@
-1362,7 +1388,6 @@ static int wave5_vpu_dec_start_streaming(struct
vb2_queue*q, unsigned int countQuoted 21 lines
goto
return_buffers; }
}
-
}pm_runtime_mark_last_busy(inst->dev->dev);pm_runtime_put_autosuspend(inst->dev->dev);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7
-0@@ -1385,6 +1410,13 @@ static int streamoff_output(struct vb2_queue *q) dma_addr_t new_rd_ptr; struct dec_output_info dec_info; unsigned int i; +struct vpu_src_buffer *vpu_buf, *tmp; + + inst->retry = false; + inst->queuing_num =0 ; + +list_for_each_entry_safe (vpu_buf, tmp, &inst- >avail_src_bufs, list) +list_del_init (&vpu_buf->list);Quoted 4 lines
for (i = 0; i <
v4l2_m2m_num_dst_bufs_ready(m2m_ctx); i++) {
ret =wave5_vpu_dec_set_disp_flag(inst, i); @@ - 1481,10+1513,8 @@
Quoted 4 lines
static void
wave5_vpu_dec_stop_streaming(struct vb2_queue *q)wave5_vpu_dec_give_command(inst,
DEC_GET_QUEUE_STATUS,
&q_status);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-4-if (q_status.report_queue_count ==0 ) -break ; - - if (wave5_vpu_wait_interrupt(inst,
VPU_DEC_TIMEOUT) < 0)drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0+ if ((inst->state == VPU_INST_STATE_STOP || +q_status.instance_queue_count == 0) &&State should be read with appropriate locking.
Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ q_status.report_queue_count == 0)break;
This loop have nothing to wait on anymore. It just actively pool on the HW.
I noticed your third patch, but if this is what you address there, squash
it somehow, this would break bisects.Quoted 5 lines
The goal of that code was to ensure that this code will never run concurrently to
finish_decode()once out of that while loop. I don't think
you break that, but that will cause possibly high CPU load.Quoted 5 lines
The code is not related with the third patch, there was sometimes HW interrupt misses since changing to asynchronous, so an infinite wait happened when closed
So I removed it, but I think there is no high CPU load because it runs very short time.If the firmware get stuck, Linux can spin, this cannot be done that way I'm sorry. That its most of the time spinning only few times does not seems like valid justification. We need a limit, we need some preemption, etc.
Quoted 50 lines
Quoted 37 lines
Quoted 4 lines
if (
wave5_vpu_dec_get_output_info(inst, &dec_output_info))@@
Quoted 9 lines
-1577,13 +1607,23 @@ static void
wave5_vpu_dec_device_run(void *priv)
structqueue_status_infoq_status;
u32fail_res= 0;
int ret = 0;drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+unsigned long flags;dev_dbg(inst->dev->dev, "%s: Fill the ring buffer with new
bitstream data",
__func__);Quoted 12 lines
pm_runtime_resume_and_get(inst->dev->dev);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-3- ret =fill_ringbuffer (inst); -if (ret) { - dev_warn(inst->dev->dev, "Filling ring bufferfailed\n");
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +6
-1- goto finish_job_and_return; +if (!inst->retry) { +mutex_lock (&inst->feed_lock); + ret =fill_ringbuffer (inst); +mutex_unlock (&inst->feed_lock); +if (ret <0 ) { + dev_warn(inst->dev->dev, "Filling ringQuoted 1 line
buffer
failed\n");
Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +4
-0+ goto finish_job_and_return; + } else if (!inst->eos && + inst->queuing_num == 0 && + inst->state ==
VPU_INST_STATE_PIC_RUN) {drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ dev_dbg(inst->dev->dev, "%s: nobitstream for feeding,
so skip ",
__func__);Quoted 11 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0+ goto finish_job_and_return; + }There is quite some overlap between that local src buffer queue and all
the code that depends onv4l2_m2m_num_src_bufs_ready(). Are you sure you
aren't re-implementing something ? Also, this code belong to thePIC_RUN
case in the below switch.Quoted 5 lines
After src buffer was queued,
v4l2_m2m_num_src_bufs_ready()always returns true until src buffer was removed in thefinish_decodesinceget_result.
So checking if there is buffer or not in the src bufs list was needed in thedevice_run.Ack.
Quoted 306 lines
Quoted 296 lines
Quoted 15 lines
}
switch (
inst->state) {drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -1619,7 +1659,9 @@ static void wave5_vpu_dec_device_run(void *priv) * we had a chance to switch, which leads to aninvalid state
* change.
*/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+spin_lock_irqsave (&inst->state_spinlock, flags);
switch_state(inst, VPU_INST_STATE_PIC_RUN);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ spin_unlock_irqrestore(&inst->state_spinlock,Quoted 1 line
flags);
Should be its own patch, and fixed everywhere. Add missing lockdep assert
(if there is a spinlock version of it).Quoted 4 lines
/*
* During DRC, the picture decoding remains pending, so justleave
Quoted 1 line
the job @@ -1634,12 +1676,14 @@ static void
wave5_vpu_dec_device_run(void *priv)Quoted 4 lines
ret =
wave5_prepare_fb(inst);
if (ret) {
dev_warn(inst->dev->dev, "Framebuffer preparation,fail: %d\n",
Quoted 6 lines
ret);
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ spin_lock_irqsave(&inst->state_spinlock,flags);
switch_state(inst, VPU_INST_STATE_STOP);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ spin_unlock_irqrestore(&inst-Quoted 1 line
state_spinlock, flags);Same, this is separate fix.
Quoted 6 lines
break;
}if (q_status.instance_queue_count) {
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-1- dev_dbg(inst->dev->dev, "%s: leave withactive job",
__func__);Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-0+ v4l2_m2m_job_finish(inst->v4l2_m2m_dev,
m2m_ctx);
return;
}drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-0@@ -1650,14 +1694,22 @@ static void wave5_vpu_dec_device_run(void *priv) dev_err(inst->dev->dev,"Frame decoding on m2m context (%p), fail: %d
(result: %d)\n",
Quoted 14 lines
m2m_ctx, ret,fail_res);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +9
-1-break ; + goto finish_job_and_return; + } + +if (fail_res ==WAVE5_SYSERR_QUEUEING_FAIL ) { + inst->retry = true; + }else { + inst->retry = false; +if (!inst->eos) + inst->queuing_num--;}
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-2-/* Return so that we leave this job active */ - dev_dbg(inst->dev->dev, "%s: leave with activeQuoted 1 line
job",
__func__);Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-3-return ; -default : - WARN(1, "Execution of a job in state %sillegal.\n",
state_to_str(inst->state));Quoted 71 lines
break;
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3
-0+default : +if (!v4l2_m2m_has_stopped (m2m_ctx)) + WARN(1, "Execution of a job in state %sillegal.\n",
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0+ state_to_str(inst->state)); +return ;}
finish_job_and_return:
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1
-4@@ -1676,10 +1728,7 @@ static void wave5_vpu_dec_job_abort(void *priv) if (ret) return; - ret =wave5_vpu_dec_set_eos_on_firmware (inst); -if (ret) -dev_warn (inst->dev->dev, - "Setting EOS for the bitstream, fail: %d\n", ret); + v4l2_m2m_job_finish(inst->v4l2_m2m_dev, inst- >v4l2_fh.m2m_ctx);}
static int
wave5_vpu_dec_job_ready(void *priv)@@ -1755,6 +1804,8 @@
static intwave5_vpu_open_dec(struct file *filp)inst->ops=&wave5_vpu_dec_inst_ops;
spin_lock_init(&inst->state_spinlock);drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2
-0+mutex_init (&inst->feed_lock); +INIT_LIST_HEAD (&inst->avail_src_bufs);
inst->codec_info=kzalloc(sizeof(*inst->codec_info), GFP_KERNEL);
if (!inst->codec_info)drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0
-3@@ -1830,9 +1881,6 @@ static int wave5_vpu_open_dec(struct file *filp) if (ret) goto cleanup_inst; -if (list_empty (&dev->instances)) -pm_runtime_use_autosuspend (inst->dev->dev); - list_add_tail(&inst->list, &dev->instances);
mutex_unlock(&dev->dev_lock);(quoted diff) +0
-0
enc.cb/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.cindex1e5fc5f8b856..cf20f774ed1b100644drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c +0
-3@@ -1774,9 +1774,6 @@ static int wave5_vpu_open_enc(struct file *filp) if (ret) goto cleanup_inst; -if (list_empty (&dev->instances)) -pm_runtime_use_autosuspend (inst->dev->dev); - list_add_tail(&inst->list, &dev->instances);
mutex_unlock(&dev->dev_lock);(quoted diff) +0
-0
vpuapi.cb/drivers/media/platform/chips-media/wave5/wave5-vpuapi.cindexe5e879a13e8b..c1737fac6edd100644drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance · line 207
Quoted 1 line
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-0@@ -207,8 +207,6 @@ int wave5_vpu_dec_close(struct vpu_instance *inst,u32
*fail_res)drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance · line 231
Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-2int retry = 0; struct vpu_device *vpu_dev = inst->dev; int i; -int inst_count =0 ; -struct vpu_instance *inst_elm; *fail_res = 0;if (!inst->codec_info)
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-0@@ -233,6 +231,16 @@ int wave5_vpu_dec_close(struct vpu_instance *inst,u32
*fail_res)Quoted 15 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +6
-0retry++ >= MAX_FIRMWARE_CALL_RETRY) { ret = -ETIMEDOUT; goto unlock_and_return; + } else if (*fail_res ==WAVE5_SYSERR_VPU_STILL_RUNNING ) { +struct dec_output_info dec_info; + +mutex_unlock (&vpu_dev->hw_lock); +wave5_vpu_dec_get_output_info (inst, &dec_info); + ret = mutex_lock_interruptible(&vpu_dev-Quoted 1 line
hw_lock);drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +2
-0+if (ret) { + pm_runtime_put_sync(inst->dev-Quoted 1 line
dev);
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c
Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +2
-0+return ret; + }Please try and simplify all the branches. Something like:
if (ret == 0) break; if (*fail_res != WAVE5_SYSERR_VPU_STILL_RUNNING) { dev_warn(inst->dev->dev, "dec_finish_seq timed out\n"); goto unlock_and_return; } if (retry++ >= MAX_FIRMWARE_CALL_RETRY) { ret = -ETIMEDOUT; goto unlock_and_return; } mutex_unlock(&vpu_dev->hw_lock); wave5_vpu_dec_get_output_info(inst,
&dec_info);
ret = mutex_lock_interruptible(&vpu_dev-Quoted 1 line
hw_lock);if (ret) { pm_runtime_put_sync(inst->dev-Quoted 1 line
dev);
return ret; }My impression on that change is that it is independent and could be introduced separately.
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance · line 257
Quoted 5 lines
}
} while (ret != 0);drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-0@@ -249,11 +257,7 @@ int wave5_vpu_dec_close(struct vpu_instance *inst,u32
*fail_res)drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res) · line 256
Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-0}wave5_vdi_free_dma_memory(vpu_dev, &p_dec_info-
Quoted 1 line
vb_task);drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-5- -list_for_each_entry (inst_elm, &vpu_dev->instances, list) - inst_count++; -if (inst_count ==1 ) -pm_runtime_dont_use_autosuspend (vpu_dev->dev);This seems like some other fix, that should be in its own commit.
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_enc_close(struct vpu_instance · line 724
Quoted 6 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +1
-0+mutex_destroy (&inst->feed_lock);unlock_and_return:
mutex_unlock(&vpu_dev->hw_lock);drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-0@@ -720,8 +724,6 @@ int wave5_vpu_enc_close(struct vpu_instance *inst,u32
*fail_res)Quoted 14 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-2int ret; int retry = 0; struct vpu_device *vpu_dev = inst->dev; -int inst_count =0 ; -struct vpu_instance *inst_elm; *fail_res = 0;if (!inst->codec_info)
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-0@@ -765,11 +767,6 @@ int wave5_vpu_enc_close(struct vpu_instance *inst, u32 *fail_res)wave5_vdi_free_dma_memory(vpu_dev, &p_enc_info-
Quoted 1 line
vb_task);drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_enc_close(struct vpu_instance *inst, u32 *fail_res) · line 772
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0
-5-list_for_each_entry (inst_elm, &vpu_dev->instances, list) - inst_count++; -if (inst_count ==1 ) -pm_runtime_dont_use_autosuspend (vpu_dev->dev); -Same.
Quoted 10 lines
mutex_unlock(&vpu_dev->hw_lock);
pm_runtime_put_sync(inst->dev->dev);(quoted diff) +0
-0
vpuapi.hb/drivers/media/platform/chips-media/wave5/wave5-vpuapi.hindex45615c15beca..6ca1ddc67c64100644drivers/media/platform/chips-media/wave5/wave5-vpuapi.h enum set_param_option { · line 166
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +1
-1@@ -163,7 +163,7 @@ enum set_param_option { #define BUFFER_MARGIN 4096 -#define MAX_FIRMWARE_CALL_RETRY 10 +#define MAX_FIRMWARE_CALL_RETRY 30I thought 10s was already very long, am I right this means it can get
stuck for 30s now ?drivers/media/platform/chips-media/wave5/wave5-vpuapi.h struct vpu_instance { · line 816
Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +2
-0#define VDI_LITTLE_ENDIAN 0x0@@ -812,6 +812,9 @@ struct vpu_instance { bool cbcr_interleave; bool nv21; bool eos; +bool retry; +int queuing_num;Some comment would be helpful. What does "retry" stand for, and how is
queing_numdifferent fromv4l2_m2m_num_src_bufs_ready()?Quoted 9 lines
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +1
-0+struct mutex feed_lock;/* lock for feeding bitstream buffers */
structvpu_bufbitstream_vbuf;
dma_addr_tlast_rd_ptr;
size_tremaining_consumed_bytes;--
Nicolas Dufresne
Principal Engineer at CollaboraQuoted 2 lines
Thanks
Jackson--
Nicolas Dufresne
Principal Engineer at Collabora
Jackson.lee
maintainerPATCH RESEND 2/3 media: chips-media: wave5: Reduce high CPU load
In reply to Jackson.leeFrom: Jackson Lee <[email protected]>
Since applying changes for performance improvement of decoder, there was a problem related to high CPU load.
CPU load was more than 4 times when comparing CPU load. The root cause was the device_run was called many times even if there was no bitstream which should be queued.
--- .../media/platform/chips-media/wave5/wave5-vpu-dec.c | 12 +++++++++--- .../media/platform/chips-media/wave5/wave5-vpuapi.h | 1 + 2 files changed, 10 insertions(+), 3 deletions(-)
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +9 -3
@@ -1313,10 +1313,13 @@ static void wave5_vpu_dec_buf_queue(struct vb2_buffer *vb) __func__, vb->type, vb->index,vb2_plane_size (&vbuf->vb2_buf,0 ), vb2_plane_size(&vbuf->vb2_buf, 1), vb2_plane_size(&vbuf->vb2_buf, 2)); -if (vb->type ==V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE ) +if (vb->type ==V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE ) { +if (inst->empty_queue) + inst->empty_queue = false;wave5_vpu_dec_buf_queue_src (vb); -else if (vb->type ==V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE ) + }else if (vb->type ==V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE ) {wave5_vpu_dec_buf_queue_dst (vb); + } }static int wave5_vpu_dec_allocate_ring_buffer (struct vpu_instance *inst)
@@ -1506,6 +1509,7 @@ static void wave5_vpu_dec_stop_streaming(struct vb2_queue *q)dev_dbg (inst->dev->dev,"%s: type: %u\n" , __func__, q->type);pm_runtime_resume_and_get (inst->dev->dev); + inst->empty_queue = false;while (check_cmd) {struct queue_status_info q_status;
@@ -1622,6 +1626,7 @@ static void wave5_vpu_dec_device_run(void *priv) inst->queuing_num == 0 && inst->state == VPU_INST_STATE_PIC_RUN) {dev_dbg (inst->dev->dev,"%s: no bitstream for feeding, so skip " , __func__); + inst->empty_queue = true; goto finish_job_and_return; } }
@@ -1764,7 +1769,8 @@ static int wave5_vpu_dec_job_ready(void *priv) "No capture buffer ready to decode!\n");break ; }else if (!wave5_is_draining_or_eos (inst) && - !v4l2_m2m_num_src_bufs_ready (m2m_ctx)) { + (!v4l2_m2m_num_src_bufs_ready (m2m_ctx) || + inst->empty_queue)) {dev_dbg (inst->dev->dev,"No bitstream data to decode!\n" );break ;
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +1 -0
@@ -813,6 +813,7 @@ struct vpu_instance {bool nv21;bool eos;bool retry; +bool empty_queue;int queuing_num;struct mutex feed_lock;/* lock for feeding bitstream buffers */ struct vpu_buf bitstream_vbuf;
Signature · 1 line
2.43.0
Jackson.lee
maintainerPATCH RESEND 3/3 media: chips-media: wave5: Fix Null reference while testing fluster
In reply to Jackson.leeFrom: Jackson Lee <[email protected]>
When multi instances are created/destroyed, many interrupts happens or structures for decoder are removed.
"struct vpu_instance" this structure is shared for all flow in decoder, so if the structure is not protected by lock, Null reference exception could happens sometimes.
IRQ Handler was spilt to two phases and Lock was added as well.
--- .../platform/chips-media/wave5/wave5-helper.c | 10 ++- .../chips-media/wave5/wave5-vpu-dec.c | 5 ++ .../chips-media/wave5/wave5-vpu-enc.c | 5 ++ .../platform/chips-media/wave5/wave5-vpu.c | 69 ++++++++++++++++--- .../platform/chips-media/wave5/wave5-vpuapi.h | 6 ++ 5 files changed, 86 insertions(+), 9 deletions(-)
drivers/media/platform/chips-media/wave5/wave5-helper.c +9 -1
@@ -49,7 +49,7 @@ void wave5_cleanup_instance(struct vpu_instance *inst)v4l2_fh_del (&inst->v4l2_fh);v4l2_fh_exit (&inst->v4l2_fh); } -list_del_init (&inst->list); +kfifo_free (&inst->irq_status);ida_free (&inst->dev->inst_ida, inst->id);kfree (inst->codec_info);kfree (inst);
@@ -61,8 +61,16 @@ int wave5_vpu_release_device(struct file *filp, {struct vpu_instance *inst =wave5_to_vpu_inst (filp->private_data);int ret =0 ; +unsigned long flags;v4l2_m2m_ctx_release (inst->v4l2_fh.m2m_ctx); + ret =mutex_lock_interruptible (&inst->dev->irq_lock); +if (ret) +return ret; +spin_lock_irqsave (&inst->dev->irq_spinlock, flags); +list_del_init (&inst->list); +spin_unlock_irqrestore (&inst->dev->irq_spinlock, flags); +mutex_unlock (&inst->dev->irq_lock);if (inst->state !=VPU_INST_STATE_NONE ) {u32 fail_res;
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5 -0
@@ -1868,6 +1868,11 @@ static int wave5_vpu_open_dec(struct file *filp) inst->xfer_func =V4L2_XFER_FUNC_DEFAULT ;init_completion (&inst->irq_done); + ret =kfifo_alloc (&inst->irq_status,16 *sizeof (int ),GFP_KERNEL ); +if (ret) { +dev_err (inst->dev->dev,"failed to allocate fifo\n" ); + goto cleanup_inst; + } inst->id =ida_alloc (&inst->dev->inst_ida,GFP_KERNEL );if (inst->id <0 ) {
drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c +5 -0
@@ -1760,6 +1760,11 @@ static int wave5_vpu_open_enc(struct file *filp) inst->frame_rate =30 ;init_completion (&inst->irq_done); + ret =kfifo_alloc (&inst->irq_status,16 *sizeof (int ),GFP_KERNEL ); +if (ret) { +dev_err (inst->dev->dev,"failed to allocate fifo\n" ); + goto cleanup_inst; + } inst->id =ida_alloc (&inst->dev->inst_ida,GFP_KERNEL );if (inst->id <0 ) {
drivers/media/platform/chips-media/wave5/wave5-vpu.c +61 -8
127 lines not on this page · raw message
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +6 -0
@@ -8,6 +8,7 @@#ifndef VPUAPI_H_INCLUDED #define VPUAPI_H_INCLUDED +#include <linux/kfifo.h> #include <linux/idr.h> #include <linux/genalloc.h> #include <media/v4l2-device.h>
@@ -747,6 +748,7 @@ struct vpu_device {struct video_device *video_dev_enc;struct mutex dev_lock;/* lock for the src, dst v4l2 queues */ struct mutex hw_lock;/* lock hw configurations */ +struct mutex irq_lock;int irq;enum product_id product;struct vpu_attr attr;
@@ -764,7 +766,10 @@ struct vpu_device {struct kthread_worker *worker;int vpu_poll_interval;int num_clks; +struct task_struct *irq_thread; +struct semaphore irq_sem;struct reset_control *resets; +spinlock_t irq_spinlock;/* protect instances list */ };struct vpu_instance ;
@@ -788,6 +793,7 @@ struct vpu_instance {enum v4l2_ycbcr_encoding ycbcr_enc;enum v4l2_quantization quantization; +struct kfifo irq_status;enum vpu_instance_state state;enum vpu_instance_type type;const struct vpu_instance_ops *ops;
Signature · 1 line
2.43.0
Hi,
Le mercredi 19 mars 2025 à 12:50 +0900, Jackson.lee a écrit :
Quoted 24 lines
From: Jackson Lee <[email protected]>
The wave5 codec driver is a stateful encoder/decoder. The following patches is for improving decoder performance.
v4l2-compliance results:
========================v4l2-compliance 1.28.1-5233, 64 bits, 64-bit
time_tBuffer ioctls:
warn: v4l2-test-buffers.cpp(693):VIDIOC_CREATE_BUFSnot supported
warn: v4l2-test-buffers.cpp(693):VIDIOC_CREATE_BUFSnot supported test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
testCREATE_BUFSmaximum buffers: OK
testVIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)Total for wave5-dec device
/dev/video0: 46, Succeeded: 46, Failed: 0, Warnings: 2 Total for wave5-enc device/dev/video1: 46, Succeeded: 46, Failed: 0, Warnings: 0Fluster test results:
=====================
Quoted 8 lines
Running test suite JCT-VC-HEVC_V1 with decoder GStreamer-H.265-V4L2-Gst1.0 Using 1 parallel job(s) Ran 133/147 tests successfully in 68.517 secs
(1 test fails because of not supporting to parse multi frames, 1 test fails because of a missing frame and slight corruption, 2 tests fail because of sizes which are incompatible with the IP, 11 tests fail because of unsupported 10 bit format)
Running test suite JVT-AVC_V1 with decoder GStreamer-H.264-V4L2-Gst1.0 Using 1 parallel job(s) Ran 78/135 tests successfully in 35.015 secs
I'd be more confortable if I could get test report using 2 or 3 parallel instances.
Also, have you conducted manual "seek" tests, and stress tested the dynamic resolution changes ? We don't have automated tests for these anywhere yet.
Nicolas
Quoted 16 lines
(57 fail because the hardware is unable to decode MBAFF / FMO / Field / Extended profile streams.)
Jackson Lee (3):
media: chips-media: wave5: Improve performance of decoder
media: chips-media: wave5: Reduce high CPU load
media: chips-media: wave5: Fix Null reference while testing fluster.../platform/chips-media/wave5/wave5-helper.c | 10 +- .../chips-media/wave5/wave5-vpu-dec.c | 361 ++++++++++-------- .../chips-media/wave5/wave5-vpu-enc.c | 8 +- .../platform/chips-media/wave5/wave5-vpu.c | 69 +++-
.../platform/chips-media/wave5/wave5-vpuapi.c| 25 +-.../platform/chips-media/wave5/wave5-vpuapi.h| 12 +-6 files changed, 307 insertions(+), 178 deletions(-)
jackson.lee
maintainerHi Nicolas
Quoted 54 lines
-----Original Message-----
From: Nicolas Dufresne <[email protected]>
Sent: Thursday, April 3, 2025 12:44 AM
To: jackson.lee <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; lafley.kim <[email protected]>; [email protected]; [email protected]; Nas Chung <[email protected]>
Subject: Re: [RESEND PATCH v0 0/3] Performance improvement of decoderHi,
Le mercredi 19 mars 2025 à 12:50 +0900, Jackson.lee a écrit :
Quoted 31 lines
From: Jackson Lee <[email protected]>
The wave5 codec driver is a stateful encoder/decoder. The following patches is for improving decoder performance.
v4l2-compliance results:
========================v4l2-compliance 1.28.1-5233, 64 bits, 64-bit
time_tBuffer ioctls:
warn: v4l2-test-buffers.cpp(693):VIDIOC_CREATE_BUFSnot supported
warn: v4l2-test-buffers.cpp(693):VIDIOC_CREATE_BUFSnot supported
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
testCREATE_BUFSmaximum buffers: OK
testVIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)Total for wave5-dec device
/dev/video0: 46, Succeeded: 46, Failed: 0,
Warnings: 2 Total for wave5-enc device/dev/video1: 46, Succeeded: 46,
Failed: 0, Warnings: 0Fluster test results:
=====================Running test suite JCT-VC-HEVC_V1 with decoder GStreamer-H.265-V4L2-Gst1.0 Using 1 parallel job(s) Ran 133/147 tests successfully in 68.517 secs
Quoted 8 lines
(1 test fails because of not supporting to parse multi frames, 1 test fails because of a missing frame and slight corruption,
2 tests fail because of sizes which are incompatible with the IP, 11 tests fail because of unsupported 10 bit format)Running test suite JVT-AVC_V1 with decoder GStreamer-H.264-V4L2-Gst1.0 Using 1 parallel job(s) Ran 78/135 tests successfully in 35.015 secs
Quoted 9 lines
I'd be more confortable if I could get test report using 2 or 3 parallel instances.
Also, have you conducted manual "seek" tests, and stress tested the dynamic resolution changes ? We don't have automated tests for these anywhere yet.
Nicolas
I will test them and update it.
Thanks
Jackson
Quoted 18 lines
Quoted 18 lines
(57 fail because the hardware is unable to decode MBAFF / FMO / Field / Extended profile streams.)
Jackson Lee (3):
media: chips-media: wave5: Improve performance of decoder
media: chips-media: wave5: Reduce high CPU load
media: chips-media: wave5: Fix Null reference while testing fluster.../platform/chips-media/wave5/wave5-helper.c | 10 +- .../chips-media/wave5/wave5-vpu-dec.c | 361
++++++++++--------
.../chips-media/wave5/wave5-vpu-enc.c | 8 +- .../platform/chips-media/wave5/wave5-vpu.c | 69 +++-.../platform/chips-media/wave5/wave5-vpuapi.c| 25 +-.../platform/chips-media/wave5/wave5-vpuapi.h| 12 +-6 files changed, 307 insertions(+), 178 deletions(-)