Skip to contentSkip to conversation

Performance improvement of decoder

PATCH RESEND In review Jackson.lee 3 participants 10 messages Mar 19, 2025 – Apr 7, 2025

Subsystems (2)
lore
Share & tools
lore

Jackson.lee

author maintainer
#0 lore raw

cover Performance improvement of decoder

From: Jackson Lee <[email protected]>

The wave5 codec driver is a stateful encoder/decoder. The following patches is for improving decoder performance.

v4l2-compliance results:
========================

v4l2-compliance 1.28.1-5233, 64 bits, 64-bit time_t

Buffer ioctls:
warn: v4l2-test-buffers.cpp(693): VIDIOC_CREATE_BUFS not supported
warn: v4l2-test-buffers.cpp(693): VIDIOC_CREATE_BUFS not supported
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
test CREATE_BUFS maximum buffers: OK
test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)

Total for wave5-dec device /dev/video0: 46, Succeeded: 46, Failed: 0, Warnings: 2 Total for wave5-enc device /dev/video1: 46, Succeeded: 46, Failed: 0, Warnings: 0

Fluster test results:
=====================

Running test suite JCT-VC-HEVC_V1 with decoder GStreamer-H.265-V4L2-Gst1.0 Using 1 parallel job(s)
Ran 133/147 tests successfully in 68.517 secs

(1 test fails because of not supporting to parse multi frames, 1 test fails because of a missing frame and slight corruption,
2 tests fail because of sizes which are incompatible with the IP, 11 tests fail because of unsupported 10 bit format)

Running test suite JVT-AVC_V1 with decoder GStreamer-H.264-V4L2-Gst1.0 Using 1 parallel job(s)
Ran 78/135 tests successfully in 35.015 secs

(57 fail because the hardware is unable to decode MBAFF / FMO / Field / Extended profile streams.)

Jackson Lee (3):
media: chips-media: wave5: Improve performance of decoder
media: chips-media: wave5: Reduce high CPU load
media: chips-media: wave5: Fix Null reference while testing fluster

 .../platform/chips-media/wave5/wave5-helper.c |  10 +-
 .../chips-media/wave5/wave5-vpu-dec.c         | 361 ++++++++++--------
 .../chips-media/wave5/wave5-vpu-enc.c         |   8 +-
 .../platform/chips-media/wave5/wave5-vpu.c    |  69 +++-
 .../platform/chips-media/wave5/wave5-vpuapi.c |  25 +-
 .../platform/chips-media/wave5/wave5-vpuapi.h |  12 +-
 6 files changed, 307 insertions(+), 178 deletions(-)
Signature · 1 line
2.43.0
author maintainer
#1 lore raw

PATCH RESEND 1/3 media: chips-media: wave5: Improve performance of decoder

From: Jackson Lee <[email protected]>

The existing way for decoding frames was to wait until each frame was decoded after feeding a bitstream. As a result, performance was low and Wave5 could not achieve max pixel processing rate.

Update driver to use an asynchronous approach for decoding and feeding a bitstream in order to achieve full capabilities of the device.

---
 .../chips-media/wave5/wave5-vpu-dec.c         | 344 ++++++++++--------
 .../chips-media/wave5/wave5-vpu-enc.c         |   3 -
 .../platform/chips-media/wave5/wave5-vpuapi.c |  25 +-
 .../platform/chips-media/wave5/wave5-vpuapi.h |   5 +-
 4 files changed, 211 insertions(+), 166 deletions(-)
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +196 -148
index fd71f0c43ac3..cc47da509703 100644
@@ -230,12 +230,147 @@ static int start_decode(struct vpu_instance *inst, u32 *fail_res)
 		switch_state(inst, VPU_INST_STATE_STOP);
 
 		dev_dbg(inst->dev->dev, "%s: pic run failed / finish job", __func__);
-		v4l2_m2m_job_finish(inst->v4l2_m2m_dev, m2m_ctx);
 	}
 
 	return ret;
 }
 
+static int write_to_ringbuffer(struct vpu_instance *inst, void *buffer, size_t buffer_size,
+			       struct vpu_buf *ring_buffer, dma_addr_t wr_ptr)
+{
+	size_t size;
+	size_t offset = wr_ptr - ring_buffer->daddr;
+	int ret;
+
+	if (wr_ptr + buffer_size > ring_buffer->daddr + ring_buffer->size) {
+		size = ring_buffer->daddr + ring_buffer->size - wr_ptr;
+		ret = wave5_vdi_write_memory(inst->dev, ring_buffer, offset, (u8 *)buffer, size);
+		if (ret < 0)
+			return ret;
+
+		ret = wave5_vdi_write_memory(inst->dev, ring_buffer, 0, (u8 *)buffer + size,
+					     buffer_size - size);
+		if (ret < 0)
+			return ret;
+	} else {
+		ret = wave5_vdi_write_memory(inst->dev, ring_buffer, offset, (u8 *)buffer,
+					     buffer_size);
+		if (ret < 0)
+			return ret;
+	}
+
+	return 0;
+}
+
+static int fill_ringbuffer(struct vpu_instance *inst)
+{
+	struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
+	struct vpu_src_buffer *vpu_buf;
+	int ret = 0;
+
+	if (m2m_ctx->last_src_buf)  {
+		struct vpu_src_buffer *vpu_buf = wave5_to_vpu_src_buf(m2m_ctx->last_src_buf);
+
+		if (vpu_buf->consumed) {
+			dev_dbg(inst->dev->dev, "last src buffer already written\n");
+			return 0;
+		}
+	}
+
+	list_for_each_entry(vpu_buf, &inst->avail_src_bufs, list) {
+		struct vb2_v4l2_buffer *vbuf = &vpu_buf->v4l2_m2m_buf.vb;
+		struct vpu_buf *ring_buffer = &inst->bitstream_vbuf;
+		size_t src_size = vb2_get_plane_payload(&vbuf->vb2_buf, 0);
+		void *src_buf = vb2_plane_vaddr(&vbuf->vb2_buf, 0);
+		dma_addr_t rd_ptr = 0;
+		dma_addr_t wr_ptr = 0;
+		size_t remain_size = 0;
+
+		if (vpu_buf->consumed) {
+			dev_dbg(inst->dev->dev, "already copied src buf (%u) to the ring buffer\n",
+				vbuf->vb2_buf.index);
+			continue;
+		}
+
+		if (!src_buf) {
+			dev_dbg(inst->dev->dev,
+				"%s: Acquiring kernel pointer to src buf (%u), fail\n",
+				__func__, vbuf->vb2_buf.index);
+			break;
+		}
+
+		ret = wave5_vpu_dec_get_bitstream_buffer(inst, &rd_ptr, &wr_ptr, &remain_size);
+		if (ret) {
+			/* Unable to acquire the mutex */
+			dev_err(inst->dev->dev, "Getting the bitstream buffer, fail: %d\n",
+				ret);
+			return ret;
+		}
+
+		dev_dbg(inst->dev->dev, "%s: rd_ptr %pad wr_ptr %pad", __func__, &rd_ptr, &wr_ptr);
+
+		if (remain_size < src_size) {
+			dev_dbg(inst->dev->dev,
+				"%s: remaining size: %zu < source size: %zu for src buf (%u)\n",
+				__func__, remain_size, src_size, vbuf->vb2_buf.index);
+			break;
+		}
+
+		ret = write_to_ringbuffer(inst, src_buf, src_size, ring_buffer, wr_ptr);
+		if (ret) {
+			dev_err(inst->dev->dev, "Write src buf (%u) to ring buffer, fail: %d\n",
+				vbuf->vb2_buf.index, ret);
+			return ret;
+		}
+
+		ret = wave5_vpu_dec_update_bitstream_buffer(inst, src_size);
+		if (ret) {
+			dev_dbg(inst->dev->dev,
+				"update_bitstream_buffer fail: %d for src buf (%u)\n",
+				ret, vbuf->vb2_buf.index);
+			break;
+		}
+
+		vpu_buf->consumed = true;
+
+		/* Don't write buffers passed the last one while draining. */
+		if (v4l2_m2m_is_last_draining_src_buf(m2m_ctx, vbuf)) {
+			dev_dbg(inst->dev->dev, "last src buffer written to the ring buffer\n");
+			break;
+		}
+
+		inst->queuing_num++;
+		list_del_init(&vpu_buf->list);
+		break;
+	}
+
+	return ret;
+}
+
+static void wave5_vpu_dec_feed_remaining(struct vpu_instance *inst)
+{
+	int ret = 0;
+	struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
+	u32 fail_res = 0;
+
+	mutex_lock(&inst->feed_lock);
+	ret = fill_ringbuffer(inst);
+	mutex_unlock(&inst->feed_lock);
+	if (ret) {
+		dev_warn(inst->dev->dev, "Filling ring buffer failed\n");
+		return;
+	}
+
+	ret = start_decode(inst, &fail_res);
+	if (ret) {
+		dev_err(inst->dev->dev,
+			"Frame decoding on m2m context (%p), fail: %d (result: %d)\n",
+			m2m_ctx, ret, fail_res);
+	}
+
+	v4l2_m2m_job_finish(inst->v4l2_m2m_dev, m2m_ctx);
+}
+
 static void flag_last_buffer_done(struct vpu_instance *inst)
 {
 	struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
@@ -347,7 +482,6 @@ static void wave5_vpu_dec_finish_decode(struct vpu_instance *inst)
 	struct vb2_v4l2_buffer *dec_buf = NULL;
 	struct vb2_v4l2_buffer *disp_buf = NULL;
 	struct vb2_queue *dst_vq = v4l2_m2m_get_dst_vq(m2m_ctx);
-	struct queue_status_info q_status;
 
 	dev_dbg(inst->dev->dev, "%s: Fetch output info from firmware.", __func__);
 
@@ -441,20 +575,6 @@ static void wave5_vpu_dec_finish_decode(struct vpu_instance *inst)
 		}
 		spin_unlock_irqrestore(&inst->state_spinlock, flags);
 	}
-
-	/*
-	 * During a resolution change and while draining, the firmware may flush
-	 * the reorder queue regardless of having a matching decoding operation
-	 * pending. Only terminate the job if there are no more IRQ coming.
-	 */
-	wave5_vpu_dec_give_command(inst, DEC_GET_QUEUE_STATUS, &q_status);
-	if (q_status.report_queue_count == 0 &&
-	    (q_status.instance_queue_count == 0 || dec_info.sequence_changed)) {
-		dev_dbg(inst->dev->dev, "%s: finishing job.\n", __func__);
-		pm_runtime_mark_last_busy(inst->dev->dev);
-		pm_runtime_put_autosuspend(inst->dev->dev);
-		v4l2_m2m_job_finish(inst->v4l2_m2m_dev, m2m_ctx);
-	}
 }
 
 static int wave5_vpu_dec_querycap(struct file *file, void *fh, struct v4l2_capability *cap)
@@ -794,11 +914,21 @@ static int wave5_vpu_dec_stop(struct vpu_instance *inst)
 	}
 
 	if (inst->state != VPU_INST_STATE_NONE) {
+		struct vb2_v4l2_buffer *vbuf;
+		struct vpu_src_buffer *vpu_buf;
+
 		/*
 		 * Temporarily release the state_spinlock so that subsequent
 		 * calls do not block on a mutex while inside this spinlock.
 		 */
 		spin_unlock_irqrestore(&inst->state_spinlock, flags);
+		vbuf = v4l2_m2m_last_src_buf(m2m_ctx);
+		if (vbuf) {
+			vpu_buf = wave5_to_vpu_src_buf(vbuf);
+			if (!vpu_buf->consumed)
+				wave5_vpu_dec_feed_remaining(inst);
+		}
+
 		ret = wave5_vpu_dec_set_eos_on_firmware(inst);
 		if (ret)
 			return ret;
@@ -1116,115 +1246,6 @@ static int wave5_prepare_fb(struct vpu_instance *inst)
 	return 0;
 }
 
-static int write_to_ringbuffer(struct vpu_instance *inst, void *buffer, size_t buffer_size,
-			       struct vpu_buf *ring_buffer, dma_addr_t wr_ptr)
-{
-	size_t size;
-	size_t offset = wr_ptr - ring_buffer->daddr;
-	int ret;
-
-	if (wr_ptr + buffer_size > ring_buffer->daddr + ring_buffer->size) {
-		size = ring_buffer->daddr + ring_buffer->size - wr_ptr;
-		ret = wave5_vdi_write_memory(inst->dev, ring_buffer, offset, (u8 *)buffer, size);
-		if (ret < 0)
-			return ret;
-
-		ret = wave5_vdi_write_memory(inst->dev, ring_buffer, 0, (u8 *)buffer + size,
-					     buffer_size - size);
-		if (ret < 0)
-			return ret;
-	} else {
-		ret = wave5_vdi_write_memory(inst->dev, ring_buffer, offset, (u8 *)buffer,
-					     buffer_size);
-		if (ret < 0)
-			return ret;
-	}
-
-	return 0;
-}
-
-static int fill_ringbuffer(struct vpu_instance *inst)
-{
-	struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
-	struct v4l2_m2m_buffer *buf, *n;
-	int ret;
-
-	if (m2m_ctx->last_src_buf)  {
-		struct vpu_src_buffer *vpu_buf = wave5_to_vpu_src_buf(m2m_ctx->last_src_buf);
-
-		if (vpu_buf->consumed) {
-			dev_dbg(inst->dev->dev, "last src buffer already written\n");
-			return 0;
-		}
-	}
-
-	v4l2_m2m_for_each_src_buf_safe(m2m_ctx, buf, n) {
-		struct vb2_v4l2_buffer *vbuf = &buf->vb;
-		struct vpu_src_buffer *vpu_buf = wave5_to_vpu_src_buf(vbuf);
-		struct vpu_buf *ring_buffer = &inst->bitstream_vbuf;
-		size_t src_size = vb2_get_plane_payload(&vbuf->vb2_buf, 0);
-		void *src_buf = vb2_plane_vaddr(&vbuf->vb2_buf, 0);
-		dma_addr_t rd_ptr = 0;
-		dma_addr_t wr_ptr = 0;
-		size_t remain_size = 0;
-
-		if (vpu_buf->consumed) {
-			dev_dbg(inst->dev->dev, "already copied src buf (%u) to the ring buffer\n",
-				vbuf->vb2_buf.index);
-			continue;
-		}
-
-		if (!src_buf) {
-			dev_dbg(inst->dev->dev,
-				"%s: Acquiring kernel pointer to src buf (%u), fail\n",
-				__func__, vbuf->vb2_buf.index);
-			break;
-		}
-
-		ret = wave5_vpu_dec_get_bitstream_buffer(inst, &rd_ptr, &wr_ptr, &remain_size);
-		if (ret) {
-			/* Unable to acquire the mutex */
-			dev_err(inst->dev->dev, "Getting the bitstream buffer, fail: %d\n",
-				ret);
-			return ret;
-		}
-
-		dev_dbg(inst->dev->dev, "%s: rd_ptr %pad wr_ptr %pad", __func__, &rd_ptr, &wr_ptr);
-
-		if (remain_size < src_size) {
-			dev_dbg(inst->dev->dev,
-				"%s: remaining size: %zu < source size: %zu for src buf (%u)\n",
-				__func__, remain_size, src_size, vbuf->vb2_buf.index);
-			break;
-		}
-
-		ret = write_to_ringbuffer(inst, src_buf, src_size, ring_buffer, wr_ptr);
-		if (ret) {
-			dev_err(inst->dev->dev, "Write src buf (%u) to ring buffer, fail: %d\n",
-				vbuf->vb2_buf.index, ret);
-			return ret;
-		}
-
-		ret = wave5_vpu_dec_update_bitstream_buffer(inst, src_size);
-		if (ret) {
-			dev_dbg(inst->dev->dev,
-				"update_bitstream_buffer fail: %d for src buf (%u)\n",
-				ret, vbuf->vb2_buf.index);
-			break;
-		}
-
-		vpu_buf->consumed = true;
-
-		/* Don't write buffers passed the last one while draining. */
-		if (v4l2_m2m_is_last_draining_src_buf(m2m_ctx, vbuf)) {
-			dev_dbg(inst->dev->dev, "last src buffer written to the ring buffer\n");
-			break;
-		}
-	}
-
-	return 0;
-}
-
 static void wave5_vpu_dec_buf_queue_src(struct vb2_buffer *vb)
 {
 	struct vpu_instance *inst = vb2_get_drv_priv(vb->vb2_queue);
@@ -1236,6 +1257,11 @@ static void wave5_vpu_dec_buf_queue_src(struct vb2_buffer *vb)
 	vbuf->sequence = inst->queued_src_buf_num++;
 
 	v4l2_m2m_buf_queue(m2m_ctx, vbuf);
+
+	INIT_LIST_HEAD(&vpu_buf->list);
+	mutex_lock(&inst->feed_lock);
+	list_add_tail(&vpu_buf->list, &inst->avail_src_bufs);
+	mutex_unlock(&inst->feed_lock);
 }
 
 static void wave5_vpu_dec_buf_queue_dst(struct vb2_buffer *vb)
@@ -1362,7 +1388,6 @@ static int wave5_vpu_dec_start_streaming(struct vb2_queue *q, unsigned int count
 				goto return_buffers;
 			}
 		}
-
 	}
 	pm_runtime_mark_last_busy(inst->dev->dev);
 	pm_runtime_put_autosuspend(inst->dev->dev);
@@ -1385,6 +1410,13 @@ static int streamoff_output(struct vb2_queue *q)
 	dma_addr_t new_rd_ptr;
 	struct dec_output_info dec_info;
 	unsigned int i;
+	struct vpu_src_buffer *vpu_buf, *tmp;
+
+	inst->retry = false;
+	inst->queuing_num = 0;
+
+	list_for_each_entry_safe(vpu_buf, tmp, &inst->avail_src_bufs, list)
+		list_del_init(&vpu_buf->list);
 
 	for (i = 0; i < v4l2_m2m_num_dst_bufs_ready(m2m_ctx); i++) {
 		ret = wave5_vpu_dec_set_disp_flag(inst, i);
@@ -1481,10 +1513,8 @@ static void wave5_vpu_dec_stop_streaming(struct vb2_queue *q)
 
 		wave5_vpu_dec_give_command(inst, DEC_GET_QUEUE_STATUS, &q_status);
 
-		if (q_status.report_queue_count == 0)
-			break;
-
-		if (wave5_vpu_wait_interrupt(inst, VPU_DEC_TIMEOUT) < 0)
+		if ((inst->state == VPU_INST_STATE_STOP || q_status.instance_queue_count == 0) &&
+		    q_status.report_queue_count == 0)
 			break;
 
 		if (wave5_vpu_dec_get_output_info(inst, &dec_output_info))
@@ -1577,13 +1607,23 @@ static void wave5_vpu_dec_device_run(void *priv)
 	struct queue_status_info q_status;
 	u32 fail_res = 0;
 	int ret = 0;
+	unsigned long flags;
 
 	dev_dbg(inst->dev->dev, "%s: Fill the ring buffer with new bitstream data", __func__);
 	pm_runtime_resume_and_get(inst->dev->dev);
-	ret = fill_ringbuffer(inst);
-	if (ret) {
-		dev_warn(inst->dev->dev, "Filling ring buffer failed\n");
-		goto finish_job_and_return;
+	if (!inst->retry) {
+		mutex_lock(&inst->feed_lock);
+		ret = fill_ringbuffer(inst);
+		mutex_unlock(&inst->feed_lock);
+		if (ret < 0) {
+			dev_warn(inst->dev->dev, "Filling ring buffer failed\n");
+			goto finish_job_and_return;
+		} else if (!inst->eos &&
+			   inst->queuing_num == 0 &&
+			   inst->state == VPU_INST_STATE_PIC_RUN) {
+			dev_dbg(inst->dev->dev, "%s: no bitstream for feeding, so skip ", __func__);
+			goto finish_job_and_return;
+		}
 	}
 
 	switch (inst->state) {
@@ -1619,7 +1659,9 @@ static void wave5_vpu_dec_device_run(void *priv)
 		 * we had a chance to switch, which leads to an invalid state
 		 * change.
 		 */
+		spin_lock_irqsave(&inst->state_spinlock, flags);
 		switch_state(inst, VPU_INST_STATE_PIC_RUN);
+		spin_unlock_irqrestore(&inst->state_spinlock, flags);
 
 		/*
 		 * During DRC, the picture decoding remains pending, so just leave the job
@@ -1634,12 +1676,14 @@ static void wave5_vpu_dec_device_run(void *priv)
 		ret = wave5_prepare_fb(inst);
 		if (ret) {
 			dev_warn(inst->dev->dev, "Framebuffer preparation, fail: %d\n", ret);
+			spin_lock_irqsave(&inst->state_spinlock, flags);
 			switch_state(inst, VPU_INST_STATE_STOP);
+			spin_unlock_irqrestore(&inst->state_spinlock, flags);
 			break;
 		}
 
 		if (q_status.instance_queue_count) {
-			dev_dbg(inst->dev->dev, "%s: leave with active job", __func__);
+			v4l2_m2m_job_finish(inst->v4l2_m2m_dev, m2m_ctx);
 			return;
 		}
 
@@ -1650,14 +1694,22 @@ static void wave5_vpu_dec_device_run(void *priv)
 			dev_err(inst->dev->dev,
 				"Frame decoding on m2m context (%p), fail: %d (result: %d)\n",
 				m2m_ctx, ret, fail_res);
-			break;
+			goto finish_job_and_return;
+		}
+
+		if (fail_res == WAVE5_SYSERR_QUEUEING_FAIL) {
+			inst->retry = true;
+		} else {
+			inst->retry = false;
+			if (!inst->eos)
+				inst->queuing_num--;
 		}
-		/* Return so that we leave this job active */
-		dev_dbg(inst->dev->dev, "%s: leave with active job", __func__);
-		return;
-	default:
-		WARN(1, "Execution of a job in state %s illegal.\n", state_to_str(inst->state));
 		break;
+	default:
+		if (!v4l2_m2m_has_stopped(m2m_ctx))
+			WARN(1, "Execution of a job in state %s illegal.\n",
+			     state_to_str(inst->state));
+		return;
 	}
 
 finish_job_and_return:
@@ -1676,10 +1728,7 @@ static void wave5_vpu_dec_job_abort(void *priv)
 	if (ret)
 		return;
 
-	ret = wave5_vpu_dec_set_eos_on_firmware(inst);
-	if (ret)
-		dev_warn(inst->dev->dev,
-			 "Setting EOS for the bitstream, fail: %d\n", ret);
+	v4l2_m2m_job_finish(inst->v4l2_m2m_dev, inst->v4l2_fh.m2m_ctx);
 }
 
 static int wave5_vpu_dec_job_ready(void *priv)
@@ -1755,6 +1804,8 @@ static int wave5_vpu_open_dec(struct file *filp)
 	inst->ops = &wave5_vpu_dec_inst_ops;
 
 	spin_lock_init(&inst->state_spinlock);
+	mutex_init(&inst->feed_lock);
+	INIT_LIST_HEAD(&inst->avail_src_bufs);
 
 	inst->codec_info = kzalloc(sizeof(*inst->codec_info), GFP_KERNEL);
 	if (!inst->codec_info)
@@ -1830,9 +1881,6 @@ static int wave5_vpu_open_dec(struct file *filp)
 	if (ret)
 		goto cleanup_inst;
 
-	if (list_empty(&dev->instances))
-		pm_runtime_use_autosuspend(inst->dev->dev);
-
 	list_add_tail(&inst->list, &dev->instances);
 
 	mutex_unlock(&dev->dev_lock);
drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c +0 -3
index 1e5fc5f8b856..cf20f774ed1b 100644
@@ -1774,9 +1774,6 @@ static int wave5_vpu_open_enc(struct file *filp)
 	if (ret)
 		goto cleanup_inst;
 
-	if (list_empty(&dev->instances))
-		pm_runtime_use_autosuspend(inst->dev->dev);
-
 	list_add_tail(&inst->list, &dev->instances);
 
 	mutex_unlock(&dev->dev_lock);
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +11 -14
index e5e879a13e8b..c1737fac6edd 100644
@@ -207,8 +207,6 @@ int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res)
 	int retry = 0;
 	struct vpu_device *vpu_dev = inst->dev;
 	int i;
-	int inst_count = 0;
-	struct vpu_instance *inst_elm;
 
 	*fail_res = 0;
 	if (!inst->codec_info)
@@ -233,6 +231,16 @@ int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res)
 		    retry++ >= MAX_FIRMWARE_CALL_RETRY) {
 			ret = -ETIMEDOUT;
 			goto unlock_and_return;
+		} else if (*fail_res == WAVE5_SYSERR_VPU_STILL_RUNNING) {
+			struct dec_output_info dec_info;
+
+			mutex_unlock(&vpu_dev->hw_lock);
+			wave5_vpu_dec_get_output_info(inst, &dec_info);
+			ret = mutex_lock_interruptible(&vpu_dev->hw_lock);
+			if (ret) {
+				pm_runtime_put_sync(inst->dev->dev);
+				return ret;
+			}
 		}
 	} while (ret != 0);
 
@@ -249,11 +257,7 @@ int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res)
 	}
 
 	wave5_vdi_free_dma_memory(vpu_dev, &p_dec_info->vb_task);
-
-	list_for_each_entry(inst_elm, &vpu_dev->instances, list)
-		inst_count++;
-	if (inst_count == 1)
-		pm_runtime_dont_use_autosuspend(vpu_dev->dev);
+	mutex_destroy(&inst->feed_lock);
 
 unlock_and_return:
 	mutex_unlock(&vpu_dev->hw_lock);
@@ -720,8 +724,6 @@ int wave5_vpu_enc_close(struct vpu_instance *inst, u32 *fail_res)
 	int ret;
 	int retry = 0;
 	struct vpu_device *vpu_dev = inst->dev;
-	int inst_count = 0;
-	struct vpu_instance *inst_elm;
 
 	*fail_res = 0;
 	if (!inst->codec_info)
@@ -765,11 +767,6 @@ int wave5_vpu_enc_close(struct vpu_instance *inst, u32 *fail_res)
 
 	wave5_vdi_free_dma_memory(vpu_dev, &p_enc_info->vb_task);
 
-	list_for_each_entry(inst_elm, &vpu_dev->instances, list)
-		inst_count++;
-	if (inst_count == 1)
-		pm_runtime_dont_use_autosuspend(vpu_dev->dev);
-
 	mutex_unlock(&vpu_dev->hw_lock);
 	pm_runtime_put_sync(inst->dev->dev);
 
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +4 -1
index 45615c15beca..6ca1ddc67c64 100644
@@ -163,7 +163,7 @@ enum set_param_option {
 
 #define BUFFER_MARGIN				4096
 
-#define MAX_FIRMWARE_CALL_RETRY			10
+#define MAX_FIRMWARE_CALL_RETRY			30
 
 #define VDI_LITTLE_ENDIAN	0x0
 
@@ -812,6 +812,9 @@ struct vpu_instance {
 	bool cbcr_interleave;
 	bool nv21;
 	bool eos;
+	bool retry;
+	int queuing_num;
+	struct mutex feed_lock; /* lock for feeding bitstream buffers */
 	struct vpu_buf bitstream_vbuf;
 	dma_addr_t last_rd_ptr;
 	size_t remaining_consumed_bytes;
Signature · 1 line
2.43.0
#2 lore raw

Re: [RESEND PATCH v0 1/3] media: chips-media: wave5: Improve performance of decoder

Le mercredi 19 mars 2025 à 12:50 +0900, Jackson.lee a écrit :

Quoted 3 lines

From: Jackson Lee <[email protected]>

The existing way for decoding frames was to wait until each frame was

Suggestion:
The current decoding method was to...

Quoted 4 lines

decoded after feeding a bitstream. As a result, performance was low and Wave5 could not achieve max pixel processing rate.

Update driver to use an asynchronous approach for decoding and feeding a

Update the driver ..

Quoted 1 line

bitstream in order to achieve full capabilities of the device.

This second part lacks the technical design choices needed for your reviewer ;-P I did read the patch once, and am under the impression that your approach is to not finish the job anymore.

Quoted 24 lines

---
 .../chips-media/wave5/wave5-vpu-dec.c         | 344 ++++++++++-------- .../chips-media/wave5/wave5-vpu-enc.c         |   3 - .../platform/chips-media/wave5/wave5-vpuapi.c |  25 +- .../platform/chips-media/wave5/wave5-vpuapi.h |   5 +-

 4 files changed, 211 insertions(+), 166 deletions(-)
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -0
index fd71f0c43ac3..cc47da509703 100644
@@ -230,12 +230,147 @@ static int start_decode(struct vpu_instance *inst, u32 *fail_res)
  		switch_state(inst, VPU_INST_STATE_STOP);


  dev_dbg(inst->dev->dev, "%s: pic run failed / finish job", __func__);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-		v4l2_m2m_job_finish(inst->v4l2_m2m_dev, m2m_ctx);

  }

  return ret;
 }

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 240

Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3 -0
+static int write_to_ringbuffer(struct vpu_instance *inst, void *buffer, size_t buffer_size,
+			       struct vpu_buf *ring_buffer, dma_addr_t wr_ptr)
+{

It seems that from now on this needs to be called with the feed_lock, so add:

	 lockdep_assert_held(&inst->feed_lock);
Quoted 13 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +13 -0
+	size_t size;
+	size_t offset = wr_ptr - ring_buffer->daddr;
+	int ret;
+
+	if (wr_ptr + buffer_size > ring_buffer->daddr + ring_buffer->size) {
+		size = ring_buffer->daddr + ring_buffer->size - wr_ptr;
+		ret = wave5_vdi_write_memory(inst->dev, ring_buffer, offset, (u8 *)buffer, size);
+		if (ret < 0)
+			return ret;
+
+		ret = wave5_vdi_write_memory(inst->dev, ring_buffer, 0, (u8 *)buffer + size,
+					     buffer_size - size);
+		if (ret < 0)

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 263

Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +10 -0
+			return ret;
+	} else {
+		ret = wave5_vdi_write_memory(inst->dev, ring_buffer, offset, (u8 *)buffer,
+					     buffer_size);
+		if (ret < 0)
+			return ret;
+	}
+
+	return 0;
+}

I have a pretty strong preference in seeing code being moved around in seperate patch documented as "non functional changes". This way I know I don't have to review again that code.

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 266

Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3 -0
+
+static int fill_ringbuffer(struct vpu_instance *inst)
+{

Same, I don't want to have to do back and fourth to check if you changed anything. Also, after the refactoring, the lockdeup assert can be added in the new patch, which makes the introduction of the new lock very obvious.

Quoted 28 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +28 -0
+	struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
+	struct vpu_src_buffer *vpu_buf;
+	int ret = 0;
+
+	if (m2m_ctx->last_src_buf)  {
+		struct vpu_src_buffer *vpu_buf = wave5_to_vpu_src_buf(m2m_ctx->last_src_buf);
+
+		if (vpu_buf->consumed) {
+			dev_dbg(inst->dev->dev, "last src buffer already written\n");
+			return 0;
+		}
+	}
+
+	list_for_each_entry(vpu_buf, &inst->avail_src_bufs, list) {
+		struct vb2_v4l2_buffer *vbuf = &vpu_buf->v4l2_m2m_buf.vb;
+		struct vpu_buf *ring_buffer = &inst->bitstream_vbuf;
+		size_t src_size = vb2_get_plane_payload(&vbuf->vb2_buf, 0);
+		void *src_buf = vb2_plane_vaddr(&vbuf->vb2_buf, 0);
+		dma_addr_t rd_ptr = 0;
+		dma_addr_t wr_ptr = 0;
+		size_t remain_size = 0;
+
+		if (vpu_buf->consumed) {
+			dev_dbg(inst->dev->dev, "already copied src buf (%u) to the ring buffer\n",
+				vbuf->vb2_buf.index);
+			continue;
+		}
+

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 304

Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +10 -0
+		if (!src_buf) {
+			dev_dbg(inst->dev->dev,
+				"%s: Acquiring kernel pointer to src buf (%u), fail\n",
+				__func__, vbuf->vb2_buf.index);
+			break;
+		}
+
+		ret = wave5_vpu_dec_get_bitstream_buffer(inst, &rd_ptr, &wr_ptr, &remain_size);
+		if (ret) {
+			/* Unable to acquire the mutex */

Might want to fix that comment in a separate patch, perhaps even remove that ?

Quoted 47 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +47 -0
+			dev_err(inst->dev->dev, "Getting the bitstream buffer, fail: %d\n",
+				ret);
+			return ret;
+		}
+
+		dev_dbg(inst->dev->dev, "%s: rd_ptr %pad wr_ptr %pad", __func__, &rd_ptr, &wr_ptr);
+
+		if (remain_size < src_size) {
+			dev_dbg(inst->dev->dev,
+				"%s: remaining size: %zu < source size: %zu for src buf (%u)\n",
+				__func__, remain_size, src_size, vbuf->vb2_buf.index);
+			break;
+		}
+
+		ret = write_to_ringbuffer(inst, src_buf, src_size, ring_buffer, wr_ptr);
+		if (ret) {
+			dev_err(inst->dev->dev, "Write src buf (%u) to ring buffer, fail: %d\n",
+				vbuf->vb2_buf.index, ret);
+			return ret;
+		}
+
+		ret = wave5_vpu_dec_update_bitstream_buffer(inst, src_size);
+		if (ret) {
+			dev_dbg(inst->dev->dev,
+				"update_bitstream_buffer fail: %d for src buf (%u)\n",
+				ret, vbuf->vb2_buf.index);
+			break;
+		}
+
+		vpu_buf->consumed = true;
+
+		/* Don't write buffers passed the last one while draining. */
+		if (v4l2_m2m_is_last_draining_src_buf(m2m_ctx, vbuf)) {
+			dev_dbg(inst->dev->dev, "last src buffer written to the ring buffer\n");
+			break;
+		}
+
+		inst->queuing_num++;
+		list_del_init(&vpu_buf->list);
+		break;
+	}
+
+	return ret;
+}
+
+static void wave5_vpu_dec_feed_remaining(struct vpu_instance *inst)
+{

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 361

Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +10 -0
+	int ret = 0;
+	struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
+	u32 fail_res = 0;
+
+	mutex_lock(&inst->feed_lock);
+	ret = fill_ringbuffer(inst);
+	mutex_unlock(&inst->feed_lock);
+	if (ret) {
+		dev_warn(inst->dev->dev, "Filling ring buffer failed\n");
+		return;

I don't think the error handling is correct. What will happen with the job ?

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 364

Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3 -0
+	}
+
+	ret = start_decode(inst, &fail_res);

I'm a little worried seeing this called outside of device_run() context.

Quoted 31 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +9 -0
+	if (ret) {
+		dev_err(inst->dev->dev,
+			"Frame decoding on m2m context (%p), fail: %d (result: %d)\n",
+			m2m_ctx, ret, fail_res);
+	}
+
+	v4l2_m2m_job_finish(inst->v4l2_m2m_dev, m2m_ctx);
 
 
 
+}
+

 static void flag_last_buffer_done(struct vpu_instance *inst) {
  struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
@@ -347,7 +482,6 @@ static void wave5_vpu_dec_finish_decode(struct vpu_instance *inst)
  	struct vb2_v4l2_buffer *dec_buf = NULL;
  	struct vb2_v4l2_buffer *disp_buf = NULL;
  	struct vb2_queue *dst_vq = v4l2_m2m_get_dst_vq(m2m_ctx);
-	struct queue_status_info q_status;


  dev_dbg(inst->dev->dev, "%s: Fetch output info from firmware.", __func__);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -4
@@ -441,20 +575,6 @@ static void wave5_vpu_dec_finish_decode(struct vpu_instance *inst)
  		}
  		spin_unlock_irqrestore(&inst->state_spinlock, flags);
  	}
-
-	/*
-	 * During a resolution change and while draining, the firmware may flush
-	 * the reorder queue regardless of having a matching decoding operation

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_finish_decode(struct vpu_instance *inst) · line 457

Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -10
-	 * pending. Only terminate the job if there are no more IRQ coming.
-	 */
-	wave5_vpu_dec_give_command(inst, DEC_GET_QUEUE_STATUS, &q_status);
-	if (q_status.report_queue_count == 0 &&
-	    (q_status.instance_queue_count == 0 || dec_info.sequence_changed)) {
-		dev_dbg(inst->dev->dev, "%s: finishing job.\n", __func__);
-		pm_runtime_mark_last_busy(inst->dev->dev);
-		pm_runtime_put_autosuspend(inst->dev->dev);
-		v4l2_m2m_job_finish(inst->v4l2_m2m_dev, m2m_ctx);
-	}

This removal is why I believe some seek stress testing are needed. I'll try and find some tests file I crafted last year.

Quoted 15 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -0
  }


 static int wave5_vpu_dec_querycap(struct file *file, void *fh, struct v4l2_capability *cap)

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -0
@@ -794,11 +914,21 @@ static int wave5_vpu_dec_stop(struct vpu_instance *inst)
  	}


  if (inst->state != VPU_INST_STATE_NONE) {

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3 -0
+		struct vb2_v4l2_buffer *vbuf;
+		struct vpu_src_buffer *vpu_buf;
+

  /*
  * Temporarily release the state_spinlock so that subsequent
  * calls do not block on a mutex while inside this spinlock. */ spin_unlock_irqrestore(&inst->state_spinlock, flags);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_vpu_dec_stop(struct vpu_instance *inst) · line 931

Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7 -0
+		vbuf = v4l2_m2m_last_src_buf(m2m_ctx);
+		if (vbuf) {
+			vpu_buf = wave5_to_vpu_src_buf(vbuf);
+			if (!vpu_buf->consumed)
+				wave5_vpu_dec_feed_remaining(inst);
+		}
+

Do you really need to add all these checks ? Shouldn't you just "feed_remaining()" and move on ? I also don't think its a good idea to call v4l2_m2m_job_finish() in code that can run concurrently with the device_run() function. This should only happen if you are certain the VPU instance is stopped already.

Quoted 123 lines

  ret = wave5_vpu_dec_set_eos_on_firmware(inst); if (ret)
  return ret;

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -0
@@ -1116,115 +1246,6 @@ static int wave5_prepare_fb(struct vpu_instance *inst)
  	return 0;
  }

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -109
-static int write_to_ringbuffer(struct vpu_instance *inst, void *buffer, size_t buffer_size,
-			       struct vpu_buf *ring_buffer, dma_addr_t wr_ptr)
-{
-	size_t size;
-	size_t offset = wr_ptr - ring_buffer->daddr;
-	int ret;
-
-	if (wr_ptr + buffer_size > ring_buffer->daddr + ring_buffer->size) {
-		size = ring_buffer->daddr + ring_buffer->size - wr_ptr;
-		ret = wave5_vdi_write_memory(inst->dev, ring_buffer, offset, (u8 *)buffer, size);
-		if (ret < 0)
-			return ret;
-
-		ret = wave5_vdi_write_memory(inst->dev, ring_buffer, 0, (u8 *)buffer + size,
-					     buffer_size - size);
-		if (ret < 0)
-			return ret;
-	} else {
-		ret = wave5_vdi_write_memory(inst->dev, ring_buffer, offset, (u8 *)buffer,
-					     buffer_size);
-		if (ret < 0)
-			return ret;
-	}
-
-	return 0;
-}
-
-static int fill_ringbuffer(struct vpu_instance *inst)
-{
-	struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
-	struct v4l2_m2m_buffer *buf, *n;
-	int ret;
-
-	if (m2m_ctx->last_src_buf)  {
-		struct vpu_src_buffer *vpu_buf = wave5_to_vpu_src_buf(m2m_ctx->last_src_buf);
-
-		if (vpu_buf->consumed) {
-			dev_dbg(inst->dev->dev, "last src buffer already written\n");
-			return 0;
-		}
-	}
-
-	v4l2_m2m_for_each_src_buf_safe(m2m_ctx, buf, n) {
-		struct vb2_v4l2_buffer *vbuf = &buf->vb;
-		struct vpu_src_buffer *vpu_buf = wave5_to_vpu_src_buf(vbuf);
-		struct vpu_buf *ring_buffer = &inst->bitstream_vbuf;
-		size_t src_size = vb2_get_plane_payload(&vbuf->vb2_buf, 0);
-		void *src_buf = vb2_plane_vaddr(&vbuf->vb2_buf, 0);
-		dma_addr_t rd_ptr = 0;
-		dma_addr_t wr_ptr = 0;
-		size_t remain_size = 0;
-
-		if (vpu_buf->consumed) {
-			dev_dbg(inst->dev->dev, "already copied src buf (%u) to the ring buffer\n",
-				vbuf->vb2_buf.index);
-			continue;
-		}
-
-		if (!src_buf) {
-			dev_dbg(inst->dev->dev,
-				"%s: Acquiring kernel pointer to src buf (%u), fail\n",
-				__func__, vbuf->vb2_buf.index);
-			break;
-		}
-
-		ret = wave5_vpu_dec_get_bitstream_buffer(inst, &rd_ptr, &wr_ptr, &remain_size);
-		if (ret) {
-			/* Unable to acquire the mutex */
-			dev_err(inst->dev->dev, "Getting the bitstream buffer, fail: %d\n",
-				ret);
-			return ret;
-		}
-
-		dev_dbg(inst->dev->dev, "%s: rd_ptr %pad wr_ptr %pad", __func__, &rd_ptr, &wr_ptr);
-
-		if (remain_size < src_size) {
-			dev_dbg(inst->dev->dev,
-				"%s: remaining size: %zu < source size: %zu for src buf (%u)\n",
-				__func__, remain_size, src_size, vbuf->vb2_buf.index);
-			break;
-		}
-
-		ret = write_to_ringbuffer(inst, src_buf, src_size, ring_buffer, wr_ptr);
-		if (ret) {
-			dev_err(inst->dev->dev, "Write src buf (%u) to ring buffer, fail: %d\n",
-				vbuf->vb2_buf.index, ret);
-			return ret;
-		}
-
-		ret = wave5_vpu_dec_update_bitstream_buffer(inst, src_size);
-		if (ret) {
-			dev_dbg(inst->dev->dev,
-				"update_bitstream_buffer fail: %d for src buf (%u)\n",
-				ret, vbuf->vb2_buf.index);
-			break;
-		}
-
-		vpu_buf->consumed = true;
-
-		/* Don't write buffers passed the last one while draining. */
-		if (v4l2_m2m_is_last_draining_src_buf(m2m_ctx, vbuf)) {
-			dev_dbg(inst->dev->dev, "last src buffer written to the ring buffer\n");
-			break;
-		}
-	}
-
-	return 0;
-}
-

 static void wave5_vpu_dec_buf_queue_src(struct vb2_buffer *vb) {
  struct vpu_instance *inst = vb2_get_drv_priv(vb->vb2_queue);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -0
@@ -1236,6 +1257,11 @@ static void wave5_vpu_dec_buf_queue_src(struct vb2_buffer *vb)
  	vbuf->sequence = inst->queued_src_buf_num++;


  v4l2_m2m_buf_queue(m2m_ctx, vbuf);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_buf_queue_src(struct vb2_buffer *vb) · line 1264

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5 -0
+
+	INIT_LIST_HEAD(&vpu_buf->list);
+	mutex_lock(&inst->feed_lock);
+	list_add_tail(&vpu_buf->list, &inst->avail_src_bufs);
+	mutex_unlock(&inst->feed_lock);

This lack documentation, you didn't even mention the introduction of a new src buf list in the commit message.

Quoted 29 lines

 }

 static void wave5_vpu_dec_buf_queue_dst(struct vb2_buffer *vb)

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7 -1
@@ -1362,7 +1388,6 @@ static int wave5_vpu_dec_start_streaming(struct vb2_queue *q, unsigned int count
  				goto return_buffers;
  			}
  		}
-
  	}
  	pm_runtime_mark_last_busy(inst->dev->dev);
  	pm_runtime_put_autosuspend(inst->dev->dev);
@@ -1385,6 +1410,13 @@ static int streamoff_output(struct vb2_queue *q)
  	dma_addr_t new_rd_ptr;
  	struct dec_output_info dec_info;
  	unsigned int i;
+	struct vpu_src_buffer *vpu_buf, *tmp;
+
+	inst->retry = false;
+	inst->queuing_num = 0;
+
+	list_for_each_entry_safe(vpu_buf, tmp, &inst->avail_src_bufs, list)
+		list_del_init(&vpu_buf->list);


  for (i = 0; i < v4l2_m2m_num_dst_bufs_ready(m2m_ctx); i++) { ret = wave5_vpu_dec_set_disp_flag(inst, i);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -0
@@ -1481,10 +1513,8 @@ static void wave5_vpu_dec_stop_streaming(struct vb2_queue *q)


  wave5_vpu_dec_give_command(inst, DEC_GET_QUEUE_STATUS, &q_status);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_stop_streaming(struct vb2_queue *q) · line 1516

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -4
-		if (q_status.report_queue_count == 0)
-			break;
-
-		if (wave5_vpu_wait_interrupt(inst, VPU_DEC_TIMEOUT) < 0)
+		if ((inst->state == VPU_INST_STATE_STOP || q_status.instance_queue_count == 0) &&

State should be read with appropriate locking.

Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+		    q_status.report_queue_count == 0)

  break;

This loop have nothing to wait on anymore. It just actively pool on the HW. I noticed your third patch, but if this is what you address there, squash it somehow, this would break bisects.

The goal of that code was to ensure that this code will never run concurrently to finish_decode() once out of that while loop. I don't think you break that, but that will cause possibly high CPU load.

Quoted 17 lines


  if (wave5_vpu_dec_get_output_info(inst, &dec_output_info))

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
@@ -1577,13 +1607,23 @@ static void wave5_vpu_dec_device_run(void *priv)
  	struct queue_status_info q_status;
  	u32 fail_res = 0;
  	int ret = 0;
+	unsigned long flags;


  dev_dbg(inst->dev->dev, "%s: Fill the ring buffer with new bitstream data", __func__); pm_runtime_resume_and_get(inst->dev->dev);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3 -4
-	ret = fill_ringbuffer(inst);
-	if (ret) {
-		dev_warn(inst->dev->dev, "Filling ring buffer failed\n");
-		goto finish_job_and_return;
+	if (!inst->retry) {
+		mutex_lock(&inst->feed_lock);
+		ret = fill_ringbuffer(inst);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_device_run(void *priv) · line 1626

Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +10 -0
+		mutex_unlock(&inst->feed_lock);
+		if (ret < 0) {
+			dev_warn(inst->dev->dev, "Filling ring buffer failed\n");
+			goto finish_job_and_return;
+		} else if (!inst->eos &&
+			   inst->queuing_num == 0 &&
+			   inst->state == VPU_INST_STATE_PIC_RUN) {
+			dev_dbg(inst->dev->dev, "%s: no bitstream for feeding, so skip ", __func__);
+			goto finish_job_and_return;
+		}

There is quite some overlap between that local src buffer queue and all the code that depends on v4l2_m2m_num_src_bufs_ready(). Are you sure you aren't re-implementing something ? Also, this code belong to the PIC_RUN case in the below switch.

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_device_run(void *priv) · line 1664

Quoted 10 lines

  }

  switch (inst->state) {

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -0
@@ -1619,7 +1659,9 @@ static void wave5_vpu_dec_device_run(void *priv)
  		 * we had a chance to switch, which leads to an invalid state
  		 * change.
  		 */
+		spin_lock_irqsave(&inst->state_spinlock, flags);
  		switch_state(inst, VPU_INST_STATE_PIC_RUN);
+		spin_unlock_irqrestore(&inst->state_spinlock, flags);

Should be its own patch, and fixed everywhere. Add missing lockdep assert (if there is a spinlock version of it).

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_device_run(void *priv) · line 1681

Quoted 10 lines


  /*
  * During DRC, the picture decoding remains pending, so just leave the job

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -0
@@ -1634,12 +1676,14 @@ static void wave5_vpu_dec_device_run(void *priv)
  		ret = wave5_prepare_fb(inst);
  		if (ret) {
  			dev_warn(inst->dev->dev, "Framebuffer preparation, fail: %d\n", ret);
+			spin_lock_irqsave(&inst->state_spinlock, flags);
  			switch_state(inst, VPU_INST_STATE_STOP);
+			spin_unlock_irqrestore(&inst->state_spinlock, flags);

Same, this is separate fix.

Quoted 100 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -0
  			break;
  		}


  if (q_status.instance_queue_count) {

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -1
-			dev_dbg(inst->dev->dev, "%s: leave with active job", __func__);
+			v4l2_m2m_job_finish(inst->v4l2_m2m_dev, m2m_ctx);

  return;
  }

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +14 -6
@@ -1650,14 +1694,22 @@ static void wave5_vpu_dec_device_run(void *priv)
  			dev_err(inst->dev->dev,
  				"Frame decoding on m2m context (%p), fail: %d (result: %d)\n",
  				m2m_ctx, ret, fail_res);
-			break;
+			goto finish_job_and_return;
+		}
+
+		if (fail_res == WAVE5_SYSERR_QUEUEING_FAIL) {
+			inst->retry = true;
+		} else {
+			inst->retry = false;
+			if (!inst->eos)
+				inst->queuing_num--;
  		}
-		/* Return so that we leave this job active */
-		dev_dbg(inst->dev->dev, "%s: leave with active job", __func__);
-		return;
-	default:
-		WARN(1, "Execution of a job in state %s illegal.\n", state_to_str(inst->state));
  		break;
+	default:
+		if (!v4l2_m2m_has_stopped(m2m_ctx))
+			WARN(1, "Execution of a job in state %s illegal.\n",
+			     state_to_str(inst->state));
+		return;
  	}


 finish_job_and_return:

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -0
@@ -1676,10 +1728,7 @@ static void wave5_vpu_dec_job_abort(void *priv)
  	if (ret)
  		return;

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -4
-	ret = wave5_vpu_dec_set_eos_on_firmware(inst);
-	if (ret)
-		dev_warn(inst->dev->dev,
-			 "Setting EOS for the bitstream, fail: %d\n", ret);
+	v4l2_m2m_job_finish(inst->v4l2_m2m_dev, inst->v4l2_fh.m2m_ctx);

 }

 static int wave5_vpu_dec_job_ready(void *priv)

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -0
@@ -1755,6 +1804,8 @@ static int wave5_vpu_open_dec(struct file *filp)
  	inst->ops = &wave5_vpu_dec_inst_ops;


  spin_lock_init(&inst->state_spinlock);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -0
+	mutex_init(&inst->feed_lock);
+	INIT_LIST_HEAD(&inst->avail_src_bufs);


  inst->codec_info = kzalloc(sizeof(*inst->codec_info), GFP_KERNEL); if (!inst->codec_info)

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -0
@@ -1830,9 +1881,6 @@ static int wave5_vpu_open_dec(struct file *filp)
  	if (ret)
  		goto cleanup_inst;

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -3
-	if (list_empty(&dev->instances))
-		pm_runtime_use_autosuspend(inst->dev->dev);
-

  list_add_tail(&inst->list, &dev->instances);
  mutex_unlock(&dev->dev_lock);

drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c +0 -0
index 1e5fc5f8b856..cf20f774ed1b 100644
@@ -1774,9 +1774,6 @@ static int wave5_vpu_open_enc(struct file *filp)
  	if (ret)
  		goto cleanup_inst;

drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c +0 -3
-	if (list_empty(&dev->instances))
-		pm_runtime_use_autosuspend(inst->dev->dev);
-

  list_add_tail(&inst->list, &dev->instances);
  mutex_unlock(&dev->dev_lock);

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -2
index e5e879a13e8b..c1737fac6edd 100644
@@ -207,8 +207,6 @@ int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res)
  	int retry = 0;
  	struct vpu_device *vpu_dev = inst->dev;
  	int i;
-	int inst_count = 0;
-	struct vpu_instance *inst_elm;


  *fail_res = 0;
  if (!inst->codec_info)

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -0
@@ -233,6 +231,16 @@ int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res)
  		    retry++ >= MAX_FIRMWARE_CALL_RETRY) {
  			ret = -ETIMEDOUT;
  			goto unlock_and_return;

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res) · line 243

Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +10 -0
+		} else if (*fail_res == WAVE5_SYSERR_VPU_STILL_RUNNING) {
+			struct dec_output_info dec_info;
+
+			mutex_unlock(&vpu_dev->hw_lock);
+			wave5_vpu_dec_get_output_info(inst, &dec_info);
+			ret = mutex_lock_interruptible(&vpu_dev->hw_lock);
+			if (ret) {
+				pm_runtime_put_sync(inst->dev->dev);
+				return ret;
+			}

Please try and simplify all the branches. Something like:

			if (ret == 0)
				break;
	
			if (*fail_res != WAVE5_SYSERR_VPU_STILL_RUNNING) {
					dev_warn(inst->dev->dev, "dec_finish_seq timed out\n");
					goto unlock_and_return;
			}

			if (retry++ >= MAX_FIRMWARE_CALL_RETRY) {
					ret = -ETIMEDOUT;
					goto unlock_and_return;
			}

			mutex_unlock(&vpu_dev->hw_lock);
			wave5_vpu_dec_get_output_info(inst, &dec_info);
			ret = mutex_lock_interruptible(&vpu_dev->hw_lock);
			if (ret) {
				pm_runtime_put_sync(inst->dev->dev);
				return ret;
			}

My impression on that change is that it is independent and could be introduced separately.

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res) · line 256

Quoted 12 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -0
  		}
  	} while (ret != 0);

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -0
@@ -249,11 +257,7 @@ int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res)
  	}


  wave5_vdi_free_dma_memory(vpu_dev, &p_dec_info->vb_task);

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -5
-
-	list_for_each_entry(inst_elm, &vpu_dev->instances, list)
-		inst_count++;
-	if (inst_count == 1)
-		pm_runtime_dont_use_autosuspend(vpu_dev->dev);

This seems like some other fix, that should be in its own commit.

Quoted 17 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +1 -0
+	mutex_destroy(&inst->feed_lock);


 unlock_and_return:
  mutex_unlock(&vpu_dev->hw_lock);

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -2
@@ -720,8 +724,6 @@ int wave5_vpu_enc_close(struct vpu_instance *inst, u32 *fail_res)
  	int ret;
  	int retry = 0;
  	struct vpu_device *vpu_dev = inst->dev;
-	int inst_count = 0;
-	struct vpu_instance *inst_elm;


  *fail_res = 0;
  if (!inst->codec_info)

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -0
@@ -765,11 +767,6 @@ int wave5_vpu_enc_close(struct vpu_instance *inst, u32 *fail_res)


  wave5_vdi_free_dma_memory(vpu_dev, &p_enc_info->vb_task);

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_enc_close(struct vpu_instance *inst, u32 *fail_res) · line 772

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -5
-	list_for_each_entry(inst_elm, &vpu_dev->instances, list)
-		inst_count++;
-	if (inst_count == 1)
-		pm_runtime_dont_use_autosuspend(vpu_dev->dev);
-

Same.

drivers/media/platform/chips-media/wave5/wave5-vpuapi.h enum set_param_option { · line 166

Quoted 13 lines

  mutex_unlock(&vpu_dev->hw_lock);
  pm_runtime_put_sync(inst->dev->dev);

drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +0 -0
index 45615c15beca..6ca1ddc67c64 100644
@@ -163,7 +163,7 @@ enum set_param_option {


 #define BUFFER_MARGIN 4096

drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +1 -1
-#define MAX_FIRMWARE_CALL_RETRY			10
+#define MAX_FIRMWARE_CALL_RETRY			30

I thought 10s was already very long, am I right this means it can get stuck for 30s now ?

drivers/media/platform/chips-media/wave5/wave5-vpuapi.h struct vpu_instance { · line 816

Quoted 9 lines


 #define VDI_LITTLE_ENDIAN 0x0

drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +2 -0
@@ -812,6 +812,9 @@ struct vpu_instance {
  	bool cbcr_interleave;
  	bool nv21;
  	bool eos;
+	bool retry;
+	int queuing_num;

Some comment would be helpful. What does "retry" stand for, and how is queing_num different from v4l2_m2m_num_src_bufs_ready() ?

Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +1 -0
+	struct mutex feed_lock; /* lock for feeding bitstream buffers */
  	struct vpu_buf bitstream_vbuf;
  	dma_addr_t last_rd_ptr;
  	size_t remaining_consumed_bytes;
Signature · 2 lines
Nicolas Dufresne
Principal Engineer at Collabora
author maintainer
#3 lore raw

RE: [RESEND PATCH v0 1/3] media: chips-media: wave5: Improve performance of decoder

Hi Nicolas

Thanks for your review.
Can you check my comment for your review?

Quoted 63 lines

-----Original Message-----
From: Nicolas Dufresne <[email protected]>
Sent: Thursday, April 3, 2025 3:03 AM
To: jackson.lee <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; lafley.kim <[email protected]>; [email protected]; [email protected]; Nas Chung <[email protected]>
Subject: Re: [RESEND PATCH v0 1/3] media: chips-media: wave5: Improve performance of decoder

Le mercredi 19 mars 2025 à 12:50 +0900, Jackson.lee a écrit :

Quoted 3 lines

From: Jackson Lee <[email protected]>

The existing way for decoding frames was to wait until each frame was

Suggestion:
The current decoding method was to...

Quoted 5 lines

decoded after feeding a bitstream. As a result, performance was low and Wave5 could not achieve max pixel processing rate.

Update driver to use an asynchronous approach for decoding and feeding a

Update the driver ..

Quoted 1 line

bitstream in order to achieve full capabilities of the device.

This second part lacks the technical design choices needed for your reviewer ;-P I did read the patch once, and am under the impression that your approach is to not finish the job anymore.

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst,

Quoted 17 lines

---
 .../chips-media/wave5/wave5-vpu-dec.c         | 344
++++++++++--------
 .../chips-media/wave5/wave5-vpu-enc.c         |   3 - .../platform/chips-media/wave5/wave5-vpuapi.c |  25 +- .../platform/chips-media/wave5/wave5-vpuapi.h |   5 +-

 4 files changed, 211 insertions(+), 166 deletions(-)
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -0
index fd71f0c43ac3..cc47da509703 100644
@@ -230,12 +230,147 @@ static int start_decode(struct vpu_instance *inst,

u32 *fail_res)

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)

Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -0
  		switch_state(inst, VPU_INST_STATE_STOP);
 
  		dev_dbg(inst->dev->dev, "%s: pic run failed / finish job",

__func__);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)

Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -1
-		v4l2_m2m_job_finish(inst->v4l2_m2m_dev, m2m_ctx);
  	}
 
  	return ret;
  }
 
+static int write_to_ringbuffer(struct vpu_instance *inst, void *buffer,

size_t buffer_size,

Quoted 8 lines

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 239

Quoted 1 line
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+			       struct vpu_buf *ring_buffer, dma_addr_t wr_ptr)

{

It seems that from now on this needs to be called with the feed_lock, so add:

	 lockdep_assert_held(&inst->feed_lock);

The "feed_lock" mutex is for protecting "inst->avail_src_bufs" this list, so the feed_lock should be used when "static int fill_ringbuffer(struct vpu_instance *inst)" is called, not the write_to_ringbuffer. So I think the feed_lock should not be added in the write_to_ringbuffer function.

Quoted 20 lines

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5 -0
+	size_t size;
+	size_t offset = wr_ptr - ring_buffer->daddr;
+	int ret;
+
+	if (wr_ptr + buffer_size > ring_buffer->daddr + ring_buffer->size)

{

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, · line 246

Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -0
+		size = ring_buffer->daddr + ring_buffer->size - wr_ptr;
+		ret = wave5_vdi_write_memory(inst->dev, ring_buffer, offset,

(u8 *)buffer, size);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)

Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +4 -0
+		if (ret < 0)
+			return ret;
+
+		ret = wave5_vdi_write_memory(inst->dev, ring_buffer, 0, (u8

*)buffer + size,

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5 -0
+					     buffer_size - size);
+		if (ret < 0)
+			return ret;
+	} else {
+		ret = wave5_vdi_write_memory(inst->dev, ring_buffer, offset,

(u8 *)buffer,

Quoted 11 lines

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 263

Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7 -0
+					     buffer_size);
+		if (ret < 0)
+			return ret;
+	}
+
+	return 0;
+}

I have a pretty strong preference in seeing code being moved around in seperate patch documented as "non functional changes". This way I know I don't have to review again that code.

I will do it.

Quoted 49 lines

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, · line 264

Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -0
+
+static int fill_ringbuffer(struct vpu_instance *inst) {

Same, I don't want to have to do back and fourth to check if you changed anything. Also, after the refactoring, the lockdeup assert can be added in the new patch, which makes the introduction of the new lock very obvious.

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)

Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +10 -0
+	struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
+	struct vpu_src_buffer *vpu_buf;
+	int ret = 0;
+
+	if (m2m_ctx->last_src_buf)  {
+		struct vpu_src_buffer *vpu_buf =
+wave5_to_vpu_src_buf(m2m_ctx->last_src_buf);
+
+		if (vpu_buf->consumed) {
+			dev_dbg(inst->dev->dev, "last src buffer already

written\n");

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5 -0
+			return 0;
+		}
+	}
+
+	list_for_each_entry(vpu_buf, &inst->avail_src_bufs, list) {

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)

Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +10 -0
+		struct vb2_v4l2_buffer *vbuf = &vpu_buf->v4l2_m2m_buf.vb;
+		struct vpu_buf *ring_buffer = &inst->bitstream_vbuf;
+		size_t src_size = vb2_get_plane_payload(&vbuf->vb2_buf, 0);
+		void *src_buf = vb2_plane_vaddr(&vbuf->vb2_buf, 0);
+		dma_addr_t rd_ptr = 0;
+		dma_addr_t wr_ptr = 0;
+		size_t remain_size = 0;
+
+		if (vpu_buf->consumed) {
+			dev_dbg(inst->dev->dev, "already copied src buf (%u)

to the ring buffer\n",

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)

Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7 -0
+				vbuf->vb2_buf.index);
+			continue;
+		}
+
+		if (!src_buf) {
+			dev_dbg(inst->dev->dev,
+				"%s: Acquiring kernel pointer to src buf (%u),

fail\n",

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5 -0
+				__func__, vbuf->vb2_buf.index);
+			break;
+		}
+
+		ret = wave5_vpu_dec_get_bitstream_buffer(inst, &rd_ptr,

&wr_ptr, &remain_size);

Quoted 6 lines

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 304

Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -0
+		if (ret) {
+			/* Unable to acquire the mutex */

Might want to fix that comment in a separate patch, perhaps even remove that ?

Do you want to remove the comment in a separate patch ?

Quoted 149 lines

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, · line 304

Quoted 1 line
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+			dev_err(inst->dev->dev, "Getting the bitstream buffer,

fail: %d\n",

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5 -0
+				ret);
+			return ret;
+		}
+
+		dev_dbg(inst->dev->dev, "%s: rd_ptr %pad wr_ptr %pad",

__func__,

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5 -0
+&rd_ptr, &wr_ptr);
+
+		if (remain_size < src_size) {
+			dev_dbg(inst->dev->dev,
+				"%s: remaining size: %zu < source size: %zu for

src buf (%u)\n",

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)

Quoted 6 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5 -0
+				__func__, remain_size, src_size, vbuf-
 vb2_buf.index);
+			break;
+		}
+
+		ret = write_to_ringbuffer(inst, src_buf, src_size,

ring_buffer, wr_ptr);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, · line 322

Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -0
+		if (ret) {
+			dev_err(inst->dev->dev, "Write src buf (%u) to ring

buffer, fail: %d\n",

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)

Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +8 -0
+				vbuf->vb2_buf.index, ret);
+			return ret;
+		}
+
+		ret = wave5_vpu_dec_update_bitstream_buffer(inst, src_size);
+		if (ret) {
+			dev_dbg(inst->dev->dev,
+				"update_bitstream_buffer fail: %d for src buf

(%u)\n",

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)

Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7 -0
+				ret, vbuf->vb2_buf.index);
+			break;
+		}
+
+		vpu_buf->consumed = true;
+
+		/* Don't write buffers passed the last one while draining.

*/

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, · line 339

Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -0
+		if (v4l2_m2m_is_last_draining_src_buf(m2m_ctx, vbuf)) {
+			dev_dbg(inst->dev->dev, "last src buffer written to

the ring buffer\n");

Quoted 12 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +12 -0
+			break;
+		}
+
+		inst->queuing_num++;
+		list_del_init(&vpu_buf->list);
+		break;
+	}
+
+	return ret;
+}
+
+static void wave5_vpu_dec_feed_remaining(struct vpu_instance *inst) {

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 361

Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +10 -0
+	int ret = 0;
+	struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
+	u32 fail_res = 0;
+
+	mutex_lock(&inst->feed_lock);
+	ret = fill_ringbuffer(inst);
+	mutex_unlock(&inst->feed_lock);
+	if (ret) {
+		dev_warn(inst->dev->dev, "Filling ring buffer failed\n");
+		return;

I don't think the error handling is correct. What will happen with the job ?

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 364

Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3 -0
+	}
+
+	ret = start_decode(inst, &fail_res);

I'm a little worried seeing this called outside of device_run() context.

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res)

Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3 -0
+	if (ret) {
+		dev_err(inst->dev->dev,
+			"Frame decoding on m2m context (%p), fail: %d

(result: %d)\n",

Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +6 -0
+			m2m_ctx, ret, fail_res);
+	}
+
+	v4l2_m2m_job_finish(inst->v4l2_m2m_dev, m2m_ctx);
 
 
 
+}
+
Quoted 4 lines

 static void flag_last_buffer_done(struct vpu_instance *inst) {
  struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx; @@ -347,7 +482,6 @@ static void wave5_vpu_dec_finish_decode(struct vpu_instance

*inst)

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_finish_decode(struct

Quoted 9 lines

  struct vb2_v4l2_buffer *dec_buf = NULL;
  struct vb2_v4l2_buffer *disp_buf = NULL; struct vb2_queue *dst_vq = v4l2_m2m_get_dst_vq(m2m_ctx);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-	struct queue_status_info q_status;

  dev_dbg(inst->dev->dev, "%s: Fetch output info from firmware.", __func__);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -0
@@ -441,20 +575,6 @@ static void wave5_vpu_dec_finish_decode(struct

vpu_instance *inst)

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_finish_decode(struct vpu_instance *inst)

Quoted 6 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -3
  		}
  		spin_unlock_irqrestore(&inst->state_spinlock, flags);
  	}
-
-	/*
-	 * During a resolution change and while draining, the firmware may

flush

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_finish_decode(struct · line 447

Quoted 1 line
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-	 * the reorder queue regardless of having a matching decoding

operation

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_finish_decode(struct vpu_instance *inst)

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -5
-	 * pending. Only terminate the job if there are no more IRQ coming.
-	 */
-	wave5_vpu_dec_give_command(inst, DEC_GET_QUEUE_STATUS, &q_status);
-	if (q_status.report_queue_count == 0 &&
-	    (q_status.instance_queue_count == 0 ||

dec_info.sequence_changed)) {

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_finish_decode(struct vpu_instance *inst) · line 457

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -5
-		dev_dbg(inst->dev->dev, "%s: finishing job.\n", __func__);
-		pm_runtime_mark_last_busy(inst->dev->dev);
-		pm_runtime_put_autosuspend(inst->dev->dev);
-		v4l2_m2m_job_finish(inst->v4l2_m2m_dev, m2m_ctx);
-	}

This removal is why I believe some seek stress testing are needed. I'll try and find some tests file I crafted last year.

Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -0
  }
 

 static int wave5_vpu_dec_querycap(struct file *file, void *fh, struct v4l2_capability *cap) @@ -794,11 +914,21 @@ static int

wave5_vpu_dec_stop(struct vpu_instance *inst)

Quoted 16 lines

  }

  if (inst->state != VPU_INST_STATE_NONE) {

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3 -0
+		struct vb2_v4l2_buffer *vbuf;
+		struct vpu_src_buffer *vpu_buf;
+

  /*
  * Temporarily release the state_spinlock so that subsequent
  * calls do not block on a mutex while inside this spinlock. */ spin_unlock_irqrestore(&inst->state_spinlock, flags);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5 -0
+		vbuf = v4l2_m2m_last_src_buf(m2m_ctx);
+		if (vbuf) {
+			vpu_buf = wave5_to_vpu_src_buf(vbuf);
+			if (!vpu_buf->consumed)
+				wave5_vpu_dec_feed_remaining(inst);
Quoted 9 lines

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c

Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -0
+		}
+

Do you really need to add all these checks ? Shouldn't you just "feed_remaining()" and move on ? I also don't think its a good idea to call v4l2_m2m_job_finish() in code that can run concurrently with the device_run() function. This should only happen if you are certain the VPU instance is stopped already.

While testing fluster, stop command is sometimes called before the last bitstream queued is handled. The stop command sets EOS before handling the last bitstream, so sometimes mismatch happens. I think the code should be add in the wave5_vpu_dec_stop.

Quoted 201 lines

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance

Quoted 5 lines

  ret = wave5_vpu_dec_set_eos_on_firmware(inst); if (ret)
  return ret;

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -0
@@ -1116,115 +1246,6 @@ static int wave5_prepare_fb(struct vpu_instance

*inst)

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)

Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
  	return 0;
  }
 
-static int write_to_ringbuffer(struct vpu_instance *inst, void *buffer,

size_t buffer_size,

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)

Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -7
-			       struct vpu_buf *ring_buffer, dma_addr_t wr_ptr)
-{
-	size_t size;
-	size_t offset = wr_ptr - ring_buffer->daddr;
-	int ret;
-
-	if (wr_ptr + buffer_size > ring_buffer->daddr + ring_buffer->size)

{

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance · line 1128

Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -2
-		size = ring_buffer->daddr + ring_buffer->size - wr_ptr;
-		ret = wave5_vdi_write_memory(inst->dev, ring_buffer, offset,

(u8 *)buffer, size);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)

Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -4
-		if (ret < 0)
-			return ret;
-
-		ret = wave5_vdi_write_memory(inst->dev, ring_buffer, 0, (u8

*)buffer + size,

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -5
-					     buffer_size - size);
-		if (ret < 0)
-			return ret;
-	} else {
-		ret = wave5_vdi_write_memory(inst->dev, ring_buffer, offset,

(u8 *)buffer,

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -5
-					     buffer_size);
-		if (ret < 0)
-			return ret;
-	}
-

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)

Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -10
-	return 0;
-}
-
-static int fill_ringbuffer(struct vpu_instance *inst) -{
-	struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
-	struct v4l2_m2m_buffer *buf, *n;
-	int ret;
-
-	if (m2m_ctx->last_src_buf)  {
-		struct vpu_src_buffer *vpu_buf =

wave5_to_vpu_src_buf(m2m_ctx->last_src_buf);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)

Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -3
-
-		if (vpu_buf->consumed) {
-			dev_dbg(inst->dev->dev, "last src buffer already

written\n");

Quoted 6 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -6
-			return 0;
-		}
-	}
-
-	v4l2_m2m_for_each_src_buf_safe(m2m_ctx, buf, n) {
-		struct vb2_v4l2_buffer *vbuf = &buf->vb;

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)

Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -10
-		struct vpu_src_buffer *vpu_buf = wave5_to_vpu_src_buf(vbuf);
-		struct vpu_buf *ring_buffer = &inst->bitstream_vbuf;
-		size_t src_size = vb2_get_plane_payload(&vbuf->vb2_buf, 0);
-		void *src_buf = vb2_plane_vaddr(&vbuf->vb2_buf, 0);
-		dma_addr_t rd_ptr = 0;
-		dma_addr_t wr_ptr = 0;
-		size_t remain_size = 0;
-
-		if (vpu_buf->consumed) {
-			dev_dbg(inst->dev->dev, "already copied src buf (%u)

to the ring buffer\n",

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)

Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -7
-				vbuf->vb2_buf.index);
-			continue;
-		}
-
-		if (!src_buf) {
-			dev_dbg(inst->dev->dev,
-				"%s: Acquiring kernel pointer to src buf (%u),

fail\n",

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -5
-				__func__, vbuf->vb2_buf.index);
-			break;
-		}
-
-		ret = wave5_vpu_dec_get_bitstream_buffer(inst, &rd_ptr,

&wr_ptr, &remain_size);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)

Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -3
-		if (ret) {
-			/* Unable to acquire the mutex */
-			dev_err(inst->dev->dev, "Getting the bitstream buffer,

fail: %d\n",

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -5
-				ret);
-			return ret;
-		}
-
-		dev_dbg(inst->dev->dev, "%s: rd_ptr %pad wr_ptr %pad",

__func__, &rd_ptr, &wr_ptr);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)

Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -4
-
-		if (remain_size < src_size) {
-			dev_dbg(inst->dev->dev,
-				"%s: remaining size: %zu < source size: %zu for

src buf (%u)\n",

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)

Quoted 6 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -5
-				__func__, remain_size, src_size, vbuf-
 vb2_buf.index);
-			break;
-		}
-
-		ret = write_to_ringbuffer(inst, src_buf, src_size,

ring_buffer, wr_ptr);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance · line 1203

Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -2
-		if (ret) {
-			dev_err(inst->dev->dev, "Write src buf (%u) to ring

buffer, fail: %d\n",

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)

Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -8
-				vbuf->vb2_buf.index, ret);
-			return ret;
-		}
-
-		ret = wave5_vpu_dec_update_bitstream_buffer(inst, src_size);
-		if (ret) {
-			dev_dbg(inst->dev->dev,
-				"update_bitstream_buffer fail: %d for src buf

(%u)\n",

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)

Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -7
-				ret, vbuf->vb2_buf.index);
-			break;
-		}
-
-		vpu_buf->consumed = true;
-
-		/* Don't write buffers passed the last one while draining.

*/

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance · line 1220

Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -2
-		if (v4l2_m2m_is_last_draining_src_buf(m2m_ctx, vbuf)) {
-			dev_dbg(inst->dev->dev, "last src buffer written to

the ring buffer\n");

Quoted 11 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -7
-			break;
-		}
-	}
-
-	return 0;
-}
-
  static void wave5_vpu_dec_buf_queue_src(struct vb2_buffer *vb)
  {

  struct vpu_instance *inst = vb2_get_drv_priv(vb->vb2_queue); @@ -1236,6 +1257,11 @@ static void wave5_vpu_dec_buf_queue_src(struct

vb2_buffer *vb)

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_buf_queue_src(struct vb2_buffer *vb) · line 1264

Quoted 8 lines

  vbuf->sequence = inst->queued_src_buf_num++;

  v4l2_m2m_buf_queue(m2m_ctx, vbuf);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5 -0
+
+	INIT_LIST_HEAD(&vpu_buf->list);
+	mutex_lock(&inst->feed_lock);
+	list_add_tail(&vpu_buf->list, &inst->avail_src_bufs);
+	mutex_unlock(&inst->feed_lock);

This lack documentation, you didn't even mention the introduction of a new src buf list in the commit message.

Quoted 4 lines

 }

 static void wave5_vpu_dec_buf_queue_dst(struct vb2_buffer *vb) @@ -1362,7 +1388,6 @@ static int wave5_vpu_dec_start_streaming(struct

vb2_queue *q, unsigned int count

Quoted 20 lines

  goto return_buffers; }
  }
-
  } pm_runtime_mark_last_busy(inst->dev->dev); pm_runtime_put_autosuspend(inst->dev->dev);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7 -0
@@ -1385,6 +1410,13 @@ static int streamoff_output(struct vb2_queue *q)
  	dma_addr_t new_rd_ptr;
  	struct dec_output_info dec_info;
  	unsigned int i;
+	struct vpu_src_buffer *vpu_buf, *tmp;
+
+	inst->retry = false;
+	inst->queuing_num = 0;
+
+	list_for_each_entry_safe(vpu_buf, tmp, &inst->avail_src_bufs, list)
+		list_del_init(&vpu_buf->list);
 
  	for (i = 0; i < v4l2_m2m_num_dst_bufs_ready(m2m_ctx); i++) {
Quoted 1 line

  ret = wave5_vpu_dec_set_disp_flag(inst, i); @@ -1481,10

+1513,8 @@

Quoted 3 lines

static void wave5_vpu_dec_stop_streaming(struct vb2_queue *q)

  wave5_vpu_dec_give_command(inst, DEC_GET_QUEUE_STATUS,

&q_status);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_stop_streaming(struct vb2_queue *q)

Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -4
-		if (q_status.report_queue_count == 0)
-			break;
-
-		if (wave5_vpu_wait_interrupt(inst, VPU_DEC_TIMEOUT) < 0)
+		if ((inst->state == VPU_INST_STATE_STOP ||
+q_status.instance_queue_count == 0) &&

State should be read with appropriate locking.

Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+		    q_status.report_queue_count == 0)

  break;

This loop have nothing to wait on anymore. It just actively pool on the HW. I noticed your third patch, but if this is what you address there, squash it somehow, this would break bisects.

Quoted 4 lines

The goal of that code was to ensure that this code will never run concurrently to finish_decode() once out of that while loop. I don't think you break that, but that will cause possibly high CPU load.

The code is not related with the third patch, there was sometimes HW interrupt misses since changing to asynchronous, so an infinite wait happened when closed So I removed it, but I think there is no high CPU load because it runs very short time.

Quoted 29 lines
Quoted 2 lines

  if (wave5_vpu_dec_get_output_info(inst, &dec_output_info))

@@

Quoted 7 lines

-1577,13 +1607,23 @@ static void wave5_vpu_dec_device_run(void *priv) struct queue_status_info q_status;
  u32 fail_res = 0;
  int ret = 0;

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+	unsigned long flags;

  dev_dbg(inst->dev->dev, "%s: Fill the ring buffer with new

bitstream data", __func__);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_device_run(void *priv)

Quoted 11 lines

  pm_runtime_resume_and_get(inst->dev->dev);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +6 -4
-	ret = fill_ringbuffer(inst);
-	if (ret) {
-		dev_warn(inst->dev->dev, "Filling ring buffer failed\n");
-		goto finish_job_and_return;
+	if (!inst->retry) {
+		mutex_lock(&inst->feed_lock);
+		ret = fill_ringbuffer(inst);
+		mutex_unlock(&inst->feed_lock);
+		if (ret < 0) {
+			dev_warn(inst->dev->dev, "Filling ring buffer

failed\n");

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_device_run(void *priv)

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5 -0
+			goto finish_job_and_return;
+		} else if (!inst->eos &&
+			   inst->queuing_num == 0 &&
+			   inst->state == VPU_INST_STATE_PIC_RUN) {
+			dev_dbg(inst->dev->dev, "%s: no bitstream for feeding,

so skip ", __func__);

Quoted 8 lines

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c

Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -0
+			goto finish_job_and_return;
+		}

There is quite some overlap between that local src buffer queue and all the code that depends on v4l2_m2m_num_src_bufs_ready(). Are you sure you aren't re-implementing something ? Also, this code belong to the PIC_RUN case in the below switch.

After src buffer was queued, v4l2_m2m_num_src_bufs_ready() always returns true until src buffer was removed in the finish_decode since get_result. So checking if there is buffer or not in the src bufs list was needed in the device_run.

Quoted 251 lines

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_device_run(void *priv) · line 1664

Quoted 10 lines

  }

  switch (inst->state) {

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -0
@@ -1619,7 +1659,9 @@ static void wave5_vpu_dec_device_run(void *priv)
  		 * we had a chance to switch, which leads to an invalid state
  		 * change.
  		 */
+		spin_lock_irqsave(&inst->state_spinlock, flags);
  		switch_state(inst, VPU_INST_STATE_PIC_RUN);
+		spin_unlock_irqrestore(&inst->state_spinlock, flags);

Should be its own patch, and fixed everywhere. Add missing lockdep assert (if there is a spinlock version of it).

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_device_run(void *priv) · line 1666

Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -0
  		/*
  		 * During DRC, the picture decoding remains pending, so just

leave

Quoted 1 line

the job @@ -1634,12 +1676,14 @@ static void

wave5_vpu_dec_device_run(void *priv)

Quoted 3 lines

  ret = wave5_prepare_fb(inst);
  if (ret) {
  dev_warn(inst->dev->dev, "Framebuffer preparation,

fail: %d\n",

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c

Quoted 4 lines

ret);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+			spin_lock_irqsave(&inst->state_spinlock, flags);

  switch_state(inst, VPU_INST_STATE_STOP);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+			spin_unlock_irqrestore(&inst->state_spinlock, flags);

Same, this is separate fix.

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c

Quoted 5 lines

  break;
  }

  if (q_status.instance_queue_count) {

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-			dev_dbg(inst->dev->dev, "%s: leave with active job",

__func__);

Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+			v4l2_m2m_job_finish(inst->v4l2_m2m_dev, m2m_ctx);

  return;
  }

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -0
@@ -1650,14 +1694,22 @@ static void wave5_vpu_dec_device_run(void *priv)
  			dev_err(inst->dev->dev,

  "Frame decoding on m2m context (%p), fail: %d

(result: %d)\n",

Quoted 12 lines

  m2m_ctx, ret, fail_res);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +9 -1
-			break;
+			goto finish_job_and_return;
+		}
+
+		if (fail_res == WAVE5_SYSERR_QUEUEING_FAIL) {
+			inst->retry = true;
+		} else {
+			inst->retry = false;
+			if (!inst->eos)
+				inst->queuing_num--;

  }

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c

Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -2
-		/* Return so that we leave this job active */
-		dev_dbg(inst->dev->dev, "%s: leave with active job",

__func__);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_device_run(void *priv)

Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -3
-		return;
-	default:
-		WARN(1, "Execution of a job in state %s illegal.\n",

state_to_str(inst->state));

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance *inst,

Quoted 61 lines

  break;

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5 -0
+	default:
+		if (!v4l2_m2m_has_stopped(m2m_ctx))
+			WARN(1, "Execution of a job in state %s illegal.\n",
+			     state_to_str(inst->state));
+		return;

  }

 finish_job_and_return:

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -4
@@ -1676,10 +1728,7 @@ static void wave5_vpu_dec_job_abort(void *priv)
  	if (ret)
  		return;
 
-	ret = wave5_vpu_dec_set_eos_on_firmware(inst);
-	if (ret)
-		dev_warn(inst->dev->dev,
-			 "Setting EOS for the bitstream, fail: %d\n", ret);
+	v4l2_m2m_job_finish(inst->v4l2_m2m_dev, inst->v4l2_fh.m2m_ctx);
  }
 

 static int wave5_vpu_dec_job_ready(void *priv) @@ -1755,6 +1804,8 @@ static int wave5_vpu_open_dec(struct file *filp)
  inst->ops = &wave5_vpu_dec_inst_ops;

  spin_lock_init(&inst->state_spinlock);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -0
+	mutex_init(&inst->feed_lock);
+	INIT_LIST_HEAD(&inst->avail_src_bufs);

  inst->codec_info = kzalloc(sizeof(*inst->codec_info), GFP_KERNEL); if (!inst->codec_info)

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -3
@@ -1830,9 +1881,6 @@ static int wave5_vpu_open_dec(struct file *filp)
  	if (ret)
  		goto cleanup_inst;
 
-	if (list_empty(&dev->instances))
-		pm_runtime_use_autosuspend(inst->dev->dev);
-
  	list_add_tail(&inst->list, &dev->instances);
 
  	mutex_unlock(&dev->dev_lock);
drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c +0 -3
index 1e5fc5f8b856..cf20f774ed1b 100644
@@ -1774,9 +1774,6 @@ static int wave5_vpu_open_enc(struct file *filp)
  	if (ret)
  		goto cleanup_inst;
 
-	if (list_empty(&dev->instances))
-		pm_runtime_use_autosuspend(inst->dev->dev);
-
  	list_add_tail(&inst->list, &dev->instances);
 
  	mutex_unlock(&dev->dev_lock);
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -0
index e5e879a13e8b..c1737fac6edd 100644
@@ -207,8 +207,6 @@ int wave5_vpu_dec_close(struct vpu_instance *inst,

u32 *fail_res)

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance *inst,

Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -2
  	int retry = 0;
  	struct vpu_device *vpu_dev = inst->dev;
  	int i;
-	int inst_count = 0;
-	struct vpu_instance *inst_elm;
 
  	*fail_res = 0;
  	if (!inst->codec_info)
@@ -233,6 +231,16 @@ int wave5_vpu_dec_close(struct vpu_instance *inst,

u32 *fail_res)

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res) · line 243

Quoted 13 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +10 -0
  		    retry++ >= MAX_FIRMWARE_CALL_RETRY) {
  			ret = -ETIMEDOUT;
  			goto unlock_and_return;
+		} else if (*fail_res == WAVE5_SYSERR_VPU_STILL_RUNNING) {
+			struct dec_output_info dec_info;
+
+			mutex_unlock(&vpu_dev->hw_lock);
+			wave5_vpu_dec_get_output_info(inst, &dec_info);
+			ret = mutex_lock_interruptible(&vpu_dev->hw_lock);
+			if (ret) {
+				pm_runtime_put_sync(inst->dev->dev);
+				return ret;
+			}

Please try and simplify all the branches. Something like:

			if (ret == 0)
				break;

			if (*fail_res != WAVE5_SYSERR_VPU_STILL_RUNNING) {
					dev_warn(inst->dev->dev, "dec_finish_seq
timed out\n");
					goto unlock_and_return;
			}

			if (retry++ >= MAX_FIRMWARE_CALL_RETRY) {
					ret = -ETIMEDOUT;
					goto unlock_and_return;
			}

			mutex_unlock(&vpu_dev->hw_lock);
			wave5_vpu_dec_get_output_info(inst, &dec_info);
			ret = mutex_lock_interruptible(&vpu_dev->hw_lock);
			if (ret) {
				pm_runtime_put_sync(inst->dev->dev);
				return ret;
			}

My impression on that change is that it is independent and could be introduced separately.

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance *inst,

Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -0
  		}
  	} while (ret != 0);
 
@@ -249,11 +257,7 @@ int wave5_vpu_dec_close(struct vpu_instance *inst,

u32 *fail_res)

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res) · line 256

Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -5
  	}
 
  	wave5_vdi_free_dma_memory(vpu_dev, &p_dec_info->vb_task);
-
-	list_for_each_entry(inst_elm, &vpu_dev->instances, list)
-		inst_count++;
-	if (inst_count == 1)
-		pm_runtime_dont_use_autosuspend(vpu_dev->dev);

This seems like some other fix, that should be in its own commit.

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_enc_close(struct vpu_instance *inst,

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +1 -0
+	mutex_destroy(&inst->feed_lock);
 

 unlock_and_return:
  mutex_unlock(&vpu_dev->hw_lock);

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -0
@@ -720,8 +724,6 @@ int wave5_vpu_enc_close(struct vpu_instance *inst,

u32 *fail_res)

Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -2
  	int ret;
  	int retry = 0;
  	struct vpu_device *vpu_dev = inst->dev;
-	int inst_count = 0;
-	struct vpu_instance *inst_elm;
 
  	*fail_res = 0;
  	if (!inst->codec_info)

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_enc_close(struct vpu_instance · line 773

Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -5
@@ -765,11 +767,6 @@ int wave5_vpu_enc_close(struct vpu_instance
 *inst, u32 *fail_res)
 
  	wave5_vdi_free_dma_memory(vpu_dev, &p_enc_info->vb_task);
 
-	list_for_each_entry(inst_elm, &vpu_dev->instances, list)
-		inst_count++;
-	if (inst_count == 1)
-		pm_runtime_dont_use_autosuspend(vpu_dev->dev);
-

Same.

Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -0
  	mutex_unlock(&vpu_dev->hw_lock);
  	pm_runtime_put_sync(inst->dev->dev);
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +0 -0
index 45615c15beca..6ca1ddc67c64 100644

drivers/media/platform/chips-media/wave5/wave5-vpuapi.h enum set_param_option { · line 166

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +1 -1
@@ -163,7 +163,7 @@ enum set_param_option {
 
  #define BUFFER_MARGIN				4096
 
-#define MAX_FIRMWARE_CALL_RETRY			10
+#define MAX_FIRMWARE_CALL_RETRY			30

I thought 10s was already very long, am I right this means it can get stuck for 30s now ?

drivers/media/platform/chips-media/wave5/wave5-vpuapi.h struct vpu_instance { · line 816

Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +2 -0
  #define VDI_LITTLE_ENDIAN	0x0
 
@@ -812,6 +812,9 @@ struct vpu_instance {
  	bool cbcr_interleave;
  	bool nv21;
  	bool eos;
+	bool retry;
+	int queuing_num;

Some comment would be helpful. What does "retry" stand for, and how is queing_num different from v4l2_m2m_num_src_bufs_ready() ?

Quoted 8 lines

drivers/media/platform/chips-media/wave5/wave5-vpuapi.h struct vpu_instance { · line 820

Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +1 -0
+	struct mutex feed_lock; /* lock for feeding bitstream buffers */
  	struct vpu_buf bitstream_vbuf;
  	dma_addr_t last_rd_ptr;
  	size_t remaining_consumed_bytes;

--
Nicolas Dufresne
Principal Engineer at Collabora

Thanks
Jackson

#4 lore raw

Re: [RESEND PATCH v0 1/3] media: chips-media: wave5: Improve performance of decoder

Hi,

Le vendredi 04 avril 2025 à 04:16 +0000, jackson.lee a écrit :

Quoted 93 lines

Hi Nicolas

Thanks for your review.
Can you check my comment for your review?

Quoted 79 lines

-----Original Message-----
From: Nicolas Dufresne <[email protected]>
Sent: Thursday, April 3, 2025 3:03 AM
To: jackson.lee <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; lafley.kim
<[email protected]>; [email protected]; [email protected]; Nas
Chung <[email protected]>
Subject: Re: [RESEND PATCH v0 1/3] media: chips-media: wave5:
Improve
performance of decoder

Le mercredi 19 mars 2025 à 12:50 +0900, Jackson.lee a écrit :

Quoted 4 lines

From: Jackson Lee <[email protected]>

The existing way for decoding frames was to wait until each frame was

Suggestion:
  The current decoding method was to...

Quoted 7 lines

decoded after feeding a bitstream. As a result, performance was low
and Wave5 could not achieve max pixel processing rate.

Update driver to use an asynchronous approach for decoding and feeding
a

  Update the driver ..

Quoted 1 line

bitstream in order to achieve full capabilities of the device.

This second part lacks the technical design choices needed for your reviewer ;-P I did read the patch once, and am under the impression that
your approach is to not finish the job anymore.

Quoted 18 lines

---
 .../chips-media/wave5/wave5-vpu-dec.c         | 344
++++++++++--------
 .../chips-media/wave5/wave5-vpu-enc.c         |   3 - .../platform/chips-media/wave5/wave5-vpuapi.c |  25 +- .../platform/chips-media/wave5/wave5-vpuapi.h |   5 +-

 4 files changed, 211 insertions(+), 166 deletions(-)
(quoted diff) +0 -0

dec.c b/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c index fd71f0c43ac3..cc47da509703 100644

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct · line 230

Quoted 1 line
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -0
@@ -230,12 +230,147 @@ static int start_decode(struct
 vpu_instance *inst,

u32 *fail_res)

Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -0
  		switch_state(inst, VPU_INST_STATE_STOP);
 

  dev_dbg(inst->dev->dev, "%s: pic run failed / finish job",

__func__);

Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-		v4l2_m2m_job_finish(inst->v4l2_m2m_dev,

m2m_ctx);
  }

  return ret;
 }

+static int write_to_ringbuffer(struct vpu_instance *inst, void *buffer,

size_t buffer_size,

Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+			       struct vpu_buf *ring_buffer,

dma_addr_t wr_ptr)

{

Quoted 6 lines

It seems that from now on this needs to be called with the feed_lock, so
add:

	 lockdep_assert_held(&inst->feed_lock);
Quoted 6 lines

The "feed_lock" mutex is for protecting "inst->avail_src_bufs" this list, so the feed_lock should be used when "static int fill_ringbuffer(struct vpu_instance *inst)" is called, not the write_to_ringbuffer.
So I think the feed_lock should not be added in the write_to_ringbuffer function.

This is called from "fill_ringbuffer()" and both calls to that seems to take that lock, my comment was just to make it explicit with lockdep. This is called from CMD_STOP and from device_run() which can be concurrent. I think the code effectively protecting the ring buffer state with.

+ mutex_lock(&inst->feed_lock);
+ ret = fill_ringbuffer(inst);
+ mutex_unlock(&inst->feed_lock);

note that there is bunch of random locking too in the abstraction, this driver will need some cleanup in the long run imho.

Quoted 114 lines
Quoted 28 lines
Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5 -0
+	size_t size;
+	size_t offset = wr_ptr - ring_buffer->daddr;
+	int ret;
+
+	if (wr_ptr + buffer_size > ring_buffer->daddr +

ring_buffer->size)

{

Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+		size = ring_buffer->daddr + ring_buffer->size -

wr_ptr;

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+		ret = wave5_vdi_write_memory(inst->dev,

ring_buffer, offset,

(u8 *)buffer, size);

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +4 -0
+		if (ret < 0)
+			return ret;
+
+		ret = wave5_vdi_write_memory(inst->dev,

ring_buffer, 0, (u8

*)buffer + size,

Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+					     buffer_size -

size);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +4 -0
+		if (ret < 0)
+			return ret;
+	} else {
+		ret = wave5_vdi_write_memory(inst->dev,

ring_buffer, offset,

(u8 *)buffer,

Quoted 13 lines

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 263

Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7 -0
+					     buffer_size);
+		if (ret < 0)
+			return ret;
+	}
+
+	return 0;
+}

I have a pretty strong preference in seeing code being moved around in
seperate patch documented as "non functional changes". This way I know I
don't have to review again that code.

I will do it.

Quoted 61 lines

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c

Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -0
+
+static int fill_ringbuffer(struct vpu_instance *inst) {

Same, I don't want to have to do back and fourth to check if you changed
anything. Also, after the refactoring, the lockdeup assert can be added in
the new patch, which makes the introduction of the new lock very obvious.

Quoted 11 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +10 -0
+	struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
+	struct vpu_src_buffer *vpu_buf;
+	int ret = 0;
+
+	if (m2m_ctx->last_src_buf)  {
+		struct vpu_src_buffer *vpu_buf =
+wave5_to_vpu_src_buf(m2m_ctx->last_src_buf);
+
+		if (vpu_buf->consumed) {
+			dev_dbg(inst->dev->dev, "last src buffer

already

written\n");

Quoted 20 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5 -0
+			return 0;
+		}
+	}
+
+	list_for_each_entry(vpu_buf, &inst->avail_src_bufs,

list) {

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+		struct vb2_v4l2_buffer *vbuf = &vpu_buf-
Quoted 1 line

v4l2_m2m_buf.vb;

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+		struct vpu_buf *ring_buffer = &inst-
Quoted 1 line

bitstream_vbuf;

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+		size_t src_size = vb2_get_plane_payload(&vbuf-
Quoted 1 line

vb2_buf, 0);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+		void *src_buf = vb2_plane_vaddr(&vbuf->vb2_buf,

0);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +6 -0
+		dma_addr_t rd_ptr = 0;
+		dma_addr_t wr_ptr = 0;
+		size_t remain_size = 0;
+
+		if (vpu_buf->consumed) {
+			dev_dbg(inst->dev->dev, "already copied
Quoted 1 line

src buf (%u)

to the ring buffer\n",

Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7 -0
+				vbuf->vb2_buf.index);
+			continue;
+		}
+
+		if (!src_buf) {
+			dev_dbg(inst->dev->dev,
+				"%s: Acquiring kernel pointer to

src buf (%u),

fail\n",

Quoted 6 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5 -0
+				__func__, vbuf->vb2_buf.index);
+			break;
+		}
+
+		ret = wave5_vpu_dec_get_bitstream_buffer(inst,

&rd_ptr,

&wr_ptr, &remain_size);

Quoted 7 lines

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 304

Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -0
+		if (ret) {
+			/* Unable to acquire the mutex */

Might want to fix that comment in a separate patch, perhaps even remove
that ?

Quoted 1 line

Do you want to remove the comment in a separate patch ?

Yes please.

Quoted 198 lines
Quoted 188 lines
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+			dev_err(inst->dev->dev, "Getting the

bitstream buffer,

fail: %d\n",

Quoted 6 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5 -0
+				ret);
+			return ret;
+		}
+
+		dev_dbg(inst->dev->dev, "%s: rd_ptr %pad wr_ptr

%pad",

__func__,

Quoted 6 lines

+&rd_ptr, &wr_ptr);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +4 -0
+
+		if (remain_size < src_size) {
+			dev_dbg(inst->dev->dev,
+				"%s: remaining size: %zu <

source size: %zu for

src buf (%u)\n",

Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+				__func__, remain_size, src_size,

vbuf-
vb2_buf.index);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +4 -0
+			break;
+		}
+
+		ret = write_to_ringbuffer(inst, src_buf,

src_size,

ring_buffer, wr_ptr);

Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -0
+		if (ret) {
+			dev_err(inst->dev->dev, "Write src buf

(%u) to ring

buffer, fail: %d\n",

Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5 -0
+				vbuf->vb2_buf.index, ret);
+			return ret;
+		}
+
+		ret =

wave5_vpu_dec_update_bitstream_buffer(inst, src_size);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3 -0
+		if (ret) {
+			dev_dbg(inst->dev->dev,
+				"update_bitstream_buffer fail:

%d for src buf

(%u)\n",

Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7 -0
+				ret, vbuf->vb2_buf.index);
+			break;
+		}
+
+		vpu_buf->consumed = true;
+
+		/* Don't write buffers passed the last one while

draining.

*/

Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+		if (v4l2_m2m_is_last_draining_src_buf(m2m_ctx,

vbuf)) {

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+			dev_dbg(inst->dev->dev, "last src buffer

written to

the ring buffer\n");

Quoted 23 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +12 -0
+			break;
+		}
+
+		inst->queuing_num++;
+		list_del_init(&vpu_buf->list);
+		break;
+	}
+
+	return ret;
+}
+
+static void wave5_vpu_dec_feed_remaining(struct vpu_instance

*inst) {

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +9 -0
+	int ret = 0;
+	struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
+	u32 fail_res = 0;
+
+	mutex_lock(&inst->feed_lock);
+	ret = fill_ringbuffer(inst);
+	mutex_unlock(&inst->feed_lock);
+	if (ret) {
+		dev_warn(inst->dev->dev, "Filling ring buffer

failed\n");

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c

Quoted 1 line
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+		return;

I don't think the error handling is correct. What will happen with the
job ?

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 364

Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3 -0
+	}
+
+	ret = start_decode(inst, &fail_res);

I'm a little worried seeing this called outside of device_run() context.

Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3 -0
+	if (ret) {
+		dev_err(inst->dev->dev,
+			"Frame decoding on m2m context (%p),

fail: %d

(result: %d)\n",

Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +6 -0
+			m2m_ctx, ret, fail_res);
+	}
+
+	v4l2_m2m_job_finish(inst->v4l2_m2m_dev, m2m_ctx);
 
 
 
+}
+
Quoted 6 lines

 static void flag_last_buffer_done(struct vpu_instance *inst) {
  struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx; @@ -347,7
+482,6 @@ static void wave5_vpu_dec_finish_decode(struct vpu_instance

*inst)

Quoted 11 lines

  struct vb2_v4l2_buffer *dec_buf = NULL;
  struct vb2_v4l2_buffer *disp_buf = NULL; struct vb2_queue *dst_vq = v4l2_m2m_get_dst_vq(m2m_ctx);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-	struct queue_status_info q_status;

  dev_dbg(inst->dev->dev, "%s: Fetch output info from firmware.",
__func__);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -0
@@ -441,20 +575,6 @@ static void

wave5_vpu_dec_finish_decode(struct

vpu_instance *inst)

Quoted 8 lines

  } spin_unlock_irqrestore(&inst->state_spinlock, flags);
  }

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -3
-
-	/*
-	 * During a resolution change and while draining, the

firmware may

flush

Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-	 * the reorder queue regardless of having a matching

decoding

operation

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c

Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-	 * pending. Only terminate the job if there are no more

IRQ coming.

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -2
-	 */
-	wave5_vpu_dec_give_command(inst, DEC_GET_QUEUE_STATUS,

&q_status);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -2
-	if (q_status.report_queue_count == 0 &&
-	    (q_status.instance_queue_count == 0 ||

dec_info.sequence_changed)) {

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c

Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-		dev_dbg(inst->dev->dev, "%s: finishing job.\n",

__func__);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -3
-		pm_runtime_mark_last_busy(inst->dev->dev);
-		pm_runtime_put_autosuspend(inst->dev->dev);
-		v4l2_m2m_job_finish(inst->v4l2_m2m_dev,

m2m_ctx);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-	}

This removal is why I believe some seek stress testing are needed. I'll
try and find some tests file I crafted last year.

Quoted 5 lines

 }

 static int wave5_vpu_dec_querycap(struct file *file, void *fh, struct
v4l2_capability *cap) @@ -794,11 +914,21 @@ static int

wave5_vpu_dec_stop(struct vpu_instance *inst)

Quoted 20 lines

  }

  if (inst->state != VPU_INST_STATE_NONE) {

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3 -0
+		struct vb2_v4l2_buffer *vbuf;
+		struct vpu_src_buffer *vpu_buf;
+

  /*
  * Temporarily release the state_spinlock so that subsequent
  * calls do not block on a mutex while inside this spinlock.
  */ spin_unlock_irqrestore(&inst->state_spinlock, flags);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5 -0
+		vbuf = v4l2_m2m_last_src_buf(m2m_ctx);
+		if (vbuf) {
+			vpu_buf = wave5_to_vpu_src_buf(vbuf);
+			if (!vpu_buf->consumed)
+				wave5_vpu_dec_feed_remaining(ins

t);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c

Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -0
+		}
+
Quoted 9 lines

Do you really need to add all these checks ? Shouldn't you just "feed_remaining()" and move on ? I also don't think its a good idea to
call v4l2_m2m_job_finish() in code that can run concurrently with the
device_run() function. This should only happen if you are certain the VPU
instance is stopped already.

Quoted 5 lines

While testing fluster,  stop command is sometimes called  before the last bitstream queued  is handled.
The stop command sets EOS before handling the last bitstream, so sometimes mismatch happens.
I think the code should be add in the wave5_vpu_dec_stop.

This would be a regression, we took good care of handling before and after case correctly.

Quoted 251 lines
Quoted 244 lines
Quoted 8 lines

  ret = wave5_vpu_dec_set_eos_on_firmware(inst); if (ret)
  return ret;

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -0
@@ -1116,115 +1246,6 @@ static int wave5_prepare_fb(struct

vpu_instance

*inst)

Quoted 5 lines

  return 0;
 }

-static int write_to_ringbuffer(struct vpu_instance *inst, void *buffer,

size_t buffer_size,

Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-			       struct vpu_buf *ring_buffer,

dma_addr_t wr_ptr)

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -6
-{
-	size_t size;
-	size_t offset = wr_ptr - ring_buffer->daddr;
-	int ret;
-
-	if (wr_ptr + buffer_size > ring_buffer->daddr +

ring_buffer->size)

{

Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-		size = ring_buffer->daddr + ring_buffer->size -

wr_ptr;

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-		ret = wave5_vdi_write_memory(inst->dev,

ring_buffer, offset,

(u8 *)buffer, size);

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -4
-		if (ret < 0)
-			return ret;
-
-		ret = wave5_vdi_write_memory(inst->dev,

ring_buffer, 0, (u8

*)buffer + size,

Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-					     buffer_size -

size);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -4
-		if (ret < 0)
-			return ret;
-	} else {
-		ret = wave5_vdi_write_memory(inst->dev,

ring_buffer, offset,

(u8 *)buffer,

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -5
-					     buffer_size);
-		if (ret < 0)
-			return ret;
-	}
-

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)

Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -10
-	return 0;
-}
-
-static int fill_ringbuffer(struct vpu_instance *inst) -{
-	struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
-	struct v4l2_m2m_buffer *buf, *n;
-	int ret;
-
-	if (m2m_ctx->last_src_buf)  {
-		struct vpu_src_buffer *vpu_buf =

wave5_to_vpu_src_buf(m2m_ctx->last_src_buf);

Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -3
-
-		if (vpu_buf->consumed) {
-			dev_dbg(inst->dev->dev, "last src buffer

already

written\n");

Quoted 20 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -7
-			return 0;
-		}
-	}
-
-	v4l2_m2m_for_each_src_buf_safe(m2m_ctx, buf, n) {
-		struct vb2_v4l2_buffer *vbuf = &buf->vb;
-		struct vpu_src_buffer *vpu_buf =

wave5_to_vpu_src_buf(vbuf);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-		struct vpu_buf *ring_buffer = &inst-
Quoted 1 line

bitstream_vbuf;

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-		size_t src_size = vb2_get_plane_payload(&vbuf-
Quoted 1 line

vb2_buf, 0);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-		void *src_buf = vb2_plane_vaddr(&vbuf->vb2_buf,

0);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -6
-		dma_addr_t rd_ptr = 0;
-		dma_addr_t wr_ptr = 0;
-		size_t remain_size = 0;
-
-		if (vpu_buf->consumed) {
-			dev_dbg(inst->dev->dev, "already copied
Quoted 1 line

src buf (%u)

to the ring buffer\n",

Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -7
-				vbuf->vb2_buf.index);
-			continue;
-		}
-
-		if (!src_buf) {
-			dev_dbg(inst->dev->dev,
-				"%s: Acquiring kernel pointer to

src buf (%u),

fail\n",

Quoted 6 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -5
-				__func__, vbuf->vb2_buf.index);
-			break;
-		}
-
-		ret = wave5_vpu_dec_get_bitstream_buffer(inst,

&rd_ptr,

&wr_ptr, &remain_size);

Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -3
-		if (ret) {
-			/* Unable to acquire the mutex */
-			dev_err(inst->dev->dev, "Getting the

bitstream buffer,

fail: %d\n",

Quoted 6 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -5
-				ret);
-			return ret;
-		}
-
-		dev_dbg(inst->dev->dev, "%s: rd_ptr %pad wr_ptr

%pad",

__func__, &rd_ptr, &wr_ptr);

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -4
-
-		if (remain_size < src_size) {
-			dev_dbg(inst->dev->dev,
-				"%s: remaining size: %zu <

source size: %zu for

src buf (%u)\n",

Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-				__func__, remain_size, src_size,

vbuf-
vb2_buf.index);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -4
-			break;
-		}
-
-		ret = write_to_ringbuffer(inst, src_buf,

src_size,

ring_buffer, wr_ptr);

Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -2
-		if (ret) {
-			dev_err(inst->dev->dev, "Write src buf

(%u) to ring

buffer, fail: %d\n",

Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -5
-				vbuf->vb2_buf.index, ret);
-			return ret;
-		}
-
-		ret =

wave5_vpu_dec_update_bitstream_buffer(inst, src_size);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -3
-		if (ret) {
-			dev_dbg(inst->dev->dev,
-				"update_bitstream_buffer fail:

%d for src buf

(%u)\n",

Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -7
-				ret, vbuf->vb2_buf.index);
-			break;
-		}
-
-		vpu_buf->consumed = true;
-
-		/* Don't write buffers passed the last one while

draining.

*/

Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-		if (v4l2_m2m_is_last_draining_src_buf(m2m_ctx,

vbuf)) {

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-			dev_dbg(inst->dev->dev, "last src buffer

written to

the ring buffer\n");

Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -7
-			break;
-		}
-	}
-
-	return 0;
-}
-

 static void wave5_vpu_dec_buf_queue_src(struct vb2_buffer *vb) {
  struct vpu_instance *inst = vb2_get_drv_priv(vb-

Quoted 3 lines
Quoted 1 line

vb2_queue); @@

-1236,6 +1257,11 @@ static void
wave5_vpu_dec_buf_queue_src(struct

vb2_buffer *vb)

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_buf_queue_src(struct vb2_buffer *vb) · line 1264

Quoted 8 lines

  vbuf->sequence = inst->queued_src_buf_num++;

  v4l2_m2m_buf_queue(m2m_ctx, vbuf);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5 -0
+
+	INIT_LIST_HEAD(&vpu_buf->list);
+	mutex_lock(&inst->feed_lock);
+	list_add_tail(&vpu_buf->list, &inst->avail_src_bufs);
+	mutex_unlock(&inst->feed_lock);

This lack documentation, you didn't even mention the introduction of a new
src buf list in the commit message.

Quoted 6 lines

 }

 static void wave5_vpu_dec_buf_queue_dst(struct vb2_buffer *vb) @@
-1362,7 +1388,6 @@ static int wave5_vpu_dec_start_streaming(struct

vb2_queue *q, unsigned int count

Quoted 21 lines

  goto return_buffers; }
  }
-
  } pm_runtime_mark_last_busy(inst->dev->dev); pm_runtime_put_autosuspend(inst->dev->dev);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7 -0
@@ -1385,6 +1410,13 @@ static int streamoff_output(struct
 vb2_queue *q)
  	dma_addr_t new_rd_ptr;
  	struct dec_output_info dec_info;
  	unsigned int i;
+	struct vpu_src_buffer *vpu_buf, *tmp;
+
+	inst->retry = false;
+	inst->queuing_num = 0;
+
+	list_for_each_entry_safe(vpu_buf, tmp, &inst-
 >avail_src_bufs, list)
+		list_del_init(&vpu_buf->list);
 
Quoted 4 lines

  for (i = 0; i < v4l2_m2m_num_dst_bufs_ready(m2m_ctx); i++) {
  ret = wave5_vpu_dec_set_disp_flag(inst, i); @@ - 1481,10

+1513,8 @@

Quoted 4 lines

static void wave5_vpu_dec_stop_streaming(struct vb2_queue *q)

  wave5_vpu_dec_give_command(inst,
DEC_GET_QUEUE_STATUS,

&q_status);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c

Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -4
-		if (q_status.report_queue_count == 0)
-			break;
-
-		if (wave5_vpu_wait_interrupt(inst,

VPU_DEC_TIMEOUT) < 0)

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -0
+		if ((inst->state == VPU_INST_STATE_STOP ||
+q_status.instance_queue_count == 0) &&

State should be read with appropriate locking.

Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+		    q_status.report_queue_count == 0)

  break;

This loop have nothing to wait on anymore. It just actively pool on the HW.
I noticed your third patch, but if this is what you address there, squash
it somehow, this would break bisects.

Quoted 5 lines

The goal of that code was to ensure that this code will never run concurrently to finish_decode() once out of that while loop. I don't think
you break that, but that will cause possibly high CPU load.

Quoted 5 lines

The code is not related with the third patch, there was sometimes HW interrupt misses since changing to asynchronous, so an infinite wait happened when closed
So I removed it, but I think there is no high CPU load because it runs very short time.

If the firmware get stuck, Linux can spin, this cannot be done that way I'm sorry. That its most of the time spinning only few times does not seems like valid justification. We need a limit, we need some preemption, etc.

Quoted 50 lines
Quoted 37 lines
Quoted 4 lines

  if (wave5_vpu_dec_get_output_info(inst, &dec_output_info))

@@

Quoted 9 lines

-1577,13 +1607,23 @@ static void wave5_vpu_dec_device_run(void *priv)
  struct queue_status_info q_status;
  u32 fail_res = 0;
  int ret = 0;

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+	unsigned long flags;

  dev_dbg(inst->dev->dev, "%s: Fill the ring buffer with new

bitstream data", __func__);

Quoted 12 lines

  pm_runtime_resume_and_get(inst->dev->dev);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -3
-	ret = fill_ringbuffer(inst);
-	if (ret) {
-		dev_warn(inst->dev->dev, "Filling ring buffer

failed\n");

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +6 -1
-		goto finish_job_and_return;
+	if (!inst->retry) {
+		mutex_lock(&inst->feed_lock);
+		ret = fill_ringbuffer(inst);
+		mutex_unlock(&inst->feed_lock);
+		if (ret < 0) {
+			dev_warn(inst->dev->dev, "Filling ring
Quoted 1 line

buffer

failed\n");

Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +4 -0
+			goto finish_job_and_return;
+		} else if (!inst->eos &&
+			   inst->queuing_num == 0 &&
+			   inst->state ==

VPU_INST_STATE_PIC_RUN) {

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+			dev_dbg(inst->dev->dev, "%s: no

bitstream for feeding,

so skip ", __func__);

Quoted 11 lines

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c

Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -0
+			goto finish_job_and_return;
+		}

There is quite some overlap between that local src buffer queue and all
the code that depends on v4l2_m2m_num_src_bufs_ready(). Are you sure you
aren't re-implementing something ? Also, this code belong to the PIC_RUN
case in the below switch.

Quoted 5 lines

After src buffer was queued, v4l2_m2m_num_src_bufs_ready() always returns true until src buffer was removed in the finish_decode since get_result.
So checking if there is buffer or not in the src bufs list was needed in the device_run.

Ack.

Quoted 308 lines
Quoted 296 lines
Quoted 15 lines

  }

  switch (inst->state) {

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -0
@@ -1619,7 +1659,9 @@ static void wave5_vpu_dec_device_run(void
 *priv)
  		 * we had a chance to switch, which leads to an

invalid state
  * change.
  */

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+		spin_lock_irqsave(&inst->state_spinlock, flags);

  switch_state(inst, VPU_INST_STATE_PIC_RUN);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+		spin_unlock_irqrestore(&inst->state_spinlock,
Quoted 1 line

flags);

Should be its own patch, and fixed everywhere. Add missing lockdep assert
(if there is a spinlock version of it).

Quoted 4 lines

  /*
  * During DRC, the picture decoding remains pending, so just

leave

Quoted 1 line

the job @@ -1634,12 +1676,14 @@ static void

wave5_vpu_dec_device_run(void *priv)

Quoted 4 lines

  ret = wave5_prepare_fb(inst);
  if (ret) {
  dev_warn(inst->dev->dev, "Framebuffer preparation,

fail: %d\n",

Quoted 6 lines

ret);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+			spin_lock_irqsave(&inst->state_spinlock,

flags);
  switch_state(inst, VPU_INST_STATE_STOP);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+			spin_unlock_irqrestore(&inst-
Quoted 1 line

state_spinlock, flags);

Same, this is separate fix.

Quoted 6 lines

  break;
  }

  if (q_status.instance_queue_count) {

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-			dev_dbg(inst->dev->dev, "%s: leave with

active job",

__func__);

Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+			v4l2_m2m_job_finish(inst->v4l2_m2m_dev,

m2m_ctx);
  return;
  }

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -0
@@ -1650,14 +1694,22 @@ static void wave5_vpu_dec_device_run(void
 *priv)
  			dev_err(inst->dev->dev,

  "Frame decoding on m2m context (%p), fail: %d

(result: %d)\n",

Quoted 14 lines

  m2m_ctx, ret, fail_res);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +9 -1
-			break;
+			goto finish_job_and_return;
+		}
+
+		if (fail_res == WAVE5_SYSERR_QUEUEING_FAIL) {
+			inst->retry = true;
+		} else {
+			inst->retry = false;
+			if (!inst->eos)
+				inst->queuing_num--;

  }

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -2
-		/* Return so that we leave this job active */
-		dev_dbg(inst->dev->dev, "%s: leave with active
Quoted 1 line

job",

__func__);

Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -3
-		return;
-	default:
-		WARN(1, "Execution of a job in state %s

illegal.\n",

state_to_str(inst->state));

Quoted 71 lines

  break;

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3 -0
+	default:
+		if (!v4l2_m2m_has_stopped(m2m_ctx))
+			WARN(1, "Execution of a job in state %s

illegal.\n",

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -0
+			     state_to_str(inst->state));
+		return;

  }

 finish_job_and_return:

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -4
@@ -1676,10 +1728,7 @@ static void wave5_vpu_dec_job_abort(void
 *priv)
  	if (ret)
  		return;
 
-	ret = wave5_vpu_dec_set_eos_on_firmware(inst);
-	if (ret)
-		dev_warn(inst->dev->dev,
-			 "Setting EOS for the bitstream, fail:
 %d\n", ret);
+	v4l2_m2m_job_finish(inst->v4l2_m2m_dev, inst-
 >v4l2_fh.m2m_ctx);

 }

 static int wave5_vpu_dec_job_ready(void *priv) @@ -1755,6 +1804,8 @@
static int wave5_vpu_open_dec(struct file *filp) inst->ops = &wave5_vpu_dec_inst_ops;

  spin_lock_init(&inst->state_spinlock);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -0
+	mutex_init(&inst->feed_lock);
+	INIT_LIST_HEAD(&inst->avail_src_bufs);

  inst->codec_info = kzalloc(sizeof(*inst->codec_info), GFP_KERNEL);
  if (!inst->codec_info)

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -3
@@ -1830,9 +1881,6 @@ static int wave5_vpu_open_dec(struct file
 *filp)
  	if (ret)
  		goto cleanup_inst;
 
-	if (list_empty(&dev->instances))
-		pm_runtime_use_autosuspend(inst->dev->dev);
-
  	list_add_tail(&inst->list, &dev->instances);
 

  mutex_unlock(&dev->dev_lock);

(quoted diff) +0 -0

enc.c b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c index 1e5fc5f8b856..cf20f774ed1b 100644

drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c +0 -3
@@ -1774,9 +1774,6 @@ static int wave5_vpu_open_enc(struct file
 *filp)
  	if (ret)
  		goto cleanup_inst;
 
-	if (list_empty(&dev->instances))
-		pm_runtime_use_autosuspend(inst->dev->dev);
-
  	list_add_tail(&inst->list, &dev->instances);
 

  mutex_unlock(&dev->dev_lock);

(quoted diff) +0 -0

vpuapi.c b/drivers/media/platform/chips-media/wave5/wave5-vpuapi.c index e5e879a13e8b..c1737fac6edd 100644

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance · line 207

Quoted 1 line
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -0
@@ -207,8 +207,6 @@ int wave5_vpu_dec_close(struct vpu_instance
 *inst,

u32 *fail_res)

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance · line 231

Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -2
  	int retry = 0;
  	struct vpu_device *vpu_dev = inst->dev;
  	int i;
-	int inst_count = 0;
-	struct vpu_instance *inst_elm;
 
  	*fail_res = 0;

  if (!inst->codec_info)

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -0
@@ -233,6 +231,16 @@ int wave5_vpu_dec_close(struct vpu_instance
 *inst,

u32 *fail_res)

Quoted 15 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +6 -0
  		    retry++ >= MAX_FIRMWARE_CALL_RETRY) {
  			ret = -ETIMEDOUT;
  			goto unlock_and_return;
+		} else if (*fail_res ==
 WAVE5_SYSERR_VPU_STILL_RUNNING) {
+			struct dec_output_info dec_info;
+
+			mutex_unlock(&vpu_dev->hw_lock);
+			wave5_vpu_dec_get_output_info(inst,
 &dec_info);
+			ret = mutex_lock_interruptible(&vpu_dev-
Quoted 1 line

hw_lock);

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +2 -0
+			if (ret) {
+				pm_runtime_put_sync(inst->dev-
Quoted 1 line

dev);

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c

Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +2 -0
+				return ret;
+			}

Please try and simplify all the branches. Something like:

			if (ret == 0)
				break;

			if (*fail_res !=
WAVE5_SYSERR_VPU_STILL_RUNNING) {
					dev_warn(inst->dev->dev,
"dec_finish_seq
timed out\n");
					goto unlock_and_return;
			}

			if (retry++ >= MAX_FIRMWARE_CALL_RETRY) {
					ret = -ETIMEDOUT;
					goto unlock_and_return;
			}

			mutex_unlock(&vpu_dev->hw_lock);
			wave5_vpu_dec_get_output_info(inst,

&dec_info);
ret = mutex_lock_interruptible(&vpu_dev-

Quoted 1 line

hw_lock);

			if (ret) {
				pm_runtime_put_sync(inst->dev-
Quoted 1 line

dev);

				return ret;
			}

My impression on that change is that it is independent and could be introduced separately.

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance · line 257

Quoted 5 lines

  }
  } while (ret != 0);

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -0
@@ -249,11 +257,7 @@ int wave5_vpu_dec_close(struct vpu_instance
 *inst,

u32 *fail_res)

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res) · line 256

Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -0
  	}
 

  wave5_vdi_free_dma_memory(vpu_dev, &p_dec_info-

Quoted 1 line

vb_task);

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -5
-
-	list_for_each_entry(inst_elm, &vpu_dev->instances, list)
-		inst_count++;
-	if (inst_count == 1)
-		pm_runtime_dont_use_autosuspend(vpu_dev->dev);

This seems like some other fix, that should be in its own commit.

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_enc_close(struct vpu_instance · line 724

Quoted 6 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +1 -0
+	mutex_destroy(&inst->feed_lock);

 unlock_and_return:
  mutex_unlock(&vpu_dev->hw_lock);

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -0
@@ -720,8 +724,6 @@ int wave5_vpu_enc_close(struct vpu_instance
 *inst,

u32 *fail_res)

Quoted 14 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -2
  	int ret;
  	int retry = 0;
  	struct vpu_device *vpu_dev = inst->dev;
-	int inst_count = 0;
-	struct vpu_instance *inst_elm;
 
  	*fail_res = 0;

  if (!inst->codec_info)

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -0
@@ -765,11 +767,6 @@ int wave5_vpu_enc_close(struct vpu_instance
 *inst, u32 *fail_res)
 

  wave5_vdi_free_dma_memory(vpu_dev, &p_enc_info-

Quoted 1 line

vb_task);

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_enc_close(struct vpu_instance *inst, u32 *fail_res) · line 772

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -5
-	list_for_each_entry(inst_elm, &vpu_dev->instances, list)
-		inst_count++;
-	if (inst_count == 1)
-		pm_runtime_dont_use_autosuspend(vpu_dev->dev);
-

Same.

Quoted 10 lines

  mutex_unlock(&vpu_dev->hw_lock);
  pm_runtime_put_sync(inst->dev->dev);

(quoted diff) +0 -0

vpuapi.h b/drivers/media/platform/chips-media/wave5/wave5-vpuapi.h index 45615c15beca..6ca1ddc67c64 100644

drivers/media/platform/chips-media/wave5/wave5-vpuapi.h enum set_param_option { · line 166

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +1 -1
@@ -163,7 +163,7 @@ enum set_param_option {
 
  #define BUFFER_MARGIN				4096
 
-#define MAX_FIRMWARE_CALL_RETRY			10
+#define MAX_FIRMWARE_CALL_RETRY			30

I thought 10s was already very long, am I right this means it can get
stuck for 30s now ?

drivers/media/platform/chips-media/wave5/wave5-vpuapi.h struct vpu_instance { · line 816

Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +2 -0
  #define VDI_LITTLE_ENDIAN	0x0
 
@@ -812,6 +812,9 @@ struct vpu_instance {
  	bool cbcr_interleave;
  	bool nv21;
  	bool eos;
+	bool retry;
+	int queuing_num;

Some comment would be helpful. What does "retry" stand for, and how is
queing_num different from v4l2_m2m_num_src_bufs_ready() ?

Quoted 9 lines
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +1 -0
+	struct mutex feed_lock; /* lock for feeding bitstream

buffers */
  struct vpu_buf bitstream_vbuf;
  dma_addr_t last_rd_ptr;
  size_t remaining_consumed_bytes;

--
Nicolas Dufresne
Principal Engineer at Collabora

Thanks
Jackson

Signature · 2 lines
Nicolas Dufresne
Principal Engineer at Collabora
author maintainer
#5 lore raw

RE: [RESEND PATCH v0 1/3] media: chips-media: wave5: Improve performance of decoder

Hi Nicolas

Quoted 441 lines

-----Original Message-----
From: Nicolas Dufresne <[email protected]>
Sent: Saturday, April 5, 2025 1:00 AM
To: jackson.lee <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; lafley.kim <[email protected]>; [email protected]; [email protected]; Nas Chung <[email protected]>
Subject: Re: [RESEND PATCH v0 1/3] media: chips-media: wave5: Improve performance of decoder

Hi,

Le vendredi 04 avril 2025 à 04:16 +0000, jackson.lee a écrit :

Quoted 89 lines

Hi Nicolas

Thanks for your review.
Can you check my comment for your review?

Quoted 75 lines

-----Original Message-----
From: Nicolas Dufresne <[email protected]>
Sent: Thursday, April 3, 2025 3:03 AM
To: jackson.lee <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; lafley.kim <[email protected]>; [email protected]; [email protected]; Nas Chung <[email protected]>
Subject: Re: [RESEND PATCH v0 1/3] media: chips-media: wave5:
Improve
performance of decoder

Le mercredi 19 mars 2025 à 12:50 +0900, Jackson.lee a écrit :

Quoted 4 lines

From: Jackson Lee <[email protected]>

The existing way for decoding frames was to wait until each frame was

Suggestion:
  The current decoding method was to...

Quoted 5 lines

decoded after feeding a bitstream. As a result, performance was low and Wave5 could not achieve max pixel processing rate.

Update driver to use an asynchronous approach for decoding and feeding a

  Update the driver ..

Quoted 1 line

bitstream in order to achieve full capabilities of the device.

This second part lacks the technical design choices needed for your reviewer ;-P I did read the patch once, and am under the impression that
your approach is to not finish the job anymore.

Quoted 18 lines

---
 .../chips-media/wave5/wave5-vpu-dec.c         | 344
++++++++++--------
 .../chips-media/wave5/wave5-vpu-enc.c         |   3 - .../platform/chips-media/wave5/wave5-vpuapi.c |  25 +- .../platform/chips-media/wave5/wave5-vpuapi.h |   5 +-

 4 files changed, 211 insertions(+), 166 deletions(-)
(quoted diff) +0 -0

dec.c b/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c index fd71f0c43ac3..cc47da509703 100644

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct · line 230

Quoted 1 line
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -0
@@ -230,12 +230,147 @@ static int start_decode(struct
 vpu_instance *inst,

u32 *fail_res)

Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -0
  		switch_state(inst, VPU_INST_STATE_STOP);
 

  dev_dbg(inst->dev->dev, "%s: pic run failed / finish job",

__func__);

Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-		v4l2_m2m_job_finish(inst->v4l2_m2m_dev,

m2m_ctx);
  }

  return ret;
 }

+static int write_to_ringbuffer(struct vpu_instance *inst, void *buffer,

size_t buffer_size,

Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+			       struct vpu_buf *ring_buffer,

dma_addr_t wr_ptr)

{

Quoted 6 lines

It seems that from now on this needs to be called with the feed_lock, so
add:

	 lockdep_assert_held(&inst->feed_lock);
Quoted 6 lines

The "feed_lock" mutex is for protecting "inst->avail_src_bufs" this list, so the feed_lock should be used when "static int fill_ringbuffer(struct vpu_instance *inst)" is called, not the write_to_ringbuffer.
So I think the feed_lock should not be added in the write_to_ringbuffer function.

This is called from "fill_ringbuffer()" and both calls to that seems to take that lock, my comment was just to make it explicit with lockdep. This is called from CMD_STOP and from device_run() which can be concurrent. I think the code effectively protecting the ring buffer state with.

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3 -0
+	mutex_lock(&inst->feed_lock);
+	ret = fill_ringbuffer(inst);
+	mutex_unlock(&inst->feed_lock);

note that there is bunch of random locking too in the abstraction, this driver will need some cleanup in the long run imho.

Quoted 114 lines
Quoted 28 lines
Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5 -0
+	size_t size;
+	size_t offset = wr_ptr - ring_buffer->daddr;
+	int ret;
+
+	if (wr_ptr + buffer_size > ring_buffer->daddr +

ring_buffer->size)

{

Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+		size = ring_buffer->daddr + ring_buffer->size -

wr_ptr;

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+		ret = wave5_vdi_write_memory(inst->dev,

ring_buffer, offset,

(u8 *)buffer, size);

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +4 -0
+		if (ret < 0)
+			return ret;
+
+		ret = wave5_vdi_write_memory(inst->dev,

ring_buffer, 0, (u8

*)buffer + size,

Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+					     buffer_size -

size);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +4 -0
+		if (ret < 0)
+			return ret;
+	} else {
+		ret = wave5_vdi_write_memory(inst->dev,

ring_buffer, offset,

(u8 *)buffer,

Quoted 13 lines

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 263

Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7 -0
+					     buffer_size);
+		if (ret < 0)
+			return ret;
+	}
+
+	return 0;
+}

I have a pretty strong preference in seeing code being moved around in
seperate patch documented as "non functional changes". This way I know I
don't have to review again that code.

I will do it.

Quoted 61 lines

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c

Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -0
+
+static int fill_ringbuffer(struct vpu_instance *inst) {

Same, I don't want to have to do back and fourth to check if you changed
anything. Also, after the refactoring, the lockdeup assert can be added in
the new patch, which makes the introduction of the new lock very obvious.

Quoted 11 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +10 -0
+	struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
+	struct vpu_src_buffer *vpu_buf;
+	int ret = 0;
+
+	if (m2m_ctx->last_src_buf)  {
+		struct vpu_src_buffer *vpu_buf =
+wave5_to_vpu_src_buf(m2m_ctx->last_src_buf);
+
+		if (vpu_buf->consumed) {
+			dev_dbg(inst->dev->dev, "last src buffer

already

written\n");

Quoted 20 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5 -0
+			return 0;
+		}
+	}
+
+	list_for_each_entry(vpu_buf, &inst->avail_src_bufs,

list) {

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+		struct vb2_v4l2_buffer *vbuf = &vpu_buf-
Quoted 1 line

v4l2_m2m_buf.vb;

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+		struct vpu_buf *ring_buffer = &inst-
Quoted 1 line

bitstream_vbuf;

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+		size_t src_size = vb2_get_plane_payload(&vbuf-
Quoted 1 line

vb2_buf, 0);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+		void *src_buf = vb2_plane_vaddr(&vbuf->vb2_buf,

0);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +6 -0
+		dma_addr_t rd_ptr = 0;
+		dma_addr_t wr_ptr = 0;
+		size_t remain_size = 0;
+
+		if (vpu_buf->consumed) {
+			dev_dbg(inst->dev->dev, "already copied
Quoted 1 line

src buf (%u)

to the ring buffer\n",

Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7 -0
+				vbuf->vb2_buf.index);
+			continue;
+		}
+
+		if (!src_buf) {
+			dev_dbg(inst->dev->dev,
+				"%s: Acquiring kernel pointer to

src buf (%u),

fail\n",

Quoted 6 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5 -0
+				__func__, vbuf->vb2_buf.index);
+			break;
+		}
+
+		ret = wave5_vpu_dec_get_bitstream_buffer(inst,

&rd_ptr,

&wr_ptr, &remain_size);

Quoted 7 lines

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 304

Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -0
+		if (ret) {
+			/* Unable to acquire the mutex */

Might want to fix that comment in a separate patch, perhaps even remove
that ?

Quoted 1 line

Do you want to remove the comment in a separate patch ?

Yes please.

Quoted 198 lines
Quoted 197 lines
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+			dev_err(inst->dev->dev, "Getting the

bitstream buffer,

fail: %d\n",

Quoted 6 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5 -0
+				ret);
+			return ret;
+		}
+
+		dev_dbg(inst->dev->dev, "%s: rd_ptr %pad wr_ptr

%pad",

__func__,

Quoted 6 lines

+&rd_ptr, &wr_ptr);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +4 -0
+
+		if (remain_size < src_size) {
+			dev_dbg(inst->dev->dev,
+				"%s: remaining size: %zu <

source size: %zu for

src buf (%u)\n",

Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+				__func__, remain_size, src_size,

vbuf-
vb2_buf.index);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +4 -0
+			break;
+		}
+
+		ret = write_to_ringbuffer(inst, src_buf,

src_size,

ring_buffer, wr_ptr);

Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -0
+		if (ret) {
+			dev_err(inst->dev->dev, "Write src buf

(%u) to ring

buffer, fail: %d\n",

Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5 -0
+				vbuf->vb2_buf.index, ret);
+			return ret;
+		}
+
+		ret =

wave5_vpu_dec_update_bitstream_buffer(inst, src_size);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3 -0
+		if (ret) {
+			dev_dbg(inst->dev->dev,
+				"update_bitstream_buffer fail:

%d for src buf

(%u)\n",

Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7 -0
+				ret, vbuf->vb2_buf.index);
+			break;
+		}
+
+		vpu_buf->consumed = true;
+
+		/* Don't write buffers passed the last one while

draining.

*/

Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+		if (v4l2_m2m_is_last_draining_src_buf(m2m_ctx,

vbuf)) {

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+			dev_dbg(inst->dev->dev, "last src buffer

written to

the ring buffer\n");

Quoted 23 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +12 -0
+			break;
+		}
+
+		inst->queuing_num++;
+		list_del_init(&vpu_buf->list);
+		break;
+	}
+
+	return ret;
+}
+
+static void wave5_vpu_dec_feed_remaining(struct vpu_instance

*inst) {

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +9 -0
+	int ret = 0;
+	struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
+	u32 fail_res = 0;
+
+	mutex_lock(&inst->feed_lock);
+	ret = fill_ringbuffer(inst);
+	mutex_unlock(&inst->feed_lock);
+	if (ret) {
+		dev_warn(inst->dev->dev, "Filling ring buffer

failed\n");

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c

Quoted 1 line
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+		return;

I don't think the error handling is correct. What will happen with the
job ?

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int start_decode(struct vpu_instance *inst, u32 *fail_res) · line 364

Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3 -0
+	}
+
+	ret = start_decode(inst, &fail_res);

I'm a little worried seeing this called outside of device_run() context.

Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3 -0
+	if (ret) {
+		dev_err(inst->dev->dev,
+			"Frame decoding on m2m context (%p),

fail: %d

(result: %d)\n",

Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +6 -0
+			m2m_ctx, ret, fail_res);
+	}
+
+	v4l2_m2m_job_finish(inst->v4l2_m2m_dev, m2m_ctx);
 
 
 
+}
+
Quoted 6 lines

 static void flag_last_buffer_done(struct vpu_instance *inst) {
  struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx; @@ -347,7
+482,6 @@ static void wave5_vpu_dec_finish_decode(struct vpu_instance

*inst)

Quoted 11 lines

  struct vb2_v4l2_buffer *dec_buf = NULL;
  struct vb2_v4l2_buffer *disp_buf = NULL; struct vb2_queue *dst_vq = v4l2_m2m_get_dst_vq(m2m_ctx);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-	struct queue_status_info q_status;

  dev_dbg(inst->dev->dev, "%s: Fetch output info from firmware.",
__func__);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -0
@@ -441,20 +575,6 @@ static void

wave5_vpu_dec_finish_decode(struct

vpu_instance *inst)

Quoted 8 lines

  } spin_unlock_irqrestore(&inst->state_spinlock, flags);
  }

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -3
-
-	/*
-	 * During a resolution change and while draining, the

firmware may

flush

Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-	 * the reorder queue regardless of having a matching

decoding

operation

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c

Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-	 * pending. Only terminate the job if there are no more

IRQ coming.

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -2
-	 */
-	wave5_vpu_dec_give_command(inst, DEC_GET_QUEUE_STATUS,

&q_status);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -2
-	if (q_status.report_queue_count == 0 &&
-	    (q_status.instance_queue_count == 0 ||

dec_info.sequence_changed)) {

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c

Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-		dev_dbg(inst->dev->dev, "%s: finishing job.\n",

__func__);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -3
-		pm_runtime_mark_last_busy(inst->dev->dev);
-		pm_runtime_put_autosuspend(inst->dev->dev);
-		v4l2_m2m_job_finish(inst->v4l2_m2m_dev,

m2m_ctx);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-	}

This removal is why I believe some seek stress testing are needed. I'll
try and find some tests file I crafted last year.

Quoted 5 lines

 }

 static int wave5_vpu_dec_querycap(struct file *file, void *fh, struct
v4l2_capability *cap) @@ -794,11 +914,21 @@ static int

wave5_vpu_dec_stop(struct vpu_instance *inst)

Quoted 20 lines

  }

  if (inst->state != VPU_INST_STATE_NONE) {

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3 -0
+		struct vb2_v4l2_buffer *vbuf;
+		struct vpu_src_buffer *vpu_buf;
+

  /*
  * Temporarily release the state_spinlock so that subsequent
  * calls do not block on a mutex while inside this spinlock.
  */ spin_unlock_irqrestore(&inst->state_spinlock, flags);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5 -0
+		vbuf = v4l2_m2m_last_src_buf(m2m_ctx);
+		if (vbuf) {
+			vpu_buf = wave5_to_vpu_src_buf(vbuf);
+			if (!vpu_buf->consumed)
+				wave5_vpu_dec_feed_remaining(ins

t);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c

Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -0
+		}
+

Do you really need to add all these checks ? Shouldn't you just "feed_remaining()" and move on ? I also don't think its a good idea to
call v4l2_m2m_job_finish() in code that can run concurrently with the
device_run() function. This should only happen if you are certain the VPU
instance is stopped already.

Quoted 8 lines
Quoted 5 lines

While testing fluster,  stop command is sometimes called  before the last bitstream queued  is handled.
The stop command sets EOS before handling the last bitstream, so sometimes mismatch happens.
I think the code should be add in the wave5_vpu_dec_stop.

This would be a regression, we took good care of handling before and after case correctly.

I don't think this is a regression issue, but I will check it.

Quoted 632 lines
Quoted 251 lines
Quoted 244 lines
Quoted 8 lines

  ret = wave5_vpu_dec_set_eos_on_firmware(inst); if (ret)
  return ret;

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -0
@@ -1116,115 +1246,6 @@ static int wave5_prepare_fb(struct

vpu_instance

*inst)

Quoted 5 lines

  return 0;
 }

-static int write_to_ringbuffer(struct vpu_instance *inst, void *buffer,

size_t buffer_size,

Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-			       struct vpu_buf *ring_buffer,

dma_addr_t wr_ptr)

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -6
-{
-	size_t size;
-	size_t offset = wr_ptr - ring_buffer->daddr;
-	int ret;
-
-	if (wr_ptr + buffer_size > ring_buffer->daddr +

ring_buffer->size)

{

Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-		size = ring_buffer->daddr + ring_buffer->size -

wr_ptr;

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-		ret = wave5_vdi_write_memory(inst->dev,

ring_buffer, offset,

(u8 *)buffer, size);

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -4
-		if (ret < 0)
-			return ret;
-
-		ret = wave5_vdi_write_memory(inst->dev,

ring_buffer, 0, (u8

*)buffer + size,

Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-					     buffer_size -

size);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -4
-		if (ret < 0)
-			return ret;
-	} else {
-		ret = wave5_vdi_write_memory(inst->dev,

ring_buffer, offset,

(u8 *)buffer,

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -5
-					     buffer_size);
-		if (ret < 0)
-			return ret;
-	}
-

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static int wave5_prepare_fb(struct vpu_instance *inst)

Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -10
-	return 0;
-}
-
-static int fill_ringbuffer(struct vpu_instance *inst) -{
-	struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
-	struct v4l2_m2m_buffer *buf, *n;
-	int ret;
-
-	if (m2m_ctx->last_src_buf)  {
-		struct vpu_src_buffer *vpu_buf =

wave5_to_vpu_src_buf(m2m_ctx->last_src_buf);

Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -3
-
-		if (vpu_buf->consumed) {
-			dev_dbg(inst->dev->dev, "last src buffer

already

written\n");

Quoted 20 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -7
-			return 0;
-		}
-	}
-
-	v4l2_m2m_for_each_src_buf_safe(m2m_ctx, buf, n) {
-		struct vb2_v4l2_buffer *vbuf = &buf->vb;
-		struct vpu_src_buffer *vpu_buf =

wave5_to_vpu_src_buf(vbuf);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-		struct vpu_buf *ring_buffer = &inst-
Quoted 1 line

bitstream_vbuf;

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-		size_t src_size = vb2_get_plane_payload(&vbuf-
Quoted 1 line

vb2_buf, 0);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-		void *src_buf = vb2_plane_vaddr(&vbuf->vb2_buf,

0);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -6
-		dma_addr_t rd_ptr = 0;
-		dma_addr_t wr_ptr = 0;
-		size_t remain_size = 0;
-
-		if (vpu_buf->consumed) {
-			dev_dbg(inst->dev->dev, "already copied
Quoted 1 line

src buf (%u)

to the ring buffer\n",

Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -7
-				vbuf->vb2_buf.index);
-			continue;
-		}
-
-		if (!src_buf) {
-			dev_dbg(inst->dev->dev,
-				"%s: Acquiring kernel pointer to

src buf (%u),

fail\n",

Quoted 6 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -5
-				__func__, vbuf->vb2_buf.index);
-			break;
-		}
-
-		ret = wave5_vpu_dec_get_bitstream_buffer(inst,

&rd_ptr,

&wr_ptr, &remain_size);

Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -3
-		if (ret) {
-			/* Unable to acquire the mutex */
-			dev_err(inst->dev->dev, "Getting the

bitstream buffer,

fail: %d\n",

Quoted 6 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -5
-				ret);
-			return ret;
-		}
-
-		dev_dbg(inst->dev->dev, "%s: rd_ptr %pad wr_ptr

%pad",

__func__, &rd_ptr, &wr_ptr);

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -4
-
-		if (remain_size < src_size) {
-			dev_dbg(inst->dev->dev,
-				"%s: remaining size: %zu <

source size: %zu for

src buf (%u)\n",

Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-				__func__, remain_size, src_size,

vbuf-
vb2_buf.index);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -4
-			break;
-		}
-
-		ret = write_to_ringbuffer(inst, src_buf,

src_size,

ring_buffer, wr_ptr);

Quoted 3 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -2
-		if (ret) {
-			dev_err(inst->dev->dev, "Write src buf

(%u) to ring

buffer, fail: %d\n",

Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -5
-				vbuf->vb2_buf.index, ret);
-			return ret;
-		}
-
-		ret =

wave5_vpu_dec_update_bitstream_buffer(inst, src_size);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -3
-		if (ret) {
-			dev_dbg(inst->dev->dev,
-				"update_bitstream_buffer fail:

%d for src buf

(%u)\n",

Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -7
-				ret, vbuf->vb2_buf.index);
-			break;
-		}
-
-		vpu_buf->consumed = true;
-
-		/* Don't write buffers passed the last one while

draining.

*/

Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-		if (v4l2_m2m_is_last_draining_src_buf(m2m_ctx,

vbuf)) {

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-			dev_dbg(inst->dev->dev, "last src buffer

written to

the ring buffer\n");

Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -7
-			break;
-		}
-	}
-
-	return 0;
-}
-

 static void wave5_vpu_dec_buf_queue_src(struct vb2_buffer *vb) {
  struct vpu_instance *inst = vb2_get_drv_priv(vb-

Quoted 3 lines
Quoted 1 line

vb2_queue); @@

-1236,6 +1257,11 @@ static void
wave5_vpu_dec_buf_queue_src(struct

vb2_buffer *vb)

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c static void wave5_vpu_dec_buf_queue_src(struct vb2_buffer *vb) · line 1264

Quoted 8 lines

  vbuf->sequence = inst->queued_src_buf_num++;

  v4l2_m2m_buf_queue(m2m_ctx, vbuf);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5 -0
+
+	INIT_LIST_HEAD(&vpu_buf->list);
+	mutex_lock(&inst->feed_lock);
+	list_add_tail(&vpu_buf->list, &inst->avail_src_bufs);
+	mutex_unlock(&inst->feed_lock);

This lack documentation, you didn't even mention the introduction of a new
src buf list in the commit message.

Quoted 6 lines

 }

 static void wave5_vpu_dec_buf_queue_dst(struct vb2_buffer *vb) @@
-1362,7 +1388,6 @@ static int wave5_vpu_dec_start_streaming(struct

vb2_queue *q, unsigned int count

Quoted 21 lines

  goto return_buffers; }
  }
-
  } pm_runtime_mark_last_busy(inst->dev->dev); pm_runtime_put_autosuspend(inst->dev->dev);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +7 -0
@@ -1385,6 +1410,13 @@ static int streamoff_output(struct
 vb2_queue *q)
  	dma_addr_t new_rd_ptr;
  	struct dec_output_info dec_info;
  	unsigned int i;
+	struct vpu_src_buffer *vpu_buf, *tmp;
+
+	inst->retry = false;
+	inst->queuing_num = 0;
+
+	list_for_each_entry_safe(vpu_buf, tmp, &inst-
 >avail_src_bufs, list)
+		list_del_init(&vpu_buf->list);
 
Quoted 4 lines

  for (i = 0; i < v4l2_m2m_num_dst_bufs_ready(m2m_ctx); i++) {
  ret = wave5_vpu_dec_set_disp_flag(inst, i); @@ - 1481,10

+1513,8 @@

Quoted 4 lines

static void wave5_vpu_dec_stop_streaming(struct vb2_queue *q)

  wave5_vpu_dec_give_command(inst,
DEC_GET_QUEUE_STATUS,

&q_status);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c

Quoted 8 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -4
-		if (q_status.report_queue_count == 0)
-			break;
-
-		if (wave5_vpu_wait_interrupt(inst,

VPU_DEC_TIMEOUT) < 0)

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -0
+		if ((inst->state == VPU_INST_STATE_STOP ||
+q_status.instance_queue_count == 0) &&

State should be read with appropriate locking.

Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+		    q_status.report_queue_count == 0)

  break;

This loop have nothing to wait on anymore. It just actively pool on the HW.
I noticed your third patch, but if this is what you address there, squash
it somehow, this would break bisects.

Quoted 5 lines

The goal of that code was to ensure that this code will never run concurrently to finish_decode() once out of that while loop. I don't think
you break that, but that will cause possibly high CPU load.

Quoted 5 lines

The code is not related with the third patch, there was sometimes HW interrupt misses since changing to asynchronous, so an infinite wait happened when closed
So I removed it, but I think there is no high CPU load because it runs very short time.

If the firmware get stuck, Linux can spin, this cannot be done that way I'm sorry. That its most of the time spinning only few times does not seems like valid justification. We need a limit, we need some preemption, etc.

Quoted 50 lines
Quoted 37 lines
Quoted 4 lines

  if (wave5_vpu_dec_get_output_info(inst, &dec_output_info))

@@

Quoted 9 lines

-1577,13 +1607,23 @@ static void wave5_vpu_dec_device_run(void *priv)
  struct queue_status_info q_status;
  u32 fail_res = 0;
  int ret = 0;

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+	unsigned long flags;

  dev_dbg(inst->dev->dev, "%s: Fill the ring buffer with new

bitstream data", __func__);

Quoted 12 lines

  pm_runtime_resume_and_get(inst->dev->dev);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -3
-	ret = fill_ringbuffer(inst);
-	if (ret) {
-		dev_warn(inst->dev->dev, "Filling ring buffer

failed\n");

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +6 -1
-		goto finish_job_and_return;
+	if (!inst->retry) {
+		mutex_lock(&inst->feed_lock);
+		ret = fill_ringbuffer(inst);
+		mutex_unlock(&inst->feed_lock);
+		if (ret < 0) {
+			dev_warn(inst->dev->dev, "Filling ring
Quoted 1 line

buffer

failed\n");

Quoted 7 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +4 -0
+			goto finish_job_and_return;
+		} else if (!inst->eos &&
+			   inst->queuing_num == 0 &&
+			   inst->state ==

VPU_INST_STATE_PIC_RUN) {

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+			dev_dbg(inst->dev->dev, "%s: no

bitstream for feeding,

so skip ", __func__);

Quoted 11 lines

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c

Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -0
+			goto finish_job_and_return;
+		}

There is quite some overlap between that local src buffer queue and all
the code that depends on v4l2_m2m_num_src_bufs_ready(). Are you sure you
aren't re-implementing something ? Also, this code belong to the PIC_RUN
case in the below switch.

Quoted 5 lines

After src buffer was queued, v4l2_m2m_num_src_bufs_ready() always returns true until src buffer was removed in the finish_decode since get_result.
So checking if there is buffer or not in the src bufs list was needed in the device_run.

Ack.

Quoted 306 lines
Quoted 296 lines
Quoted 15 lines

  }

  switch (inst->state) {

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -0
@@ -1619,7 +1659,9 @@ static void wave5_vpu_dec_device_run(void
 *priv)
  		 * we had a chance to switch, which leads to an

invalid state
  * change.
  */

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+		spin_lock_irqsave(&inst->state_spinlock, flags);

  switch_state(inst, VPU_INST_STATE_PIC_RUN);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+		spin_unlock_irqrestore(&inst->state_spinlock,
Quoted 1 line

flags);

Should be its own patch, and fixed everywhere. Add missing lockdep assert
(if there is a spinlock version of it).

Quoted 4 lines

  /*
  * During DRC, the picture decoding remains pending, so just

leave

Quoted 1 line

the job @@ -1634,12 +1676,14 @@ static void

wave5_vpu_dec_device_run(void *priv)

Quoted 4 lines

  ret = wave5_prepare_fb(inst);
  if (ret) {
  dev_warn(inst->dev->dev, "Framebuffer preparation,

fail: %d\n",

Quoted 6 lines

ret);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+			spin_lock_irqsave(&inst->state_spinlock,

flags);
  switch_state(inst, VPU_INST_STATE_STOP);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+			spin_unlock_irqrestore(&inst-
Quoted 1 line

state_spinlock, flags);

Same, this is separate fix.

Quoted 6 lines

  break;
  }

  if (q_status.instance_queue_count) {

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -1
-			dev_dbg(inst->dev->dev, "%s: leave with

active job",

__func__);

Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -0
+			v4l2_m2m_job_finish(inst->v4l2_m2m_dev,

m2m_ctx);
  return;
  }

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -0
@@ -1650,14 +1694,22 @@ static void wave5_vpu_dec_device_run(void
 *priv)
  			dev_err(inst->dev->dev,

  "Frame decoding on m2m context (%p), fail: %d

(result: %d)\n",

Quoted 14 lines

  m2m_ctx, ret, fail_res);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +9 -1
-			break;
+			goto finish_job_and_return;
+		}
+
+		if (fail_res == WAVE5_SYSERR_QUEUEING_FAIL) {
+			inst->retry = true;
+		} else {
+			inst->retry = false;
+			if (!inst->eos)
+				inst->queuing_num--;

  }

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -2
-		/* Return so that we leave this job active */
-		dev_dbg(inst->dev->dev, "%s: leave with active
Quoted 1 line

job",

__func__);

Quoted 4 lines
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -3
-		return;
-	default:
-		WARN(1, "Execution of a job in state %s

illegal.\n",

state_to_str(inst->state));

Quoted 71 lines

  break;

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +3 -0
+	default:
+		if (!v4l2_m2m_has_stopped(m2m_ctx))
+			WARN(1, "Execution of a job in state %s

illegal.\n",

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -0
+			     state_to_str(inst->state));
+		return;

  }

 finish_job_and_return:

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +1 -4
@@ -1676,10 +1728,7 @@ static void wave5_vpu_dec_job_abort(void
 *priv)
  	if (ret)
  		return;
 
-	ret = wave5_vpu_dec_set_eos_on_firmware(inst);
-	if (ret)
-		dev_warn(inst->dev->dev,
-			 "Setting EOS for the bitstream, fail:
 %d\n", ret);
+	v4l2_m2m_job_finish(inst->v4l2_m2m_dev, inst-
 >v4l2_fh.m2m_ctx);

 }

 static int wave5_vpu_dec_job_ready(void *priv) @@ -1755,6 +1804,8 @@
static int wave5_vpu_open_dec(struct file *filp) inst->ops = &wave5_vpu_dec_inst_ops;

  spin_lock_init(&inst->state_spinlock);

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +2 -0
+	mutex_init(&inst->feed_lock);
+	INIT_LIST_HEAD(&inst->avail_src_bufs);

  inst->codec_info = kzalloc(sizeof(*inst->codec_info), GFP_KERNEL);
  if (!inst->codec_info)

drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +0 -3
@@ -1830,9 +1881,6 @@ static int wave5_vpu_open_dec(struct file
 *filp)
  	if (ret)
  		goto cleanup_inst;
 
-	if (list_empty(&dev->instances))
-		pm_runtime_use_autosuspend(inst->dev->dev);
-
  	list_add_tail(&inst->list, &dev->instances);
 

  mutex_unlock(&dev->dev_lock);

(quoted diff) +0 -0

enc.c b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c index 1e5fc5f8b856..cf20f774ed1b 100644

drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c +0 -3
@@ -1774,9 +1774,6 @@ static int wave5_vpu_open_enc(struct file
 *filp)
  	if (ret)
  		goto cleanup_inst;
 
-	if (list_empty(&dev->instances))
-		pm_runtime_use_autosuspend(inst->dev->dev);
-
  	list_add_tail(&inst->list, &dev->instances);
 

  mutex_unlock(&dev->dev_lock);

(quoted diff) +0 -0

vpuapi.c b/drivers/media/platform/chips-media/wave5/wave5-vpuapi.c index e5e879a13e8b..c1737fac6edd 100644

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance · line 207

Quoted 1 line
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -0
@@ -207,8 +207,6 @@ int wave5_vpu_dec_close(struct vpu_instance
 *inst,

u32 *fail_res)

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance · line 231

Quoted 10 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -2
  	int retry = 0;
  	struct vpu_device *vpu_dev = inst->dev;
  	int i;
-	int inst_count = 0;
-	struct vpu_instance *inst_elm;
 
  	*fail_res = 0;

  if (!inst->codec_info)

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -0
@@ -233,6 +231,16 @@ int wave5_vpu_dec_close(struct vpu_instance
 *inst,

u32 *fail_res)

Quoted 15 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +6 -0
  		    retry++ >= MAX_FIRMWARE_CALL_RETRY) {
  			ret = -ETIMEDOUT;
  			goto unlock_and_return;
+		} else if (*fail_res ==
 WAVE5_SYSERR_VPU_STILL_RUNNING) {
+			struct dec_output_info dec_info;
+
+			mutex_unlock(&vpu_dev->hw_lock);
+			wave5_vpu_dec_get_output_info(inst,
 &dec_info);
+			ret = mutex_lock_interruptible(&vpu_dev-
Quoted 1 line

hw_lock);

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +2 -0
+			if (ret) {
+				pm_runtime_put_sync(inst->dev-
Quoted 1 line

dev);

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c

Quoted 2 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +2 -0
+				return ret;
+			}

Please try and simplify all the branches. Something like:

			if (ret == 0)
				break;

			if (*fail_res !=
WAVE5_SYSERR_VPU_STILL_RUNNING) {
					dev_warn(inst->dev->dev,
"dec_finish_seq
timed out\n");
					goto unlock_and_return;
			}

			if (retry++ >= MAX_FIRMWARE_CALL_RETRY) {
					ret = -ETIMEDOUT;
					goto unlock_and_return;
			}

			mutex_unlock(&vpu_dev->hw_lock);
			wave5_vpu_dec_get_output_info(inst,

&dec_info);
ret = mutex_lock_interruptible(&vpu_dev-

Quoted 1 line

hw_lock);

			if (ret) {
				pm_runtime_put_sync(inst->dev-
Quoted 1 line

dev);

				return ret;
			}

My impression on that change is that it is independent and could be introduced separately.

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance · line 257

Quoted 5 lines

  }
  } while (ret != 0);

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -0
@@ -249,11 +257,7 @@ int wave5_vpu_dec_close(struct vpu_instance
 *inst,

u32 *fail_res)

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res) · line 256

Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -0
  	}
 

  wave5_vdi_free_dma_memory(vpu_dev, &p_dec_info-

Quoted 1 line

vb_task);

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -5
-
-	list_for_each_entry(inst_elm, &vpu_dev->instances, list)
-		inst_count++;
-	if (inst_count == 1)
-		pm_runtime_dont_use_autosuspend(vpu_dev->dev);

This seems like some other fix, that should be in its own commit.

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_enc_close(struct vpu_instance · line 724

Quoted 6 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +1 -0
+	mutex_destroy(&inst->feed_lock);

 unlock_and_return:
  mutex_unlock(&vpu_dev->hw_lock);

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -0
@@ -720,8 +724,6 @@ int wave5_vpu_enc_close(struct vpu_instance
 *inst,

u32 *fail_res)

Quoted 14 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -2
  	int ret;
  	int retry = 0;
  	struct vpu_device *vpu_dev = inst->dev;
-	int inst_count = 0;
-	struct vpu_instance *inst_elm;
 
  	*fail_res = 0;

  if (!inst->codec_info)

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -0
@@ -765,11 +767,6 @@ int wave5_vpu_enc_close(struct vpu_instance
 *inst, u32 *fail_res)
 

  wave5_vdi_free_dma_memory(vpu_dev, &p_enc_info-

Quoted 1 line

vb_task);

drivers/media/platform/chips-media/wave5/wave5-vpuapi.c int wave5_vpu_enc_close(struct vpu_instance *inst, u32 *fail_res) · line 772

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.c +0 -5
-	list_for_each_entry(inst_elm, &vpu_dev->instances, list)
-		inst_count++;
-	if (inst_count == 1)
-		pm_runtime_dont_use_autosuspend(vpu_dev->dev);
-

Same.

Quoted 10 lines

  mutex_unlock(&vpu_dev->hw_lock);
  pm_runtime_put_sync(inst->dev->dev);

(quoted diff) +0 -0

vpuapi.h b/drivers/media/platform/chips-media/wave5/wave5-vpuapi.h index 45615c15beca..6ca1ddc67c64 100644

drivers/media/platform/chips-media/wave5/wave5-vpuapi.h enum set_param_option { · line 166

Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +1 -1
@@ -163,7 +163,7 @@ enum set_param_option {
 
  #define BUFFER_MARGIN				4096
 
-#define MAX_FIRMWARE_CALL_RETRY			10
+#define MAX_FIRMWARE_CALL_RETRY			30

I thought 10s was already very long, am I right this means it can get
stuck for 30s now ?

drivers/media/platform/chips-media/wave5/wave5-vpuapi.h struct vpu_instance { · line 816

Quoted 9 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +2 -0
  #define VDI_LITTLE_ENDIAN	0x0
 
@@ -812,6 +812,9 @@ struct vpu_instance {
  	bool cbcr_interleave;
  	bool nv21;
  	bool eos;
+	bool retry;
+	int queuing_num;

Some comment would be helpful. What does "retry" stand for, and how is
queing_num different from v4l2_m2m_num_src_bufs_ready() ?

Quoted 9 lines
Quoted 5 lines
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +1 -0
+	struct mutex feed_lock; /* lock for feeding bitstream

buffers */
  struct vpu_buf bitstream_vbuf;
  dma_addr_t last_rd_ptr;
  size_t remaining_consumed_bytes;

--
Nicolas Dufresne
Principal Engineer at Collabora

Quoted 2 lines

Thanks
Jackson

--
Nicolas Dufresne
Principal Engineer at Collabora

author maintainer
#6 lore raw

PATCH RESEND 2/3 media: chips-media: wave5: Reduce high CPU load

In reply to Jackson.lee

From: Jackson Lee <[email protected]>

Since applying changes for performance improvement of decoder, there was a problem related to high CPU load.
CPU load was more than 4 times when comparing CPU load. The root cause was the device_run was called many times even if there was no bitstream which should be queued.

---
 .../media/platform/chips-media/wave5/wave5-vpu-dec.c | 12 +++++++++---
 .../media/platform/chips-media/wave5/wave5-vpuapi.h  |  1 +
 2 files changed, 10 insertions(+), 3 deletions(-)
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +9 -3
index cc47da509703..5ba3aaba24ce 100644
@@ -1313,10 +1313,13 @@ static void wave5_vpu_dec_buf_queue(struct vb2_buffer *vb)
 		__func__, vb->type, vb->index, vb2_plane_size(&vbuf->vb2_buf, 0),
 		vb2_plane_size(&vbuf->vb2_buf, 1), vb2_plane_size(&vbuf->vb2_buf, 2));
 
-	if (vb->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+	if (vb->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+		if (inst->empty_queue)
+			inst->empty_queue = false;
 		wave5_vpu_dec_buf_queue_src(vb);
-	else if (vb->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+	} else if (vb->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
 		wave5_vpu_dec_buf_queue_dst(vb);
+	}
 }
 
 static int wave5_vpu_dec_allocate_ring_buffer(struct vpu_instance *inst)
@@ -1506,6 +1509,7 @@ static void wave5_vpu_dec_stop_streaming(struct vb2_queue *q)
 
 	dev_dbg(inst->dev->dev, "%s: type: %u\n", __func__, q->type);
 	pm_runtime_resume_and_get(inst->dev->dev);
+	inst->empty_queue = false;
 
 	while (check_cmd) {
 		struct queue_status_info q_status;
@@ -1622,6 +1626,7 @@ static void wave5_vpu_dec_device_run(void *priv)
 			   inst->queuing_num == 0 &&
 			   inst->state == VPU_INST_STATE_PIC_RUN) {
 			dev_dbg(inst->dev->dev, "%s: no bitstream for feeding, so skip ", __func__);
+			inst->empty_queue = true;
 			goto finish_job_and_return;
 		}
 	}
@@ -1764,7 +1769,8 @@ static int wave5_vpu_dec_job_ready(void *priv)
 				"No capture buffer ready to decode!\n");
 			break;
 		} else if (!wave5_is_draining_or_eos(inst) &&
-			   !v4l2_m2m_num_src_bufs_ready(m2m_ctx)) {
+			   (!v4l2_m2m_num_src_bufs_ready(m2m_ctx) ||
+			    inst->empty_queue)) {
 			dev_dbg(inst->dev->dev,
 				"No bitstream data to decode!\n");
 			break;
drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +1 -0
index 6ca1ddc67c64..3f6e37b6be86 100644
@@ -813,6 +813,7 @@ struct vpu_instance {
 	bool nv21;
 	bool eos;
 	bool retry;
+	bool empty_queue;
 	int queuing_num;
 	struct mutex feed_lock; /* lock for feeding bitstream buffers */
 	struct vpu_buf bitstream_vbuf;
Signature · 1 line
2.43.0
author maintainer
#7 lore raw

PATCH RESEND 3/3 media: chips-media: wave5: Fix Null reference while testing fluster

In reply to Jackson.lee

From: Jackson Lee <[email protected]>

When multi instances are created/destroyed, many interrupts happens or structures for decoder are removed.
"struct vpu_instance" this structure is shared for all flow in decoder, so if the structure is not protected by lock, Null reference exception could happens sometimes.
IRQ Handler was spilt to two phases and Lock was added as well.

---
 .../platform/chips-media/wave5/wave5-helper.c | 10 ++-
 .../chips-media/wave5/wave5-vpu-dec.c         |  5 ++
 .../chips-media/wave5/wave5-vpu-enc.c         |  5 ++
 .../platform/chips-media/wave5/wave5-vpu.c    | 69 ++++++++++++++++---
 .../platform/chips-media/wave5/wave5-vpuapi.h |  6 ++
 5 files changed, 86 insertions(+), 9 deletions(-)
drivers/media/platform/chips-media/wave5/wave5-helper.c +9 -1
index 2c9d8cbca6e4..5d9969bb7ada 100644
@@ -49,7 +49,7 @@ void wave5_cleanup_instance(struct vpu_instance *inst)
 		v4l2_fh_del(&inst->v4l2_fh);
 		v4l2_fh_exit(&inst->v4l2_fh);
 	}
-	list_del_init(&inst->list);
+	kfifo_free(&inst->irq_status);
 	ida_free(&inst->dev->inst_ida, inst->id);
 	kfree(inst->codec_info);
 	kfree(inst);
@@ -61,8 +61,16 @@ int wave5_vpu_release_device(struct file *filp,
 {
 	struct vpu_instance *inst = wave5_to_vpu_inst(filp->private_data);
 	int ret = 0;
+	unsigned long flags;
 
 	v4l2_m2m_ctx_release(inst->v4l2_fh.m2m_ctx);
+	ret = mutex_lock_interruptible(&inst->dev->irq_lock);
+	if (ret)
+		return ret;
+	spin_lock_irqsave(&inst->dev->irq_spinlock, flags);
+	list_del_init(&inst->list);
+	spin_unlock_irqrestore(&inst->dev->irq_spinlock, flags);
+	mutex_unlock(&inst->dev->irq_lock);
 	if (inst->state != VPU_INST_STATE_NONE) {
 		u32 fail_res;
 
drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c +5 -0
index 5ba3aaba24ce..38f7a98192ad 100644
@@ -1868,6 +1868,11 @@ static int wave5_vpu_open_dec(struct file *filp)
 	inst->xfer_func = V4L2_XFER_FUNC_DEFAULT;
 
 	init_completion(&inst->irq_done);
+	ret = kfifo_alloc(&inst->irq_status, 16 * sizeof(int), GFP_KERNEL);
+	if (ret) {
+		dev_err(inst->dev->dev, "failed to allocate fifo\n");
+		goto cleanup_inst;
+	}
 
 	inst->id = ida_alloc(&inst->dev->inst_ida, GFP_KERNEL);
 	if (inst->id < 0) {
drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c +5 -0
index cf20f774ed1b..7f1aa392805f 100644
@@ -1760,6 +1760,11 @@ static int wave5_vpu_open_enc(struct file *filp)
 	inst->frame_rate = 30;
 
 	init_completion(&inst->irq_done);
+	ret = kfifo_alloc(&inst->irq_status, 16 * sizeof(int), GFP_KERNEL);
+	if (ret) {
+		dev_err(inst->dev->dev, "failed to allocate fifo\n");
+		goto cleanup_inst;
+	}
 
 	inst->id = ida_alloc(&inst->dev->inst_ida, GFP_KERNEL);
 	if (inst->id < 0) {
drivers/media/platform/chips-media/wave5/wave5-vpu.c +61 -8

127 lines not on this page · raw message

drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +6 -0
index 3f6e37b6be86..6fcaae62bd73 100644
@@ -8,6 +8,7 @@
 #ifndef VPUAPI_H_INCLUDED
 #define VPUAPI_H_INCLUDED
 
+#include <linux/kfifo.h>
 #include <linux/idr.h>
 #include <linux/genalloc.h>
 #include <media/v4l2-device.h>
@@ -747,6 +748,7 @@ struct vpu_device {
 	struct video_device *video_dev_enc;
 	struct mutex dev_lock; /* lock for the src, dst v4l2 queues */
 	struct mutex hw_lock; /* lock hw configurations */
+	struct mutex irq_lock;
 	int irq;
 	enum product_id product;
 	struct vpu_attr attr;
@@ -764,7 +766,10 @@ struct vpu_device {
 	struct kthread_worker *worker;
 	int vpu_poll_interval;
 	int num_clks;
+	struct task_struct *irq_thread;
+	struct semaphore irq_sem;
 	struct reset_control *resets;
+	spinlock_t irq_spinlock; /* protect instances list */
 };
 
 struct vpu_instance;
@@ -788,6 +793,7 @@ struct vpu_instance {
 	enum v4l2_ycbcr_encoding ycbcr_enc;
 	enum v4l2_quantization quantization;
 
+	struct kfifo irq_status;
 	enum vpu_instance_state state;
 	enum vpu_instance_type type;
 	const struct vpu_instance_ops *ops;
Signature · 1 line
2.43.0
#8 lore raw
In reply to Jackson.lee

Hi,

Le mercredi 19 mars 2025 à 12:50 +0900, Jackson.lee a écrit :

Quoted 24 lines

From: Jackson Lee <[email protected]>

The wave5 codec driver is a stateful encoder/decoder. The following patches is for improving decoder performance.

v4l2-compliance results:
========================

v4l2-compliance 1.28.1-5233, 64 bits, 64-bit time_t

Buffer ioctls:
                warn: v4l2-test-buffers.cpp(693): VIDIOC_CREATE_BUFS not supported
                warn: v4l2-test-buffers.cpp(693): VIDIOC_CREATE_BUFS not supported test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
        test CREATE_BUFS maximum buffers: OK
        test VIDIOC_EXPBUF: OK
        test Requests: OK (Not Supported)

Total for wave5-dec device /dev/video0: 46, Succeeded: 46, Failed: 0, Warnings: 2 Total for wave5-enc device /dev/video1: 46, Succeeded: 46, Failed: 0, Warnings: 0

Fluster test results:
=====================

Quoted 8 lines

Running test suite JCT-VC-HEVC_V1 with decoder GStreamer-H.265-V4L2-Gst1.0 Using 1 parallel job(s) Ran 133/147 tests successfully               in 68.517 secs

(1 test fails because of not supporting to parse multi frames, 1 test fails because of a missing frame and slight corruption, 2 tests fail because of sizes which are incompatible with the IP, 11 tests fail because of unsupported 10 bit format)

Running test suite JVT-AVC_V1 with decoder GStreamer-H.264-V4L2-Gst1.0 Using 1 parallel job(s) Ran 78/135 tests successfully               in 35.015 secs

I'd be more confortable if I could get test report using 2 or 3 parallel instances.

Also, have you conducted manual "seek" tests, and stress tested the dynamic resolution changes ? We don't have automated tests for these anywhere yet.

Nicolas

Quoted 16 lines

(57 fail because the hardware is unable to decode  MBAFF / FMO / Field / Extended profile streams.)

Jackson Lee (3):
  media: chips-media: wave5: Improve performance of decoder
  media: chips-media: wave5: Reduce high CPU load
  media: chips-media: wave5: Fix Null reference while testing fluster

 .../platform/chips-media/wave5/wave5-helper.c |  10 +- .../chips-media/wave5/wave5-vpu-dec.c         | 361 ++++++++++-------- .../chips-media/wave5/wave5-vpu-enc.c         |   8 +- .../platform/chips-media/wave5/wave5-vpu.c    |  69 +++- .../platform/chips-media/wave5/wave5-vpuapi.c |  25 +- .../platform/chips-media/wave5/wave5-vpuapi.h |  12 +-

 6 files changed, 307 insertions(+), 178 deletions(-)
author maintainer
#9 lore raw

Hi Nicolas

Quoted 54 lines

-----Original Message-----
From: Nicolas Dufresne <[email protected]>
Sent: Thursday, April 3, 2025 12:44 AM
To: jackson.lee <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; lafley.kim <[email protected]>; [email protected]; [email protected]; Nas Chung <[email protected]>
Subject: Re: [RESEND PATCH v0 0/3] Performance improvement of decoder

Hi,

Le mercredi 19 mars 2025 à 12:50 +0900, Jackson.lee a écrit :

Quoted 31 lines

From: Jackson Lee <[email protected]>

The wave5 codec driver is a stateful encoder/decoder. The following patches is for improving decoder performance.

v4l2-compliance results:
========================

v4l2-compliance 1.28.1-5233, 64 bits, 64-bit time_t

Buffer ioctls:
                warn: v4l2-test-buffers.cpp(693): VIDIOC_CREATE_BUFS not supported
                warn: v4l2-test-buffers.cpp(693): VIDIOC_CREATE_BUFS not supported
        test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
        test CREATE_BUFS maximum buffers: OK
        test VIDIOC_EXPBUF: OK
        test Requests: OK (Not Supported)

Total for wave5-dec device /dev/video0: 46, Succeeded: 46, Failed: 0,
Warnings: 2 Total for wave5-enc device /dev/video1: 46, Succeeded: 46,
Failed: 0, Warnings: 0

Fluster test results:
=====================

Running test suite JCT-VC-HEVC_V1 with decoder GStreamer-H.265-V4L2-Gst1.0 Using 1 parallel job(s) Ran 133/147 tests successfully               in 68.517 secs

Quoted 8 lines

(1 test fails because of not supporting to parse multi frames, 1 test fails because of a missing frame and slight corruption,
 2 tests fail because of sizes which are incompatible with the IP, 11 tests fail because of unsupported 10 bit format)

Running test suite JVT-AVC_V1 with decoder GStreamer-H.264-V4L2-Gst1.0 Using 1 parallel job(s) Ran 78/135 tests successfully               in 35.015 secs

Quoted 9 lines

I'd be more confortable if I could get test report using 2 or 3 parallel instances.

Also, have you conducted manual "seek" tests, and stress tested the dynamic resolution changes ? We don't have automated tests for these anywhere yet.

Nicolas

I will test them and update it.

Thanks
Jackson

Quoted 18 lines
Quoted 18 lines

(57 fail because the hardware is unable to decode  MBAFF / FMO / Field / Extended profile streams.)

Jackson Lee (3):
  media: chips-media: wave5: Improve performance of decoder
  media: chips-media: wave5: Reduce high CPU load
  media: chips-media: wave5: Fix Null reference while testing fluster

 .../platform/chips-media/wave5/wave5-helper.c |  10 +- .../chips-media/wave5/wave5-vpu-dec.c         | 361
++++++++++--------
 .../chips-media/wave5/wave5-vpu-enc.c         |   8 +- .../platform/chips-media/wave5/wave5-vpu.c    |  69 +++- .../platform/chips-media/wave5/wave5-vpuapi.c |  25 +- .../platform/chips-media/wave5/wave5-vpuapi.h |  12 +-

 6 files changed, 307 insertions(+), 178 deletions(-)