Skip to contentSkip to conversation

Add support for mt2701 JPEG ENC support

PATCH Superseded Xia Jiang 3 participants 17 messages Oct 17, 2019 – May 1, 2020

Subsystems (3)

v4 of 12 Superseded Latest is v12 →

lore
Share & tools
lore

Xia Jiang

author
#0 lore raw

v4 cover Add support for mt2701 JPEG ENC support

This patchset add support for mt2701 JPEG ENC support.

This is the compliance test result for jpeg dec and enc.

The JPEG dec log:
------------------------------------------------------------
v4l2-compliance -d /dev/video0
v4l2-compliance SHA: b51e9a8a74eed6da15127fe5eadf515914347f76, 32 bits

Compliance test for mtk-jpeg device /dev/video0:

Driver Info:

        Driver name      : mtk-jpeg                                                
        Card type        : mtk-jpeg decoder                                        
        Bus info         : platform:15004000.jpegdec                               
        Driver version   : 5.4.0                                                   
        Capabilities     : 0x84204000                                              

Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Detected JPEG Decoder

Required ioctls:
test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
test second /dev/video0 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK

Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 0 Private Controls: 0

Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK
test Composing: OK
test Scaling: OK

Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)

Total for mtk-jpeg device /dev/video0: 44, Succeeded: 44, Failed: 0, Warnings: 0
------------------------------------------------------------

The JPEG enc log:

------------------------------------------------------------
v4l2-compliance -d /dev/video1
v4l2-compliance SHA: b51e9a8a74eed6da15127fe5eadf515914347f76, 32 bits

Compliance test for mtk-jpeg device /dev/video1:

Driver Info:

        Driver name      : mtk-jpeg
        Card type        : mtk-jpeg encoder
        Bus info         : platform:1500a000.jpegenc
        Driver version   : 5.4.0
        Capabilities     : 0x84204000

Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Detected JPEG Encoder

Required ioctls:
test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
test second /dev/video1 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK

Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 4 Private Controls: 0

Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK
test Composing: OK
test Scaling: OK (Not Supported)

Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)

Total for mtk-jpeg device /dev/video1: 44, Succeeded: 44, Failed: 0, Warnings: 0
------------------------------------------------------------

Change compared to v3:
-put original v4l2-compliance bug fix in a separate patch. -split mtk_jpeg_try_fmt_mplane() to two functions, one for encoder, one
for decoder.
-split mtk_jpeg_set_default_params() to two functions, one for encoder,
one for decoder.
-add cropping support for encoder in g/s_selection ioctls. -cancle v3 patch 'media:v4l2-ctrl: Add jpeg enc exif mode control',
because that exif information can use V4L2_JPEG_ACTIVE_MARKER_APP1
annotation: Exif metadata are restricted in size to 64kB in JPEG images because according to the specification this information be contained within a single JPEG APP1 segment (from wikipedia).
-change MTK_JPEG_MAX_WIDTH/MTK_JPEG_MAX_HEIGH from 8192 to 65535 by
specification.
-move width shifting operation behind aligning operation in
mtk_jpeg_try_enc_fmt_mplane() for bug fix.
-fix user abuseing data_offset issue for DMABUF in
mtk_jpeg_set_enc_src().
-fix kbuild warings: change MTK_JPEG_MIN_HEIGHT/MTK_JPEG_MAX_HEIGHT and
MTK_JPEG_MIN_WIDTH/MTK_JPEG_MAX_WIDTH from int type to unsigned int type.
fix msleadingly indented of 'else'.

Xia Jiang (5):
media: dt-bindings: Add jpeg enc device tree node document
arm: dts: Add jpeg enc device tree node
media: platform: Rename jpeg dec file name
media: platform: Fix v4l2-compliance test bug
media: platform: Add jpeg dec/enc feature

 .../bindings/media/mediatek-jpeg-encoder.txt  |  37 +
 arch/arm/boot/dts/mt2701.dtsi                 |  13 +
 drivers/media/platform/mtk-jpeg/Makefile      |   5 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 746 ++++++++++++++----
 .../media/platform/mtk-jpeg/mtk_jpeg_core.h   | 123 ++-
 .../{mtk_jpeg_hw.c => mtk_jpeg_dec_hw.c}      |   2 +-
 .../{mtk_jpeg_hw.h => mtk_jpeg_dec_hw.h}      |   9 +-
 ...{mtk_jpeg_parse.c => mtk_jpeg_dec_parse.c} |   2 +-
 ...{mtk_jpeg_parse.h => mtk_jpeg_dec_parse.h} |   2 +-
 .../{mtk_jpeg_reg.h => mtk_jpeg_dec_reg.h}    |   0
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c | 175 ++++
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h |  60 ++
 .../platform/mtk-jpeg/mtk_jpeg_enc_reg.h      |  49 ++
 13 files changed, 1069 insertions(+), 154 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek-jpeg-encoder.txt
 rename drivers/media/platform/mtk-jpeg/{mtk_jpeg_hw.c => mtk_jpeg_dec_hw.c} (99%)
 rename drivers/media/platform/mtk-jpeg/{mtk_jpeg_hw.h => mtk_jpeg_dec_hw.h} (91%)
 rename drivers/media/platform/mtk-jpeg/{mtk_jpeg_parse.c => mtk_jpeg_dec_parse.c} (98%)
 rename drivers/media/platform/mtk-jpeg/{mtk_jpeg_parse.h => mtk_jpeg_dec_parse.h} (92%)
 rename drivers/media/platform/mtk-jpeg/{mtk_jpeg_reg.h => mtk_jpeg_dec_reg.h} (100%)
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h
Signature · 1 line
2.18.0
author
#1 lore raw

PATCH v4 1/5 media: dt-bindings: Add jpeg enc device tree node document

Add jpeg enc device tree node document

---
v4: no changes

v3: change compatible to SoC specific compatible

v2: no changes

---
 .../bindings/media/mediatek-jpeg-encoder.txt  | 37 +++++++++++++++++++
 1 file changed, 37 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek-jpeg-encoder.txt
Documentation/devicetree/bindings/media/mediatek-jpeg-encoder.txt new file +37 -0
new file mode 100644
index 000000000000..fa8da699493b
@@ -0,0 +1,37 @@
+* MediaTek JPEG Encoder
+
+MediaTek JPEG Encoder is the JPEG encode hardware present in MediaTek SoCs
+
+Required properties:
+- compatible : should be one of:
+               "mediatek,mt2701-jpgenc"
+               ...
+               followed by "mediatek,mtk-jpgenc"
+- reg : physical base address of the JPEG encoder registers and length of
+  memory mapped region.
+- interrupts : interrupt number to the interrupt controller.
+- clocks: device clocks, see
+  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
+- clock-names: must contain "jpgenc". It is the clock of JPEG encoder.
+- power-domains: a phandle to the power domain, see
+  Documentation/devicetree/bindings/power/power_domain.txt for details.
+- mediatek,larb: must contain the local arbiters in the current SoCs, see
+  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
+  for details.
+- iommus: should point to the respective IOMMU block with master port as
+  argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
+  for details.
+
+Example:
+	jpegenc: jpegenc@1500a000 {
+		compatible = "mediatek,mt2701-jpgenc",
+			     "mediatek,mtk-jpgenc";
+		reg = <0 0x1500a000 0 0x1000>;
+		interrupts = <GIC_SPI 141 IRQ_TYPE_LEVEL_LOW>;
+		clocks =  <&imgsys CLK_IMG_VENC>;
+		clock-names = "jpgenc";
+		power-domains = <&scpsys MT2701_POWER_DOMAIN_ISP>;
+		mediatek,larb = <&larb2>;
+		iommus = <&iommu MT2701_M4U_PORT_JPGENC_RDMA>,
+			 <&iommu MT2701_M4U_PORT_JPGENC_BSDMA>;
+	};
Signature · 1 line
2.18.0
author
#2 lore raw

PATCH v4 2/5 arm: dts: Add jpeg enc device tree node

In reply to Xia Jiang

Add jpeg enc device tree node

---
v4: no changes

v3: change compatible to SoC specific compatible

v2: no changes

---
 arch/arm/boot/dts/mt2701.dtsi | 13 +++++++++++++
 1 file changed, 13 insertions(+)
arch/arm/boot/dts/mt2701.dtsi +13 -0
index 51e1305c6471..f2f92150b3fb 100644
@@ -569,6 +569,19 @@
 			 <&iommu MT2701_M4U_PORT_JPGDEC_BSDMA>;
 	};
 
+	jpegenc: jpegenc@1500a000 {
+		compatible = "mediatek,mt2701-jpgenc",
+			     "mediatek,mtk-jpgenc";
+		reg = <0 0x1500a000 0 0x1000>;
+		interrupts = <GIC_SPI 141 IRQ_TYPE_LEVEL_LOW>;
+		clocks =  <&imgsys CLK_IMG_VENC>;
+		clock-names = "jpgenc";
+		power-domains = <&scpsys MT2701_POWER_DOMAIN_ISP>;
+		mediatek,larb = <&larb2>;
+		iommus = <&iommu MT2701_M4U_PORT_JPGENC_RDMA>,
+			 <&iommu MT2701_M4U_PORT_JPGENC_BSDMA>;
+	};
+
 	vdecsys: syscon@16000000 {
 		compatible = "mediatek,mt2701-vdecsys", "syscon";
 		reg = <0 0x16000000 0 0x1000>;
Signature · 1 line
2.18.0
author
#3 lore raw

PATCH v4 3/5 media: platform: Rename jpeg dec file name

In reply to Xia Jiang

Rename the files which are for decode feature. This is preparing path since the jpeg enc patch will be added later.

---
v4: no changes

v3: no changes

v2: no changes

---
 drivers/media/platform/mtk-jpeg/Makefile                      | 2 +-
 drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c               | 4 ++--
 .../platform/mtk-jpeg/{mtk_jpeg_hw.c => mtk_jpeg_dec_hw.c}    | 2 +-
 .../platform/mtk-jpeg/{mtk_jpeg_hw.h => mtk_jpeg_dec_hw.h}    | 2 +-
 .../mtk-jpeg/{mtk_jpeg_parse.c => mtk_jpeg_dec_parse.c}       | 2 +-
 .../mtk-jpeg/{mtk_jpeg_parse.h => mtk_jpeg_dec_parse.h}       | 2 +-
 .../platform/mtk-jpeg/{mtk_jpeg_reg.h => mtk_jpeg_dec_reg.h}  | 0
 7 files changed, 7 insertions(+), 7 deletions(-)
 rename drivers/media/platform/mtk-jpeg/{mtk_jpeg_hw.c => mtk_jpeg_dec_hw.c} (99%)
 rename drivers/media/platform/mtk-jpeg/{mtk_jpeg_hw.h => mtk_jpeg_dec_hw.h} (98%)
 rename drivers/media/platform/mtk-jpeg/{mtk_jpeg_parse.c => mtk_jpeg_dec_parse.c} (98%)
 rename drivers/media/platform/mtk-jpeg/{mtk_jpeg_parse.h => mtk_jpeg_dec_parse.h} (92%)
 rename drivers/media/platform/mtk-jpeg/{mtk_jpeg_reg.h => mtk_jpeg_dec_reg.h} (100%)
drivers/media/platform/mtk-jpeg/Makefile +1 -1
index 92a4fc046bfe..48516dcf96e6 100644
@@ -1,3 +1,3 @@
 # SPDX-License-Identifier: GPL-2.0-only
-mtk_jpeg-objs := mtk_jpeg_core.o mtk_jpeg_hw.o mtk_jpeg_parse.o
+mtk_jpeg-objs := mtk_jpeg_core.o mtk_jpeg_dec_hw.o mtk_jpeg_dec_parse.o
 obj-$(CONFIG_VIDEO_MEDIATEK_JPEG) += mtk_jpeg.o
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -2
index ee802fc3bcdf..5f0990fce8ef 100644
@@ -23,9 +23,9 @@
 #include <media/videobuf2-dma-contig.h>
 #include <soc/mediatek/smi.h>
 
-#include "mtk_jpeg_hw.h"
+#include "mtk_jpeg_dec_hw.h"
 #include "mtk_jpeg_core.h"
-#include "mtk_jpeg_parse.h"
+#include "mtk_jpeg_dec_parse.h"
 
 static struct mtk_jpeg_fmt mtk_jpeg_formats[] = {
 	{
drivers/media/platform/mtk-jpeg/mtk_jpeg_dec_hw.c renamed from drivers/media/platform/mtk-jpeg/mtk_jpeg_hw.c +1 -1
similarity index 99%
rename from drivers/media/platform/mtk-jpeg/mtk_jpeg_hw.c
rename to drivers/media/platform/mtk-jpeg/mtk_jpeg_dec_hw.c
index ddf0dfa78e20..922dbe421fae 100644
@@ -9,7 +9,7 @@
 #include <linux/kernel.h>
 #include <media/videobuf2-core.h>
 
-#include "mtk_jpeg_hw.h"
+#include "mtk_jpeg_dec_hw.h"
 
 #define MTK_JPEG_DUNUM_MASK(val)	(((val) - 1) & 0x3)
 
drivers/media/platform/mtk-jpeg/mtk_jpeg_dec_hw.h renamed from drivers/media/platform/mtk-jpeg/mtk_jpeg_hw.h +1 -1
similarity index 98%
rename from drivers/media/platform/mtk-jpeg/mtk_jpeg_hw.h
rename to drivers/media/platform/mtk-jpeg/mtk_jpeg_dec_hw.h
index 9c6584eaad99..a37be1a48415 100644
@@ -11,7 +11,7 @@
 #include <media/videobuf2-core.h>
 
 #include "mtk_jpeg_core.h"
-#include "mtk_jpeg_reg.h"
+#include "mtk_jpeg_dec_reg.h"
 
 enum {
 	MTK_JPEG_DEC_RESULT_EOF_DONE		= 0,
drivers/media/platform/mtk-jpeg/mtk_jpeg_dec_parse.c renamed from drivers/media/platform/mtk-jpeg/mtk_jpeg_parse.c +1 -1
similarity index 98%
rename from drivers/media/platform/mtk-jpeg/mtk_jpeg_parse.c
rename to drivers/media/platform/mtk-jpeg/mtk_jpeg_dec_parse.c
index f862d38f3af7..b95c45791c29 100644
@@ -8,7 +8,7 @@
 #include <linux/kernel.h>
 #include <linux/videodev2.h>
 
-#include "mtk_jpeg_parse.h"
+#include "mtk_jpeg_dec_parse.h"
 
 #define TEM	0x01
 #define SOF0	0xc0
drivers/media/platform/mtk-jpeg/mtk_jpeg_dec_parse.h renamed from drivers/media/platform/mtk-jpeg/mtk_jpeg_parse.h +1 -1
similarity index 92%
rename from drivers/media/platform/mtk-jpeg/mtk_jpeg_parse.h
rename to drivers/media/platform/mtk-jpeg/mtk_jpeg_dec_parse.h
index 0a48eeabaff2..2918f15811f8 100644
@@ -8,7 +8,7 @@
 #ifndef _MTK_JPEG_PARSE_H
 #define _MTK_JPEG_PARSE_H
 
-#include "mtk_jpeg_hw.h"
+#include "mtk_jpeg_dec_hw.h"
 
 bool mtk_jpeg_parse(struct mtk_jpeg_dec_param *param, u8 *src_addr_va,
 		    u32 src_size);
drivers/media/platform/mtk-jpeg/mtk_jpeg_dec_reg.h renamed from drivers/media/platform/mtk-jpeg/mtk_jpeg_reg.h +0 -0
similarity index 100%
rename from drivers/media/platform/mtk-jpeg/mtk_jpeg_reg.h
rename to drivers/media/platform/mtk-jpeg/mtk_jpeg_dec_reg.h
Signature · 1 line
2.18.0
author
#4 lore raw

PATCH v4 4/5 media: platform: Fix v4l2-compliance test bug

In reply to Xia Jiang

Improve subscribe event handling: let v4l2_ctrl_subscribe_event() do the job for other types except V4L2_EVENT_SOURCE_CHANGE.
Add checking created buffer size follow in mtk_jpeg_queue_setup().

---
v4: new add patch for v4l2-compliance test bug fix

---
 drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +9 -2
index 5f0990fce8ef..abc506a552c1 100644
@@ -446,9 +446,9 @@ static int mtk_jpeg_subscribe_event(struct v4l2_fh *fh,
 	switch (sub->type) {
 	case V4L2_EVENT_SOURCE_CHANGE:
 		return v4l2_src_change_event_subscribe(fh, sub);
-	default:
-		return -EINVAL;
 	}
+
+	return v4l2_ctrl_subscribe_event(fh, sub);
 }
 
 static int mtk_jpeg_g_selection(struct file *file, void *priv,
@@ -571,6 +571,13 @@ static int mtk_jpeg_queue_setup(struct vb2_queue *q,
 	if (!q_data)
 		return -EINVAL;
 
+	if (*num_planes) {
+		for (i = 0; i < *num_planes; i++)
+			if (sizes[i] < q_data->sizeimage[i])
+				return -EINVAL;
+		return 0;
+	}
+
 	*num_planes = q_data->fmt->colplanes;
 	for (i = 0; i < q_data->fmt->colplanes; i++) {
 		sizes[i] = q_data->sizeimage[i];
Signature · 1 line
2.18.0
author
#5 lore raw

PATCH v4 5/5 media: platform: Add jpeg dec/enc feature

In reply to Xia Jiang

Add mtk jpeg encode v4l2 driver based on jpeg decode, because that jpeg decode and encode have great similarities with function operation.

---
v4: split mtk_jpeg_try_fmt_mplane() to two functions, one for encoder,
one for decoder.
split mtk_jpeg_set_default_params() to two functions, one for encoder, one for decoder.
add cropping support for encoder in g/s_selection ioctls. change exif mode support by using V4L2_JPEG_ACTIVE_MARKER_APP1. change MTK_JPEG_MAX_WIDTH/MTK_JPEG_MAX_HEIGH from 8192 to 65535 by specification.
move width shifting operation behind aligning operation in mtk_jpeg_try_enc_fmt_mplane() for bug fix.
fix user abuseing data_offset issue for DMABUF in mtk_jpeg_set_enc_src().
fix kbuild warings: change MTK_JPEG_MIN_HEIGHT/MTK_JPEG_MAX_HEIGHT
and MTK_JPEG_MIN_WIDTH/MTK_JPEG_MAX_WIDTH from 'int' type to 'unsigned int' type. fix msleadingly indented of 'else'.

v3: delete Change-Id.
only test once handler->error after the last v4l2_ctrl_new_std(). seperate changes of v4l2-ctrls.c and v4l2-controls.h to new patch.

v2: fix compliance test fail, check created buffer size in driver.

---
 drivers/media/platform/mtk-jpeg/Makefile      |   5 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 731 +++++++++++++++---
 .../media/platform/mtk-jpeg/mtk_jpeg_core.h   | 123 ++-
 .../media/platform/mtk-jpeg/mtk_jpeg_dec_hw.h |   7 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c | 175 +++++
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h |  60 ++
 .../platform/mtk-jpeg/mtk_jpeg_enc_reg.h      |  49 ++
 7 files changed, 1004 insertions(+), 146 deletions(-)
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h
drivers/media/platform/mtk-jpeg/Makefile +4 -1
index 48516dcf96e6..76c33aad0f3f 100644
@@ -1,3 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0-only
-mtk_jpeg-objs := mtk_jpeg_core.o mtk_jpeg_dec_hw.o mtk_jpeg_dec_parse.o
+mtk_jpeg-objs := mtk_jpeg_core.o \
+		 mtk_jpeg_dec_hw.o \
+		 mtk_jpeg_dec_parse.o \
+		 mtk_jpeg_enc_hw.o
 obj-$(CONFIG_VIDEO_MEDIATEK_JPEG) += mtk_jpeg.o
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +603 -128

1,063 lines not on this page · raw message

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +109 -14

181 lines not on this page · raw message

drivers/media/platform/mtk-jpeg/mtk_jpeg_dec_hw.h +4 -3
index a37be1a48415..39d743fb572f 100644
@@ -3,10 +3,11 @@
  * Copyright (c) 2016 MediaTek Inc.
  * Author: Ming Hsiu Tsai <[email protected]>
  *         Rick Chang <[email protected]>
+ *         Xia Jiang <[email protected]>
  */
 
-#ifndef _MTK_JPEG_HW_H
-#define _MTK_JPEG_HW_H
+#ifndef _MTK_JPEG_DEC_HW_H
+#define _MTK_JPEG_DEC_HW_H
 
 #include <media/videobuf2-core.h>
 
@@ -80,4 +81,4 @@ void mtk_jpeg_dec_set_config(void __iomem *base,
 void mtk_jpeg_dec_reset(void __iomem *dec_reg_base);
 void mtk_jpeg_dec_start(void __iomem *dec_reg_base);
 
-#endif /* _MTK_JPEG_HW_H */
+#endif /* _MTK_JPEG_DEC_HW_H */
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +175 -0

175 lines not on this page · raw message

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h new file +60 -0

60 lines not on this page · raw message

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h new file +49 -0

49 lines not on this page · raw message

Signature · 1 line
2.18.0
#6 lore raw

Re: [PATCH v4 5/5] media: platform: Add jpeg dec/enc feature

Hi Xia,

Some comments about the selection code:

Quoted 40 lines · Xia Jiang

Add mtk jpeg encode v4l2 driver based on jpeg decode, because that jpeg decode and encode have great similarities with function operation.

---
v4: split mtk_jpeg_try_fmt_mplane() to two functions, one for encoder,
one for decoder.
split mtk_jpeg_set_default_params() to two functions, one for encoder, one for decoder.
add cropping support for encoder in g/s_selection ioctls. change exif mode support by using V4L2_JPEG_ACTIVE_MARKER_APP1. change MTK_JPEG_MAX_WIDTH/MTK_JPEG_MAX_HEIGH from 8192 to 65535 by specification.
move width shifting operation behind aligning operation in mtk_jpeg_try_enc_fmt_mplane() for bug fix.
fix user abuseing data_offset issue for DMABUF in mtk_jpeg_set_enc_src().
fix kbuild warings: change MTK_JPEG_MIN_HEIGHT/MTK_JPEG_MAX_HEIGHT
and MTK_JPEG_MIN_WIDTH/MTK_JPEG_MAX_WIDTH from 'int' type to 'unsigned int' type. fix msleadingly indented of 'else'.

v3: delete Change-Id.
only test once handler->error after the last v4l2_ctrl_new_std(). seperate changes of v4l2-ctrls.c and v4l2-controls.h to new patch.

v2: fix compliance test fail, check created buffer size in driver.

---
 drivers/media/platform/mtk-jpeg/Makefile      |   5 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 731 +++++++++++++++---
 .../media/platform/mtk-jpeg/mtk_jpeg_core.h   | 123 ++-
 .../media/platform/mtk-jpeg/mtk_jpeg_dec_hw.h |   7 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c | 175 +++++
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h |  60 ++
 .../platform/mtk-jpeg/mtk_jpeg_enc_reg.h      |  49 ++
 7 files changed, 1004 insertions(+), 146 deletions(-)
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h

<snip>

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -1
@@ -455,11 +679,19 @@ static int mtk_jpeg_g_selection(struct file *file, void *priv,
 				struct v4l2_selection *s)
 {
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
 
-	if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+	if (jpeg->mode == MTK_JPEG_ENC && s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_g_selection(struct file *file, void *priv, · line 694

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +7 -0
+		return -EINVAL;
+
+	if (jpeg->mode == MTK_JPEG_DEC &&
+	    s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
 		return -EINVAL;
 
 	switch (s->target) {
+	case V4L2_SEL_TGT_CROP:
+	case V4L2_SEL_TGT_CROP_BOUNDS:
+	case V4L2_SEL_TGT_CROP_DEFAULT:

This is wrong...

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -0
 	case V4L2_SEL_TGT_COMPOSE:
 	case V4L2_SEL_TGT_COMPOSE_DEFAULT:
 		s->r.width = ctx->out_q.w;
@@ -484,11 +716,17 @@ static int mtk_jpeg_s_selection(struct file *file, void *priv,
 				struct v4l2_selection *s)
 {
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
 

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_s_selection(struct file *file, void *priv, · line 729

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -1
-	if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+	if (jpeg->mode == MTK_JPEG_ENC && s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)
+		return -EINVAL;
+
+	if (jpeg->mode == MTK_JPEG_DEC &&
+	    s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
 		return -EINVAL;
 
 	switch (s->target) {
+	case V4L2_SEL_TGT_CROP:

...and so is this.

The decoder only supports COMPOSE, the encoder only supports CROP.

This signals support for both cropping and composition for both encoder and decoder, and that's wrong. You can see this in the compliance output as well:
it says that both cropping and composition are 'OK', meaning that both features are implemented.

It also claims that the decoder supports scaling. Is that correct? Is there a scaler in the JPEG decoder? Usually codecs do not have a scaler.

Regards,

Hans

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +0 -0
 	case V4L2_SEL_TGT_COMPOSE:
 		s->r.left = 0;
 		s->r.top = 0;
@@ -658,10 +896,92 @@ static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx,
 		 param->dec_w, param->dec_h);
 }
author
#7 lore raw

Re: [PATCH v4 5/5] media: platform: Add jpeg dec/enc feature

Quoted 98 lines · Hans Verkuil

Hi Xia,

Some comments about the selection code:

Quoted 40 lines · Xia Jiang

Add mtk jpeg encode v4l2 driver based on jpeg decode, because that jpeg decode and encode have great similarities with function operation.

---
v4: split mtk_jpeg_try_fmt_mplane() to two functions, one for encoder,
one for decoder.
split mtk_jpeg_set_default_params() to two functions, one for encoder, one for decoder.
add cropping support for encoder in g/s_selection ioctls. change exif mode support by using V4L2_JPEG_ACTIVE_MARKER_APP1. change MTK_JPEG_MAX_WIDTH/MTK_JPEG_MAX_HEIGH from 8192 to 65535 by specification.
move width shifting operation behind aligning operation in mtk_jpeg_try_enc_fmt_mplane() for bug fix.
fix user abuseing data_offset issue for DMABUF in mtk_jpeg_set_enc_src().
fix kbuild warings: change MTK_JPEG_MIN_HEIGHT/MTK_JPEG_MAX_HEIGHT
and MTK_JPEG_MIN_WIDTH/MTK_JPEG_MAX_WIDTH from 'int' type to 'unsigned int' type. fix msleadingly indented of 'else'.

v3: delete Change-Id.
only test once handler->error after the last v4l2_ctrl_new_std(). seperate changes of v4l2-ctrls.c and v4l2-controls.h to new patch.

v2: fix compliance test fail, check created buffer size in driver.

---
 drivers/media/platform/mtk-jpeg/Makefile      |   5 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 731 +++++++++++++++---
 .../media/platform/mtk-jpeg/mtk_jpeg_core.h   | 123 ++-
 .../media/platform/mtk-jpeg/mtk_jpeg_dec_hw.h |   7 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c | 175 +++++
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h |  60 ++
 .../platform/mtk-jpeg/mtk_jpeg_enc_reg.h      |  49 ++
 7 files changed, 1004 insertions(+), 146 deletions(-)
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h

<snip>

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -1
@@ -455,11 +679,19 @@ static int mtk_jpeg_g_selection(struct file *file, void *priv,
 				struct v4l2_selection *s)
 {
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
 
-	if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+	if (jpeg->mode == MTK_JPEG_ENC && s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_g_selection(struct file *file, void *priv, · line 694

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +7 -0
+		return -EINVAL;
+
+	if (jpeg->mode == MTK_JPEG_DEC &&
+	    s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
 		return -EINVAL;
 
 	switch (s->target) {
+	case V4L2_SEL_TGT_CROP:
+	case V4L2_SEL_TGT_CROP_BOUNDS:
+	case V4L2_SEL_TGT_CROP_DEFAULT:

This is wrong...

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -0
 	case V4L2_SEL_TGT_COMPOSE:
 	case V4L2_SEL_TGT_COMPOSE_DEFAULT:
 		s->r.width = ctx->out_q.w;
@@ -484,11 +716,17 @@ static int mtk_jpeg_s_selection(struct file *file, void *priv,
 				struct v4l2_selection *s)
 {
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
 

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_s_selection(struct file *file, void *priv, · line 729

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -1
-	if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+	if (jpeg->mode == MTK_JPEG_ENC && s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)
+		return -EINVAL;
+
+	if (jpeg->mode == MTK_JPEG_DEC &&
+	    s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
 		return -EINVAL;
 
 	switch (s->target) {
+	case V4L2_SEL_TGT_CROP:

...and so is this.

The decoder only supports COMPOSE, the encoder only supports CROP.

This signals support for both cropping and composition for both encoder and decoder, and that's wrong. You can see this in the compliance output as well:
it says that both cropping and composition are 'OK', meaning that both features are implemented.

Quoted 6 lines

It also claims that the decoder supports scaling. Is that correct? Is there a scaler in the JPEG decoder? Usually codecs do not have a scaler.

Regards,

Hans

Dear Hans,
Crop only for encoder and compose only for decoder in the latest vesion.

Best Regards,
Xia Jiang

Quoted 9 lines
Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +0 -0
 	case V4L2_SEL_TGT_COMPOSE:
 		s->r.left = 0;
 		s->r.top = 0;
@@ -658,10 +896,92 @@ static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx,
 		 param->dec_w, param->dec_h);
 }
#8 lore raw

Re: [PATCH v4 5/5] media: platform: Add jpeg dec/enc feature

In reply to Xia Jiang

Hi Xia,

Quoted 27 lines · Xia Jiang

Add mtk jpeg encode v4l2 driver based on jpeg decode, because that jpeg decode and encode have great similarities with function operation.

---
v4: split mtk_jpeg_try_fmt_mplane() to two functions, one for encoder,
one for decoder.
split mtk_jpeg_set_default_params() to two functions, one for encoder, one for decoder.
add cropping support for encoder in g/s_selection ioctls. change exif mode support by using V4L2_JPEG_ACTIVE_MARKER_APP1. change MTK_JPEG_MAX_WIDTH/MTK_JPEG_MAX_HEIGH from 8192 to 65535 by specification.
move width shifting operation behind aligning operation in mtk_jpeg_try_enc_fmt_mplane() for bug fix.
fix user abuseing data_offset issue for DMABUF in mtk_jpeg_set_enc_src().
fix kbuild warings: change MTK_JPEG_MIN_HEIGHT/MTK_JPEG_MAX_HEIGHT
and MTK_JPEG_MIN_WIDTH/MTK_JPEG_MAX_WIDTH from 'int' type to 'unsigned int' type. fix msleadingly indented of 'else'.

v3: delete Change-Id.
only test once handler->error after the last v4l2_ctrl_new_std(). seperate changes of v4l2-ctrls.c and v4l2-controls.h to new patch.

v2: fix compliance test fail, check created buffer size in driver.

Quoted 13 lines
---
 drivers/media/platform/mtk-jpeg/Makefile      |   5 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 731 +++++++++++++++---
 .../media/platform/mtk-jpeg/mtk_jpeg_core.h   | 123 ++-
 .../media/platform/mtk-jpeg/mtk_jpeg_dec_hw.h |   7 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c | 175 +++++
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h |  60 ++
 .../platform/mtk-jpeg/mtk_jpeg_enc_reg.h      |  49 ++
 7 files changed, 1004 insertions(+), 146 deletions(-)
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h

First of all, thanks for the patch!

Please check my comments below.

My general feeling about this code is that the encoder hardware block is completely orthogonal from the decoder block and there is very little code reuse from the original decoder driver.

Moreover, a lot of existing code now needs if (decoder) { ... } else {... } segments, which complicates the code.

Would it perhaps make sense to instead create a separate mtk-jpeg-enc driver?

It would also give us a fresh start in terms of code quality, as the existing mtk-jpeg driver has a lot of quality issues unfortunately. (Some of my comments to this patch actually relate to the issues with the original code, not introduced by this patch, but we need to fix them if changing this driver already.)

Quoted 86 lines
drivers/media/platform/mtk-jpeg/Makefile +4 -1
index 48516dcf96e6..76c33aad0f3f 100644
@@ -1,3 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0-only
-mtk_jpeg-objs := mtk_jpeg_core.o mtk_jpeg_dec_hw.o mtk_jpeg_dec_parse.o
+mtk_jpeg-objs := mtk_jpeg_core.o \
+		 mtk_jpeg_dec_hw.o \
+		 mtk_jpeg_dec_parse.o \
+		 mtk_jpeg_enc_hw.o
 obj-$(CONFIG_VIDEO_MEDIATEK_JPEG) += mtk_jpeg.o
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +40 -1
index abc506a552c1..087b62bb1d5c 100644
@@ -3,6 +3,7 @@
  * Copyright (c) 2016 MediaTek Inc.
  * Author: Ming Hsiu Tsai <[email protected]>
  *         Rick Chang <[email protected]>
+ *         Xia Jiang <[email protected]>
  */
 
 #include <linux/clk.h>
@@ -23,6 +24,7 @@
 #include <media/videobuf2-dma-contig.h>
 #include <soc/mediatek/smi.h>
 
+#include "mtk_jpeg_enc_hw.h"
 #include "mtk_jpeg_dec_hw.h"
 #include "mtk_jpeg_core.h"
 #include "mtk_jpeg_dec_parse.h"
@@ -31,7 +33,8 @@ static struct mtk_jpeg_fmt mtk_jpeg_formats[] = {
 	{
 		.fourcc		= V4L2_PIX_FMT_JPEG,
 		.colplanes	= 1,
-		.flags		= MTK_JPEG_FMT_FLAG_DEC_OUTPUT,
+		.flags		= MTK_JPEG_FMT_FLAG_DEC_OUTPUT |
+				  MTK_JPEG_FMT_FLAG_ENC_CAPTURE,
 	},
 	{
 		.fourcc		= V4L2_PIX_FMT_YUV420M,
@@ -51,6 +54,42 @@ static struct mtk_jpeg_fmt mtk_jpeg_formats[] = {
 		.v_align	= 3,
 		.flags		= MTK_JPEG_FMT_FLAG_DEC_CAPTURE,
 	},
+	{
+		.fourcc		= V4L2_PIX_FMT_NV12M,
+		.h_sample	= {4, 2, 2},
+		.v_sample	= {4, 2, 2},
+		.colplanes	= 2,
+		.h_align	= 4,
+		.v_align	= 4,
+		.flags		= MTK_JPEG_FMT_FLAG_ENC_OUTPUT,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_NV21M,
+		.h_sample	= {4, 2, 2},
+		.v_sample	= {4, 2, 2},
+		.colplanes	= 2,
+		.h_align	= 4,
+		.v_align	= 4,
+		.flags		= MTK_JPEG_FMT_FLAG_ENC_OUTPUT,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_YUYV,
+		.h_sample	= {4, 2, 2},
+		.v_sample	= {4, 4, 4},
+		.colplanes	= 1,
+		.h_align	= 4,
+		.v_align	= 3,
+		.flags		= MTK_JPEG_FMT_FLAG_ENC_OUTPUT,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_YVYU,
+		.h_sample	= {4, 2, 2},
+		.v_sample	= {4, 4, 4},
+		.colplanes	= 1,
+		.h_align	= 4,
+		.v_align	= 3,
+		.flags		= MTK_JPEG_FMT_FLAG_ENC_OUTPUT,
+	},
 };
 
 #define MTK_JPEG_NUM_FORMATS ARRAY_SIZE(mtk_jpeg_formats)

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c struct mtk_jpeg_src_buf { · line 110

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -0
@@ -65,11 +104,19 @@ struct mtk_jpeg_src_buf {
 	struct list_head list;
 	int flags;
 	struct mtk_jpeg_dec_param dec_param;
+	struct mtk_jpeg_enc_param enc_param;
 };
 
+#define MTK_MAX_CTRLS_HINT	20

I can see only 3 controls registered by this driver. Moreover, wouldn't it be more readable if the v4l2_ctrl_handler_init() call had "3" directly passed to it? Then the reader can see that there are 3 calls to v4l2_ctrl_new_std() just below it and the meaning behind the value of 3 is clear.

Quoted 40 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +22 -2
+
 static int debug;
 module_param(debug, int, 0644);
 
+static inline struct mtk_jpeg_ctx *ctrl_to_ctx(struct v4l2_ctrl *ctrl)
+{
+	return container_of(ctrl->handler, struct mtk_jpeg_ctx, ctrl_hdl);
+}
+
 static inline struct mtk_jpeg_ctx *mtk_jpeg_fh_to_ctx(struct v4l2_fh *fh)
 {
 	return container_of(fh, struct mtk_jpeg_ctx, fh);
@@ -86,14 +133,74 @@ static int mtk_jpeg_querycap(struct file *file, void *priv,
 {
 	struct mtk_jpeg_dev *jpeg = video_drvdata(file);
 
-	strscpy(cap->driver, MTK_JPEG_NAME " decoder", sizeof(cap->driver));
-	strscpy(cap->card, MTK_JPEG_NAME " decoder", sizeof(cap->card));
+	strscpy(cap->driver, MTK_JPEG_NAME, sizeof(cap->driver));
+	if (jpeg->mode ==  MTK_JPEG_ENC)
+		strscpy(cap->card, MTK_JPEG_NAME " encoder", sizeof(cap->card));
+	else
+		strscpy(cap->card, MTK_JPEG_NAME " decoder", sizeof(cap->card));
 	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
 		 dev_name(jpeg->dev));
 
 	return 0;
 }
 
+static int vidioc_jpeg_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_jpeg_ctx *ctx = ctrl_to_ctx(ctrl);
+	struct jpeg_enc_param *p = &ctx->jpeg_param;
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+	int ret = 0;
+
+	switch (ctrl->id) {
+	case V4L2_CID_JPEG_RESTART_INTERVAL:
+		p->restart_interval = ctrl->val;
+		break;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_querycap(struct file *file, void *priv, · line 167

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+	case V4L2_CID_JPEG_COMPRESSION_QUALITY:
+		p->enc_quality = ctrl->val;
+		break;
+	case V4L2_CID_JPEG_ACTIVE_MARKER:
+		p->enable_exif = ctrl->val & V4L2_JPEG_ACTIVE_MARKER_APP1;
+		break;
+	}
+
+	v4l2_dbg(2, debug, &jpeg->v4l2_dev, "%s val = %d",
+		 v4l2_ctrl_get_name(ctrl->id), ctrl->val);

Generic V4L2 logs can already dump all V4L2 IOCTL calls, so perhaps we could remove this?

Quoted 61 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +44 -0
+
+	return ret;
+}
+
+static const struct v4l2_ctrl_ops mtk_jpeg_ctrl_ops = {
+	.s_ctrl = vidioc_jpeg_s_ctrl,
+};
+
+int mtk_jpeg_ctrls_setup(struct mtk_jpeg_ctx *ctx)
+{
+	const struct v4l2_ctrl_ops *ops = &mtk_jpeg_ctrl_ops;
+	struct v4l2_ctrl_handler *handler = &ctx->ctrl_hdl;
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+
+	v4l2_ctrl_handler_init(handler, MTK_MAX_CTRLS_HINT);
+
+	if (jpeg->mode == MTK_JPEG_ENC) {
+		v4l2_ctrl_new_std(handler, ops, V4L2_CID_JPEG_RESTART_INTERVAL,
+				  0, 100, 1, 0);
+		v4l2_ctrl_new_std(handler, ops,
+				  V4L2_CID_JPEG_COMPRESSION_QUALITY, 48, 100, 1,
+				  90);
+		v4l2_ctrl_new_std(handler, ops, V4L2_CID_JPEG_ACTIVE_MARKER, 0,
+				  V4L2_JPEG_ACTIVE_MARKER_APP1, 0, 0);
+
+		if (handler->error) {
+			v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
+			return handler->error;
+		}
+	}
+
+	v4l2_ctrl_handler_setup(&ctx->ctrl_hdl);
+
+	return 0;
+}
+
 static int mtk_jpeg_enum_fmt(struct mtk_jpeg_fmt *mtk_jpeg_formats, int n,
 			     struct v4l2_fmtdesc *f, u32 type)
 {
@@ -118,23 +225,29 @@ static int mtk_jpeg_enum_fmt(struct mtk_jpeg_fmt *mtk_jpeg_formats, int n,
 static int mtk_jpeg_enum_fmt_vid_cap(struct file *file, void *priv,
 				     struct v4l2_fmtdesc *f)
 {
+	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+
 	return mtk_jpeg_enum_fmt(mtk_jpeg_formats, MTK_JPEG_NUM_FORMATS, f,
+				 ctx->jpeg->mode == MTK_JPEG_ENC ?
+				 MTK_JPEG_FMT_FLAG_ENC_CAPTURE :
 				 MTK_JPEG_FMT_FLAG_DEC_CAPTURE);
 }
 
 static int mtk_jpeg_enum_fmt_vid_out(struct file *file, void *priv,
 				     struct v4l2_fmtdesc *f)
 {
+	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+
 	return mtk_jpeg_enum_fmt(mtk_jpeg_formats, MTK_JPEG_NUM_FORMATS, f,
+				 ctx->jpeg->mode == MTK_JPEG_ENC ?
+				 MTK_JPEG_FMT_FLAG_ENC_OUTPUT :
 				 MTK_JPEG_FMT_FLAG_DEC_OUTPUT);
 }

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_enum_fmt(struct mtk_jpeg_fmt *mtk_jpeg_formats, int n, · line 250

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -5
 
-static struct mtk_jpeg_q_data *mtk_jpeg_get_q_data(struct mtk_jpeg_ctx *ctx,
-						   enum v4l2_buf_type type)
+static struct mtk_jpeg_q_data *
+mtk_jpeg_get_q_data(struct mtk_jpeg_ctx *ctx, enum v4l2_buf_type type)
 {
-	if (V4L2_TYPE_IS_OUTPUT(type))
-		return &ctx->out_q;
-	return &ctx->cap_q;
+	return V4L2_TYPE_IS_OUTPUT(type) ? &ctx->out_q : &ctx->cap_q;

Is this change necessary?

Quoted 27 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +8 -5
 }
 
 static struct mtk_jpeg_fmt *mtk_jpeg_find_format(struct mtk_jpeg_ctx *ctx,
@@ -143,9 +256,14 @@ static struct mtk_jpeg_fmt *mtk_jpeg_find_format(struct mtk_jpeg_ctx *ctx,
 {
 	unsigned int k, fmt_flag;
 
-	fmt_flag = (fmt_type == MTK_JPEG_FMT_TYPE_OUTPUT) ?
-		   MTK_JPEG_FMT_FLAG_DEC_OUTPUT :
-		   MTK_JPEG_FMT_FLAG_DEC_CAPTURE;
+	if (ctx->jpeg->mode ==  MTK_JPEG_ENC)
+		fmt_flag = (fmt_type == MTK_JPEG_FMT_TYPE_OUTPUT) ?
+			   MTK_JPEG_FMT_FLAG_ENC_OUTPUT :
+			   MTK_JPEG_FMT_FLAG_ENC_CAPTURE;
+	else
+		fmt_flag = (fmt_type == MTK_JPEG_FMT_TYPE_OUTPUT) ?
+			   MTK_JPEG_FMT_FLAG_DEC_OUTPUT :
+			   MTK_JPEG_FMT_FLAG_DEC_CAPTURE;
 
 	for (k = 0; k < MTK_JPEG_NUM_FORMATS; k++) {
 		struct mtk_jpeg_fmt *fmt = &mtk_jpeg_formats[k];
@@ -196,9 +314,99 @@ static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx,
 	}
 }
 
-static int mtk_jpeg_try_fmt_mplane(struct v4l2_format *f,
-				   struct mtk_jpeg_fmt *fmt,

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 325

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +9 -1
-				   struct mtk_jpeg_ctx *ctx, int q_type)
+static int mtk_jpeg_try_enc_fmt_mplane(struct v4l2_format *f,
+				       struct mtk_jpeg_fmt *fmt,
+				       struct mtk_jpeg_ctx *ctx, int q_type)
+{
+	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+	int i, align_w, align_h;
+
+	memset(pix_mp->reserved, 0, sizeof(pix_mp->reserved));

The core already takes care of zeroing this.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 336

Quoted 11 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +11 -0
+	pix_mp->field = V4L2_FIELD_NONE;
+
+	if (ctx->state != MTK_JPEG_INIT) {
+		mtk_jpeg_adjust_fmt_mplane(ctx, f);
+		goto end;
+	}
+
+	pix_mp->num_planes = fmt->colplanes;
+	pix_mp->pixelformat = fmt->fourcc;
+
+	if (q_type == MTK_JPEG_FMT_TYPE_OUTPUT) {

There are separate ops for out_mplane and cap_mplane. Wouldn't it make more sense to have them handled in separate functions?

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -0
+		pix_mp->height = clamp(pix_mp->height, MTK_JPEG_MIN_HEIGHT,
+				       MTK_JPEG_MAX_HEIGHT);
+		pix_mp->width = clamp(pix_mp->width, MTK_JPEG_MIN_WIDTH,
+				      MTK_JPEG_MAX_WIDTH);

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 350

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+		align_w = pix_mp->width;
+		align_h = pix_mp->height;
+		align_w = round_up(align_w, 2);
+		if (pix_mp->num_planes == 1U) {
+			mtk_jpeg_bound_align_image(&align_w,
+						   MTK_JPEG_MIN_WIDTH,
+						   MTK_JPEG_MAX_WIDTH, 5,
+						   &align_h,
+						   MTK_JPEG_MIN_HEIGHT,
+						   MTK_JPEG_MAX_HEIGHT, 3);

I'd suggest removing the mtk_jpeg_bound_align_image(), as we don't need the complex computation that's happening there. All we need is:

	pix_mp->height = clamp(round_up(pix_mp->height, 8), MTK_JPEG_MIN_HEIGHT,
			       MTK_JPEG_MAX_HEIGHT);
	pix_mp->width = clamp(round_up(pix_mp->width, 32), MTK_JPEG_MIN_WIDTH,
			      MTK_JPEG_MAX_WIDTH);

Note that contrary to the existing code, the aligned width and height _must_ be returned back to the application and not only hidden inside bytesperline and sizeimage.

Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -0
+			align_w = align_w << 1;
+			pix_mp->plane_fmt[0].bytesperline = align_w;
+			pix_mp->plane_fmt[0].sizeimage = align_w * align_h;
+		} else if (pix_mp->num_planes == 2U) {
+			mtk_jpeg_bound_align_image(&align_w,

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 365

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+						   MTK_JPEG_MIN_WIDTH,
+						   MTK_JPEG_MAX_WIDTH, 4,
+						   &align_h,
+						   MTK_JPEG_MIN_HEIGHT,
+						   MTK_JPEG_MAX_HEIGHT, 4);
+			pix_mp->plane_fmt[0].bytesperline = align_w;
+			pix_mp->plane_fmt[0].sizeimage = align_w * align_h;
+			pix_mp->plane_fmt[1].bytesperline = align_w;
+			pix_mp->plane_fmt[1].sizeimage =
+				(align_w * align_h) / 2;

Could we parametrize these calculations from the format array and use a for loop over the planes instead?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 369

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -0
+		} else {
+			v4l2_err(&jpeg->v4l2_dev,
+				 "Unsupport num planes = %d\n",
+				 pix_mp->num_planes);

Isn't this impossible?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 376

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +7 -0
+		}
+	} else {
+		mtk_jpeg_bound_align_image(&pix_mp->width, MTK_JPEG_MIN_WIDTH,
+					   MTK_JPEG_MAX_WIDTH, 0,
+					   &pix_mp->height,
+					   MTK_JPEG_MIN_HEIGHT,
+					   MTK_JPEG_MAX_HEIGHT, 0);

Same here, could we just use clamp() directly?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 378

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -0
+
+		if (fmt->fourcc == V4L2_PIX_FMT_JPEG) {

Can we have any other format on this queue?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 382

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -0
+			pix_mp->plane_fmt[0].bytesperline = 0;
+			pix_mp->plane_fmt[0].sizeimage =
+				mtk_jpeg_align(pix_mp->plane_fmt[0].sizeimage,
+					       128);

What's mtk_jpeg_align()? Why couldn't we just use round_up() directly?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 385

Quoted 3 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -0
+			if (pix_mp->plane_fmt[0].sizeimage == 0)
+				pix_mp->plane_fmt[0].sizeimage =
+					MTK_JPEG_DEFAULT_SIZEIMAGE;

I think there is some reasonable upper bound of the compressed image size given the resolution. Could we use that here instead?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 392

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +7 -0
+		}
+	}
+
+	for (i = 0; i < fmt->colplanes; i++) {
+		struct v4l2_plane_pix_format *pfmt =
+				&pix_mp->plane_fmt[i];
+		memset(pfmt->reserved, 0, sizeof(pfmt->reserved));

The core already zeroes this.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 403

Quoted 11 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +11 -0
+	}
+end:
+	v4l2_dbg(2, debug, &jpeg->v4l2_dev, "wxh:%ux%u\n",
+		 pix_mp->width, pix_mp->height);
+	for (i = 0; i < pix_mp->num_planes; i++) {
+		v4l2_dbg(2, debug, &jpeg->v4l2_dev,
+			 "plane[%d] bpl=%u, size=%u\n",
+			 i,
+			 pix_mp->plane_fmt[i].bytesperline,
+			 pix_mp->plane_fmt[i].sizeimage);
+	}

Generic V4L2 debug logs already print this.

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
+	return 0;
+}
+
+static int mtk_jpeg_try_dec_fmt_mplane(struct v4l2_format *f,
+				       struct mtk_jpeg_fmt *fmt,
+				       struct mtk_jpeg_ctx *ctx, int q_type)
 {
 	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
 	struct mtk_jpeg_dev *jpeg = ctx->jpeg;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_try_fmt_mplane(struct v4l2_format *f, · line 447

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -0
@@ -233,6 +441,10 @@ static int mtk_jpeg_try_fmt_mplane(struct v4l2_format *f,
 	}
 
 	/* type is MTK_JPEG_FMT_TYPE_CAPTURE */
+	pix_mp->height = clamp(pix_mp->height, MTK_JPEG_MIN_HEIGHT,
+			       MTK_JPEG_MAX_HEIGHT);
+	pix_mp->width = clamp(pix_mp->width, MTK_JPEG_MIN_WIDTH,
+			      MTK_JPEG_MAX_WIDTH);

Why is this needed? Doesn't the call below already include these clamps?

(That said, it would be cleaner if mtk_jpeg_bound_align_image() was removed and replaced with clamp() calls directly, because of the confusing behavior of v4l_bound_align_image().

Quoted 12 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -0
 	mtk_jpeg_bound_align_image(&pix_mp->width, MTK_JPEG_MIN_WIDTH,
 				   MTK_JPEG_MAX_WIDTH, fmt->h_align,
 				   &pix_mp->height, MTK_JPEG_MIN_HEIGHT,
@@ -316,6 +528,7 @@ static int mtk_jpeg_try_fmt_vid_cap_mplane(struct file *file, void *priv,
 {
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
 	struct mtk_jpeg_fmt *fmt;
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
 
 	fmt = mtk_jpeg_find_format(ctx, f->fmt.pix_mp.pixelformat,
 				   MTK_JPEG_FMT_TYPE_CAPTURE);

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_try_fmt_vid_cap_mplane(struct file *file, void *priv, · line 550

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -1
@@ -329,7 +542,12 @@ static int mtk_jpeg_try_fmt_vid_cap_mplane(struct file *file, void *priv,
 		 (fmt->fourcc >> 16 & 0xff),
 		 (fmt->fourcc >> 24 & 0xff));
 
-	return mtk_jpeg_try_fmt_mplane(f, fmt, ctx, MTK_JPEG_FMT_TYPE_CAPTURE);
+
+	return jpeg->mode == MTK_JPEG_ENC ?
+	       mtk_jpeg_try_enc_fmt_mplane(f, fmt, ctx,
+					   MTK_JPEG_FMT_TYPE_CAPTURE) :
+	       mtk_jpeg_try_dec_fmt_mplane(f, fmt, ctx,
+					   MTK_JPEG_FMT_TYPE_CAPTURE);

The ternary operator isn't a universal replacement for if/else. It should be limited to simple expressions only. Function calls don't count as simple expressions.

Quoted 12 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -0
 }
 
 static int mtk_jpeg_try_fmt_vid_out_mplane(struct file *file, void *priv,
@@ -337,6 +555,7 @@ static int mtk_jpeg_try_fmt_vid_out_mplane(struct file *file, void *priv,
 {
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
 	struct mtk_jpeg_fmt *fmt;
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
 
 	fmt = mtk_jpeg_find_format(ctx, f->fmt.pix_mp.pixelformat,
 				   MTK_JPEG_FMT_TYPE_OUTPUT);

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_try_fmt_vid_out_mplane(struct file *file, void *priv, · line 576

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -1
@@ -350,7 +569,12 @@ static int mtk_jpeg_try_fmt_vid_out_mplane(struct file *file, void *priv,
 		 (fmt->fourcc >> 16 & 0xff),
 		 (fmt->fourcc >> 24 & 0xff));
 
-	return mtk_jpeg_try_fmt_mplane(f, fmt, ctx, MTK_JPEG_FMT_TYPE_OUTPUT);
+	return jpeg->mode == MTK_JPEG_ENC ?
+	       mtk_jpeg_try_enc_fmt_mplane(f, fmt, ctx,
+					   MTK_JPEG_FMT_TYPE_OUTPUT) :
+	       mtk_jpeg_try_dec_fmt_mplane(f, fmt, ctx,
+					   MTK_JPEG_FMT_TYPE_OUTPUT);

Ditto.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_try_fmt_vid_out_mplane(struct file *file, void *priv, · line 577

Quoted 1 line
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -0
+

Unnecessary blank line.

Quoted 61 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +30 -2
 }
 
 static int mtk_jpeg_s_fmt_mplane(struct mtk_jpeg_ctx *ctx,
@@ -455,11 +679,19 @@ static int mtk_jpeg_g_selection(struct file *file, void *priv,
 				struct v4l2_selection *s)
 {
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
 
-	if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+	if (jpeg->mode == MTK_JPEG_ENC && s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)
+		return -EINVAL;
+
+	if (jpeg->mode == MTK_JPEG_DEC &&
+	    s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
 		return -EINVAL;
 
 	switch (s->target) {
+	case V4L2_SEL_TGT_CROP:
+	case V4L2_SEL_TGT_CROP_BOUNDS:
+	case V4L2_SEL_TGT_CROP_DEFAULT:
 	case V4L2_SEL_TGT_COMPOSE:
 	case V4L2_SEL_TGT_COMPOSE_DEFAULT:
 		s->r.width = ctx->out_q.w;
@@ -484,11 +716,17 @@ static int mtk_jpeg_s_selection(struct file *file, void *priv,
 				struct v4l2_selection *s)
 {
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
 
-	if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+	if (jpeg->mode == MTK_JPEG_ENC && s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)
+		return -EINVAL;
+
+	if (jpeg->mode == MTK_JPEG_DEC &&
+	    s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
 		return -EINVAL;
 
 	switch (s->target) {
+	case V4L2_SEL_TGT_CROP:
 	case V4L2_SEL_TGT_COMPOSE:
 		s->r.left = 0;
 		s->r.top = 0;
@@ -658,10 +896,92 @@ static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx,
 		 param->dec_w, param->dec_h);
 }
 
+static void mtk_jpeg_set_param(struct mtk_jpeg_ctx *ctx,
+			       struct mtk_jpeg_enc_param *param)
+{
+	struct mtk_jpeg_q_data *q_data_src = &ctx->out_q;
+	struct jpeg_enc_param *jpeg_params = &ctx->jpeg_param;
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+	u32 width_even;
+	u32 is_420;
+	u32 padding_width;
+	u32 padding_height;
+
+	switch (q_data_src->fmt->fourcc) {
+	case V4L2_PIX_FMT_YUYV:
+		param->enc_format = JPEG_YUV_FORMAT_YUYV;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx, · line 922

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+		break;
+	case V4L2_PIX_FMT_YVYU:
+		param->enc_format = JPEG_YUV_FORMAT_YVYU;
+		break;
+	case V4L2_PIX_FMT_NV12M:
+		param->enc_format = JPEG_YUV_FORMAT_NV12;
+		break;
+	case V4L2_PIX_FMT_NV21M:
+		param->enc_format = JPEG_YUV_FORMAT_NV12;
+		break;

Could we include enc_format in the mtk_jpeg_fmt struct and take it from there, eliminating the need for a switch over all supported formats?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx, · line 926

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -0
+	default:
+		v4l2_err(&jpeg->v4l2_dev, "Unsupport fourcc =%d\n",
+			 q_data_src->fmt->fourcc);
+		break;

Isn't this impossible?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx, · line 934

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +8 -0
+	}
+	param->enc_w = q_data_src->w;
+	param->enc_h = q_data_src->h;
+
+	if (jpeg_params->enc_quality >= 97)
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q97;
+	else if (jpeg_params->enc_quality >= 95)
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q95;

I'm wondering if the application requests 96, it doesn't expect the quality to be _at_least_ 96.

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +8 -0
+	else if (jpeg_params->enc_quality >= 92)
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q92;
+	else if (jpeg_params->enc_quality >= 90)
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q90;
+	else if (jpeg_params->enc_quality >= 87)
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q87;
+	else if (jpeg_params->enc_quality >= 84)
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q84;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx, · line 952

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+	else if (jpeg_params->enc_quality >= 80)
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q80;
+	else if (jpeg_params->enc_quality >= 74)
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q74;
+	else if (jpeg_params->enc_quality >= 64)
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q64;
+	else if (jpeg_params->enc_quality >= 60)
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q60;
+	else
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q48;

We could put this in a sorted array of <quality paramter, hardware value> pairs and use a simple for loop instead.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx, · line 957

Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -0
+
+	param->enable_exif = jpeg_params->enable_exif;
+	param->restart_interval = jpeg_params->restart_interval;
+
+	width_even = ((param->enc_w + 1) >> 1) << 1;

Isn't this equivalent to round_up(param->enc_w, 2)?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx, · line 965

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +8 -0
+	is_420 = (param->enc_format == JPEG_YUV_FORMAT_NV12 ||
+		  param->enc_format == JPEG_YUV_FORMAT_NV12) ? 1 : 0;
+	padding_width = mtk_jpeg_align(param->enc_w, 16);
+	padding_height = mtk_jpeg_align(param->enc_h, is_420 ? 16 : 8);
+	if (!is_420)
+		width_even = width_even << 1;
+
+	param->img_stride = mtk_jpeg_align(width_even, (is_420 ? 16 : 32));

Isn't this the same as bytesperline in the v4l2_pix_fmt_mplane struct?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx, · line 966

Quoted 1 line
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -0
+	param->mem_stride = mtk_jpeg_align(width_even, (is_420 ? 16 : 32));

What's the difference between img_stride and mem_stride?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx, · line 969

Quoted 3 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -0
+	param->total_encdu =
+		((padding_width >> 4) * (padding_height >> (is_420 ? 4 : 3)) *
+		(is_420 ? 6 : 4)) - 1;

The comment above the struct says this is the total number of 8x8 blocks. Why would it depend on whether the format is YUV 4:2:0? Since we should have already aligned the width and height in try_fmt, this should be as simple as (width / 8) * (height / 8).

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx, · line 978

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +9 -0
+
+	v4l2_dbg(0, 2, &jpeg->v4l2_dev, "fmt %d, w,h %d,%d, enable_exif %d,",
+		 "enc_quality %d, restart_interval %d,img_stride %d,",
+		 "mem_stride %d,totalEncDu %d\n",
+		 param->enc_format, param->enc_w, param->enc_h,
+		 param->enable_exif, param->enc_quality,
+		 param->restart_interval, param->img_stride,
+		 param->mem_stride, param->total_encdu);
+}

Do we need this intermediate step, with an intermediate representation of the parameters? Couldn't we just program the registers directly based on the original V4L2 values, as other drivers normally do?

Quoted 22 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -9
+
 static void mtk_jpeg_buf_queue(struct vb2_buffer *vb)
 {
 	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
 	struct mtk_jpeg_dec_param *param;
+	struct mtk_jpeg_enc_param *enc_param;
 	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
 	struct mtk_jpeg_src_buf *jpeg_src_buf;
 	bool header_valid;
@@ -673,29 +993,45 @@ static void mtk_jpeg_buf_queue(struct vb2_buffer *vb)
 		goto end;
 
 	jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(vb);
-	param = &jpeg_src_buf->dec_param;
-	memset(param, 0, sizeof(*param));
-
-	if (jpeg_src_buf->flags & MTK_JPEG_BUF_FLAGS_LAST_FRAME) {
-		v4l2_dbg(1, debug, &jpeg->v4l2_dev, "Got eos\n");
-		goto end;
-	}
-	header_valid = mtk_jpeg_parse(param, (u8 *)vb2_plane_vaddr(vb, 0),
-				      vb2_get_plane_payload(vb, 0));

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_buf_queue(struct vb2_buffer *vb) · line 996

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -9
-	if (!header_valid) {
-		v4l2_err(&jpeg->v4l2_dev, "Header invalid.\n");
-		vb2_buffer_done(vb, VB2_BUF_STATE_ERROR);
-		return;
-	}
-
-	if (ctx->state == MTK_JPEG_INIT) {
-		struct vb2_queue *dst_vq = v4l2_m2m_get_vq(
-			ctx->fh.m2m_ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
+	if (jpeg->mode ==  MTK_JPEG_ENC) {

nit: Double space after ==.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_buf_queue(struct vb2_buffer *vb) · line 1002

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
+		enc_param = &jpeg_src_buf->enc_param;
+		memset(enc_param, 0, sizeof(*enc_param));
+		mtk_jpeg_set_param(ctx, enc_param);
+		if (jpeg_src_buf->flags & MTK_JPEG_BUF_FLAGS_LAST_FRAME) {
+			v4l2_dbg(1, debug, &jpeg->v4l2_dev, "Got eos");
+			goto end;

Shouldn't we still change the state to RUNNING? The last frame still needs to be encoded and the condition for getting back to the INIT state is stopping the streaming.

Quoted 61 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +34 -7
+		}
+		if (ctx->state == MTK_JPEG_INIT)
+			ctx->state = MTK_JPEG_RUNNING;
+	} else {
+		param = &jpeg_src_buf->dec_param;
+		memset(param, 0, sizeof(*param));
+
+		if (jpeg_src_buf->flags & MTK_JPEG_BUF_FLAGS_LAST_FRAME) {
+			v4l2_dbg(1, debug, &jpeg->v4l2_dev, "Got eos\n");
+			goto end;
+		}
+		header_valid = mtk_jpeg_parse(param,
+					      (u8 *)vb2_plane_vaddr(vb, 0),
+					      vb2_get_plane_payload(vb, 0));
+		if (!header_valid) {
+			v4l2_err(&jpeg->v4l2_dev, "Header invalid.\n");
+			vb2_buffer_done(vb, VB2_BUF_STATE_ERROR);
+			return;
+		}
 
-		mtk_jpeg_queue_src_chg_event(ctx);
-		mtk_jpeg_set_queue_data(ctx, param);
-		ctx->state = vb2_is_streaming(dst_vq) ?
-				MTK_JPEG_SOURCE_CHANGE : MTK_JPEG_RUNNING;
+		if (ctx->state == MTK_JPEG_INIT) {
+			struct vb2_queue *dst_vq;
+
+			dst_vq = v4l2_m2m_get_vq
+					(ctx->fh.m2m_ctx,
+					 V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
+			mtk_jpeg_queue_src_chg_event(ctx);
+			mtk_jpeg_set_queue_data(ctx, param);
+			ctx->state = vb2_is_streaming(dst_vq) ?
+					MTK_JPEG_SOURCE_CHANGE :
+					MTK_JPEG_RUNNING;
+		}
 	}
 end:
 	v4l2_m2m_buf_queue(ctx->fh.m2m_ctx, to_vb2_v4l2_buffer(vb));
@@ -738,16 +1074,16 @@ static void mtk_jpeg_stop_streaming(struct vb2_queue *q)
 	 * subsampling. Update capture queue when the stream is off.
 	 */
 	if (ctx->state == MTK_JPEG_SOURCE_CHANGE &&
-	    !V4L2_TYPE_IS_OUTPUT(q->type)) {
+	    !V4L2_TYPE_IS_OUTPUT(q->type) &&
+	    ctx->jpeg->mode == MTK_JPEG_DEC) {
 		struct mtk_jpeg_src_buf *src_buf;
 
 		vb = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
 		src_buf = mtk_jpeg_vb2_to_srcbuf(&vb->vb2_buf);
 		mtk_jpeg_set_queue_data(ctx, &src_buf->dec_param);
 		ctx->state = MTK_JPEG_RUNNING;
-	} else if (V4L2_TYPE_IS_OUTPUT(q->type)) {
+	} else if (V4L2_TYPE_IS_OUTPUT(q->type))
 		ctx->state = MTK_JPEG_INIT;
-	}
 
 	while ((vb = mtk_jpeg_buf_remove(ctx, q->type)))
 		v4l2_m2m_buf_done(vb, VB2_BUF_STATE_ERROR);
@@ -802,6 +1138,34 @@ static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx,
 	return 0;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx, · line 1148

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +8 -0
 }
 
+static void mtk_jpeg_set_enc_dst(struct mtk_jpeg_ctx *ctx,
+				 struct vb2_buffer *dst_buf,
+				 struct mtk_jpeg_enc_bs *bs)
+{
+	struct jpeg_enc_param *p = &ctx->jpeg_param;
+
+	bs->dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0) &
+		      (~JPEG_ENC_DST_ADDR_OFFSET_MASK);

Wouldn't the hardware attempt to access the memory before the buffer in this case? This is a recipe for a disaster (e.g. the hardware overwriting some memory it shouldn't touch).

Could we instead check the buffer address alignment in .buf_prepare and fail if it's not big enough?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx, · line 1150

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -0
+	bs->dma_addr_offset = p->enable_exif ? MTK_JPEG_DEFAULT_EXIF_SIZE : 0;
+	bs->dma_addr_offsetmask = bs->dma_addr & JPEG_ENC_DST_ADDR_OFFSET_MASK;

What is the meaning of this offset mask?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx, · line 1151

Quoted 1 line
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -0
+	bs->size = mtk_jpeg_align(vb2_plane_size(dst_buf, 0), 128);

Wouldn't this let the hardware write beyond the buffer if the original size was not a multiple of 128? Shouldn't we round_down() insted?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx, · line 1164

Quoted 13 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +13 -0
+}
+
+static void mtk_jpeg_set_enc_src(struct mtk_jpeg_ctx *ctx,
+				 struct vb2_buffer *src_buf,
+				 struct mtk_jpeg_enc_fb *fb)
+{
+	int i;
+
+	for (i = 0; i < src_buf->num_planes; i++) {
+		fb->fb_addr[i].dma_addr =
+			vb2_dma_contig_plane_dma_addr(src_buf, i) +
+			src_buf->planes[i].data_offset;
+		fb->fb_addr[i].size = (size_t)src_buf->planes[i].length -

The proper way to access plane length is to call vb2_plane_size().

Quoted 25 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +7 -2
+				      src_buf->planes[i].data_offset;
+	}
+}
+
 static void mtk_jpeg_device_run(void *priv)
 {
 	struct mtk_jpeg_ctx *ctx = priv;
@@ -812,6 +1176,8 @@ static void mtk_jpeg_device_run(void *priv)
 	struct mtk_jpeg_src_buf *jpeg_src_buf;
 	struct mtk_jpeg_bs bs;
 	struct mtk_jpeg_fb fb;
+	struct mtk_jpeg_enc_bs enc_bs;
+	struct mtk_jpeg_enc_fb enc_fb;
 	int i;
 
 	src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
@@ -822,30 +1188,45 @@ static void mtk_jpeg_device_run(void *priv)
 		for (i = 0; i < dst_buf->vb2_buf.num_planes; i++)
 			vb2_set_plane_payload(&dst_buf->vb2_buf, i, 0);
 		buf_state = VB2_BUF_STATE_DONE;
-		goto dec_end;
+		goto device_run_end;
 	}
 
-	if (mtk_jpeg_check_resolution_change(ctx, &jpeg_src_buf->dec_param)) {

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_device_run(void *priv) · line 1198

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -5
-		mtk_jpeg_queue_src_chg_event(ctx);
-		ctx->state = MTK_JPEG_SOURCE_CHANGE;
-		v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
-		return;
-	}
+	if (jpeg->mode == MTK_JPEG_ENC) {
+		mtk_jpeg_set_enc_dst(ctx, &dst_buf->vb2_buf, &enc_bs);
+		mtk_jpeg_set_enc_src(ctx, &src_buf->vb2_buf, &enc_fb);
+
+		spin_lock_irqsave(&jpeg->hw_lock, flags);

Do we actually need this lock? device_run can be only called once and only after the hardware finished processing the current job.

Quoted 26 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +18 -6
+		mtk_jpeg_enc_reset(jpeg->reg_base);
+		mtk_jpeg_enc_set_config(jpeg->reg_base,
+					&jpeg_src_buf->enc_param, &enc_bs,
+					&enc_fb);
+
+		mtk_jpeg_enc_start(jpeg->reg_base);
+	} else {
+		if (mtk_jpeg_check_resolution_change
+			(ctx, &jpeg_src_buf->dec_param)) {
+			mtk_jpeg_queue_src_chg_event(ctx);
+			ctx->state = MTK_JPEG_SOURCE_CHANGE;
+			v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
+			return;
+		}
 
-	mtk_jpeg_set_dec_src(ctx, &src_buf->vb2_buf, &bs);
-	if (mtk_jpeg_set_dec_dst(ctx, &jpeg_src_buf->dec_param, &dst_buf->vb2_buf, &fb))
-		goto dec_end;
+		mtk_jpeg_set_dec_src(ctx, &src_buf->vb2_buf, &bs);
+		if (mtk_jpeg_set_dec_dst(ctx, &jpeg_src_buf->dec_param,
+					 &dst_buf->vb2_buf, &fb))
+			goto device_run_end;
 
-	spin_lock_irqsave(&jpeg->hw_lock, flags);
-	mtk_jpeg_dec_reset(jpeg->dec_reg_base);
-	mtk_jpeg_dec_set_config(jpeg->dec_reg_base,

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_device_run(void *priv) · line 1226

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -2
-				&jpeg_src_buf->dec_param, &bs, &fb);
+		spin_lock_irqsave(&jpeg->hw_lock, flags);
+		mtk_jpeg_dec_reset(jpeg->reg_base);
+		mtk_jpeg_dec_set_config(jpeg->reg_base,
+					&jpeg_src_buf->dec_param, &bs, &fb);
 
-	mtk_jpeg_dec_start(jpeg->dec_reg_base);
+		mtk_jpeg_dec_start(jpeg->reg_base);
+	}
 	spin_unlock_irqrestore(&jpeg->hw_lock, flags);

It is a good practice for spin_lock and spin_unlock to be in the same scope, for readability reasons.

Quoted 29 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +7 -6
 	return;
 
-dec_end:
+device_run_end:
 	v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
 	v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
 	v4l2_m2m_buf_done(src_buf, buf_state);
@@ -905,30 +1286,30 @@ static void mtk_jpeg_clk_on(struct mtk_jpeg_dev *jpeg)
 	ret = mtk_smi_larb_get(jpeg->larb);
 	if (ret)
 		dev_err(jpeg->dev, "mtk_smi_larb_get larbvdec fail %d\n", ret);
-	clk_prepare_enable(jpeg->clk_jdec_smi);
-	clk_prepare_enable(jpeg->clk_jdec);
+	if (jpeg->mode == MTK_JPEG_DEC)
+		clk_prepare_enable(jpeg->clk_jpeg_smi);
+	clk_prepare_enable(jpeg->clk_jpeg);
 }
 
 static void mtk_jpeg_clk_off(struct mtk_jpeg_dev *jpeg)
 {
-	clk_disable_unprepare(jpeg->clk_jdec);
-	clk_disable_unprepare(jpeg->clk_jdec_smi);
+	clk_disable_unprepare(jpeg->clk_jpeg);
+	if (jpeg->mode == MTK_JPEG_DEC)
+		clk_disable_unprepare(jpeg->clk_jpeg_smi);
 	mtk_smi_larb_put(jpeg->larb);
 }
 
-static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv)

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_clk_on(struct mtk_jpeg_dev *jpeg) · line 1309

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -2
+static irqreturn_t mtk_jpeg_irq(int irq, void *priv)
 {
 	struct mtk_jpeg_dev *jpeg = priv;
 	struct mtk_jpeg_ctx *ctx;
 	struct vb2_v4l2_buffer *src_buf, *dst_buf;
 	struct mtk_jpeg_src_buf *jpeg_src_buf;
 	enum vb2_buffer_state buf_state = VB2_BUF_STATE_ERROR;
-	u32	dec_irq_ret;
-	u32 dec_ret;
+	u32	irq_ret;

In the kernel coding style, type is separated from variable name with a single space.

Quoted 99 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +66 -13
+	u32 ret, result_size;
 	int i;
 
-	dec_ret = mtk_jpeg_dec_get_int_status(jpeg->dec_reg_base);
-	dec_irq_ret = mtk_jpeg_dec_enum_result(dec_ret);
 	ctx = v4l2_m2m_get_curr_priv(jpeg->m2m_dev);
 	if (!ctx) {
 		v4l2_err(&jpeg->v4l2_dev, "Context is NULL\n");
@@ -939,39 +1320,107 @@ static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv)
 	dst_buf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
 	jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(&src_buf->vb2_buf);
 
-	if (dec_irq_ret >= MTK_JPEG_DEC_RESULT_UNDERFLOW)
-		mtk_jpeg_dec_reset(jpeg->dec_reg_base);
+	if (jpeg->mode == MTK_JPEG_ENC) {
+		ret = mtk_jpeg_enc_get_int_status(jpeg->reg_base);
+		irq_ret = mtk_jpeg_enc_enum_result(jpeg->reg_base, ret,
+						   &result_size);
 
-	if (dec_irq_ret != MTK_JPEG_DEC_RESULT_EOF_DONE) {
-		dev_err(jpeg->dev, "decode failed\n");
-		goto dec_end;
-	}
+		if (irq_ret >= MTK_JPEG_ENC_RESULT_STALL)
+			mtk_jpeg_enc_reset(jpeg->reg_base);
+
+		if (irq_ret != MTK_JPEG_ENC_RESULT_DONE) {
+			dev_err(jpeg->dev, "encode failed\n");
+			goto irq_end;
+		}
+
+		vb2_set_plane_payload(&dst_buf->vb2_buf, 0,
+				      result_size);
+	} else {
+		ret = mtk_jpeg_dec_get_int_status(jpeg->reg_base);
+		irq_ret = mtk_jpeg_dec_enum_result(ret);
 
-	for (i = 0; i < dst_buf->vb2_buf.num_planes; i++)
-		vb2_set_plane_payload(&dst_buf->vb2_buf, i,
-				      jpeg_src_buf->dec_param.comp_size[i]);
+		if (irq_ret >= MTK_JPEG_DEC_RESULT_UNDERFLOW)
+			mtk_jpeg_dec_reset(jpeg->reg_base);
+
+		if (irq_ret != MTK_JPEG_DEC_RESULT_EOF_DONE) {
+			dev_err(jpeg->dev, "decode failed\n");
+			goto irq_end;
+		}
+
+		for (i = 0; i < dst_buf->vb2_buf.num_planes; i++)
+			vb2_set_plane_payload
+				(&dst_buf->vb2_buf, i,
+				 jpeg_src_buf->dec_param.comp_size[i]);
+	}
 
 	buf_state = VB2_BUF_STATE_DONE;
 
-dec_end:
+irq_end:
 	v4l2_m2m_buf_done(src_buf, buf_state);
 	v4l2_m2m_buf_done(dst_buf, buf_state);
 	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
 	return IRQ_HANDLED;
 }
 
-static void mtk_jpeg_set_default_params(struct mtk_jpeg_ctx *ctx)
+static void mtk_jpeg_set_enc_default_params(struct mtk_jpeg_ctx *ctx)
+{
+	struct mtk_jpeg_q_data *q = &ctx->out_q;
+	int align_w, align_h;
+
+	ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+
+	ctx->colorspace = V4L2_COLORSPACE_JPEG,
+	ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
+	ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+
+	q->w = MTK_JPEG_MIN_WIDTH;
+	q->h = MTK_JPEG_MIN_HEIGHT;
+
+	q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_YUYV,
+				      MTK_JPEG_FMT_TYPE_OUTPUT);
+
+	align_w = q->w;
+	align_h = q->h;
+	align_w = round_up(align_w, 2);
+	v4l_bound_align_image(&align_w, MTK_JPEG_MIN_WIDTH, MTK_JPEG_MAX_WIDTH,
+			      5, &align_h, MTK_JPEG_MIN_HEIGHT,
+			      MTK_JPEG_MAX_HEIGHT, 3, 0);
+	align_w = align_w << 1;
+
+	if (align_w < MTK_JPEG_MIN_WIDTH &&
+	    (align_w + 32) <= MTK_JPEG_MAX_WIDTH)
+		align_w += 32;
+	if (align_h < MTK_JPEG_MIN_HEIGHT &&
+	    (align_h + 8) <= MTK_JPEG_MAX_HEIGHT)
+		align_h += 8;
+
+	q->sizeimage[0] = align_w * align_h;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1408

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+	q->bytesperline[0] = align_w;
+
+	q = &ctx->cap_q;
+	q->w = MTK_JPEG_MIN_WIDTH;
+	q->h = MTK_JPEG_MIN_HEIGHT;
+	q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_JPEG,
+				      MTK_JPEG_FMT_TYPE_CAPTURE);
+	q->bytesperline[0] = 0;
+	q->sizeimage[0] = MTK_JPEG_DEFAULT_SIZEIMAGE;
+}

Could we just create an arbitrary v4l2_pix_format_mplane struct and call s_fmt instead? In general, all of the constant values and alignments should be already ensured by try_fmt, so this function should be redundant.

Quoted 32 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -2
+
+static void mtk_jpeg_set_dec_default_params(struct mtk_jpeg_ctx *ctx)
 {
 	struct mtk_jpeg_q_data *q = &ctx->out_q;
 	int i;
 
+	ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+
 	ctx->colorspace = V4L2_COLORSPACE_JPEG,
 	ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
 	ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
 	ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;
 
 	q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_JPEG,
-					      MTK_JPEG_FMT_TYPE_OUTPUT);
+				      MTK_JPEG_FMT_TYPE_OUTPUT);
 	q->w = MTK_JPEG_MIN_WIDTH;
 	q->h = MTK_JPEG_MIN_HEIGHT;
 	q->bytesperline[0] = 0;
@@ -979,7 +1428,7 @@ static void mtk_jpeg_set_default_params(struct mtk_jpeg_ctx *ctx)
 
 	q = &ctx->cap_q;
 	q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_YUV420M,
-					      MTK_JPEG_FMT_TYPE_CAPTURE);
+				      MTK_JPEG_FMT_TYPE_CAPTURE);
 	q->w = MTK_JPEG_MIN_WIDTH;
 	q->h = MTK_JPEG_MIN_HEIGHT;
 
@@ -1020,7 +1469,15 @@ static int mtk_jpeg_open(struct file *file)
 		goto error;
 	}
 

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_open(struct file *file) · line 1480

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +9 -1
-	mtk_jpeg_set_default_params(ctx);
+	ret = mtk_jpeg_ctrls_setup(ctx);
+	if (ret) {
+		v4l2_err(&jpeg->v4l2_dev, "Failed to setup controls() (%d)\n",
+			 ret);
+		goto error;
+	}
+
+	jpeg->mode == MTK_JPEG_ENC ? mtk_jpeg_set_enc_default_params(ctx) :
+				     mtk_jpeg_set_dec_default_params(ctx);

As mentioned before, this kind of usage of the ternary operator is generaly discouraged, because it makes the code difficult to read.

Quoted 12 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -0
 	mutex_unlock(&jpeg->lock);
 	return 0;
 
@@ -1040,6 +1497,7 @@ static int mtk_jpeg_release(struct file *file)
 
 	mutex_lock(&jpeg->lock);
 	v4l2_m2m_ctx_release(ctx->fh.m2m_ctx);
+	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
 	v4l2_fh_del(&ctx->fh);
 	v4l2_fh_exit(&ctx->fh);
 	kfree(ctx);

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_clk_init(struct mtk_jpeg_dev *jpeg) · line 1525

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -0
@@ -1064,6 +1522,7 @@ static int mtk_jpeg_clk_init(struct mtk_jpeg_dev *jpeg)
 	node = of_parse_phandle(jpeg->dev->of_node, "mediatek,larb", 0);
 	if (!node)
 		return -EINVAL;
+

Unnecessary change?

Quoted 59 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +18 -15
 	pdev = of_find_device_by_node(node);
 	if (WARN_ON(!pdev)) {
 		of_node_put(node);
@@ -1073,19 +1532,24 @@ static int mtk_jpeg_clk_init(struct mtk_jpeg_dev *jpeg)
 
 	jpeg->larb = &pdev->dev;
 
-	jpeg->clk_jdec = devm_clk_get(jpeg->dev, "jpgdec");
-	if (IS_ERR(jpeg->clk_jdec))
-		return PTR_ERR(jpeg->clk_jdec);
+	if (jpeg->mode == MTK_JPEG_ENC) {
+		jpeg->clk_jpeg = devm_clk_get(jpeg->dev, "jpgenc");
+		return PTR_ERR_OR_ZERO(jpeg->clk_jpeg);
+	}
+
+	jpeg->clk_jpeg = devm_clk_get(jpeg->dev, "jpgdec");
+	if (IS_ERR(jpeg->clk_jpeg))
+		return PTR_ERR(jpeg->clk_jpeg);
 
-	jpeg->clk_jdec_smi = devm_clk_get(jpeg->dev, "jpgdec-smi");
-	return PTR_ERR_OR_ZERO(jpeg->clk_jdec_smi);
+	jpeg->clk_jpeg_smi = devm_clk_get(jpeg->dev, "jpgdec-smi");
+	return PTR_ERR_OR_ZERO(jpeg->clk_jpeg_smi);
 }
 
 static int mtk_jpeg_probe(struct platform_device *pdev)
 {
 	struct mtk_jpeg_dev *jpeg;
 	struct resource *res;
-	int dec_irq;
+	int jpeg_irq;
 	int ret;
 
 	jpeg = devm_kzalloc(&pdev->dev, sizeof(*jpeg), GFP_KERNEL);
@@ -1095,28 +1559,26 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 	mutex_init(&jpeg->lock);
 	spin_lock_init(&jpeg->hw_lock);
 	jpeg->dev = &pdev->dev;
+	jpeg->mode = (enum mtk_jpeg_mode)of_device_get_match_data(jpeg->dev);
 
 	res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
-	jpeg->dec_reg_base = devm_ioremap_resource(&pdev->dev, res);
-	if (IS_ERR(jpeg->dec_reg_base)) {
-		ret = PTR_ERR(jpeg->dec_reg_base);
+	jpeg->reg_base = devm_ioremap_resource(&pdev->dev, res);
+	if (IS_ERR(jpeg->reg_base)) {
+		ret = PTR_ERR(jpeg->reg_base);
 		return ret;
 	}
 
-	res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
-	dec_irq = platform_get_irq(pdev, 0);
-	if (!res || dec_irq < 0) {
-		dev_err(&pdev->dev, "Failed to get dec_irq %d.\n", dec_irq);
-		ret = -EINVAL;
-		return ret;
+	jpeg_irq = platform_get_irq(pdev, 0);
+	if (jpeg_irq < 0) {
+		dev_err(&pdev->dev, "Failed to get jpeg_irq %d.\n", jpeg_irq);

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_probe(struct platform_device *pdev) · line 1119

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -4
+		return jpeg_irq;
 	}
 
-	ret = devm_request_irq(&pdev->dev, dec_irq, mtk_jpeg_dec_irq, 0,
+	ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_irq, 0,
 			       pdev->name, jpeg);
 	if (ret) {
-		dev_err(&pdev->dev, "Failed to request dec_irq %d (%d)\n",
-			dec_irq, ret);
-		ret = -EINVAL;

This removal of ret assignment looks like a separate fix that should be done in its own patch.

Quoted 32 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -13
+		dev_err(&pdev->dev, "Failed to request jpeg_irq %d (%d)\n",
+			jpeg_irq, ret);
 		goto err_req_irq;
 	}
 
@@ -1140,33 +1602,35 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 		goto err_m2m_init;
 	}
 
-	jpeg->dec_vdev = video_device_alloc();
-	if (!jpeg->dec_vdev) {
+	jpeg->vfd_jpeg = video_device_alloc();
+	if (!jpeg->vfd_jpeg) {
 		ret = -ENOMEM;
-		goto err_dec_vdev_alloc;
+		goto err_vfd_jpeg_alloc;
 	}
-	snprintf(jpeg->dec_vdev->name, sizeof(jpeg->dec_vdev->name),
-		 "%s-dec", MTK_JPEG_NAME);
-	jpeg->dec_vdev->fops = &mtk_jpeg_fops;
-	jpeg->dec_vdev->ioctl_ops = &mtk_jpeg_ioctl_ops;
-	jpeg->dec_vdev->minor = -1;
-	jpeg->dec_vdev->release = video_device_release;
-	jpeg->dec_vdev->lock = &jpeg->lock;
-	jpeg->dec_vdev->v4l2_dev = &jpeg->v4l2_dev;
-	jpeg->dec_vdev->vfl_dir = VFL_DIR_M2M;
-	jpeg->dec_vdev->device_caps = V4L2_CAP_STREAMING |
+	snprintf(jpeg->vfd_jpeg->name, sizeof(jpeg->vfd_jpeg->name),
+		 "%s-%s", MTK_JPEG_NAME,
+		 jpeg->mode == MTK_JPEG_ENC ? "enc" : "dec");
+	jpeg->vfd_jpeg->fops = &mtk_jpeg_fops;
+	jpeg->vfd_jpeg->ioctl_ops = &mtk_jpeg_ioctl_ops;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_probe(struct platform_device *pdev) · line 1623

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +7 -1
+	jpeg->vfd_jpeg->minor = -1;
+	jpeg->vfd_jpeg->release = video_device_release;
+	jpeg->vfd_jpeg->lock = &jpeg->lock;
+	jpeg->vfd_jpeg->v4l2_dev = &jpeg->v4l2_dev;
+	jpeg->vfd_jpeg->vfl_dir = VFL_DIR_M2M;
+	jpeg->vfd_jpeg->device_caps = V4L2_CAP_STREAMING |
 				      V4L2_CAP_VIDEO_M2M_MPLANE;
 
-	ret = video_register_device(jpeg->dec_vdev, VFL_TYPE_GRABBER, 3);
+	ret = video_register_device(jpeg->vfd_jpeg, VFL_TYPE_GRABBER, -1);

The change from 3 to -1 also looks like something for a separate patch.

Quoted 105 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +25 -13
 	if (ret) {
 		v4l2_err(&jpeg->v4l2_dev, "Failed to register video device\n");
-		goto err_dec_vdev_register;
+		goto err_vfd_jpeg_register;
 	}
 
-	video_set_drvdata(jpeg->dec_vdev, jpeg);
+	video_set_drvdata(jpeg->vfd_jpeg, jpeg);
 	v4l2_info(&jpeg->v4l2_dev,
-		  "decoder device registered as /dev/video%d (%d,%d)\n",
-		  jpeg->dec_vdev->num, VIDEO_MAJOR, jpeg->dec_vdev->minor);
+		  "jpeg device %d registered as /dev/video%d (%d,%d)\n",
+		  jpeg->mode, jpeg->vfd_jpeg->num, VIDEO_MAJOR,
+		  jpeg->vfd_jpeg->minor);
 
 	platform_set_drvdata(pdev, jpeg);
 
@@ -1174,10 +1638,10 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 
 	return 0;
 
-err_dec_vdev_register:
-	video_device_release(jpeg->dec_vdev);
+err_vfd_jpeg_register:
+	video_device_release(jpeg->vfd_jpeg);
 
-err_dec_vdev_alloc:
+err_vfd_jpeg_alloc:
 	v4l2_m2m_release(jpeg->m2m_dev);
 
 err_m2m_init:
@@ -1197,8 +1661,8 @@ static int mtk_jpeg_remove(struct platform_device *pdev)
 	struct mtk_jpeg_dev *jpeg = platform_get_drvdata(pdev);
 
 	pm_runtime_disable(&pdev->dev);
-	video_unregister_device(jpeg->dec_vdev);
-	video_device_release(jpeg->dec_vdev);
+	video_unregister_device(jpeg->vfd_jpeg);
+	video_device_release(jpeg->vfd_jpeg);
 	v4l2_m2m_release(jpeg->m2m_dev);
 	v4l2_device_unregister(&jpeg->v4l2_dev);
 
@@ -1209,7 +1673,11 @@ static __maybe_unused int mtk_jpeg_pm_suspend(struct device *dev)
 {
 	struct mtk_jpeg_dev *jpeg = dev_get_drvdata(dev);
 
-	mtk_jpeg_dec_reset(jpeg->dec_reg_base);
+	if (jpeg->mode == MTK_JPEG_ENC)
+		mtk_jpeg_enc_reset(jpeg->reg_base);
+	else
+		mtk_jpeg_dec_reset(jpeg->reg_base);
+
 	mtk_jpeg_clk_off(jpeg);
 
 	return 0;
@@ -1220,7 +1688,10 @@ static __maybe_unused int mtk_jpeg_pm_resume(struct device *dev)
 	struct mtk_jpeg_dev *jpeg = dev_get_drvdata(dev);
 
 	mtk_jpeg_clk_on(jpeg);
-	mtk_jpeg_dec_reset(jpeg->dec_reg_base);
+	if (jpeg->mode == MTK_JPEG_ENC)
+		mtk_jpeg_enc_reset(jpeg->reg_base);
+	else
+		mtk_jpeg_dec_reset(jpeg->reg_base);
 
 	return 0;
 }
@@ -1256,11 +1727,15 @@ static const struct dev_pm_ops mtk_jpeg_pm_ops = {
 static const struct of_device_id mtk_jpeg_match[] = {
 	{
 		.compatible = "mediatek,mt8173-jpgdec",
-		.data       = NULL,
+		.data       = (void *)MTK_JPEG_DEC,
 	},
 	{
 		.compatible = "mediatek,mt2701-jpgdec",
-		.data       = NULL,
+		.data       = (void *)MTK_JPEG_DEC,
+	},
+	{
+		.compatible = "mediatek,mtk-jpgenc",
+		.data       = (void *)MTK_JPEG_ENC,
 	},
 	{},
 };
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +3 -0
index 999bd1427809..35ddc64af554 100644
@@ -3,6 +3,7 @@
  * Copyright (c) 2016 MediaTek Inc.
  * Author: Ming Hsiu Tsai <[email protected]>
  *         Rick Chang <[email protected]>
+ *         Xia Jiang <[email protected]>
  */
 
 #ifndef _MTK_JPEG_CORE_H
@@ -17,23 +18,77 @@
 
 #define MTK_JPEG_FMT_FLAG_DEC_OUTPUT	BIT(0)
 #define MTK_JPEG_FMT_FLAG_DEC_CAPTURE	BIT(1)
+#define MTK_JPEG_FMT_FLAG_ENC_OUTPUT	BIT(2)
+#define MTK_JPEG_FMT_FLAG_ENC_CAPTURE	BIT(3)
 
 #define MTK_JPEG_FMT_TYPE_OUTPUT	1

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 30

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +4 -4
 #define MTK_JPEG_FMT_TYPE_CAPTURE	2
 
-#define MTK_JPEG_MIN_WIDTH	32
-#define MTK_JPEG_MIN_HEIGHT	32
-#define MTK_JPEG_MAX_WIDTH	8192
-#define MTK_JPEG_MAX_HEIGHT	8192
+#define MTK_JPEG_MIN_WIDTH	32U
+#define MTK_JPEG_MIN_HEIGHT	32U
+#define MTK_JPEG_MAX_WIDTH	65535U
+#define MTK_JPEG_MAX_HEIGHT	65535U

Why is it okay to change this from 8192 to 65535?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 36

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +3 -0
 #define MTK_JPEG_DEFAULT_SIZEIMAGE	(1 * 1024 * 1024)
+#define MTK_JPEG_DEFAULT_EXIF_SIZE	(64 * 1024)
 
+#define MTK_JPEG_ENCODE		0
+#define MTK_JPEG_DECODE		1

These two are unused.

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +4 -0
+
+/**
+ * enum mtk_jpeg_ctx_state - contex state of jpeg
+ */
 enum mtk_jpeg_ctx_state {
 	MTK_JPEG_INIT = 0,
 	MTK_JPEG_RUNNING,

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 53

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +7 -0
 	MTK_JPEG_SOURCE_CHANGE,
 };
 
+/**
+ * enum mtk_jpeg_mode - mode of jpeg
+ */
+enum mtk_jpeg_mode {
+	MTK_JPEG_ENC,
+	MTK_JPEG_DEC,
+};

Perhaps we could just have a bool is_encoder?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 63

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +10 -0
+
+/**
+ * enum jpeg_enc_yuv_fmt - yuv format of jpeg enc
+ */
+enum jpeg_enc_yuv_fmt {
+	JPEG_YUV_FORMAT_YUYV = 0,
+	JPEG_YUV_FORMAT_YVYU = 1,
+	JPEG_YUV_FORMAT_NV12 = 2,
+	JEPG_YUV_FORMAT_NV21 = 3,
+};

#define macros are typically used for hardware values.

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +6 -0
+
+/**
+ * enum JPEG_ENCODE_QUALITY_ENUM - number of jpeg encoder quality
+ */
+enum JPEG_ENCODE_QUALITY_ENUM {
+	JPEG_ENCODE_QUALITY_Q60 = 0x0,

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 79

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +10 -0
+	JPEG_ENCODE_QUALITY_Q80 = 0x1,
+	JPEG_ENCODE_QUALITY_Q90 = 0x2,
+	JPEG_ENCODE_QUALITY_Q95 = 0x3,
+
+	JPEG_ENCODE_QUALITY_Q39 = 0x4,
+	JPEG_ENCODE_QUALITY_Q68 = 0x5,
+	JPEG_ENCODE_QUALITY_Q84 = 0x6,
+	JPEG_ENCODE_QUALITY_Q92 = 0x7,
+
+	JPEG_ENCODE_QUALITY_Q48 = 0x8,

No enum for 0x9?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 83

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +4 -0
+	JPEG_ENCODE_QUALITY_Q74 = 0xA,
+	JPEG_ENCODE_QUALITY_Q87 = 0xB,
+
+	JPEG_ENCODE_QUALITY_Q34 = 0xC,

No enum for 0xd?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 89

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +6 -0
+	JPEG_ENCODE_QUALITY_Q64 = 0xE,
+	JPEG_ENCODE_QUALITY_Q82 = 0xF,
+
+	JPEG_ENCODE_QUALITY_Q97 = 0x10,
+
+	JPEG_ENCODE_QUALITY_ALL = 0xFFFFFFFF

What is the meaning of this value?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 89

Quoted 1 line
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +1 -0
+};

Since these are hardware register values, #define macros are preferred.

Quoted 40 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +11 -8
+
 /**
  * struct mt_jpeg - JPEG IP abstraction
  * @lock:		the mutex protecting this structure
@@ -43,11 +98,12 @@ enum mtk_jpeg_ctx_state {
  * @v4l2_dev:		v4l2 device for mem2mem mode
  * @m2m_dev:		v4l2 mem2mem device data
  * @alloc_ctx:		videobuf2 memory allocator's context
- * @dec_vdev:		video device node for decoder mem2mem mode
- * @dec_reg_base:	JPEG registers mapping
- * @clk_jdec:		JPEG hw working clock
- * @clk_jdec_smi:	JPEG SMI bus clock
+ * @vfd_jpeg:		video device node for jpeg mem2mem mode
+ * @reg_base:		JPEG registers mapping
+ * @clk_jpeg:		JPEG hw working clock
+ * @clk_jpeg_smi:	JPEG SMI bus clock
  * @larb:		SMI device
+ * @mode:		compression (encode) operation or decompression (decode)
  */
 struct mtk_jpeg_dev {
 	struct mutex		lock;
@@ -57,11 +113,12 @@ struct mtk_jpeg_dev {
 	struct v4l2_device	v4l2_dev;
 	struct v4l2_m2m_dev	*m2m_dev;
 	void			*alloc_ctx;
-	struct video_device	*dec_vdev;
-	void __iomem		*dec_reg_base;
-	struct clk		*clk_jdec;
-	struct clk		*clk_jdec_smi;
+	struct video_device	*vfd_jpeg;
+	void __iomem		*reg_base;
+	struct clk		*clk_jpeg;
+	struct clk		*clk_jpeg_smi;
 	struct device		*larb;
+	enum mtk_jpeg_mode	mode;
 };
 
 /**
@@ -101,15 +158,51 @@ struct mtk_jpeg_q_data {
 	u32			sizeimage[VIDEO_MAX_PLANES];

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h struct mtk_jpeg_q_data { · line 168

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +8 -0
 };
 
+/**
+ * jpeg_enc_param - parameters of jpeg encode control
+ * @enable_exif:	EXIF enable for jpeg encode mode
+ * @enc_quality:	destination image quality in encode mode
+ * @restart_interval:	JPEG restart interval for JPEG encoding
+ */
+struct jpeg_enc_param {
+	u32 enable_exif;

Shouldn't this be a bool?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h struct mtk_jpeg_q_data { · line 169

Quoted 1 line
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +1 -0
+	u32 enc_quality;

This is from 0 to 100, so perhaps unsigned char?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h struct mtk_jpeg_q_data { · line 170

Quoted 1 line
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +1 -0
+	u32 restart_interval;

Ditto.

Quoted 15 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +15 -0
+};
+
+/**
+ * mtk_jpeg_enc_param:  General jpeg encoding parameters
+ * @enc_w:		image width
+ * @enc_h:		image height
+ * @enable_exif:	EXIF enable for jpeg encode mode
+ * @enc_quality:	destination image quality in encode mode
+ * @enc_format:		input image format
+ * @restart_interval:	JPEG restart interval for JPEG encoding
+ * @img_stride:		jpeg encoder image stride
+ * @mem_stride:		jpeg encoder memory stride
+ * @total_encdu:	total 8x8 block number
+ */
+struct mtk_jpeg_enc_param {

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h struct mtk_jpeg_q_data { · line 195

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +10 -0
+	u32 enc_w;
+	u32 enc_h;
+	u32 enable_exif;
+	u32 enc_quality;
+	u32 enc_format;
+	u32 restart_interval;
+	u32 img_stride;
+	u32 mem_stride;
+	u32 total_encdu;
+};

All of the parameters here are already represented by corresponding generic V4L2 state. We should be able to just convert them to the hardware representation at the time of programing the registers, without a need to store them in this intermediate struct.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h struct mtk_jpeg_q_data { · line 112

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +1 -2
+
 /**
  * mtk_jpeg_ctx - the device context data
  * @jpeg:		JPEG IP device for this context
  * @out_q:		source (output) queue information
  * @cap_q:		destination (capture) queue queue information
  * @fh:			V4L2 file handle
- * @dec_param		parameters for HW decoding
  * @state:		state of the context
- * @header_valid:	set if header has been parsed and valid

Why removing these two?

Quoted 70 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +4 -0
+ * @jpeg_param:		jpeg encode parameters
+ * @ctrl_hdl:		controls handler
  * @colorspace: enum v4l2_colorspace; supplemental to pixelformat
  * @ycbcr_enc: enum v4l2_ycbcr_encoding, Y'CbCr encoding
  * @quantization: enum v4l2_quantization, colorspace quantization
@@ -121,6 +214,8 @@ struct mtk_jpeg_ctx {
 	struct mtk_jpeg_q_data		cap_q;
 	struct v4l2_fh			fh;
 	enum mtk_jpeg_ctx_state		state;
+	struct jpeg_enc_param		jpeg_param;
+	struct v4l2_ctrl_handler	ctrl_hdl;
 
 	enum v4l2_colorspace colorspace;
 	enum v4l2_ycbcr_encoding ycbcr_enc;
drivers/media/platform/mtk-jpeg/mtk_jpeg_dec_hw.h +4 -3
index a37be1a48415..39d743fb572f 100644
@@ -3,10 +3,11 @@
  * Copyright (c) 2016 MediaTek Inc.
  * Author: Ming Hsiu Tsai <[email protected]>
  *         Rick Chang <[email protected]>
+ *         Xia Jiang <[email protected]>
  */
 
-#ifndef _MTK_JPEG_HW_H
-#define _MTK_JPEG_HW_H
+#ifndef _MTK_JPEG_DEC_HW_H
+#define _MTK_JPEG_DEC_HW_H
 
 #include <media/videobuf2-core.h>
 
@@ -80,4 +81,4 @@ void mtk_jpeg_dec_set_config(void __iomem *base,
 void mtk_jpeg_dec_reset(void __iomem *dec_reg_base);
 void mtk_jpeg_dec_start(void __iomem *dec_reg_base);
 
-#endif /* _MTK_JPEG_HW_H */
+#endif /* _MTK_JPEG_DEC_HW_H */
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +26 -0
new file mode 100644
index 000000000000..ac27774ccfed
@@ -0,0 +1,175 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ * Author: Xia Jiang <[email protected]>
+ *
+ */
+
+#include <linux/io.h>
+#include <linux/kernel.h>
+#include <media/videobuf2-core.h>
+
+#include "mtk_jpeg_enc_hw.h"
+
+void mtk_jpeg_enc_reset(void __iomem *base)
+{
+	writel(0x00, base + JPGENC_RSTB);
+	writel(JPEG_ENC_RESET_BIT, base + JPGENC_RSTB);
+	writel(0x00, base + JPGENC_CODEC_SEL);
+}
+
+u32 mtk_jpeg_enc_get_int_status(void __iomem *base)
+{
+	u32 ret;
+
+	ret = readl(base + JPGENC_INT_STS) &
+		    JPEG_DRV_ENC_INT_STATUS_MASK_ALLIRQ;

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 36

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +10 -0
+	if (ret)
+		writel(0, base + JPGENC_INT_STS);
+
+	return ret;
+}
+
+u32 mtk_jpeg_enc_get_file_size(void __iomem *base)
+{
+	return readl(base + JPGENC_DMA_ADDR0) - readl(base + JPGENC_DST_ADDR0);
+}

This is just used in the function below, so the code could just be moved there.

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 47

Quoted 11 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +11 -0
+
+u32 mtk_jpeg_enc_enum_result(void __iomem *base, u32 irq_status, u32 *file_size)
+{
+	*file_size = mtk_jpeg_enc_get_file_size(base);
+	if (irq_status & JPEG_DRV_ENC_INT_STATUS_DONE)
+		return MTK_JPEG_ENC_RESULT_DONE;
+	else if (irq_status & JPEG_DRV_ENC_INT_STATUS_STALL)
+		return MTK_JPEG_ENC_RESULT_STALL;
+	else if (irq_status & JPEG_DRV_ENC_INT_STATUS_VCODEC_IRQ)
+		return MTK_JPEG_ENC_RESULT_VCODEC_IRQ;
+	return MTK_JPEG_ENC_RESULT_ERROR_UNKNOWN;

This function seems to do two things at a time, which is confusing, given that the name only refers to the result and doesn't mention the "file size".

So an alternative option to my suggestion above would be removing the file size handling from this function and using the dedicated function for it.

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 53

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +6 -0
+}
+
+static void mtk_jpeg_enc_set_blk_num(void __iomem *base, u32 blk_num)
+{
+	writel(blk_num, base + JPGENC_BLK_NUM);
+}

Is there any benefit of having a lot of such small functions rather than just doing everything needed in one function? This hardware doesn't have so many parameters to configure and the caller side is generic V4L2 and vb2, so there is no need for additional layer of abstraction.

How about the following model, as used by many other drivers:

mtk_jpeg_enc_set_src()

{
	// Set any registers related to source format and buffer
}

mtk_jpeg_enc_set_dst()

{
	// Set any registers related to destination format and buffer
}

mtk_jpeg_enc_set_params()

{
	// Set any registers related to additional encoding parameters
}

mtk_jpeg_enc_device_run(enc, ctx)

{
	mtk_jpeg_enc_set_src(enc, src_buf, src_fmt);
	mtk_jpeg_enc_set_dst(enc, dst_buf, dst_fmt);
	mtk_jpeg_enc_set_params(enc, ctx);
	// Trigger the hardware run
}

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 62

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +9 -0
+
+static void mtk_jpeg_enc_set_encFormat(void __iomem *base, u32 enc_format)
+{
+	u32 value;
+
+	value = readl(base + JPGENC_CTRL);
+	value &= ~JPEG_ENC_CTRL_YUV_BIT;
+	value |= JPGENC_FORMAT(enc_format);
+	writel(value, base + JPGENC_CTRL);

The model I suggested above also avoids this kind of read modify write operations, which just unnecessarily add synchronous MMIO round trips, which means more CPU overhead.

Given that the full state is always known by the driver, it can just write all the register values without the need to read them back.

Quoted 36 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +36 -0
+}
+
+static void mtk_jpeg_enc_set_img_size(void __iomem *base, u32 width, u32 height)
+{
+	u32 value;
+
+	value = JPGENC_WIDTH_HEIGHT(width, height);
+	writel(value, base + JPGENC_IMG_SIZE);
+}
+
+static void mtk_jpeg_enc_set_src_img(void __iomem *base, u32 width,
+				     u32 height, u32 yuv_format,
+				     u32 total_encdu)
+{
+	mtk_jpeg_enc_set_img_size(base, width, height);
+	mtk_jpeg_enc_set_encFormat(base, yuv_format);
+	mtk_jpeg_enc_set_blk_num(base, total_encdu);
+}
+
+static void mtk_jpeg_enc_set_src_buf(void __iomem *base, u32 img_stride,
+				     u32 mem_stride, u32 src_addr,
+				     u32 src_addr_c)
+{
+	writel(img_stride, base + JPGENC_IMG_STRIDE);
+	writel(mem_stride, base + JPGENC_STRIDE);
+	writel(src_addr, base + JPGENC_SRC_LUMA_ADDR);
+	writel(src_addr_c, base + JPGENC_SRC_CHROMA_ADDR);
+}
+
+static void mtk_jpeg_enc_set_dst_buf(void __iomem *base, u32 dst_addr,
+				     u32 stall_size, u32 init_offset,
+				     u32 offset_mask)
+{
+	writel(JPGENC_INIT_OFFSET(init_offset), base + JPGENC_OFFSET_ADDR);
+	writel(JPGENC_OFFSET_MASK(offset_mask), base + JPGENC_BYTE_OFFSET_MASK);
+	writel(JPGENC_DST_ADDR(dst_addr), base + JPGENC_DST_ADDR0);

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 108

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +10 -0
+	writel(JPGENC_STALL_ADDR(dst_addr, stall_size),
+	       base + JPGENC_STALL_ADDR0);
+}
+
+static void mtk_jpeg_enc_set_quality(void __iomem *base, u32 quality)
+{
+	u32 value;
+
+	value = readl(base + JPGENC_QUALITY);
+	value = JPGENC_SET_QUALITY(value, quality);

This macro is confusing. Could we just clear the quality bits explicitly and then bitwise-OR the new value?

Quoted 81 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +67 -0
+	writel(value, base + JPGENC_QUALITY);
+}
+
+static void mtk_jpeg_enc_set_restart_interval(void __iomem *base,
+					      u32 restart_interval)
+{
+	u32 value;
+
+	value = readl(base + JPGENC_CTRL);
+	if (restart_interval)
+		value |= JPEG_ENC_CTRL_RESTART_EN_BIT;
+	else
+		value &= ~JPEG_ENC_CTRL_RESTART_EN_BIT;
+	writel(value, base + JPGENC_CTRL);
+	writel(restart_interval, base + JPGENC_RST_MCU_NUM);
+}
+
+static void mtk_jpeg_enc_set_encode_mode(void __iomem *base, u32 exif_en)
+{
+	u32 value;
+
+	value = readl(base + JPGENC_CTRL);
+	value &= ~JPEG_ENC_CTRL_FILE_FORMAT_BIT;
+	writel(value, base + JPGENC_CTRL);
+
+	if (exif_en) {
+		value = readl(base + JPGENC_CTRL);
+		value |= JPEG_ENC_EN_JFIF_EXIF;
+		writel(value, base + JPGENC_CTRL);
+	}
+}
+
+static void mtk_jpeg_enc_set_ctrl_cfg(void __iomem *base, u32 exif_en,
+				      u32 quality, u32 restart_interval)
+{
+	mtk_jpeg_enc_set_quality(base, quality);
+
+	mtk_jpeg_enc_set_restart_interval(base, restart_interval);
+
+	mtk_jpeg_enc_set_encode_mode(base, exif_en);
+}
+
+void mtk_jpeg_enc_start(void __iomem *base)
+{
+	u32 value;
+
+	value = readl(base + JPGENC_CTRL);
+	value |= JPEG_ENC_CTRL_INT_EN_BIT | JPEG_ENC_CTRL_ENABLE_BIT;
+	writel(value, base + JPGENC_CTRL);
+}
+
+void mtk_jpeg_enc_set_config(void __iomem *base,
+			     struct mtk_jpeg_enc_param *config,
+			     struct mtk_jpeg_enc_bs *bs,
+			     struct mtk_jpeg_enc_fb *fb)
+{
+	mtk_jpeg_enc_set_src_img(base, config->enc_w, config->enc_h,
+				 config->enc_format, config->total_encdu);
+	mtk_jpeg_enc_set_src_buf(base, config->img_stride, config->mem_stride,
+				 fb->fb_addr[0].dma_addr,
+				 fb->fb_addr[1].dma_addr);
+	mtk_jpeg_enc_set_dst_buf(base, bs->dma_addr, bs->size,
+				 bs->dma_addr_offset, bs->dma_addr_offsetmask);
+	mtk_jpeg_enc_set_ctrl_cfg(base, config->enable_exif,
+				  config->enc_quality,
+				  config->restart_interval);
+}
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h new file +8 -0
new file mode 100644
index 000000000000..7080240f6dac
@@ -0,0 +1,60 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ * Author: Xia Jiang <[email protected]>
+ *
+ */
+
+#ifndef _MTK_JPEG_ENC_HW_H

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h line 18

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h new file +10 -0
+#define _MTK_JPEG_ENC_HW_H
+
+#include <media/videobuf2-core.h>
+
+#include "mtk_jpeg_core.h"
+#include "mtk_jpeg_enc_reg.h"
+
+#define JPEG_ENC_DST_ADDR_OFFSET_MASK GENMASK(3, 0)
+
+#define JPEG_ENC_CTRL_YUV_BIT 0x18

0x18 isn't a single bit as the _BIT suffix would suggest. Perhaps JPEG_ENC_CTRL_FORMAT_MASK would be more appropriate?

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h line 31

Quoted 13 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h new file +13 -0
+#define JPEG_ENC_CTRL_RESTART_EN_BIT            BIT(10)
+#define JPEG_ENC_CTRL_FILE_FORMAT_BIT           BIT(5)
+#define JPEG_ENC_EN_JFIF_EXIF                   BIT(5)
+#define JPEG_ENC_CTRL_INT_EN_BIT                BIT(2)
+#define JPEG_ENC_CTRL_ENABLE_BIT                BIT(0)
+#define JPEG_ENC_RESET_BIT                      BIT(0)
+
+enum {
+	MTK_JPEG_ENC_RESULT_DONE		= 0,
+	MTK_JPEG_ENC_RESULT_STALL,
+	MTK_JPEG_ENC_RESULT_VCODEC_IRQ,
+	MTK_JPEG_ENC_RESULT_ERROR_UNKNOWN
+};

Do we need these intermediate error codes? Could we just use errno values instead?

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h new file +7 -0
+
+struct mtk_jpeg_enc_bs {
+	dma_addr_t	dma_addr;
+	size_t		size;
+	u32			dma_addr_offset;
+	u32			dma_addr_offsetmask;
+};

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h line 48

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h new file +10 -0
+
+struct mtk_jpeg_mem {
+	dma_addr_t	dma_addr;
+	size_t		size;
+};
+
+struct mtk_jpeg_enc_fb {
+	struct mtk_jpeg_mem	fb_addr[MTK_JPEG_COMP_MAX];
+	u32			num_planes;
+};

Please add kerneldoc comments to the structs above.

nit: Separating types from names with a single space is more future-proof, in case someone needs to change the layout of the struct or rename fields.

Quoted 18 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h new file +12 -0
+
+void mtk_jpeg_enc_reset(void __iomem *base);
+u32 mtk_jpeg_enc_get_int_status(void __iomem *base);
+u32 mtk_jpeg_enc_get_file_size(void __iomem *base);
+u32 mtk_jpeg_enc_enum_result(void __iomem *base, u32 irq_status,
+			     u32 *file_size);
+void mtk_jpeg_enc_start(void __iomem *enc_reg_base);
+void mtk_jpeg_enc_set_config(void __iomem *base,
+			     struct mtk_jpeg_enc_param *config,
+			     struct mtk_jpeg_enc_bs *bs,
+			     struct mtk_jpeg_enc_fb *fb);
+#endif /* _MTK_JPEG_ENC_HW_H */
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h new file +0 -0
new file mode 100644
index 000000000000..d785621cb4c6

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h line 9

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h new file +9 -0
@@ -0,0 +1,49 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ * Author: Xia Jiang <[email protected]>
+ *
+ */
+#ifndef _MTK_JPEG_ENC_REG_H
+#define _MTK_JPEG_ENC_REG_H
+#define MTK_JPEG_COMP_MAX		3

This doesn't look like a register-specific definition.

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h line 14

Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h new file +5 -0
+
+#define JPEG_DRV_ENC_INT_STATUS_DONE			BIT(0)
+#define JPEG_DRV_ENC_INT_STATUS_STALL			BIT(1)
+#define JPEG_DRV_ENC_INT_STATUS_VCODEC_IRQ		BIT(4)
+#define JPEG_DRV_ENC_INT_STATUS_MASK_ALLIRQ		0x13

Could we define this as an | of the specific bit definitions above rather than hardcoding?

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h line 16

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h new file +2 -0
+
+#define JPGENC_RSTB				0x100

Could we agree on a consistent prefix for macros? Some use JPEG_ENC, some JPEG_DRV_ENC and some JPGENC. How about using JPEG_ENC everywhere?

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h line 19

Quoted 3 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h new file +3 -0
+#define JPGENC_CTRL				0x104
+#define JPGENC_QUALITY				0x108
+#define JPGENC_BLK_NUM				0x10C

nit: Hex numerals in the kernel are lowercase.

Quoted 18 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h new file +18 -0
+#define JPGENC_BLK_CNT				0x110
+#define JPGENC_INT_STS				0x11C
+#define JPGENC_DST_ADDR0			0x120
+#define JPGENC_DMA_ADDR0			0x124
+#define JPGENC_STALL_ADDR0			0x128
+#define JPGENC_OFFSET_ADDR			0x138
+#define JPGENC_RST_MCU_NUM			0x150
+#define JPGENC_IMG_SIZE				0x154
+#define JPGENC_DEBUG_INFO0			0x160
+#define JPGENC_DEBUG_INFO1			0x164
+#define JPGENC_TOTAL_CYCLE			0x168
+#define JPGENC_BYTE_OFFSET_MASK			0x16C
+#define JPGENC_SRC_LUMA_ADDR			0x170
+#define JPGENC_SRC_CHROMA_ADDR			0x174
+#define JPGENC_STRIDE				0x178
+#define JPGENC_IMG_STRIDE			0x17C
+#define JPGENC_DCM_CTRL				0x300
+#define JPGENC_CODEC_SEL			0x314

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h line 47

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h new file +10 -0
+#define JPGENC_ULTRA_THRES			0x318
+
+#define JPGENC_FORMAT(x)		(((x) & 3) << 3)
+#define JPGENC_WIDTH_HEIGHT(w, h)	(((w) << 16) | (h))
+#define JPGENC_INIT_OFFSET(x)		((x) & (~0xF))
+#define JPGENC_OFFSET_MASK(x)		((x) & 0xF)
+#define JPGENC_DST_ADDR(x)		((x) & (~0xF))
+#define JPGENC_STALL_ADDR(x, y)		(((x) + (y)) & (~0xF))
+#define JPGENC_QUALITY_MASK		0xFFFF0000
+#define JPGENC_SET_QUALITY(x, y)	(((x) & JPGENC_QUALITY_MASK) | (y))

JPGENC_QUALITY_MASK is only used here, so 0xffff0000 could be just used directly.

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h line 49

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h new file +2 -0
+
+#endif /* _MTK_JPEG_ENC_REG_H */

I can see some bits defined in mtk_jpeg_enc_hw.h as well. Perhaps all these could be moved there too, which would make us have 1 file less?

Best regards,
Tomasz

author
#9 lore raw

Re: [PATCH v4 5/5] media: platform: Add jpeg dec/enc feature

Quoted 52 lines · Tomasz Figa

Hi Xia,

Quoted 27 lines · Xia Jiang

Add mtk jpeg encode v4l2 driver based on jpeg decode, because that jpeg decode and encode have great similarities with function operation.

---
v4: split mtk_jpeg_try_fmt_mplane() to two functions, one for encoder,
one for decoder.
split mtk_jpeg_set_default_params() to two functions, one for encoder, one for decoder.
add cropping support for encoder in g/s_selection ioctls. change exif mode support by using V4L2_JPEG_ACTIVE_MARKER_APP1. change MTK_JPEG_MAX_WIDTH/MTK_JPEG_MAX_HEIGH from 8192 to 65535 by specification.
move width shifting operation behind aligning operation in mtk_jpeg_try_enc_fmt_mplane() for bug fix.
fix user abuseing data_offset issue for DMABUF in mtk_jpeg_set_enc_src().
fix kbuild warings: change MTK_JPEG_MIN_HEIGHT/MTK_JPEG_MAX_HEIGHT
and MTK_JPEG_MIN_WIDTH/MTK_JPEG_MAX_WIDTH from 'int' type to 'unsigned int' type. fix msleadingly indented of 'else'.

v3: delete Change-Id.
only test once handler->error after the last v4l2_ctrl_new_std(). seperate changes of v4l2-ctrls.c and v4l2-controls.h to new patch.

v2: fix compliance test fail, check created buffer size in driver.

Quoted 13 lines
---
 drivers/media/platform/mtk-jpeg/Makefile      |   5 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 731 +++++++++++++++---
 .../media/platform/mtk-jpeg/mtk_jpeg_core.h   | 123 ++-
 .../media/platform/mtk-jpeg/mtk_jpeg_dec_hw.h |   7 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c | 175 +++++
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h |  60 ++
 .../platform/mtk-jpeg/mtk_jpeg_enc_reg.h      |  49 ++
 7 files changed, 1004 insertions(+), 146 deletions(-)
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h

First of all, thanks for the patch!

Please check my comments below.

My general feeling about this code is that the encoder hardware block is completely orthogonal from the decoder block and there is very little code reuse from the original decoder driver.

Quoted 6 lines

Moreover, a lot of existing code now needs if (decoder) { ... } else {... } segments, which complicates the code.

Would it perhaps make sense to instead create a separate mtk-jpeg-enc driver?

Dear Tomasz,

Thanks for your comments.

My reasons about the architecture of jpeg enc driver are as follows:

The first internal design and realization of jpeg enc driver was a separate driver, but found that mtk_jpeg_core.c and mtk_jpeg_enc_core.c have lots of reuse.Because that the core.c mainly contains realization of v4L2 ioctl functions and some logic which are high similarity between encoder and decoder.

The jpeg encoder and decoder are two independent hardwares exactly, so the code about hardware specification(register setting) are separated(mtk_jpeg_enc_hw.c and mtk_jpeg_dec_hw.c).

As for 17 existing code segments contain if(decoder){} else {}, they are not complicated IMHO.The complicated(multilayer nested) functions are separated in V4 version as Hans recommendation.

By the way,the upstreamed module s5p-jpeg
(https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/s5p-jpeg/jpeg-core.c#L1998) also use encoder and decoder mode in the common core.c, but their encoder and decoder are the same hardware.Maybe our jpeg enc and dec are designed into one hardware in the future.In that case the current architecture is more compatible.

So I prefer the current design.

Quoted 6 lines

It would also give us a fresh start in terms of code quality, as the existing mtk-jpeg driver has a lot of quality issues unfortunately. (Some of my comments to this patch actually relate to the issues with the original code, not introduced by this patch, but we need to fix them if changing this driver already.)

I will fix the issues one by one later.

Quoted 1,937 lines
Quoted 86 lines
drivers/media/platform/mtk-jpeg/Makefile +4 -1
index 48516dcf96e6..76c33aad0f3f 100644
@@ -1,3 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0-only
-mtk_jpeg-objs := mtk_jpeg_core.o mtk_jpeg_dec_hw.o mtk_jpeg_dec_parse.o
+mtk_jpeg-objs := mtk_jpeg_core.o \
+		 mtk_jpeg_dec_hw.o \
+		 mtk_jpeg_dec_parse.o \
+		 mtk_jpeg_enc_hw.o
 obj-$(CONFIG_VIDEO_MEDIATEK_JPEG) += mtk_jpeg.o
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +40 -1
index abc506a552c1..087b62bb1d5c 100644
@@ -3,6 +3,7 @@
  * Copyright (c) 2016 MediaTek Inc.
  * Author: Ming Hsiu Tsai <[email protected]>
  *         Rick Chang <[email protected]>
+ *         Xia Jiang <[email protected]>
  */
 
 #include <linux/clk.h>
@@ -23,6 +24,7 @@
 #include <media/videobuf2-dma-contig.h>
 #include <soc/mediatek/smi.h>
 
+#include "mtk_jpeg_enc_hw.h"
 #include "mtk_jpeg_dec_hw.h"
 #include "mtk_jpeg_core.h"
 #include "mtk_jpeg_dec_parse.h"
@@ -31,7 +33,8 @@ static struct mtk_jpeg_fmt mtk_jpeg_formats[] = {
 	{
 		.fourcc		= V4L2_PIX_FMT_JPEG,
 		.colplanes	= 1,
-		.flags		= MTK_JPEG_FMT_FLAG_DEC_OUTPUT,
+		.flags		= MTK_JPEG_FMT_FLAG_DEC_OUTPUT |
+				  MTK_JPEG_FMT_FLAG_ENC_CAPTURE,
 	},
 	{
 		.fourcc		= V4L2_PIX_FMT_YUV420M,
@@ -51,6 +54,42 @@ static struct mtk_jpeg_fmt mtk_jpeg_formats[] = {
 		.v_align	= 3,
 		.flags		= MTK_JPEG_FMT_FLAG_DEC_CAPTURE,
 	},
+	{
+		.fourcc		= V4L2_PIX_FMT_NV12M,
+		.h_sample	= {4, 2, 2},
+		.v_sample	= {4, 2, 2},
+		.colplanes	= 2,
+		.h_align	= 4,
+		.v_align	= 4,
+		.flags		= MTK_JPEG_FMT_FLAG_ENC_OUTPUT,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_NV21M,
+		.h_sample	= {4, 2, 2},
+		.v_sample	= {4, 2, 2},
+		.colplanes	= 2,
+		.h_align	= 4,
+		.v_align	= 4,
+		.flags		= MTK_JPEG_FMT_FLAG_ENC_OUTPUT,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_YUYV,
+		.h_sample	= {4, 2, 2},
+		.v_sample	= {4, 4, 4},
+		.colplanes	= 1,
+		.h_align	= 4,
+		.v_align	= 3,
+		.flags		= MTK_JPEG_FMT_FLAG_ENC_OUTPUT,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_YVYU,
+		.h_sample	= {4, 2, 2},
+		.v_sample	= {4, 4, 4},
+		.colplanes	= 1,
+		.h_align	= 4,
+		.v_align	= 3,
+		.flags		= MTK_JPEG_FMT_FLAG_ENC_OUTPUT,
+	},
 };
 
 #define MTK_JPEG_NUM_FORMATS ARRAY_SIZE(mtk_jpeg_formats)

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c struct mtk_jpeg_src_buf { · line 110

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -0
@@ -65,11 +104,19 @@ struct mtk_jpeg_src_buf {
 	struct list_head list;
 	int flags;
 	struct mtk_jpeg_dec_param dec_param;
+	struct mtk_jpeg_enc_param enc_param;
 };
 
+#define MTK_MAX_CTRLS_HINT	20

I can see only 3 controls registered by this driver. Moreover, wouldn't it be more readable if the v4l2_ctrl_handler_init() call had "3" directly passed to it? Then the reader can see that there are 3 calls to v4l2_ctrl_new_std() just below it and the meaning behind the value of 3 is clear.

Quoted 40 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +22 -2
+
 static int debug;
 module_param(debug, int, 0644);
 
+static inline struct mtk_jpeg_ctx *ctrl_to_ctx(struct v4l2_ctrl *ctrl)
+{
+	return container_of(ctrl->handler, struct mtk_jpeg_ctx, ctrl_hdl);
+}
+
 static inline struct mtk_jpeg_ctx *mtk_jpeg_fh_to_ctx(struct v4l2_fh *fh)
 {
 	return container_of(fh, struct mtk_jpeg_ctx, fh);
@@ -86,14 +133,74 @@ static int mtk_jpeg_querycap(struct file *file, void *priv,
 {
 	struct mtk_jpeg_dev *jpeg = video_drvdata(file);
 
-	strscpy(cap->driver, MTK_JPEG_NAME " decoder", sizeof(cap->driver));
-	strscpy(cap->card, MTK_JPEG_NAME " decoder", sizeof(cap->card));
+	strscpy(cap->driver, MTK_JPEG_NAME, sizeof(cap->driver));
+	if (jpeg->mode ==  MTK_JPEG_ENC)
+		strscpy(cap->card, MTK_JPEG_NAME " encoder", sizeof(cap->card));
+	else
+		strscpy(cap->card, MTK_JPEG_NAME " decoder", sizeof(cap->card));
 	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
 		 dev_name(jpeg->dev));
 
 	return 0;
 }
 
+static int vidioc_jpeg_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_jpeg_ctx *ctx = ctrl_to_ctx(ctrl);
+	struct jpeg_enc_param *p = &ctx->jpeg_param;
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+	int ret = 0;
+
+	switch (ctrl->id) {
+	case V4L2_CID_JPEG_RESTART_INTERVAL:
+		p->restart_interval = ctrl->val;
+		break;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_querycap(struct file *file, void *priv, · line 167

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+	case V4L2_CID_JPEG_COMPRESSION_QUALITY:
+		p->enc_quality = ctrl->val;
+		break;
+	case V4L2_CID_JPEG_ACTIVE_MARKER:
+		p->enable_exif = ctrl->val & V4L2_JPEG_ACTIVE_MARKER_APP1;
+		break;
+	}
+
+	v4l2_dbg(2, debug, &jpeg->v4l2_dev, "%s val = %d",
+		 v4l2_ctrl_get_name(ctrl->id), ctrl->val);

Generic V4L2 logs can already dump all V4L2 IOCTL calls, so perhaps we could remove this?

Quoted 61 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +44 -0
+
+	return ret;
+}
+
+static const struct v4l2_ctrl_ops mtk_jpeg_ctrl_ops = {
+	.s_ctrl = vidioc_jpeg_s_ctrl,
+};
+
+int mtk_jpeg_ctrls_setup(struct mtk_jpeg_ctx *ctx)
+{
+	const struct v4l2_ctrl_ops *ops = &mtk_jpeg_ctrl_ops;
+	struct v4l2_ctrl_handler *handler = &ctx->ctrl_hdl;
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+
+	v4l2_ctrl_handler_init(handler, MTK_MAX_CTRLS_HINT);
+
+	if (jpeg->mode == MTK_JPEG_ENC) {
+		v4l2_ctrl_new_std(handler, ops, V4L2_CID_JPEG_RESTART_INTERVAL,
+				  0, 100, 1, 0);
+		v4l2_ctrl_new_std(handler, ops,
+				  V4L2_CID_JPEG_COMPRESSION_QUALITY, 48, 100, 1,
+				  90);
+		v4l2_ctrl_new_std(handler, ops, V4L2_CID_JPEG_ACTIVE_MARKER, 0,
+				  V4L2_JPEG_ACTIVE_MARKER_APP1, 0, 0);
+
+		if (handler->error) {
+			v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
+			return handler->error;
+		}
+	}
+
+	v4l2_ctrl_handler_setup(&ctx->ctrl_hdl);
+
+	return 0;
+}
+
 static int mtk_jpeg_enum_fmt(struct mtk_jpeg_fmt *mtk_jpeg_formats, int n,
 			     struct v4l2_fmtdesc *f, u32 type)
 {
@@ -118,23 +225,29 @@ static int mtk_jpeg_enum_fmt(struct mtk_jpeg_fmt *mtk_jpeg_formats, int n,
 static int mtk_jpeg_enum_fmt_vid_cap(struct file *file, void *priv,
 				     struct v4l2_fmtdesc *f)
 {
+	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+
 	return mtk_jpeg_enum_fmt(mtk_jpeg_formats, MTK_JPEG_NUM_FORMATS, f,
+				 ctx->jpeg->mode == MTK_JPEG_ENC ?
+				 MTK_JPEG_FMT_FLAG_ENC_CAPTURE :
 				 MTK_JPEG_FMT_FLAG_DEC_CAPTURE);
 }
 
 static int mtk_jpeg_enum_fmt_vid_out(struct file *file, void *priv,
 				     struct v4l2_fmtdesc *f)
 {
+	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+
 	return mtk_jpeg_enum_fmt(mtk_jpeg_formats, MTK_JPEG_NUM_FORMATS, f,
+				 ctx->jpeg->mode == MTK_JPEG_ENC ?
+				 MTK_JPEG_FMT_FLAG_ENC_OUTPUT :
 				 MTK_JPEG_FMT_FLAG_DEC_OUTPUT);
 }

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_enum_fmt(struct mtk_jpeg_fmt *mtk_jpeg_formats, int n, · line 250

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -5
 
-static struct mtk_jpeg_q_data *mtk_jpeg_get_q_data(struct mtk_jpeg_ctx *ctx,
-						   enum v4l2_buf_type type)
+static struct mtk_jpeg_q_data *
+mtk_jpeg_get_q_data(struct mtk_jpeg_ctx *ctx, enum v4l2_buf_type type)
 {
-	if (V4L2_TYPE_IS_OUTPUT(type))
-		return &ctx->out_q;
-	return &ctx->cap_q;
+	return V4L2_TYPE_IS_OUTPUT(type) ? &ctx->out_q : &ctx->cap_q;

Is this change necessary?

Quoted 27 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +8 -5
 }
 
 static struct mtk_jpeg_fmt *mtk_jpeg_find_format(struct mtk_jpeg_ctx *ctx,
@@ -143,9 +256,14 @@ static struct mtk_jpeg_fmt *mtk_jpeg_find_format(struct mtk_jpeg_ctx *ctx,
 {
 	unsigned int k, fmt_flag;
 
-	fmt_flag = (fmt_type == MTK_JPEG_FMT_TYPE_OUTPUT) ?
-		   MTK_JPEG_FMT_FLAG_DEC_OUTPUT :
-		   MTK_JPEG_FMT_FLAG_DEC_CAPTURE;
+	if (ctx->jpeg->mode ==  MTK_JPEG_ENC)
+		fmt_flag = (fmt_type == MTK_JPEG_FMT_TYPE_OUTPUT) ?
+			   MTK_JPEG_FMT_FLAG_ENC_OUTPUT :
+			   MTK_JPEG_FMT_FLAG_ENC_CAPTURE;
+	else
+		fmt_flag = (fmt_type == MTK_JPEG_FMT_TYPE_OUTPUT) ?
+			   MTK_JPEG_FMT_FLAG_DEC_OUTPUT :
+			   MTK_JPEG_FMT_FLAG_DEC_CAPTURE;
 
 	for (k = 0; k < MTK_JPEG_NUM_FORMATS; k++) {
 		struct mtk_jpeg_fmt *fmt = &mtk_jpeg_formats[k];
@@ -196,9 +314,99 @@ static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx,
 	}
 }
 
-static int mtk_jpeg_try_fmt_mplane(struct v4l2_format *f,
-				   struct mtk_jpeg_fmt *fmt,

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 325

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +9 -1
-				   struct mtk_jpeg_ctx *ctx, int q_type)
+static int mtk_jpeg_try_enc_fmt_mplane(struct v4l2_format *f,
+				       struct mtk_jpeg_fmt *fmt,
+				       struct mtk_jpeg_ctx *ctx, int q_type)
+{
+	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+	int i, align_w, align_h;
+
+	memset(pix_mp->reserved, 0, sizeof(pix_mp->reserved));

The core already takes care of zeroing this.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 336

Quoted 11 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +11 -0
+	pix_mp->field = V4L2_FIELD_NONE;
+
+	if (ctx->state != MTK_JPEG_INIT) {
+		mtk_jpeg_adjust_fmt_mplane(ctx, f);
+		goto end;
+	}
+
+	pix_mp->num_planes = fmt->colplanes;
+	pix_mp->pixelformat = fmt->fourcc;
+
+	if (q_type == MTK_JPEG_FMT_TYPE_OUTPUT) {

There are separate ops for out_mplane and cap_mplane. Wouldn't it make more sense to have them handled in separate functions?

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -0
+		pix_mp->height = clamp(pix_mp->height, MTK_JPEG_MIN_HEIGHT,
+				       MTK_JPEG_MAX_HEIGHT);
+		pix_mp->width = clamp(pix_mp->width, MTK_JPEG_MIN_WIDTH,
+				      MTK_JPEG_MAX_WIDTH);

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 350

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+		align_w = pix_mp->width;
+		align_h = pix_mp->height;
+		align_w = round_up(align_w, 2);
+		if (pix_mp->num_planes == 1U) {
+			mtk_jpeg_bound_align_image(&align_w,
+						   MTK_JPEG_MIN_WIDTH,
+						   MTK_JPEG_MAX_WIDTH, 5,
+						   &align_h,
+						   MTK_JPEG_MIN_HEIGHT,
+						   MTK_JPEG_MAX_HEIGHT, 3);

I'd suggest removing the mtk_jpeg_bound_align_image(), as we don't need the complex computation that's happening there. All we need is:

	pix_mp->height = clamp(round_up(pix_mp->height, 8), MTK_JPEG_MIN_HEIGHT,
			       MTK_JPEG_MAX_HEIGHT);
	pix_mp->width = clamp(round_up(pix_mp->width, 32), MTK_JPEG_MIN_WIDTH,
			      MTK_JPEG_MAX_WIDTH);

Note that contrary to the existing code, the aligned width and height _must_ be returned back to the application and not only hidden inside bytesperline and sizeimage.

Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -0
+			align_w = align_w << 1;
+			pix_mp->plane_fmt[0].bytesperline = align_w;
+			pix_mp->plane_fmt[0].sizeimage = align_w * align_h;
+		} else if (pix_mp->num_planes == 2U) {
+			mtk_jpeg_bound_align_image(&align_w,

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 365

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+						   MTK_JPEG_MIN_WIDTH,
+						   MTK_JPEG_MAX_WIDTH, 4,
+						   &align_h,
+						   MTK_JPEG_MIN_HEIGHT,
+						   MTK_JPEG_MAX_HEIGHT, 4);
+			pix_mp->plane_fmt[0].bytesperline = align_w;
+			pix_mp->plane_fmt[0].sizeimage = align_w * align_h;
+			pix_mp->plane_fmt[1].bytesperline = align_w;
+			pix_mp->plane_fmt[1].sizeimage =
+				(align_w * align_h) / 2;

Could we parametrize these calculations from the format array and use a for loop over the planes instead?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 369

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -0
+		} else {
+			v4l2_err(&jpeg->v4l2_dev,
+				 "Unsupport num planes = %d\n",
+				 pix_mp->num_planes);

Isn't this impossible?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 376

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +7 -0
+		}
+	} else {
+		mtk_jpeg_bound_align_image(&pix_mp->width, MTK_JPEG_MIN_WIDTH,
+					   MTK_JPEG_MAX_WIDTH, 0,
+					   &pix_mp->height,
+					   MTK_JPEG_MIN_HEIGHT,
+					   MTK_JPEG_MAX_HEIGHT, 0);

Same here, could we just use clamp() directly?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 378

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -0
+
+		if (fmt->fourcc == V4L2_PIX_FMT_JPEG) {

Can we have any other format on this queue?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 382

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -0
+			pix_mp->plane_fmt[0].bytesperline = 0;
+			pix_mp->plane_fmt[0].sizeimage =
+				mtk_jpeg_align(pix_mp->plane_fmt[0].sizeimage,
+					       128);

What's mtk_jpeg_align()? Why couldn't we just use round_up() directly?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 385

Quoted 3 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -0
+			if (pix_mp->plane_fmt[0].sizeimage == 0)
+				pix_mp->plane_fmt[0].sizeimage =
+					MTK_JPEG_DEFAULT_SIZEIMAGE;

I think there is some reasonable upper bound of the compressed image size given the resolution. Could we use that here instead?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 392

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +7 -0
+		}
+	}
+
+	for (i = 0; i < fmt->colplanes; i++) {
+		struct v4l2_plane_pix_format *pfmt =
+				&pix_mp->plane_fmt[i];
+		memset(pfmt->reserved, 0, sizeof(pfmt->reserved));

The core already zeroes this.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 403

Quoted 11 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +11 -0
+	}
+end:
+	v4l2_dbg(2, debug, &jpeg->v4l2_dev, "wxh:%ux%u\n",
+		 pix_mp->width, pix_mp->height);
+	for (i = 0; i < pix_mp->num_planes; i++) {
+		v4l2_dbg(2, debug, &jpeg->v4l2_dev,
+			 "plane[%d] bpl=%u, size=%u\n",
+			 i,
+			 pix_mp->plane_fmt[i].bytesperline,
+			 pix_mp->plane_fmt[i].sizeimage);
+	}

Generic V4L2 debug logs already print this.

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
+	return 0;
+}
+
+static int mtk_jpeg_try_dec_fmt_mplane(struct v4l2_format *f,
+				       struct mtk_jpeg_fmt *fmt,
+				       struct mtk_jpeg_ctx *ctx, int q_type)
 {
 	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
 	struct mtk_jpeg_dev *jpeg = ctx->jpeg;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_try_fmt_mplane(struct v4l2_format *f, · line 447

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -0
@@ -233,6 +441,10 @@ static int mtk_jpeg_try_fmt_mplane(struct v4l2_format *f,
 	}
 
 	/* type is MTK_JPEG_FMT_TYPE_CAPTURE */
+	pix_mp->height = clamp(pix_mp->height, MTK_JPEG_MIN_HEIGHT,
+			       MTK_JPEG_MAX_HEIGHT);
+	pix_mp->width = clamp(pix_mp->width, MTK_JPEG_MIN_WIDTH,
+			      MTK_JPEG_MAX_WIDTH);

Why is this needed? Doesn't the call below already include these clamps?

(That said, it would be cleaner if mtk_jpeg_bound_align_image() was removed and replaced with clamp() calls directly, because of the confusing behavior of v4l_bound_align_image().

Quoted 12 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -0
 	mtk_jpeg_bound_align_image(&pix_mp->width, MTK_JPEG_MIN_WIDTH,
 				   MTK_JPEG_MAX_WIDTH, fmt->h_align,
 				   &pix_mp->height, MTK_JPEG_MIN_HEIGHT,
@@ -316,6 +528,7 @@ static int mtk_jpeg_try_fmt_vid_cap_mplane(struct file *file, void *priv,
 {
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
 	struct mtk_jpeg_fmt *fmt;
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
 
 	fmt = mtk_jpeg_find_format(ctx, f->fmt.pix_mp.pixelformat,
 				   MTK_JPEG_FMT_TYPE_CAPTURE);

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_try_fmt_vid_cap_mplane(struct file *file, void *priv, · line 550

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -1
@@ -329,7 +542,12 @@ static int mtk_jpeg_try_fmt_vid_cap_mplane(struct file *file, void *priv,
 		 (fmt->fourcc >> 16 & 0xff),
 		 (fmt->fourcc >> 24 & 0xff));
 
-	return mtk_jpeg_try_fmt_mplane(f, fmt, ctx, MTK_JPEG_FMT_TYPE_CAPTURE);
+
+	return jpeg->mode == MTK_JPEG_ENC ?
+	       mtk_jpeg_try_enc_fmt_mplane(f, fmt, ctx,
+					   MTK_JPEG_FMT_TYPE_CAPTURE) :
+	       mtk_jpeg_try_dec_fmt_mplane(f, fmt, ctx,
+					   MTK_JPEG_FMT_TYPE_CAPTURE);

The ternary operator isn't a universal replacement for if/else. It should be limited to simple expressions only. Function calls don't count as simple expressions.

Quoted 12 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -0
 }
 
 static int mtk_jpeg_try_fmt_vid_out_mplane(struct file *file, void *priv,
@@ -337,6 +555,7 @@ static int mtk_jpeg_try_fmt_vid_out_mplane(struct file *file, void *priv,
 {
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
 	struct mtk_jpeg_fmt *fmt;
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
 
 	fmt = mtk_jpeg_find_format(ctx, f->fmt.pix_mp.pixelformat,
 				   MTK_JPEG_FMT_TYPE_OUTPUT);

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_try_fmt_vid_out_mplane(struct file *file, void *priv, · line 576

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -1
@@ -350,7 +569,12 @@ static int mtk_jpeg_try_fmt_vid_out_mplane(struct file *file, void *priv,
 		 (fmt->fourcc >> 16 & 0xff),
 		 (fmt->fourcc >> 24 & 0xff));
 
-	return mtk_jpeg_try_fmt_mplane(f, fmt, ctx, MTK_JPEG_FMT_TYPE_OUTPUT);
+	return jpeg->mode == MTK_JPEG_ENC ?
+	       mtk_jpeg_try_enc_fmt_mplane(f, fmt, ctx,
+					   MTK_JPEG_FMT_TYPE_OUTPUT) :
+	       mtk_jpeg_try_dec_fmt_mplane(f, fmt, ctx,
+					   MTK_JPEG_FMT_TYPE_OUTPUT);

Ditto.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_try_fmt_vid_out_mplane(struct file *file, void *priv, · line 577

Quoted 1 line
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -0
+

Unnecessary blank line.

Quoted 61 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +30 -2
 }
 
 static int mtk_jpeg_s_fmt_mplane(struct mtk_jpeg_ctx *ctx,
@@ -455,11 +679,19 @@ static int mtk_jpeg_g_selection(struct file *file, void *priv,
 				struct v4l2_selection *s)
 {
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
 
-	if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+	if (jpeg->mode == MTK_JPEG_ENC && s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)
+		return -EINVAL;
+
+	if (jpeg->mode == MTK_JPEG_DEC &&
+	    s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
 		return -EINVAL;
 
 	switch (s->target) {
+	case V4L2_SEL_TGT_CROP:
+	case V4L2_SEL_TGT_CROP_BOUNDS:
+	case V4L2_SEL_TGT_CROP_DEFAULT:
 	case V4L2_SEL_TGT_COMPOSE:
 	case V4L2_SEL_TGT_COMPOSE_DEFAULT:
 		s->r.width = ctx->out_q.w;
@@ -484,11 +716,17 @@ static int mtk_jpeg_s_selection(struct file *file, void *priv,
 				struct v4l2_selection *s)
 {
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
 
-	if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+	if (jpeg->mode == MTK_JPEG_ENC && s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)
+		return -EINVAL;
+
+	if (jpeg->mode == MTK_JPEG_DEC &&
+	    s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
 		return -EINVAL;
 
 	switch (s->target) {
+	case V4L2_SEL_TGT_CROP:
 	case V4L2_SEL_TGT_COMPOSE:
 		s->r.left = 0;
 		s->r.top = 0;
@@ -658,10 +896,92 @@ static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx,
 		 param->dec_w, param->dec_h);
 }
 
+static void mtk_jpeg_set_param(struct mtk_jpeg_ctx *ctx,
+			       struct mtk_jpeg_enc_param *param)
+{
+	struct mtk_jpeg_q_data *q_data_src = &ctx->out_q;
+	struct jpeg_enc_param *jpeg_params = &ctx->jpeg_param;
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+	u32 width_even;
+	u32 is_420;
+	u32 padding_width;
+	u32 padding_height;
+
+	switch (q_data_src->fmt->fourcc) {
+	case V4L2_PIX_FMT_YUYV:
+		param->enc_format = JPEG_YUV_FORMAT_YUYV;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx, · line 922

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+		break;
+	case V4L2_PIX_FMT_YVYU:
+		param->enc_format = JPEG_YUV_FORMAT_YVYU;
+		break;
+	case V4L2_PIX_FMT_NV12M:
+		param->enc_format = JPEG_YUV_FORMAT_NV12;
+		break;
+	case V4L2_PIX_FMT_NV21M:
+		param->enc_format = JPEG_YUV_FORMAT_NV12;
+		break;

Could we include enc_format in the mtk_jpeg_fmt struct and take it from there, eliminating the need for a switch over all supported formats?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx, · line 926

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -0
+	default:
+		v4l2_err(&jpeg->v4l2_dev, "Unsupport fourcc =%d\n",
+			 q_data_src->fmt->fourcc);
+		break;

Isn't this impossible?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx, · line 934

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +8 -0
+	}
+	param->enc_w = q_data_src->w;
+	param->enc_h = q_data_src->h;
+
+	if (jpeg_params->enc_quality >= 97)
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q97;
+	else if (jpeg_params->enc_quality >= 95)
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q95;

I'm wondering if the application requests 96, it doesn't expect the quality to be _at_least_ 96.

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +8 -0
+	else if (jpeg_params->enc_quality >= 92)
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q92;
+	else if (jpeg_params->enc_quality >= 90)
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q90;
+	else if (jpeg_params->enc_quality >= 87)
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q87;
+	else if (jpeg_params->enc_quality >= 84)
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q84;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx, · line 952

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+	else if (jpeg_params->enc_quality >= 80)
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q80;
+	else if (jpeg_params->enc_quality >= 74)
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q74;
+	else if (jpeg_params->enc_quality >= 64)
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q64;
+	else if (jpeg_params->enc_quality >= 60)
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q60;
+	else
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q48;

We could put this in a sorted array of <quality paramter, hardware value> pairs and use a simple for loop instead.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx, · line 957

Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -0
+
+	param->enable_exif = jpeg_params->enable_exif;
+	param->restart_interval = jpeg_params->restart_interval;
+
+	width_even = ((param->enc_w + 1) >> 1) << 1;

Isn't this equivalent to round_up(param->enc_w, 2)?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx, · line 965

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +8 -0
+	is_420 = (param->enc_format == JPEG_YUV_FORMAT_NV12 ||
+		  param->enc_format == JPEG_YUV_FORMAT_NV12) ? 1 : 0;
+	padding_width = mtk_jpeg_align(param->enc_w, 16);
+	padding_height = mtk_jpeg_align(param->enc_h, is_420 ? 16 : 8);
+	if (!is_420)
+		width_even = width_even << 1;
+
+	param->img_stride = mtk_jpeg_align(width_even, (is_420 ? 16 : 32));

Isn't this the same as bytesperline in the v4l2_pix_fmt_mplane struct?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx, · line 966

Quoted 1 line
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -0
+	param->mem_stride = mtk_jpeg_align(width_even, (is_420 ? 16 : 32));

What's the difference between img_stride and mem_stride?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx, · line 969

Quoted 3 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -0
+	param->total_encdu =
+		((padding_width >> 4) * (padding_height >> (is_420 ? 4 : 3)) *
+		(is_420 ? 6 : 4)) - 1;

The comment above the struct says this is the total number of 8x8 blocks. Why would it depend on whether the format is YUV 4:2:0? Since we should have already aligned the width and height in try_fmt, this should be as simple as (width / 8) * (height / 8).

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx, · line 978

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +9 -0
+
+	v4l2_dbg(0, 2, &jpeg->v4l2_dev, "fmt %d, w,h %d,%d, enable_exif %d,",
+		 "enc_quality %d, restart_interval %d,img_stride %d,",
+		 "mem_stride %d,totalEncDu %d\n",
+		 param->enc_format, param->enc_w, param->enc_h,
+		 param->enable_exif, param->enc_quality,
+		 param->restart_interval, param->img_stride,
+		 param->mem_stride, param->total_encdu);
+}

Do we need this intermediate step, with an intermediate representation of the parameters? Couldn't we just program the registers directly based on the original V4L2 values, as other drivers normally do?

Quoted 22 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -9
+
 static void mtk_jpeg_buf_queue(struct vb2_buffer *vb)
 {
 	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
 	struct mtk_jpeg_dec_param *param;
+	struct mtk_jpeg_enc_param *enc_param;
 	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
 	struct mtk_jpeg_src_buf *jpeg_src_buf;
 	bool header_valid;
@@ -673,29 +993,45 @@ static void mtk_jpeg_buf_queue(struct vb2_buffer *vb)
 		goto end;
 
 	jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(vb);
-	param = &jpeg_src_buf->dec_param;
-	memset(param, 0, sizeof(*param));
-
-	if (jpeg_src_buf->flags & MTK_JPEG_BUF_FLAGS_LAST_FRAME) {
-		v4l2_dbg(1, debug, &jpeg->v4l2_dev, "Got eos\n");
-		goto end;
-	}
-	header_valid = mtk_jpeg_parse(param, (u8 *)vb2_plane_vaddr(vb, 0),
-				      vb2_get_plane_payload(vb, 0));

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_buf_queue(struct vb2_buffer *vb) · line 996

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -9
-	if (!header_valid) {
-		v4l2_err(&jpeg->v4l2_dev, "Header invalid.\n");
-		vb2_buffer_done(vb, VB2_BUF_STATE_ERROR);
-		return;
-	}
-
-	if (ctx->state == MTK_JPEG_INIT) {
-		struct vb2_queue *dst_vq = v4l2_m2m_get_vq(
-			ctx->fh.m2m_ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
+	if (jpeg->mode ==  MTK_JPEG_ENC) {

nit: Double space after ==.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_buf_queue(struct vb2_buffer *vb) · line 1002

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
+		enc_param = &jpeg_src_buf->enc_param;
+		memset(enc_param, 0, sizeof(*enc_param));
+		mtk_jpeg_set_param(ctx, enc_param);
+		if (jpeg_src_buf->flags & MTK_JPEG_BUF_FLAGS_LAST_FRAME) {
+			v4l2_dbg(1, debug, &jpeg->v4l2_dev, "Got eos");
+			goto end;

Shouldn't we still change the state to RUNNING? The last frame still needs to be encoded and the condition for getting back to the INIT state is stopping the streaming.

Quoted 61 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +34 -7
+		}
+		if (ctx->state == MTK_JPEG_INIT)
+			ctx->state = MTK_JPEG_RUNNING;
+	} else {
+		param = &jpeg_src_buf->dec_param;
+		memset(param, 0, sizeof(*param));
+
+		if (jpeg_src_buf->flags & MTK_JPEG_BUF_FLAGS_LAST_FRAME) {
+			v4l2_dbg(1, debug, &jpeg->v4l2_dev, "Got eos\n");
+			goto end;
+		}
+		header_valid = mtk_jpeg_parse(param,
+					      (u8 *)vb2_plane_vaddr(vb, 0),
+					      vb2_get_plane_payload(vb, 0));
+		if (!header_valid) {
+			v4l2_err(&jpeg->v4l2_dev, "Header invalid.\n");
+			vb2_buffer_done(vb, VB2_BUF_STATE_ERROR);
+			return;
+		}
 
-		mtk_jpeg_queue_src_chg_event(ctx);
-		mtk_jpeg_set_queue_data(ctx, param);
-		ctx->state = vb2_is_streaming(dst_vq) ?
-				MTK_JPEG_SOURCE_CHANGE : MTK_JPEG_RUNNING;
+		if (ctx->state == MTK_JPEG_INIT) {
+			struct vb2_queue *dst_vq;
+
+			dst_vq = v4l2_m2m_get_vq
+					(ctx->fh.m2m_ctx,
+					 V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
+			mtk_jpeg_queue_src_chg_event(ctx);
+			mtk_jpeg_set_queue_data(ctx, param);
+			ctx->state = vb2_is_streaming(dst_vq) ?
+					MTK_JPEG_SOURCE_CHANGE :
+					MTK_JPEG_RUNNING;
+		}
 	}
 end:
 	v4l2_m2m_buf_queue(ctx->fh.m2m_ctx, to_vb2_v4l2_buffer(vb));
@@ -738,16 +1074,16 @@ static void mtk_jpeg_stop_streaming(struct vb2_queue *q)
 	 * subsampling. Update capture queue when the stream is off.
 	 */
 	if (ctx->state == MTK_JPEG_SOURCE_CHANGE &&
-	    !V4L2_TYPE_IS_OUTPUT(q->type)) {
+	    !V4L2_TYPE_IS_OUTPUT(q->type) &&
+	    ctx->jpeg->mode == MTK_JPEG_DEC) {
 		struct mtk_jpeg_src_buf *src_buf;
 
 		vb = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
 		src_buf = mtk_jpeg_vb2_to_srcbuf(&vb->vb2_buf);
 		mtk_jpeg_set_queue_data(ctx, &src_buf->dec_param);
 		ctx->state = MTK_JPEG_RUNNING;
-	} else if (V4L2_TYPE_IS_OUTPUT(q->type)) {
+	} else if (V4L2_TYPE_IS_OUTPUT(q->type))
 		ctx->state = MTK_JPEG_INIT;
-	}
 
 	while ((vb = mtk_jpeg_buf_remove(ctx, q->type)))
 		v4l2_m2m_buf_done(vb, VB2_BUF_STATE_ERROR);
@@ -802,6 +1138,34 @@ static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx,
 	return 0;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx, · line 1148

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +8 -0
 }
 
+static void mtk_jpeg_set_enc_dst(struct mtk_jpeg_ctx *ctx,
+				 struct vb2_buffer *dst_buf,
+				 struct mtk_jpeg_enc_bs *bs)
+{
+	struct jpeg_enc_param *p = &ctx->jpeg_param;
+
+	bs->dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0) &
+		      (~JPEG_ENC_DST_ADDR_OFFSET_MASK);

Wouldn't the hardware attempt to access the memory before the buffer in this case? This is a recipe for a disaster (e.g. the hardware overwriting some memory it shouldn't touch).

Could we instead check the buffer address alignment in .buf_prepare and fail if it's not big enough?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx, · line 1150

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -0
+	bs->dma_addr_offset = p->enable_exif ? MTK_JPEG_DEFAULT_EXIF_SIZE : 0;
+	bs->dma_addr_offsetmask = bs->dma_addr & JPEG_ENC_DST_ADDR_OFFSET_MASK;

What is the meaning of this offset mask?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx, · line 1151

Quoted 1 line
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -0
+	bs->size = mtk_jpeg_align(vb2_plane_size(dst_buf, 0), 128);

Wouldn't this let the hardware write beyond the buffer if the original size was not a multiple of 128? Shouldn't we round_down() insted?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx, · line 1164

Quoted 13 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +13 -0
+}
+
+static void mtk_jpeg_set_enc_src(struct mtk_jpeg_ctx *ctx,
+				 struct vb2_buffer *src_buf,
+				 struct mtk_jpeg_enc_fb *fb)
+{
+	int i;
+
+	for (i = 0; i < src_buf->num_planes; i++) {
+		fb->fb_addr[i].dma_addr =
+			vb2_dma_contig_plane_dma_addr(src_buf, i) +
+			src_buf->planes[i].data_offset;
+		fb->fb_addr[i].size = (size_t)src_buf->planes[i].length -

The proper way to access plane length is to call vb2_plane_size().

Quoted 25 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +7 -2
+				      src_buf->planes[i].data_offset;
+	}
+}
+
 static void mtk_jpeg_device_run(void *priv)
 {
 	struct mtk_jpeg_ctx *ctx = priv;
@@ -812,6 +1176,8 @@ static void mtk_jpeg_device_run(void *priv)
 	struct mtk_jpeg_src_buf *jpeg_src_buf;
 	struct mtk_jpeg_bs bs;
 	struct mtk_jpeg_fb fb;
+	struct mtk_jpeg_enc_bs enc_bs;
+	struct mtk_jpeg_enc_fb enc_fb;
 	int i;
 
 	src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
@@ -822,30 +1188,45 @@ static void mtk_jpeg_device_run(void *priv)
 		for (i = 0; i < dst_buf->vb2_buf.num_planes; i++)
 			vb2_set_plane_payload(&dst_buf->vb2_buf, i, 0);
 		buf_state = VB2_BUF_STATE_DONE;
-		goto dec_end;
+		goto device_run_end;
 	}
 
-	if (mtk_jpeg_check_resolution_change(ctx, &jpeg_src_buf->dec_param)) {

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_device_run(void *priv) · line 1198

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -5
-		mtk_jpeg_queue_src_chg_event(ctx);
-		ctx->state = MTK_JPEG_SOURCE_CHANGE;
-		v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
-		return;
-	}
+	if (jpeg->mode == MTK_JPEG_ENC) {
+		mtk_jpeg_set_enc_dst(ctx, &dst_buf->vb2_buf, &enc_bs);
+		mtk_jpeg_set_enc_src(ctx, &src_buf->vb2_buf, &enc_fb);
+
+		spin_lock_irqsave(&jpeg->hw_lock, flags);

Do we actually need this lock? device_run can be only called once and only after the hardware finished processing the current job.

Quoted 26 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +18 -6
+		mtk_jpeg_enc_reset(jpeg->reg_base);
+		mtk_jpeg_enc_set_config(jpeg->reg_base,
+					&jpeg_src_buf->enc_param, &enc_bs,
+					&enc_fb);
+
+		mtk_jpeg_enc_start(jpeg->reg_base);
+	} else {
+		if (mtk_jpeg_check_resolution_change
+			(ctx, &jpeg_src_buf->dec_param)) {
+			mtk_jpeg_queue_src_chg_event(ctx);
+			ctx->state = MTK_JPEG_SOURCE_CHANGE;
+			v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
+			return;
+		}
 
-	mtk_jpeg_set_dec_src(ctx, &src_buf->vb2_buf, &bs);
-	if (mtk_jpeg_set_dec_dst(ctx, &jpeg_src_buf->dec_param, &dst_buf->vb2_buf, &fb))
-		goto dec_end;
+		mtk_jpeg_set_dec_src(ctx, &src_buf->vb2_buf, &bs);
+		if (mtk_jpeg_set_dec_dst(ctx, &jpeg_src_buf->dec_param,
+					 &dst_buf->vb2_buf, &fb))
+			goto device_run_end;
 
-	spin_lock_irqsave(&jpeg->hw_lock, flags);
-	mtk_jpeg_dec_reset(jpeg->dec_reg_base);
-	mtk_jpeg_dec_set_config(jpeg->dec_reg_base,

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_device_run(void *priv) · line 1226

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -2
-				&jpeg_src_buf->dec_param, &bs, &fb);
+		spin_lock_irqsave(&jpeg->hw_lock, flags);
+		mtk_jpeg_dec_reset(jpeg->reg_base);
+		mtk_jpeg_dec_set_config(jpeg->reg_base,
+					&jpeg_src_buf->dec_param, &bs, &fb);
 
-	mtk_jpeg_dec_start(jpeg->dec_reg_base);
+		mtk_jpeg_dec_start(jpeg->reg_base);
+	}
 	spin_unlock_irqrestore(&jpeg->hw_lock, flags);

It is a good practice for spin_lock and spin_unlock to be in the same scope, for readability reasons.

Quoted 29 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +7 -6
 	return;
 
-dec_end:
+device_run_end:
 	v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
 	v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
 	v4l2_m2m_buf_done(src_buf, buf_state);
@@ -905,30 +1286,30 @@ static void mtk_jpeg_clk_on(struct mtk_jpeg_dev *jpeg)
 	ret = mtk_smi_larb_get(jpeg->larb);
 	if (ret)
 		dev_err(jpeg->dev, "mtk_smi_larb_get larbvdec fail %d\n", ret);
-	clk_prepare_enable(jpeg->clk_jdec_smi);
-	clk_prepare_enable(jpeg->clk_jdec);
+	if (jpeg->mode == MTK_JPEG_DEC)
+		clk_prepare_enable(jpeg->clk_jpeg_smi);
+	clk_prepare_enable(jpeg->clk_jpeg);
 }
 
 static void mtk_jpeg_clk_off(struct mtk_jpeg_dev *jpeg)
 {
-	clk_disable_unprepare(jpeg->clk_jdec);
-	clk_disable_unprepare(jpeg->clk_jdec_smi);
+	clk_disable_unprepare(jpeg->clk_jpeg);
+	if (jpeg->mode == MTK_JPEG_DEC)
+		clk_disable_unprepare(jpeg->clk_jpeg_smi);
 	mtk_smi_larb_put(jpeg->larb);
 }
 
-static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv)

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_clk_on(struct mtk_jpeg_dev *jpeg) · line 1309

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -2
+static irqreturn_t mtk_jpeg_irq(int irq, void *priv)
 {
 	struct mtk_jpeg_dev *jpeg = priv;
 	struct mtk_jpeg_ctx *ctx;
 	struct vb2_v4l2_buffer *src_buf, *dst_buf;
 	struct mtk_jpeg_src_buf *jpeg_src_buf;
 	enum vb2_buffer_state buf_state = VB2_BUF_STATE_ERROR;
-	u32	dec_irq_ret;
-	u32 dec_ret;
+	u32	irq_ret;

In the kernel coding style, type is separated from variable name with a single space.

Quoted 99 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +66 -13
+	u32 ret, result_size;
 	int i;
 
-	dec_ret = mtk_jpeg_dec_get_int_status(jpeg->dec_reg_base);
-	dec_irq_ret = mtk_jpeg_dec_enum_result(dec_ret);
 	ctx = v4l2_m2m_get_curr_priv(jpeg->m2m_dev);
 	if (!ctx) {
 		v4l2_err(&jpeg->v4l2_dev, "Context is NULL\n");
@@ -939,39 +1320,107 @@ static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv)
 	dst_buf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
 	jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(&src_buf->vb2_buf);
 
-	if (dec_irq_ret >= MTK_JPEG_DEC_RESULT_UNDERFLOW)
-		mtk_jpeg_dec_reset(jpeg->dec_reg_base);
+	if (jpeg->mode == MTK_JPEG_ENC) {
+		ret = mtk_jpeg_enc_get_int_status(jpeg->reg_base);
+		irq_ret = mtk_jpeg_enc_enum_result(jpeg->reg_base, ret,
+						   &result_size);
 
-	if (dec_irq_ret != MTK_JPEG_DEC_RESULT_EOF_DONE) {
-		dev_err(jpeg->dev, "decode failed\n");
-		goto dec_end;
-	}
+		if (irq_ret >= MTK_JPEG_ENC_RESULT_STALL)
+			mtk_jpeg_enc_reset(jpeg->reg_base);
+
+		if (irq_ret != MTK_JPEG_ENC_RESULT_DONE) {
+			dev_err(jpeg->dev, "encode failed\n");
+			goto irq_end;
+		}
+
+		vb2_set_plane_payload(&dst_buf->vb2_buf, 0,
+				      result_size);
+	} else {
+		ret = mtk_jpeg_dec_get_int_status(jpeg->reg_base);
+		irq_ret = mtk_jpeg_dec_enum_result(ret);
 
-	for (i = 0; i < dst_buf->vb2_buf.num_planes; i++)
-		vb2_set_plane_payload(&dst_buf->vb2_buf, i,
-				      jpeg_src_buf->dec_param.comp_size[i]);
+		if (irq_ret >= MTK_JPEG_DEC_RESULT_UNDERFLOW)
+			mtk_jpeg_dec_reset(jpeg->reg_base);
+
+		if (irq_ret != MTK_JPEG_DEC_RESULT_EOF_DONE) {
+			dev_err(jpeg->dev, "decode failed\n");
+			goto irq_end;
+		}
+
+		for (i = 0; i < dst_buf->vb2_buf.num_planes; i++)
+			vb2_set_plane_payload
+				(&dst_buf->vb2_buf, i,
+				 jpeg_src_buf->dec_param.comp_size[i]);
+	}
 
 	buf_state = VB2_BUF_STATE_DONE;
 
-dec_end:
+irq_end:
 	v4l2_m2m_buf_done(src_buf, buf_state);
 	v4l2_m2m_buf_done(dst_buf, buf_state);
 	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
 	return IRQ_HANDLED;
 }
 
-static void mtk_jpeg_set_default_params(struct mtk_jpeg_ctx *ctx)
+static void mtk_jpeg_set_enc_default_params(struct mtk_jpeg_ctx *ctx)
+{
+	struct mtk_jpeg_q_data *q = &ctx->out_q;
+	int align_w, align_h;
+
+	ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+
+	ctx->colorspace = V4L2_COLORSPACE_JPEG,
+	ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
+	ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+
+	q->w = MTK_JPEG_MIN_WIDTH;
+	q->h = MTK_JPEG_MIN_HEIGHT;
+
+	q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_YUYV,
+				      MTK_JPEG_FMT_TYPE_OUTPUT);
+
+	align_w = q->w;
+	align_h = q->h;
+	align_w = round_up(align_w, 2);
+	v4l_bound_align_image(&align_w, MTK_JPEG_MIN_WIDTH, MTK_JPEG_MAX_WIDTH,
+			      5, &align_h, MTK_JPEG_MIN_HEIGHT,
+			      MTK_JPEG_MAX_HEIGHT, 3, 0);
+	align_w = align_w << 1;
+
+	if (align_w < MTK_JPEG_MIN_WIDTH &&
+	    (align_w + 32) <= MTK_JPEG_MAX_WIDTH)
+		align_w += 32;
+	if (align_h < MTK_JPEG_MIN_HEIGHT &&
+	    (align_h + 8) <= MTK_JPEG_MAX_HEIGHT)
+		align_h += 8;
+
+	q->sizeimage[0] = align_w * align_h;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1408

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+	q->bytesperline[0] = align_w;
+
+	q = &ctx->cap_q;
+	q->w = MTK_JPEG_MIN_WIDTH;
+	q->h = MTK_JPEG_MIN_HEIGHT;
+	q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_JPEG,
+				      MTK_JPEG_FMT_TYPE_CAPTURE);
+	q->bytesperline[0] = 0;
+	q->sizeimage[0] = MTK_JPEG_DEFAULT_SIZEIMAGE;
+}

Could we just create an arbitrary v4l2_pix_format_mplane struct and call s_fmt instead? In general, all of the constant values and alignments should be already ensured by try_fmt, so this function should be redundant.

Quoted 32 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -2
+
+static void mtk_jpeg_set_dec_default_params(struct mtk_jpeg_ctx *ctx)
 {
 	struct mtk_jpeg_q_data *q = &ctx->out_q;
 	int i;
 
+	ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+
 	ctx->colorspace = V4L2_COLORSPACE_JPEG,
 	ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
 	ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
 	ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;
 
 	q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_JPEG,
-					      MTK_JPEG_FMT_TYPE_OUTPUT);
+				      MTK_JPEG_FMT_TYPE_OUTPUT);
 	q->w = MTK_JPEG_MIN_WIDTH;
 	q->h = MTK_JPEG_MIN_HEIGHT;
 	q->bytesperline[0] = 0;
@@ -979,7 +1428,7 @@ static void mtk_jpeg_set_default_params(struct mtk_jpeg_ctx *ctx)
 
 	q = &ctx->cap_q;
 	q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_YUV420M,
-					      MTK_JPEG_FMT_TYPE_CAPTURE);
+				      MTK_JPEG_FMT_TYPE_CAPTURE);
 	q->w = MTK_JPEG_MIN_WIDTH;
 	q->h = MTK_JPEG_MIN_HEIGHT;
 
@@ -1020,7 +1469,15 @@ static int mtk_jpeg_open(struct file *file)
 		goto error;
 	}
 

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_open(struct file *file) · line 1480

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +9 -1
-	mtk_jpeg_set_default_params(ctx);
+	ret = mtk_jpeg_ctrls_setup(ctx);
+	if (ret) {
+		v4l2_err(&jpeg->v4l2_dev, "Failed to setup controls() (%d)\n",
+			 ret);
+		goto error;
+	}
+
+	jpeg->mode == MTK_JPEG_ENC ? mtk_jpeg_set_enc_default_params(ctx) :
+				     mtk_jpeg_set_dec_default_params(ctx);

As mentioned before, this kind of usage of the ternary operator is generaly discouraged, because it makes the code difficult to read.

Quoted 12 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -0
 	mutex_unlock(&jpeg->lock);
 	return 0;
 
@@ -1040,6 +1497,7 @@ static int mtk_jpeg_release(struct file *file)
 
 	mutex_lock(&jpeg->lock);
 	v4l2_m2m_ctx_release(ctx->fh.m2m_ctx);
+	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
 	v4l2_fh_del(&ctx->fh);
 	v4l2_fh_exit(&ctx->fh);
 	kfree(ctx);

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_clk_init(struct mtk_jpeg_dev *jpeg) · line 1525

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -0
@@ -1064,6 +1522,7 @@ static int mtk_jpeg_clk_init(struct mtk_jpeg_dev *jpeg)
 	node = of_parse_phandle(jpeg->dev->of_node, "mediatek,larb", 0);
 	if (!node)
 		return -EINVAL;
+

Unnecessary change?

Quoted 59 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +18 -15
 	pdev = of_find_device_by_node(node);
 	if (WARN_ON(!pdev)) {
 		of_node_put(node);
@@ -1073,19 +1532,24 @@ static int mtk_jpeg_clk_init(struct mtk_jpeg_dev *jpeg)
 
 	jpeg->larb = &pdev->dev;
 
-	jpeg->clk_jdec = devm_clk_get(jpeg->dev, "jpgdec");
-	if (IS_ERR(jpeg->clk_jdec))
-		return PTR_ERR(jpeg->clk_jdec);
+	if (jpeg->mode == MTK_JPEG_ENC) {
+		jpeg->clk_jpeg = devm_clk_get(jpeg->dev, "jpgenc");
+		return PTR_ERR_OR_ZERO(jpeg->clk_jpeg);
+	}
+
+	jpeg->clk_jpeg = devm_clk_get(jpeg->dev, "jpgdec");
+	if (IS_ERR(jpeg->clk_jpeg))
+		return PTR_ERR(jpeg->clk_jpeg);
 
-	jpeg->clk_jdec_smi = devm_clk_get(jpeg->dev, "jpgdec-smi");
-	return PTR_ERR_OR_ZERO(jpeg->clk_jdec_smi);
+	jpeg->clk_jpeg_smi = devm_clk_get(jpeg->dev, "jpgdec-smi");
+	return PTR_ERR_OR_ZERO(jpeg->clk_jpeg_smi);
 }
 
 static int mtk_jpeg_probe(struct platform_device *pdev)
 {
 	struct mtk_jpeg_dev *jpeg;
 	struct resource *res;
-	int dec_irq;
+	int jpeg_irq;
 	int ret;
 
 	jpeg = devm_kzalloc(&pdev->dev, sizeof(*jpeg), GFP_KERNEL);
@@ -1095,28 +1559,26 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 	mutex_init(&jpeg->lock);
 	spin_lock_init(&jpeg->hw_lock);
 	jpeg->dev = &pdev->dev;
+	jpeg->mode = (enum mtk_jpeg_mode)of_device_get_match_data(jpeg->dev);
 
 	res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
-	jpeg->dec_reg_base = devm_ioremap_resource(&pdev->dev, res);
-	if (IS_ERR(jpeg->dec_reg_base)) {
-		ret = PTR_ERR(jpeg->dec_reg_base);
+	jpeg->reg_base = devm_ioremap_resource(&pdev->dev, res);
+	if (IS_ERR(jpeg->reg_base)) {
+		ret = PTR_ERR(jpeg->reg_base);
 		return ret;
 	}
 
-	res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
-	dec_irq = platform_get_irq(pdev, 0);
-	if (!res || dec_irq < 0) {
-		dev_err(&pdev->dev, "Failed to get dec_irq %d.\n", dec_irq);
-		ret = -EINVAL;
-		return ret;
+	jpeg_irq = platform_get_irq(pdev, 0);
+	if (jpeg_irq < 0) {
+		dev_err(&pdev->dev, "Failed to get jpeg_irq %d.\n", jpeg_irq);

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_probe(struct platform_device *pdev) · line 1119

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -4
+		return jpeg_irq;
 	}
 
-	ret = devm_request_irq(&pdev->dev, dec_irq, mtk_jpeg_dec_irq, 0,
+	ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_irq, 0,
 			       pdev->name, jpeg);
 	if (ret) {
-		dev_err(&pdev->dev, "Failed to request dec_irq %d (%d)\n",
-			dec_irq, ret);
-		ret = -EINVAL;

This removal of ret assignment looks like a separate fix that should be done in its own patch.

Quoted 32 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -13
+		dev_err(&pdev->dev, "Failed to request jpeg_irq %d (%d)\n",
+			jpeg_irq, ret);
 		goto err_req_irq;
 	}
 
@@ -1140,33 +1602,35 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 		goto err_m2m_init;
 	}
 
-	jpeg->dec_vdev = video_device_alloc();
-	if (!jpeg->dec_vdev) {
+	jpeg->vfd_jpeg = video_device_alloc();
+	if (!jpeg->vfd_jpeg) {
 		ret = -ENOMEM;
-		goto err_dec_vdev_alloc;
+		goto err_vfd_jpeg_alloc;
 	}
-	snprintf(jpeg->dec_vdev->name, sizeof(jpeg->dec_vdev->name),
-		 "%s-dec", MTK_JPEG_NAME);
-	jpeg->dec_vdev->fops = &mtk_jpeg_fops;
-	jpeg->dec_vdev->ioctl_ops = &mtk_jpeg_ioctl_ops;
-	jpeg->dec_vdev->minor = -1;
-	jpeg->dec_vdev->release = video_device_release;
-	jpeg->dec_vdev->lock = &jpeg->lock;
-	jpeg->dec_vdev->v4l2_dev = &jpeg->v4l2_dev;
-	jpeg->dec_vdev->vfl_dir = VFL_DIR_M2M;
-	jpeg->dec_vdev->device_caps = V4L2_CAP_STREAMING |
+	snprintf(jpeg->vfd_jpeg->name, sizeof(jpeg->vfd_jpeg->name),
+		 "%s-%s", MTK_JPEG_NAME,
+		 jpeg->mode == MTK_JPEG_ENC ? "enc" : "dec");
+	jpeg->vfd_jpeg->fops = &mtk_jpeg_fops;
+	jpeg->vfd_jpeg->ioctl_ops = &mtk_jpeg_ioctl_ops;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_probe(struct platform_device *pdev) · line 1623

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +7 -1
+	jpeg->vfd_jpeg->minor = -1;
+	jpeg->vfd_jpeg->release = video_device_release;
+	jpeg->vfd_jpeg->lock = &jpeg->lock;
+	jpeg->vfd_jpeg->v4l2_dev = &jpeg->v4l2_dev;
+	jpeg->vfd_jpeg->vfl_dir = VFL_DIR_M2M;
+	jpeg->vfd_jpeg->device_caps = V4L2_CAP_STREAMING |
 				      V4L2_CAP_VIDEO_M2M_MPLANE;
 
-	ret = video_register_device(jpeg->dec_vdev, VFL_TYPE_GRABBER, 3);
+	ret = video_register_device(jpeg->vfd_jpeg, VFL_TYPE_GRABBER, -1);

The change from 3 to -1 also looks like something for a separate patch.

Quoted 105 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +25 -13
 	if (ret) {
 		v4l2_err(&jpeg->v4l2_dev, "Failed to register video device\n");
-		goto err_dec_vdev_register;
+		goto err_vfd_jpeg_register;
 	}
 
-	video_set_drvdata(jpeg->dec_vdev, jpeg);
+	video_set_drvdata(jpeg->vfd_jpeg, jpeg);
 	v4l2_info(&jpeg->v4l2_dev,
-		  "decoder device registered as /dev/video%d (%d,%d)\n",
-		  jpeg->dec_vdev->num, VIDEO_MAJOR, jpeg->dec_vdev->minor);
+		  "jpeg device %d registered as /dev/video%d (%d,%d)\n",
+		  jpeg->mode, jpeg->vfd_jpeg->num, VIDEO_MAJOR,
+		  jpeg->vfd_jpeg->minor);
 
 	platform_set_drvdata(pdev, jpeg);
 
@@ -1174,10 +1638,10 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 
 	return 0;
 
-err_dec_vdev_register:
-	video_device_release(jpeg->dec_vdev);
+err_vfd_jpeg_register:
+	video_device_release(jpeg->vfd_jpeg);
 
-err_dec_vdev_alloc:
+err_vfd_jpeg_alloc:
 	v4l2_m2m_release(jpeg->m2m_dev);
 
 err_m2m_init:
@@ -1197,8 +1661,8 @@ static int mtk_jpeg_remove(struct platform_device *pdev)
 	struct mtk_jpeg_dev *jpeg = platform_get_drvdata(pdev);
 
 	pm_runtime_disable(&pdev->dev);
-	video_unregister_device(jpeg->dec_vdev);
-	video_device_release(jpeg->dec_vdev);
+	video_unregister_device(jpeg->vfd_jpeg);
+	video_device_release(jpeg->vfd_jpeg);
 	v4l2_m2m_release(jpeg->m2m_dev);
 	v4l2_device_unregister(&jpeg->v4l2_dev);
 
@@ -1209,7 +1673,11 @@ static __maybe_unused int mtk_jpeg_pm_suspend(struct device *dev)
 {
 	struct mtk_jpeg_dev *jpeg = dev_get_drvdata(dev);
 
-	mtk_jpeg_dec_reset(jpeg->dec_reg_base);
+	if (jpeg->mode == MTK_JPEG_ENC)
+		mtk_jpeg_enc_reset(jpeg->reg_base);
+	else
+		mtk_jpeg_dec_reset(jpeg->reg_base);
+
 	mtk_jpeg_clk_off(jpeg);
 
 	return 0;
@@ -1220,7 +1688,10 @@ static __maybe_unused int mtk_jpeg_pm_resume(struct device *dev)
 	struct mtk_jpeg_dev *jpeg = dev_get_drvdata(dev);
 
 	mtk_jpeg_clk_on(jpeg);
-	mtk_jpeg_dec_reset(jpeg->dec_reg_base);
+	if (jpeg->mode == MTK_JPEG_ENC)
+		mtk_jpeg_enc_reset(jpeg->reg_base);
+	else
+		mtk_jpeg_dec_reset(jpeg->reg_base);
 
 	return 0;
 }
@@ -1256,11 +1727,15 @@ static const struct dev_pm_ops mtk_jpeg_pm_ops = {
 static const struct of_device_id mtk_jpeg_match[] = {
 	{
 		.compatible = "mediatek,mt8173-jpgdec",
-		.data       = NULL,
+		.data       = (void *)MTK_JPEG_DEC,
 	},
 	{
 		.compatible = "mediatek,mt2701-jpgdec",
-		.data       = NULL,
+		.data       = (void *)MTK_JPEG_DEC,
+	},
+	{
+		.compatible = "mediatek,mtk-jpgenc",
+		.data       = (void *)MTK_JPEG_ENC,
 	},
 	{},
 };
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +3 -0
index 999bd1427809..35ddc64af554 100644
@@ -3,6 +3,7 @@
  * Copyright (c) 2016 MediaTek Inc.
  * Author: Ming Hsiu Tsai <[email protected]>
  *         Rick Chang <[email protected]>
+ *         Xia Jiang <[email protected]>
  */
 
 #ifndef _MTK_JPEG_CORE_H
@@ -17,23 +18,77 @@
 
 #define MTK_JPEG_FMT_FLAG_DEC_OUTPUT	BIT(0)
 #define MTK_JPEG_FMT_FLAG_DEC_CAPTURE	BIT(1)
+#define MTK_JPEG_FMT_FLAG_ENC_OUTPUT	BIT(2)
+#define MTK_JPEG_FMT_FLAG_ENC_CAPTURE	BIT(3)
 
 #define MTK_JPEG_FMT_TYPE_OUTPUT	1

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 30

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +4 -4
 #define MTK_JPEG_FMT_TYPE_CAPTURE	2
 
-#define MTK_JPEG_MIN_WIDTH	32
-#define MTK_JPEG_MIN_HEIGHT	32
-#define MTK_JPEG_MAX_WIDTH	8192
-#define MTK_JPEG_MAX_HEIGHT	8192
+#define MTK_JPEG_MIN_WIDTH	32U
+#define MTK_JPEG_MIN_HEIGHT	32U
+#define MTK_JPEG_MAX_WIDTH	65535U
+#define MTK_JPEG_MAX_HEIGHT	65535U

Why is it okay to change this from 8192 to 65535?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 36

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +3 -0
 #define MTK_JPEG_DEFAULT_SIZEIMAGE	(1 * 1024 * 1024)
+#define MTK_JPEG_DEFAULT_EXIF_SIZE	(64 * 1024)
 
+#define MTK_JPEG_ENCODE		0
+#define MTK_JPEG_DECODE		1

These two are unused.

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +4 -0
+
+/**
+ * enum mtk_jpeg_ctx_state - contex state of jpeg
+ */
 enum mtk_jpeg_ctx_state {
 	MTK_JPEG_INIT = 0,
 	MTK_JPEG_RUNNING,

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 53

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +7 -0
 	MTK_JPEG_SOURCE_CHANGE,
 };
 
+/**
+ * enum mtk_jpeg_mode - mode of jpeg
+ */
+enum mtk_jpeg_mode {
+	MTK_JPEG_ENC,
+	MTK_JPEG_DEC,
+};

Perhaps we could just have a bool is_encoder?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 63

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +10 -0
+
+/**
+ * enum jpeg_enc_yuv_fmt - yuv format of jpeg enc
+ */
+enum jpeg_enc_yuv_fmt {
+	JPEG_YUV_FORMAT_YUYV = 0,
+	JPEG_YUV_FORMAT_YVYU = 1,
+	JPEG_YUV_FORMAT_NV12 = 2,
+	JEPG_YUV_FORMAT_NV21 = 3,
+};

#define macros are typically used for hardware values.

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +6 -0
+
+/**
+ * enum JPEG_ENCODE_QUALITY_ENUM - number of jpeg encoder quality
+ */
+enum JPEG_ENCODE_QUALITY_ENUM {
+	JPEG_ENCODE_QUALITY_Q60 = 0x0,

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 79

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +10 -0
+	JPEG_ENCODE_QUALITY_Q80 = 0x1,
+	JPEG_ENCODE_QUALITY_Q90 = 0x2,
+	JPEG_ENCODE_QUALITY_Q95 = 0x3,
+
+	JPEG_ENCODE_QUALITY_Q39 = 0x4,
+	JPEG_ENCODE_QUALITY_Q68 = 0x5,
+	JPEG_ENCODE_QUALITY_Q84 = 0x6,
+	JPEG_ENCODE_QUALITY_Q92 = 0x7,
+
+	JPEG_ENCODE_QUALITY_Q48 = 0x8,

No enum for 0x9?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 83

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +4 -0
+	JPEG_ENCODE_QUALITY_Q74 = 0xA,
+	JPEG_ENCODE_QUALITY_Q87 = 0xB,
+
+	JPEG_ENCODE_QUALITY_Q34 = 0xC,

No enum for 0xd?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 89

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +6 -0
+	JPEG_ENCODE_QUALITY_Q64 = 0xE,
+	JPEG_ENCODE_QUALITY_Q82 = 0xF,
+
+	JPEG_ENCODE_QUALITY_Q97 = 0x10,
+
+	JPEG_ENCODE_QUALITY_ALL = 0xFFFFFFFF

What is the meaning of this value?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 89

Quoted 1 line
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +1 -0
+};

Since these are hardware register values, #define macros are preferred.

Quoted 40 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +11 -8
+
 /**
  * struct mt_jpeg - JPEG IP abstraction
  * @lock:		the mutex protecting this structure
@@ -43,11 +98,12 @@ enum mtk_jpeg_ctx_state {
  * @v4l2_dev:		v4l2 device for mem2mem mode
  * @m2m_dev:		v4l2 mem2mem device data
  * @alloc_ctx:		videobuf2 memory allocator's context
- * @dec_vdev:		video device node for decoder mem2mem mode
- * @dec_reg_base:	JPEG registers mapping
- * @clk_jdec:		JPEG hw working clock
- * @clk_jdec_smi:	JPEG SMI bus clock
+ * @vfd_jpeg:		video device node for jpeg mem2mem mode
+ * @reg_base:		JPEG registers mapping
+ * @clk_jpeg:		JPEG hw working clock
+ * @clk_jpeg_smi:	JPEG SMI bus clock
  * @larb:		SMI device
+ * @mode:		compression (encode) operation or decompression (decode)
  */
 struct mtk_jpeg_dev {
 	struct mutex		lock;
@@ -57,11 +113,12 @@ struct mtk_jpeg_dev {
 	struct v4l2_device	v4l2_dev;
 	struct v4l2_m2m_dev	*m2m_dev;
 	void			*alloc_ctx;
-	struct video_device	*dec_vdev;
-	void __iomem		*dec_reg_base;
-	struct clk		*clk_jdec;
-	struct clk		*clk_jdec_smi;
+	struct video_device	*vfd_jpeg;
+	void __iomem		*reg_base;
+	struct clk		*clk_jpeg;
+	struct clk		*clk_jpeg_smi;
 	struct device		*larb;
+	enum mtk_jpeg_mode	mode;
 };
 
 /**
@@ -101,15 +158,51 @@ struct mtk_jpeg_q_data {
 	u32			sizeimage[VIDEO_MAX_PLANES];

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h struct mtk_jpeg_q_data { · line 168

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +8 -0
 };
 
+/**
+ * jpeg_enc_param - parameters of jpeg encode control
+ * @enable_exif:	EXIF enable for jpeg encode mode
+ * @enc_quality:	destination image quality in encode mode
+ * @restart_interval:	JPEG restart interval for JPEG encoding
+ */
+struct jpeg_enc_param {
+	u32 enable_exif;

Shouldn't this be a bool?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h struct mtk_jpeg_q_data { · line 169

Quoted 1 line
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +1 -0
+	u32 enc_quality;

This is from 0 to 100, so perhaps unsigned char?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h struct mtk_jpeg_q_data { · line 170

Quoted 1 line
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +1 -0
+	u32 restart_interval;

Ditto.

Quoted 15 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +15 -0
+};
+
+/**
+ * mtk_jpeg_enc_param:  General jpeg encoding parameters
+ * @enc_w:		image width
+ * @enc_h:		image height
+ * @enable_exif:	EXIF enable for jpeg encode mode
+ * @enc_quality:	destination image quality in encode mode
+ * @enc_format:		input image format
+ * @restart_interval:	JPEG restart interval for JPEG encoding
+ * @img_stride:		jpeg encoder image stride
+ * @mem_stride:		jpeg encoder memory stride
+ * @total_encdu:	total 8x8 block number
+ */
+struct mtk_jpeg_enc_param {

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h struct mtk_jpeg_q_data { · line 195

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +10 -0
+	u32 enc_w;
+	u32 enc_h;
+	u32 enable_exif;
+	u32 enc_quality;
+	u32 enc_format;
+	u32 restart_interval;
+	u32 img_stride;
+	u32 mem_stride;
+	u32 total_encdu;
+};

All of the parameters here are already represented by corresponding generic V4L2 state. We should be able to just convert them to the hardware representation at the time of programing the registers, without a need to store them in this intermediate struct.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h struct mtk_jpeg_q_data { · line 112

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +1 -2
+
 /**
  * mtk_jpeg_ctx - the device context data
  * @jpeg:		JPEG IP device for this context
  * @out_q:		source (output) queue information
  * @cap_q:		destination (capture) queue queue information
  * @fh:			V4L2 file handle
- * @dec_param		parameters for HW decoding
  * @state:		state of the context
- * @header_valid:	set if header has been parsed and valid

Why removing these two?

Quoted 70 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +4 -0
+ * @jpeg_param:		jpeg encode parameters
+ * @ctrl_hdl:		controls handler
  * @colorspace: enum v4l2_colorspace; supplemental to pixelformat
  * @ycbcr_enc: enum v4l2_ycbcr_encoding, Y'CbCr encoding
  * @quantization: enum v4l2_quantization, colorspace quantization
@@ -121,6 +214,8 @@ struct mtk_jpeg_ctx {
 	struct mtk_jpeg_q_data		cap_q;
 	struct v4l2_fh			fh;
 	enum mtk_jpeg_ctx_state		state;
+	struct jpeg_enc_param		jpeg_param;
+	struct v4l2_ctrl_handler	ctrl_hdl;
 
 	enum v4l2_colorspace colorspace;
 	enum v4l2_ycbcr_encoding ycbcr_enc;
drivers/media/platform/mtk-jpeg/mtk_jpeg_dec_hw.h +4 -3
index a37be1a48415..39d743fb572f 100644
@@ -3,10 +3,11 @@
  * Copyright (c) 2016 MediaTek Inc.
  * Author: Ming Hsiu Tsai <[email protected]>
  *         Rick Chang <[email protected]>
+ *         Xia Jiang <[email protected]>
  */
 
-#ifndef _MTK_JPEG_HW_H
-#define _MTK_JPEG_HW_H
+#ifndef _MTK_JPEG_DEC_HW_H
+#define _MTK_JPEG_DEC_HW_H
 
 #include <media/videobuf2-core.h>
 
@@ -80,4 +81,4 @@ void mtk_jpeg_dec_set_config(void __iomem *base,
 void mtk_jpeg_dec_reset(void __iomem *dec_reg_base);
 void mtk_jpeg_dec_start(void __iomem *dec_reg_base);
 
-#endif /* _MTK_JPEG_HW_H */
+#endif /* _MTK_JPEG_DEC_HW_H */
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +26 -0
new file mode 100644
index 000000000000..ac27774ccfed
@@ -0,0 +1,175 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ * Author: Xia Jiang <[email protected]>
+ *
+ */
+
+#include <linux/io.h>
+#include <linux/kernel.h>
+#include <media/videobuf2-core.h>
+
+#include "mtk_jpeg_enc_hw.h"
+
+void mtk_jpeg_enc_reset(void __iomem *base)
+{
+	writel(0x00, base + JPGENC_RSTB);
+	writel(JPEG_ENC_RESET_BIT, base + JPGENC_RSTB);
+	writel(0x00, base + JPGENC_CODEC_SEL);
+}
+
+u32 mtk_jpeg_enc_get_int_status(void __iomem *base)
+{
+	u32 ret;
+
+	ret = readl(base + JPGENC_INT_STS) &
+		    JPEG_DRV_ENC_INT_STATUS_MASK_ALLIRQ;

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 36

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +10 -0
+	if (ret)
+		writel(0, base + JPGENC_INT_STS);
+
+	return ret;
+}
+
+u32 mtk_jpeg_enc_get_file_size(void __iomem *base)
+{
+	return readl(base + JPGENC_DMA_ADDR0) - readl(base + JPGENC_DST_ADDR0);
+}

This is just used in the function below, so the code could just be moved there.

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 47

Quoted 11 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +11 -0
+
+u32 mtk_jpeg_enc_enum_result(void __iomem *base, u32 irq_status, u32 *file_size)
+{
+	*file_size = mtk_jpeg_enc_get_file_size(base);
+	if (irq_status & JPEG_DRV_ENC_INT_STATUS_DONE)
+		return MTK_JPEG_ENC_RESULT_DONE;
+	else if (irq_status & JPEG_DRV_ENC_INT_STATUS_STALL)
+		return MTK_JPEG_ENC_RESULT_STALL;
+	else if (irq_status & JPEG_DRV_ENC_INT_STATUS_VCODEC_IRQ)
+		return MTK_JPEG_ENC_RESULT_VCODEC_IRQ;
+	return MTK_JPEG_ENC_RESULT_ERROR_UNKNOWN;

This function seems to do two things at a time, which is confusing, given that the name only refers to the result and doesn't mention the "file size".

So an alternative option to my suggestion above would be removing the file size handling from this function and using the dedicated function for it.

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 53

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +6 -0
+}
+
+static void mtk_jpeg_enc_set_blk_num(void __iomem *base, u32 blk_num)
+{
+	writel(blk_num, base + JPGENC_BLK_NUM);
+}

Is there any benefit of having a lot of such small functions rather than just doing everything needed in one function? This hardware doesn't have so many parameters to configure and the caller side is generic V4L2 and vb2, so there is no need for additional layer of abstraction.

How about the following model, as used by many other drivers:

mtk_jpeg_enc_set_src()

{
	// Set any registers related to source format and buffer
}

mtk_jpeg_enc_set_dst()

{
	// Set any registers related to destination format and buffer
}

mtk_jpeg_enc_set_params()

{
	// Set any registers related to additional encoding parameters
}

mtk_jpeg_enc_device_run(enc, ctx)

{
	mtk_jpeg_enc_set_src(enc, src_buf, src_fmt);
	mtk_jpeg_enc_set_dst(enc, dst_buf, dst_fmt);
	mtk_jpeg_enc_set_params(enc, ctx);
	// Trigger the hardware run
}

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 62

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +9 -0
+
+static void mtk_jpeg_enc_set_encFormat(void __iomem *base, u32 enc_format)
+{
+	u32 value;
+
+	value = readl(base + JPGENC_CTRL);
+	value &= ~JPEG_ENC_CTRL_YUV_BIT;
+	value |= JPGENC_FORMAT(enc_format);
+	writel(value, base + JPGENC_CTRL);

The model I suggested above also avoids this kind of read modify write operations, which just unnecessarily add synchronous MMIO round trips, which means more CPU overhead.

Given that the full state is always known by the driver, it can just write all the register values without the need to read them back.

Quoted 36 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +36 -0
+}
+
+static void mtk_jpeg_enc_set_img_size(void __iomem *base, u32 width, u32 height)
+{
+	u32 value;
+
+	value = JPGENC_WIDTH_HEIGHT(width, height);
+	writel(value, base + JPGENC_IMG_SIZE);
+}
+
+static void mtk_jpeg_enc_set_src_img(void __iomem *base, u32 width,
+				     u32 height, u32 yuv_format,
+				     u32 total_encdu)
+{
+	mtk_jpeg_enc_set_img_size(base, width, height);
+	mtk_jpeg_enc_set_encFormat(base, yuv_format);
+	mtk_jpeg_enc_set_blk_num(base, total_encdu);
+}
+
+static void mtk_jpeg_enc_set_src_buf(void __iomem *base, u32 img_stride,
+				     u32 mem_stride, u32 src_addr,
+				     u32 src_addr_c)
+{
+	writel(img_stride, base + JPGENC_IMG_STRIDE);
+	writel(mem_stride, base + JPGENC_STRIDE);
+	writel(src_addr, base + JPGENC_SRC_LUMA_ADDR);
+	writel(src_addr_c, base + JPGENC_SRC_CHROMA_ADDR);
+}
+
+static void mtk_jpeg_enc_set_dst_buf(void __iomem *base, u32 dst_addr,
+				     u32 stall_size, u32 init_offset,
+				     u32 offset_mask)
+{
+	writel(JPGENC_INIT_OFFSET(init_offset), base + JPGENC_OFFSET_ADDR);
+	writel(JPGENC_OFFSET_MASK(offset_mask), base + JPGENC_BYTE_OFFSET_MASK);
+	writel(JPGENC_DST_ADDR(dst_addr), base + JPGENC_DST_ADDR0);

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 108

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +10 -0
+	writel(JPGENC_STALL_ADDR(dst_addr, stall_size),
+	       base + JPGENC_STALL_ADDR0);
+}
+
+static void mtk_jpeg_enc_set_quality(void __iomem *base, u32 quality)
+{
+	u32 value;
+
+	value = readl(base + JPGENC_QUALITY);
+	value = JPGENC_SET_QUALITY(value, quality);

This macro is confusing. Could we just clear the quality bits explicitly and then bitwise-OR the new value?

Quoted 81 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +67 -0
+	writel(value, base + JPGENC_QUALITY);
+}
+
+static void mtk_jpeg_enc_set_restart_interval(void __iomem *base,
+					      u32 restart_interval)
+{
+	u32 value;
+
+	value = readl(base + JPGENC_CTRL);
+	if (restart_interval)
+		value |= JPEG_ENC_CTRL_RESTART_EN_BIT;
+	else
+		value &= ~JPEG_ENC_CTRL_RESTART_EN_BIT;
+	writel(value, base + JPGENC_CTRL);
+	writel(restart_interval, base + JPGENC_RST_MCU_NUM);
+}
+
+static void mtk_jpeg_enc_set_encode_mode(void __iomem *base, u32 exif_en)
+{
+	u32 value;
+
+	value = readl(base + JPGENC_CTRL);
+	value &= ~JPEG_ENC_CTRL_FILE_FORMAT_BIT;
+	writel(value, base + JPGENC_CTRL);
+
+	if (exif_en) {
+		value = readl(base + JPGENC_CTRL);
+		value |= JPEG_ENC_EN_JFIF_EXIF;
+		writel(value, base + JPGENC_CTRL);
+	}
+}
+
+static void mtk_jpeg_enc_set_ctrl_cfg(void __iomem *base, u32 exif_en,
+				      u32 quality, u32 restart_interval)
+{
+	mtk_jpeg_enc_set_quality(base, quality);
+
+	mtk_jpeg_enc_set_restart_interval(base, restart_interval);
+
+	mtk_jpeg_enc_set_encode_mode(base, exif_en);
+}
+
+void mtk_jpeg_enc_start(void __iomem *base)
+{
+	u32 value;
+
+	value = readl(base + JPGENC_CTRL);
+	value |= JPEG_ENC_CTRL_INT_EN_BIT | JPEG_ENC_CTRL_ENABLE_BIT;
+	writel(value, base + JPGENC_CTRL);
+}
+
+void mtk_jpeg_enc_set_config(void __iomem *base,
+			     struct mtk_jpeg_enc_param *config,
+			     struct mtk_jpeg_enc_bs *bs,
+			     struct mtk_jpeg_enc_fb *fb)
+{
+	mtk_jpeg_enc_set_src_img(base, config->enc_w, config->enc_h,
+				 config->enc_format, config->total_encdu);
+	mtk_jpeg_enc_set_src_buf(base, config->img_stride, config->mem_stride,
+				 fb->fb_addr[0].dma_addr,
+				 fb->fb_addr[1].dma_addr);
+	mtk_jpeg_enc_set_dst_buf(base, bs->dma_addr, bs->size,
+				 bs->dma_addr_offset, bs->dma_addr_offsetmask);
+	mtk_jpeg_enc_set_ctrl_cfg(base, config->enable_exif,
+				  config->enc_quality,
+				  config->restart_interval);
+}
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h new file +8 -0
new file mode 100644
index 000000000000..7080240f6dac
@@ -0,0 +1,60 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ * Author: Xia Jiang <[email protected]>
+ *
+ */
+
+#ifndef _MTK_JPEG_ENC_HW_H

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h line 18

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h new file +10 -0
+#define _MTK_JPEG_ENC_HW_H
+
+#include <media/videobuf2-core.h>
+
+#include "mtk_jpeg_core.h"
+#include "mtk_jpeg_enc_reg.h"
+
+#define JPEG_ENC_DST_ADDR_OFFSET_MASK GENMASK(3, 0)
+
+#define JPEG_ENC_CTRL_YUV_BIT 0x18

0x18 isn't a single bit as the _BIT suffix would suggest. Perhaps JPEG_ENC_CTRL_FORMAT_MASK would be more appropriate?

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h line 31

Quoted 13 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h new file +13 -0
+#define JPEG_ENC_CTRL_RESTART_EN_BIT            BIT(10)
+#define JPEG_ENC_CTRL_FILE_FORMAT_BIT           BIT(5)
+#define JPEG_ENC_EN_JFIF_EXIF                   BIT(5)
+#define JPEG_ENC_CTRL_INT_EN_BIT                BIT(2)
+#define JPEG_ENC_CTRL_ENABLE_BIT                BIT(0)
+#define JPEG_ENC_RESET_BIT                      BIT(0)
+
+enum {
+	MTK_JPEG_ENC_RESULT_DONE		= 0,
+	MTK_JPEG_ENC_RESULT_STALL,
+	MTK_JPEG_ENC_RESULT_VCODEC_IRQ,
+	MTK_JPEG_ENC_RESULT_ERROR_UNKNOWN
+};

Do we need these intermediate error codes? Could we just use errno values instead?

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h new file +7 -0
+
+struct mtk_jpeg_enc_bs {
+	dma_addr_t	dma_addr;
+	size_t		size;
+	u32			dma_addr_offset;
+	u32			dma_addr_offsetmask;
+};

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h line 48

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h new file +10 -0
+
+struct mtk_jpeg_mem {
+	dma_addr_t	dma_addr;
+	size_t		size;
+};
+
+struct mtk_jpeg_enc_fb {
+	struct mtk_jpeg_mem	fb_addr[MTK_JPEG_COMP_MAX];
+	u32			num_planes;
+};

Please add kerneldoc comments to the structs above.

nit: Separating types from names with a single space is more future-proof, in case someone needs to change the layout of the struct or rename fields.

Quoted 18 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h new file +12 -0
+
+void mtk_jpeg_enc_reset(void __iomem *base);
+u32 mtk_jpeg_enc_get_int_status(void __iomem *base);
+u32 mtk_jpeg_enc_get_file_size(void __iomem *base);
+u32 mtk_jpeg_enc_enum_result(void __iomem *base, u32 irq_status,
+			     u32 *file_size);
+void mtk_jpeg_enc_start(void __iomem *enc_reg_base);
+void mtk_jpeg_enc_set_config(void __iomem *base,
+			     struct mtk_jpeg_enc_param *config,
+			     struct mtk_jpeg_enc_bs *bs,
+			     struct mtk_jpeg_enc_fb *fb);
+#endif /* _MTK_JPEG_ENC_HW_H */
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h new file +0 -0
new file mode 100644
index 000000000000..d785621cb4c6

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h line 9

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h new file +9 -0
@@ -0,0 +1,49 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ * Author: Xia Jiang <[email protected]>
+ *
+ */
+#ifndef _MTK_JPEG_ENC_REG_H
+#define _MTK_JPEG_ENC_REG_H
+#define MTK_JPEG_COMP_MAX		3

This doesn't look like a register-specific definition.

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h line 14

Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h new file +5 -0
+
+#define JPEG_DRV_ENC_INT_STATUS_DONE			BIT(0)
+#define JPEG_DRV_ENC_INT_STATUS_STALL			BIT(1)
+#define JPEG_DRV_ENC_INT_STATUS_VCODEC_IRQ		BIT(4)
+#define JPEG_DRV_ENC_INT_STATUS_MASK_ALLIRQ		0x13

Could we define this as an | of the specific bit definitions above rather than hardcoding?

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h line 16

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h new file +2 -0
+
+#define JPGENC_RSTB				0x100

Could we agree on a consistent prefix for macros? Some use JPEG_ENC, some JPEG_DRV_ENC and some JPGENC. How about using JPEG_ENC everywhere?

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h line 19

Quoted 3 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h new file +3 -0
+#define JPGENC_CTRL				0x104
+#define JPGENC_QUALITY				0x108
+#define JPGENC_BLK_NUM				0x10C

nit: Hex numerals in the kernel are lowercase.

Quoted 18 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h new file +18 -0
+#define JPGENC_BLK_CNT				0x110
+#define JPGENC_INT_STS				0x11C
+#define JPGENC_DST_ADDR0			0x120
+#define JPGENC_DMA_ADDR0			0x124
+#define JPGENC_STALL_ADDR0			0x128
+#define JPGENC_OFFSET_ADDR			0x138
+#define JPGENC_RST_MCU_NUM			0x150
+#define JPGENC_IMG_SIZE				0x154
+#define JPGENC_DEBUG_INFO0			0x160
+#define JPGENC_DEBUG_INFO1			0x164
+#define JPGENC_TOTAL_CYCLE			0x168
+#define JPGENC_BYTE_OFFSET_MASK			0x16C
+#define JPGENC_SRC_LUMA_ADDR			0x170
+#define JPGENC_SRC_CHROMA_ADDR			0x174
+#define JPGENC_STRIDE				0x178
+#define JPGENC_IMG_STRIDE			0x17C
+#define JPGENC_DCM_CTRL				0x300
+#define JPGENC_CODEC_SEL			0x314

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h line 47

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h new file +10 -0
+#define JPGENC_ULTRA_THRES			0x318
+
+#define JPGENC_FORMAT(x)		(((x) & 3) << 3)
+#define JPGENC_WIDTH_HEIGHT(w, h)	(((w) << 16) | (h))
+#define JPGENC_INIT_OFFSET(x)		((x) & (~0xF))
+#define JPGENC_OFFSET_MASK(x)		((x) & 0xF)
+#define JPGENC_DST_ADDR(x)		((x) & (~0xF))
+#define JPGENC_STALL_ADDR(x, y)		(((x) + (y)) & (~0xF))
+#define JPGENC_QUALITY_MASK		0xFFFF0000
+#define JPGENC_SET_QUALITY(x, y)	(((x) & JPGENC_QUALITY_MASK) | (y))

JPGENC_QUALITY_MASK is only used here, so 0xffff0000 could be just used directly.

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h line 49

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h new file +2 -0
+
+#endif /* _MTK_JPEG_ENC_REG_H */

I can see some bits defined in mtk_jpeg_enc_hw.h as well. Perhaps all these could be moved there too, which would make us have 1 file less?

Best regards,
Tomasz

#10 lore raw

Re: [PATCH v4 5/5] media: platform: Add jpeg dec/enc feature

Quoted 76 lines · Xia Jiang
Quoted 52 lines · Tomasz Figa

Hi Xia,

Quoted 27 lines · Xia Jiang

Add mtk jpeg encode v4l2 driver based on jpeg decode, because that jpeg decode and encode have great similarities with function operation.

---
v4: split mtk_jpeg_try_fmt_mplane() to two functions, one for encoder,
one for decoder.
split mtk_jpeg_set_default_params() to two functions, one for encoder, one for decoder.
add cropping support for encoder in g/s_selection ioctls. change exif mode support by using V4L2_JPEG_ACTIVE_MARKER_APP1. change MTK_JPEG_MAX_WIDTH/MTK_JPEG_MAX_HEIGH from 8192 to 65535 by specification.
move width shifting operation behind aligning operation in mtk_jpeg_try_enc_fmt_mplane() for bug fix.
fix user abuseing data_offset issue for DMABUF in mtk_jpeg_set_enc_src().
fix kbuild warings: change MTK_JPEG_MIN_HEIGHT/MTK_JPEG_MAX_HEIGHT
and MTK_JPEG_MIN_WIDTH/MTK_JPEG_MAX_WIDTH from 'int' type to 'unsigned int' type. fix msleadingly indented of 'else'.

v3: delete Change-Id.
only test once handler->error after the last v4l2_ctrl_new_std(). seperate changes of v4l2-ctrls.c and v4l2-controls.h to new patch.

v2: fix compliance test fail, check created buffer size in driver.

Quoted 13 lines
---
 drivers/media/platform/mtk-jpeg/Makefile      |   5 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 731 +++++++++++++++---
 .../media/platform/mtk-jpeg/mtk_jpeg_core.h   | 123 ++-
 .../media/platform/mtk-jpeg/mtk_jpeg_dec_hw.h |   7 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c | 175 +++++
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h |  60 ++
 .../platform/mtk-jpeg/mtk_jpeg_enc_reg.h      |  49 ++
 7 files changed, 1004 insertions(+), 146 deletions(-)
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h

First of all, thanks for the patch!

Please check my comments below.

My general feeling about this code is that the encoder hardware block is completely orthogonal from the decoder block and there is very little code reuse from the original decoder driver.

Quoted 6 lines

Moreover, a lot of existing code now needs if (decoder) { ... } else {... } segments, which complicates the code.

Would it perhaps make sense to instead create a separate mtk-jpeg-enc driver?

Dear Tomasz,

Thanks for your comments.

My reasons about the architecture of jpeg enc driver are as follows:

The first internal design and realization of jpeg enc driver was a separate driver, but found that mtk_jpeg_core.c and mtk_jpeg_enc_core.c have lots of reuse.Because that the core.c mainly contains realization of v4L2 ioctl functions and some logic which are high similarity between encoder and decoder.

The jpeg encoder and decoder are two independent hardwares exactly, so the code about hardware specification(register setting) are separated(mtk_jpeg_enc_hw.c and mtk_jpeg_dec_hw.c).

Quoted 9 lines

As for 17 existing code segments contain if(decoder){} else {}, they are not complicated IMHO.The complicated(multilayer nested) functions are separated in V4 version as Hans recommendation.

By the way,the upstreamed module s5p-jpeg
(https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/s5p-jpeg/jpeg-core.c#L1998) also use encoder and decoder mode in the common core.c, but their encoder and decoder are the same hardware.Maybe our jpeg enc and dec are designed into one hardware in the future.In that case the current architecture is more compatible.

So I prefer the current design.

Would you be able to give some numbers to show the code reuse to justify using the same driver? From my observation, a new driver would result in a significantly cleaner code. If there is a further hardware architecture change, that would likely require another driver, because it wouldn't be compatible with existing programming model anyway.

Regardless of that, if we end up with reusing the same driver, I'd like you to fix the issues existing in the current base before adding the encoder functionality.

Best regards,
Tomasz

author
#11 lore raw

Re: [PATCH v4 5/5] media: platform: Add jpeg dec/enc feature

Quoted 93 lines · Tomasz Figa
Quoted 76 lines · Xia Jiang
Quoted 52 lines · Tomasz Figa

Hi Xia,

Quoted 27 lines · Xia Jiang

Add mtk jpeg encode v4l2 driver based on jpeg decode, because that jpeg decode and encode have great similarities with function operation.

---
v4: split mtk_jpeg_try_fmt_mplane() to two functions, one for encoder,
one for decoder.
split mtk_jpeg_set_default_params() to two functions, one for encoder, one for decoder.
add cropping support for encoder in g/s_selection ioctls. change exif mode support by using V4L2_JPEG_ACTIVE_MARKER_APP1. change MTK_JPEG_MAX_WIDTH/MTK_JPEG_MAX_HEIGH from 8192 to 65535 by specification.
move width shifting operation behind aligning operation in mtk_jpeg_try_enc_fmt_mplane() for bug fix.
fix user abuseing data_offset issue for DMABUF in mtk_jpeg_set_enc_src().
fix kbuild warings: change MTK_JPEG_MIN_HEIGHT/MTK_JPEG_MAX_HEIGHT
and MTK_JPEG_MIN_WIDTH/MTK_JPEG_MAX_WIDTH from 'int' type to 'unsigned int' type. fix msleadingly indented of 'else'.

v3: delete Change-Id.
only test once handler->error after the last v4l2_ctrl_new_std(). seperate changes of v4l2-ctrls.c and v4l2-controls.h to new patch.

v2: fix compliance test fail, check created buffer size in driver.

Quoted 13 lines
---
 drivers/media/platform/mtk-jpeg/Makefile      |   5 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 731 +++++++++++++++---
 .../media/platform/mtk-jpeg/mtk_jpeg_core.h   | 123 ++-
 .../media/platform/mtk-jpeg/mtk_jpeg_dec_hw.h |   7 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c | 175 +++++
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h |  60 ++
 .../platform/mtk-jpeg/mtk_jpeg_enc_reg.h      |  49 ++
 7 files changed, 1004 insertions(+), 146 deletions(-)
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h

First of all, thanks for the patch!

Please check my comments below.

My general feeling about this code is that the encoder hardware block is completely orthogonal from the decoder block and there is very little code reuse from the original decoder driver.

Quoted 6 lines

Moreover, a lot of existing code now needs if (decoder) { ... } else {... } segments, which complicates the code.

Would it perhaps make sense to instead create a separate mtk-jpeg-enc driver?

Dear Tomasz,

Thanks for your comments.

My reasons about the architecture of jpeg enc driver are as follows:

The first internal design and realization of jpeg enc driver was a separate driver, but found that mtk_jpeg_core.c and mtk_jpeg_enc_core.c have lots of reuse.Because that the core.c mainly contains realization of v4L2 ioctl functions and some logic which are high similarity between encoder and decoder.

The jpeg encoder and decoder are two independent hardwares exactly, so the code about hardware specification(register setting) are separated(mtk_jpeg_enc_hw.c and mtk_jpeg_dec_hw.c).

Quoted 9 lines

As for 17 existing code segments contain if(decoder){} else {}, they are not complicated IMHO.The complicated(multilayer nested) functions are separated in V4 version as Hans recommendation.

By the way,the upstreamed module s5p-jpeg
(https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/s5p-jpeg/jpeg-core.c#L1998) also use encoder and decoder mode in the common core.c, but their encoder and decoder are the same hardware.Maybe our jpeg enc and dec are designed into one hardware in the future.In that case the current architecture is more compatible.

So I prefer the current design.

Would you be able to give some numbers to show the code reuse to justify using the same driver? From my observation, a new driver would result in a significantly cleaner code. If there is a further hardware architecture change, that would likely require another driver, because it wouldn't be compatible with existing programming model anyway.

Quoted 3 lines

Regardless of that, if we end up with reusing the same driver, I'd like you to fix the issues existing in the current base before adding the encoder functionality.

Dear Tomasz,
I've counted about 1000 lines of code that can be reused.The reused code is 75 percent of the original code.

If you agree to reuse the same driver,I will fix the issues existing in the current driver.

Best Regards,
Xia Jiang

Quoted 3 lines

Best regards,
Tomasz

#12 lore raw

Re: [PATCH v4 5/5] media: platform: Add jpeg dec/enc feature

Hi Xia,

Quoted 98 lines · Xia Jiang
Quoted 93 lines · Tomasz Figa
Quoted 76 lines · Xia Jiang
Quoted 52 lines · Tomasz Figa

Hi Xia,

Quoted 27 lines · Xia Jiang

Add mtk jpeg encode v4l2 driver based on jpeg decode, because that jpeg decode and encode have great similarities with function operation.

---
v4: split mtk_jpeg_try_fmt_mplane() to two functions, one for encoder,
one for decoder.
split mtk_jpeg_set_default_params() to two functions, one for encoder, one for decoder.
add cropping support for encoder in g/s_selection ioctls. change exif mode support by using V4L2_JPEG_ACTIVE_MARKER_APP1. change MTK_JPEG_MAX_WIDTH/MTK_JPEG_MAX_HEIGH from 8192 to 65535 by specification.
move width shifting operation behind aligning operation in mtk_jpeg_try_enc_fmt_mplane() for bug fix.
fix user abuseing data_offset issue for DMABUF in mtk_jpeg_set_enc_src().
fix kbuild warings: change MTK_JPEG_MIN_HEIGHT/MTK_JPEG_MAX_HEIGHT
and MTK_JPEG_MIN_WIDTH/MTK_JPEG_MAX_WIDTH from 'int' type to 'unsigned int' type. fix msleadingly indented of 'else'.

v3: delete Change-Id.
only test once handler->error after the last v4l2_ctrl_new_std(). seperate changes of v4l2-ctrls.c and v4l2-controls.h to new patch.

v2: fix compliance test fail, check created buffer size in driver.

Quoted 13 lines
---
 drivers/media/platform/mtk-jpeg/Makefile      |   5 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 731 +++++++++++++++---
 .../media/platform/mtk-jpeg/mtk_jpeg_core.h   | 123 ++-
 .../media/platform/mtk-jpeg/mtk_jpeg_dec_hw.h |   7 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c | 175 +++++
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h |  60 ++
 .../platform/mtk-jpeg/mtk_jpeg_enc_reg.h      |  49 ++
 7 files changed, 1004 insertions(+), 146 deletions(-)
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h

First of all, thanks for the patch!

Please check my comments below.

My general feeling about this code is that the encoder hardware block is completely orthogonal from the decoder block and there is very little code reuse from the original decoder driver.

Quoted 6 lines

Moreover, a lot of existing code now needs if (decoder) { ... } else {... } segments, which complicates the code.

Would it perhaps make sense to instead create a separate mtk-jpeg-enc driver?

Dear Tomasz,

Thanks for your comments.

My reasons about the architecture of jpeg enc driver are as follows:

The first internal design and realization of jpeg enc driver was a separate driver, but found that mtk_jpeg_core.c and mtk_jpeg_enc_core.c have lots of reuse.Because that the core.c mainly contains realization of v4L2 ioctl functions and some logic which are high similarity between encoder and decoder.

The jpeg encoder and decoder are two independent hardwares exactly, so the code about hardware specification(register setting) are separated(mtk_jpeg_enc_hw.c and mtk_jpeg_dec_hw.c).

Quoted 9 lines

As for 17 existing code segments contain if(decoder){} else {}, they are not complicated IMHO.The complicated(multilayer nested) functions are separated in V4 version as Hans recommendation.

By the way,the upstreamed module s5p-jpeg
(https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/s5p-jpeg/jpeg-core.c#L1998) also use encoder and decoder mode in the common core.c, but their encoder and decoder are the same hardware.Maybe our jpeg enc and dec are designed into one hardware in the future.In that case the current architecture is more compatible.

So I prefer the current design.

Would you be able to give some numbers to show the code reuse to justify using the same driver? From my observation, a new driver would result in a significantly cleaner code. If there is a further hardware architecture change, that would likely require another driver, because it wouldn't be compatible with existing programming model anyway.

Quoted 3 lines

Regardless of that, if we end up with reusing the same driver, I'd like you to fix the issues existing in the current base before adding the encoder functionality.

Quoted 6 lines

Dear Tomasz,
I've counted about 1000 lines of code that can be reused.The reused code is 75 percent of the original code.

If you agree to reuse the same driver,I will fix the issues existing in the current driver.

Sorry, I was out of the office.

Okay, let's reuse the driver. I guess the hardware programming part itself is smaller than the V4L2 boiler plate needed for it and that's where the 1000 lines of code comes from.

The first step I'd suggest then would be running the latest v4l2-compliance, from the master branch of v4l2-utils [1] and making sure there are no issues.

Then check if any comments I posted to the new code added by your patch apply to the existing code as well and fixing those issues in a prerequisite patches, at the beginning of the series.

[1] https://git.linuxtv.org/v4l-utils.git/

Best regards,
Tomasz

author
#13 lore raw

Re: [PATCH v4 5/5] media: platform: Add jpeg dec/enc feature

In reply to Tomasz Figa
Quoted 55 lines · Tomasz Figa

Hi Xia,

Quoted 27 lines · Xia Jiang

Add mtk jpeg encode v4l2 driver based on jpeg decode, because that jpeg decode and encode have great similarities with function operation.

---
v4: split mtk_jpeg_try_fmt_mplane() to two functions, one for encoder,
one for decoder.
split mtk_jpeg_set_default_params() to two functions, one for encoder, one for decoder.
add cropping support for encoder in g/s_selection ioctls. change exif mode support by using V4L2_JPEG_ACTIVE_MARKER_APP1. change MTK_JPEG_MAX_WIDTH/MTK_JPEG_MAX_HEIGH from 8192 to 65535 by specification.
move width shifting operation behind aligning operation in mtk_jpeg_try_enc_fmt_mplane() for bug fix.
fix user abuseing data_offset issue for DMABUF in mtk_jpeg_set_enc_src().
fix kbuild warings: change MTK_JPEG_MIN_HEIGHT/MTK_JPEG_MAX_HEIGHT
and MTK_JPEG_MIN_WIDTH/MTK_JPEG_MAX_WIDTH from 'int' type to 'unsigned int' type. fix msleadingly indented of 'else'.

v3: delete Change-Id.
only test once handler->error after the last v4l2_ctrl_new_std(). seperate changes of v4l2-ctrls.c and v4l2-controls.h to new patch.

v2: fix compliance test fail, check created buffer size in driver.

Quoted 13 lines
---
 drivers/media/platform/mtk-jpeg/Makefile      |   5 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 731 +++++++++++++++---
 .../media/platform/mtk-jpeg/mtk_jpeg_core.h   | 123 ++-
 .../media/platform/mtk-jpeg/mtk_jpeg_dec_hw.h |   7 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c | 175 +++++
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h |  60 ++
 .../platform/mtk-jpeg/mtk_jpeg_enc_reg.h      |  49 ++
 7 files changed, 1004 insertions(+), 146 deletions(-)
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h

First of all, thanks for the patch!

Please check my comments below.

My general feeling about this code is that the encoder hardware block is completely orthogonal from the decoder block and there is very little code reuse from the original decoder driver.

Moreover, a lot of existing code now needs if (decoder) { ... } else {... } segments, which complicates the code.

Quoted 9 lines

Would it perhaps make sense to instead create a separate mtk-jpeg-enc driver?

It would also give us a fresh start in terms of code quality, as the existing mtk-jpeg driver has a lot of quality issues unfortunately. (Some of my comments to this patch actually relate to the issues with the original code, not introduced by this patch, but we need to fix them if changing this driver already.)

Dear Tomasz,

I haved fixed the driver by following your advice in general.

Please check my reply below.

Quoted 92 lines
Quoted 92 lines
drivers/media/platform/mtk-jpeg/Makefile +4 -1
index 48516dcf96e6..76c33aad0f3f 100644
@@ -1,3 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0-only
-mtk_jpeg-objs := mtk_jpeg_core.o mtk_jpeg_dec_hw.o mtk_jpeg_dec_parse.o
+mtk_jpeg-objs := mtk_jpeg_core.o \
+		 mtk_jpeg_dec_hw.o \
+		 mtk_jpeg_dec_parse.o \
+		 mtk_jpeg_enc_hw.o
 obj-$(CONFIG_VIDEO_MEDIATEK_JPEG) += mtk_jpeg.o
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +41 -1
index abc506a552c1..087b62bb1d5c 100644
@@ -3,6 +3,7 @@
  * Copyright (c) 2016 MediaTek Inc.
  * Author: Ming Hsiu Tsai <[email protected]>
  *         Rick Chang <[email protected]>
+ *         Xia Jiang <[email protected]>
  */
 
 #include <linux/clk.h>
@@ -23,6 +24,7 @@
 #include <media/videobuf2-dma-contig.h>
 #include <soc/mediatek/smi.h>
 
+#include "mtk_jpeg_enc_hw.h"
 #include "mtk_jpeg_dec_hw.h"
 #include "mtk_jpeg_core.h"
 #include "mtk_jpeg_dec_parse.h"
@@ -31,7 +33,8 @@ static struct mtk_jpeg_fmt mtk_jpeg_formats[] = {
 	{
 		.fourcc		= V4L2_PIX_FMT_JPEG,
 		.colplanes	= 1,
-		.flags		= MTK_JPEG_FMT_FLAG_DEC_OUTPUT,
+		.flags		= MTK_JPEG_FMT_FLAG_DEC_OUTPUT |
+				  MTK_JPEG_FMT_FLAG_ENC_CAPTURE,
 	},
 	{
 		.fourcc		= V4L2_PIX_FMT_YUV420M,
@@ -51,6 +54,42 @@ static struct mtk_jpeg_fmt mtk_jpeg_formats[] = {
 		.v_align	= 3,
 		.flags		= MTK_JPEG_FMT_FLAG_DEC_CAPTURE,
 	},
+	{
+		.fourcc		= V4L2_PIX_FMT_NV12M,
+		.h_sample	= {4, 2, 2},
+		.v_sample	= {4, 2, 2},
+		.colplanes	= 2,
+		.h_align	= 4,
+		.v_align	= 4,
+		.flags		= MTK_JPEG_FMT_FLAG_ENC_OUTPUT,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_NV21M,
+		.h_sample	= {4, 2, 2},
+		.v_sample	= {4, 2, 2},
+		.colplanes	= 2,
+		.h_align	= 4,
+		.v_align	= 4,
+		.flags		= MTK_JPEG_FMT_FLAG_ENC_OUTPUT,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_YUYV,
+		.h_sample	= {4, 2, 2},
+		.v_sample	= {4, 4, 4},
+		.colplanes	= 1,
+		.h_align	= 4,
+		.v_align	= 3,
+		.flags		= MTK_JPEG_FMT_FLAG_ENC_OUTPUT,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_YVYU,
+		.h_sample	= {4, 2, 2},
+		.v_sample	= {4, 4, 4},
+		.colplanes	= 1,
+		.h_align	= 4,
+		.v_align	= 3,
+		.flags		= MTK_JPEG_FMT_FLAG_ENC_OUTPUT,
+	},
 };
 
 #define MTK_JPEG_NUM_FORMATS ARRAY_SIZE(mtk_jpeg_formats)
@@ -65,11 +104,19 @@ struct mtk_jpeg_src_buf {
 	struct list_head list;
 	int flags;
 	struct mtk_jpeg_dec_param dec_param;
+	struct mtk_jpeg_enc_param enc_param;
 };
 
Quoted 7 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c struct mtk_jpeg_src_buf { · line 110

Quoted 1 line
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -0
+#define MTK_MAX_CTRLS_HINT	20

I can see only 3 controls registered by this driver. Moreover, wouldn't it be more readable if the v4l2_ctrl_handler_init() call had "3" directly passed to it? Then the reader can see that there are 3 calls to v4l2_ctrl_new_std() just below it and the meaning behind the value of 3 is clear.

done

Quoted 46 lines
Quoted 46 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +27 -2
+
 static int debug;
 module_param(debug, int, 0644);
 
+static inline struct mtk_jpeg_ctx *ctrl_to_ctx(struct v4l2_ctrl *ctrl)
+{
+	return container_of(ctrl->handler, struct mtk_jpeg_ctx, ctrl_hdl);
+}
+
 static inline struct mtk_jpeg_ctx *mtk_jpeg_fh_to_ctx(struct v4l2_fh *fh)
 {
 	return container_of(fh, struct mtk_jpeg_ctx, fh);
@@ -86,14 +133,74 @@ static int mtk_jpeg_querycap(struct file *file, void *priv,
 {
 	struct mtk_jpeg_dev *jpeg = video_drvdata(file);
 
-	strscpy(cap->driver, MTK_JPEG_NAME " decoder", sizeof(cap->driver));
-	strscpy(cap->card, MTK_JPEG_NAME " decoder", sizeof(cap->card));
+	strscpy(cap->driver, MTK_JPEG_NAME, sizeof(cap->driver));
+	if (jpeg->mode ==  MTK_JPEG_ENC)
+		strscpy(cap->card, MTK_JPEG_NAME " encoder", sizeof(cap->card));
+	else
+		strscpy(cap->card, MTK_JPEG_NAME " decoder", sizeof(cap->card));
 	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
 		 dev_name(jpeg->dev));
 
 	return 0;
 }
 
+static int vidioc_jpeg_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_jpeg_ctx *ctx = ctrl_to_ctx(ctrl);
+	struct jpeg_enc_param *p = &ctx->jpeg_param;
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+	int ret = 0;
+
+	switch (ctrl->id) {
+	case V4L2_CID_JPEG_RESTART_INTERVAL:
+		p->restart_interval = ctrl->val;
+		break;
+	case V4L2_CID_JPEG_COMPRESSION_QUALITY:
+		p->enc_quality = ctrl->val;
+		break;
+	case V4L2_CID_JPEG_ACTIVE_MARKER:
+		p->enable_exif = ctrl->val & V4L2_JPEG_ACTIVE_MARKER_APP1;
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_querycap(struct file *file, void *priv, · line 167

Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -0
+		break;
+	}
+
+	v4l2_dbg(2, debug, &jpeg->v4l2_dev, "%s val = %d",
+		 v4l2_ctrl_get_name(ctrl->id), ctrl->val);

Generic V4L2 logs can already dump all V4L2 IOCTL calls, so perhaps we could remove this?

done

Quoted 63 lines
Quoted 63 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +44 -0
+
+	return ret;
+}
+
+static const struct v4l2_ctrl_ops mtk_jpeg_ctrl_ops = {
+	.s_ctrl = vidioc_jpeg_s_ctrl,
+};
+
+int mtk_jpeg_ctrls_setup(struct mtk_jpeg_ctx *ctx)
+{
+	const struct v4l2_ctrl_ops *ops = &mtk_jpeg_ctrl_ops;
+	struct v4l2_ctrl_handler *handler = &ctx->ctrl_hdl;
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+
+	v4l2_ctrl_handler_init(handler, MTK_MAX_CTRLS_HINT);
+
+	if (jpeg->mode == MTK_JPEG_ENC) {
+		v4l2_ctrl_new_std(handler, ops, V4L2_CID_JPEG_RESTART_INTERVAL,
+				  0, 100, 1, 0);
+		v4l2_ctrl_new_std(handler, ops,
+				  V4L2_CID_JPEG_COMPRESSION_QUALITY, 48, 100, 1,
+				  90);
+		v4l2_ctrl_new_std(handler, ops, V4L2_CID_JPEG_ACTIVE_MARKER, 0,
+				  V4L2_JPEG_ACTIVE_MARKER_APP1, 0, 0);
+
+		if (handler->error) {
+			v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
+			return handler->error;
+		}
+	}
+
+	v4l2_ctrl_handler_setup(&ctx->ctrl_hdl);
+
+	return 0;
+}
+
 static int mtk_jpeg_enum_fmt(struct mtk_jpeg_fmt *mtk_jpeg_formats, int n,
 			     struct v4l2_fmtdesc *f, u32 type)
 {
@@ -118,23 +225,29 @@ static int mtk_jpeg_enum_fmt(struct mtk_jpeg_fmt *mtk_jpeg_formats, int n,
 static int mtk_jpeg_enum_fmt_vid_cap(struct file *file, void *priv,
 				     struct v4l2_fmtdesc *f)
 {
+	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+
 	return mtk_jpeg_enum_fmt(mtk_jpeg_formats, MTK_JPEG_NUM_FORMATS, f,
+				 ctx->jpeg->mode == MTK_JPEG_ENC ?
+				 MTK_JPEG_FMT_FLAG_ENC_CAPTURE :
 				 MTK_JPEG_FMT_FLAG_DEC_CAPTURE);
 }
 
 static int mtk_jpeg_enum_fmt_vid_out(struct file *file, void *priv,
 				     struct v4l2_fmtdesc *f)
 {
+	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+
 	return mtk_jpeg_enum_fmt(mtk_jpeg_formats, MTK_JPEG_NUM_FORMATS, f,
+				 ctx->jpeg->mode == MTK_JPEG_ENC ?
+				 MTK_JPEG_FMT_FLAG_ENC_OUTPUT :
 				 MTK_JPEG_FMT_FLAG_DEC_OUTPUT);
 }
 
Quoted 11 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_enum_fmt(struct mtk_jpeg_fmt *mtk_jpeg_formats, int n, · line 250

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -5
-static struct mtk_jpeg_q_data *mtk_jpeg_get_q_data(struct mtk_jpeg_ctx *ctx,
-						   enum v4l2_buf_type type)
+static struct mtk_jpeg_q_data *
+mtk_jpeg_get_q_data(struct mtk_jpeg_ctx *ctx, enum v4l2_buf_type type)
 {
-	if (V4L2_TYPE_IS_OUTPUT(type))
-		return &ctx->out_q;
-	return &ctx->cap_q;
+	return V4L2_TYPE_IS_OUTPUT(type) ? &ctx->out_q : &ctx->cap_q;

Is this change necessary?

cancel this change.

Quoted 26 lines
Quoted 26 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +8 -3
 }
 
 static struct mtk_jpeg_fmt *mtk_jpeg_find_format(struct mtk_jpeg_ctx *ctx,
@@ -143,9 +256,14 @@ static struct mtk_jpeg_fmt *mtk_jpeg_find_format(struct mtk_jpeg_ctx *ctx,
 {
 	unsigned int k, fmt_flag;
 
-	fmt_flag = (fmt_type == MTK_JPEG_FMT_TYPE_OUTPUT) ?
-		   MTK_JPEG_FMT_FLAG_DEC_OUTPUT :
-		   MTK_JPEG_FMT_FLAG_DEC_CAPTURE;
+	if (ctx->jpeg->mode ==  MTK_JPEG_ENC)
+		fmt_flag = (fmt_type == MTK_JPEG_FMT_TYPE_OUTPUT) ?
+			   MTK_JPEG_FMT_FLAG_ENC_OUTPUT :
+			   MTK_JPEG_FMT_FLAG_ENC_CAPTURE;
+	else
+		fmt_flag = (fmt_type == MTK_JPEG_FMT_TYPE_OUTPUT) ?
+			   MTK_JPEG_FMT_FLAG_DEC_OUTPUT :
+			   MTK_JPEG_FMT_FLAG_DEC_CAPTURE;
 
 	for (k = 0; k < MTK_JPEG_NUM_FORMATS; k++) {
 		struct mtk_jpeg_fmt *fmt = &mtk_jpeg_formats[k];
@@ -196,9 +314,99 @@ static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx,
 	}
 }
 
Quoted 14 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 325

Quoted 12 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +9 -3
-static int mtk_jpeg_try_fmt_mplane(struct v4l2_format *f,
-				   struct mtk_jpeg_fmt *fmt,
-				   struct mtk_jpeg_ctx *ctx, int q_type)
+static int mtk_jpeg_try_enc_fmt_mplane(struct v4l2_format *f,
+				       struct mtk_jpeg_fmt *fmt,
+				       struct mtk_jpeg_ctx *ctx, int q_type)
+{
+	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+	int i, align_w, align_h;
+
+	memset(pix_mp->reserved, 0, sizeof(pix_mp->reserved));

The core already takes care of zeroing this.

keep this, cancel following zeroing pix_mp->reserved.

Quoted 15 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 336

Quoted 12 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +11 -0
+	pix_mp->field = V4L2_FIELD_NONE;
+
+	if (ctx->state != MTK_JPEG_INIT) {
+		mtk_jpeg_adjust_fmt_mplane(ctx, f);
+		goto end;
+	}
+
+	pix_mp->num_planes = fmt->colplanes;
+	pix_mp->pixelformat = fmt->fourcc;
+
+	if (q_type == MTK_JPEG_FMT_TYPE_OUTPUT) {

There are separate ops for out_mplane and cap_mplane. Wouldn't it make more sense to have them handled in separate functions?

don't separate ops for out_mplane and cap_mplane, because the modified code of this function as your following advice is brief.

Quoted 10 lines
Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +9 -0
+		pix_mp->height = clamp(pix_mp->height, MTK_JPEG_MIN_HEIGHT,
+				       MTK_JPEG_MAX_HEIGHT);
+		pix_mp->width = clamp(pix_mp->width, MTK_JPEG_MIN_WIDTH,
+				      MTK_JPEG_MAX_WIDTH);
+		align_w = pix_mp->width;
+		align_h = pix_mp->height;
+		align_w = round_up(align_w, 2);
+		if (pix_mp->num_planes == 1U) {
+			mtk_jpeg_bound_align_image(&align_w,
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 350

Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -0
+						   MTK_JPEG_MIN_WIDTH,
+						   MTK_JPEG_MAX_WIDTH, 5,
+						   &align_h,
+						   MTK_JPEG_MIN_HEIGHT,
+						   MTK_JPEG_MAX_HEIGHT, 3);

I'd suggest removing the mtk_jpeg_bound_align_image(), as we don't need the complex computation that's happening there. All we need is:

done

Quoted 9 lines
	pix_mp->height = clamp(round_up(pix_mp->height, 8), MTK_JPEG_MIN_HEIGHT,
			       MTK_JPEG_MAX_HEIGHT);
	pix_mp->width = clamp(round_up(pix_mp->width, 32), MTK_JPEG_MIN_WIDTH,
			      MTK_JPEG_MAX_WIDTH);

Note that contrary to the existing code, the aligned width and height _must_ be returned back to the application and not only hidden inside bytesperline and sizeimage.

done

Quoted 20 lines
Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -0
+			align_w = align_w << 1;
+			pix_mp->plane_fmt[0].bytesperline = align_w;
+			pix_mp->plane_fmt[0].sizeimage = align_w * align_h;
+		} else if (pix_mp->num_planes == 2U) {
+			mtk_jpeg_bound_align_image(&align_w,

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 365

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+						   MTK_JPEG_MIN_WIDTH,
+						   MTK_JPEG_MAX_WIDTH, 4,
+						   &align_h,
+						   MTK_JPEG_MIN_HEIGHT,
+						   MTK_JPEG_MAX_HEIGHT, 4);
+			pix_mp->plane_fmt[0].bytesperline = align_w;
+			pix_mp->plane_fmt[0].sizeimage = align_w * align_h;
+			pix_mp->plane_fmt[1].bytesperline = align_w;
+			pix_mp->plane_fmt[1].sizeimage =
+				(align_w * align_h) / 2;

Could we parametrize these calculations from the format array and use a for loop over the planes instead?

Quoted 6 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 369

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -0
+		} else {
+			v4l2_err(&jpeg->v4l2_dev,
+				 "Unsupport num planes = %d\n",
+				 pix_mp->num_planes);

Isn't this impossible?

yes, deleted it.

Quoted 10 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 376

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +7 -0
+		}
+	} else {
+		mtk_jpeg_bound_align_image(&pix_mp->width, MTK_JPEG_MIN_WIDTH,
+					   MTK_JPEG_MAX_WIDTH, 0,
+					   &pix_mp->height,
+					   MTK_JPEG_MIN_HEIGHT,
+					   MTK_JPEG_MAX_HEIGHT, 0);

Same here, could we just use clamp() directly?

done

Quoted 5 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 378

Quoted 3 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -0
+
+		if (fmt->fourcc == V4L2_PIX_FMT_JPEG) {

Can we have any other format on this queue?

for jpeg enc ,only V4L2_PIX_FMT_JPEG format

Quoted 7 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 382

Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -0
+			pix_mp->plane_fmt[0].bytesperline = 0;
+			pix_mp->plane_fmt[0].sizeimage =
+				mtk_jpeg_align(pix_mp->plane_fmt[0].sizeimage,
+					       128);

What's mtk_jpeg_align()? Why couldn't we just use round_up() directly?

done

Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 385

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -0
+			if (pix_mp->plane_fmt[0].sizeimage == 0)
+				pix_mp->plane_fmt[0].sizeimage =
+					MTK_JPEG_DEFAULT_SIZEIMAGE;

I think there is some reasonable upper bound of the compressed image size given the resolution. Could we use that here instead?

Quoted 9 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 392

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +7 -0
+		}
+	}
+
+	for (i = 0; i < fmt->colplanes; i++) {
+		struct v4l2_plane_pix_format *pfmt =
+				&pix_mp->plane_fmt[i];
+		memset(pfmt->reserved, 0, sizeof(pfmt->reserved));

The core already zeroes this.

canceled this

Quoted 14 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx, · line 403

Quoted 12 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +11 -0
+	}
+end:
+	v4l2_dbg(2, debug, &jpeg->v4l2_dev, "wxh:%ux%u\n",
+		 pix_mp->width, pix_mp->height);
+	for (i = 0; i < pix_mp->num_planes; i++) {
+		v4l2_dbg(2, debug, &jpeg->v4l2_dev,
+			 "plane[%d] bpl=%u, size=%u\n",
+			 i,
+			 pix_mp->plane_fmt[i].bytesperline,
+			 pix_mp->plane_fmt[i].sizeimage);
+	}

Generic V4L2 debug logs already print this.

canceled this

Quoted 13 lines
Quoted 13 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
+	return 0;
+}
+
+static int mtk_jpeg_try_dec_fmt_mplane(struct v4l2_format *f,
+				       struct mtk_jpeg_fmt *fmt,
+				       struct mtk_jpeg_ctx *ctx, int q_type)
 {
 	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
 	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
@@ -233,6 +441,10 @@ static int mtk_jpeg_try_fmt_mplane(struct v4l2_format *f,
 	}
 
Quoted 7 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_try_fmt_mplane(struct v4l2_format *f, · line 447

Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -0
 	/* type is MTK_JPEG_FMT_TYPE_CAPTURE */
+	pix_mp->height = clamp(pix_mp->height, MTK_JPEG_MIN_HEIGHT,
+			       MTK_JPEG_MAX_HEIGHT);
+	pix_mp->width = clamp(pix_mp->width, MTK_JPEG_MIN_WIDTH,
+			      MTK_JPEG_MAX_WIDTH);

Why is this needed? Doesn't the call below already include these clamps?

cancel this

Quoted 4 lines

(That said, it would be cleaner if mtk_jpeg_bound_align_image() was removed and replaced with clamp() calls directly, because of the confusing behavior of v4l_bound_align_image().

done

Quoted 16 lines
Quoted 16 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -0
 	mtk_jpeg_bound_align_image(&pix_mp->width, MTK_JPEG_MIN_WIDTH,
 				   MTK_JPEG_MAX_WIDTH, fmt->h_align,
 				   &pix_mp->height, MTK_JPEG_MIN_HEIGHT,
@@ -316,6 +528,7 @@ static int mtk_jpeg_try_fmt_vid_cap_mplane(struct file *file, void *priv,
 {
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
 	struct mtk_jpeg_fmt *fmt;
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
 
 	fmt = mtk_jpeg_find_format(ctx, f->fmt.pix_mp.pixelformat,
 				   MTK_JPEG_FMT_TYPE_CAPTURE);
@@ -329,7 +542,12 @@ static int mtk_jpeg_try_fmt_vid_cap_mplane(struct file *file, void *priv,
 		 (fmt->fourcc >> 16 & 0xff),
 		 (fmt->fourcc >> 24 & 0xff));
 
Quoted 11 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_try_fmt_vid_cap_mplane(struct file *file, void *priv, · line 550

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -1
-	return mtk_jpeg_try_fmt_mplane(f, fmt, ctx, MTK_JPEG_FMT_TYPE_CAPTURE);
+
+	return jpeg->mode == MTK_JPEG_ENC ?
+	       mtk_jpeg_try_enc_fmt_mplane(f, fmt, ctx,
+					   MTK_JPEG_FMT_TYPE_CAPTURE) :
+	       mtk_jpeg_try_dec_fmt_mplane(f, fmt, ctx,
+					   MTK_JPEG_FMT_TYPE_CAPTURE);

The ternary operator isn't a universal replacement for if/else. It should be limited to simple expressions only. Function calls don't count as simple expressions.

used if/else to replace this

Quoted 16 lines
Quoted 16 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -0
 }
 
 static int mtk_jpeg_try_fmt_vid_out_mplane(struct file *file, void *priv,
@@ -337,6 +555,7 @@ static int mtk_jpeg_try_fmt_vid_out_mplane(struct file *file, void *priv,
 {
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
 	struct mtk_jpeg_fmt *fmt;
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
 
 	fmt = mtk_jpeg_find_format(ctx, f->fmt.pix_mp.pixelformat,
 				   MTK_JPEG_FMT_TYPE_OUTPUT);
@@ -350,7 +569,12 @@ static int mtk_jpeg_try_fmt_vid_out_mplane(struct file *file, void *priv,
 		 (fmt->fourcc >> 16 & 0xff),
 		 (fmt->fourcc >> 24 & 0xff));
 
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_try_fmt_vid_out_mplane(struct file *file, void *priv, · line 576

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -1
-	return mtk_jpeg_try_fmt_mplane(f, fmt, ctx, MTK_JPEG_FMT_TYPE_OUTPUT);
+	return jpeg->mode == MTK_JPEG_ENC ?
+	       mtk_jpeg_try_enc_fmt_mplane(f, fmt, ctx,
+					   MTK_JPEG_FMT_TYPE_OUTPUT) :
+	       mtk_jpeg_try_dec_fmt_mplane(f, fmt, ctx,
+					   MTK_JPEG_FMT_TYPE_OUTPUT);

Ditto.

done

Quoted 4 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_try_fmt_vid_out_mplane(struct file *file, void *priv, · line 577

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -0
+

Unnecessary blank line.

deleted

Quoted 67 lines
Quoted 67 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +35 -2
 }
 
 static int mtk_jpeg_s_fmt_mplane(struct mtk_jpeg_ctx *ctx,
@@ -455,11 +679,19 @@ static int mtk_jpeg_g_selection(struct file *file, void *priv,
 				struct v4l2_selection *s)
 {
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
 
-	if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+	if (jpeg->mode == MTK_JPEG_ENC && s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)
+		return -EINVAL;
+
+	if (jpeg->mode == MTK_JPEG_DEC &&
+	    s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
 		return -EINVAL;
 
 	switch (s->target) {
+	case V4L2_SEL_TGT_CROP:
+	case V4L2_SEL_TGT_CROP_BOUNDS:
+	case V4L2_SEL_TGT_CROP_DEFAULT:
 	case V4L2_SEL_TGT_COMPOSE:
 	case V4L2_SEL_TGT_COMPOSE_DEFAULT:
 		s->r.width = ctx->out_q.w;
@@ -484,11 +716,17 @@ static int mtk_jpeg_s_selection(struct file *file, void *priv,
 				struct v4l2_selection *s)
 {
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
 
-	if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+	if (jpeg->mode == MTK_JPEG_ENC && s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)
+		return -EINVAL;
+
+	if (jpeg->mode == MTK_JPEG_DEC &&
+	    s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
 		return -EINVAL;
 
 	switch (s->target) {
+	case V4L2_SEL_TGT_CROP:
 	case V4L2_SEL_TGT_COMPOSE:
 		s->r.left = 0;
 		s->r.top = 0;
@@ -658,10 +896,92 @@ static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx,
 		 param->dec_w, param->dec_h);
 }
 
+static void mtk_jpeg_set_param(struct mtk_jpeg_ctx *ctx,
+			       struct mtk_jpeg_enc_param *param)
+{
+	struct mtk_jpeg_q_data *q_data_src = &ctx->out_q;
+	struct jpeg_enc_param *jpeg_params = &ctx->jpeg_param;
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+	u32 width_even;
+	u32 is_420;
+	u32 padding_width;
+	u32 padding_height;
+
+	switch (q_data_src->fmt->fourcc) {
+	case V4L2_PIX_FMT_YUYV:
+		param->enc_format = JPEG_YUV_FORMAT_YUYV;
+		break;
+	case V4L2_PIX_FMT_YVYU:
+		param->enc_format = JPEG_YUV_FORMAT_YVYU;
+		break;
+	case V4L2_PIX_FMT_NV12M:
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx, · line 922

Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -0
+		param->enc_format = JPEG_YUV_FORMAT_NV12;
+		break;
+	case V4L2_PIX_FMT_NV21M:
+		param->enc_format = JPEG_YUV_FORMAT_NV12;
+		break;

Could we include enc_format in the mtk_jpeg_fmt struct and take it from there, eliminating the need for a switch over all supported formats?

the enc_format is just for encoder hw,but mtk_jpeg_fmt_struct is for encoder and decoder,so I didn't modify this, but I canceled this function as your advice

Quoted 7 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx, · line 926

Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -0
+	default:
+		v4l2_err(&jpeg->v4l2_dev, "Unsupport fourcc =%d\n",
+			 q_data_src->fmt->fourcc);
+		break;

Isn't this impossible?

yes

Quoted 12 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx, · line 934

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +8 -0
+	}
+	param->enc_w = q_data_src->w;
+	param->enc_h = q_data_src->h;
+
+	if (jpeg_params->enc_quality >= 97)
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q97;
+	else if (jpeg_params->enc_quality >= 95)
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q95;

I'm wondering if the application requests 96, it doesn't expect the quality to be _at_least_ 96.

our jpeg enc hw do not support quality 96,only support 15 kinds of quant table listed here, so if the application requests 96,a nearest and highest quality will be given.

Quoted 14 lines
Quoted 14 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +13 -0
+	else if (jpeg_params->enc_quality >= 92)
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q92;
+	else if (jpeg_params->enc_quality >= 90)
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q90;
+	else if (jpeg_params->enc_quality >= 87)
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q87;
+	else if (jpeg_params->enc_quality >= 84)
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q84;
+	else if (jpeg_params->enc_quality >= 80)
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q80;
+	else if (jpeg_params->enc_quality >= 74)
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q74;
+	else if (jpeg_params->enc_quality >= 64)
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx, · line 952

Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -0
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q64;
+	else if (jpeg_params->enc_quality >= 60)
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q60;
+	else
+		param->enc_quality = JPEG_ENCODE_QUALITY_Q48;

We could put this in a sorted array of <quality paramter, hardware value> pairs and use a simple for loop instead.

done

Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx, · line 957

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -0
+
+	param->enable_exif = jpeg_params->enable_exif;
+	param->restart_interval = jpeg_params->restart_interval;
+
+	width_even = ((param->enc_w + 1) >> 1) << 1;

Isn't this equivalent to round_up(param->enc_w, 2)?

yes

Quoted 11 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx, · line 965

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +8 -0
+	is_420 = (param->enc_format == JPEG_YUV_FORMAT_NV12 ||
+		  param->enc_format == JPEG_YUV_FORMAT_NV12) ? 1 : 0;
+	padding_width = mtk_jpeg_align(param->enc_w, 16);
+	padding_height = mtk_jpeg_align(param->enc_h, is_420 ? 16 : 8);
+	if (!is_420)
+		width_even = width_even << 1;
+
+	param->img_stride = mtk_jpeg_align(width_even, (is_420 ? 16 : 32));

Isn't this the same as bytesperline in the v4l2_pix_fmt_mplane struct?

yes

Quoted 4 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx, · line 966

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -0
+	param->mem_stride = mtk_jpeg_align(width_even, (is_420 ? 16 : 32));

What's the difference between img_stride and mem_stride?

In theory, mem_stride need >= img_stride,but we use the same is ok

Quoted 9 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx, · line 969

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -0
+	param->total_encdu =
+		((padding_width >> 4) * (padding_height >> (is_420 ? 4 : 3)) *
+		(is_420 ? 6 : 4)) - 1;

The comment above the struct says this is the total number of 8x8 blocks. Why would it depend on whether the format is YUV 4:2:0? Since we should have already aligned the width and height in try_fmt, this should be as simple as (width / 8) * (height / 8).

becuase the image size is w*h*1.5 for yuv420 format, but w*h*2 for yuv422,so for yuv420: w_16/8*h_16/8*1.5-1(because the hw will start at number 0), yuv422: w_32/8*h_8/8*2-1,this number is equal to my code.

Quoted 14 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx, · line 978

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +9 -0
+
+	v4l2_dbg(0, 2, &jpeg->v4l2_dev, "fmt %d, w,h %d,%d, enable_exif %d,",
+		 "enc_quality %d, restart_interval %d,img_stride %d,",
+		 "mem_stride %d,totalEncDu %d\n",
+		 param->enc_format, param->enc_w, param->enc_h,
+		 param->enable_exif, param->enc_quality,
+		 param->restart_interval, param->img_stride,
+		 param->mem_stride, param->total_encdu);
+}

Do we need this intermediate step, with an intermediate representation of the parameters? Couldn't we just program the registers directly based on the original V4L2 values, as other drivers normally do?

canceled this

Quoted 27 lines
Quoted 27 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -13
+
 static void mtk_jpeg_buf_queue(struct vb2_buffer *vb)
 {
 	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
 	struct mtk_jpeg_dec_param *param;
+	struct mtk_jpeg_enc_param *enc_param;
 	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
 	struct mtk_jpeg_src_buf *jpeg_src_buf;
 	bool header_valid;
@@ -673,29 +993,45 @@ static void mtk_jpeg_buf_queue(struct vb2_buffer *vb)
 		goto end;
 
 	jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(vb);
-	param = &jpeg_src_buf->dec_param;
-	memset(param, 0, sizeof(*param));
-
-	if (jpeg_src_buf->flags & MTK_JPEG_BUF_FLAGS_LAST_FRAME) {
-		v4l2_dbg(1, debug, &jpeg->v4l2_dev, "Got eos\n");
-		goto end;
-	}
-	header_valid = mtk_jpeg_parse(param, (u8 *)vb2_plane_vaddr(vb, 0),
-				      vb2_get_plane_payload(vb, 0));
-	if (!header_valid) {
-		v4l2_err(&jpeg->v4l2_dev, "Header invalid.\n");
-		vb2_buffer_done(vb, VB2_BUF_STATE_ERROR);
-		return;
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_buf_queue(struct vb2_buffer *vb) · line 996

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -5
-	}
-
-	if (ctx->state == MTK_JPEG_INIT) {
-		struct vb2_queue *dst_vq = v4l2_m2m_get_vq(
-			ctx->fh.m2m_ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
+	if (jpeg->mode ==  MTK_JPEG_ENC) {

nit: Double space after ==.

deleted it

Quoted 11 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_buf_queue(struct vb2_buffer *vb) · line 1002

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
+		enc_param = &jpeg_src_buf->enc_param;
+		memset(enc_param, 0, sizeof(*enc_param));
+		mtk_jpeg_set_param(ctx, enc_param);
+		if (jpeg_src_buf->flags & MTK_JPEG_BUF_FLAGS_LAST_FRAME) {
+			v4l2_dbg(1, debug, &jpeg->v4l2_dev, "Got eos");
+			goto end;

Shouldn't we still change the state to RUNNING? The last frame still needs to be encoded and the condition for getting back to the INIT state is stopping the streaming.

done

Quoted 63 lines
Quoted 63 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +34 -7
+		}
+		if (ctx->state == MTK_JPEG_INIT)
+			ctx->state = MTK_JPEG_RUNNING;
+	} else {
+		param = &jpeg_src_buf->dec_param;
+		memset(param, 0, sizeof(*param));
+
+		if (jpeg_src_buf->flags & MTK_JPEG_BUF_FLAGS_LAST_FRAME) {
+			v4l2_dbg(1, debug, &jpeg->v4l2_dev, "Got eos\n");
+			goto end;
+		}
+		header_valid = mtk_jpeg_parse(param,
+					      (u8 *)vb2_plane_vaddr(vb, 0),
+					      vb2_get_plane_payload(vb, 0));
+		if (!header_valid) {
+			v4l2_err(&jpeg->v4l2_dev, "Header invalid.\n");
+			vb2_buffer_done(vb, VB2_BUF_STATE_ERROR);
+			return;
+		}
 
-		mtk_jpeg_queue_src_chg_event(ctx);
-		mtk_jpeg_set_queue_data(ctx, param);
-		ctx->state = vb2_is_streaming(dst_vq) ?
-				MTK_JPEG_SOURCE_CHANGE : MTK_JPEG_RUNNING;
+		if (ctx->state == MTK_JPEG_INIT) {
+			struct vb2_queue *dst_vq;
+
+			dst_vq = v4l2_m2m_get_vq
+					(ctx->fh.m2m_ctx,
+					 V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
+			mtk_jpeg_queue_src_chg_event(ctx);
+			mtk_jpeg_set_queue_data(ctx, param);
+			ctx->state = vb2_is_streaming(dst_vq) ?
+					MTK_JPEG_SOURCE_CHANGE :
+					MTK_JPEG_RUNNING;
+		}
 	}
 end:
 	v4l2_m2m_buf_queue(ctx->fh.m2m_ctx, to_vb2_v4l2_buffer(vb));
@@ -738,16 +1074,16 @@ static void mtk_jpeg_stop_streaming(struct vb2_queue *q)
 	 * subsampling. Update capture queue when the stream is off.
 	 */
 	if (ctx->state == MTK_JPEG_SOURCE_CHANGE &&
-	    !V4L2_TYPE_IS_OUTPUT(q->type)) {
+	    !V4L2_TYPE_IS_OUTPUT(q->type) &&
+	    ctx->jpeg->mode == MTK_JPEG_DEC) {
 		struct mtk_jpeg_src_buf *src_buf;
 
 		vb = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
 		src_buf = mtk_jpeg_vb2_to_srcbuf(&vb->vb2_buf);
 		mtk_jpeg_set_queue_data(ctx, &src_buf->dec_param);
 		ctx->state = MTK_JPEG_RUNNING;
-	} else if (V4L2_TYPE_IS_OUTPUT(q->type)) {
+	} else if (V4L2_TYPE_IS_OUTPUT(q->type))
 		ctx->state = MTK_JPEG_INIT;
-	}
 
 	while ((vb = mtk_jpeg_buf_remove(ctx, q->type)))
 		v4l2_m2m_buf_done(vb, VB2_BUF_STATE_ERROR);
@@ -802,6 +1138,34 @@ static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx,
 	return 0;
 }
 
Quoted 12 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx, · line 1148

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +8 -0
+static void mtk_jpeg_set_enc_dst(struct mtk_jpeg_ctx *ctx,
+				 struct vb2_buffer *dst_buf,
+				 struct mtk_jpeg_enc_bs *bs)
+{
+	struct jpeg_enc_param *p = &ctx->jpeg_param;
+
+	bs->dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0) &
+		      (~JPEG_ENC_DST_ADDR_OFFSET_MASK);

Wouldn't the hardware attempt to access the memory before the buffer in this case? This is a recipe for a disaster (e.g. the hardware overwriting some memory it shouldn't touch).

have deleted this

Quoted 8 lines

Could we instead check the buffer address alignment in .buf_prepare and fail if it's not big enough?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx, · line 1150

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -0
+	bs->dma_addr_offset = p->enable_exif ? MTK_JPEG_DEFAULT_EXIF_SIZE : 0;
+	bs->dma_addr_offsetmask = bs->dma_addr & JPEG_ENC_DST_ADDR_OFFSET_MASK;

What is the meaning of this offset mask?

our actual destination address = destination address + offset address+ destination address offset mask.The mask 0:No offset,1~15:offset byte from the 16-byte aligned

Quoted 5 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx, · line 1151

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -0
+	bs->size = mtk_jpeg_align(vb2_plane_size(dst_buf, 0), 128);

Wouldn't this let the hardware write beyond the buffer if the original size was not a multiple of 128? Shouldn't we round_down() insted?

done

Quoted 16 lines
Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -0
+}
+
+static void mtk_jpeg_set_enc_src(struct mtk_jpeg_ctx *ctx,

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx, · line 1164

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+				 struct vb2_buffer *src_buf,
+				 struct mtk_jpeg_enc_fb *fb)
+{
+	int i;
+
+	for (i = 0; i < src_buf->num_planes; i++) {
+		fb->fb_addr[i].dma_addr =
+			vb2_dma_contig_plane_dma_addr(src_buf, i) +
+			src_buf->planes[i].data_offset;
+		fb->fb_addr[i].size = (size_t)src_buf->planes[i].length -

The proper way to access plane length is to call vb2_plane_size().

done

Quoted 25 lines
Quoted 25 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +7 -1
+				      src_buf->planes[i].data_offset;
+	}
+}
+
 static void mtk_jpeg_device_run(void *priv)
 {
 	struct mtk_jpeg_ctx *ctx = priv;
@@ -812,6 +1176,8 @@ static void mtk_jpeg_device_run(void *priv)
 	struct mtk_jpeg_src_buf *jpeg_src_buf;
 	struct mtk_jpeg_bs bs;
 	struct mtk_jpeg_fb fb;
+	struct mtk_jpeg_enc_bs enc_bs;
+	struct mtk_jpeg_enc_fb enc_fb;
 	int i;
 
 	src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
@@ -822,30 +1188,45 @@ static void mtk_jpeg_device_run(void *priv)
 		for (i = 0; i < dst_buf->vb2_buf.num_planes; i++)
 			vb2_set_plane_payload(&dst_buf->vb2_buf, i, 0);
 		buf_state = VB2_BUF_STATE_DONE;
-		goto dec_end;
+		goto device_run_end;
 	}
 
Quoted 14 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_device_run(void *priv) · line 1198

Quoted 11 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -6
-	if (mtk_jpeg_check_resolution_change(ctx, &jpeg_src_buf->dec_param)) {
-		mtk_jpeg_queue_src_chg_event(ctx);
-		ctx->state = MTK_JPEG_SOURCE_CHANGE;
-		v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
-		return;
-	}
+	if (jpeg->mode == MTK_JPEG_ENC) {
+		mtk_jpeg_set_enc_dst(ctx, &dst_buf->vb2_buf, &enc_bs);
+		mtk_jpeg_set_enc_src(ctx, &src_buf->vb2_buf, &enc_fb);
+
+		spin_lock_irqsave(&jpeg->hw_lock, flags);

Do we actually need this lock? device_run can be only called once and only after the hardware finished processing the current job.

canceled this

Quoted 69 lines
Quoted 27 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +18 -6
+		mtk_jpeg_enc_reset(jpeg->reg_base);
+		mtk_jpeg_enc_set_config(jpeg->reg_base,
+					&jpeg_src_buf->enc_param, &enc_bs,
+					&enc_fb);
+
+		mtk_jpeg_enc_start(jpeg->reg_base);
+	} else {
+		if (mtk_jpeg_check_resolution_change
+			(ctx, &jpeg_src_buf->dec_param)) {
+			mtk_jpeg_queue_src_chg_event(ctx);
+			ctx->state = MTK_JPEG_SOURCE_CHANGE;
+			v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
+			return;
+		}
 
-	mtk_jpeg_set_dec_src(ctx, &src_buf->vb2_buf, &bs);
-	if (mtk_jpeg_set_dec_dst(ctx, &jpeg_src_buf->dec_param, &dst_buf->vb2_buf, &fb))
-		goto dec_end;
+		mtk_jpeg_set_dec_src(ctx, &src_buf->vb2_buf, &bs);
+		if (mtk_jpeg_set_dec_dst(ctx, &jpeg_src_buf->dec_param,
+					 &dst_buf->vb2_buf, &fb))
+			goto device_run_end;
 
-	spin_lock_irqsave(&jpeg->hw_lock, flags);
-	mtk_jpeg_dec_reset(jpeg->dec_reg_base);
-	mtk_jpeg_dec_set_config(jpeg->dec_reg_base,

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_device_run(void *priv) · line 1226

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -2
-				&jpeg_src_buf->dec_param, &bs, &fb);
+		spin_lock_irqsave(&jpeg->hw_lock, flags);
+		mtk_jpeg_dec_reset(jpeg->reg_base);
+		mtk_jpeg_dec_set_config(jpeg->reg_base,
+					&jpeg_src_buf->dec_param, &bs, &fb);
 
-	mtk_jpeg_dec_start(jpeg->dec_reg_base);
+		mtk_jpeg_dec_start(jpeg->reg_base);
+	}
 	spin_unlock_irqrestore(&jpeg->hw_lock, flags);

It is a good practice for spin_lock and spin_unlock to be in the same scope, for readability reasons.

Quoted 28 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +7 -5
 	return;
 
-dec_end:
+device_run_end:
 	v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
 	v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
 	v4l2_m2m_buf_done(src_buf, buf_state);
@@ -905,30 +1286,30 @@ static void mtk_jpeg_clk_on(struct mtk_jpeg_dev *jpeg)
 	ret = mtk_smi_larb_get(jpeg->larb);
 	if (ret)
 		dev_err(jpeg->dev, "mtk_smi_larb_get larbvdec fail %d\n", ret);
-	clk_prepare_enable(jpeg->clk_jdec_smi);
-	clk_prepare_enable(jpeg->clk_jdec);
+	if (jpeg->mode == MTK_JPEG_DEC)
+		clk_prepare_enable(jpeg->clk_jpeg_smi);
+	clk_prepare_enable(jpeg->clk_jpeg);
 }
 
 static void mtk_jpeg_clk_off(struct mtk_jpeg_dev *jpeg)
 {
-	clk_disable_unprepare(jpeg->clk_jdec);
-	clk_disable_unprepare(jpeg->clk_jdec_smi);
+	clk_disable_unprepare(jpeg->clk_jpeg);
+	if (jpeg->mode == MTK_JPEG_DEC)
+		clk_disable_unprepare(jpeg->clk_jpeg_smi);
 	mtk_smi_larb_put(jpeg->larb);
 }
 
Quoted 14 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_clk_on(struct mtk_jpeg_dev *jpeg) · line 1309

Quoted 11 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -3
-static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv)
+static irqreturn_t mtk_jpeg_irq(int irq, void *priv)
 {
 	struct mtk_jpeg_dev *jpeg = priv;
 	struct mtk_jpeg_ctx *ctx;
 	struct vb2_v4l2_buffer *src_buf, *dst_buf;
 	struct mtk_jpeg_src_buf *jpeg_src_buf;
 	enum vb2_buffer_state buf_state = VB2_BUF_STATE_ERROR;
-	u32	dec_irq_ret;
-	u32 dec_ret;
+	u32	irq_ret;

In the kernel coding style, type is separated from variable name with a single space.

done

Quoted 106 lines
Quoted 106 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +72 -13
+	u32 ret, result_size;
 	int i;
 
-	dec_ret = mtk_jpeg_dec_get_int_status(jpeg->dec_reg_base);
-	dec_irq_ret = mtk_jpeg_dec_enum_result(dec_ret);
 	ctx = v4l2_m2m_get_curr_priv(jpeg->m2m_dev);
 	if (!ctx) {
 		v4l2_err(&jpeg->v4l2_dev, "Context is NULL\n");
@@ -939,39 +1320,107 @@ static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv)
 	dst_buf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
 	jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(&src_buf->vb2_buf);
 
-	if (dec_irq_ret >= MTK_JPEG_DEC_RESULT_UNDERFLOW)
-		mtk_jpeg_dec_reset(jpeg->dec_reg_base);
+	if (jpeg->mode == MTK_JPEG_ENC) {
+		ret = mtk_jpeg_enc_get_int_status(jpeg->reg_base);
+		irq_ret = mtk_jpeg_enc_enum_result(jpeg->reg_base, ret,
+						   &result_size);
 
-	if (dec_irq_ret != MTK_JPEG_DEC_RESULT_EOF_DONE) {
-		dev_err(jpeg->dev, "decode failed\n");
-		goto dec_end;
-	}
+		if (irq_ret >= MTK_JPEG_ENC_RESULT_STALL)
+			mtk_jpeg_enc_reset(jpeg->reg_base);
+
+		if (irq_ret != MTK_JPEG_ENC_RESULT_DONE) {
+			dev_err(jpeg->dev, "encode failed\n");
+			goto irq_end;
+		}
+
+		vb2_set_plane_payload(&dst_buf->vb2_buf, 0,
+				      result_size);
+	} else {
+		ret = mtk_jpeg_dec_get_int_status(jpeg->reg_base);
+		irq_ret = mtk_jpeg_dec_enum_result(ret);
 
-	for (i = 0; i < dst_buf->vb2_buf.num_planes; i++)
-		vb2_set_plane_payload(&dst_buf->vb2_buf, i,
-				      jpeg_src_buf->dec_param.comp_size[i]);
+		if (irq_ret >= MTK_JPEG_DEC_RESULT_UNDERFLOW)
+			mtk_jpeg_dec_reset(jpeg->reg_base);
+
+		if (irq_ret != MTK_JPEG_DEC_RESULT_EOF_DONE) {
+			dev_err(jpeg->dev, "decode failed\n");
+			goto irq_end;
+		}
+
+		for (i = 0; i < dst_buf->vb2_buf.num_planes; i++)
+			vb2_set_plane_payload
+				(&dst_buf->vb2_buf, i,
+				 jpeg_src_buf->dec_param.comp_size[i]);
+	}
 
 	buf_state = VB2_BUF_STATE_DONE;
 
-dec_end:
+irq_end:
 	v4l2_m2m_buf_done(src_buf, buf_state);
 	v4l2_m2m_buf_done(dst_buf, buf_state);
 	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
 	return IRQ_HANDLED;
 }
 
-static void mtk_jpeg_set_default_params(struct mtk_jpeg_ctx *ctx)
+static void mtk_jpeg_set_enc_default_params(struct mtk_jpeg_ctx *ctx)
+{
+	struct mtk_jpeg_q_data *q = &ctx->out_q;
+	int align_w, align_h;
+
+	ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+
+	ctx->colorspace = V4L2_COLORSPACE_JPEG,
+	ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
+	ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+
+	q->w = MTK_JPEG_MIN_WIDTH;
+	q->h = MTK_JPEG_MIN_HEIGHT;
+
+	q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_YUYV,
+				      MTK_JPEG_FMT_TYPE_OUTPUT);
+
+	align_w = q->w;
+	align_h = q->h;
+	align_w = round_up(align_w, 2);
+	v4l_bound_align_image(&align_w, MTK_JPEG_MIN_WIDTH, MTK_JPEG_MAX_WIDTH,
+			      5, &align_h, MTK_JPEG_MIN_HEIGHT,
+			      MTK_JPEG_MAX_HEIGHT, 3, 0);
+	align_w = align_w << 1;
+
+	if (align_w < MTK_JPEG_MIN_WIDTH &&
+	    (align_w + 32) <= MTK_JPEG_MAX_WIDTH)
+		align_w += 32;
+	if (align_h < MTK_JPEG_MIN_HEIGHT &&
+	    (align_h + 8) <= MTK_JPEG_MAX_HEIGHT)
+		align_h += 8;
+
+	q->sizeimage[0] = align_w * align_h;
+	q->bytesperline[0] = align_w;
+
+	q = &ctx->cap_q;
+	q->w = MTK_JPEG_MIN_WIDTH;
+	q->h = MTK_JPEG_MIN_HEIGHT;
+	q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_JPEG,
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1408

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -0
+				      MTK_JPEG_FMT_TYPE_CAPTURE);
+	q->bytesperline[0] = 0;
+	q->sizeimage[0] = MTK_JPEG_DEFAULT_SIZEIMAGE;
+}

Could we just create an arbitrary v4l2_pix_format_mplane struct and call s_fmt instead? In general, all of the constant values and alignments should be already ensured by try_fmt, so this function should be redundant.

if cancel this function,the v4l2-compliance test will fail

Quoted 33 lines
Quoted 33 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -2
+
+static void mtk_jpeg_set_dec_default_params(struct mtk_jpeg_ctx *ctx)
 {
 	struct mtk_jpeg_q_data *q = &ctx->out_q;
 	int i;
 
+	ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+
 	ctx->colorspace = V4L2_COLORSPACE_JPEG,
 	ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
 	ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
 	ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;
 
 	q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_JPEG,
-					      MTK_JPEG_FMT_TYPE_OUTPUT);
+				      MTK_JPEG_FMT_TYPE_OUTPUT);
 	q->w = MTK_JPEG_MIN_WIDTH;
 	q->h = MTK_JPEG_MIN_HEIGHT;
 	q->bytesperline[0] = 0;
@@ -979,7 +1428,7 @@ static void mtk_jpeg_set_default_params(struct mtk_jpeg_ctx *ctx)
 
 	q = &ctx->cap_q;
 	q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_YUV420M,
-					      MTK_JPEG_FMT_TYPE_CAPTURE);
+				      MTK_JPEG_FMT_TYPE_CAPTURE);
 	q->w = MTK_JPEG_MIN_WIDTH;
 	q->h = MTK_JPEG_MIN_HEIGHT;
 
@@ -1020,7 +1469,15 @@ static int mtk_jpeg_open(struct file *file)
 		goto error;
 	}
 
Quoted 13 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_open(struct file *file) · line 1480

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +9 -1
-	mtk_jpeg_set_default_params(ctx);
+	ret = mtk_jpeg_ctrls_setup(ctx);
+	if (ret) {
+		v4l2_err(&jpeg->v4l2_dev, "Failed to setup controls() (%d)\n",
+			 ret);
+		goto error;
+	}
+
+	jpeg->mode == MTK_JPEG_ENC ? mtk_jpeg_set_enc_default_params(ctx) :
+				     mtk_jpeg_set_dec_default_params(ctx);

As mentioned before, this kind of usage of the ternary operator is generaly discouraged, because it makes the code difficult to read.

done

Quoted 6 lines
Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +0 -0
 	mutex_unlock(&jpeg->lock);
 	return 0;
 
@@ -1040,6 +1497,7 @@ static int mtk_jpeg_release(struct file *file)
 
Quoted 13 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_clk_init(struct mtk_jpeg_dev *jpeg) · line 1525

Quoted 11 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -0
 	mutex_lock(&jpeg->lock);
 	v4l2_m2m_ctx_release(ctx->fh.m2m_ctx);
+	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
 	v4l2_fh_del(&ctx->fh);
 	v4l2_fh_exit(&ctx->fh);
 	kfree(ctx);
@@ -1064,6 +1522,7 @@ static int mtk_jpeg_clk_init(struct mtk_jpeg_dev *jpeg)
 	node = of_parse_phandle(jpeg->dev->of_node, "mediatek,larb", 0);
 	if (!node)
 		return -EINVAL;
+

Unnecessary change?

yes

Quoted 63 lines
Quoted 63 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +19 -15
 	pdev = of_find_device_by_node(node);
 	if (WARN_ON(!pdev)) {
 		of_node_put(node);
@@ -1073,19 +1532,24 @@ static int mtk_jpeg_clk_init(struct mtk_jpeg_dev *jpeg)
 
 	jpeg->larb = &pdev->dev;
 
-	jpeg->clk_jdec = devm_clk_get(jpeg->dev, "jpgdec");
-	if (IS_ERR(jpeg->clk_jdec))
-		return PTR_ERR(jpeg->clk_jdec);
+	if (jpeg->mode == MTK_JPEG_ENC) {
+		jpeg->clk_jpeg = devm_clk_get(jpeg->dev, "jpgenc");
+		return PTR_ERR_OR_ZERO(jpeg->clk_jpeg);
+	}
+
+	jpeg->clk_jpeg = devm_clk_get(jpeg->dev, "jpgdec");
+	if (IS_ERR(jpeg->clk_jpeg))
+		return PTR_ERR(jpeg->clk_jpeg);
 
-	jpeg->clk_jdec_smi = devm_clk_get(jpeg->dev, "jpgdec-smi");
-	return PTR_ERR_OR_ZERO(jpeg->clk_jdec_smi);
+	jpeg->clk_jpeg_smi = devm_clk_get(jpeg->dev, "jpgdec-smi");
+	return PTR_ERR_OR_ZERO(jpeg->clk_jpeg_smi);
 }
 
 static int mtk_jpeg_probe(struct platform_device *pdev)
 {
 	struct mtk_jpeg_dev *jpeg;
 	struct resource *res;
-	int dec_irq;
+	int jpeg_irq;
 	int ret;
 
 	jpeg = devm_kzalloc(&pdev->dev, sizeof(*jpeg), GFP_KERNEL);
@@ -1095,28 +1559,26 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 	mutex_init(&jpeg->lock);
 	spin_lock_init(&jpeg->hw_lock);
 	jpeg->dev = &pdev->dev;
+	jpeg->mode = (enum mtk_jpeg_mode)of_device_get_match_data(jpeg->dev);
 
 	res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
-	jpeg->dec_reg_base = devm_ioremap_resource(&pdev->dev, res);
-	if (IS_ERR(jpeg->dec_reg_base)) {
-		ret = PTR_ERR(jpeg->dec_reg_base);
+	jpeg->reg_base = devm_ioremap_resource(&pdev->dev, res);
+	if (IS_ERR(jpeg->reg_base)) {
+		ret = PTR_ERR(jpeg->reg_base);
 		return ret;
 	}
 
-	res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
-	dec_irq = platform_get_irq(pdev, 0);
-	if (!res || dec_irq < 0) {
-		dev_err(&pdev->dev, "Failed to get dec_irq %d.\n", dec_irq);
-		ret = -EINVAL;
-		return ret;
+	jpeg_irq = platform_get_irq(pdev, 0);
+	if (jpeg_irq < 0) {
+		dev_err(&pdev->dev, "Failed to get jpeg_irq %d.\n", jpeg_irq);
+		return jpeg_irq;
 	}
 
Quoted 10 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_probe(struct platform_device *pdev) · line 1119

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -4
-	ret = devm_request_irq(&pdev->dev, dec_irq, mtk_jpeg_dec_irq, 0,
+	ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_irq, 0,
 			       pdev->name, jpeg);
 	if (ret) {
-		dev_err(&pdev->dev, "Failed to request dec_irq %d (%d)\n",
-			dec_irq, ret);
-		ret = -EINVAL;

This removal of ret assignment looks like a separate fix that should be done in its own patch.

this change is because of the adding of jpeg enc driver,not the orignal driver' question, should I move it to the orignal driver's patch?

Quoted 41 lines
Quoted 41 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +16 -13
+		dev_err(&pdev->dev, "Failed to request jpeg_irq %d (%d)\n",
+			jpeg_irq, ret);
 		goto err_req_irq;
 	}
 
@@ -1140,33 +1602,35 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 		goto err_m2m_init;
 	}
 
-	jpeg->dec_vdev = video_device_alloc();
-	if (!jpeg->dec_vdev) {
+	jpeg->vfd_jpeg = video_device_alloc();
+	if (!jpeg->vfd_jpeg) {
 		ret = -ENOMEM;
-		goto err_dec_vdev_alloc;
+		goto err_vfd_jpeg_alloc;
 	}
-	snprintf(jpeg->dec_vdev->name, sizeof(jpeg->dec_vdev->name),
-		 "%s-dec", MTK_JPEG_NAME);
-	jpeg->dec_vdev->fops = &mtk_jpeg_fops;
-	jpeg->dec_vdev->ioctl_ops = &mtk_jpeg_ioctl_ops;
-	jpeg->dec_vdev->minor = -1;
-	jpeg->dec_vdev->release = video_device_release;
-	jpeg->dec_vdev->lock = &jpeg->lock;
-	jpeg->dec_vdev->v4l2_dev = &jpeg->v4l2_dev;
-	jpeg->dec_vdev->vfl_dir = VFL_DIR_M2M;
-	jpeg->dec_vdev->device_caps = V4L2_CAP_STREAMING |
+	snprintf(jpeg->vfd_jpeg->name, sizeof(jpeg->vfd_jpeg->name),
+		 "%s-%s", MTK_JPEG_NAME,
+		 jpeg->mode == MTK_JPEG_ENC ? "enc" : "dec");
+	jpeg->vfd_jpeg->fops = &mtk_jpeg_fops;
+	jpeg->vfd_jpeg->ioctl_ops = &mtk_jpeg_ioctl_ops;
+	jpeg->vfd_jpeg->minor = -1;
+	jpeg->vfd_jpeg->release = video_device_release;
+	jpeg->vfd_jpeg->lock = &jpeg->lock;
+	jpeg->vfd_jpeg->v4l2_dev = &jpeg->v4l2_dev;
+	jpeg->vfd_jpeg->vfl_dir = VFL_DIR_M2M;
+	jpeg->vfd_jpeg->device_caps = V4L2_CAP_STREAMING |
 				      V4L2_CAP_VIDEO_M2M_MPLANE;
 
Quoted 4 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_probe(struct platform_device *pdev) · line 1623

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -1
-	ret = video_register_device(jpeg->dec_vdev, VFL_TYPE_GRABBER, 3);
+	ret = video_register_device(jpeg->vfd_jpeg, VFL_TYPE_GRABBER, -1);

The change from 3 to -1 also looks like something for a separate patch.

same as the above reply

Quoted 108 lines
Quoted 108 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +25 -13
 	if (ret) {
 		v4l2_err(&jpeg->v4l2_dev, "Failed to register video device\n");
-		goto err_dec_vdev_register;
+		goto err_vfd_jpeg_register;
 	}
 
-	video_set_drvdata(jpeg->dec_vdev, jpeg);
+	video_set_drvdata(jpeg->vfd_jpeg, jpeg);
 	v4l2_info(&jpeg->v4l2_dev,
-		  "decoder device registered as /dev/video%d (%d,%d)\n",
-		  jpeg->dec_vdev->num, VIDEO_MAJOR, jpeg->dec_vdev->minor);
+		  "jpeg device %d registered as /dev/video%d (%d,%d)\n",
+		  jpeg->mode, jpeg->vfd_jpeg->num, VIDEO_MAJOR,
+		  jpeg->vfd_jpeg->minor);
 
 	platform_set_drvdata(pdev, jpeg);
 
@@ -1174,10 +1638,10 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 
 	return 0;
 
-err_dec_vdev_register:
-	video_device_release(jpeg->dec_vdev);
+err_vfd_jpeg_register:
+	video_device_release(jpeg->vfd_jpeg);
 
-err_dec_vdev_alloc:
+err_vfd_jpeg_alloc:
 	v4l2_m2m_release(jpeg->m2m_dev);
 
 err_m2m_init:
@@ -1197,8 +1661,8 @@ static int mtk_jpeg_remove(struct platform_device *pdev)
 	struct mtk_jpeg_dev *jpeg = platform_get_drvdata(pdev);
 
 	pm_runtime_disable(&pdev->dev);
-	video_unregister_device(jpeg->dec_vdev);
-	video_device_release(jpeg->dec_vdev);
+	video_unregister_device(jpeg->vfd_jpeg);
+	video_device_release(jpeg->vfd_jpeg);
 	v4l2_m2m_release(jpeg->m2m_dev);
 	v4l2_device_unregister(&jpeg->v4l2_dev);
 
@@ -1209,7 +1673,11 @@ static __maybe_unused int mtk_jpeg_pm_suspend(struct device *dev)
 {
 	struct mtk_jpeg_dev *jpeg = dev_get_drvdata(dev);
 
-	mtk_jpeg_dec_reset(jpeg->dec_reg_base);
+	if (jpeg->mode == MTK_JPEG_ENC)
+		mtk_jpeg_enc_reset(jpeg->reg_base);
+	else
+		mtk_jpeg_dec_reset(jpeg->reg_base);
+
 	mtk_jpeg_clk_off(jpeg);
 
 	return 0;
@@ -1220,7 +1688,10 @@ static __maybe_unused int mtk_jpeg_pm_resume(struct device *dev)
 	struct mtk_jpeg_dev *jpeg = dev_get_drvdata(dev);
 
 	mtk_jpeg_clk_on(jpeg);
-	mtk_jpeg_dec_reset(jpeg->dec_reg_base);
+	if (jpeg->mode == MTK_JPEG_ENC)
+		mtk_jpeg_enc_reset(jpeg->reg_base);
+	else
+		mtk_jpeg_dec_reset(jpeg->reg_base);
 
 	return 0;
 }
@@ -1256,11 +1727,15 @@ static const struct dev_pm_ops mtk_jpeg_pm_ops = {
 static const struct of_device_id mtk_jpeg_match[] = {
 	{
 		.compatible = "mediatek,mt8173-jpgdec",
-		.data       = NULL,
+		.data       = (void *)MTK_JPEG_DEC,
 	},
 	{
 		.compatible = "mediatek,mt2701-jpgdec",
-		.data       = NULL,
+		.data       = (void *)MTK_JPEG_DEC,
+	},
+	{
+		.compatible = "mediatek,mtk-jpgenc",
+		.data       = (void *)MTK_JPEG_ENC,
 	},
 	{},
 };
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +3 -0
index 999bd1427809..35ddc64af554 100644
@@ -3,6 +3,7 @@
  * Copyright (c) 2016 MediaTek Inc.
  * Author: Ming Hsiu Tsai <[email protected]>
  *         Rick Chang <[email protected]>
+ *         Xia Jiang <[email protected]>
  */
 
 #ifndef _MTK_JPEG_CORE_H
@@ -17,23 +18,77 @@
 
 #define MTK_JPEG_FMT_FLAG_DEC_OUTPUT	BIT(0)
 #define MTK_JPEG_FMT_FLAG_DEC_CAPTURE	BIT(1)
+#define MTK_JPEG_FMT_FLAG_ENC_OUTPUT	BIT(2)
+#define MTK_JPEG_FMT_FLAG_ENC_CAPTURE	BIT(3)
 
 #define MTK_JPEG_FMT_TYPE_OUTPUT	1
 #define MTK_JPEG_FMT_TYPE_CAPTURE	2
 
Quoted 10 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 30

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +4 -4
-#define MTK_JPEG_MIN_WIDTH	32
-#define MTK_JPEG_MIN_HEIGHT	32
-#define MTK_JPEG_MAX_WIDTH	8192
-#define MTK_JPEG_MAX_HEIGHT	8192
+#define MTK_JPEG_MIN_WIDTH	32U
+#define MTK_JPEG_MIN_HEIGHT	32U
+#define MTK_JPEG_MAX_WIDTH	65535U
+#define MTK_JPEG_MAX_HEIGHT	65535U

Why is it okay to change this from 8192 to 65535?

our hw support max width/height to 65535

Quoted 9 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 36

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +3 -0
 #define MTK_JPEG_DEFAULT_SIZEIMAGE	(1 * 1024 * 1024)
+#define MTK_JPEG_DEFAULT_EXIF_SIZE	(64 * 1024)
 
+#define MTK_JPEG_ENCODE		0
+#define MTK_JPEG_DECODE		1

These two are unused.

deleted it

Quoted 21 lines
Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +4 -0
+
+/**
+ * enum mtk_jpeg_ctx_state - contex state of jpeg
+ */
 enum mtk_jpeg_ctx_state {
 	MTK_JPEG_INIT = 0,
 	MTK_JPEG_RUNNING,

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 53

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +7 -0
 	MTK_JPEG_SOURCE_CHANGE,
 };
 
+/**
+ * enum mtk_jpeg_mode - mode of jpeg
+ */
+enum mtk_jpeg_mode {
+	MTK_JPEG_ENC,
+	MTK_JPEG_DEC,
+};

Perhaps we could just have a bool is_encoder?

Quoted 12 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 63

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +10 -0
+
+/**
+ * enum jpeg_enc_yuv_fmt - yuv format of jpeg enc
+ */
+enum jpeg_enc_yuv_fmt {
+	JPEG_YUV_FORMAT_YUYV = 0,
+	JPEG_YUV_FORMAT_YVYU = 1,
+	JPEG_YUV_FORMAT_NV12 = 2,
+	JEPG_YUV_FORMAT_NV21 = 3,
+};

#define macros are typically used for hardware values.

done

Quoted 11 lines
Quoted 11 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +10 -0
+
+/**
+ * enum JPEG_ENCODE_QUALITY_ENUM - number of jpeg encoder quality
+ */
+enum JPEG_ENCODE_QUALITY_ENUM {
+	JPEG_ENCODE_QUALITY_Q60 = 0x0,
+	JPEG_ENCODE_QUALITY_Q80 = 0x1,
+	JPEG_ENCODE_QUALITY_Q90 = 0x2,
+	JPEG_ENCODE_QUALITY_Q95 = 0x3,
+
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 79

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +6 -0
+	JPEG_ENCODE_QUALITY_Q39 = 0x4,
+	JPEG_ENCODE_QUALITY_Q68 = 0x5,
+	JPEG_ENCODE_QUALITY_Q84 = 0x6,
+	JPEG_ENCODE_QUALITY_Q92 = 0x7,
+
+	JPEG_ENCODE_QUALITY_Q48 = 0x8,

No enum for 0x9?

yes, these values are defined according by hw

Quoted 7 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 83

Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +4 -0
+	JPEG_ENCODE_QUALITY_Q74 = 0xA,
+	JPEG_ENCODE_QUALITY_Q87 = 0xB,
+
+	JPEG_ENCODE_QUALITY_Q34 = 0xC,

No enum for 0xd?

yes

Quoted 9 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 89

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +6 -0
+	JPEG_ENCODE_QUALITY_Q64 = 0xE,
+	JPEG_ENCODE_QUALITY_Q82 = 0xF,
+
+	JPEG_ENCODE_QUALITY_Q97 = 0x10,
+
+	JPEG_ENCODE_QUALITY_ALL = 0xFFFFFFFF

What is the meaning of this value?

means the quantization table setting for frame

Quoted 4 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 89

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +1 -0
+};

Since these are hardware register values, #define macros are preferred.

done

Quoted 43 lines
Quoted 43 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +11 -8
+
 /**
  * struct mt_jpeg - JPEG IP abstraction
  * @lock:		the mutex protecting this structure
@@ -43,11 +98,12 @@ enum mtk_jpeg_ctx_state {
  * @v4l2_dev:		v4l2 device for mem2mem mode
  * @m2m_dev:		v4l2 mem2mem device data
  * @alloc_ctx:		videobuf2 memory allocator's context
- * @dec_vdev:		video device node for decoder mem2mem mode
- * @dec_reg_base:	JPEG registers mapping
- * @clk_jdec:		JPEG hw working clock
- * @clk_jdec_smi:	JPEG SMI bus clock
+ * @vfd_jpeg:		video device node for jpeg mem2mem mode
+ * @reg_base:		JPEG registers mapping
+ * @clk_jpeg:		JPEG hw working clock
+ * @clk_jpeg_smi:	JPEG SMI bus clock
  * @larb:		SMI device
+ * @mode:		compression (encode) operation or decompression (decode)
  */
 struct mtk_jpeg_dev {
 	struct mutex		lock;
@@ -57,11 +113,12 @@ struct mtk_jpeg_dev {
 	struct v4l2_device	v4l2_dev;
 	struct v4l2_m2m_dev	*m2m_dev;
 	void			*alloc_ctx;
-	struct video_device	*dec_vdev;
-	void __iomem		*dec_reg_base;
-	struct clk		*clk_jdec;
-	struct clk		*clk_jdec_smi;
+	struct video_device	*vfd_jpeg;
+	void __iomem		*reg_base;
+	struct clk		*clk_jpeg;
+	struct clk		*clk_jpeg_smi;
 	struct device		*larb;
+	enum mtk_jpeg_mode	mode;
 };
 
 /**
@@ -101,15 +158,51 @@ struct mtk_jpeg_q_data {
 	u32			sizeimage[VIDEO_MAX_PLANES];
 };
 
Quoted 10 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h struct mtk_jpeg_q_data { · line 168

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +8 -0
+/**
+ * jpeg_enc_param - parameters of jpeg encode control
+ * @enable_exif:	EXIF enable for jpeg encode mode
+ * @enc_quality:	destination image quality in encode mode
+ * @restart_interval:	JPEG restart interval for JPEG encoding
+ */
+struct jpeg_enc_param {
+	u32 enable_exif;

Shouldn't this be a bool?

this value is seted by V4L2_CID_JPEG_ACTIVE_MARKER control,its' value is V4L2_JPEG_ACTIVE_MARKER_APP1(1<<1),not a bool

Quoted 4 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h struct mtk_jpeg_q_data { · line 169

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +1 -0
+	u32 enc_quality;

This is from 0 to 100, so perhaps unsigned char?

yes,has modified it to u8

Quoted 4 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h struct mtk_jpeg_q_data { · line 170

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +1 -0
+	u32 restart_interval;

Ditto.

done

Quoted 23 lines
Quoted 23 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +22 -0
+};
+
+/**
+ * mtk_jpeg_enc_param:  General jpeg encoding parameters
+ * @enc_w:		image width
+ * @enc_h:		image height
+ * @enable_exif:	EXIF enable for jpeg encode mode
+ * @enc_quality:	destination image quality in encode mode
+ * @enc_format:		input image format
+ * @restart_interval:	JPEG restart interval for JPEG encoding
+ * @img_stride:		jpeg encoder image stride
+ * @mem_stride:		jpeg encoder memory stride
+ * @total_encdu:	total 8x8 block number
+ */
+struct mtk_jpeg_enc_param {
+	u32 enc_w;
+	u32 enc_h;
+	u32 enable_exif;
+	u32 enc_quality;
+	u32 enc_format;
+	u32 restart_interval;
+	u32 img_stride;
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h struct mtk_jpeg_q_data { · line 195

Quoted 3 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +3 -0
+	u32 mem_stride;
+	u32 total_encdu;
+};

All of the parameters here are already represented by corresponding generic V4L2 state. We should be able to just convert them to the hardware representation at the time of programing the registers, without a need to store them in this intermediate struct.

deleted this structure

Quoted 12 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h struct mtk_jpeg_q_data { · line 112

Quoted 11 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +1 -2
+
 /**
  * mtk_jpeg_ctx - the device context data
  * @jpeg:		JPEG IP device for this context
  * @out_q:		source (output) queue information
  * @cap_q:		destination (capture) queue queue information
  * @fh:			V4L2 file handle
- * @dec_param		parameters for HW decoding
  * @state:		state of the context
- * @header_valid:	set if header has been parsed and valid
Quoted 1 line

Why removing these two?

the orignal structure didn't have these two member variables

Quoted 101 lines
Quoted 71 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +4 -0
+ * @jpeg_param:		jpeg encode parameters
+ * @ctrl_hdl:		controls handler
  * @colorspace: enum v4l2_colorspace; supplemental to pixelformat
  * @ycbcr_enc: enum v4l2_ycbcr_encoding, Y'CbCr encoding
  * @quantization: enum v4l2_quantization, colorspace quantization
@@ -121,6 +214,8 @@ struct mtk_jpeg_ctx {
 	struct mtk_jpeg_q_data		cap_q;
 	struct v4l2_fh			fh;
 	enum mtk_jpeg_ctx_state		state;
+	struct jpeg_enc_param		jpeg_param;
+	struct v4l2_ctrl_handler	ctrl_hdl;
 
 	enum v4l2_colorspace colorspace;
 	enum v4l2_ycbcr_encoding ycbcr_enc;
drivers/media/platform/mtk-jpeg/mtk_jpeg_dec_hw.h +4 -3
index a37be1a48415..39d743fb572f 100644
@@ -3,10 +3,11 @@
  * Copyright (c) 2016 MediaTek Inc.
  * Author: Ming Hsiu Tsai <[email protected]>
  *         Rick Chang <[email protected]>
+ *         Xia Jiang <[email protected]>
  */
 
-#ifndef _MTK_JPEG_HW_H
-#define _MTK_JPEG_HW_H
+#ifndef _MTK_JPEG_DEC_HW_H
+#define _MTK_JPEG_DEC_HW_H
 
 #include <media/videobuf2-core.h>
 
@@ -80,4 +81,4 @@ void mtk_jpeg_dec_set_config(void __iomem *base,
 void mtk_jpeg_dec_reset(void __iomem *dec_reg_base);
 void mtk_jpeg_dec_start(void __iomem *dec_reg_base);
 
-#endif /* _MTK_JPEG_HW_H */
+#endif /* _MTK_JPEG_DEC_HW_H */
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +26 -0
new file mode 100644
index 000000000000..ac27774ccfed
@@ -0,0 +1,175 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ * Author: Xia Jiang <[email protected]>
+ *
+ */
+
+#include <linux/io.h>
+#include <linux/kernel.h>
+#include <media/videobuf2-core.h>
+
+#include "mtk_jpeg_enc_hw.h"
+
+void mtk_jpeg_enc_reset(void __iomem *base)
+{
+	writel(0x00, base + JPGENC_RSTB);
+	writel(JPEG_ENC_RESET_BIT, base + JPGENC_RSTB);
+	writel(0x00, base + JPGENC_CODEC_SEL);
+}
+
+u32 mtk_jpeg_enc_get_int_status(void __iomem *base)
+{
+	u32 ret;
+
+	ret = readl(base + JPGENC_INT_STS) &
+		    JPEG_DRV_ENC_INT_STATUS_MASK_ALLIRQ;

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 36

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +10 -0
+	if (ret)
+		writel(0, base + JPGENC_INT_STS);
+
+	return ret;
+}
+
+u32 mtk_jpeg_enc_get_file_size(void __iomem *base)
+{
+	return readl(base + JPGENC_DMA_ADDR0) - readl(base + JPGENC_DST_ADDR0);
+}

This is just used in the function below, so the code could just be moved there.

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 47

Quoted 11 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +11 -0
+
+u32 mtk_jpeg_enc_enum_result(void __iomem *base, u32 irq_status, u32 *file_size)
+{
+	*file_size = mtk_jpeg_enc_get_file_size(base);
+	if (irq_status & JPEG_DRV_ENC_INT_STATUS_DONE)
+		return MTK_JPEG_ENC_RESULT_DONE;
+	else if (irq_status & JPEG_DRV_ENC_INT_STATUS_STALL)
+		return MTK_JPEG_ENC_RESULT_STALL;
+	else if (irq_status & JPEG_DRV_ENC_INT_STATUS_VCODEC_IRQ)
+		return MTK_JPEG_ENC_RESULT_VCODEC_IRQ;
+	return MTK_JPEG_ENC_RESULT_ERROR_UNKNOWN;

This function seems to do two things at a time, which is confusing, given that the name only refers to the result and doesn't mention the "file size".

Quoted 2 lines

So an alternative option to my suggestion above would be removing the file size handling from this function and using the dedicated function for it.

done

Quoted 30 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 53

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +6 -0
+}
+
+static void mtk_jpeg_enc_set_blk_num(void __iomem *base, u32 blk_num)
+{
+	writel(blk_num, base + JPGENC_BLK_NUM);
+}

Is there any benefit of having a lot of such small functions rather than just doing everything needed in one function? This hardware doesn't have so many parameters to configure and the caller side is generic V4L2 and vb2, so there is no need for additional layer of abstraction.

How about the following model, as used by many other drivers:

mtk_jpeg_enc_set_src()

{
	// Set any registers related to source format and buffer
}

mtk_jpeg_enc_set_dst()

{
	// Set any registers related to destination format and buffer
}

mtk_jpeg_enc_set_params()

{
	// Set any registers related to additional encoding parameters
}
Quoted 7 lines

mtk_jpeg_enc_device_run(enc, ctx)

{
	mtk_jpeg_enc_set_src(enc, src_buf, src_fmt);
	mtk_jpeg_enc_set_dst(enc, dst_buf, dst_fmt);
	mtk_jpeg_enc_set_params(enc, ctx);
	// Trigger the hardware run
}

done

Quoted 15 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 62

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +9 -0
+
+static void mtk_jpeg_enc_set_encFormat(void __iomem *base, u32 enc_format)
+{
+	u32 value;
+
+	value = readl(base + JPGENC_CTRL);
+	value &= ~JPEG_ENC_CTRL_YUV_BIT;
+	value |= JPGENC_FORMAT(enc_format);
+	writel(value, base + JPGENC_CTRL);

The model I suggested above also avoids this kind of read modify write operations, which just unnecessarily add synchronous MMIO round trips, which means more CPU overhead.

Quoted 2 lines

Given that the full state is always known by the driver, it can just write all the register values without the need to read them back.

the JPGENC_CTRL register has 32 bits,different bits have different meanings. encformat is bit 3~4,so we need ready before write to guarantee other bits' value not be changed

Quoted 42 lines
Quoted 42 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +41 -0
+}
+
+static void mtk_jpeg_enc_set_img_size(void __iomem *base, u32 width, u32 height)
+{
+	u32 value;
+
+	value = JPGENC_WIDTH_HEIGHT(width, height);
+	writel(value, base + JPGENC_IMG_SIZE);
+}
+
+static void mtk_jpeg_enc_set_src_img(void __iomem *base, u32 width,
+				     u32 height, u32 yuv_format,
+				     u32 total_encdu)
+{
+	mtk_jpeg_enc_set_img_size(base, width, height);
+	mtk_jpeg_enc_set_encFormat(base, yuv_format);
+	mtk_jpeg_enc_set_blk_num(base, total_encdu);
+}
+
+static void mtk_jpeg_enc_set_src_buf(void __iomem *base, u32 img_stride,
+				     u32 mem_stride, u32 src_addr,
+				     u32 src_addr_c)
+{
+	writel(img_stride, base + JPGENC_IMG_STRIDE);
+	writel(mem_stride, base + JPGENC_STRIDE);
+	writel(src_addr, base + JPGENC_SRC_LUMA_ADDR);
+	writel(src_addr_c, base + JPGENC_SRC_CHROMA_ADDR);
+}
+
+static void mtk_jpeg_enc_set_dst_buf(void __iomem *base, u32 dst_addr,
+				     u32 stall_size, u32 init_offset,
+				     u32 offset_mask)
+{
+	writel(JPGENC_INIT_OFFSET(init_offset), base + JPGENC_OFFSET_ADDR);
+	writel(JPGENC_OFFSET_MASK(offset_mask), base + JPGENC_BYTE_OFFSET_MASK);
+	writel(JPGENC_DST_ADDR(dst_addr), base + JPGENC_DST_ADDR0);
+	writel(JPGENC_STALL_ADDR(dst_addr, stall_size),
+	       base + JPGENC_STALL_ADDR0);
+}
+
+static void mtk_jpeg_enc_set_quality(void __iomem *base, u32 quality)
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 108

Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +5 -0
+{
+	u32 value;
+
+	value = readl(base + JPGENC_QUALITY);
+	value = JPGENC_SET_QUALITY(value, quality);

This macro is confusing. Could we just clear the quality bits explicitly and then bitwise-OR the new value?

yes.guarantee other bits not be changed,quality bits is new value

Quoted 87 lines
Quoted 87 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +67 -0
+	writel(value, base + JPGENC_QUALITY);
+}
+
+static void mtk_jpeg_enc_set_restart_interval(void __iomem *base,
+					      u32 restart_interval)
+{
+	u32 value;
+
+	value = readl(base + JPGENC_CTRL);
+	if (restart_interval)
+		value |= JPEG_ENC_CTRL_RESTART_EN_BIT;
+	else
+		value &= ~JPEG_ENC_CTRL_RESTART_EN_BIT;
+	writel(value, base + JPGENC_CTRL);
+	writel(restart_interval, base + JPGENC_RST_MCU_NUM);
+}
+
+static void mtk_jpeg_enc_set_encode_mode(void __iomem *base, u32 exif_en)
+{
+	u32 value;
+
+	value = readl(base + JPGENC_CTRL);
+	value &= ~JPEG_ENC_CTRL_FILE_FORMAT_BIT;
+	writel(value, base + JPGENC_CTRL);
+
+	if (exif_en) {
+		value = readl(base + JPGENC_CTRL);
+		value |= JPEG_ENC_EN_JFIF_EXIF;
+		writel(value, base + JPGENC_CTRL);
+	}
+}
+
+static void mtk_jpeg_enc_set_ctrl_cfg(void __iomem *base, u32 exif_en,
+				      u32 quality, u32 restart_interval)
+{
+	mtk_jpeg_enc_set_quality(base, quality);
+
+	mtk_jpeg_enc_set_restart_interval(base, restart_interval);
+
+	mtk_jpeg_enc_set_encode_mode(base, exif_en);
+}
+
+void mtk_jpeg_enc_start(void __iomem *base)
+{
+	u32 value;
+
+	value = readl(base + JPGENC_CTRL);
+	value |= JPEG_ENC_CTRL_INT_EN_BIT | JPEG_ENC_CTRL_ENABLE_BIT;
+	writel(value, base + JPGENC_CTRL);
+}
+
+void mtk_jpeg_enc_set_config(void __iomem *base,
+			     struct mtk_jpeg_enc_param *config,
+			     struct mtk_jpeg_enc_bs *bs,
+			     struct mtk_jpeg_enc_fb *fb)
+{
+	mtk_jpeg_enc_set_src_img(base, config->enc_w, config->enc_h,
+				 config->enc_format, config->total_encdu);
+	mtk_jpeg_enc_set_src_buf(base, config->img_stride, config->mem_stride,
+				 fb->fb_addr[0].dma_addr,
+				 fb->fb_addr[1].dma_addr);
+	mtk_jpeg_enc_set_dst_buf(base, bs->dma_addr, bs->size,
+				 bs->dma_addr_offset, bs->dma_addr_offsetmask);
+	mtk_jpeg_enc_set_ctrl_cfg(base, config->enable_exif,
+				  config->enc_quality,
+				  config->restart_interval);
+}
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h new file +13 -0
new file mode 100644
index 000000000000..7080240f6dac
@@ -0,0 +1,60 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ * Author: Xia Jiang <[email protected]>
+ *
+ */
+
+#ifndef _MTK_JPEG_ENC_HW_H
+#define _MTK_JPEG_ENC_HW_H
+
+#include <media/videobuf2-core.h>
+
+#include "mtk_jpeg_core.h"
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h line 18

Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h new file +5 -0
+#include "mtk_jpeg_enc_reg.h"
+
+#define JPEG_ENC_DST_ADDR_OFFSET_MASK GENMASK(3, 0)
+
+#define JPEG_ENC_CTRL_YUV_BIT 0x18

0x18 isn't a single bit as the _BIT suffix would suggest. Perhaps JPEG_ENC_CTRL_FORMAT_MASK would be more appropriate?

done

Quoted 17 lines
Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h new file +3 -0
+#define JPEG_ENC_CTRL_RESTART_EN_BIT            BIT(10)
+#define JPEG_ENC_CTRL_FILE_FORMAT_BIT           BIT(5)
+#define JPEG_ENC_EN_JFIF_EXIF                   BIT(5)

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h line 31

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h new file +10 -0
+#define JPEG_ENC_CTRL_INT_EN_BIT                BIT(2)
+#define JPEG_ENC_CTRL_ENABLE_BIT                BIT(0)
+#define JPEG_ENC_RESET_BIT                      BIT(0)
+
+enum {
+	MTK_JPEG_ENC_RESULT_DONE		= 0,
+	MTK_JPEG_ENC_RESULT_STALL,
+	MTK_JPEG_ENC_RESULT_VCODEC_IRQ,
+	MTK_JPEG_ENC_RESULT_ERROR_UNKNOWN
+};

Do we need these intermediate error codes? Could we just use errno values instead?

we can just use errno values,but our interrupt status have three state,done/stall,vcodec irq,maybe listing them here makes more clarity

Quoted 21 lines
Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h new file +7 -0
+
+struct mtk_jpeg_enc_bs {
+	dma_addr_t	dma_addr;
+	size_t		size;
+	u32			dma_addr_offset;
+	u32			dma_addr_offsetmask;
+};

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h line 48

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h new file +10 -0
+
+struct mtk_jpeg_mem {
+	dma_addr_t	dma_addr;
+	size_t		size;
+};
+
+struct mtk_jpeg_enc_fb {
+	struct mtk_jpeg_mem	fb_addr[MTK_JPEG_COMP_MAX];
+	u32			num_planes;
+};

Please add kerneldoc comments to the structs above.

Quoted 2 lines

nit: Separating types from names with a single space is more future-proof, in case someone needs to change the layout of the struct or rename fields.

done

Quoted 22 lines
Quoted 22 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h new file +12 -0
+
+void mtk_jpeg_enc_reset(void __iomem *base);
+u32 mtk_jpeg_enc_get_int_status(void __iomem *base);
+u32 mtk_jpeg_enc_get_file_size(void __iomem *base);
+u32 mtk_jpeg_enc_enum_result(void __iomem *base, u32 irq_status,
+			     u32 *file_size);
+void mtk_jpeg_enc_start(void __iomem *enc_reg_base);
+void mtk_jpeg_enc_set_config(void __iomem *base,
+			     struct mtk_jpeg_enc_param *config,
+			     struct mtk_jpeg_enc_bs *bs,
+			     struct mtk_jpeg_enc_fb *fb);
+#endif /* _MTK_JPEG_ENC_HW_H */
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h new file +3 -0
new file mode 100644
index 000000000000..d785621cb4c6
@@ -0,0 +1,49 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h line 9

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h new file +6 -0
+ * Author: Xia Jiang <[email protected]>
+ *
+ */
+#ifndef _MTK_JPEG_ENC_REG_H
+#define _MTK_JPEG_ENC_REG_H
+#define MTK_JPEG_COMP_MAX		3

This doesn't look like a register-specific definition.

move it to mtk_jpeg_core.h which dec also uses it

Quoted 10 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h line 14

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h new file +5 -0
+
+#define JPEG_DRV_ENC_INT_STATUS_DONE			BIT(0)
+#define JPEG_DRV_ENC_INT_STATUS_STALL			BIT(1)
+#define JPEG_DRV_ENC_INT_STATUS_VCODEC_IRQ		BIT(4)
+#define JPEG_DRV_ENC_INT_STATUS_MASK_ALLIRQ		0x13

Could we define this as an | of the specific bit definitions above rather than hardcoding?

Quoted 5 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h line 16

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h new file +2 -0
+
+#define JPGENC_RSTB				0x100

Could we agree on a consistent prefix for macros? Some use JPEG_ENC, some JPEG_DRV_ENC and some JPGENC. How about using JPEG_ENC everywhere?

done

Quoted 6 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h line 19

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h new file +3 -0
+#define JPGENC_CTRL				0x104
+#define JPGENC_QUALITY				0x108
+#define JPGENC_BLK_NUM				0x10C

nit: Hex numerals in the kernel are lowercase.

done

Quoted 24 lines
Quoted 24 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h new file +23 -0
+#define JPGENC_BLK_CNT				0x110
+#define JPGENC_INT_STS				0x11C
+#define JPGENC_DST_ADDR0			0x120
+#define JPGENC_DMA_ADDR0			0x124
+#define JPGENC_STALL_ADDR0			0x128
+#define JPGENC_OFFSET_ADDR			0x138
+#define JPGENC_RST_MCU_NUM			0x150
+#define JPGENC_IMG_SIZE				0x154
+#define JPGENC_DEBUG_INFO0			0x160
+#define JPGENC_DEBUG_INFO1			0x164
+#define JPGENC_TOTAL_CYCLE			0x168
+#define JPGENC_BYTE_OFFSET_MASK			0x16C
+#define JPGENC_SRC_LUMA_ADDR			0x170
+#define JPGENC_SRC_CHROMA_ADDR			0x174
+#define JPGENC_STRIDE				0x178
+#define JPGENC_IMG_STRIDE			0x17C
+#define JPGENC_DCM_CTRL				0x300
+#define JPGENC_CODEC_SEL			0x314
+#define JPGENC_ULTRA_THRES			0x318
+
+#define JPGENC_FORMAT(x)		(((x) & 3) << 3)
+#define JPGENC_WIDTH_HEIGHT(w, h)	(((w) << 16) | (h))
+#define JPGENC_INIT_OFFSET(x)		((x) & (~0xF))
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h line 47

Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h new file +5 -0
+#define JPGENC_OFFSET_MASK(x)		((x) & 0xF)
+#define JPGENC_DST_ADDR(x)		((x) & (~0xF))
+#define JPGENC_STALL_ADDR(x, y)		(((x) + (y)) & (~0xF))
+#define JPGENC_QUALITY_MASK		0xFFFF0000
+#define JPGENC_SET_QUALITY(x, y)	(((x) & JPGENC_QUALITY_MASK) | (y))

JPGENC_QUALITY_MASK is only used here, so 0xffff0000 could be just used directly.

done

Quoted 6 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h line 49

Quoted 3 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h new file +2 -0
+
+#endif /* _MTK_JPEG_ENC_REG_H */

I can see some bits defined in mtk_jpeg_enc_hw.h as well. Perhaps all these could be moved there too, which would make us have 1 file less?

moved bits defined and register defined to mtk_jpeg_enc_reg.h, but the functions declaration about jpeg enc hw were in mtk_jpeg_enc_hw.h, is that ok?

Quoted 4 lines

Best regards,
Tomasz

_______________________________________________
Linux-mediatek mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-mediatek

#14 lore raw

Re: [PATCH v4 5/5] media: platform: Add jpeg dec/enc feature

Hi Xia,

Quoted 66 lines · Xia Jiang
Quoted 55 lines · Tomasz Figa

Hi Xia,

Quoted 27 lines · Xia Jiang

Add mtk jpeg encode v4l2 driver based on jpeg decode, because that jpeg decode and encode have great similarities with function operation.

---
v4: split mtk_jpeg_try_fmt_mplane() to two functions, one for encoder,
one for decoder.
split mtk_jpeg_set_default_params() to two functions, one for encoder, one for decoder.
add cropping support for encoder in g/s_selection ioctls. change exif mode support by using V4L2_JPEG_ACTIVE_MARKER_APP1. change MTK_JPEG_MAX_WIDTH/MTK_JPEG_MAX_HEIGH from 8192 to 65535 by specification.
move width shifting operation behind aligning operation in mtk_jpeg_try_enc_fmt_mplane() for bug fix.
fix user abuseing data_offset issue for DMABUF in mtk_jpeg_set_enc_src().
fix kbuild warings: change MTK_JPEG_MIN_HEIGHT/MTK_JPEG_MAX_HEIGHT
and MTK_JPEG_MIN_WIDTH/MTK_JPEG_MAX_WIDTH from 'int' type to 'unsigned int' type. fix msleadingly indented of 'else'.

v3: delete Change-Id.
only test once handler->error after the last v4l2_ctrl_new_std(). seperate changes of v4l2-ctrls.c and v4l2-controls.h to new patch.

v2: fix compliance test fail, check created buffer size in driver.

Quoted 13 lines
---
 drivers/media/platform/mtk-jpeg/Makefile      |   5 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 731 +++++++++++++++---
 .../media/platform/mtk-jpeg/mtk_jpeg_core.h   | 123 ++-
 .../media/platform/mtk-jpeg/mtk_jpeg_dec_hw.h |   7 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c | 175 +++++
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h |  60 ++
 .../platform/mtk-jpeg/mtk_jpeg_enc_reg.h      |  49 ++
 7 files changed, 1004 insertions(+), 146 deletions(-)
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h

First of all, thanks for the patch!

Please check my comments below.

My general feeling about this code is that the encoder hardware block is completely orthogonal from the decoder block and there is very little code reuse from the original decoder driver.

Moreover, a lot of existing code now needs if (decoder) { ... } else {... } segments, which complicates the code.

Quoted 9 lines

Would it perhaps make sense to instead create a separate mtk-jpeg-enc driver?

It would also give us a fresh start in terms of code quality, as the existing mtk-jpeg driver has a lot of quality issues unfortunately. (Some of my comments to this patch actually relate to the issues with the original code, not introduced by this patch, but we need to fix them if changing this driver already.)

Quoted 5 lines

Dear Tomasz,

I haved fixed the driver by following your advice in general.

Please check my reply below.

Sorry, I missed this message originally. Replied below.

[snip]

Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +8 -0
+   }
+   param->enc_w = q_data_src->w;
+   param->enc_h = q_data_src->h;
+
+   if (jpeg_params->enc_quality >= 97)
+           param->enc_quality = JPEG_ENCODE_QUALITY_Q97;
+   else if (jpeg_params->enc_quality >= 95)
+           param->enc_quality = JPEG_ENCODE_QUALITY_Q95;
Quoted 6 lines
Quoted 2 lines

I'm wondering if the application requests 96, it doesn't expect the quality to be _at_least_ 96.

our jpeg enc hw do not support quality 96,only support 15 kinds of quant table listed here, so if the application requests 96,a nearest and highest quality will be given.

Quoted 1 line

Just to clarify my comment, if I remember correctly, the JPEG standard defines the 100 levels, so if the application requests level 96, but the hardware provides only 95 and 97, the quality should be favored and 97 used.

[snip]

Quoted 11 lines
Quoted 4 lines
Quoted 2 lines

+ param->mem_stride = mtk_jpeg_align(width_even, (is_420 ? 16 : 32));

What's the difference between img_stride and mem_stride?

In theory, mem_stride need >= img_stride,but we use the same is ok

Quoted 5 lines
Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -0
+   param->total_encdu =
+           ((padding_width >> 4) * (padding_height >> (is_420 ? 4 : 3)) *
+           (is_420 ? 6 : 4)) - 1;
Quoted 7 lines
Quoted 4 lines

The comment above the struct says this is the total number of 8x8 blocks. Why would it depend on whether the format is YUV 4:2:0? Since we should have already aligned the width and height in try_fmt, this should be as simple as (width / 8) * (height / 8).

becuase the image size is w*h*1.5 for yuv420 format, but w*h*2 for yuv422,so for yuv420: w_16/8*h_16/8*1.5-1(because the hw will start at number 0), yuv422: w_32/8*h_8/8*2-1,this number is equal to my code.

Do you mean that this also includes the Cb and Cr 8x8 blocks separately? If so, could it be rewritten as below to improve the readability?

luma_blocks = width / 8 * height / 8;
if (is_420)

    chroma_blocks = luma_blocks / 4;
else
    chroma_blocks = luma_blocks / 2;

param->last_encdu = luma_blocks + 2 * chroma_blocks - 1;

Also, does it mean that this number is actually the index of the last block, not the total number of blocks?
If so, the field should be probably renamed to last_encdu and the description updated accordingly.

[snip]

Quoted 10 lines
Quoted 7 lines

Could we instead check the buffer address alignment in .buf_prepare and fail if it's not big enough?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx, · line 1150

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -0
+   bs->dma_addr_offset = p->enable_exif ? MTK_JPEG_DEFAULT_EXIF_SIZE : 0;
+   bs->dma_addr_offsetmask = bs->dma_addr & JPEG_ENC_DST_ADDR_OFFSET_MASK;

What is the meaning of this offset mask?

our actual destination address = destination address + offset address+ destination address offset mask.The mask 0:No offset,1~15:offset byte from the 16-byte aligned

So we have dma_addr, dma_addr_offset and dma_addr_offsetmask. Why do we need dma_addr_offset? Would the same operation be achieved with the code below?

dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0); if (p->enable_exif)

    dma_addr += MTK_JPEG_DEFAULT_EXIF_SIZE;

bs->dma_addr = dma_addr & ~JPEG_ENC_DST_ADDR_OFFSET_MASK; bs->dma_addr_offset = 0;
bs->dma_addr_offsetmask = dma_addr & JPEG_ENC_DST_ADDR_OFFSET_MASK;

Or does the hardware write something directly at bs->dma_addr (some tags?) and then the encoded image at the final desintation address?

[snip]

Quoted 49 lines
Quoted 41 lines
Quoted 41 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +40 -1
-static void mtk_jpeg_set_default_params(struct mtk_jpeg_ctx *ctx)
+static void mtk_jpeg_set_enc_default_params(struct mtk_jpeg_ctx *ctx)
+{
+   struct mtk_jpeg_q_data *q = &ctx->out_q;
+   int align_w, align_h;
+
+   ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+
+   ctx->colorspace = V4L2_COLORSPACE_JPEG,
+   ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+   ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
+   ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+
+   q->w = MTK_JPEG_MIN_WIDTH;
+   q->h = MTK_JPEG_MIN_HEIGHT;
+
+   q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_YUYV,
+                                 MTK_JPEG_FMT_TYPE_OUTPUT);
+
+   align_w = q->w;
+   align_h = q->h;
+   align_w = round_up(align_w, 2);
+   v4l_bound_align_image(&align_w, MTK_JPEG_MIN_WIDTH, MTK_JPEG_MAX_WIDTH,
+                         5, &align_h, MTK_JPEG_MIN_HEIGHT,
+                         MTK_JPEG_MAX_HEIGHT, 3, 0);
+   align_w = align_w << 1;
+
+   if (align_w < MTK_JPEG_MIN_WIDTH &&
+       (align_w + 32) <= MTK_JPEG_MAX_WIDTH)
+           align_w += 32;
+   if (align_h < MTK_JPEG_MIN_HEIGHT &&
+       (align_h + 8) <= MTK_JPEG_MAX_HEIGHT)
+           align_h += 8;
+
+   q->sizeimage[0] = align_w * align_h;
+   q->bytesperline[0] = align_w;
+
+   q = &ctx->cap_q;
+   q->w = MTK_JPEG_MIN_WIDTH;
+   q->h = MTK_JPEG_MIN_HEIGHT;
+   q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_JPEG,
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1408

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -0
+                                 MTK_JPEG_FMT_TYPE_CAPTURE);
+   q->bytesperline[0] = 0;
+   q->sizeimage[0] = MTK_JPEG_DEFAULT_SIZEIMAGE;
+}

Could we just create an arbitrary v4l2_pix_format_mplane struct and call s_fmt instead? In general, all of the constant values and alignments should be already ensured by try_fmt, so this function should be redundant.

Quoted 1 line

if cancel this function,the v4l2-compliance test will fail

Sorry, I guess my comment was not clear. We need to initialize the default parameters. However, the contents of this function seem to heavily duplicate with the code that should be in try_fmt, so could we just call try_fmt from here instead of repeating the calculations?

[snip]

Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -4
-   ret = devm_request_irq(&pdev->dev, dec_irq, mtk_jpeg_dec_irq, 0,
+   ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_irq, 0,
                           pdev->name, jpeg);
    if (ret) {
-           dev_err(&pdev->dev, "Failed to request dec_irq %d (%d)\n",
-                   dec_irq, ret);
-           ret = -EINVAL;
Quoted 4 lines
Quoted 2 lines

This removal of ret assignment looks like a separate fix that should be done in its own patch.

this change is because of the adding of jpeg enc driver,not the orignal driver' question, should I move it to the orignal driver's patch?

Yes, please.

Quoted 41 lines
Quoted 41 lines
Quoted 41 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +16 -13
+           dev_err(&pdev->dev, "Failed to request jpeg_irq %d (%d)\n",
+                   jpeg_irq, ret);
            goto err_req_irq;
    }
@@ -1140,33 +1602,35 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
            goto err_m2m_init;
    }
 
-   jpeg->dec_vdev = video_device_alloc();
-   if (!jpeg->dec_vdev) {
+   jpeg->vfd_jpeg = video_device_alloc();
+   if (!jpeg->vfd_jpeg) {
            ret = -ENOMEM;
-           goto err_dec_vdev_alloc;
+           goto err_vfd_jpeg_alloc;
    }
-   snprintf(jpeg->dec_vdev->name, sizeof(jpeg->dec_vdev->name),
-            "%s-dec", MTK_JPEG_NAME);
-   jpeg->dec_vdev->fops = &mtk_jpeg_fops;
-   jpeg->dec_vdev->ioctl_ops = &mtk_jpeg_ioctl_ops;
-   jpeg->dec_vdev->minor = -1;
-   jpeg->dec_vdev->release = video_device_release;
-   jpeg->dec_vdev->lock = &jpeg->lock;
-   jpeg->dec_vdev->v4l2_dev = &jpeg->v4l2_dev;
-   jpeg->dec_vdev->vfl_dir = VFL_DIR_M2M;
-   jpeg->dec_vdev->device_caps = V4L2_CAP_STREAMING |
+   snprintf(jpeg->vfd_jpeg->name, sizeof(jpeg->vfd_jpeg->name),
+            "%s-%s", MTK_JPEG_NAME,
+            jpeg->mode == MTK_JPEG_ENC ? "enc" : "dec");
+   jpeg->vfd_jpeg->fops = &mtk_jpeg_fops;
+   jpeg->vfd_jpeg->ioctl_ops = &mtk_jpeg_ioctl_ops;
+   jpeg->vfd_jpeg->minor = -1;
+   jpeg->vfd_jpeg->release = video_device_release;
+   jpeg->vfd_jpeg->lock = &jpeg->lock;
+   jpeg->vfd_jpeg->v4l2_dev = &jpeg->v4l2_dev;
+   jpeg->vfd_jpeg->vfl_dir = VFL_DIR_M2M;
+   jpeg->vfd_jpeg->device_caps = V4L2_CAP_STREAMING |
                                  V4L2_CAP_VIDEO_M2M_MPLANE;
 
Quoted 5 lines
Quoted 4 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_probe(struct platform_device *pdev) · line 1623

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -1
-   ret = video_register_device(jpeg->dec_vdev, VFL_TYPE_GRABBER, 3);
+   ret = video_register_device(jpeg->vfd_jpeg, VFL_TYPE_GRABBER, -1);

The change from 3 to -1 also looks like something for a separate patch.

same as the above reply

Ditto.

[snip]

Quoted 20 lines
Quoted 10 lines
Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +2 -0
@@ -17,23 +18,77 @@
 
 #define MTK_JPEG_FMT_FLAG_DEC_OUTPUT       BIT(0)
 #define MTK_JPEG_FMT_FLAG_DEC_CAPTURE      BIT(1)
+#define MTK_JPEG_FMT_FLAG_ENC_OUTPUT       BIT(2)
+#define MTK_JPEG_FMT_FLAG_ENC_CAPTURE      BIT(3)
 
 #define MTK_JPEG_FMT_TYPE_OUTPUT   1
 #define MTK_JPEG_FMT_TYPE_CAPTURE  2
 
Quoted 10 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 30

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +4 -4
-#define MTK_JPEG_MIN_WIDTH 32
-#define MTK_JPEG_MIN_HEIGHT        32
-#define MTK_JPEG_MAX_WIDTH 8192
-#define MTK_JPEG_MAX_HEIGHT        8192
+#define MTK_JPEG_MIN_WIDTH 32U
+#define MTK_JPEG_MIN_HEIGHT        32U
+#define MTK_JPEG_MAX_WIDTH 65535U
+#define MTK_JPEG_MAX_HEIGHT        65535U

Why is it okay to change this from 8192 to 65535?

Quoted 1 line

our hw support max width/height to 65535

Does this also apply to the JPEG decoder on MT8173 for which the driver was developed?

[snip]

Quoted 10 lines
Quoted 10 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h struct mtk_jpeg_q_data { · line 168

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +8 -0
+/**
+ * jpeg_enc_param - parameters of jpeg encode control
+ * @enable_exif:   EXIF enable for jpeg encode mode
+ * @enc_quality:   destination image quality in encode mode
+ * @restart_interval:      JPEG restart interval for JPEG encoding
+ */
+struct jpeg_enc_param {
+   u32 enable_exif;

Shouldn't this be a bool?

Quoted 2 lines

this value is seted by V4L2_CID_JPEG_ACTIVE_MARKER control,its' value is V4L2_JPEG_ACTIVE_MARKER_APP1(1<<1),not a bool

In this case, please call it active_marker and also update the comment above accordingly.

Still, if this driver only cares about V4L2_JPEG_ACTIVE_MARKER_APP1, bool enable_exif, assigned by the code appropriately, would make more sense.

[snip]

Quoted 14 lines
Quoted 14 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 62

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +9 -0
+
+static void mtk_jpeg_enc_set_encFormat(void __iomem *base, u32 enc_format)
+{
+   u32 value;
+
+   value = readl(base + JPGENC_CTRL);
+   value &= ~JPEG_ENC_CTRL_YUV_BIT;
+   value |= JPGENC_FORMAT(enc_format);
+   writel(value, base + JPGENC_CTRL);

The model I suggested above also avoids this kind of read modify write operations, which just unnecessarily add synchronous MMIO round trips, which means more CPU overhead.

Quoted 5 lines
Quoted 2 lines

Given that the full state is always known by the driver, it can just write all the register values without the need to read them back.

the JPGENC_CTRL register has 32 bits,different bits have different meanings. encformat is bit 3~4,so we need ready before write to guarantee other bits' value not be changed

I explained this more in my review comments to the latest revision, but this is just solving a problem that is introduced by the design of the code. If the code was designed so that 1 register is only changed in 1 function, there would be no need to read back the registers, because the function would initialize the full register at a time. That's what most of the other drivers do.

[snip]

Quoted 11 lines
Quoted 9 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h line 31

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h new file +6 -0
+enum {
+   MTK_JPEG_ENC_RESULT_DONE                = 0,
+   MTK_JPEG_ENC_RESULT_STALL,
+   MTK_JPEG_ENC_RESULT_VCODEC_IRQ,
+   MTK_JPEG_ENC_RESULT_ERROR_UNKNOWN
+};

Do we need these intermediate error codes? Could we just use errno values instead?

we can just use errno values,but our interrupt status have three state,done/stall,vcodec irq,maybe listing them here makes more clarity

I suggested another approach in my comments for the latest revision. Generally the interrupt handler is the only place where this error handling is done, so the intermediate error codes shouldn't be necessary, as the interrupt handler would directly signal any issues to V4L2.

[snip]

Quoted 13 lines
Quoted 11 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h line 47

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h new file +8 -0
+#define JPGENC_FORMAT(x)           (((x) & 3) << 3)
+#define JPGENC_WIDTH_HEIGHT(w, h)  (((w) << 16) | (h))
+#define JPGENC_INIT_OFFSET(x)              ((x) & (~0xF))
+#define JPGENC_OFFSET_MASK(x)              ((x) & 0xF)
+#define JPGENC_DST_ADDR(x)         ((x) & (~0xF))
+#define JPGENC_STALL_ADDR(x, y)            (((x) + (y)) & (~0xF))
+#define JPGENC_QUALITY_MASK                0xFFFF0000
+#define JPGENC_SET_QUALITY(x, y)   (((x) & JPGENC_QUALITY_MASK) | (y))

JPGENC_QUALITY_MASK is only used here, so 0xffff0000 could be just used directly.

done

Quoted 1 line
Quoted 8 lines
Quoted 5 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h line 49

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h new file +2 -0
+
+#endif /* _MTK_JPEG_ENC_REG_H */

I can see some bits defined in mtk_jpeg_enc_hw.h as well. Perhaps all these could be moved there too, which would make us have 1 file less?

moved bits defined and register defined to mtk_jpeg_enc_reg.h, but the functions declaration about jpeg enc hw were in mtk_jpeg_enc_hw.h, is that ok?

My suggestion was the other way around - have all the encoder-specific definitions in mtk_jpeg_enc_hw.h and remove mtk_jpeg_enc_reg.h.

In general, this is a relatively simple hardware block and we should be able to just have one file that deals with hardware registers. Then there would be no need for headers, as the register definitions could be put directly in the source file. A rule of thumb is to use a header when something needs to be shared between multiple source files.

Best regards,
Tomasz

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

author
#15 lore raw

Re: [PATCH v4 5/5] media: platform: Add jpeg dec/enc feature

Quoted 69 lines · Tomasz Figa

Hi Xia,

Quoted 66 lines · Xia Jiang
Quoted 64 lines · Tomasz Figa

Hi Xia,

Quoted 27 lines · Xia Jiang

Add mtk jpeg encode v4l2 driver based on jpeg decode, because that jpeg decode and encode have great similarities with function operation.

---
v4: split mtk_jpeg_try_fmt_mplane() to two functions, one for encoder,
one for decoder.
split mtk_jpeg_set_default_params() to two functions, one for encoder, one for decoder.
add cropping support for encoder in g/s_selection ioctls. change exif mode support by using V4L2_JPEG_ACTIVE_MARKER_APP1. change MTK_JPEG_MAX_WIDTH/MTK_JPEG_MAX_HEIGH from 8192 to 65535 by specification.
move width shifting operation behind aligning operation in mtk_jpeg_try_enc_fmt_mplane() for bug fix.
fix user abuseing data_offset issue for DMABUF in mtk_jpeg_set_enc_src().
fix kbuild warings: change MTK_JPEG_MIN_HEIGHT/MTK_JPEG_MAX_HEIGHT
and MTK_JPEG_MIN_WIDTH/MTK_JPEG_MAX_WIDTH from 'int' type to 'unsigned int' type. fix msleadingly indented of 'else'.

v3: delete Change-Id.
only test once handler->error after the last v4l2_ctrl_new_std(). seperate changes of v4l2-ctrls.c and v4l2-controls.h to new patch.

v2: fix compliance test fail, check created buffer size in driver.

Quoted 13 lines
---
 drivers/media/platform/mtk-jpeg/Makefile      |   5 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 731 +++++++++++++++---
 .../media/platform/mtk-jpeg/mtk_jpeg_core.h   | 123 ++-
 .../media/platform/mtk-jpeg/mtk_jpeg_dec_hw.h |   7 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c | 175 +++++
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h |  60 ++
 .../platform/mtk-jpeg/mtk_jpeg_enc_reg.h      |  49 ++
 7 files changed, 1004 insertions(+), 146 deletions(-)
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h

First of all, thanks for the patch!

Please check my comments below.

My general feeling about this code is that the encoder hardware block is completely orthogonal from the decoder block and there is very little code reuse from the original decoder driver.

Moreover, a lot of existing code now needs if (decoder) { ... } else {... } segments, which complicates the code.

Would it perhaps make sense to instead create a separate mtk-jpeg-enc driver?

It would also give us a fresh start in terms of code quality, as the existing mtk-jpeg driver has a lot of quality issues unfortunately. (Some of my comments to this patch actually relate to the issues with the original code, not introduced by this patch, but we need to fix them if changing this driver already.)

Quoted 7 lines
Quoted 5 lines

Dear Tomasz,

I haved fixed the driver by following your advice in general.

Please check my reply below.

Sorry, I missed this message originally. Replied below.

Dear Tomasz,

Thank you for your reply.I have changed the driver in the V8 version by following your good advice.

Quoted 18 lines

[snip]

Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +8 -0
+   }
+   param->enc_w = q_data_src->w;
+   param->enc_h = q_data_src->h;
+
+   if (jpeg_params->enc_quality >= 97)
+           param->enc_quality = JPEG_ENCODE_QUALITY_Q97;
+   else if (jpeg_params->enc_quality >= 95)
+           param->enc_quality = JPEG_ENCODE_QUALITY_Q95;
Quoted 6 lines
Quoted 2 lines

I'm wondering if the application requests 96, it doesn't expect the quality to be _at_least_ 96.

our jpeg enc hw do not support quality 96,only support 15 kinds of quant table listed here, so if the application requests 96,a nearest and highest quality will be given.

Quoted 1 line
Quoted 4 lines

Just to clarify my comment, if I remember correctly, the JPEG standard defines the 100 levels, so if the application requests level 96, but the hardware provides only 95 and 97, the quality should be favored and 97 used.

done.

Quoted 26 lines

[snip]

Quoted 11 lines
Quoted 4 lines
Quoted 2 lines

+ param->mem_stride = mtk_jpeg_align(width_even, (is_420 ? 16 : 32));

What's the difference between img_stride and mem_stride?

In theory, mem_stride need >= img_stride,but we use the same is ok

Quoted 5 lines
Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -0
+   param->total_encdu =
+           ((padding_width >> 4) * (padding_height >> (is_420 ? 4 : 3)) *
+           (is_420 ? 6 : 4)) - 1;
Quoted 7 lines
Quoted 4 lines

The comment above the struct says this is the total number of 8x8 blocks. Why would it depend on whether the format is YUV 4:2:0? Since we should have already aligned the width and height in try_fmt, this should be as simple as (width / 8) * (height / 8).

becuase the image size is w*h*1.5 for yuv420 format, but w*h*2 for yuv422,so for yuv420: w_16/8*h_16/8*1.5-1(because the hw will start at number 0), yuv422: w_32/8*h_8/8*2-1,this number is equal to my code.

Do you mean that this also includes the Cb and Cr 8x8 blocks separately? If so, could it be rewritten as below to improve the readability?

luma_blocks = width / 8 * height / 8;
if (is_420)

Quoted 7 lines
    chroma_blocks = luma_blocks / 4;
else
    chroma_blocks = luma_blocks / 2;

param->last_encdu = luma_blocks + 2 * chroma_blocks - 1;

Also, does it mean that this number is actually the index of the last block, not the total number of blocks?

yes.

Quoted 10 lines

If so, the field should be probably renamed to last_encdu and the description updated accordingly.

[snip]

Quoted 6 lines
Quoted 6 lines

Could we instead check the buffer address alignment in .buf_prepare and fail if it's not big enough?

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -0
+   bs->dma_addr_offset = p->enable_exif ? MTK_JPEG_DEFAULT_EXIF_SIZE : 0;
+   bs->dma_addr_offsetmask = bs->dma_addr & JPEG_ENC_DST_ADDR_OFFSET_MASK;
Quoted 8 lines
Quoted 4 lines
Quoted 1 line

What is the meaning of this offset mask?

our actual destination address = destination address + offset address+ destination address offset mask.The mask 0:No offset,1~15:offset byte from the 16-byte aligned

So we have dma_addr, dma_addr_offset and dma_addr_offsetmask. Why do we need dma_addr_offset? Would the same operation be achieved with the code below?

Because in exif mode, the beginning of the destination buffer(offset) should be reserved for the application to fill(I explained this more in my reply to the v7 revision).

Quoted 10 lines

dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0); if (p->enable_exif)

    dma_addr += MTK_JPEG_DEFAULT_EXIF_SIZE;

bs->dma_addr = dma_addr & ~JPEG_ENC_DST_ADDR_OFFSET_MASK; bs->dma_addr_offset = 0;
bs->dma_addr_offsetmask = dma_addr & JPEG_ENC_DST_ADDR_OFFSET_MASK;

Or does the hardware write something directly at bs->dma_addr (some tags?) and then the encoded image at the final desintation address?

Hardware just write encoded image at the final destination address.

Quoted 50 lines

[snip]

Quoted 49 lines
Quoted 49 lines
Quoted 35 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +34 -1
-static void mtk_jpeg_set_default_params(struct mtk_jpeg_ctx *ctx)
+static void mtk_jpeg_set_enc_default_params(struct mtk_jpeg_ctx *ctx)
+{
+   struct mtk_jpeg_q_data *q = &ctx->out_q;
+   int align_w, align_h;
+
+   ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+
+   ctx->colorspace = V4L2_COLORSPACE_JPEG,
+   ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+   ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
+   ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+
+   q->w = MTK_JPEG_MIN_WIDTH;
+   q->h = MTK_JPEG_MIN_HEIGHT;
+
+   q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_YUYV,
+                                 MTK_JPEG_FMT_TYPE_OUTPUT);
+
+   align_w = q->w;
+   align_h = q->h;
+   align_w = round_up(align_w, 2);
+   v4l_bound_align_image(&align_w, MTK_JPEG_MIN_WIDTH, MTK_JPEG_MAX_WIDTH,
+                         5, &align_h, MTK_JPEG_MIN_HEIGHT,
+                         MTK_JPEG_MAX_HEIGHT, 3, 0);
+   align_w = align_w << 1;
+
+   if (align_w < MTK_JPEG_MIN_WIDTH &&
+       (align_w + 32) <= MTK_JPEG_MAX_WIDTH)
+           align_w += 32;
+   if (align_h < MTK_JPEG_MIN_HEIGHT &&
+       (align_h + 8) <= MTK_JPEG_MAX_HEIGHT)
+           align_h += 8;
+
+   q->sizeimage[0] = align_w * align_h;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1408

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+   q->bytesperline[0] = align_w;
+
+   q = &ctx->cap_q;
+   q->w = MTK_JPEG_MIN_WIDTH;
+   q->h = MTK_JPEG_MIN_HEIGHT;
+   q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_JPEG,
+                                 MTK_JPEG_FMT_TYPE_CAPTURE);
+   q->bytesperline[0] = 0;
+   q->sizeimage[0] = MTK_JPEG_DEFAULT_SIZEIMAGE;
+}

Could we just create an arbitrary v4l2_pix_format_mplane struct and call s_fmt instead? In general, all of the constant values and alignments should be already ensured by try_fmt, so this function should be redundant.

Quoted 6 lines
Quoted 1 line

if cancel this function,the v4l2-compliance test will fail

Sorry, I guess my comment was not clear. We need to initialize the default parameters. However, the contents of this function seem to heavily duplicate with the code that should be in try_fmt, so could we just call try_fmt from here instead of repeating the calculations?

done.

Quoted 10 lines

[snip]

Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -4
-   ret = devm_request_irq(&pdev->dev, dec_irq, mtk_jpeg_dec_irq, 0,
+   ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_irq, 0,
                           pdev->name, jpeg);
    if (ret) {
-           dev_err(&pdev->dev, "Failed to request dec_irq %d (%d)\n",
-                   dec_irq, ret);
-           ret = -EINVAL;
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

This removal of ret assignment looks like a separate fix that should be done in its own patch.

this change is because of the adding of jpeg enc driver,not the orignal driver' question, should I move it to the orignal driver's patch?

Yes, please.

done.

Quoted 42 lines
Quoted 42 lines
Quoted 42 lines
Quoted 42 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +16 -13
+           dev_err(&pdev->dev, "Failed to request jpeg_irq %d (%d)\n",
+                   jpeg_irq, ret);
            goto err_req_irq;
    }
@@ -1140,33 +1602,35 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
            goto err_m2m_init;
    }
 
-   jpeg->dec_vdev = video_device_alloc();
-   if (!jpeg->dec_vdev) {
+   jpeg->vfd_jpeg = video_device_alloc();
+   if (!jpeg->vfd_jpeg) {
            ret = -ENOMEM;
-           goto err_dec_vdev_alloc;
+           goto err_vfd_jpeg_alloc;
    }
-   snprintf(jpeg->dec_vdev->name, sizeof(jpeg->dec_vdev->name),
-            "%s-dec", MTK_JPEG_NAME);
-   jpeg->dec_vdev->fops = &mtk_jpeg_fops;
-   jpeg->dec_vdev->ioctl_ops = &mtk_jpeg_ioctl_ops;
-   jpeg->dec_vdev->minor = -1;
-   jpeg->dec_vdev->release = video_device_release;
-   jpeg->dec_vdev->lock = &jpeg->lock;
-   jpeg->dec_vdev->v4l2_dev = &jpeg->v4l2_dev;
-   jpeg->dec_vdev->vfl_dir = VFL_DIR_M2M;
-   jpeg->dec_vdev->device_caps = V4L2_CAP_STREAMING |
+   snprintf(jpeg->vfd_jpeg->name, sizeof(jpeg->vfd_jpeg->name),
+            "%s-%s", MTK_JPEG_NAME,
+            jpeg->mode == MTK_JPEG_ENC ? "enc" : "dec");
+   jpeg->vfd_jpeg->fops = &mtk_jpeg_fops;
+   jpeg->vfd_jpeg->ioctl_ops = &mtk_jpeg_ioctl_ops;
+   jpeg->vfd_jpeg->minor = -1;
+   jpeg->vfd_jpeg->release = video_device_release;
+   jpeg->vfd_jpeg->lock = &jpeg->lock;
+   jpeg->vfd_jpeg->v4l2_dev = &jpeg->v4l2_dev;
+   jpeg->vfd_jpeg->vfl_dir = VFL_DIR_M2M;
+   jpeg->vfd_jpeg->device_caps = V4L2_CAP_STREAMING |
                                  V4L2_CAP_VIDEO_M2M_MPLANE;
 
Quoted 7 lines
Quoted 5 lines
Quoted 4 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_probe(struct platform_device *pdev) · line 1623

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -1
-   ret = video_register_device(jpeg->dec_vdev, VFL_TYPE_GRABBER, 3);
+   ret = video_register_device(jpeg->vfd_jpeg, VFL_TYPE_GRABBER, -1);

The change from 3 to -1 also looks like something for a separate patch.

same as the above reply

Ditto.

done.

Quoted 22 lines

[snip]

Quoted 20 lines
Quoted 20 lines
Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +2 -0
@@ -17,23 +18,77 @@
 
 #define MTK_JPEG_FMT_FLAG_DEC_OUTPUT       BIT(0)
 #define MTK_JPEG_FMT_FLAG_DEC_CAPTURE      BIT(1)
+#define MTK_JPEG_FMT_FLAG_ENC_OUTPUT       BIT(2)
+#define MTK_JPEG_FMT_FLAG_ENC_CAPTURE      BIT(3)
 
 #define MTK_JPEG_FMT_TYPE_OUTPUT   1

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 30

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +4 -4
 #define MTK_JPEG_FMT_TYPE_CAPTURE  2
 
-#define MTK_JPEG_MIN_WIDTH 32
-#define MTK_JPEG_MIN_HEIGHT        32
-#define MTK_JPEG_MAX_WIDTH 8192
-#define MTK_JPEG_MAX_HEIGHT        8192
+#define MTK_JPEG_MIN_WIDTH 32U
+#define MTK_JPEG_MIN_HEIGHT        32U
+#define MTK_JPEG_MAX_WIDTH 65535U
+#define MTK_JPEG_MAX_HEIGHT        65535U

Why is it okay to change this from 8192 to 65535?

Quoted 4 lines
Quoted 1 line

our hw support max width/height to 65535

Does this also apply to the JPEG decoder on MT8173 for which the driver was developed?

yes.

Quoted 15 lines

[snip]

Quoted 10 lines
Quoted 10 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h struct mtk_jpeg_q_data { · line 168

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +8 -0
+/**
+ * jpeg_enc_param - parameters of jpeg encode control
+ * @enable_exif:   EXIF enable for jpeg encode mode
+ * @enc_quality:   destination image quality in encode mode
+ * @restart_interval:      JPEG restart interval for JPEG encoding
+ */
+struct jpeg_enc_param {
+   u32 enable_exif;

Shouldn't this be a bool?

Quoted 2 lines

this value is seted by V4L2_CID_JPEG_ACTIVE_MARKER control,its' value is V4L2_JPEG_ACTIVE_MARKER_APP1(1<<1),not a bool

Quoted 6 lines

In this case, please call it active_marker and also update the comment above accordingly.

Still, if this driver only cares about V4L2_JPEG_ACTIVE_MARKER_APP1, bool enable_exif, assigned by the code appropriately, would make more sense.

done.

Quoted 22 lines

[snip]

Quoted 14 lines
Quoted 14 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 62

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +9 -0
+
+static void mtk_jpeg_enc_set_encFormat(void __iomem *base, u32 enc_format)
+{
+   u32 value;
+
+   value = readl(base + JPGENC_CTRL);
+   value &= ~JPEG_ENC_CTRL_YUV_BIT;
+   value |= JPGENC_FORMAT(enc_format);
+   writel(value, base + JPGENC_CTRL);

The model I suggested above also avoids this kind of read modify write operations, which just unnecessarily add synchronous MMIO round trips, which means more CPU overhead.

Quoted 5 lines
Quoted 2 lines

Given that the full state is always known by the driver, it can just write all the register values without the need to read them back.

the JPGENC_CTRL register has 32 bits,different bits have different meanings. encformat is bit 3~4,so we need ready before write to guarantee other bits' value not be changed

Quoted 6 lines

I explained this more in my review comments to the latest revision, but this is just solving a problem that is introduced by the design of the code. If the code was designed so that 1 register is only changed in 1 function, there would be no need to read back the registers, because the function would initialize the full register at a time. That's what most of the other drivers do.

done.

Quoted 11 lines

[snip]

Quoted 9 lines
Quoted 9 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h line 31

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h new file +6 -0
+enum {
+   MTK_JPEG_ENC_RESULT_DONE                = 0,
+   MTK_JPEG_ENC_RESULT_STALL,
+   MTK_JPEG_ENC_RESULT_VCODEC_IRQ,
+   MTK_JPEG_ENC_RESULT_ERROR_UNKNOWN
+};

Do we need these intermediate error codes? Could we just use errno values instead?

Quoted 8 lines
Quoted 2 lines

we can just use errno values,but our interrupt status have three state,done/stall,vcodec irq,maybe listing them here makes more clarity

I suggested another approach in my comments for the latest revision. Generally the interrupt handler is the only place where this error handling is done, so the intermediate error codes shouldn't be necessary, as the interrupt handler would directly signal any issues to V4L2.

I have removed the MTK_JPEG_ENC_RESULT_ERROR_UNKNOWN.

Quoted 24 lines

[snip]

Quoted 13 lines
Quoted 11 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h line 47

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h new file +8 -0
+#define JPGENC_FORMAT(x)           (((x) & 3) << 3)
+#define JPGENC_WIDTH_HEIGHT(w, h)  (((w) << 16) | (h))
+#define JPGENC_INIT_OFFSET(x)              ((x) & (~0xF))
+#define JPGENC_OFFSET_MASK(x)              ((x) & 0xF)
+#define JPGENC_DST_ADDR(x)         ((x) & (~0xF))
+#define JPGENC_STALL_ADDR(x, y)            (((x) + (y)) & (~0xF))
+#define JPGENC_QUALITY_MASK                0xFFFF0000
+#define JPGENC_SET_QUALITY(x, y)   (((x) & JPGENC_QUALITY_MASK) | (y))

JPGENC_QUALITY_MASK is only used here, so 0xffff0000 could be just used directly.

done

Quoted 1 line
Quoted 8 lines
Quoted 5 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h line 49

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h new file +2 -0
+
+#endif /* _MTK_JPEG_ENC_REG_H */

I can see some bits defined in mtk_jpeg_enc_hw.h as well. Perhaps all these could be moved there too, which would make us have 1 file less?

moved bits defined and register defined to mtk_jpeg_enc_reg.h, but the functions declaration about jpeg enc hw were in mtk_jpeg_enc_hw.h, is that ok?

Quoted 8 lines

My suggestion was the other way around - have all the encoder-specific definitions in mtk_jpeg_enc_hw.h and remove mtk_jpeg_enc_reg.h.

In general, this is a relatively simple hardware block and we should be able to just have one file that deals with hardware registers. Then there would be no need for headers, as the register definitions could be put directly in the source file. A rule of thumb is to use a header when something needs to be shared between multiple source files.

done.

Quoted 3 lines

Best regards,
Tomasz

#16 lore raw

Re: [PATCH v4 5/5] media: platform: Add jpeg dec/enc feature

Hi Xia,

Finally found time to take a look again. Really sorry for the late reply.

Quoted 154 lines · Xia Jiang
Quoted 69 lines · Tomasz Figa

Hi Xia,

Quoted 66 lines · Xia Jiang
Quoted 64 lines · Tomasz Figa

Hi Xia,

Quoted 27 lines · Xia Jiang

Add mtk jpeg encode v4l2 driver based on jpeg decode, because that jpeg decode and encode have great similarities with function operation.

---
v4: split mtk_jpeg_try_fmt_mplane() to two functions, one for encoder,
one for decoder.
split mtk_jpeg_set_default_params() to two functions, one for encoder, one for decoder.
add cropping support for encoder in g/s_selection ioctls. change exif mode support by using V4L2_JPEG_ACTIVE_MARKER_APP1. change MTK_JPEG_MAX_WIDTH/MTK_JPEG_MAX_HEIGH from 8192 to 65535 by specification.
move width shifting operation behind aligning operation in mtk_jpeg_try_enc_fmt_mplane() for bug fix.
fix user abuseing data_offset issue for DMABUF in mtk_jpeg_set_enc_src().
fix kbuild warings: change MTK_JPEG_MIN_HEIGHT/MTK_JPEG_MAX_HEIGHT
and MTK_JPEG_MIN_WIDTH/MTK_JPEG_MAX_WIDTH from 'int' type to 'unsigned int' type. fix msleadingly indented of 'else'.

v3: delete Change-Id.
only test once handler->error after the last v4l2_ctrl_new_std(). seperate changes of v4l2-ctrls.c and v4l2-controls.h to new patch.

v2: fix compliance test fail, check created buffer size in driver.

Quoted 13 lines
---
 drivers/media/platform/mtk-jpeg/Makefile      |   5 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 731 +++++++++++++++---
 .../media/platform/mtk-jpeg/mtk_jpeg_core.h   | 123 ++-
 .../media/platform/mtk-jpeg/mtk_jpeg_dec_hw.h |   7 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c | 175 +++++
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h |  60 ++
 .../platform/mtk-jpeg/mtk_jpeg_enc_reg.h      |  49 ++
 7 files changed, 1004 insertions(+), 146 deletions(-)
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_reg.h

First of all, thanks for the patch!

Please check my comments below.

My general feeling about this code is that the encoder hardware block is completely orthogonal from the decoder block and there is very little code reuse from the original decoder driver.

Moreover, a lot of existing code now needs if (decoder) { ... } else {... } segments, which complicates the code.

Would it perhaps make sense to instead create a separate mtk-jpeg-enc driver?

It would also give us a fresh start in terms of code quality, as the existing mtk-jpeg driver has a lot of quality issues unfortunately. (Some of my comments to this patch actually relate to the issues with the original code, not introduced by this patch, but we need to fix them if changing this driver already.)

Quoted 7 lines
Quoted 5 lines

Dear Tomasz,

I haved fixed the driver by following your advice in general.

Please check my reply below.

Sorry, I missed this message originally. Replied below.

Dear Tomasz,

Thank you for your reply.I have changed the driver in the V8 version by following your good advice.

Quoted 18 lines

[snip]

Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +8 -0
+   }
+   param->enc_w = q_data_src->w;
+   param->enc_h = q_data_src->h;
+
+   if (jpeg_params->enc_quality >= 97)
+           param->enc_quality = JPEG_ENCODE_QUALITY_Q97;
+   else if (jpeg_params->enc_quality >= 95)
+           param->enc_quality = JPEG_ENCODE_QUALITY_Q95;
Quoted 6 lines
Quoted 2 lines

I'm wondering if the application requests 96, it doesn't expect the quality to be _at_least_ 96.

our jpeg enc hw do not support quality 96,only support 15 kinds of quant table listed here, so if the application requests 96,a nearest and highest quality will be given.

Quoted 1 line
Quoted 4 lines

Just to clarify my comment, if I remember correctly, the JPEG standard defines the 100 levels, so if the application requests level 96, but the hardware provides only 95 and 97, the quality should be favored and 97 used.

done.

Quoted 26 lines

[snip]

Quoted 11 lines
Quoted 4 lines
Quoted 2 lines

+ param->mem_stride = mtk_jpeg_align(width_even, (is_420 ? 16 : 32));

What's the difference between img_stride and mem_stride?

In theory, mem_stride need >= img_stride,but we use the same is ok

Quoted 5 lines
Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -0
+   param->total_encdu =
+           ((padding_width >> 4) * (padding_height >> (is_420 ? 4 : 3)) *
+           (is_420 ? 6 : 4)) - 1;
Quoted 7 lines
Quoted 4 lines

The comment above the struct says this is the total number of 8x8 blocks. Why would it depend on whether the format is YUV 4:2:0? Since we should have already aligned the width and height in try_fmt, this should be as simple as (width / 8) * (height / 8).

becuase the image size is w*h*1.5 for yuv420 format, but w*h*2 for yuv422,so for yuv420: w_16/8*h_16/8*1.5-1(because the hw will start at number 0), yuv422: w_32/8*h_8/8*2-1,this number is equal to my code.

Do you mean that this also includes the Cb and Cr 8x8 blocks separately? If so, could it be rewritten as below to improve the readability?

luma_blocks = width / 8 * height / 8;
if (is_420)

Quoted 7 lines
    chroma_blocks = luma_blocks / 4;
else
    chroma_blocks = luma_blocks / 2;

param->last_encdu = luma_blocks + 2 * chroma_blocks - 1;

Also, does it mean that this number is actually the index of the last block, not the total number of blocks?

yes.

Quoted 15 lines

If so, the field should be probably renamed to last_encdu and the description updated accordingly.

[snip]

Quoted 10 lines
Quoted 7 lines

Could we instead check the buffer address alignment in .buf_prepare and fail if it's not big enough?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx, · line 1150

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -0
+   bs->dma_addr_offset = p->enable_exif ? MTK_JPEG_DEFAULT_EXIF_SIZE : 0;
+   bs->dma_addr_offsetmask = bs->dma_addr & JPEG_ENC_DST_ADDR_OFFSET_MASK;

What is the meaning of this offset mask?

our actual destination address = destination address + offset address+ destination address offset mask.The mask 0:No offset,1~15:offset byte from the 16-byte aligned

Quoted 6 lines
Quoted 3 lines

So we have dma_addr, dma_addr_offset and dma_addr_offsetmask. Why do we need dma_addr_offset? Would the same operation be achieved with the code below?

Because in exif mode, the beginning of the destination buffer(offset) should be reserved for the application to fill(I explained this more in my reply to the v7 revision).

Thanks. I'll check the explanation in v7.

Quoted 11 lines
Quoted 10 lines

dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0); if (p->enable_exif)

    dma_addr += MTK_JPEG_DEFAULT_EXIF_SIZE;

bs->dma_addr = dma_addr & ~JPEG_ENC_DST_ADDR_OFFSET_MASK; bs->dma_addr_offset = 0;
bs->dma_addr_offsetmask = dma_addr & JPEG_ENC_DST_ADDR_OFFSET_MASK;

Or does the hardware write something directly at bs->dma_addr (some tags?) and then the encoded image at the final desintation address?

Hardware just write encoded image at the final destination address.

Okay, so it sounds like we should be able to simplify the calculation.

Quoted 146 lines
Quoted 50 lines

[snip]

Quoted 49 lines
Quoted 49 lines
Quoted 35 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +34 -1
-static void mtk_jpeg_set_default_params(struct mtk_jpeg_ctx *ctx)
+static void mtk_jpeg_set_enc_default_params(struct mtk_jpeg_ctx *ctx)
+{
+   struct mtk_jpeg_q_data *q = &ctx->out_q;
+   int align_w, align_h;
+
+   ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+
+   ctx->colorspace = V4L2_COLORSPACE_JPEG,
+   ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+   ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
+   ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+
+   q->w = MTK_JPEG_MIN_WIDTH;
+   q->h = MTK_JPEG_MIN_HEIGHT;
+
+   q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_YUYV,
+                                 MTK_JPEG_FMT_TYPE_OUTPUT);
+
+   align_w = q->w;
+   align_h = q->h;
+   align_w = round_up(align_w, 2);
+   v4l_bound_align_image(&align_w, MTK_JPEG_MIN_WIDTH, MTK_JPEG_MAX_WIDTH,
+                         5, &align_h, MTK_JPEG_MIN_HEIGHT,
+                         MTK_JPEG_MAX_HEIGHT, 3, 0);
+   align_w = align_w << 1;
+
+   if (align_w < MTK_JPEG_MIN_WIDTH &&
+       (align_w + 32) <= MTK_JPEG_MAX_WIDTH)
+           align_w += 32;
+   if (align_h < MTK_JPEG_MIN_HEIGHT &&
+       (align_h + 8) <= MTK_JPEG_MAX_HEIGHT)
+           align_h += 8;
+
+   q->sizeimage[0] = align_w * align_h;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1408

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+   q->bytesperline[0] = align_w;
+
+   q = &ctx->cap_q;
+   q->w = MTK_JPEG_MIN_WIDTH;
+   q->h = MTK_JPEG_MIN_HEIGHT;
+   q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_JPEG,
+                                 MTK_JPEG_FMT_TYPE_CAPTURE);
+   q->bytesperline[0] = 0;
+   q->sizeimage[0] = MTK_JPEG_DEFAULT_SIZEIMAGE;
+}

Could we just create an arbitrary v4l2_pix_format_mplane struct and call s_fmt instead? In general, all of the constant values and alignments should be already ensured by try_fmt, so this function should be redundant.

Quoted 6 lines
Quoted 1 line

if cancel this function,the v4l2-compliance test will fail

Sorry, I guess my comment was not clear. We need to initialize the default parameters. However, the contents of this function seem to heavily duplicate with the code that should be in try_fmt, so could we just call try_fmt from here instead of repeating the calculations?

done.

Quoted 10 lines

[snip]

Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -4
-   ret = devm_request_irq(&pdev->dev, dec_irq, mtk_jpeg_dec_irq, 0,
+   ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_irq, 0,
                           pdev->name, jpeg);
    if (ret) {
-           dev_err(&pdev->dev, "Failed to request dec_irq %d (%d)\n",
-                   dec_irq, ret);
-           ret = -EINVAL;
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

This removal of ret assignment looks like a separate fix that should be done in its own patch.

this change is because of the adding of jpeg enc driver,not the orignal driver' question, should I move it to the orignal driver's patch?

Yes, please.

done.

Quoted 42 lines
Quoted 42 lines
Quoted 42 lines
Quoted 42 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +16 -13
+           dev_err(&pdev->dev, "Failed to request jpeg_irq %d (%d)\n",
+                   jpeg_irq, ret);
            goto err_req_irq;
    }
@@ -1140,33 +1602,35 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
            goto err_m2m_init;
    }
 
-   jpeg->dec_vdev = video_device_alloc();
-   if (!jpeg->dec_vdev) {
+   jpeg->vfd_jpeg = video_device_alloc();
+   if (!jpeg->vfd_jpeg) {
            ret = -ENOMEM;
-           goto err_dec_vdev_alloc;
+           goto err_vfd_jpeg_alloc;
    }
-   snprintf(jpeg->dec_vdev->name, sizeof(jpeg->dec_vdev->name),
-            "%s-dec", MTK_JPEG_NAME);
-   jpeg->dec_vdev->fops = &mtk_jpeg_fops;
-   jpeg->dec_vdev->ioctl_ops = &mtk_jpeg_ioctl_ops;
-   jpeg->dec_vdev->minor = -1;
-   jpeg->dec_vdev->release = video_device_release;
-   jpeg->dec_vdev->lock = &jpeg->lock;
-   jpeg->dec_vdev->v4l2_dev = &jpeg->v4l2_dev;
-   jpeg->dec_vdev->vfl_dir = VFL_DIR_M2M;
-   jpeg->dec_vdev->device_caps = V4L2_CAP_STREAMING |
+   snprintf(jpeg->vfd_jpeg->name, sizeof(jpeg->vfd_jpeg->name),
+            "%s-%s", MTK_JPEG_NAME,
+            jpeg->mode == MTK_JPEG_ENC ? "enc" : "dec");
+   jpeg->vfd_jpeg->fops = &mtk_jpeg_fops;
+   jpeg->vfd_jpeg->ioctl_ops = &mtk_jpeg_ioctl_ops;
+   jpeg->vfd_jpeg->minor = -1;
+   jpeg->vfd_jpeg->release = video_device_release;
+   jpeg->vfd_jpeg->lock = &jpeg->lock;
+   jpeg->vfd_jpeg->v4l2_dev = &jpeg->v4l2_dev;
+   jpeg->vfd_jpeg->vfl_dir = VFL_DIR_M2M;
+   jpeg->vfd_jpeg->device_caps = V4L2_CAP_STREAMING |
                                  V4L2_CAP_VIDEO_M2M_MPLANE;
 
Quoted 7 lines
Quoted 5 lines
Quoted 4 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_probe(struct platform_device *pdev) · line 1623

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -1
-   ret = video_register_device(jpeg->dec_vdev, VFL_TYPE_GRABBER, 3);
+   ret = video_register_device(jpeg->vfd_jpeg, VFL_TYPE_GRABBER, -1);

The change from 3 to -1 also looks like something for a separate patch.

same as the above reply

Ditto.

done.

Quoted 22 lines

[snip]

Quoted 20 lines
Quoted 20 lines
Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +2 -0
@@ -17,23 +18,77 @@
 
 #define MTK_JPEG_FMT_FLAG_DEC_OUTPUT       BIT(0)
 #define MTK_JPEG_FMT_FLAG_DEC_CAPTURE      BIT(1)
+#define MTK_JPEG_FMT_FLAG_ENC_OUTPUT       BIT(2)
+#define MTK_JPEG_FMT_FLAG_ENC_CAPTURE      BIT(3)
 
 #define MTK_JPEG_FMT_TYPE_OUTPUT   1

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 30

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +4 -4
 #define MTK_JPEG_FMT_TYPE_CAPTURE  2
 
-#define MTK_JPEG_MIN_WIDTH 32
-#define MTK_JPEG_MIN_HEIGHT        32
-#define MTK_JPEG_MAX_WIDTH 8192
-#define MTK_JPEG_MAX_HEIGHT        8192
+#define MTK_JPEG_MIN_WIDTH 32U
+#define MTK_JPEG_MIN_HEIGHT        32U
+#define MTK_JPEG_MAX_WIDTH 65535U
+#define MTK_JPEG_MAX_HEIGHT        65535U

Why is it okay to change this from 8192 to 65535?

Quoted 5 lines
Quoted 4 lines
Quoted 1 line

our hw support max width/height to 65535

Does this also apply to the JPEG decoder on MT8173 for which the driver was developed?

yes.

Okay, then the limit change should be a separate patch that fixes the wrong limits in existing code. It could be also merged to stable kernels.

Best regards,
Tomasz