Skip to contentSkip to conversation

Add support for mt2701 JPEG ENC support

PATCH RESEND Superseded Xia Jiang 4 participants 45 messages Jun 4, 2020 – Jul 24, 2020

Subsystems (3)

v9 of 12 Superseded Latest is v12 →

lore
Share & tools
lore

Xia Jiang

author
#0 lore raw

v9 cover Add support for mt2701 JPEG ENC support

This patchset add support for mt2701 JPEG ENC support.

This is the compliance test result for jpeg dec and enc.

The JPEG dec log:
------------------------------------------------------------
v4l2-compliance -d /dev/video0
v4l2-compliance SHA: 74377da4f5f3b63203c599d5dd75db6af91fdbb9, 32 bits, 32-bit time_t

Compliance test for mtk-jpeg device /dev/video0:

Driver Info:

        Driver name      : mtk-jpeg                                             
        Card type        : mtk-jpeg decoder                                     
        Bus info         : platform:15004000.jpegdec                            
        Driver version   : 5.7.0                                                
        Capabilities     : 0x84204000                                           

Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Detected JPEG Decoder

Required ioctls:
test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
test second /dev/video0 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK

test invalid ioctls: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 0 Private Controls: 0

Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK (Not Supported)
test Composing: OK
test Scaling: OK (Not Supported)

Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)

Total for mtk-jpeg device /dev/video0: 45, Succeeded: 45, Failed: 0, Warnings: 0
------------------------------------------------------------

The JPEG enc log:

------------------------------------------------------------
v4l2-compliance -d /dev/video1
v4l2-compliance SHA: 74377da4f5f3b63203c599d5dd75db6af91fdbb9, 32 bits, 32-bit time_t

Compliance test for mtk-jpeg device /dev/video1:

Driver Info:

        Driver name      : mtk-jpeg                                                                
        Card type        : mtk-jpeg encoder                                                        
        Bus info         : platform:1500a000.jpegenc                                               
        Driver version   : 5.7.0                                                                   
        Capabilities     : 0x84204000                                                              

Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Detected JPEG Encoder

Required ioctls:
test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
test second /dev/video1 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK

test invalid ioctls: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 4 Private Controls: 0

Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)

Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)

Total for mtk-jpeg device /dev/video1: 45, Succeeded: 45, Failed: 0, Warnings: 0
------------------------------------------------------------

Change compared to v8:
-change commit message of patch 02/18
-use pm_runtime_put() to replace pm_runtime_put_sync() of patch 05/18
-add one patch for deletting the resetting hardware flow in the system PM ops
-use v4l2_m2m_suspend() and v4l2_m2m_resume() to improve the implemention
of the system PM ops. This patch(07/18) depends on [RFC,V4,1/4] media:
v4l2_mem2mem: add v4l2_m2m_suspend, v4l2_m2m_resume(https://patchwork.kernel.org/patch/11272917/)
-add one patch for cancelling the last frame handling flow
-add one patch for deletting zeroing the reserved fields
-move changing data type of max/min width/height to patch 10/18
-add one patch for renaming existing functions/defines/variables

Reason for resend:
-delete check.txt in patch 06/18

Xia Jiang (18):
media: platform: Improve subscribe event flow for bug fixing
media: platform: Improve queue set up flow for bug fixing
media: platform: Improve getting and requesting irq flow for bug
fixing
media: platform: Change the fixed device node number to unfixed value
media: platform: Improve power on and power off flow
media: platform: Delete the resetting hardware flow in the system PM
ops
media: platform: Improve the implementation of the system PM ops
media: platform: Cancel the last frame handling flow
media: platform: Delete zeroing the reserved fields
media: platform: Stylistic changes for improving code quality
media: platform: Use generic rounding helpers
media: platform: Change MTK_JPEG_COMP_MAX macro definition location
media: platform: Delete redundant code and add annotation for an enum
media: dt-bindings: Add jpeg enc device tree node document
arm: dts: mt2701: Add jpeg enc device tree node
media: platform: Rename jpeg dec file name
media: platform: Rename existing functions/defines/variables
media: platform: Add jpeg enc feature

 .../bindings/media/mediatek-jpeg-encoder.txt  |   37 +
 arch/arm/boot/dts/mt2701.dtsi                 |   13 +
 drivers/media/platform/mtk-jpeg/Makefile      |    5 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 1130 ++++++++++++-----
 .../media/platform/mtk-jpeg/mtk_jpeg_core.h   |   66 +-
 .../{mtk_jpeg_hw.c => mtk_jpeg_dec_hw.c}      |   10 +-
 .../{mtk_jpeg_hw.h => mtk_jpeg_dec_hw.h}      |   14 +-
 ...{mtk_jpeg_parse.c => mtk_jpeg_dec_parse.c} |    2 +-
 ...{mtk_jpeg_parse.h => mtk_jpeg_dec_parse.h} |    2 +-
 .../{mtk_jpeg_reg.h => mtk_jpeg_dec_reg.h}    |   19 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c |  193 +++
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h |  123 ++
 12 files changed, 1282 insertions(+), 332 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek-jpeg-encoder.txt
 rename drivers/media/platform/mtk-jpeg/{mtk_jpeg_hw.c => mtk_jpeg_dec_hw.c} (98%)
 rename drivers/media/platform/mtk-jpeg/{mtk_jpeg_hw.h => mtk_jpeg_dec_hw.h} (89%)
 rename drivers/media/platform/mtk-jpeg/{mtk_jpeg_parse.c => mtk_jpeg_dec_parse.c} (98%)
 rename drivers/media/platform/mtk-jpeg/{mtk_jpeg_parse.h => mtk_jpeg_dec_parse.h} (92%)
 rename drivers/media/platform/mtk-jpeg/{mtk_jpeg_reg.h => mtk_jpeg_dec_reg.h} (77%)
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h
Signature · 6 lines
2.18.0 

_______________________________________________
Linux-mediatek mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-mediatek
author
#1 lore raw

PATCH RESEND v9 1/18 media: platform: Improve subscribe event flow for bug fixing

Let v4l2_ctrl_subscribe_event() do the job for other types except V4L2_EVENT_SOURCE_CHANGE.

---
v9: no changes

---
 drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -2
index f82a81a3bdee..4ad4a4b30a0e 100644
@@ -446,9 +446,9 @@ static int mtk_jpeg_subscribe_event(struct v4l2_fh *fh,
 	switch (sub->type) {
 	case V4L2_EVENT_SOURCE_CHANGE:
 		return v4l2_src_change_event_subscribe(fh, sub);
-	default:
-		return -EINVAL;
 	}
+
+	return v4l2_ctrl_subscribe_event(fh, sub);
 }
 
 static int mtk_jpeg_g_selection(struct file *file, void *priv,
Signature · 1 line
2.18.0
author
#2 lore raw

PATCH RESEND v9 2/18 media: platform: Improve queue set up flow for bug fixing

In reply to Xia Jiang

Add checking created buffer size follow in mtk_jpeg_queue_setup().

---
v9: no changes

---
 drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c | 7 +++++++
 1 file changed, 7 insertions(+)
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +7 -0
index 4ad4a4b30a0e..67a022d04df7 100644
@@ -571,6 +571,13 @@ static int mtk_jpeg_queue_setup(struct vb2_queue *q,
 	if (!q_data)
 		return -EINVAL;
 
+	if (*num_planes) {
+		for (i = 0; i < *num_planes; i++)
+			if (sizes[i] < q_data->sizeimage[i])
+				return -EINVAL;
+		return 0;
+	}
+
 	*num_planes = q_data->fmt->colplanes;
 	for (i = 0; i < q_data->fmt->colplanes; i++) {
 		sizes[i] = q_data->sizeimage[i];
Signature · 5 lines
2.18.0
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
author
#3 lore raw

PATCH RESEND v9 3/18 media: platform: Improve getting and requesting irq flow for bug fixing

In reply to Xia Jiang

Delete platform_get_resource operation for irq.
Return actual value rather than EINVAL when fail to get and request irq.

---
v9: no changes

---
 drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -5
index 67a022d04df7..2677580941b0 100644
@@ -1103,12 +1103,10 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 		return ret;
 	}
 
-	res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
 	dec_irq = platform_get_irq(pdev, 0);
-	if (!res || dec_irq < 0) {
+	if (dec_irq < 0) {
 		dev_err(&pdev->dev, "Failed to get dec_irq %d.\n", dec_irq);
-		ret = -EINVAL;
-		return ret;
+		return dec_irq;
 	}
 
 	ret = devm_request_irq(&pdev->dev, dec_irq, mtk_jpeg_dec_irq, 0,
@@ -1116,7 +1114,6 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 	if (ret) {
 		dev_err(&pdev->dev, "Failed to request dec_irq %d (%d)\n",
 			dec_irq, ret);
-		ret = -EINVAL;
 		goto err_req_irq;
 	}
 
Signature · 1 line
2.18.0
author
#4 lore raw

v9 cover Add support for mt2701 JPEG ENC support

In reply to Xia Jiang

This patchset add support for mt2701 JPEG ENC support.

This is the compliance test result for jpeg dec and enc.

The JPEG dec log:
------------------------------------------------------------
v4l2-compliance -d /dev/video0
v4l2-compliance SHA: 74377da4f5f3b63203c599d5dd75db6af91fdbb9, 32 bits, 32-bit time_t

Compliance test for mtk-jpeg device /dev/video0:

Driver Info:

        Driver name      : mtk-jpeg                                             
        Card type        : mtk-jpeg decoder                                     
        Bus info         : platform:15004000.jpegdec                            
        Driver version   : 5.7.0                                                
        Capabilities     : 0x84204000                                           

Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Detected JPEG Decoder

Required ioctls:
test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
test second /dev/video0 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK

test invalid ioctls: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 0 Private Controls: 0

Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK (Not Supported)
test Composing: OK
test Scaling: OK (Not Supported)

Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)

Total for mtk-jpeg device /dev/video0: 45, Succeeded: 45, Failed: 0, Warnings: 0
------------------------------------------------------------

The JPEG enc log:

------------------------------------------------------------
v4l2-compliance -d /dev/video1
v4l2-compliance SHA: 74377da4f5f3b63203c599d5dd75db6af91fdbb9, 32 bits, 32-bit time_t

Compliance test for mtk-jpeg device /dev/video1:

Driver Info:

        Driver name      : mtk-jpeg                                                                
        Card type        : mtk-jpeg encoder                                                        
        Bus info         : platform:1500a000.jpegenc                                               
        Driver version   : 5.7.0                                                                   
        Capabilities     : 0x84204000                                                              

Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Detected JPEG Encoder

Required ioctls:
test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
test second /dev/video1 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK

test invalid ioctls: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 4 Private Controls: 0

Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)

Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)

Total for mtk-jpeg device /dev/video1: 45, Succeeded: 45, Failed: 0, Warnings: 0
------------------------------------------------------------

Change compared to v8:
-change commit message of patch 02/18
-use pm_runtime_put() to replace pm_runtime_put_sync() of patch 05/18
-add one patch for deletting the resetting hardware flow in the system PM ops
-use v4l2_m2m_suspend() and v4l2_m2m_resume() to improve the implemention
of the system PM ops. This patch(07/18) depends on [RFC,V4,1/4] media:
v4l2_mem2mem: add v4l2_m2m_suspend, v4l2_m2m_resume(https://patchwork.kernel.org/patch/11272917/)
-add one patch for cancelling the last frame handling flow
-add one patch for deletting zeroing the reserved fields
-move changing data type of max/min width/height to patch 10/18
-add one patch for renaming existing functions/defines/variables

Reason for resend:
-delete check.txt in patch 06/18

Xia Jiang (18):
media: platform: Improve subscribe event flow for bug fixing
media: platform: Improve queue set up flow for bug fixing
media: platform: Improve getting and requesting irq flow for bug
fixing
media: platform: Change the fixed device node number to unfixed value
media: platform: Improve power on and power off flow
media: platform: Delete the resetting hardware flow in the system PM
ops
media: platform: Improve the implementation of the system PM ops
media: platform: Cancel the last frame handling flow
media: platform: Delete zeroing the reserved fields
media: platform: Stylistic changes for improving code quality
media: platform: Use generic rounding helpers
media: platform: Change MTK_JPEG_COMP_MAX macro definition location
media: platform: Delete redundant code and add annotation for an enum
media: dt-bindings: Add jpeg enc device tree node document
arm: dts: mt2701: Add jpeg enc device tree node
media: platform: Rename jpeg dec file name
media: platform: Rename existing functions/defines/variables
media: platform: Add jpeg enc feature

 .../bindings/media/mediatek-jpeg-encoder.txt  |   37 +
 arch/arm/boot/dts/mt2701.dtsi                 |   13 +
 drivers/media/platform/mtk-jpeg/Makefile      |    5 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 1134 ++++++++++++-----
 .../media/platform/mtk-jpeg/mtk_jpeg_core.h   |   66 +-
 .../{mtk_jpeg_hw.c => mtk_jpeg_dec_hw.c}      |   10 +-
 .../{mtk_jpeg_hw.h => mtk_jpeg_dec_hw.h}      |   14 +-
 ...{mtk_jpeg_parse.c => mtk_jpeg_dec_parse.c} |    2 +-
 ...{mtk_jpeg_parse.h => mtk_jpeg_dec_parse.h} |    2 +-
 .../{mtk_jpeg_reg.h => mtk_jpeg_dec_reg.h}    |   19 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c |  193 +++
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h |  123 ++
 12 files changed, 1284 insertions(+), 334 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek-jpeg-encoder.txt
 rename drivers/media/platform/mtk-jpeg/{mtk_jpeg_hw.c => mtk_jpeg_dec_hw.c} (98%)
 rename drivers/media/platform/mtk-jpeg/{mtk_jpeg_hw.h => mtk_jpeg_dec_hw.h} (89%)
 rename drivers/media/platform/mtk-jpeg/{mtk_jpeg_parse.c => mtk_jpeg_dec_parse.c} (98%)
 rename drivers/media/platform/mtk-jpeg/{mtk_jpeg_parse.h => mtk_jpeg_dec_parse.h} (92%)
 rename drivers/media/platform/mtk-jpeg/{mtk_jpeg_reg.h => mtk_jpeg_dec_reg.h} (77%)
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h
Signature · 5 lines
2.18.0
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
author
#5 lore raw

PATCH RESEND v9 4/18 media: platform: Change the fixed device node number to unfixed value

In reply to Xia Jiang

The driver can be instantiated multiple times, e.g. for a decoder and an encoder. Moreover, other drivers could coexist on the same system. This makes the static video node number assignment pointless, so switch to automatic assignment instead.

---
v9: change the commit message

---
 drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -1
index 2677580941b0..12609ca46fd9 100644
@@ -1154,7 +1154,7 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 	jpeg->dec_vdev->device_caps = V4L2_CAP_STREAMING |
 				      V4L2_CAP_VIDEO_M2M_MPLANE;
 
-	ret = video_register_device(jpeg->dec_vdev, VFL_TYPE_VIDEO, 3);
+	ret = video_register_device(jpeg->dec_vdev, VFL_TYPE_VIDEO, -1);
 	if (ret) {
 		v4l2_err(&jpeg->v4l2_dev, "Failed to register video device\n");
 		goto err_dec_vdev_register;
Signature · 1 line
2.18.0
Reviewed-by on v9 4/18
#6 lore raw

Re: [PATCH RESEND v9 04/18] media: platform: Change the fixed device node number to unfixed value

Hi Xia,

Quoted 5 lines · Xia Jiang

The driver can be instantiated multiple times, e.g. for a decoder and an encoder. Moreover, other drivers could coexist on the same system. This makes the static video node number assignment pointless, so switch to automatic assignment instead.

Quoted 7 lines

---
v9: change the commit message

---
 drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Best regards,
Tomasz

author
#7 lore raw

PATCH RESEND v9 5/18 media: platform: Improve power on and power off flow

In reply to Xia Jiang

Call pm_runtime_get_sync() before starting a frame and then pm_runtime_put() after completing it. This can save power for the time between processing two frames.

---
v9: use pm_runtime_put() to replace pm_runtime_put_sync()

---
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 27 +++++--------------
 1 file changed, 6 insertions(+), 21 deletions(-)
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -21
index 12609ca46fd9..fb624385969e 100644
@@ -710,23 +710,6 @@ static struct vb2_v4l2_buffer *mtk_jpeg_buf_remove(struct mtk_jpeg_ctx *ctx,
 		return v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
 }
 
-static int mtk_jpeg_start_streaming(struct vb2_queue *q, unsigned int count)
-{
-	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(q);
-	struct vb2_v4l2_buffer *vb;
-	int ret = 0;
-
-	ret = pm_runtime_get_sync(ctx->jpeg->dev);
-	if (ret < 0)
-		goto err;
-
-	return 0;
-err:
-	while ((vb = mtk_jpeg_buf_remove(ctx, q->type)))
-		v4l2_m2m_buf_done(vb, VB2_BUF_STATE_QUEUED);
-	return ret;
-}
-
 static void mtk_jpeg_stop_streaming(struct vb2_queue *q)
 {
 	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(q);
@@ -751,8 +734,6 @@ static void mtk_jpeg_stop_streaming(struct vb2_queue *q)
 
 	while ((vb = mtk_jpeg_buf_remove(ctx, q->type)))
 		v4l2_m2m_buf_done(vb, VB2_BUF_STATE_ERROR);
-
-	pm_runtime_put_sync(ctx->jpeg->dev);
 }
 
 static const struct vb2_ops mtk_jpeg_qops = {
@@ -761,7 +742,6 @@ static const struct vb2_ops mtk_jpeg_qops = {
 	.buf_queue          = mtk_jpeg_buf_queue,
 	.wait_prepare       = vb2_ops_wait_prepare,
 	.wait_finish        = vb2_ops_wait_finish,
-	.start_streaming    = mtk_jpeg_start_streaming,
 	.stop_streaming     = mtk_jpeg_stop_streaming,
 };
 
@@ -812,7 +792,7 @@ static void mtk_jpeg_device_run(void *priv)
 	struct mtk_jpeg_src_buf *jpeg_src_buf;
 	struct mtk_jpeg_bs bs;
 	struct mtk_jpeg_fb fb;
-	int i;
+	int i, ret;
 
 	src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
 	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
@@ -832,6 +812,10 @@ static void mtk_jpeg_device_run(void *priv)
 		return;
 	}
 
+	ret = pm_runtime_get_sync(jpeg->dev);
+	if (ret < 0)
+		goto dec_end;
+
 	mtk_jpeg_set_dec_src(ctx, &src_buf->vb2_buf, &bs);
 	if (mtk_jpeg_set_dec_dst(ctx, &jpeg_src_buf->dec_param, &dst_buf->vb2_buf, &fb))
 		goto dec_end;
@@ -957,6 +941,7 @@ static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv)
 	v4l2_m2m_buf_done(src_buf, buf_state);
 	v4l2_m2m_buf_done(dst_buf, buf_state);
 	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
+	pm_runtime_put(ctx->jpeg->dev);
 	return IRQ_HANDLED;
 }
 
Signature · 1 line
2.18.0
Reviewed-by on v9 5/18
#8 lore raw

Re: [PATCH RESEND v9 05/18] media: platform: Improve power on and power off flow

Hi Xia,

Quoted 77 lines · Xia Jiang

Call pm_runtime_get_sync() before starting a frame and then pm_runtime_put() after completing it. This can save power for the time between processing two frames.

---
v9: use pm_runtime_put() to replace pm_runtime_put_sync()

---
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 27 +++++--------------
 1 file changed, 6 insertions(+), 21 deletions(-)
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -21
index 12609ca46fd9..fb624385969e 100644
@@ -710,23 +710,6 @@ static struct vb2_v4l2_buffer *mtk_jpeg_buf_remove(struct mtk_jpeg_ctx *ctx,
 		return v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
 }
 
-static int mtk_jpeg_start_streaming(struct vb2_queue *q, unsigned int count)
-{
-	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(q);
-	struct vb2_v4l2_buffer *vb;
-	int ret = 0;
-
-	ret = pm_runtime_get_sync(ctx->jpeg->dev);
-	if (ret < 0)
-		goto err;
-
-	return 0;
-err:
-	while ((vb = mtk_jpeg_buf_remove(ctx, q->type)))
-		v4l2_m2m_buf_done(vb, VB2_BUF_STATE_QUEUED);
-	return ret;
-}
-
 static void mtk_jpeg_stop_streaming(struct vb2_queue *q)
 {
 	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(q);
@@ -751,8 +734,6 @@ static void mtk_jpeg_stop_streaming(struct vb2_queue *q)
 
 	while ((vb = mtk_jpeg_buf_remove(ctx, q->type)))
 		v4l2_m2m_buf_done(vb, VB2_BUF_STATE_ERROR);
-
-	pm_runtime_put_sync(ctx->jpeg->dev);
 }
 
 static const struct vb2_ops mtk_jpeg_qops = {
@@ -761,7 +742,6 @@ static const struct vb2_ops mtk_jpeg_qops = {
 	.buf_queue          = mtk_jpeg_buf_queue,
 	.wait_prepare       = vb2_ops_wait_prepare,
 	.wait_finish        = vb2_ops_wait_finish,
-	.start_streaming    = mtk_jpeg_start_streaming,
 	.stop_streaming     = mtk_jpeg_stop_streaming,
 };
 
@@ -812,7 +792,7 @@ static void mtk_jpeg_device_run(void *priv)
 	struct mtk_jpeg_src_buf *jpeg_src_buf;
 	struct mtk_jpeg_bs bs;
 	struct mtk_jpeg_fb fb;
-	int i;
+	int i, ret;
 
 	src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
 	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
@@ -832,6 +812,10 @@ static void mtk_jpeg_device_run(void *priv)
 		return;
 	}
 
+	ret = pm_runtime_get_sync(jpeg->dev);
+	if (ret < 0)
+		goto dec_end;
+
 	mtk_jpeg_set_dec_src(ctx, &src_buf->vb2_buf, &bs);
 	if (mtk_jpeg_set_dec_dst(ctx, &jpeg_src_buf->dec_param, &dst_buf->vb2_buf, &fb))
 		goto dec_end;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 944

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -0
@@ -957,6 +941,7 @@ static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv)
 	v4l2_m2m_buf_done(src_buf, buf_state);
 	v4l2_m2m_buf_done(dst_buf, buf_state);
 	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
+	pm_runtime_put(ctx->jpeg->dev);

This patch itself is correct and feel free to add my

However, it looks like there might be one more problem with this driver. What happens if the hardware locks up? The driver doesn't seem to take any measures to detect that and recover the system.

If you take a look at other drivers, e.g. the MTK FD driver, there is a delayed work scheduled before starting the hardware and canceled in the interrupt handler. If the delayed work is executed, it resets the hardware and reports the failure to V4L2, so that the execution can continue from next frames.

This should be fixed in a separate patch, could be outside of this series.

Best regards,
Tomasz

author
#9 lore raw

PATCH RESEND v9 6/18 media: platform: Delete the resetting hardware flow in the system PM ops

In reply to Xia Jiang

Delete the resetting hardware flow in suspend and resume function because that resetting operation will be done in device_run().

---
v9: new patch

---
 drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c | 2 --
 1 file changed, 2 deletions(-)
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +0 -2
index fb624385969e..7f74597262fc 100644
@@ -1191,7 +1191,6 @@ static __maybe_unused int mtk_jpeg_pm_suspend(struct device *dev)
 {
 	struct mtk_jpeg_dev *jpeg = dev_get_drvdata(dev);
 
-	mtk_jpeg_dec_reset(jpeg->dec_reg_base);
 	mtk_jpeg_clk_off(jpeg);
 
 	return 0;
@@ -1202,7 +1201,6 @@ static __maybe_unused int mtk_jpeg_pm_resume(struct device *dev)
 	struct mtk_jpeg_dev *jpeg = dev_get_drvdata(dev);
 
 	mtk_jpeg_clk_on(jpeg);
-	mtk_jpeg_dec_reset(jpeg->dec_reg_base);
 
 	return 0;
 }
Signature · 1 line
2.18.0
Reviewed-by on v9 6/18
#10 lore raw

Re: [PATCH RESEND v9 06/18] media: platform: Delete the resetting hardware flow in the system PM ops

Hi Xia,

Quoted 10 lines · Xia Jiang

Delete the resetting hardware flow in suspend and resume function because that resetting operation will be done in device_run().

---
v9: new patch

---
 drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c | 2 --
 1 file changed, 2 deletions(-)

Best regards,
Tomasz

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

author
#11 lore raw

PATCH RESEND v9 7/18 media: platform: Improve the implementation of the system PM ops

In reply to Xia Jiang

Add v4l2_m2m_suspend() function call in mtk_jpeg_suspend() to make sure that the current frame is processed completely before suspend. Add v4l2_m2m_resume() function call in mtk_jpeg_resume() to unblock the driver from scheduling next frame.

---
v9: use v4l2_m2m_suspend() and v4l2_m2m_resume() to improve the
implemention of the system PM ops

---
 drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c | 6 ++++++
 1 file changed, 6 insertions(+)
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
index 7f74597262fc..49bdbf1c435f 100644
@@ -1208,10 +1208,13 @@ static __maybe_unused int mtk_jpeg_pm_resume(struct device *dev)
 static __maybe_unused int mtk_jpeg_suspend(struct device *dev)
 {
 	int ret;
+	struct mtk_jpeg_dev *jpeg = dev_get_drvdata(dev);
 
 	if (pm_runtime_suspended(dev))
 		return 0;
 
+	v4l2_m2m_suspend(jpeg->m2m_dev);
+
 	ret = mtk_jpeg_pm_suspend(dev);
 	return ret;
 }
@@ -1219,12 +1222,15 @@ static __maybe_unused int mtk_jpeg_suspend(struct device *dev)
 static __maybe_unused int mtk_jpeg_resume(struct device *dev)
 {
 	int ret;
+	struct mtk_jpeg_dev *jpeg = dev_get_drvdata(dev);
 
 	if (pm_runtime_suspended(dev))
 		return 0;
 
 	ret = mtk_jpeg_pm_resume(dev);
 
+	v4l2_m2m_resume(jpeg->m2m_dev);
+
 	return ret;
 }
 
Signature · 1 line
2.18.0
#12 lore raw

Re: [PATCH RESEND v9 07/18] media: platform: Improve the implementation of the system PM ops

Hi Xia,

Quoted 5 lines · Xia Jiang

Add v4l2_m2m_suspend() function call in mtk_jpeg_suspend() to make sure that the current frame is processed completely before suspend. Add v4l2_m2m_resume() function call in mtk_jpeg_resume() to unblock the driver from scheduling next frame.

Quoted 8 lines

---
v9: use v4l2_m2m_suspend() and v4l2_m2m_resume() to improve the
implemention of the system PM ops

---
 drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c | 6 ++++++
 1 file changed, 6 insertions(+)

Thank you for the patch. Please see my comments inline.

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +0 -0
index 7f74597262fc..49bdbf1c435f 100644
@@ -1208,10 +1208,13 @@ static __maybe_unused int mtk_jpeg_pm_resume(struct device *dev)
 static __maybe_unused int mtk_jpeg_suspend(struct device *dev)
 {
 	int ret;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static __maybe_unused int mtk_jpeg_pm_resume(struct device *dev) · line 1220

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -0
+	struct mtk_jpeg_dev *jpeg = dev_get_drvdata(dev);
 
 	if (pm_runtime_suspended(dev))
 		return 0;
 
+	v4l2_m2m_suspend(jpeg->m2m_dev);
+
 	ret = mtk_jpeg_pm_suspend(dev);
 	return ret;
 }

This could be simplified into:

{
	struct mtk_jpeg_dev *jpeg = dev_get_drvdata(dev);

	v4l2_m2m_suspend(jpeg->m2m_dev);
	return pm_runtime_force_suspend(dev);
}
Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -0
@@ -1219,12 +1222,15 @@ static __maybe_unused int mtk_jpeg_suspend(struct device *dev)
 static __maybe_unused int mtk_jpeg_resume(struct device *dev)
 {
 	int ret;
+	struct mtk_jpeg_dev *jpeg = dev_get_drvdata(dev);

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static __maybe_unused int mtk_jpeg_suspend(struct device *dev) · line 1235

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -0
 
 	if (pm_runtime_suspended(dev))
 		return 0;
 
 	ret = mtk_jpeg_pm_resume(dev);
 
+	v4l2_m2m_resume(jpeg->m2m_dev);
+
 	return ret;
 }

Similarly here:

{
	struct mtk_jpeg_dev *jpeg = dev_get_drvdata(dev);
	int ret;

	ret = pm_runtime_force_resume(dev);
	if (ret < 0)
		return ret;

	v4l2_m2m_resume(jpeg->m2m_dev);
}

The pm_runtime_force_*() helpers will make sure that the right runtime PM state is restored.

Best regards,
Tomasz

author
#13 lore raw

PATCH RESEND v9 8/18 media: platform: Cancel the last frame handling flow

In reply to Xia Jiang

There is no need to queue an empty buffer for signaling a last frame, because all frames are separate from each other in JPEG.

---
v9: new patch

---
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 21 +------------------
 1 file changed, 1 insertion(+), 20 deletions(-)
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -20
index 49bdbf1c435f..bb4ebce881ee 100644
@@ -55,15 +55,9 @@ static struct mtk_jpeg_fmt mtk_jpeg_formats[] = {
 
 #define MTK_JPEG_NUM_FORMATS ARRAY_SIZE(mtk_jpeg_formats)
 
-enum {
-	MTK_JPEG_BUF_FLAGS_INIT			= 0,
-	MTK_JPEG_BUF_FLAGS_LAST_FRAME		= 1,
-};
-
 struct mtk_jpeg_src_buf {
 	struct vb2_v4l2_buffer b;
 	struct list_head list;
-	int flags;
 	struct mtk_jpeg_dec_param dec_param;
 };
 
@@ -520,8 +514,6 @@ static int mtk_jpeg_qbuf(struct file *file, void *priv, struct v4l2_buffer *buf)
 
 	vb = vb2_get_buffer(vq, buf->index);
 	jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(vb);
-	jpeg_src_buf->flags = (buf->m.planes[0].bytesused == 0) ?
-		MTK_JPEG_BUF_FLAGS_LAST_FRAME : MTK_JPEG_BUF_FLAGS_INIT;
 end:
 	return v4l2_m2m_qbuf(file, fh->m2m_ctx, buf);
 }
@@ -676,10 +668,6 @@ static void mtk_jpeg_buf_queue(struct vb2_buffer *vb)
 	param = &jpeg_src_buf->dec_param;
 	memset(param, 0, sizeof(*param));
 
-	if (jpeg_src_buf->flags & MTK_JPEG_BUF_FLAGS_LAST_FRAME) {
-		v4l2_dbg(1, debug, &jpeg->v4l2_dev, "Got eos\n");
-		goto end;
-	}
 	header_valid = mtk_jpeg_parse(param, (u8 *)vb2_plane_vaddr(vb, 0),
 				      vb2_get_plane_payload(vb, 0));
 	if (!header_valid) {
@@ -792,19 +780,12 @@ static void mtk_jpeg_device_run(void *priv)
 	struct mtk_jpeg_src_buf *jpeg_src_buf;
 	struct mtk_jpeg_bs bs;
 	struct mtk_jpeg_fb fb;
-	int i, ret;
+	int ret;
 
 	src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
 	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
 	jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(&src_buf->vb2_buf);
 
-	if (jpeg_src_buf->flags & MTK_JPEG_BUF_FLAGS_LAST_FRAME) {
-		for (i = 0; i < dst_buf->vb2_buf.num_planes; i++)
-			vb2_set_plane_payload(&dst_buf->vb2_buf, i, 0);
-		buf_state = VB2_BUF_STATE_DONE;
-		goto dec_end;
-	}
-
 	if (mtk_jpeg_check_resolution_change(ctx, &jpeg_src_buf->dec_param)) {
 		mtk_jpeg_queue_src_chg_event(ctx);
 		ctx->state = MTK_JPEG_SOURCE_CHANGE;
Signature · 1 line
2.18.0
Reviewed-by on v9 8/18
#14 lore raw

Re: [PATCH RESEND v9 08/18] media: platform: Cancel the last frame handling flow

Hi Xia,

Quoted 10 lines · Xia Jiang

There is no need to queue an empty buffer for signaling a last frame, because all frames are separate from each other in JPEG.

---
v9: new patch

---
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 21 +------------------
 1 file changed, 1 insertion(+), 20 deletions(-)

Best regards,
Tomasz

author
#15 lore raw

PATCH RESEND v9 9/18 media: platform: Delete zeroing the reserved fields

In reply to Xia Jiang

Delete zeroing the reserved fields because that the core already does it.

---
v9: new patch

---
 drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c | 5 -----
 1 file changed, 5 deletions(-)
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +0 -5
index bb4ebce881ee..bd1cc58324c6 100644
@@ -198,7 +198,6 @@ static int mtk_jpeg_try_fmt_mplane(struct v4l2_format *f,
 	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
 	int i;
 
-	memset(pix_mp->reserved, 0, sizeof(pix_mp->reserved));
 	pix_mp->field = V4L2_FIELD_NONE;
 
 	if (ctx->state != MTK_JPEG_INIT) {
@@ -217,7 +216,6 @@ static int mtk_jpeg_try_fmt_mplane(struct v4l2_format *f,
 					   &pix_mp->height, MTK_JPEG_MIN_HEIGHT,
 					   MTK_JPEG_MAX_HEIGHT, 0);
 
-		memset(pfmt->reserved, 0, sizeof(pfmt->reserved));
 		pfmt->bytesperline = 0;
 		/* Source size must be aligned to 128 */
 		pfmt->sizeimage = mtk_jpeg_align(pfmt->sizeimage, 128);
@@ -237,7 +235,6 @@ static int mtk_jpeg_try_fmt_mplane(struct v4l2_format *f,
 		u32 stride = pix_mp->width * fmt->h_sample[i] / 4;
 		u32 h = pix_mp->height * fmt->v_sample[i] / 4;
 
-		memset(pfmt->reserved, 0, sizeof(pfmt->reserved));
 		pfmt->bytesperline = stride;
 		pfmt->sizeimage = stride * h;
 	}
@@ -270,7 +267,6 @@ static int mtk_jpeg_g_fmt_vid_mplane(struct file *file, void *priv,
 
 	q_data = mtk_jpeg_get_q_data(ctx, f->type);
 
-	memset(pix_mp->reserved, 0, sizeof(pix_mp->reserved));
 	pix_mp->width = q_data->w;
 	pix_mp->height = q_data->h;
 	pix_mp->field = V4L2_FIELD_NONE;
@@ -294,7 +290,6 @@ static int mtk_jpeg_g_fmt_vid_mplane(struct file *file, void *priv,
 
 		pfmt->bytesperline = q_data->bytesperline[i];
 		pfmt->sizeimage = q_data->sizeimage[i];
-		memset(pfmt->reserved, 0, sizeof(pfmt->reserved));
 
 		v4l2_dbg(1, debug, &jpeg->v4l2_dev,
 			 "plane[%d] bpl=%u, size=%u\n",
Signature · 1 line
2.18.0
Reviewed-by on v9 9/18
#16 lore raw

Re: [PATCH RESEND v9 09/18] media: platform: Delete zeroing the reserved fields

Hi Xia,

Quoted 10 lines · Xia Jiang

Delete zeroing the reserved fields because that the core already does it.

---
v9: new patch

---
 drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c | 5 -----
 1 file changed, 5 deletions(-)

Best regards,
Tomasz

author
#17 lore raw

PATCH RESEND v9 10/18 media: platform: Stylistic changes for improving code quality

In reply to Xia Jiang

Change register offset hex numerals from uppercase to lowercase. Change data type of max/min width/height from integer to unsigned integer.

---
v9: move changing data type of max/min width/height to this patch

---
 .../media/platform/mtk-jpeg/mtk_jpeg_core.h    |  8 ++++----
 drivers/media/platform/mtk-jpeg/mtk_jpeg_reg.h | 18 +++++++++---------
 2 files changed, 13 insertions(+), 13 deletions(-)
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +4 -4
index 999bd1427809..28e9b30ad5c3 100644
@@ -21,10 +21,10 @@
 #define MTK_JPEG_FMT_TYPE_OUTPUT	1
 #define MTK_JPEG_FMT_TYPE_CAPTURE	2
 
-#define MTK_JPEG_MIN_WIDTH	32
-#define MTK_JPEG_MIN_HEIGHT	32
-#define MTK_JPEG_MAX_WIDTH	8192
-#define MTK_JPEG_MAX_HEIGHT	8192
+#define MTK_JPEG_MIN_WIDTH	32U
+#define MTK_JPEG_MIN_HEIGHT	32U
+#define MTK_JPEG_MAX_WIDTH	8192U
+#define MTK_JPEG_MAX_HEIGHT	8192U
 
 #define MTK_JPEG_DEFAULT_SIZEIMAGE	(1 * 1024 * 1024)
 
drivers/media/platform/mtk-jpeg/mtk_jpeg_reg.h +9 -9
index 94db04e9cdb6..2945da842dfa 100644
@@ -20,29 +20,29 @@
 #define BIT_INQST_MASK_ALLIRQ		0x37
 
 #define JPGDEC_REG_RESET		0x0090
-#define JPGDEC_REG_BRZ_FACTOR		0x00F8
-#define JPGDEC_REG_DU_NUM		0x00FC
+#define JPGDEC_REG_BRZ_FACTOR		0x00f8
+#define JPGDEC_REG_DU_NUM		0x00fc
 #define JPGDEC_REG_DEST_ADDR0_Y		0x0140
 #define JPGDEC_REG_DEST_ADDR0_U		0x0144
 #define JPGDEC_REG_DEST_ADDR0_V		0x0148
-#define JPGDEC_REG_DEST_ADDR1_Y		0x014C
+#define JPGDEC_REG_DEST_ADDR1_Y		0x014c
 #define JPGDEC_REG_DEST_ADDR1_U		0x0150
 #define JPGDEC_REG_DEST_ADDR1_V		0x0154
 #define JPGDEC_REG_STRIDE_Y		0x0158
-#define JPGDEC_REG_STRIDE_UV		0x015C
+#define JPGDEC_REG_STRIDE_UV		0x015c
 #define JPGDEC_REG_IMG_STRIDE_Y		0x0160
 #define JPGDEC_REG_IMG_STRIDE_UV	0x0164
-#define JPGDEC_REG_WDMA_CTRL		0x016C
+#define JPGDEC_REG_WDMA_CTRL		0x016c
 #define JPGDEC_REG_PAUSE_MCU_NUM	0x0170
-#define JPGDEC_REG_OPERATION_MODE	0x017C
+#define JPGDEC_REG_OPERATION_MODE	0x017c
 #define JPGDEC_REG_FILE_ADDR		0x0200
-#define JPGDEC_REG_COMP_ID		0x020C
+#define JPGDEC_REG_COMP_ID		0x020c
 #define JPGDEC_REG_TOTAL_MCU_NUM	0x0210
 #define JPGDEC_REG_COMP0_DATA_UNIT_NUM	0x0224
-#define JPGDEC_REG_DU_CTRL		0x023C
+#define JPGDEC_REG_DU_CTRL		0x023c
 #define JPGDEC_REG_TRIG			0x0240
 #define JPGDEC_REG_FILE_BRP		0x0248
-#define JPGDEC_REG_FILE_TOTAL_SIZE	0x024C
+#define JPGDEC_REG_FILE_TOTAL_SIZE	0x024c
 #define JPGDEC_REG_QT_ID		0x0270
 #define JPGDEC_REG_INTERRUPT_STATUS	0x0274
 #define JPGDEC_REG_STATUS		0x0278
Signature · 1 line
2.18.0
Reviewed-by on v9 10/18
#18 lore raw

Re: [PATCH RESEND v9 10/18] media: platform: Stylistic changes for improving code quality

Hi Xia,

Quoted 4 lines · Xia Jiang

Change register offset hex numerals from uppercase to lowercase. Change data type of max/min width/height from integer to unsigned integer.

Quoted 8 lines

---
v9: move changing data type of max/min width/height to this patch

---
 .../media/platform/mtk-jpeg/mtk_jpeg_core.h    |  8 ++++----
 drivers/media/platform/mtk-jpeg/mtk_jpeg_reg.h | 18 +++++++++---------
 2 files changed, 13 insertions(+), 13 deletions(-)

Best regards,
Tomasz

author
#19 lore raw

PATCH RESEND v9 11/18 media: platform: Use generic rounding helpers

In reply to Xia Jiang

Use clamp() to replace mtk_jpeg_bound_align_image() and round() to replace mtk_jpeg_align().

---
v9: change the patch title description

---
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 41 +++++--------------
 drivers/media/platform/mtk-jpeg/mtk_jpeg_hw.c |  8 ++--
 drivers/media/platform/mtk-jpeg/mtk_jpeg_hw.h |  5 ---
 3 files changed, 15 insertions(+), 39 deletions(-)
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +11 -30
index bd1cc58324c6..c9c0357b2d6c 100644
@@ -151,25 +151,6 @@ static struct mtk_jpeg_fmt *mtk_jpeg_find_format(struct mtk_jpeg_ctx *ctx,
 	return NULL;
 }
 
-static void mtk_jpeg_bound_align_image(u32 *w, unsigned int wmin,
-				       unsigned int wmax, unsigned int walign,
-				       u32 *h, unsigned int hmin,
-				       unsigned int hmax, unsigned int halign)
-{
-	int width, height, w_step, h_step;
-
-	width = *w;
-	height = *h;
-	w_step = 1 << walign;
-	h_step = 1 << halign;
-
-	v4l_bound_align_image(w, wmin, wmax, walign, h, hmin, hmax, halign, 0);
-	if (*w < width && (*w + w_step) <= wmax)
-		*w += w_step;
-	if (*h < height && (*h + h_step) <= hmax)
-		*h += h_step;
-}
-
 static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx,
 				       struct v4l2_format *f)
 {
@@ -211,24 +192,24 @@ static int mtk_jpeg_try_fmt_mplane(struct v4l2_format *f,
 	if (q_type == MTK_JPEG_FMT_TYPE_OUTPUT) {
 		struct v4l2_plane_pix_format *pfmt = &pix_mp->plane_fmt[0];
 
-		mtk_jpeg_bound_align_image(&pix_mp->width, MTK_JPEG_MIN_WIDTH,
-					   MTK_JPEG_MAX_WIDTH, 0,
-					   &pix_mp->height, MTK_JPEG_MIN_HEIGHT,
-					   MTK_JPEG_MAX_HEIGHT, 0);
+		pix_mp->height = clamp(pix_mp->height, MTK_JPEG_MIN_HEIGHT,
+				       MTK_JPEG_MAX_HEIGHT);
+		pix_mp->width = clamp(pix_mp->width, MTK_JPEG_MIN_WIDTH,
+				      MTK_JPEG_MAX_WIDTH);
 
 		pfmt->bytesperline = 0;
 		/* Source size must be aligned to 128 */
-		pfmt->sizeimage = mtk_jpeg_align(pfmt->sizeimage, 128);
+		pfmt->sizeimage = round_up(pfmt->sizeimage, 128);
 		if (pfmt->sizeimage == 0)
 			pfmt->sizeimage = MTK_JPEG_DEFAULT_SIZEIMAGE;
 		goto end;
 	}
 
 	/* type is MTK_JPEG_FMT_TYPE_CAPTURE */
-	mtk_jpeg_bound_align_image(&pix_mp->width, MTK_JPEG_MIN_WIDTH,
-				   MTK_JPEG_MAX_WIDTH, fmt->h_align,
-				   &pix_mp->height, MTK_JPEG_MIN_HEIGHT,
-				   MTK_JPEG_MAX_HEIGHT, fmt->v_align);
+	pix_mp->height = clamp(round_up(pix_mp->height, fmt->v_align),
+			       MTK_JPEG_MIN_HEIGHT, MTK_JPEG_MAX_HEIGHT);
+	pix_mp->width = clamp(round_up(pix_mp->width, fmt->h_align),
+			      MTK_JPEG_MIN_WIDTH, MTK_JPEG_MAX_WIDTH);
 
 	for (i = 0; i < fmt->colplanes; i++) {
 		struct v4l2_plane_pix_format *pfmt = &pix_mp->plane_fmt[i];
@@ -734,8 +715,8 @@ static void mtk_jpeg_set_dec_src(struct mtk_jpeg_ctx *ctx,
 {
 	bs->str_addr = vb2_dma_contig_plane_dma_addr(src_buf, 0);
 	bs->end_addr = bs->str_addr +
-			 mtk_jpeg_align(vb2_get_plane_payload(src_buf, 0), 16);
-	bs->size = mtk_jpeg_align(vb2_plane_size(src_buf, 0), 128);
+		       round_up(vb2_get_plane_payload(src_buf, 0), 16);
+	bs->size = round_up(vb2_plane_size(src_buf, 0), 128);
 }
 
 static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx,
drivers/media/platform/mtk-jpeg/mtk_jpeg_hw.c +4 -4
index ddf0dfa78e20..68abcfd7494d 100644
@@ -153,10 +153,10 @@ static int mtk_jpeg_calc_dst_size(struct mtk_jpeg_dec_param *param)
 				param->sampling_w[i];
 		/* output format is 420/422 */
 		param->comp_w[i] = padding_w >> brz_w[i];
-		param->comp_w[i] = mtk_jpeg_align(param->comp_w[i],
-						  MTK_JPEG_DCTSIZE);
-		param->img_stride[i] = i ? mtk_jpeg_align(param->comp_w[i], 16)
-					: mtk_jpeg_align(param->comp_w[i], 32);
+		param->comp_w[i] = round_up(param->comp_w[i],
+					    MTK_JPEG_DCTSIZE);
+		param->img_stride[i] = i ? round_up(param->comp_w[i], 16)
+					: round_up(param->comp_w[i], 32);
 		ds_row_h[i] = (MTK_JPEG_DCTSIZE * param->sampling_h[i]);
 	}
 	param->dec_w = param->img_stride[0];
drivers/media/platform/mtk-jpeg/mtk_jpeg_hw.h +0 -5
index 9c6584eaad99..7b0687f8f4b6 100644
@@ -54,11 +54,6 @@ struct mtk_jpeg_dec_param {
 	u8 uv_brz_w;
 };
 
-static inline u32 mtk_jpeg_align(u32 val, u32 align)
-{
-	return (val + align - 1) & ~(align - 1);
-}
-
 struct mtk_jpeg_bs {
 	dma_addr_t	str_addr;
 	dma_addr_t	end_addr;
Signature · 1 line
2.18.0
author
#20 lore raw

PATCH RESEND v9 12/18 media: platform: Change MTK_JPEG_COMP_MAX macro definition location

In reply to Xia Jiang

Move MTK_JPEG_COMP_MAX definition to mtk_jpeg_core.h file, because it is used by mtk_jpeg_core.c file.

---
v9: no changes

---
 drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h | 2 ++
 drivers/media/platform/mtk-jpeg/mtk_jpeg_reg.h  | 1 -
 2 files changed, 2 insertions(+), 1 deletion(-)
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +2 -0
index 28e9b30ad5c3..64a731261214 100644
@@ -15,6 +15,8 @@
 
 #define MTK_JPEG_NAME		"mtk-jpeg"
 
+#define MTK_JPEG_COMP_MAX		3
+
 #define MTK_JPEG_FMT_FLAG_DEC_OUTPUT	BIT(0)
 #define MTK_JPEG_FMT_FLAG_DEC_CAPTURE	BIT(1)
 
drivers/media/platform/mtk-jpeg/mtk_jpeg_reg.h +0 -1
index 2945da842dfa..21ec8f96797f 100644
@@ -8,7 +8,6 @@
 #ifndef _MTK_JPEG_REG_H
 #define _MTK_JPEG_REG_H
 
-#define MTK_JPEG_COMP_MAX		3
 #define MTK_JPEG_BLOCK_MAX		10
 #define MTK_JPEG_DCTSIZE		8
 
Signature · 1 line
2.18.0
author
#21 lore raw

PATCH RESEND v9 13/18 media: platform: Delete redundant code and add annotation for an enum

In reply to Xia Jiang

Delete unused member variables annotation.
Delete unused variable definition.
Delete redundant log print, because V4L2 debug logs already print it. Add annotation for enum mtk_jpeg_ctx_state.

---
v9: add annotation for enum mtk_jpeg_ctx_state

---
 drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c | 15 ++-------------
 drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h |  8 ++++++--
 2 files changed, 8 insertions(+), 15 deletions(-)
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -13
index c9c0357b2d6c..6c82134d6b3d 100644
@@ -176,14 +176,13 @@ static int mtk_jpeg_try_fmt_mplane(struct v4l2_format *f,
 				   struct mtk_jpeg_ctx *ctx, int q_type)
 {
 	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
-	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
 	int i;
 
 	pix_mp->field = V4L2_FIELD_NONE;
 
 	if (ctx->state != MTK_JPEG_INIT) {
 		mtk_jpeg_adjust_fmt_mplane(ctx, f);
-		goto end;
+		return 0;
 	}
 
 	pix_mp->num_planes = fmt->colplanes;
@@ -202,7 +201,7 @@ static int mtk_jpeg_try_fmt_mplane(struct v4l2_format *f,
 		pfmt->sizeimage = round_up(pfmt->sizeimage, 128);
 		if (pfmt->sizeimage == 0)
 			pfmt->sizeimage = MTK_JPEG_DEFAULT_SIZEIMAGE;
-		goto end;
+		return 0;
 	}
 
 	/* type is MTK_JPEG_FMT_TYPE_CAPTURE */
@@ -219,16 +218,6 @@ static int mtk_jpeg_try_fmt_mplane(struct v4l2_format *f,
 		pfmt->bytesperline = stride;
 		pfmt->sizeimage = stride * h;
 	}
-end:
-	v4l2_dbg(2, debug, &jpeg->v4l2_dev, "wxh:%ux%u\n",
-		 pix_mp->width, pix_mp->height);
-	for (i = 0; i < pix_mp->num_planes; i++) {
-		v4l2_dbg(2, debug, &jpeg->v4l2_dev,
-			 "plane[%d] bpl=%u, size=%u\n",
-			 i,
-			 pix_mp->plane_fmt[i].bytesperline,
-			 pix_mp->plane_fmt[i].sizeimage);
-	}
 	return 0;
 }
 
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +6 -2
index 64a731261214..5fcdf6950782 100644
@@ -30,6 +30,12 @@
 
 #define MTK_JPEG_DEFAULT_SIZEIMAGE	(1 * 1024 * 1024)
 
+/**
+ * enum mtk_jpeg_ctx_state - states of the context state machine
+ * @MTK_JPEG_INIT:		current state is initialized
+ * @MTK_JPEG_RUNNING:		current state is running
+ * @MTK_JPEG_SOURCE_CHANGE:	current state is source resolution change
+ */
 enum mtk_jpeg_ctx_state {
 	MTK_JPEG_INIT = 0,
 	MTK_JPEG_RUNNING,
@@ -109,9 +115,7 @@ struct mtk_jpeg_q_data {
  * @out_q:		source (output) queue information
  * @cap_q:		destination (capture) queue queue information
  * @fh:			V4L2 file handle
- * @dec_param		parameters for HW decoding
  * @state:		state of the context
- * @header_valid:	set if header has been parsed and valid
  * @colorspace: enum v4l2_colorspace; supplemental to pixelformat
  * @ycbcr_enc: enum v4l2_ycbcr_encoding, Y'CbCr encoding
  * @quantization: enum v4l2_quantization, colorspace quantization
Signature · 1 line
2.18.0
Reviewed-by on v9 13/18
#22 lore raw

Re: [PATCH RESEND v9 13/18] media: platform: Delete redundant code and add annotation for an enum

Hi Xia,

Quoted 5 lines · Xia Jiang

Delete unused member variables annotation.
Delete unused variable definition.
Delete redundant log print, because V4L2 debug logs already print it. Add annotation for enum mtk_jpeg_ctx_state.

Quoted 8 lines

---
v9: add annotation for enum mtk_jpeg_ctx_state

---
 drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c | 15 ++-------------
 drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h |  8 ++++++--
 2 files changed, 8 insertions(+), 15 deletions(-)

Best regards,
Tomasz

author
#23 lore raw

PATCH RESEND v9 14/18 media: dt-bindings: Add jpeg enc device tree node document

In reply to Xia Jiang

Add jpeg enc device tree node document.

---
v9: no changes

---
 .../bindings/media/mediatek-jpeg-encoder.txt  | 37 +++++++++++++++++++
 1 file changed, 37 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek-jpeg-encoder.txt
Documentation/devicetree/bindings/media/mediatek-jpeg-encoder.txt new file +37 -0
new file mode 100644
index 000000000000..fa8da699493b
@@ -0,0 +1,37 @@
+* MediaTek JPEG Encoder
+
+MediaTek JPEG Encoder is the JPEG encode hardware present in MediaTek SoCs
+
+Required properties:
+- compatible : should be one of:
+               "mediatek,mt2701-jpgenc"
+               ...
+               followed by "mediatek,mtk-jpgenc"
+- reg : physical base address of the JPEG encoder registers and length of
+  memory mapped region.
+- interrupts : interrupt number to the interrupt controller.
+- clocks: device clocks, see
+  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
+- clock-names: must contain "jpgenc". It is the clock of JPEG encoder.
+- power-domains: a phandle to the power domain, see
+  Documentation/devicetree/bindings/power/power_domain.txt for details.
+- mediatek,larb: must contain the local arbiters in the current SoCs, see
+  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
+  for details.
+- iommus: should point to the respective IOMMU block with master port as
+  argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
+  for details.
+
+Example:
+	jpegenc: jpegenc@1500a000 {
+		compatible = "mediatek,mt2701-jpgenc",
+			     "mediatek,mtk-jpgenc";
+		reg = <0 0x1500a000 0 0x1000>;
+		interrupts = <GIC_SPI 141 IRQ_TYPE_LEVEL_LOW>;
+		clocks =  <&imgsys CLK_IMG_VENC>;
+		clock-names = "jpgenc";
+		power-domains = <&scpsys MT2701_POWER_DOMAIN_ISP>;
+		mediatek,larb = <&larb2>;
+		iommus = <&iommu MT2701_M4U_PORT_JPGENC_RDMA>,
+			 <&iommu MT2701_M4U_PORT_JPGENC_BSDMA>;
+	};
Signature · 1 line
2.18.0
author
#24 lore raw

PATCH RESEND v9 15/18 arm: dts: mt2701: Add jpeg enc device tree node

In reply to Xia Jiang

Add jpeg enc device tree node.

---
v9: add "mt2701" in the title description

---
 arch/arm/boot/dts/mt2701.dtsi | 13 +++++++++++++
 1 file changed, 13 insertions(+)
arch/arm/boot/dts/mt2701.dtsi +13 -0
index 2c4ec82547ee..235bacc0e418 100644
@@ -568,6 +568,19 @@
 			 <&iommu MT2701_M4U_PORT_JPGDEC_BSDMA>;
 	};
 
+	jpegenc: jpegenc@1500a000 {
+		compatible = "mediatek,mt2701-jpgenc",
+			     "mediatek,mtk-jpgenc";
+		reg = <0 0x1500a000 0 0x1000>;
+		interrupts = <GIC_SPI 141 IRQ_TYPE_LEVEL_LOW>;
+		clocks =  <&imgsys CLK_IMG_VENC>;
+		clock-names = "jpgenc";
+		power-domains = <&scpsys MT2701_POWER_DOMAIN_ISP>;
+		mediatek,larb = <&larb2>;
+		iommus = <&iommu MT2701_M4U_PORT_JPGENC_RDMA>,
+			 <&iommu MT2701_M4U_PORT_JPGENC_BSDMA>;
+	};
+
 	vdecsys: syscon@16000000 {
 		compatible = "mediatek,mt2701-vdecsys", "syscon";
 		reg = <0 0x16000000 0 0x1000>;
Signature · 1 line
2.18.0
author
#25 lore raw

PATCH RESEND v9 16/18 media: platform: Rename jpeg dec file name

In reply to Xia Jiang

Rename the files which are for decode feature. This is preparing path since the jpeg enc patch will be added later.

---
v9: no changes

---
 drivers/media/platform/mtk-jpeg/Makefile                      | 2 +-
 drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c               | 4 ++--
 .../platform/mtk-jpeg/{mtk_jpeg_hw.c => mtk_jpeg_dec_hw.c}    | 2 +-
 .../platform/mtk-jpeg/{mtk_jpeg_hw.h => mtk_jpeg_dec_hw.h}    | 2 +-
 .../mtk-jpeg/{mtk_jpeg_parse.c => mtk_jpeg_dec_parse.c}       | 2 +-
 .../mtk-jpeg/{mtk_jpeg_parse.h => mtk_jpeg_dec_parse.h}       | 2 +-
 .../platform/mtk-jpeg/{mtk_jpeg_reg.h => mtk_jpeg_dec_reg.h}  | 0
 7 files changed, 7 insertions(+), 7 deletions(-)
 rename drivers/media/platform/mtk-jpeg/{mtk_jpeg_hw.c => mtk_jpeg_dec_hw.c} (99%)
 rename drivers/media/platform/mtk-jpeg/{mtk_jpeg_hw.h => mtk_jpeg_dec_hw.h} (98%)
 rename drivers/media/platform/mtk-jpeg/{mtk_jpeg_parse.c => mtk_jpeg_dec_parse.c} (98%)
 rename drivers/media/platform/mtk-jpeg/{mtk_jpeg_parse.h => mtk_jpeg_dec_parse.h} (92%)
 rename drivers/media/platform/mtk-jpeg/{mtk_jpeg_reg.h => mtk_jpeg_dec_reg.h} (100%)
drivers/media/platform/mtk-jpeg/Makefile +1 -1
index 92a4fc046bfe..48516dcf96e6 100644
@@ -1,3 +1,3 @@
 # SPDX-License-Identifier: GPL-2.0-only
-mtk_jpeg-objs := mtk_jpeg_core.o mtk_jpeg_hw.o mtk_jpeg_parse.o
+mtk_jpeg-objs := mtk_jpeg_core.o mtk_jpeg_dec_hw.o mtk_jpeg_dec_parse.o
 obj-$(CONFIG_VIDEO_MEDIATEK_JPEG) += mtk_jpeg.o
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -2
index 6c82134d6b3d..e0e522a502e1 100644
@@ -23,9 +23,9 @@
 #include <media/videobuf2-dma-contig.h>
 #include <soc/mediatek/smi.h>
 
-#include "mtk_jpeg_hw.h"
+#include "mtk_jpeg_dec_hw.h"
 #include "mtk_jpeg_core.h"
-#include "mtk_jpeg_parse.h"
+#include "mtk_jpeg_dec_parse.h"
 
 static struct mtk_jpeg_fmt mtk_jpeg_formats[] = {
 	{
drivers/media/platform/mtk-jpeg/mtk_jpeg_dec_hw.c renamed from drivers/media/platform/mtk-jpeg/mtk_jpeg_hw.c +1 -1
similarity index 99%
rename from drivers/media/platform/mtk-jpeg/mtk_jpeg_hw.c
rename to drivers/media/platform/mtk-jpeg/mtk_jpeg_dec_hw.c
index 68abcfd7494d..afbbfd5d02bc 100644
@@ -9,7 +9,7 @@
 #include <linux/kernel.h>
 #include <media/videobuf2-core.h>
 
-#include "mtk_jpeg_hw.h"
+#include "mtk_jpeg_dec_hw.h"
 
 #define MTK_JPEG_DUNUM_MASK(val)	(((val) - 1) & 0x3)
 
drivers/media/platform/mtk-jpeg/mtk_jpeg_dec_hw.h renamed from drivers/media/platform/mtk-jpeg/mtk_jpeg_hw.h +1 -1
similarity index 98%
rename from drivers/media/platform/mtk-jpeg/mtk_jpeg_hw.h
rename to drivers/media/platform/mtk-jpeg/mtk_jpeg_dec_hw.h
index 7b0687f8f4b6..1cc37dbfc8e7 100644
@@ -11,7 +11,7 @@
 #include <media/videobuf2-core.h>
 
 #include "mtk_jpeg_core.h"
-#include "mtk_jpeg_reg.h"
+#include "mtk_jpeg_dec_reg.h"
 
 enum {
 	MTK_JPEG_DEC_RESULT_EOF_DONE		= 0,
drivers/media/platform/mtk-jpeg/mtk_jpeg_dec_parse.c renamed from drivers/media/platform/mtk-jpeg/mtk_jpeg_parse.c +1 -1
similarity index 98%
rename from drivers/media/platform/mtk-jpeg/mtk_jpeg_parse.c
rename to drivers/media/platform/mtk-jpeg/mtk_jpeg_dec_parse.c
index f862d38f3af7..b95c45791c29 100644
@@ -8,7 +8,7 @@
 #include <linux/kernel.h>
 #include <linux/videodev2.h>
 
-#include "mtk_jpeg_parse.h"
+#include "mtk_jpeg_dec_parse.h"
 
 #define TEM	0x01
 #define SOF0	0xc0
drivers/media/platform/mtk-jpeg/mtk_jpeg_dec_parse.h renamed from drivers/media/platform/mtk-jpeg/mtk_jpeg_parse.h +1 -1
similarity index 92%
rename from drivers/media/platform/mtk-jpeg/mtk_jpeg_parse.h
rename to drivers/media/platform/mtk-jpeg/mtk_jpeg_dec_parse.h
index 0a48eeabaff2..2918f15811f8 100644
@@ -8,7 +8,7 @@
 #ifndef _MTK_JPEG_PARSE_H
 #define _MTK_JPEG_PARSE_H
 
-#include "mtk_jpeg_hw.h"
+#include "mtk_jpeg_dec_hw.h"
 
 bool mtk_jpeg_parse(struct mtk_jpeg_dec_param *param, u8 *src_addr_va,
 		    u32 src_size);
drivers/media/platform/mtk-jpeg/mtk_jpeg_dec_reg.h renamed from drivers/media/platform/mtk-jpeg/mtk_jpeg_reg.h +0 -0
similarity index 100%
rename from drivers/media/platform/mtk-jpeg/mtk_jpeg_reg.h
rename to drivers/media/platform/mtk-jpeg/mtk_jpeg_dec_reg.h
Signature · 1 line
2.18.0
author
#26 lore raw

PATCH RESEND v9 17/18 media: platform: Rename existing functions/defines/variables

In reply to Xia Jiang

Rename existing funcitons/defines/variables with a _dec prefix and without dec_ prefix to prepare for the addition of the jpeg encoder feature.

---
v9: new patch

---
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 200 +++++++++---------
 .../media/platform/mtk-jpeg/mtk_jpeg_core.h   |   8 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_dec_hw.h |   7 +-
 3 files changed, 109 insertions(+), 106 deletions(-)
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +101 -99
index e0e522a502e1..29b8b82c606c 100644
@@ -27,7 +27,7 @@
 #include "mtk_jpeg_core.h"
 #include "mtk_jpeg_dec_parse.h"
 
-static struct mtk_jpeg_fmt mtk_jpeg_formats[] = {
+static struct mtk_jpeg_fmt mtk_jpeg_dec_formats[] = {
 	{
 		.fourcc		= V4L2_PIX_FMT_JPEG,
 		.colplanes	= 1,
@@ -53,7 +53,7 @@ static struct mtk_jpeg_fmt mtk_jpeg_formats[] = {
 	},
 };
 
-#define MTK_JPEG_NUM_FORMATS ARRAY_SIZE(mtk_jpeg_formats)
+#define MTK_JPEG_DEC_NUM_FORMATS ARRAY_SIZE(mtk_jpeg_dec_formats)
 
 struct mtk_jpeg_src_buf {
 	struct vb2_v4l2_buffer b;
@@ -75,12 +75,12 @@ static inline struct mtk_jpeg_src_buf *mtk_jpeg_vb2_to_srcbuf(
 	return container_of(to_vb2_v4l2_buffer(vb), struct mtk_jpeg_src_buf, b);
 }
 
-static int mtk_jpeg_querycap(struct file *file, void *priv,
-			     struct v4l2_capability *cap)
+static int mtk_jpeg_dec_querycap(struct file *file, void *priv,
+				 struct v4l2_capability *cap)
 {
 	struct mtk_jpeg_dev *jpeg = video_drvdata(file);
 
-	strscpy(cap->driver, MTK_JPEG_NAME " decoder", sizeof(cap->driver));
+	strscpy(cap->driver, MTK_JPEG_NAME, sizeof(cap->driver));
 	strscpy(cap->card, MTK_JPEG_NAME " decoder", sizeof(cap->card));
 	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
 		 dev_name(jpeg->dev));
@@ -109,22 +109,23 @@ static int mtk_jpeg_enum_fmt(struct mtk_jpeg_fmt *mtk_jpeg_formats, int n,
 	return 0;
 }
 
-static int mtk_jpeg_enum_fmt_vid_cap(struct file *file, void *priv,
-				     struct v4l2_fmtdesc *f)
+static int mtk_jpeg_dec_enum_fmt_vid_cap(struct file *file, void *priv,
+					 struct v4l2_fmtdesc *f)
 {
-	return mtk_jpeg_enum_fmt(mtk_jpeg_formats, MTK_JPEG_NUM_FORMATS, f,
+	return mtk_jpeg_enum_fmt(mtk_jpeg_dec_formats,
+				 MTK_JPEG_DEC_NUM_FORMATS, f,
 				 MTK_JPEG_FMT_FLAG_DEC_CAPTURE);
 }
 
-static int mtk_jpeg_enum_fmt_vid_out(struct file *file, void *priv,
-				     struct v4l2_fmtdesc *f)
+static int mtk_jpeg_dec_enum_fmt_vid_out(struct file *file, void *priv,
+					 struct v4l2_fmtdesc *f)
 {
-	return mtk_jpeg_enum_fmt(mtk_jpeg_formats, MTK_JPEG_NUM_FORMATS, f,
-				 MTK_JPEG_FMT_FLAG_DEC_OUTPUT);
+	return mtk_jpeg_enum_fmt(mtk_jpeg_dec_formats, MTK_JPEG_DEC_NUM_FORMATS,
+				 f, MTK_JPEG_FMT_FLAG_DEC_OUTPUT);
 }
 
-static struct mtk_jpeg_q_data *mtk_jpeg_get_q_data(struct mtk_jpeg_ctx *ctx,
-						   enum v4l2_buf_type type)
+static struct mtk_jpeg_q_data *
+mtk_jpeg_get_q_data(struct mtk_jpeg_ctx *ctx, enum v4l2_buf_type type)
 {
 	if (V4L2_TYPE_IS_OUTPUT(type))
 		return &ctx->out_q;
@@ -141,8 +142,8 @@ static struct mtk_jpeg_fmt *mtk_jpeg_find_format(struct mtk_jpeg_ctx *ctx,
 		   MTK_JPEG_FMT_FLAG_DEC_OUTPUT :
 		   MTK_JPEG_FMT_FLAG_DEC_CAPTURE;
 
-	for (k = 0; k < MTK_JPEG_NUM_FORMATS; k++) {
-		struct mtk_jpeg_fmt *fmt = &mtk_jpeg_formats[k];
+	for (k = 0; k < MTK_JPEG_DEC_NUM_FORMATS; k++) {
+		struct mtk_jpeg_fmt *fmt = &mtk_jpeg_dec_formats[k];
 
 		if (fmt->fourcc == pixelformat && fmt->flags & fmt_flag)
 			return fmt;
@@ -270,8 +271,8 @@ static int mtk_jpeg_g_fmt_vid_mplane(struct file *file, void *priv,
 	return 0;
 }
 
-static int mtk_jpeg_try_fmt_vid_cap_mplane(struct file *file, void *priv,
-					   struct v4l2_format *f)
+static int mtk_jpeg_dec_try_fmt_vid_cap_mplane(struct file *file, void *priv,
+					       struct v4l2_format *f)
 {
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
 	struct mtk_jpeg_fmt *fmt;
@@ -291,8 +292,8 @@ static int mtk_jpeg_try_fmt_vid_cap_mplane(struct file *file, void *priv,
 	return mtk_jpeg_try_fmt_mplane(f, fmt, ctx, MTK_JPEG_FMT_TYPE_CAPTURE);
 }
 
-static int mtk_jpeg_try_fmt_vid_out_mplane(struct file *file, void *priv,
-					   struct v4l2_format *f)
+static int mtk_jpeg_dec_try_fmt_vid_out_mplane(struct file *file, void *priv,
+					       struct v4l2_format *f)
 {
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
 	struct mtk_jpeg_fmt *fmt;
@@ -364,24 +365,24 @@ static int mtk_jpeg_s_fmt_mplane(struct mtk_jpeg_ctx *ctx,
 	return 0;
 }
 
-static int mtk_jpeg_s_fmt_vid_out_mplane(struct file *file, void *priv,
-					 struct v4l2_format *f)
+static int mtk_jpeg_dec_s_fmt_vid_out_mplane(struct file *file, void *priv,
+					     struct v4l2_format *f)
 {
 	int ret;
 
-	ret = mtk_jpeg_try_fmt_vid_out_mplane(file, priv, f);
+	ret = mtk_jpeg_dec_try_fmt_vid_out_mplane(file, priv, f);
 	if (ret)
 		return ret;
 
 	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f);
 }
 
-static int mtk_jpeg_s_fmt_vid_cap_mplane(struct file *file, void *priv,
-					 struct v4l2_format *f)
+static int mtk_jpeg_dec_s_fmt_vid_cap_mplane(struct file *file, void *priv,
+					     struct v4l2_format *f)
 {
 	int ret;
 
-	ret = mtk_jpeg_try_fmt_vid_cap_mplane(file, priv, f);
+	ret = mtk_jpeg_dec_try_fmt_vid_cap_mplane(file, priv, f);
 	if (ret)
 		return ret;
 
@@ -410,8 +411,8 @@ static int mtk_jpeg_subscribe_event(struct v4l2_fh *fh,
 	return v4l2_ctrl_subscribe_event(fh, sub);
 }
 
-static int mtk_jpeg_g_selection(struct file *file, void *priv,
-				struct v4l2_selection *s)
+static int mtk_jpeg_dec_g_selection(struct file *file, void *priv,
+				    struct v4l2_selection *s)
 {
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
 
@@ -439,8 +440,8 @@ static int mtk_jpeg_g_selection(struct file *file, void *priv,
 	return 0;
 }
 
-static int mtk_jpeg_s_selection(struct file *file, void *priv,
-				struct v4l2_selection *s)
+static int mtk_jpeg_dec_s_selection(struct file *file, void *priv,
+				    struct v4l2_selection *s)
 {
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
 
@@ -483,20 +484,20 @@ static int mtk_jpeg_qbuf(struct file *file, void *priv, struct v4l2_buffer *buf)
 	return v4l2_m2m_qbuf(file, fh->m2m_ctx, buf);
 }
 
-static const struct v4l2_ioctl_ops mtk_jpeg_ioctl_ops = {
-	.vidioc_querycap                = mtk_jpeg_querycap,
-	.vidioc_enum_fmt_vid_cap	= mtk_jpeg_enum_fmt_vid_cap,
-	.vidioc_enum_fmt_vid_out	= mtk_jpeg_enum_fmt_vid_out,
-	.vidioc_try_fmt_vid_cap_mplane	= mtk_jpeg_try_fmt_vid_cap_mplane,
-	.vidioc_try_fmt_vid_out_mplane	= mtk_jpeg_try_fmt_vid_out_mplane,
+static const struct v4l2_ioctl_ops mtk_jpeg_dec_ioctl_ops = {
+	.vidioc_querycap                = mtk_jpeg_dec_querycap,
+	.vidioc_enum_fmt_vid_cap	= mtk_jpeg_dec_enum_fmt_vid_cap,
+	.vidioc_enum_fmt_vid_out	= mtk_jpeg_dec_enum_fmt_vid_out,
+	.vidioc_try_fmt_vid_cap_mplane	= mtk_jpeg_dec_try_fmt_vid_cap_mplane,
+	.vidioc_try_fmt_vid_out_mplane	= mtk_jpeg_dec_try_fmt_vid_out_mplane,
 	.vidioc_g_fmt_vid_cap_mplane    = mtk_jpeg_g_fmt_vid_mplane,
 	.vidioc_g_fmt_vid_out_mplane    = mtk_jpeg_g_fmt_vid_mplane,
-	.vidioc_s_fmt_vid_cap_mplane    = mtk_jpeg_s_fmt_vid_cap_mplane,
-	.vidioc_s_fmt_vid_out_mplane    = mtk_jpeg_s_fmt_vid_out_mplane,
+	.vidioc_s_fmt_vid_cap_mplane    = mtk_jpeg_dec_s_fmt_vid_cap_mplane,
+	.vidioc_s_fmt_vid_out_mplane    = mtk_jpeg_dec_s_fmt_vid_out_mplane,
 	.vidioc_qbuf                    = mtk_jpeg_qbuf,
 	.vidioc_subscribe_event         = mtk_jpeg_subscribe_event,
-	.vidioc_g_selection		= mtk_jpeg_g_selection,
-	.vidioc_s_selection		= mtk_jpeg_s_selection,
+	.vidioc_g_selection		= mtk_jpeg_dec_g_selection,
+	.vidioc_s_selection		= mtk_jpeg_dec_s_selection,
 
 	.vidioc_create_bufs		= v4l2_m2m_ioctl_create_bufs,
 	.vidioc_prepare_buf		= v4l2_m2m_ioctl_prepare_buf,
@@ -615,7 +616,7 @@ static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx,
 		 param->dec_w, param->dec_h);
 }
 
-static void mtk_jpeg_buf_queue(struct vb2_buffer *vb)
+static void mtk_jpeg_dec_buf_queue(struct vb2_buffer *vb)
 {
 	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
 	struct mtk_jpeg_dec_param *param;
@@ -663,7 +664,7 @@ static struct vb2_v4l2_buffer *mtk_jpeg_buf_remove(struct mtk_jpeg_ctx *ctx,
 		return v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
 }
 
-static void mtk_jpeg_stop_streaming(struct vb2_queue *q)
+static void mtk_jpeg_dec_stop_streaming(struct vb2_queue *q)
 {
 	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(q);
 	struct vb2_v4l2_buffer *vb;
@@ -689,13 +690,13 @@ static void mtk_jpeg_stop_streaming(struct vb2_queue *q)
 		v4l2_m2m_buf_done(vb, VB2_BUF_STATE_ERROR);
 }
 
-static const struct vb2_ops mtk_jpeg_qops = {
+static const struct vb2_ops mtk_jpeg_dec_qops = {
 	.queue_setup        = mtk_jpeg_queue_setup,
 	.buf_prepare        = mtk_jpeg_buf_prepare,
-	.buf_queue          = mtk_jpeg_buf_queue,
+	.buf_queue          = mtk_jpeg_dec_buf_queue,
 	.wait_prepare       = vb2_ops_wait_prepare,
 	.wait_finish        = vb2_ops_wait_finish,
-	.stop_streaming     = mtk_jpeg_stop_streaming,
+	.stop_streaming     = mtk_jpeg_dec_stop_streaming,
 };
 
 static void mtk_jpeg_set_dec_src(struct mtk_jpeg_ctx *ctx,
@@ -735,7 +736,7 @@ static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx,
 	return 0;
 }
 
-static void mtk_jpeg_device_run(void *priv)
+static void mtk_jpeg_dec_device_run(void *priv)
 {
 	struct mtk_jpeg_ctx *ctx = priv;
 	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
@@ -763,15 +764,16 @@ static void mtk_jpeg_device_run(void *priv)
 		goto dec_end;
 
 	mtk_jpeg_set_dec_src(ctx, &src_buf->vb2_buf, &bs);
-	if (mtk_jpeg_set_dec_dst(ctx, &jpeg_src_buf->dec_param, &dst_buf->vb2_buf, &fb))
+	if (mtk_jpeg_set_dec_dst(ctx, &jpeg_src_buf->dec_param,
+				 &dst_buf->vb2_buf, &fb))
 		goto dec_end;
 
 	spin_lock_irqsave(&jpeg->hw_lock, flags);
-	mtk_jpeg_dec_reset(jpeg->dec_reg_base);
-	mtk_jpeg_dec_set_config(jpeg->dec_reg_base,
+	mtk_jpeg_dec_reset(jpeg->reg_base);
+	mtk_jpeg_dec_set_config(jpeg->reg_base,
 				&jpeg_src_buf->dec_param, &bs, &fb);
 
-	mtk_jpeg_dec_start(jpeg->dec_reg_base);
+	mtk_jpeg_dec_start(jpeg->reg_base);
 	spin_unlock_irqrestore(&jpeg->hw_lock, flags);
 	return;
 
@@ -783,20 +785,20 @@ static void mtk_jpeg_device_run(void *priv)
 	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
 }
 
-static int mtk_jpeg_job_ready(void *priv)
+static int mtk_jpeg_dec_job_ready(void *priv)
 {
 	struct mtk_jpeg_ctx *ctx = priv;
 
 	return (ctx->state == MTK_JPEG_RUNNING) ? 1 : 0;
 }
 
-static const struct v4l2_m2m_ops mtk_jpeg_m2m_ops = {
-	.device_run = mtk_jpeg_device_run,
-	.job_ready  = mtk_jpeg_job_ready,
+static const struct v4l2_m2m_ops mtk_jpeg_dec_m2m_ops = {
+	.device_run = mtk_jpeg_dec_device_run,
+	.job_ready  = mtk_jpeg_dec_job_ready,
 };
 
-static int mtk_jpeg_queue_init(void *priv, struct vb2_queue *src_vq,
-			       struct vb2_queue *dst_vq)
+static int mtk_jpeg_dec_queue_init(void *priv, struct vb2_queue *src_vq,
+				   struct vb2_queue *dst_vq)
 {
 	struct mtk_jpeg_ctx *ctx = priv;
 	int ret;
@@ -805,7 +807,7 @@ static int mtk_jpeg_queue_init(void *priv, struct vb2_queue *src_vq,
 	src_vq->io_modes = VB2_DMABUF | VB2_MMAP;
 	src_vq->drv_priv = ctx;
 	src_vq->buf_struct_size = sizeof(struct mtk_jpeg_src_buf);
-	src_vq->ops = &mtk_jpeg_qops;
+	src_vq->ops = &mtk_jpeg_dec_qops;
 	src_vq->mem_ops = &vb2_dma_contig_memops;
 	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
 	src_vq->lock = &ctx->jpeg->lock;
@@ -818,7 +820,7 @@ static int mtk_jpeg_queue_init(void *priv, struct vb2_queue *src_vq,
 	dst_vq->io_modes = VB2_DMABUF | VB2_MMAP;
 	dst_vq->drv_priv = ctx;
 	dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
-	dst_vq->ops = &mtk_jpeg_qops;
+	dst_vq->ops = &mtk_jpeg_dec_qops;
 	dst_vq->mem_ops = &vb2_dma_contig_memops;
 	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
 	dst_vq->lock = &ctx->jpeg->lock;
@@ -857,7 +859,7 @@ static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv)
 	u32 dec_ret;
 	int i;
 
-	dec_ret = mtk_jpeg_dec_get_int_status(jpeg->dec_reg_base);
+	dec_ret = mtk_jpeg_dec_get_int_status(jpeg->reg_base);
 	dec_irq_ret = mtk_jpeg_dec_enum_result(dec_ret);
 	ctx = v4l2_m2m_get_curr_priv(jpeg->m2m_dev);
 	if (!ctx) {
@@ -870,7 +872,7 @@ static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv)
 	jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(&src_buf->vb2_buf);
 
 	if (dec_irq_ret >= MTK_JPEG_DEC_RESULT_UNDERFLOW)
-		mtk_jpeg_dec_reset(jpeg->dec_reg_base);
+		mtk_jpeg_dec_reset(jpeg->reg_base);
 
 	if (dec_irq_ret != MTK_JPEG_DEC_RESULT_EOF_DONE) {
 		dev_err(jpeg->dev, "decode failed\n");
@@ -891,7 +893,7 @@ static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv)
 	return IRQ_HANDLED;
 }
 
-static void mtk_jpeg_set_default_params(struct mtk_jpeg_ctx *ctx)
+static void mtk_jpeg_set_dec_default_params(struct mtk_jpeg_ctx *ctx)
 {
 	struct mtk_jpeg_q_data *q = &ctx->out_q;
 	int i;
@@ -923,7 +925,7 @@ static void mtk_jpeg_set_default_params(struct mtk_jpeg_ctx *ctx)
 	}
 }
 
-static int mtk_jpeg_open(struct file *file)
+static int mtk_jpeg_dec_open(struct file *file)
 {
 	struct mtk_jpeg_dev *jpeg = video_drvdata(file);
 	struct video_device *vfd = video_devdata(file);
@@ -945,13 +947,13 @@ static int mtk_jpeg_open(struct file *file)
 
 	ctx->jpeg = jpeg;
 	ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(jpeg->m2m_dev, ctx,
-					    mtk_jpeg_queue_init);
+					    mtk_jpeg_dec_queue_init);
 	if (IS_ERR(ctx->fh.m2m_ctx)) {
 		ret = PTR_ERR(ctx->fh.m2m_ctx);
 		goto error;
 	}
 
-	mtk_jpeg_set_default_params(ctx);
+	mtk_jpeg_set_dec_default_params(ctx);
 	mutex_unlock(&jpeg->lock);
 	return 0;
 
@@ -978,9 +980,9 @@ static int mtk_jpeg_release(struct file *file)
 	return 0;
 }
 
-static const struct v4l2_file_operations mtk_jpeg_fops = {
+static const struct v4l2_file_operations mtk_jpeg_dec_fops = {
 	.owner          = THIS_MODULE,
-	.open           = mtk_jpeg_open,
+	.open           = mtk_jpeg_dec_open,
 	.release        = mtk_jpeg_release,
 	.poll           = v4l2_m2m_fop_poll,
 	.unlocked_ioctl = video_ioctl2,
@@ -1016,7 +1018,7 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 {
 	struct mtk_jpeg_dev *jpeg;
 	struct resource *res;
-	int dec_irq;
+	int jpeg_irq;
 	int ret;
 
 	jpeg = devm_kzalloc(&pdev->dev, sizeof(*jpeg), GFP_KERNEL);
@@ -1028,23 +1030,23 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 	jpeg->dev = &pdev->dev;
 
 	res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
-	jpeg->dec_reg_base = devm_ioremap_resource(&pdev->dev, res);
-	if (IS_ERR(jpeg->dec_reg_base)) {
-		ret = PTR_ERR(jpeg->dec_reg_base);
+	jpeg->reg_base = devm_ioremap_resource(&pdev->dev, res);
+	if (IS_ERR(jpeg->reg_base)) {
+		ret = PTR_ERR(jpeg->reg_base);
 		return ret;
 	}
 
-	dec_irq = platform_get_irq(pdev, 0);
-	if (dec_irq < 0) {
-		dev_err(&pdev->dev, "Failed to get dec_irq %d.\n", dec_irq);
-		return dec_irq;
+	jpeg_irq = platform_get_irq(pdev, 0);
+	if (jpeg_irq < 0) {
+		dev_err(&pdev->dev, "Failed to get jpeg_irq %d.\n", jpeg_irq);
+		return jpeg_irq;
 	}
 
-	ret = devm_request_irq(&pdev->dev, dec_irq, mtk_jpeg_dec_irq, 0,
+	ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_dec_irq, 0,
 			       pdev->name, jpeg);
 	if (ret) {
-		dev_err(&pdev->dev, "Failed to request dec_irq %d (%d)\n",
-			dec_irq, ret);
+		dev_err(&pdev->dev, "Failed to request jpeg_irq %d (%d)\n",
+			jpeg_irq, ret);
 		goto err_req_irq;
 	}
 
@@ -1061,40 +1063,40 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 		goto err_dev_register;
 	}
 
-	jpeg->m2m_dev = v4l2_m2m_init(&mtk_jpeg_m2m_ops);
+	jpeg->m2m_dev = v4l2_m2m_init(&mtk_jpeg_dec_m2m_ops);
 	if (IS_ERR(jpeg->m2m_dev)) {
 		v4l2_err(&jpeg->v4l2_dev, "Failed to init mem2mem device\n");
 		ret = PTR_ERR(jpeg->m2m_dev);
 		goto err_m2m_init;
 	}
 
-	jpeg->dec_vdev = video_device_alloc();
-	if (!jpeg->dec_vdev) {
+	jpeg->vdev = video_device_alloc();
+	if (!jpeg->vdev) {
 		ret = -ENOMEM;
-		goto err_dec_vdev_alloc;
+		goto err_vfd_jpeg_alloc;
 	}
-	snprintf(jpeg->dec_vdev->name, sizeof(jpeg->dec_vdev->name),
+	snprintf(jpeg->vdev->name, sizeof(jpeg->vdev->name),
 		 "%s-dec", MTK_JPEG_NAME);
-	jpeg->dec_vdev->fops = &mtk_jpeg_fops;
-	jpeg->dec_vdev->ioctl_ops = &mtk_jpeg_ioctl_ops;
-	jpeg->dec_vdev->minor = -1;
-	jpeg->dec_vdev->release = video_device_release;
-	jpeg->dec_vdev->lock = &jpeg->lock;
-	jpeg->dec_vdev->v4l2_dev = &jpeg->v4l2_dev;
-	jpeg->dec_vdev->vfl_dir = VFL_DIR_M2M;
-	jpeg->dec_vdev->device_caps = V4L2_CAP_STREAMING |
+	jpeg->vdev->fops = &mtk_jpeg_dec_fops;
+	jpeg->vdev->ioctl_ops = &mtk_jpeg_dec_ioctl_ops;
+	jpeg->vdev->minor = -1;
+	jpeg->vdev->release = video_device_release;
+	jpeg->vdev->lock = &jpeg->lock;
+	jpeg->vdev->v4l2_dev = &jpeg->v4l2_dev;
+	jpeg->vdev->vfl_dir = VFL_DIR_M2M;
+	jpeg->vdev->device_caps = V4L2_CAP_STREAMING |
 				      V4L2_CAP_VIDEO_M2M_MPLANE;
 
-	ret = video_register_device(jpeg->dec_vdev, VFL_TYPE_VIDEO, -1);
+	ret = video_register_device(jpeg->vdev, VFL_TYPE_VIDEO, -1);
 	if (ret) {
 		v4l2_err(&jpeg->v4l2_dev, "Failed to register video device\n");
-		goto err_dec_vdev_register;
+		goto err_vfd_jpeg_register;
 	}
 
-	video_set_drvdata(jpeg->dec_vdev, jpeg);
+	video_set_drvdata(jpeg->vdev, jpeg);
 	v4l2_info(&jpeg->v4l2_dev,
 		  "decoder device registered as /dev/video%d (%d,%d)\n",
-		  jpeg->dec_vdev->num, VIDEO_MAJOR, jpeg->dec_vdev->minor);
+		  jpeg->vdev->num, VIDEO_MAJOR, jpeg->vdev->minor);
 
 	platform_set_drvdata(pdev, jpeg);
 
@@ -1102,10 +1104,10 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 
 	return 0;
 
-err_dec_vdev_register:
-	video_device_release(jpeg->dec_vdev);
+err_vfd_jpeg_register:
+	video_device_release(jpeg->vdev);
 
-err_dec_vdev_alloc:
+err_vfd_jpeg_alloc:
 	v4l2_m2m_release(jpeg->m2m_dev);
 
 err_m2m_init:
@@ -1125,8 +1127,8 @@ static int mtk_jpeg_remove(struct platform_device *pdev)
 	struct mtk_jpeg_dev *jpeg = platform_get_drvdata(pdev);
 
 	pm_runtime_disable(&pdev->dev);
-	video_unregister_device(jpeg->dec_vdev);
-	video_device_release(jpeg->dec_vdev);
+	video_unregister_device(jpeg->vdev);
+	video_device_release(jpeg->vdev);
 	v4l2_m2m_release(jpeg->m2m_dev);
 	v4l2_device_unregister(&jpeg->v4l2_dev);
 
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +4 -4
index 5fcdf6950782..0b59e48495d5 100644
@@ -51,8 +51,8 @@ enum mtk_jpeg_ctx_state {
  * @v4l2_dev:		v4l2 device for mem2mem mode
  * @m2m_dev:		v4l2 mem2mem device data
  * @alloc_ctx:		videobuf2 memory allocator's context
- * @dec_vdev:		video device node for decoder mem2mem mode
- * @dec_reg_base:	JPEG registers mapping
+ * @vdev:		video device node for jpeg mem2mem mode
+ * @reg_base:		JPEG registers mapping
  * @clk_jdec:		JPEG hw working clock
  * @clk_jdec_smi:	JPEG SMI bus clock
  * @larb:		SMI device
@@ -65,8 +65,8 @@ struct mtk_jpeg_dev {
 	struct v4l2_device	v4l2_dev;
 	struct v4l2_m2m_dev	*m2m_dev;
 	void			*alloc_ctx;
-	struct video_device	*dec_vdev;
-	void __iomem		*dec_reg_base;
+	struct video_device	*vdev;
+	void __iomem		*reg_base;
 	struct clk		*clk_jdec;
 	struct clk		*clk_jdec_smi;
 	struct device		*larb;
drivers/media/platform/mtk-jpeg/mtk_jpeg_dec_hw.h +4 -3
index 1cc37dbfc8e7..ce263db5f30a 100644
@@ -3,10 +3,11 @@
  * Copyright (c) 2016 MediaTek Inc.
  * Author: Ming Hsiu Tsai <[email protected]>
  *         Rick Chang <[email protected]>
+ *         Xia Jiang <[email protected]>
  */
 
-#ifndef _MTK_JPEG_HW_H
-#define _MTK_JPEG_HW_H
+#ifndef _MTK_JPEG_DEC_HW_H
+#define _MTK_JPEG_DEC_HW_H
 
 #include <media/videobuf2-core.h>
 
@@ -75,4 +76,4 @@ void mtk_jpeg_dec_set_config(void __iomem *base,
 void mtk_jpeg_dec_reset(void __iomem *dec_reg_base);
 void mtk_jpeg_dec_start(void __iomem *dec_reg_base);
 
-#endif /* _MTK_JPEG_HW_H */
+#endif /* _MTK_JPEG_DEC_HW_H */
Signature · 1 line
2.18.0
#27 lore raw

Re: [PATCH RESEND v9 17/18] media: platform: Rename existing functions/defines/variables

Quoted 1 line · Xia Jiang

Rename existing funcitons/defines/variables with a _dec prefix and

Tiny typo: funcitons -> functions

Quoted 2 lines

without dec_ prefix to prepare for the addition of the jpeg encoder feature.

Regards,

Hans

Reviewed-by on v9 17/18
#28 lore raw

Re: [PATCH RESEND v9 17/18] media: platform: Rename existing functions/defines/variables

In reply to Xia Jiang

Hi Xia,

Quoted 4 lines · Xia Jiang

Rename existing funcitons/defines/variables with a _dec prefix and without dec_ prefix to prepare for the addition of the jpeg encoder feature.

Quoted 9 lines

---
v9: new patch

---
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 200 +++++++++---------
 .../media/platform/mtk-jpeg/mtk_jpeg_core.h   |   8 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_dec_hw.h |   7 +-
 3 files changed, 109 insertions(+), 106 deletions(-)

Best regards,
Tomasz

author
#29 lore raw

PATCH RESEND v9 18/18 media: platform: Add jpeg enc feature

In reply to Xia Jiang

Add mtk jpeg encode v4l2 driver based on jpeg decode, because that jpeg decode and encode have great similarities with function operation.

---
v9: add member variable(struct v4l2_rect) in out_q structure for storing
the active crop information.
move the renaming exsting functions/defines/variables to a separate patch.

---
 drivers/media/platform/mtk-jpeg/Makefile      |   5 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 845 +++++++++++++++---
 .../media/platform/mtk-jpeg/mtk_jpeg_core.h   |  44 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c | 193 ++++
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h | 123 +++
 5 files changed, 1084 insertions(+), 126 deletions(-)
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h
drivers/media/platform/mtk-jpeg/Makefile +4 -1
index 48516dcf96e6..76c33aad0f3f 100644
@@ -1,3 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0-only
-mtk_jpeg-objs := mtk_jpeg_core.o mtk_jpeg_dec_hw.o mtk_jpeg_dec_parse.o
+mtk_jpeg-objs := mtk_jpeg_core.o \
+		 mtk_jpeg_dec_hw.o \
+		 mtk_jpeg_dec_parse.o \
+		 mtk_jpeg_enc_hw.o
 obj-$(CONFIG_VIDEO_MEDIATEK_JPEG) += mtk_jpeg.o
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +729 -116

1,171 lines not on this page · raw message

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +35 -9

113 lines not on this page · raw message

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +193 -0

193 lines not on this page · raw message

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h new file +123 -0

123 lines not on this page · raw message

Signature · 1 line
2.18.0
#30 lore raw

Re: [PATCH RESEND v9 18/18] media: platform: Add jpeg enc feature

Hi, Xia:

Xia Jiang <[email protected]> 於 2020年6月4日 週四 下午5:21寫道:

Quoted 19 lines

Add mtk jpeg encode v4l2 driver based on jpeg decode, because that jpeg decode and encode have great similarities with function operation.

---
v9: add member variable(struct v4l2_rect) in out_q structure for storing
the active crop information.
move the renaming exsting functions/defines/variables to a separate patch.

---
 drivers/media/platform/mtk-jpeg/Makefile      |   5 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 845 +++++++++++++++---
 .../media/platform/mtk-jpeg/mtk_jpeg_core.h   |  44 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c | 193 ++++
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h | 123 +++
 5 files changed, 1084 insertions(+), 126 deletions(-)
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h

[snip]

Quoted 61 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +44 -7
+static void mtk_jpeg_set_enc_default_params(struct mtk_jpeg_ctx *ctx)
+{
+       struct mtk_jpeg_q_data *q = &ctx->out_q;
+       struct v4l2_pix_format_mplane *pix_mp;
+
+       pix_mp = kmalloc(sizeof(*pix_mp), GFP_KERNEL);
+
+       ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+       ctx->colorspace = V4L2_COLORSPACE_JPEG,
+       ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+       ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
+       ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+       pix_mp->width = MTK_JPEG_MIN_WIDTH;
+       pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+
+       q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_YUYV,
+                                     MTK_JPEG_FMT_FLAG_ENC_OUTPUT);
+       vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+                                   fmt.pix_mp), q->fmt);
+       q->w = pix_mp->width;
+       q->h = pix_mp->height;
+       q->crop_rect.width = pix_mp->width;
+       q->crop_rect.height = pix_mp->height;
+       q->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
+       q->bytesperline[0] = pix_mp->plane_fmt[0].bytesperline;
+
+       q = &ctx->cap_q;
+       q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_JPEG,
+                                     MTK_JPEG_FMT_FLAG_ENC_CAPTURE);
+       pix_mp->width = MTK_JPEG_MIN_WIDTH;
+       pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+       vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+                                   fmt.pix_mp), q->fmt);
+       q->w = pix_mp->width;
+       q->h = pix_mp->height;
+       q->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
+       q->bytesperline[0] = pix_mp->plane_fmt[0].bytesperline;
+}
+
 static void mtk_jpeg_set_dec_default_params(struct mtk_jpeg_ctx *ctx)
 {
        struct mtk_jpeg_q_data *q = &ctx->out_q;
+       struct v4l2_pix_format_mplane *pix_mp;
        int i;
 
+       pix_mp = kmalloc(sizeof(*pix_mp), GFP_KERNEL);
+
+       ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
        ctx->colorspace = V4L2_COLORSPACE_JPEG,
        ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
        ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
        ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;
-
-       q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_JPEG,
-                                             MTK_JPEG_FMT_TYPE_OUTPUT);
-       q->w = MTK_JPEG_MIN_WIDTH;
-       q->h = MTK_JPEG_MIN_HEIGHT;
-       q->bytesperline[0] = 0;
-       q->sizeimage[0] = MTK_JPEG_DEFAULT_SIZEIMAGE;
+       pix_mp->width = MTK_JPEG_MIN_WIDTH;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1418

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+       pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+
+       q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_JPEG,
+                                     MTK_JPEG_FMT_FLAG_DEC_OUTPUT);
+       vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+                                   fmt.pix_mp), q->fmt);
+       q->w = pix_mp->width;
+       q->h = pix_mp->height;
+       q->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
+       q->bytesperline[0] = pix_mp->plane_fmt[0].bytesperline;

I would like non-jpeg-enc related modification to be another patch.

Quoted 208 lines
        q = &ctx->cap_q;
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +106 -22
-       q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_YUV420M,
-                                             MTK_JPEG_FMT_TYPE_CAPTURE);
-       q->w = MTK_JPEG_MIN_WIDTH;
-       q->h = MTK_JPEG_MIN_HEIGHT;
-
+       q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_YUV420M,
+                                     MTK_JPEG_FMT_FLAG_DEC_CAPTURE);
+       pix_mp->width = MTK_JPEG_MIN_WIDTH;
+       pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+       vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+                                   fmt.pix_mp), q->fmt);
+       q->w = pix_mp->width;
+       q->h = pix_mp->height;
        for (i = 0; i < q->fmt->colplanes; i++) {
-               u32 stride = q->w * q->fmt->h_sample[i] / 4;
-               u32 h = q->h * q->fmt->v_sample[i] / 4;
+               q->sizeimage[i] = pix_mp->plane_fmt[i].sizeimage;
+               q->bytesperline[i] = pix_mp->plane_fmt[i].bytesperline;
+       }
+}
 
-               q->bytesperline[i] = stride;
-               q->sizeimage[i] = stride * h;
+static int mtk_jpeg_enc_open(struct file *file)
+{
+       struct mtk_jpeg_dev *jpeg = video_drvdata(file);
+       struct video_device *vfd = video_devdata(file);
+       struct mtk_jpeg_ctx *ctx;
+       int ret = 0;
+
+       ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+       if (!ctx)
+               return -ENOMEM;
+
+       if (mutex_lock_interruptible(&jpeg->lock)) {
+               ret = -ERESTARTSYS;
+               goto free;
+       }
+
+       v4l2_fh_init(&ctx->fh, vfd);
+       file->private_data = &ctx->fh;
+       v4l2_fh_add(&ctx->fh);
+
+       ctx->jpeg = jpeg;
+       ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(jpeg->m2m_dev, ctx,
+                                           mtk_jpeg_enc_queue_init);
+       if (IS_ERR(ctx->fh.m2m_ctx)) {
+               ret = PTR_ERR(ctx->fh.m2m_ctx);
+               goto error;
        }
+
+       ret = mtk_jpeg_enc_ctrls_setup(ctx);
+       if (ret) {
+               v4l2_err(&jpeg->v4l2_dev, "Failed to setup jpeg enc controls\n");
+               goto error;
+       }
+       mtk_jpeg_set_enc_default_params(ctx);
+
+       mutex_unlock(&jpeg->lock);
+       return 0;
+
+error:
+       v4l2_fh_del(&ctx->fh);
+       v4l2_fh_exit(&ctx->fh);
+       mutex_unlock(&jpeg->lock);
+free:
+       kfree(ctx);
+       return ret;
 }
 
 static int mtk_jpeg_dec_open(struct file *file)
@@ -953,6 +1507,12 @@ static int mtk_jpeg_dec_open(struct file *file)
                goto error;
        }
 
+       v4l2_ctrl_handler_init(&ctx->ctrl_hdl, 0);
+       ret = v4l2_ctrl_handler_setup(&ctx->ctrl_hdl);
+       if (ret) {
+               v4l2_err(&jpeg->v4l2_dev, "Failed to setup jpeg dec controls\n");
+               goto error;
+       }
        mtk_jpeg_set_dec_default_params(ctx);
        mutex_unlock(&jpeg->lock);
        return 0;
@@ -973,6 +1533,7 @@ static int mtk_jpeg_release(struct file *file)
 
        mutex_lock(&jpeg->lock);
        v4l2_m2m_ctx_release(ctx->fh.m2m_ctx);
+       v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
        v4l2_fh_del(&ctx->fh);
        v4l2_fh_exit(&ctx->fh);
        kfree(ctx);
@@ -980,6 +1541,15 @@ static int mtk_jpeg_release(struct file *file)
        return 0;
 }
 
+static const struct v4l2_file_operations mtk_jpeg_enc_fops = {
+       .owner          = THIS_MODULE,
+       .open           = mtk_jpeg_enc_open,
+       .release        = mtk_jpeg_release,
+       .poll           = v4l2_m2m_fop_poll,
+       .unlocked_ioctl = video_ioctl2,
+       .mmap           = v4l2_m2m_fop_mmap,
+};
+
 static const struct v4l2_file_operations mtk_jpeg_dec_fops = {
        .owner          = THIS_MODULE,
        .open           = mtk_jpeg_dec_open,
@@ -993,6 +1563,7 @@ static int mtk_jpeg_clk_init(struct mtk_jpeg_dev *jpeg)
 {
        struct device_node *node;
        struct platform_device *pdev;
+       int i;
 
        node = of_parse_phandle(jpeg->dev->of_node, "mediatek,larb", 0);
        if (!node)
@@ -1006,12 +1577,17 @@ static int mtk_jpeg_clk_init(struct mtk_jpeg_dev *jpeg)
 
        jpeg->larb = &pdev->dev;
 
-       jpeg->clk_jdec = devm_clk_get(jpeg->dev, "jpgdec");
-       if (IS_ERR(jpeg->clk_jdec))
-               return PTR_ERR(jpeg->clk_jdec);
+       for (i = 0; i < jpeg->variant->num_clocks; i++) {
+               jpeg->clocks[i] = devm_clk_get(jpeg->dev,
+                                              jpeg->variant->clk_names[i]);
+               if (IS_ERR(jpeg->clocks[i])) {
+                       dev_err(&pdev->dev, "failed to get clock: %s\n",
+                               jpeg->variant->clk_names[i]);
+                       return PTR_ERR(jpeg->clocks[i]);
+               }
+       }
 
-       jpeg->clk_jdec_smi = devm_clk_get(jpeg->dev, "jpgdec-smi");
-       return PTR_ERR_OR_ZERO(jpeg->clk_jdec_smi);
+       return 0;
 }
 
 static int mtk_jpeg_probe(struct platform_device *pdev)
@@ -1028,6 +1604,7 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
        mutex_init(&jpeg->lock);
        spin_lock_init(&jpeg->hw_lock);
        jpeg->dev = &pdev->dev;
+       jpeg->variant = of_device_get_match_data(jpeg->dev);
 
        res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
        jpeg->reg_base = devm_ioremap_resource(&pdev->dev, res);
@@ -1042,8 +1619,12 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
                return jpeg_irq;
        }
 
-       ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_dec_irq, 0,
-                              pdev->name, jpeg);
+       if (jpeg->variant->is_encoder)
+               ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_enc_irq,
+                                      0, pdev->name, jpeg);
+       else
+               ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_dec_irq,
+                                      0, pdev->name, jpeg);
        if (ret) {
                dev_err(&pdev->dev, "Failed to request jpeg_irq %d (%d)\n",
                        jpeg_irq, ret);
@@ -1063,7 +1644,10 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
                goto err_dev_register;
        }
 
-       jpeg->m2m_dev = v4l2_m2m_init(&mtk_jpeg_dec_m2m_ops);
+       if (jpeg->variant->is_encoder)
+               jpeg->m2m_dev = v4l2_m2m_init(&mtk_jpeg_enc_m2m_ops);
+       else
+               jpeg->m2m_dev = v4l2_m2m_init(&mtk_jpeg_dec_m2m_ops);
        if (IS_ERR(jpeg->m2m_dev)) {
                v4l2_err(&jpeg->v4l2_dev, "Failed to init mem2mem device\n");
                ret = PTR_ERR(jpeg->m2m_dev);
@@ -1076,9 +1660,15 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
                goto err_vfd_jpeg_alloc;
        }
        snprintf(jpeg->vdev->name, sizeof(jpeg->vdev->name),
-                "%s-dec", MTK_JPEG_NAME);
-       jpeg->vdev->fops = &mtk_jpeg_dec_fops;
-       jpeg->vdev->ioctl_ops = &mtk_jpeg_dec_ioctl_ops;
+                "%s-%s", MTK_JPEG_NAME,
+                jpeg->variant->is_encoder ? "enc" : "dec");
+       if (jpeg->variant->is_encoder) {
+               jpeg->vdev->fops = &mtk_jpeg_enc_fops;
+               jpeg->vdev->ioctl_ops = &mtk_jpeg_enc_ioctl_ops;
+       } else {
+               jpeg->vdev->fops = &mtk_jpeg_dec_fops;
+               jpeg->vdev->ioctl_ops = &mtk_jpeg_dec_ioctl_ops;
+       }
        jpeg->vdev->minor = -1;
        jpeg->vdev->release = video_device_release;
        jpeg->vdev->lock = &jpeg->lock;
@@ -1095,8 +1685,9 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 
        video_set_drvdata(jpeg->vdev, jpeg);
        v4l2_info(&jpeg->v4l2_dev,
-                 "decoder device registered as /dev/video%d (%d,%d)\n",
-                 jpeg->vdev->num, VIDEO_MAJOR, jpeg->vdev->minor);
+                 "jpeg %s device registered as /dev/video%d (%d,%d)\n",
+                 jpeg->variant->is_encoder ? "enc" : "dec", jpeg->vdev->num,
+                 VIDEO_MAJOR, jpeg->vdev->minor);
 
        platform_set_drvdata(pdev, jpeg);
 

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static const struct dev_pm_ops mtk_jpeg_pm_ops = { · line 1785

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -0
@@ -1187,14 +1778,36 @@ static const struct dev_pm_ops mtk_jpeg_pm_ops = {
        SET_RUNTIME_PM_OPS(mtk_jpeg_pm_suspend, mtk_jpeg_pm_resume, NULL)
 };
 
+static struct mtk_jpeg_variant mt8173_jpeg_drvdata = {
+       .is_encoder     = false,
+       .clk_names      = {"jpgdec-smi", "jpgdec"},
+       .num_clocks     = 2,
+};

I would like non-jpeg-enc related modification to be another patch.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static const struct dev_pm_ops mtk_jpeg_pm_ops = { · line 1791

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
+
+static struct mtk_jpeg_variant mt2701_jpeg_drvdata = {
+       .is_encoder     = false,
+       .clk_names      = {"jpgdec-smi", "jpgdec"},
+       .num_clocks     = 2,
+};

mt2701_jpeg_drvdata is equal to mt8173_jpeg_drvdata, so keep only mt8173_jpeg_drvdata.

Quoted 47 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +13 -2
+
+static struct mtk_jpeg_variant mtk_jpeg_drvdata = {
+       .is_encoder     = true,
+       .clk_names      = {"jpgenc"},
+       .num_clocks     = 1,
+};
+
 static const struct of_device_id mtk_jpeg_match[] = {
        {
                .compatible = "mediatek,mt8173-jpgdec",
-               .data       = NULL,
+               .data = &mt8173_jpeg_drvdata,
        },
        {
                .compatible = "mediatek,mt2701-jpgdec",
-               .data       = NULL,
+               .data = &mt2701_jpeg_drvdata,
+       },
+       {
+               .compatible = "mediatek,mtk-jpgenc",
+               .data = &mtk_jpeg_drvdata,
        },
        {},
 };
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +3 -2
index 0b59e48495d5..9ec2c3350a16 100644
@@ -3,6 +3,7 @@
  * Copyright (c) 2016 MediaTek Inc.
  * Author: Ming Hsiu Tsai <[email protected]>
  *         Rick Chang <[email protected]>
+ *         Xia Jiang <[email protected]>
  */
 
 #ifndef _MTK_JPEG_CORE_H
@@ -16,19 +17,21 @@
 #define MTK_JPEG_NAME          "mtk-jpeg"
 
 #define MTK_JPEG_COMP_MAX              3
+#define MTK_JPEG_MAX_CLOCKS            2
+
 
 #define MTK_JPEG_FMT_FLAG_DEC_OUTPUT   BIT(0)
 #define MTK_JPEG_FMT_FLAG_DEC_CAPTURE  BIT(1)
-
-#define MTK_JPEG_FMT_TYPE_OUTPUT       1

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 31

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +4 -3
-#define MTK_JPEG_FMT_TYPE_CAPTURE      2
+#define MTK_JPEG_FMT_FLAG_ENC_OUTPUT   BIT(2)
+#define MTK_JPEG_FMT_FLAG_ENC_CAPTURE  BIT(3)
 
 #define MTK_JPEG_MIN_WIDTH     32U
 #define MTK_JPEG_MIN_HEIGHT    32U
-#define MTK_JPEG_MAX_WIDTH     8192U
-#define MTK_JPEG_MAX_HEIGHT    8192U
+#define MTK_JPEG_MAX_WIDTH     65535U
+#define MTK_JPEG_MAX_HEIGHT    65535U

MT8173 use this definition, why do you change this?
If the definition is wrong in MT8173, use a patch to fixup this.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h enum mtk_jpeg_ctx_state { · line 47

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +1 -0
 #define MTK_JPEG_DEFAULT_SIZEIMAGE     (1 * 1024 * 1024)
+#define MTK_JPEG_MAX_EXIF_SIZE (64 * 1024)
 
 /**
  * enum mtk_jpeg_ctx_state - states of the context state machine
@@ -42,6 +45,18 @@ enum mtk_jpeg_ctx_state {
        MTK_JPEG_SOURCE_CHANGE,
 };
 

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

author
#31 lore raw

Re: [PATCH RESEND v9 18/18] media: platform: Add jpeg enc feature

Quoted 23 lines · Chun-Kuang Hu

Hi, Xia:

Xia Jiang <[email protected]> 於 2020年6月4日 週四 下午5:21寫道:

Quoted 9 lines

Add mtk jpeg encode v4l2 driver based on jpeg decode, because that jpeg decode and encode have great similarities with function operation.

---
v9: add member variable(struct v4l2_rect) in out_q structure for storing
the active crop information.
move the renaming exsting functions/defines/variables to a separate patch.

Quoted 10 lines
---
 drivers/media/platform/mtk-jpeg/Makefile      |   5 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 845 +++++++++++++++---
 .../media/platform/mtk-jpeg/mtk_jpeg_core.h   |  44 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c | 193 ++++
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h | 123 +++
 5 files changed, 1084 insertions(+), 126 deletions(-)
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h
Quoted 2 lines

[snip]

Dear Chun-Kuang,
Thanks for your reply.
I have uploaded v10 version which contains the changes you mentioned.

Quoted 65 lines
Quoted 65 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +48 -7
+static void mtk_jpeg_set_enc_default_params(struct mtk_jpeg_ctx *ctx)
+{
+       struct mtk_jpeg_q_data *q = &ctx->out_q;
+       struct v4l2_pix_format_mplane *pix_mp;
+
+       pix_mp = kmalloc(sizeof(*pix_mp), GFP_KERNEL);
+
+       ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+       ctx->colorspace = V4L2_COLORSPACE_JPEG,
+       ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+       ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
+       ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+       pix_mp->width = MTK_JPEG_MIN_WIDTH;
+       pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+
+       q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_YUYV,
+                                     MTK_JPEG_FMT_FLAG_ENC_OUTPUT);
+       vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+                                   fmt.pix_mp), q->fmt);
+       q->w = pix_mp->width;
+       q->h = pix_mp->height;
+       q->crop_rect.width = pix_mp->width;
+       q->crop_rect.height = pix_mp->height;
+       q->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
+       q->bytesperline[0] = pix_mp->plane_fmt[0].bytesperline;
+
+       q = &ctx->cap_q;
+       q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_JPEG,
+                                     MTK_JPEG_FMT_FLAG_ENC_CAPTURE);
+       pix_mp->width = MTK_JPEG_MIN_WIDTH;
+       pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+       vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+                                   fmt.pix_mp), q->fmt);
+       q->w = pix_mp->width;
+       q->h = pix_mp->height;
+       q->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
+       q->bytesperline[0] = pix_mp->plane_fmt[0].bytesperline;
+}
+
 static void mtk_jpeg_set_dec_default_params(struct mtk_jpeg_ctx *ctx)
 {
        struct mtk_jpeg_q_data *q = &ctx->out_q;
+       struct v4l2_pix_format_mplane *pix_mp;
        int i;
 
+       pix_mp = kmalloc(sizeof(*pix_mp), GFP_KERNEL);
+
+       ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
        ctx->colorspace = V4L2_COLORSPACE_JPEG,
        ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
        ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
        ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;
-
-       q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_JPEG,
-                                             MTK_JPEG_FMT_TYPE_OUTPUT);
-       q->w = MTK_JPEG_MIN_WIDTH;
-       q->h = MTK_JPEG_MIN_HEIGHT;
-       q->bytesperline[0] = 0;
-       q->sizeimage[0] = MTK_JPEG_DEFAULT_SIZEIMAGE;
+       pix_mp->width = MTK_JPEG_MIN_WIDTH;
+       pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+
+       q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_JPEG,
+                                     MTK_JPEG_FMT_FLAG_DEC_OUTPUT);
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1380

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
+       vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+                                   fmt.pix_mp), q->fmt);
+       q->w = pix_mp->width;
+       q->h = pix_mp->height;
+       q->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
+       q->bytesperline[0] = pix_mp->plane_fmt[0].bytesperline;

I would like non-jpeg-enc related modification to be another patch.

done.

Quoted 212 lines
Quoted 212 lines

        q = &ctx->cap_q;
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +106 -22
-       q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_YUV420M,
-                                             MTK_JPEG_FMT_TYPE_CAPTURE);
-       q->w = MTK_JPEG_MIN_WIDTH;
-       q->h = MTK_JPEG_MIN_HEIGHT;
-
+       q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_YUV420M,
+                                     MTK_JPEG_FMT_FLAG_DEC_CAPTURE);
+       pix_mp->width = MTK_JPEG_MIN_WIDTH;
+       pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+       vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+                                   fmt.pix_mp), q->fmt);
+       q->w = pix_mp->width;
+       q->h = pix_mp->height;
        for (i = 0; i < q->fmt->colplanes; i++) {
-               u32 stride = q->w * q->fmt->h_sample[i] / 4;
-               u32 h = q->h * q->fmt->v_sample[i] / 4;
+               q->sizeimage[i] = pix_mp->plane_fmt[i].sizeimage;
+               q->bytesperline[i] = pix_mp->plane_fmt[i].bytesperline;
+       }
+}
 
-               q->bytesperline[i] = stride;
-               q->sizeimage[i] = stride * h;
+static int mtk_jpeg_enc_open(struct file *file)
+{
+       struct mtk_jpeg_dev *jpeg = video_drvdata(file);
+       struct video_device *vfd = video_devdata(file);
+       struct mtk_jpeg_ctx *ctx;
+       int ret = 0;
+
+       ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+       if (!ctx)
+               return -ENOMEM;
+
+       if (mutex_lock_interruptible(&jpeg->lock)) {
+               ret = -ERESTARTSYS;
+               goto free;
+       }
+
+       v4l2_fh_init(&ctx->fh, vfd);
+       file->private_data = &ctx->fh;
+       v4l2_fh_add(&ctx->fh);
+
+       ctx->jpeg = jpeg;
+       ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(jpeg->m2m_dev, ctx,
+                                           mtk_jpeg_enc_queue_init);
+       if (IS_ERR(ctx->fh.m2m_ctx)) {
+               ret = PTR_ERR(ctx->fh.m2m_ctx);
+               goto error;
        }
+
+       ret = mtk_jpeg_enc_ctrls_setup(ctx);
+       if (ret) {
+               v4l2_err(&jpeg->v4l2_dev, "Failed to setup jpeg enc controls\n");
+               goto error;
+       }
+       mtk_jpeg_set_enc_default_params(ctx);
+
+       mutex_unlock(&jpeg->lock);
+       return 0;
+
+error:
+       v4l2_fh_del(&ctx->fh);
+       v4l2_fh_exit(&ctx->fh);
+       mutex_unlock(&jpeg->lock);
+free:
+       kfree(ctx);
+       return ret;
 }
 
 static int mtk_jpeg_dec_open(struct file *file)
@@ -953,6 +1507,12 @@ static int mtk_jpeg_dec_open(struct file *file)
                goto error;
        }
 
+       v4l2_ctrl_handler_init(&ctx->ctrl_hdl, 0);
+       ret = v4l2_ctrl_handler_setup(&ctx->ctrl_hdl);
+       if (ret) {
+               v4l2_err(&jpeg->v4l2_dev, "Failed to setup jpeg dec controls\n");
+               goto error;
+       }
        mtk_jpeg_set_dec_default_params(ctx);
        mutex_unlock(&jpeg->lock);
        return 0;
@@ -973,6 +1533,7 @@ static int mtk_jpeg_release(struct file *file)
 
        mutex_lock(&jpeg->lock);
        v4l2_m2m_ctx_release(ctx->fh.m2m_ctx);
+       v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
        v4l2_fh_del(&ctx->fh);
        v4l2_fh_exit(&ctx->fh);
        kfree(ctx);
@@ -980,6 +1541,15 @@ static int mtk_jpeg_release(struct file *file)
        return 0;
 }
 
+static const struct v4l2_file_operations mtk_jpeg_enc_fops = {
+       .owner          = THIS_MODULE,
+       .open           = mtk_jpeg_enc_open,
+       .release        = mtk_jpeg_release,
+       .poll           = v4l2_m2m_fop_poll,
+       .unlocked_ioctl = video_ioctl2,
+       .mmap           = v4l2_m2m_fop_mmap,
+};
+
 static const struct v4l2_file_operations mtk_jpeg_dec_fops = {
        .owner          = THIS_MODULE,
        .open           = mtk_jpeg_dec_open,
@@ -993,6 +1563,7 @@ static int mtk_jpeg_clk_init(struct mtk_jpeg_dev *jpeg)
 {
        struct device_node *node;
        struct platform_device *pdev;
+       int i;
 
        node = of_parse_phandle(jpeg->dev->of_node, "mediatek,larb", 0);
        if (!node)
@@ -1006,12 +1577,17 @@ static int mtk_jpeg_clk_init(struct mtk_jpeg_dev *jpeg)
 
        jpeg->larb = &pdev->dev;
 
-       jpeg->clk_jdec = devm_clk_get(jpeg->dev, "jpgdec");
-       if (IS_ERR(jpeg->clk_jdec))
-               return PTR_ERR(jpeg->clk_jdec);
+       for (i = 0; i < jpeg->variant->num_clocks; i++) {
+               jpeg->clocks[i] = devm_clk_get(jpeg->dev,
+                                              jpeg->variant->clk_names[i]);
+               if (IS_ERR(jpeg->clocks[i])) {
+                       dev_err(&pdev->dev, "failed to get clock: %s\n",
+                               jpeg->variant->clk_names[i]);
+                       return PTR_ERR(jpeg->clocks[i]);
+               }
+       }
 
-       jpeg->clk_jdec_smi = devm_clk_get(jpeg->dev, "jpgdec-smi");
-       return PTR_ERR_OR_ZERO(jpeg->clk_jdec_smi);
+       return 0;
 }
 
 static int mtk_jpeg_probe(struct platform_device *pdev)
@@ -1028,6 +1604,7 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
        mutex_init(&jpeg->lock);
        spin_lock_init(&jpeg->hw_lock);
        jpeg->dev = &pdev->dev;
+       jpeg->variant = of_device_get_match_data(jpeg->dev);
 
        res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
        jpeg->reg_base = devm_ioremap_resource(&pdev->dev, res);
@@ -1042,8 +1619,12 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
                return jpeg_irq;
        }
 
-       ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_dec_irq, 0,
-                              pdev->name, jpeg);
+       if (jpeg->variant->is_encoder)
+               ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_enc_irq,
+                                      0, pdev->name, jpeg);
+       else
+               ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_dec_irq,
+                                      0, pdev->name, jpeg);
        if (ret) {
                dev_err(&pdev->dev, "Failed to request jpeg_irq %d (%d)\n",
                        jpeg_irq, ret);
@@ -1063,7 +1644,10 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
                goto err_dev_register;
        }
 
-       jpeg->m2m_dev = v4l2_m2m_init(&mtk_jpeg_dec_m2m_ops);
+       if (jpeg->variant->is_encoder)
+               jpeg->m2m_dev = v4l2_m2m_init(&mtk_jpeg_enc_m2m_ops);
+       else
+               jpeg->m2m_dev = v4l2_m2m_init(&mtk_jpeg_dec_m2m_ops);
        if (IS_ERR(jpeg->m2m_dev)) {
                v4l2_err(&jpeg->v4l2_dev, "Failed to init mem2mem device\n");
                ret = PTR_ERR(jpeg->m2m_dev);
@@ -1076,9 +1660,15 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
                goto err_vfd_jpeg_alloc;
        }
        snprintf(jpeg->vdev->name, sizeof(jpeg->vdev->name),
-                "%s-dec", MTK_JPEG_NAME);
-       jpeg->vdev->fops = &mtk_jpeg_dec_fops;
-       jpeg->vdev->ioctl_ops = &mtk_jpeg_dec_ioctl_ops;
+                "%s-%s", MTK_JPEG_NAME,
+                jpeg->variant->is_encoder ? "enc" : "dec");
+       if (jpeg->variant->is_encoder) {
+               jpeg->vdev->fops = &mtk_jpeg_enc_fops;
+               jpeg->vdev->ioctl_ops = &mtk_jpeg_enc_ioctl_ops;
+       } else {
+               jpeg->vdev->fops = &mtk_jpeg_dec_fops;
+               jpeg->vdev->ioctl_ops = &mtk_jpeg_dec_ioctl_ops;
+       }
        jpeg->vdev->minor = -1;
        jpeg->vdev->release = video_device_release;
        jpeg->vdev->lock = &jpeg->lock;
@@ -1095,8 +1685,9 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 
        video_set_drvdata(jpeg->vdev, jpeg);
        v4l2_info(&jpeg->v4l2_dev,
-                 "decoder device registered as /dev/video%d (%d,%d)\n",
-                 jpeg->vdev->num, VIDEO_MAJOR, jpeg->vdev->minor);
+                 "jpeg %s device registered as /dev/video%d (%d,%d)\n",
+                 jpeg->variant->is_encoder ? "enc" : "dec", jpeg->vdev->num,
+                 VIDEO_MAJOR, jpeg->vdev->minor);
 
        platform_set_drvdata(pdev, jpeg);
 
@@ -1187,14 +1778,36 @@ static const struct dev_pm_ops mtk_jpeg_pm_ops = {
        SET_RUNTIME_PM_OPS(mtk_jpeg_pm_suspend, mtk_jpeg_pm_resume, NULL)
 };
 
Quoted 7 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static const struct dev_pm_ops mtk_jpeg_pm_ops = { · line 1785

Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -0
+static struct mtk_jpeg_variant mt8173_jpeg_drvdata = {
+       .is_encoder     = false,
+       .clk_names      = {"jpgdec-smi", "jpgdec"},
+       .num_clocks     = 2,
+};

I would like non-jpeg-enc related modification to be another patch.

done.

Quoted 10 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static const struct dev_pm_ops mtk_jpeg_pm_ops = { · line 1791

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
+
+static struct mtk_jpeg_variant mt2701_jpeg_drvdata = {
+       .is_encoder     = false,
+       .clk_names      = {"jpgdec-smi", "jpgdec"},
+       .num_clocks     = 2,
+};

mt2701_jpeg_drvdata is equal to mt8173_jpeg_drvdata, so keep only mt8173_jpeg_drvdata.

done.

Quoted 52 lines
Quoted 52 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +13 -2
+
+static struct mtk_jpeg_variant mtk_jpeg_drvdata = {
+       .is_encoder     = true,
+       .clk_names      = {"jpgenc"},
+       .num_clocks     = 1,
+};
+
 static const struct of_device_id mtk_jpeg_match[] = {
        {
                .compatible = "mediatek,mt8173-jpgdec",
-               .data       = NULL,
+               .data = &mt8173_jpeg_drvdata,
        },
        {
                .compatible = "mediatek,mt2701-jpgdec",
-               .data       = NULL,
+               .data = &mt2701_jpeg_drvdata,
+       },
+       {
+               .compatible = "mediatek,mtk-jpgenc",
+               .data = &mtk_jpeg_drvdata,
        },
        {},
 };
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +5 -3
index 0b59e48495d5..9ec2c3350a16 100644
@@ -3,6 +3,7 @@
  * Copyright (c) 2016 MediaTek Inc.
  * Author: Ming Hsiu Tsai <[email protected]>
  *         Rick Chang <[email protected]>
+ *         Xia Jiang <[email protected]>
  */
 
 #ifndef _MTK_JPEG_CORE_H
@@ -16,19 +17,21 @@
 #define MTK_JPEG_NAME          "mtk-jpeg"
 
 #define MTK_JPEG_COMP_MAX              3
+#define MTK_JPEG_MAX_CLOCKS            2
+
 
 #define MTK_JPEG_FMT_FLAG_DEC_OUTPUT   BIT(0)
 #define MTK_JPEG_FMT_FLAG_DEC_CAPTURE  BIT(1)
-
-#define MTK_JPEG_FMT_TYPE_OUTPUT       1
-#define MTK_JPEG_FMT_TYPE_CAPTURE      2
+#define MTK_JPEG_FMT_FLAG_ENC_OUTPUT   BIT(2)
+#define MTK_JPEG_FMT_FLAG_ENC_CAPTURE  BIT(3)
 
Quoted 9 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 31

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +2 -2
 #define MTK_JPEG_MIN_WIDTH     32U
 #define MTK_JPEG_MIN_HEIGHT    32U
-#define MTK_JPEG_MAX_WIDTH     8192U
-#define MTK_JPEG_MAX_HEIGHT    8192U
+#define MTK_JPEG_MAX_WIDTH     65535U
+#define MTK_JPEG_MAX_HEIGHT    65535U

MT8173 use this definition, why do you change this?
If the definition is wrong in MT8173, use a patch to fixup this.

yes,the max width/height is also 65535 for MT8173.

Quoted 11 lines
Quoted 11 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +1 -0
 #define MTK_JPEG_DEFAULT_SIZEIMAGE     (1 * 1024 * 1024)
+#define MTK_JPEG_MAX_EXIF_SIZE (64 * 1024)
 
 /**
  * enum mtk_jpeg_ctx_state - states of the context state machine
@@ -42,6 +45,18 @@ enum mtk_jpeg_ctx_state {
        MTK_JPEG_SOURCE_CHANGE,
 };
 
#32 lore raw

Re: [PATCH RESEND v9 18/18] media: platform: Add jpeg enc feature

In reply to Xia Jiang
Quoted 284 lines · Xia Jiang

Add mtk jpeg encode v4l2 driver based on jpeg decode, because that jpeg decode and encode have great similarities with function operation.

---
v9: add member variable(struct v4l2_rect) in out_q structure for storing
the active crop information.
move the renaming exsting functions/defines/variables to a separate patch.

---
 drivers/media/platform/mtk-jpeg/Makefile      |   5 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 845 +++++++++++++++---
 .../media/platform/mtk-jpeg/mtk_jpeg_core.h   |  44 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c | 193 ++++
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h | 123 +++
 5 files changed, 1084 insertions(+), 126 deletions(-)
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h
drivers/media/platform/mtk-jpeg/Makefile +4 -1
index 48516dcf96e6..76c33aad0f3f 100644
@@ -1,3 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0-only
-mtk_jpeg-objs := mtk_jpeg_core.o mtk_jpeg_dec_hw.o mtk_jpeg_dec_parse.o
+mtk_jpeg-objs := mtk_jpeg_core.o \
+		 mtk_jpeg_dec_hw.o \
+		 mtk_jpeg_dec_parse.o \
+		 mtk_jpeg_enc_hw.o
 obj-$(CONFIG_VIDEO_MEDIATEK_JPEG) += mtk_jpeg.o
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +144 -30
index 29b8b82c606c..d7ef69920530 100644
@@ -3,6 +3,7 @@
  * Copyright (c) 2016 MediaTek Inc.
  * Author: Ming Hsiu Tsai <[email protected]>
  *         Rick Chang <[email protected]>
+ *         Xia Jiang <[email protected]>
  */
 
 #include <linux/clk.h>
@@ -23,10 +24,59 @@
 #include <media/videobuf2-dma-contig.h>
 #include <soc/mediatek/smi.h>
 
+#include "mtk_jpeg_enc_hw.h"
 #include "mtk_jpeg_dec_hw.h"
 #include "mtk_jpeg_core.h"
 #include "mtk_jpeg_dec_parse.h"
 
+static struct mtk_jpeg_fmt mtk_jpeg_enc_formats[] = {
+	{
+		.fourcc		= V4L2_PIX_FMT_JPEG,
+		.colplanes	= 1,
+		.flags		= MTK_JPEG_FMT_FLAG_ENC_CAPTURE,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_NV12M,
+		.hw_format	= JPEG_ENC_YUV_FORMAT_NV12,
+		.h_sample	= {4, 4},
+		.v_sample	= {4, 2},
+		.colplanes	= 2,
+		.h_align	= 4,
+		.v_align	= 4,
+		.flags		= MTK_JPEG_FMT_FLAG_ENC_OUTPUT,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_NV21M,
+		.hw_format	= JEPG_ENC_YUV_FORMAT_NV21,
+		.h_sample	= {4, 4},
+		.v_sample	= {4, 2},
+		.colplanes	= 2,
+		.h_align	= 4,
+		.v_align	= 4,
+		.flags		= MTK_JPEG_FMT_FLAG_ENC_OUTPUT,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_YUYV,
+		.hw_format	= JPEG_ENC_YUV_FORMAT_YUYV,
+		.h_sample	= {8},
+		.v_sample	= {4},
+		.colplanes	= 1,
+		.h_align	= 5,
+		.v_align	= 3,
+		.flags		= MTK_JPEG_FMT_FLAG_ENC_OUTPUT,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_YVYU,
+		.hw_format	= JPEG_ENC_YUV_FORMAT_YVYU,
+		.h_sample	= {8},
+		.v_sample	= {4},
+		.colplanes	= 1,
+		.h_align	= 5,
+		.v_align	= 3,
+		.flags		= MTK_JPEG_FMT_FLAG_ENC_OUTPUT,
+	},
+};
+
 static struct mtk_jpeg_fmt mtk_jpeg_dec_formats[] = {
 	{
 		.fourcc		= V4L2_PIX_FMT_JPEG,
@@ -53,6 +103,7 @@ static struct mtk_jpeg_fmt mtk_jpeg_dec_formats[] = {
 	},
 };
 
+#define MTK_JPEG_ENC_NUM_FORMATS ARRAY_SIZE(mtk_jpeg_enc_formats)
 #define MTK_JPEG_DEC_NUM_FORMATS ARRAY_SIZE(mtk_jpeg_dec_formats)
 
 struct mtk_jpeg_src_buf {
@@ -64,6 +115,11 @@ struct mtk_jpeg_src_buf {
 static int debug;
 module_param(debug, int, 0644);
 
+static inline struct mtk_jpeg_ctx *ctrl_to_ctx(struct v4l2_ctrl *ctrl)
+{
+	return container_of(ctrl->handler, struct mtk_jpeg_ctx, ctrl_hdl);
+}
+
 static inline struct mtk_jpeg_ctx *mtk_jpeg_fh_to_ctx(struct v4l2_fh *fh)
 {
 	return container_of(fh, struct mtk_jpeg_ctx, fh);
@@ -75,6 +131,19 @@ static inline struct mtk_jpeg_src_buf *mtk_jpeg_vb2_to_srcbuf(
 	return container_of(to_vb2_v4l2_buffer(vb), struct mtk_jpeg_src_buf, b);
 }
 
+static int mtk_jpeg_enc_querycap(struct file *file, void *priv,
+				 struct v4l2_capability *cap)
+{
+	struct mtk_jpeg_dev *jpeg = video_drvdata(file);
+
+	strscpy(cap->driver, MTK_JPEG_NAME, sizeof(cap->driver));
+	strscpy(cap->card, MTK_JPEG_NAME " encoder", sizeof(cap->card));
+	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
+		 dev_name(jpeg->dev));
+
+	return 0;
+}
+
 static int mtk_jpeg_dec_querycap(struct file *file, void *priv,
 				 struct v4l2_capability *cap)
 {
@@ -88,6 +157,54 @@ static int mtk_jpeg_dec_querycap(struct file *file, void *priv,
 	return 0;
 }
 
+static int vidioc_jpeg_enc_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_jpeg_ctx *ctx = ctrl_to_ctx(ctrl);
+
+	switch (ctrl->id) {
+	case V4L2_CID_JPEG_RESTART_INTERVAL:
+		ctx->restart_interval = ctrl->val;
+		break;
+	case V4L2_CID_JPEG_COMPRESSION_QUALITY:
+		ctx->enc_quality = ctrl->val;
+		break;
+	case V4L2_CID_JPEG_ACTIVE_MARKER:
+		ctx->enable_exif = ctrl->val & V4L2_JPEG_ACTIVE_MARKER_APP1 ?
+				   true : false;
+		break;
+	}
+
+	return 0;
+}
+
+static const struct v4l2_ctrl_ops mtk_jpeg_enc_ctrl_ops = {
+	.s_ctrl = vidioc_jpeg_enc_s_ctrl,
+};
+
+static int mtk_jpeg_enc_ctrls_setup(struct mtk_jpeg_ctx *ctx)
+{
+	const struct v4l2_ctrl_ops *ops = &mtk_jpeg_enc_ctrl_ops;
+	struct v4l2_ctrl_handler *handler = &ctx->ctrl_hdl;
+
+	v4l2_ctrl_handler_init(handler, 3);
+
+	v4l2_ctrl_new_std(handler, ops, V4L2_CID_JPEG_RESTART_INTERVAL, 0, 100,
+			  1, 0);
+	v4l2_ctrl_new_std(handler, ops, V4L2_CID_JPEG_COMPRESSION_QUALITY, 48,
+			  100, 1, 90);
+	v4l2_ctrl_new_std(handler, ops, V4L2_CID_JPEG_ACTIVE_MARKER, 0,
+			  V4L2_JPEG_ACTIVE_MARKER_APP1, 0, 0);
+
+	if (handler->error) {
+		v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
+		return handler->error;
+	}
+
+	v4l2_ctrl_handler_setup(&ctx->ctrl_hdl);
+
+	return 0;
+}
+
 static int mtk_jpeg_enum_fmt(struct mtk_jpeg_fmt *mtk_jpeg_formats, int n,
 			     struct v4l2_fmtdesc *f, u32 type)
 {
@@ -109,6 +226,14 @@ static int mtk_jpeg_enum_fmt(struct mtk_jpeg_fmt *mtk_jpeg_formats, int n,
 	return 0;
 }
 
+static int mtk_jpeg_enc_enum_fmt_vid_cap(struct file *file, void *priv,
+					 struct v4l2_fmtdesc *f)
+{
+	return mtk_jpeg_enum_fmt(mtk_jpeg_enc_formats,
+				 MTK_JPEG_ENC_NUM_FORMATS, f,
+				 MTK_JPEG_FMT_FLAG_ENC_CAPTURE);
+}
+
 static int mtk_jpeg_dec_enum_fmt_vid_cap(struct file *file, void *priv,
 					 struct v4l2_fmtdesc *f)
 {
@@ -117,6 +242,14 @@ static int mtk_jpeg_dec_enum_fmt_vid_cap(struct file *file, void *priv,
 				 MTK_JPEG_FMT_FLAG_DEC_CAPTURE);
 }
 
+static int mtk_jpeg_enc_enum_fmt_vid_out(struct file *file, void *priv,
+					 struct v4l2_fmtdesc *f)
+{
+	return mtk_jpeg_enum_fmt(mtk_jpeg_enc_formats,
+				 MTK_JPEG_ENC_NUM_FORMATS, f,
+				 MTK_JPEG_FMT_FLAG_ENC_OUTPUT);
+}
+
 static int mtk_jpeg_dec_enum_fmt_vid_out(struct file *file, void *priv,
 					 struct v4l2_fmtdesc *f)
 {
@@ -132,93 +265,66 @@ mtk_jpeg_get_q_data(struct mtk_jpeg_ctx *ctx, enum v4l2_buf_type type)
 	return &ctx->cap_q;
 }
 
-static struct mtk_jpeg_fmt *mtk_jpeg_find_format(struct mtk_jpeg_ctx *ctx,
-						 u32 pixelformat,
+static struct mtk_jpeg_fmt *mtk_jpeg_find_format(u32 pixelformat,
 						 unsigned int fmt_type)
 {
-	unsigned int k, fmt_flag;
+	unsigned int k;
+	struct mtk_jpeg_fmt *fmt;
 
-	fmt_flag = (fmt_type == MTK_JPEG_FMT_TYPE_OUTPUT) ?
-		   MTK_JPEG_FMT_FLAG_DEC_OUTPUT :
-		   MTK_JPEG_FMT_FLAG_DEC_CAPTURE;
+	for (k = 0; k < MTK_JPEG_ENC_NUM_FORMATS; k++) {
+		fmt = &mtk_jpeg_enc_formats[k];
+
+		if (fmt->fourcc == pixelformat && fmt->flags & fmt_type)
+			return fmt;
+	}
 
 	for (k = 0; k < MTK_JPEG_DEC_NUM_FORMATS; k++) {
-		struct mtk_jpeg_fmt *fmt = &mtk_jpeg_dec_formats[k];
+		fmt = &mtk_jpeg_dec_formats[k];
 
-		if (fmt->fourcc == pixelformat && fmt->flags & fmt_flag)
+		if (fmt->fourcc == pixelformat && fmt->flags & fmt_type)
 			return fmt;
 	}
 
 	return NULL;
 }
 
-static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx,
-				       struct v4l2_format *f)
-{
-	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
-	struct mtk_jpeg_q_data *q_data;
-	int i;
-
-	q_data = mtk_jpeg_get_q_data(ctx, f->type);
-
-	pix_mp->width = q_data->w;
-	pix_mp->height = q_data->h;
-	pix_mp->pixelformat = q_data->fmt->fourcc;
-	pix_mp->num_planes = q_data->fmt->colplanes;
-
-	for (i = 0; i < pix_mp->num_planes; i++) {
-		pix_mp->plane_fmt[i].bytesperline = q_data->bytesperline[i];
-		pix_mp->plane_fmt[i].sizeimage = q_data->sizeimage[i];
-	}
-}
-
-static int mtk_jpeg_try_fmt_mplane(struct v4l2_format *f,
-				   struct mtk_jpeg_fmt *fmt,

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c mtk_jpeg_get_q_data(struct mtk_jpeg_ctx *ctx, enum v4l2_buf_type type) · line 185

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -4
-				   struct mtk_jpeg_ctx *ctx, int q_type)
+static int vidioc_try_fmt(struct v4l2_format *f, struct mtk_jpeg_fmt *fmt)
 {
 	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
 	int i;
 
 	pix_mp->field = V4L2_FIELD_NONE;
-
-	if (ctx->state != MTK_JPEG_INIT) {
-		mtk_jpeg_adjust_fmt_mplane(ctx, f);

The removal of this function seems unrelated to adding the jpeg enc feature. It's not obvious to me why this is no longer needed and what replaces it.

I think this is something that should be split off to its own patch.

Quoted 100 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +45 -29
-		return 0;
-	}
-
 	pix_mp->num_planes = fmt->colplanes;
 	pix_mp->pixelformat = fmt->fourcc;
 
-	if (q_type == MTK_JPEG_FMT_TYPE_OUTPUT) {
-		struct v4l2_plane_pix_format *pfmt = &pix_mp->plane_fmt[0];
-
+	if (fmt->fourcc == V4L2_PIX_FMT_JPEG) {
 		pix_mp->height = clamp(pix_mp->height, MTK_JPEG_MIN_HEIGHT,
 				       MTK_JPEG_MAX_HEIGHT);
 		pix_mp->width = clamp(pix_mp->width, MTK_JPEG_MIN_WIDTH,
 				      MTK_JPEG_MAX_WIDTH);
-
-		pfmt->bytesperline = 0;
-		/* Source size must be aligned to 128 */
-		pfmt->sizeimage = round_up(pfmt->sizeimage, 128);
-		if (pfmt->sizeimage == 0)
-			pfmt->sizeimage = MTK_JPEG_DEFAULT_SIZEIMAGE;
-		return 0;
+		pix_mp->plane_fmt[0].bytesperline = 0;
+		pix_mp->plane_fmt[0].sizeimage =
+				round_up(pix_mp->plane_fmt[0].sizeimage, 128);
+		if (pix_mp->plane_fmt[0].sizeimage == 0)
+			pix_mp->plane_fmt[0].sizeimage =
+				MTK_JPEG_DEFAULT_SIZEIMAGE;
+	} else {
+		pix_mp->height = clamp(round_up(pix_mp->height, fmt->v_align),
+				       MTK_JPEG_MIN_HEIGHT,
+				       MTK_JPEG_MAX_HEIGHT);
+		pix_mp->width = clamp(round_up(pix_mp->width, fmt->h_align),
+				      MTK_JPEG_MIN_WIDTH, MTK_JPEG_MAX_WIDTH);
+		for (i = 0; i < pix_mp->num_planes; i++) {
+			struct v4l2_plane_pix_format *pfmt =
+							&pix_mp->plane_fmt[i];
+			u32 stride = pix_mp->width * fmt->h_sample[i] / 4;
+			u32 h = pix_mp->height * fmt->v_sample[i] / 4;
+
+			pfmt->bytesperline = stride;
+			pfmt->sizeimage = stride * h;
+		}
 	}
 
-	/* type is MTK_JPEG_FMT_TYPE_CAPTURE */
-	pix_mp->height = clamp(round_up(pix_mp->height, fmt->v_align),
-			       MTK_JPEG_MIN_HEIGHT, MTK_JPEG_MAX_HEIGHT);
-	pix_mp->width = clamp(round_up(pix_mp->width, fmt->h_align),
-			      MTK_JPEG_MIN_WIDTH, MTK_JPEG_MAX_WIDTH);
-
-	for (i = 0; i < fmt->colplanes; i++) {
-		struct v4l2_plane_pix_format *pfmt = &pix_mp->plane_fmt[i];
-		u32 stride = pix_mp->width * fmt->h_sample[i] / 4;
-		u32 h = pix_mp->height * fmt->v_sample[i] / 4;
-
-		pfmt->bytesperline = stride;
-		pfmt->sizeimage = stride * h;
-	}
 	return 0;
 }
 
@@ -271,14 +377,35 @@ static int mtk_jpeg_g_fmt_vid_mplane(struct file *file, void *priv,
 	return 0;
 }
 
+static int mtk_jpeg_enc_try_fmt_vid_cap_mplane(struct file *file, void *priv,
+					       struct v4l2_format *f)
+{
+	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+	struct mtk_jpeg_fmt *fmt;
+
+	fmt = mtk_jpeg_find_format(f->fmt.pix_mp.pixelformat,
+				   MTK_JPEG_FMT_FLAG_ENC_CAPTURE);
+	if (!fmt)
+		fmt = ctx->cap_q.fmt;
+
+	v4l2_dbg(2, debug, &ctx->jpeg->v4l2_dev, "(%d) try_fmt:%c%c%c%c\n",
+		 f->type,
+		 (fmt->fourcc & 0xff),
+		 (fmt->fourcc >>  8 & 0xff),
+		 (fmt->fourcc >> 16 & 0xff),
+		 (fmt->fourcc >> 24 & 0xff));
+
+	return vidioc_try_fmt(f, fmt);
+}
+
 static int mtk_jpeg_dec_try_fmt_vid_cap_mplane(struct file *file, void *priv,
 					       struct v4l2_format *f)
 {
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
 	struct mtk_jpeg_fmt *fmt;
 
-	fmt = mtk_jpeg_find_format(ctx, f->fmt.pix_mp.pixelformat,
-				   MTK_JPEG_FMT_TYPE_CAPTURE);
+	fmt = mtk_jpeg_find_format(f->fmt.pix_mp.pixelformat,
+				   MTK_JPEG_FMT_FLAG_DEC_CAPTURE);
 	if (!fmt)
 		fmt = ctx->cap_q.fmt;
 

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_try_fmt_vid_cap_mplane(struct file *file, void *priv, · line 422

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -1
@@ -289,7 +416,33 @@ static int mtk_jpeg_dec_try_fmt_vid_cap_mplane(struct file *file, void *priv,
 		 (fmt->fourcc >> 16 & 0xff),
 		 (fmt->fourcc >> 24 & 0xff));
 
-	return mtk_jpeg_try_fmt_mplane(f, fmt, ctx, MTK_JPEG_FMT_TYPE_CAPTURE);
+	if (ctx->state != MTK_JPEG_INIT) {
+		mtk_jpeg_g_fmt_vid_mplane(file, priv, f);
+		return 0;
+	}

This is probably related to the same mtk_jpeg_adjust_fmt_mplane() removal.

Quoted 38 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +25 -2
+
+	return vidioc_try_fmt(f, fmt);
+}
+
+static int mtk_jpeg_enc_try_fmt_vid_out_mplane(struct file *file, void *priv,
+					       struct v4l2_format *f)
+{
+	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+	struct mtk_jpeg_fmt *fmt;
+
+	fmt = mtk_jpeg_find_format(f->fmt.pix_mp.pixelformat,
+				   MTK_JPEG_FMT_FLAG_ENC_OUTPUT);
+	if (!fmt)
+		fmt = ctx->out_q.fmt;
+
+	v4l2_dbg(2, debug, &ctx->jpeg->v4l2_dev, "(%d) try_fmt:%c%c%c%c\n",
+		 f->type,
+		 (fmt->fourcc & 0xff),
+		 (fmt->fourcc >>  8 & 0xff),
+		 (fmt->fourcc >> 16 & 0xff),
+		 (fmt->fourcc >> 24 & 0xff));
+
+	return vidioc_try_fmt(f, fmt);
 }
 
 static int mtk_jpeg_dec_try_fmt_vid_out_mplane(struct file *file, void *priv,
@@ -298,8 +451,8 @@ static int mtk_jpeg_dec_try_fmt_vid_out_mplane(struct file *file, void *priv,
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
 	struct mtk_jpeg_fmt *fmt;
 
-	fmt = mtk_jpeg_find_format(ctx, f->fmt.pix_mp.pixelformat,
-				   MTK_JPEG_FMT_TYPE_OUTPUT);
+	fmt = mtk_jpeg_find_format(f->fmt.pix_mp.pixelformat,
+				   MTK_JPEG_FMT_FLAG_DEC_OUTPUT);
 	if (!fmt)
 		fmt = ctx->out_q.fmt;
 

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_try_fmt_vid_out_mplane(struct file *file, void *priv, · line 469

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -1
@@ -310,17 +463,21 @@ static int mtk_jpeg_dec_try_fmt_vid_out_mplane(struct file *file, void *priv,
 		 (fmt->fourcc >> 16 & 0xff),
 		 (fmt->fourcc >> 24 & 0xff));
 
-	return mtk_jpeg_try_fmt_mplane(f, fmt, ctx, MTK_JPEG_FMT_TYPE_OUTPUT);
+	if (ctx->state != MTK_JPEG_INIT) {
+		mtk_jpeg_g_fmt_vid_mplane(file, priv, f);
+		return 0;
+	}

Ditto.

Quoted 93 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +40 -8
+
+	return vidioc_try_fmt(f, fmt);
 }
 
 static int mtk_jpeg_s_fmt_mplane(struct mtk_jpeg_ctx *ctx,
-				 struct v4l2_format *f)
+				 struct v4l2_format *f, unsigned int fmt_type)
 {
 	struct vb2_queue *vq;
 	struct mtk_jpeg_q_data *q_data = NULL;
 	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
 	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
-	unsigned int f_type;
 	int i;
 
 	vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, f->type);
@@ -334,12 +491,11 @@ static int mtk_jpeg_s_fmt_mplane(struct mtk_jpeg_ctx *ctx,
 		return -EBUSY;
 	}
 
-	f_type = V4L2_TYPE_IS_OUTPUT(f->type) ?
-			 MTK_JPEG_FMT_TYPE_OUTPUT : MTK_JPEG_FMT_TYPE_CAPTURE;
-
-	q_data->fmt = mtk_jpeg_find_format(ctx, pix_mp->pixelformat, f_type);
+	q_data->fmt = mtk_jpeg_find_format(pix_mp->pixelformat, fmt_type);
 	q_data->w = pix_mp->width;
 	q_data->h = pix_mp->height;
+	q_data->crop_rect.width = pix_mp->width;
+	q_data->crop_rect.height = pix_mp->height;
 	ctx->colorspace = pix_mp->colorspace;
 	ctx->ycbcr_enc = pix_mp->ycbcr_enc;
 	ctx->xfer_func = pix_mp->xfer_func;
@@ -365,6 +521,19 @@ static int mtk_jpeg_s_fmt_mplane(struct mtk_jpeg_ctx *ctx,
 	return 0;
 }
 
+static int mtk_jpeg_enc_s_fmt_vid_out_mplane(struct file *file, void *priv,
+					     struct v4l2_format *f)
+{
+	int ret;
+
+	ret = mtk_jpeg_enc_try_fmt_vid_out_mplane(file, priv, f);
+	if (ret)
+		return ret;
+
+	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f,
+				     MTK_JPEG_FMT_FLAG_ENC_OUTPUT);
+}
+
 static int mtk_jpeg_dec_s_fmt_vid_out_mplane(struct file *file, void *priv,
 					     struct v4l2_format *f)
 {
@@ -374,7 +543,21 @@ static int mtk_jpeg_dec_s_fmt_vid_out_mplane(struct file *file, void *priv,
 	if (ret)
 		return ret;
 
-	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f);
+	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f,
+				     MTK_JPEG_FMT_FLAG_DEC_OUTPUT);
+}
+
+static int mtk_jpeg_enc_s_fmt_vid_cap_mplane(struct file *file, void *priv,
+					     struct v4l2_format *f)
+{
+	int ret;
+
+	ret = mtk_jpeg_enc_try_fmt_vid_cap_mplane(file, priv, f);
+	if (ret)
+		return ret;
+
+	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f,
+				     MTK_JPEG_FMT_FLAG_ENC_CAPTURE);
 }
 
 static int mtk_jpeg_dec_s_fmt_vid_cap_mplane(struct file *file, void *priv,
@@ -386,7 +569,8 @@ static int mtk_jpeg_dec_s_fmt_vid_cap_mplane(struct file *file, void *priv,
 	if (ret)
 		return ret;
 
-	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f);
+	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f,
+				     MTK_JPEG_FMT_FLAG_DEC_CAPTURE);
 }
 
 static void mtk_jpeg_queue_src_chg_event(struct mtk_jpeg_ctx *ctx)
@@ -411,6 +595,29 @@ static int mtk_jpeg_subscribe_event(struct v4l2_fh *fh,
 	return v4l2_ctrl_subscribe_event(fh, sub);
 }
 
+static int mtk_jpeg_enc_g_selection(struct file *file, void *priv,
+				    struct v4l2_selection *s)
+{
+	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_subscribe_event(struct v4l2_fh *fh, · line 611

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+
+	if (s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)
+		return -EINVAL;
+
+	switch (s->target) {
+	case V4L2_SEL_TGT_CROP:
+	case V4L2_SEL_TGT_CROP_BOUNDS:
+	case V4L2_SEL_TGT_CROP_DEFAULT:
+		s->r.width = ctx->out_q.w;
+		s->r.height = ctx->out_q.h;

If TGT_CROP can be set, then g_selection should return ctx->out_q.crop_rect for that target, and not ctx->out_q.w/h. That's for CROP_BOUNDS/CROP_DEFAULT only.

Quoted 18 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +11 -0
+		s->r.left = 0;
+		s->r.top = 0;
+		break;
+	default:
+		return -EINVAL;
+	}
+	return 0;
+}
+
 static int mtk_jpeg_dec_g_selection(struct file *file, void *priv,
 				    struct v4l2_selection *s)
 {
@@ -440,6 +647,29 @@ static int mtk_jpeg_dec_g_selection(struct file *file, void *priv,
 	return 0;
 }
 
+static int mtk_jpeg_enc_s_selection(struct file *file, void *priv,
+				    struct v4l2_selection *s)

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_g_selection(struct file *file, void *priv, · line 661

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+{
+	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+
+	if (s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)
+		return -EINVAL;
+
+	switch (s->target) {
+	case V4L2_SEL_TGT_CROP:
+		s->r.left = 0;
+		s->r.top = 0;

Is it a hardware limitation that left/top can only be 0?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_g_selection(struct file *file, void *priv, · line 664

Quoted 3 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -0
+		s->r.width = min(s->r.width, ctx->out_q.w);
+		s->r.height = min(s->r.height, ctx->out_q.h);
+		ctx->out_q.crop_rect = s->r;

I'd rename crop_rect to enc_crop_rect.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_g_selection(struct file *file, void *priv, · line 675

Quoted 11 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +8 -0
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
 static int mtk_jpeg_dec_s_selection(struct file *file, void *priv,
 				    struct v4l2_selection *s)
 {

I also looked at the current decoder selection code and I see no reason why s_selection is supported at all for the decoder since you can't actually set a compose rectangle, whatever you pass is just ignored.

I would recommend that mtk_jpeg_dec_s_selection() is just removed in a separate patch. The presence of that ioctl suggests that you can set the compose rectangle when in reality you can't. That can be confusing for applications.

Quoted 334 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +240 -10
@@ -484,6 +714,33 @@ static int mtk_jpeg_qbuf(struct file *file, void *priv, struct v4l2_buffer *buf)
 	return v4l2_m2m_qbuf(file, fh->m2m_ctx, buf);
 }
 
+static const struct v4l2_ioctl_ops mtk_jpeg_enc_ioctl_ops = {
+	.vidioc_querycap                = mtk_jpeg_enc_querycap,
+	.vidioc_enum_fmt_vid_cap	= mtk_jpeg_enc_enum_fmt_vid_cap,
+	.vidioc_enum_fmt_vid_out	= mtk_jpeg_enc_enum_fmt_vid_out,
+	.vidioc_try_fmt_vid_cap_mplane	= mtk_jpeg_enc_try_fmt_vid_cap_mplane,
+	.vidioc_try_fmt_vid_out_mplane	= mtk_jpeg_enc_try_fmt_vid_out_mplane,
+	.vidioc_g_fmt_vid_cap_mplane    = mtk_jpeg_g_fmt_vid_mplane,
+	.vidioc_g_fmt_vid_out_mplane    = mtk_jpeg_g_fmt_vid_mplane,
+	.vidioc_s_fmt_vid_cap_mplane    = mtk_jpeg_enc_s_fmt_vid_cap_mplane,
+	.vidioc_s_fmt_vid_out_mplane    = mtk_jpeg_enc_s_fmt_vid_out_mplane,
+	.vidioc_qbuf                    = mtk_jpeg_qbuf,
+	.vidioc_subscribe_event         = mtk_jpeg_subscribe_event,
+	.vidioc_g_selection		= mtk_jpeg_enc_g_selection,
+	.vidioc_s_selection		= mtk_jpeg_enc_s_selection,
+
+	.vidioc_create_bufs		= v4l2_m2m_ioctl_create_bufs,
+	.vidioc_prepare_buf		= v4l2_m2m_ioctl_prepare_buf,
+	.vidioc_reqbufs                 = v4l2_m2m_ioctl_reqbufs,
+	.vidioc_querybuf                = v4l2_m2m_ioctl_querybuf,
+	.vidioc_dqbuf                   = v4l2_m2m_ioctl_dqbuf,
+	.vidioc_expbuf                  = v4l2_m2m_ioctl_expbuf,
+	.vidioc_streamon                = v4l2_m2m_ioctl_streamon,
+	.vidioc_streamoff               = v4l2_m2m_ioctl_streamoff,
+
+	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
+};
+
 static const struct v4l2_ioctl_ops mtk_jpeg_dec_ioctl_ops = {
 	.vidioc_querycap                = mtk_jpeg_dec_querycap,
 	.vidioc_enum_fmt_vid_cap	= mtk_jpeg_dec_enum_fmt_vid_cap,
@@ -575,8 +832,9 @@ static bool mtk_jpeg_check_resolution_change(struct mtk_jpeg_ctx *ctx,
 	}
 
 	q_data = &ctx->cap_q;
-	if (q_data->fmt != mtk_jpeg_find_format(ctx, param->dst_fourcc,
-						MTK_JPEG_FMT_TYPE_CAPTURE)) {
+	if (q_data->fmt !=
+	    mtk_jpeg_find_format(param->dst_fourcc,
+				 MTK_JPEG_FMT_FLAG_DEC_CAPTURE)) {
 		v4l2_dbg(1, debug, &jpeg->v4l2_dev, "format change\n");
 		return true;
 	}
@@ -597,9 +855,8 @@ static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx,
 	q_data = &ctx->cap_q;
 	q_data->w = param->dec_w;
 	q_data->h = param->dec_h;
-	q_data->fmt = mtk_jpeg_find_format(ctx,
-					   param->dst_fourcc,
-					   MTK_JPEG_FMT_TYPE_CAPTURE);
+	q_data->fmt = mtk_jpeg_find_format(param->dst_fourcc,
+					   MTK_JPEG_FMT_FLAG_DEC_CAPTURE);
 
 	for (i = 0; i < q_data->fmt->colplanes; i++) {
 		q_data->bytesperline[i] = param->mem_stride[i];
@@ -616,6 +873,17 @@ static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx,
 		 param->dec_w, param->dec_h);
 }
 
+static void mtk_jpeg_enc_buf_queue(struct vb2_buffer *vb)
+{
+	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+
+	v4l2_dbg(2, debug, &jpeg->v4l2_dev, "(%d) buf_q id=%d, vb=%p\n",
+		 vb->vb2_queue->type, vb->index, vb);
+
+	v4l2_m2m_buf_queue(ctx->fh.m2m_ctx, to_vb2_v4l2_buffer(vb));
+}
+
 static void mtk_jpeg_dec_buf_queue(struct vb2_buffer *vb)
 {
 	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
@@ -664,6 +932,15 @@ static struct vb2_v4l2_buffer *mtk_jpeg_buf_remove(struct mtk_jpeg_ctx *ctx,
 		return v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
 }
 
+static void mtk_jpeg_enc_stop_streaming(struct vb2_queue *q)
+{
+	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(q);
+	struct vb2_v4l2_buffer *vb;
+
+	while ((vb = mtk_jpeg_buf_remove(ctx, q->type)))
+		v4l2_m2m_buf_done(vb, VB2_BUF_STATE_ERROR);
+}
+
 static void mtk_jpeg_dec_stop_streaming(struct vb2_queue *q)
 {
 	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(q);
@@ -699,6 +976,15 @@ static const struct vb2_ops mtk_jpeg_dec_qops = {
 	.stop_streaming     = mtk_jpeg_dec_stop_streaming,
 };
 
+static const struct vb2_ops mtk_jpeg_enc_qops = {
+	.queue_setup        = mtk_jpeg_queue_setup,
+	.buf_prepare        = mtk_jpeg_buf_prepare,
+	.buf_queue          = mtk_jpeg_enc_buf_queue,
+	.wait_prepare       = vb2_ops_wait_prepare,
+	.wait_finish        = vb2_ops_wait_finish,
+	.stop_streaming     = mtk_jpeg_enc_stop_streaming,
+};
+
 static void mtk_jpeg_set_dec_src(struct mtk_jpeg_ctx *ctx,
 				 struct vb2_buffer *src_buf,
 				 struct mtk_jpeg_bs *bs)
@@ -736,6 +1022,85 @@ static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx,
 	return 0;
 }
 
+static void mtk_jpeg_set_enc_dst(struct mtk_jpeg_ctx *ctx, void __iomem *base,
+				 struct vb2_buffer *dst_buf,
+				 struct mtk_jpeg_enc_bs *bs)
+{
+	bs->dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
+	bs->dma_addr_offset = ctx->enable_exif ? MTK_JPEG_MAX_EXIF_SIZE : 0;
+	bs->dma_addr_offsetmask = bs->dma_addr & JPEG_ENC_DST_ADDR_OFFSET_MASK;
+	bs->size = vb2_plane_size(dst_buf, 0);
+
+	mtk_jpeg_enc_set_dst_addr(base, bs->dma_addr, bs->size,
+				  bs->dma_addr_offset,
+				  bs->dma_addr_offsetmask);
+}
+
+static void mtk_jpeg_set_enc_src(struct mtk_jpeg_ctx *ctx, void __iomem *base,
+				 struct vb2_buffer *src_buf)
+{
+	int i;
+	dma_addr_t	dma_addr;
+
+	mtk_jpeg_enc_set_img_size(base, ctx->out_q.crop_rect.width,
+				  ctx->out_q.crop_rect.height);
+	mtk_jpeg_enc_set_blk_num(base, ctx->out_q.fmt->fourcc,
+				 ctx->out_q.crop_rect.width,
+				 ctx->out_q.crop_rect.height);
+	mtk_jpeg_enc_set_stride(base, ctx->out_q.fmt->fourcc, ctx->out_q.w,
+				ctx->out_q.h, ctx->out_q.bytesperline[0]);
+
+	for (i = 0; i < src_buf->num_planes; i++) {
+		dma_addr = vb2_dma_contig_plane_dma_addr(src_buf, i) +
+			   src_buf->planes[i].data_offset;
+		mtk_jpeg_enc_set_src_addr(base, dma_addr, i);
+	}
+}
+
+static void mtk_jpeg_enc_device_run(void *priv)
+{
+	struct mtk_jpeg_ctx *ctx = priv;
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+	struct vb2_v4l2_buffer *src_buf, *dst_buf;
+	enum vb2_buffer_state buf_state = VB2_BUF_STATE_ERROR;
+	unsigned long flags;
+	struct mtk_jpeg_src_buf *jpeg_src_buf;
+	struct mtk_jpeg_enc_bs enc_bs;
+	int ret;
+
+	src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
+	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
+	jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(&src_buf->vb2_buf);
+
+	ret = pm_runtime_get_sync(jpeg->dev);
+	if (ret < 0)
+		goto enc_end;
+
+	spin_lock_irqsave(&jpeg->hw_lock, flags);
+
+	/*
+	 * Resetting the hardware every frame is to ensure that all the
+	 * registers are cleared. This is a hardware requirement.
+	 */
+	mtk_jpeg_enc_reset(jpeg->reg_base);
+
+	mtk_jpeg_set_enc_dst(ctx, jpeg->reg_base, &dst_buf->vb2_buf, &enc_bs);
+	mtk_jpeg_set_enc_src(ctx, jpeg->reg_base, &src_buf->vb2_buf);
+	mtk_jpeg_enc_set_config(jpeg->reg_base, ctx->out_q.fmt->hw_format,
+				ctx->enable_exif, ctx->enc_quality,
+				ctx->restart_interval);
+	mtk_jpeg_enc_start(jpeg->reg_base);
+	spin_unlock_irqrestore(&jpeg->hw_lock, flags);
+	return;
+
+enc_end:
+	v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+	v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+	v4l2_m2m_buf_done(src_buf, buf_state);
+	v4l2_m2m_buf_done(dst_buf, buf_state);
+	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
+}
+
 static void mtk_jpeg_dec_device_run(void *priv)
 {
 	struct mtk_jpeg_ctx *ctx = priv;
@@ -785,6 +1150,11 @@ static void mtk_jpeg_dec_device_run(void *priv)
 	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
 }
 
+static int mtk_jpeg_enc_job_ready(void *priv)
+{
+		return 1;
+}
+
 static int mtk_jpeg_dec_job_ready(void *priv)
 {
 	struct mtk_jpeg_ctx *ctx = priv;
@@ -792,6 +1162,11 @@ static int mtk_jpeg_dec_job_ready(void *priv)
 	return (ctx->state == MTK_JPEG_RUNNING) ? 1 : 0;
 }
 
+static const struct v4l2_m2m_ops mtk_jpeg_enc_m2m_ops = {
+	.device_run = mtk_jpeg_enc_device_run,
+	.job_ready  = mtk_jpeg_enc_job_ready,
+};
+
 static const struct v4l2_m2m_ops mtk_jpeg_dec_m2m_ops = {
 	.device_run = mtk_jpeg_dec_device_run,
 	.job_ready  = mtk_jpeg_dec_job_ready,
@@ -830,24 +1205,109 @@ static int mtk_jpeg_dec_queue_init(void *priv, struct vb2_queue *src_vq,
 	return ret;
 }
 
-static void mtk_jpeg_clk_on(struct mtk_jpeg_dev *jpeg)
+static int mtk_jpeg_enc_queue_init(void *priv, struct vb2_queue *src_vq,
+				   struct vb2_queue *dst_vq)
 {
+	struct mtk_jpeg_ctx *ctx = priv;
 	int ret;
 
+	src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+	src_vq->io_modes = VB2_DMABUF | VB2_MMAP;
+	src_vq->drv_priv = ctx;
+	src_vq->buf_struct_size = sizeof(struct mtk_jpeg_src_buf);
+	src_vq->ops = &mtk_jpeg_enc_qops;
+	src_vq->mem_ops = &vb2_dma_contig_memops;
+	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	src_vq->lock = &ctx->jpeg->lock;
+	src_vq->dev = ctx->jpeg->dev;
+	ret = vb2_queue_init(src_vq);
+	if (ret)
+		return ret;
+
+	dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+	dst_vq->io_modes = VB2_DMABUF | VB2_MMAP;
+	dst_vq->drv_priv = ctx;
+	dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
+	dst_vq->ops = &mtk_jpeg_enc_qops;
+	dst_vq->mem_ops = &vb2_dma_contig_memops;
+	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	dst_vq->lock = &ctx->jpeg->lock;
+	dst_vq->dev = ctx->jpeg->dev;
+	ret = vb2_queue_init(dst_vq);
+
+	return ret;
+}
+
+static void mtk_jpeg_clk_on(struct mtk_jpeg_dev *jpeg)
+{
+	int ret, i;
+
 	ret = mtk_smi_larb_get(jpeg->larb);
 	if (ret)
 		dev_err(jpeg->dev, "mtk_smi_larb_get larbvdec fail %d\n", ret);
-	clk_prepare_enable(jpeg->clk_jdec_smi);
-	clk_prepare_enable(jpeg->clk_jdec);
+
+	for (i = 0; i < jpeg->variant->num_clocks; i++) {
+		ret = clk_prepare_enable(jpeg->clocks[i]);
+		if (ret) {
+			while (--i >= 0)
+				clk_disable_unprepare(jpeg->clocks[i]);
+		}
+	}
 }
 
 static void mtk_jpeg_clk_off(struct mtk_jpeg_dev *jpeg)
 {
-	clk_disable_unprepare(jpeg->clk_jdec);
-	clk_disable_unprepare(jpeg->clk_jdec_smi);
+	int i;
+
+	for (i = jpeg->variant->num_clocks - 1; i >= 0; i--)
+		clk_disable_unprepare(jpeg->clocks[i]);
 	mtk_smi_larb_put(jpeg->larb);
 }
 
+static irqreturn_t mtk_jpeg_enc_irq(int irq, void *priv)
+{
+	struct mtk_jpeg_dev *jpeg = priv;
+	struct mtk_jpeg_ctx *ctx;
+	struct vb2_v4l2_buffer *src_buf, *dst_buf;
+	struct mtk_jpeg_src_buf *jpeg_src_buf;
+	enum vb2_buffer_state buf_state = VB2_BUF_STATE_ERROR;
+	u32 enc_irq_ret;
+	u32 enc_ret, result_size;
+
+	ctx = v4l2_m2m_get_curr_priv(jpeg->m2m_dev);
+	if (!ctx) {
+		v4l2_err(&jpeg->v4l2_dev, "Context is NULL\n");
+		return IRQ_HANDLED;
+	}
+
+	src_buf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+	dst_buf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+	jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(&src_buf->vb2_buf);
+
+	enc_ret = mtk_jpeg_enc_get_and_clear_int_status(jpeg->reg_base);
+	enc_irq_ret = mtk_jpeg_enc_enum_result(jpeg->reg_base, enc_ret);
+
+	if (enc_irq_ret >= MTK_JPEG_ENC_RESULT_STALL)
+		mtk_jpeg_enc_reset(jpeg->reg_base);
+
+	if (enc_irq_ret != MTK_JPEG_ENC_RESULT_DONE) {
+		dev_err(jpeg->dev, "encode failed\n");
+		goto enc_end;
+	}
+
+	result_size = mtk_jpeg_enc_get_file_size(jpeg->reg_base);
+	vb2_set_plane_payload(&dst_buf->vb2_buf, 0, result_size);
+
+	buf_state = VB2_BUF_STATE_DONE;
+
+enc_end:
+	v4l2_m2m_buf_done(src_buf, buf_state);
+	v4l2_m2m_buf_done(dst_buf, buf_state);
+	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
+	pm_runtime_put(ctx->jpeg->dev);
+	return IRQ_HANDLED;
+}
+
 static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv)
 {
 	struct mtk_jpeg_dev *jpeg = priv;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1361

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
@@ -893,36 +1353,130 @@ static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv)
 	return IRQ_HANDLED;
 }
 
+static void mtk_jpeg_set_enc_default_params(struct mtk_jpeg_ctx *ctx)
+{
+	struct mtk_jpeg_q_data *q = &ctx->out_q;
+	struct v4l2_pix_format_mplane *pix_mp;
+
+	pix_mp = kmalloc(sizeof(*pix_mp), GFP_KERNEL);

Huh? It's allocated, but not freed?

This appears to be a temporary struct, so why not just do:

	struct v4l2_pix_format_mplane pix_mp = {};

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1374

Quoted 13 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +13 -0
+
+	ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+	ctx->colorspace = V4L2_COLORSPACE_JPEG,
+	ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
+	ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+	pix_mp->width = MTK_JPEG_MIN_WIDTH;
+	pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+
+	q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_YUYV,
+				      MTK_JPEG_FMT_FLAG_ENC_OUTPUT);
+	vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+				    fmt.pix_mp), q->fmt);

Yuck. Just change the vidioc_try_fmt prototype where the first argument is a v4l2_pix_format_mplane pointer instead of a v4l2_format pointer.

Quoted 17 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +17 -0
+	q->w = pix_mp->width;
+	q->h = pix_mp->height;
+	q->crop_rect.width = pix_mp->width;
+	q->crop_rect.height = pix_mp->height;
+	q->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
+	q->bytesperline[0] = pix_mp->plane_fmt[0].bytesperline;
+
+	q = &ctx->cap_q;
+	q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_JPEG,
+				      MTK_JPEG_FMT_FLAG_ENC_CAPTURE);
+	pix_mp->width = MTK_JPEG_MIN_WIDTH;
+	pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+	vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+				    fmt.pix_mp), q->fmt);
+	q->w = pix_mp->width;
+	q->h = pix_mp->height;
+	q->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1401

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -0
+	q->bytesperline[0] = pix_mp->plane_fmt[0].bytesperline;
+}
+
 static void mtk_jpeg_set_dec_default_params(struct mtk_jpeg_ctx *ctx)
 {
 	struct mtk_jpeg_q_data *q = &ctx->out_q;
+	struct v4l2_pix_format_mplane *pix_mp;
 	int i;
 
+	pix_mp = kmalloc(sizeof(*pix_mp), GFP_KERNEL);

Ditto.

Quoted 715 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +143 -31
+
+	ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
 	ctx->colorspace = V4L2_COLORSPACE_JPEG,
 	ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
 	ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
 	ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;
-
-	q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_JPEG,
-					      MTK_JPEG_FMT_TYPE_OUTPUT);
-	q->w = MTK_JPEG_MIN_WIDTH;
-	q->h = MTK_JPEG_MIN_HEIGHT;
-	q->bytesperline[0] = 0;
-	q->sizeimage[0] = MTK_JPEG_DEFAULT_SIZEIMAGE;
+	pix_mp->width = MTK_JPEG_MIN_WIDTH;
+	pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+
+	q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_JPEG,
+				      MTK_JPEG_FMT_FLAG_DEC_OUTPUT);
+	vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+				    fmt.pix_mp), q->fmt);
+	q->w = pix_mp->width;
+	q->h = pix_mp->height;
+	q->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
+	q->bytesperline[0] = pix_mp->plane_fmt[0].bytesperline;
 
 	q = &ctx->cap_q;
-	q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_YUV420M,
-					      MTK_JPEG_FMT_TYPE_CAPTURE);
-	q->w = MTK_JPEG_MIN_WIDTH;
-	q->h = MTK_JPEG_MIN_HEIGHT;
-
+	q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_YUV420M,
+				      MTK_JPEG_FMT_FLAG_DEC_CAPTURE);
+	pix_mp->width = MTK_JPEG_MIN_WIDTH;
+	pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+	vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+				    fmt.pix_mp), q->fmt);
+	q->w = pix_mp->width;
+	q->h = pix_mp->height;
 	for (i = 0; i < q->fmt->colplanes; i++) {
-		u32 stride = q->w * q->fmt->h_sample[i] / 4;
-		u32 h = q->h * q->fmt->v_sample[i] / 4;
+		q->sizeimage[i] = pix_mp->plane_fmt[i].sizeimage;
+		q->bytesperline[i] = pix_mp->plane_fmt[i].bytesperline;
+	}
+}
 
-		q->bytesperline[i] = stride;
-		q->sizeimage[i] = stride * h;
+static int mtk_jpeg_enc_open(struct file *file)
+{
+	struct mtk_jpeg_dev *jpeg = video_drvdata(file);
+	struct video_device *vfd = video_devdata(file);
+	struct mtk_jpeg_ctx *ctx;
+	int ret = 0;
+
+	ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+	if (!ctx)
+		return -ENOMEM;
+
+	if (mutex_lock_interruptible(&jpeg->lock)) {
+		ret = -ERESTARTSYS;
+		goto free;
+	}
+
+	v4l2_fh_init(&ctx->fh, vfd);
+	file->private_data = &ctx->fh;
+	v4l2_fh_add(&ctx->fh);
+
+	ctx->jpeg = jpeg;
+	ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(jpeg->m2m_dev, ctx,
+					    mtk_jpeg_enc_queue_init);
+	if (IS_ERR(ctx->fh.m2m_ctx)) {
+		ret = PTR_ERR(ctx->fh.m2m_ctx);
+		goto error;
 	}
+
+	ret = mtk_jpeg_enc_ctrls_setup(ctx);
+	if (ret) {
+		v4l2_err(&jpeg->v4l2_dev, "Failed to setup jpeg enc controls\n");
+		goto error;
+	}
+	mtk_jpeg_set_enc_default_params(ctx);
+
+	mutex_unlock(&jpeg->lock);
+	return 0;
+
+error:
+	v4l2_fh_del(&ctx->fh);
+	v4l2_fh_exit(&ctx->fh);
+	mutex_unlock(&jpeg->lock);
+free:
+	kfree(ctx);
+	return ret;
 }
 
 static int mtk_jpeg_dec_open(struct file *file)
@@ -953,6 +1507,12 @@ static int mtk_jpeg_dec_open(struct file *file)
 		goto error;
 	}
 
+	v4l2_ctrl_handler_init(&ctx->ctrl_hdl, 0);
+	ret = v4l2_ctrl_handler_setup(&ctx->ctrl_hdl);
+	if (ret) {
+		v4l2_err(&jpeg->v4l2_dev, "Failed to setup jpeg dec controls\n");
+		goto error;
+	}
 	mtk_jpeg_set_dec_default_params(ctx);
 	mutex_unlock(&jpeg->lock);
 	return 0;
@@ -973,6 +1533,7 @@ static int mtk_jpeg_release(struct file *file)
 
 	mutex_lock(&jpeg->lock);
 	v4l2_m2m_ctx_release(ctx->fh.m2m_ctx);
+	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
 	v4l2_fh_del(&ctx->fh);
 	v4l2_fh_exit(&ctx->fh);
 	kfree(ctx);
@@ -980,6 +1541,15 @@ static int mtk_jpeg_release(struct file *file)
 	return 0;
 }
 
+static const struct v4l2_file_operations mtk_jpeg_enc_fops = {
+	.owner          = THIS_MODULE,
+	.open           = mtk_jpeg_enc_open,
+	.release        = mtk_jpeg_release,
+	.poll           = v4l2_m2m_fop_poll,
+	.unlocked_ioctl = video_ioctl2,
+	.mmap           = v4l2_m2m_fop_mmap,
+};
+
 static const struct v4l2_file_operations mtk_jpeg_dec_fops = {
 	.owner          = THIS_MODULE,
 	.open           = mtk_jpeg_dec_open,
@@ -993,6 +1563,7 @@ static int mtk_jpeg_clk_init(struct mtk_jpeg_dev *jpeg)
 {
 	struct device_node *node;
 	struct platform_device *pdev;
+	int i;
 
 	node = of_parse_phandle(jpeg->dev->of_node, "mediatek,larb", 0);
 	if (!node)
@@ -1006,12 +1577,17 @@ static int mtk_jpeg_clk_init(struct mtk_jpeg_dev *jpeg)
 
 	jpeg->larb = &pdev->dev;
 
-	jpeg->clk_jdec = devm_clk_get(jpeg->dev, "jpgdec");
-	if (IS_ERR(jpeg->clk_jdec))
-		return PTR_ERR(jpeg->clk_jdec);
+	for (i = 0; i < jpeg->variant->num_clocks; i++) {
+		jpeg->clocks[i] = devm_clk_get(jpeg->dev,
+					       jpeg->variant->clk_names[i]);
+		if (IS_ERR(jpeg->clocks[i])) {
+			dev_err(&pdev->dev, "failed to get clock: %s\n",
+				jpeg->variant->clk_names[i]);
+			return PTR_ERR(jpeg->clocks[i]);
+		}
+	}
 
-	jpeg->clk_jdec_smi = devm_clk_get(jpeg->dev, "jpgdec-smi");
-	return PTR_ERR_OR_ZERO(jpeg->clk_jdec_smi);
+	return 0;
 }
 
 static int mtk_jpeg_probe(struct platform_device *pdev)
@@ -1028,6 +1604,7 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 	mutex_init(&jpeg->lock);
 	spin_lock_init(&jpeg->hw_lock);
 	jpeg->dev = &pdev->dev;
+	jpeg->variant = of_device_get_match_data(jpeg->dev);
 
 	res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
 	jpeg->reg_base = devm_ioremap_resource(&pdev->dev, res);
@@ -1042,8 +1619,12 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 		return jpeg_irq;
 	}
 
-	ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_dec_irq, 0,
-			       pdev->name, jpeg);
+	if (jpeg->variant->is_encoder)
+		ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_enc_irq,
+				       0, pdev->name, jpeg);
+	else
+		ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_dec_irq,
+				       0, pdev->name, jpeg);
 	if (ret) {
 		dev_err(&pdev->dev, "Failed to request jpeg_irq %d (%d)\n",
 			jpeg_irq, ret);
@@ -1063,7 +1644,10 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 		goto err_dev_register;
 	}
 
-	jpeg->m2m_dev = v4l2_m2m_init(&mtk_jpeg_dec_m2m_ops);
+	if (jpeg->variant->is_encoder)
+		jpeg->m2m_dev = v4l2_m2m_init(&mtk_jpeg_enc_m2m_ops);
+	else
+		jpeg->m2m_dev = v4l2_m2m_init(&mtk_jpeg_dec_m2m_ops);
 	if (IS_ERR(jpeg->m2m_dev)) {
 		v4l2_err(&jpeg->v4l2_dev, "Failed to init mem2mem device\n");
 		ret = PTR_ERR(jpeg->m2m_dev);
@@ -1076,9 +1660,15 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 		goto err_vfd_jpeg_alloc;
 	}
 	snprintf(jpeg->vdev->name, sizeof(jpeg->vdev->name),
-		 "%s-dec", MTK_JPEG_NAME);
-	jpeg->vdev->fops = &mtk_jpeg_dec_fops;
-	jpeg->vdev->ioctl_ops = &mtk_jpeg_dec_ioctl_ops;
+		 "%s-%s", MTK_JPEG_NAME,
+		 jpeg->variant->is_encoder ? "enc" : "dec");
+	if (jpeg->variant->is_encoder) {
+		jpeg->vdev->fops = &mtk_jpeg_enc_fops;
+		jpeg->vdev->ioctl_ops = &mtk_jpeg_enc_ioctl_ops;
+	} else {
+		jpeg->vdev->fops = &mtk_jpeg_dec_fops;
+		jpeg->vdev->ioctl_ops = &mtk_jpeg_dec_ioctl_ops;
+	}
 	jpeg->vdev->minor = -1;
 	jpeg->vdev->release = video_device_release;
 	jpeg->vdev->lock = &jpeg->lock;
@@ -1095,8 +1685,9 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 
 	video_set_drvdata(jpeg->vdev, jpeg);
 	v4l2_info(&jpeg->v4l2_dev,
-		  "decoder device registered as /dev/video%d (%d,%d)\n",
-		  jpeg->vdev->num, VIDEO_MAJOR, jpeg->vdev->minor);
+		  "jpeg %s device registered as /dev/video%d (%d,%d)\n",
+		  jpeg->variant->is_encoder ? "enc" : "dec", jpeg->vdev->num,
+		  VIDEO_MAJOR, jpeg->vdev->minor);
 
 	platform_set_drvdata(pdev, jpeg);
 
@@ -1187,14 +1778,36 @@ static const struct dev_pm_ops mtk_jpeg_pm_ops = {
 	SET_RUNTIME_PM_OPS(mtk_jpeg_pm_suspend, mtk_jpeg_pm_resume, NULL)
 };
 
+static struct mtk_jpeg_variant mt8173_jpeg_drvdata = {
+	.is_encoder	= false,
+	.clk_names	= {"jpgdec-smi", "jpgdec"},
+	.num_clocks	= 2,
+};
+
+static struct mtk_jpeg_variant mt2701_jpeg_drvdata = {
+	.is_encoder	= false,
+	.clk_names	= {"jpgdec-smi", "jpgdec"},
+	.num_clocks	= 2,
+};
+
+static struct mtk_jpeg_variant mtk_jpeg_drvdata = {
+	.is_encoder	= true,
+	.clk_names	= {"jpgenc"},
+	.num_clocks	= 1,
+};
+
 static const struct of_device_id mtk_jpeg_match[] = {
 	{
 		.compatible = "mediatek,mt8173-jpgdec",
-		.data       = NULL,
+		.data = &mt8173_jpeg_drvdata,
 	},
 	{
 		.compatible = "mediatek,mt2701-jpgdec",
-		.data       = NULL,
+		.data = &mt2701_jpeg_drvdata,
+	},
+	{
+		.compatible = "mediatek,mtk-jpgenc",
+		.data = &mtk_jpeg_drvdata,
 	},
 	{},
 };
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +35 -9
index 0b59e48495d5..9ec2c3350a16 100644
@@ -3,6 +3,7 @@
  * Copyright (c) 2016 MediaTek Inc.
  * Author: Ming Hsiu Tsai <[email protected]>
  *         Rick Chang <[email protected]>
+ *         Xia Jiang <[email protected]>
  */
 
 #ifndef _MTK_JPEG_CORE_H
@@ -16,19 +17,21 @@
 #define MTK_JPEG_NAME		"mtk-jpeg"
 
 #define MTK_JPEG_COMP_MAX		3
+#define MTK_JPEG_MAX_CLOCKS		2
+
 
 #define MTK_JPEG_FMT_FLAG_DEC_OUTPUT	BIT(0)
 #define MTK_JPEG_FMT_FLAG_DEC_CAPTURE	BIT(1)
-
-#define MTK_JPEG_FMT_TYPE_OUTPUT	1
-#define MTK_JPEG_FMT_TYPE_CAPTURE	2
+#define MTK_JPEG_FMT_FLAG_ENC_OUTPUT	BIT(2)
+#define MTK_JPEG_FMT_FLAG_ENC_CAPTURE	BIT(3)
 
 #define MTK_JPEG_MIN_WIDTH	32U
 #define MTK_JPEG_MIN_HEIGHT	32U
-#define MTK_JPEG_MAX_WIDTH	8192U
-#define MTK_JPEG_MAX_HEIGHT	8192U
+#define MTK_JPEG_MAX_WIDTH	65535U
+#define MTK_JPEG_MAX_HEIGHT	65535U
 
 #define MTK_JPEG_DEFAULT_SIZEIMAGE	(1 * 1024 * 1024)
+#define MTK_JPEG_MAX_EXIF_SIZE	(64 * 1024)
 
 /**
  * enum mtk_jpeg_ctx_state - states of the context state machine
@@ -42,6 +45,18 @@ enum mtk_jpeg_ctx_state {
 	MTK_JPEG_SOURCE_CHANGE,
 };
 
+/**
+ * mtk_jpeg_variant - mtk jpeg driver variant
+ * @is_encoder:		driver mode is jpeg encoder
+ * @clk_names:		clock names
+ * @num_clocks:		numbers of clock
+ */
+struct mtk_jpeg_variant {
+	bool is_encoder;
+	const char		*clk_names[MTK_JPEG_MAX_CLOCKS];
+	int			num_clocks;
+};
+
 /**
  * struct mt_jpeg - JPEG IP abstraction
  * @lock:		the mutex protecting this structure
@@ -53,9 +68,9 @@ enum mtk_jpeg_ctx_state {
  * @alloc_ctx:		videobuf2 memory allocator's context
  * @vdev:		video device node for jpeg mem2mem mode
  * @reg_base:		JPEG registers mapping
- * @clk_jdec:		JPEG hw working clock
- * @clk_jdec_smi:	JPEG SMI bus clock
  * @larb:		SMI device
+ * @clocks:		JPEG IP clock(s)
+ * @variant:		driver variant to be used
  */
 struct mtk_jpeg_dev {
 	struct mutex		lock;
@@ -67,14 +82,15 @@ struct mtk_jpeg_dev {
 	void			*alloc_ctx;
 	struct video_device	*vdev;
 	void __iomem		*reg_base;
-	struct clk		*clk_jdec;
-	struct clk		*clk_jdec_smi;
 	struct device		*larb;
+	struct clk		*clocks[MTK_JPEG_MAX_CLOCKS];
+	const struct mtk_jpeg_variant *variant;
 };
 
 /**
  * struct jpeg_fmt - driver's internal color format data
  * @fourcc:	the fourcc code, 0 if not applicable
+ * @hw_format:	hardware format value
  * @h_sample:	horizontal sample count of plane in 4 * 4 pixel image
  * @v_sample:	vertical sample count of plane in 4 * 4 pixel image
  * @colplanes:	number of color planes (1 for packed formats)
@@ -84,6 +100,7 @@ struct mtk_jpeg_dev {
  */
 struct mtk_jpeg_fmt {
 	u32	fourcc;
+	u32	hw_format;
 	int	h_sample[VIDEO_MAX_PLANES];
 	int	v_sample[VIDEO_MAX_PLANES];
 	int	colplanes;
@@ -107,6 +124,7 @@ struct mtk_jpeg_q_data {
 	u32			h;
 	u32			bytesperline[VIDEO_MAX_PLANES];
 	u32			sizeimage[VIDEO_MAX_PLANES];
+	struct v4l2_rect	crop_rect;
 };
 
 /**
@@ -116,6 +134,10 @@ struct mtk_jpeg_q_data {
  * @cap_q:		destination (capture) queue queue information
  * @fh:			V4L2 file handle
  * @state:		state of the context
+ * @enable_exif:	enable exif mode of jpeg encoder
+ * @enc_quality:	jpeg encoder quality
+ * @restart_interval:	jpeg encoder restart interval
+ * @ctrl_hdl:		controls handler
  * @colorspace: enum v4l2_colorspace; supplemental to pixelformat
  * @ycbcr_enc: enum v4l2_ycbcr_encoding, Y'CbCr encoding
  * @quantization: enum v4l2_quantization, colorspace quantization
@@ -127,6 +149,10 @@ struct mtk_jpeg_ctx {
 	struct mtk_jpeg_q_data		cap_q;
 	struct v4l2_fh			fh;
 	enum mtk_jpeg_ctx_state		state;
+	bool				enable_exif;
+	u8				enc_quality;
+	u8				restart_interval;
+	struct v4l2_ctrl_handler	ctrl_hdl;
 
 	enum v4l2_colorspace colorspace;
 	enum v4l2_ycbcr_encoding ycbcr_enc;
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +193 -0
new file mode 100644
index 000000000000..7fc1de920a75
@@ -0,0 +1,193 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ * Author: Xia Jiang <[email protected]>
+ *
+ */
+
+#include <linux/io.h>
+#include <linux/kernel.h>
+#include <media/videobuf2-core.h>
+
+#include "mtk_jpeg_enc_hw.h"
+
+static const struct mtk_jpeg_enc_qlt mtk_jpeg_enc_quality[] = {
+	{.quality_param = 34, .hardware_value = JPEG_ENC_QUALITY_Q34},
+	{.quality_param = 39, .hardware_value = JPEG_ENC_QUALITY_Q39},
+	{.quality_param = 48, .hardware_value = JPEG_ENC_QUALITY_Q48},
+	{.quality_param = 60, .hardware_value = JPEG_ENC_QUALITY_Q60},
+	{.quality_param = 64, .hardware_value = JPEG_ENC_QUALITY_Q64},
+	{.quality_param = 68, .hardware_value = JPEG_ENC_QUALITY_Q68},
+	{.quality_param = 74, .hardware_value = JPEG_ENC_QUALITY_Q74},
+	{.quality_param = 80, .hardware_value = JPEG_ENC_QUALITY_Q80},
+	{.quality_param = 82, .hardware_value = JPEG_ENC_QUALITY_Q82},
+	{.quality_param = 84, .hardware_value = JPEG_ENC_QUALITY_Q84},
+	{.quality_param = 87, .hardware_value = JPEG_ENC_QUALITY_Q87},
+	{.quality_param = 90, .hardware_value = JPEG_ENC_QUALITY_Q90},
+	{.quality_param = 92, .hardware_value = JPEG_ENC_QUALITY_Q92},
+	{.quality_param = 95, .hardware_value = JPEG_ENC_QUALITY_Q95},
+	{.quality_param = 97, .hardware_value = JPEG_ENC_QUALITY_Q97},
+};
+
+void mtk_jpeg_enc_reset(void __iomem *base)
+{
+	writel(0x00, base + JPEG_ENC_RSTB);
+	writel(JPEG_ENC_RESET_BIT, base + JPEG_ENC_RSTB);
+	writel(0x00, base + JPEG_ENC_CODEC_SEL);
+}
+
+u32 mtk_jpeg_enc_get_and_clear_int_status(void __iomem *base)
+{
+	u32 ret;
+
+	ret = readl(base + JPEG_ENC_INT_STS) &
+		    JPEG_ENC_INT_STATUS_MASK_ALLIRQ;
+	if (ret)
+		writel(0, base + JPEG_ENC_INT_STS);
+
+	return ret;
+}
+
+u32 mtk_jpeg_enc_get_file_size(void __iomem *base)
+{
+	return readl(base + JPEG_ENC_DMA_ADDR0) -
+	       readl(base + JPEG_ENC_DST_ADDR0);
+}
+
+u32 mtk_jpeg_enc_enum_result(void __iomem *base, u32 irq_status)
+{
+	if (irq_status & JPEG_ENC_INT_STATUS_DONE)
+		return MTK_JPEG_ENC_RESULT_DONE;
+	else if (irq_status & JPEG_ENC_INT_STATUS_STALL)
+		return MTK_JPEG_ENC_RESULT_STALL;
+	else
+		return MTK_JPEG_ENC_RESULT_VCODEC_IRQ;
+}
+
+void mtk_jpeg_enc_set_img_size(void __iomem *base, u32 width, u32 height)
+{
+	u32 value;
+
+	value = width << 16 | height;
+	writel(value, base + JPEG_ENC_IMG_SIZE);
+}
+
+void mtk_jpeg_enc_set_blk_num(void __iomem *base, u32 enc_format, u32 width,
+			      u32 height)
+{
+	u32 blk_num;
+	u32 is_420;
+	u32 padding_width;
+	u32 padding_height;
+	u32 luma_blocks;
+	u32 chroma_blocks;
+
+	is_420 = (enc_format == V4L2_PIX_FMT_NV12M ||
+		  enc_format == V4L2_PIX_FMT_NV21M) ? 1 : 0;
+	padding_width = round_up(width, 16);
+	padding_height = round_up(height, is_420 ? 16 : 8);
+
+	luma_blocks = padding_width / 8 * padding_height / 8;
+	if (is_420)
+		chroma_blocks = luma_blocks / 4;
+	else
+		chroma_blocks = luma_blocks / 2;
+
+	blk_num = luma_blocks + 2 * chroma_blocks - 1;
+
+	writel(blk_num, base + JPEG_ENC_BLK_NUM);
+}
+
+void mtk_jpeg_enc_set_stride(void __iomem *base, u32 enc_format, u32 width,
+			     u32 height, u32 bytesperline)
+{
+	u32 img_stride;
+	u32 mem_stride;
+
+	if (enc_format == V4L2_PIX_FMT_NV12M ||
+	    enc_format == V4L2_PIX_FMT_NV21M) {
+		img_stride = round_up(width, 16);
+		mem_stride = bytesperline;
+	} else {
+		img_stride = round_up(width * 2, 32);
+		mem_stride = img_stride;
+	}
+
+	writel(img_stride, base + JPEG_ENC_IMG_STRIDE);
+	writel(mem_stride, base + JPEG_ENC_STRIDE);
+}
+
+void mtk_jpeg_enc_set_src_addr(void __iomem *base, u32 src_addr,
+			       u32 plane_index)
+{
+	if (!plane_index)
+		writel(src_addr, base + JPEG_ENC_SRC_LUMA_ADDR);
+	else
+		writel(src_addr, base + JPEG_ENC_SRC_CHROMA_ADDR);
+}
+
+void mtk_jpeg_enc_set_dst_addr(void __iomem *base, u32 dst_addr,
+			       u32 stall_size, u32 init_offset,
+			       u32 offset_mask)
+{
+	writel(init_offset & ~0xf, base + JPEG_ENC_OFFSET_ADDR);
+	writel(offset_mask & 0xf, base + JPEG_ENC_BYTE_OFFSET_MASK);
+	writel(dst_addr & ~0xf, base + JPEG_ENC_DST_ADDR0);
+	writel((dst_addr + stall_size) & ~0xf, base + JPEG_ENC_STALL_ADDR0);
+}
+
+static void mtk_jpeg_enc_set_quality(void __iomem *base, u32 quality)
+{
+	u32 value;
+	u32 i, enc_quality;
+
+	enc_quality = mtk_jpeg_enc_quality[0].hardware_value;
+	for (i = 0; i < ARRAY_SIZE(mtk_jpeg_enc_quality); i++) {
+		if (quality <= mtk_jpeg_enc_quality[i].quality_param) {
+			enc_quality = mtk_jpeg_enc_quality[i].hardware_value;
+			break;
+		}
+	}
+
+	value = readl(base + JPEG_ENC_QUALITY);
+	value = (value & JPEG_ENC_QUALITY_MASK) | enc_quality;
+	writel(value, base + JPEG_ENC_QUALITY);
+}
+
+static void mtk_jpeg_enc_set_ctrl(void __iomem *base, u32 enc_format,
+				  bool exif_en, u32 restart_interval)
+{
+	u32 value;
+
+	value = readl(base + JPEG_ENC_CTRL);
+	value &= ~JPEG_ENC_CTRL_YUV_FORMAT_MASK;
+	value |= (enc_format & 3) << 3;
+	if (exif_en)
+		value |= JPEG_ENC_CTRL_FILE_FORMAT_BIT;
+	else
+		value &= ~JPEG_ENC_CTRL_FILE_FORMAT_BIT;
+	if (restart_interval)
+		value |= JPEG_ENC_CTRL_RESTART_EN_BIT;
+	else
+		value &= ~JPEG_ENC_CTRL_RESTART_EN_BIT;
+	writel(value, base + JPEG_ENC_CTRL);
+}
+
+void mtk_jpeg_enc_set_config(void __iomem *base, u32 enc_format, bool exif_en,
+			     u32 quality, u32 restart_interval)
+{
+	mtk_jpeg_enc_set_quality(base, quality);
+
+	mtk_jpeg_enc_set_ctrl(base, enc_format, exif_en, restart_interval);
+
+	writel(restart_interval, base + JPEG_ENC_RST_MCU_NUM);
+}
+
+void mtk_jpeg_enc_start(void __iomem *base)
+{
+	u32 value;
+
+	value = readl(base + JPEG_ENC_CTRL);
+	value |= JPEG_ENC_CTRL_INT_EN_BIT | JPEG_ENC_CTRL_ENABLE_BIT;
+	writel(value, base + JPEG_ENC_CTRL);
+}
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h new file +113 -0
new file mode 100644
index 000000000000..73faf49b667c
@@ -0,0 +1,123 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ * Author: Xia Jiang <[email protected]>
+ *
+ */
+
+#ifndef _MTK_JPEG_ENC_HW_H
+#define _MTK_JPEG_ENC_HW_H
+
+#include <media/videobuf2-core.h>
+
+#include "mtk_jpeg_core.h"
+
+#define JPEG_ENC_INT_STATUS_DONE	BIT(0)
+#define JPEG_ENC_INT_STATUS_STALL	BIT(1)
+#define JPEG_ENC_INT_STATUS_VCODEC_IRQ	BIT(4)
+#define JPEG_ENC_INT_STATUS_MASK_ALLIRQ	0x13
+
+#define JPEG_ENC_DST_ADDR_OFFSET_MASK	GENMASK(3, 0)
+#define JPEG_ENC_QUALITY_MASK		GENMASK(31, 16)
+
+#define JPEG_ENC_CTRL_YUV_FORMAT_MASK	0x18
+#define JPEG_ENC_CTRL_RESTART_EN_BIT	BIT(10)
+#define JPEG_ENC_CTRL_FILE_FORMAT_BIT	BIT(5)
+#define JPEG_ENC_CTRL_INT_EN_BIT	BIT(2)
+#define JPEG_ENC_CTRL_ENABLE_BIT	BIT(0)
+#define JPEG_ENC_RESET_BIT		BIT(0)
+
+#define JPEG_ENC_YUV_FORMAT_YUYV	0
+#define JPEG_ENC_YUV_FORMAT_YVYU	1
+#define JPEG_ENC_YUV_FORMAT_NV12	2
+#define JEPG_ENC_YUV_FORMAT_NV21	3
+
+#define JPEG_ENC_QUALITY_Q60		0x0
+#define JPEG_ENC_QUALITY_Q80		0x1
+#define JPEG_ENC_QUALITY_Q90		0x2
+#define JPEG_ENC_QUALITY_Q95		0x3
+#define JPEG_ENC_QUALITY_Q39		0x4
+#define JPEG_ENC_QUALITY_Q68		0x5
+#define JPEG_ENC_QUALITY_Q84		0x6
+#define JPEG_ENC_QUALITY_Q92		0x7
+#define JPEG_ENC_QUALITY_Q48		0x8
+#define JPEG_ENC_QUALITY_Q74		0xa
+#define JPEG_ENC_QUALITY_Q87		0xb
+#define JPEG_ENC_QUALITY_Q34		0xc
+#define JPEG_ENC_QUALITY_Q64		0xe
+#define JPEG_ENC_QUALITY_Q82		0xf
+#define JPEG_ENC_QUALITY_Q97		0x10
+
+#define JPEG_ENC_RSTB			0x100
+#define JPEG_ENC_CTRL			0x104
+#define JPEG_ENC_QUALITY		0x108
+#define JPEG_ENC_BLK_NUM		0x10C
+#define JPEG_ENC_BLK_CNT		0x110
+#define JPEG_ENC_INT_STS		0x11c
+#define JPEG_ENC_DST_ADDR0		0x120
+#define JPEG_ENC_DMA_ADDR0		0x124
+#define JPEG_ENC_STALL_ADDR0		0x128
+#define JPEG_ENC_OFFSET_ADDR		0x138
+#define JPEG_ENC_RST_MCU_NUM		0x150
+#define JPEG_ENC_IMG_SIZE		0x154
+#define JPEG_ENC_DEBUG_INFO0		0x160
+#define JPEG_ENC_DEBUG_INFO1		0x164
+#define JPEG_ENC_TOTAL_CYCLE		0x168
+#define JPEG_ENC_BYTE_OFFSET_MASK	0x16c
+#define JPEG_ENC_SRC_LUMA_ADDR		0x170
+#define JPEG_ENC_SRC_CHROMA_ADDR	0x174
+#define JPEG_ENC_STRIDE			0x178
+#define JPEG_ENC_IMG_STRIDE		0x17c
+#define JPEG_ENC_DCM_CTRL		0x300
+#define JPEG_ENC_CODEC_SEL		0x314
+#define JPEG_ENC_ULTRA_THRES		0x318
+
+enum {
+	MTK_JPEG_ENC_RESULT_DONE,
+	MTK_JPEG_ENC_RESULT_STALL,
+	MTK_JPEG_ENC_RESULT_VCODEC_IRQ
+};
+
+/**
+ * struct mtk_jpeg_enc_qlt - JPEG encoder quality data
+ * @quality_param:	quality value
+ * @hardware_value:	hardware value of quality
+ */
+struct mtk_jpeg_enc_qlt {
+	u8	quality_param;
+	u8	hardware_value;
+};
+
+/**
+ * struct mt_jpeg_enc_bs - JPEG encoder bitstream  buffer
+ * @dma_addr:			JPEG encoder destination address
+ * @size:			JPEG encoder bistream size
+ * @dma_addr_offset:		JPEG encoder offset address
+ * @dma_addr_offsetmask:	JPEG encoder destination address offset mask
+ */
+struct mtk_jpeg_enc_bs {
+	dma_addr_t	dma_addr;
+	size_t		size;
+	u32		dma_addr_offset;
+	u32		dma_addr_offsetmask;
+};
+
+void mtk_jpeg_enc_reset(void __iomem *base);
+u32 mtk_jpeg_enc_get_and_clear_int_status(void __iomem *base);
+u32 mtk_jpeg_enc_get_file_size(void __iomem *base);
+u32 mtk_jpeg_enc_enum_result(void __iomem *base, u32 irq_status);
+void mtk_jpeg_enc_set_img_size(void __iomem *base, u32 width, u32 height);
+void mtk_jpeg_enc_set_blk_num(void __iomem *base, u32 enc_format, u32 width,
+			      u32 height);
+void mtk_jpeg_enc_set_stride(void __iomem *base, u32 enc_format, u32 width,
+			     u32 height, u32 bytesperline);

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h line 123

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h new file +10 -0
+void mtk_jpeg_enc_set_src_addr(void __iomem *base, u32 src_addr,
+			       u32 plane_index);
+void mtk_jpeg_enc_set_dst_addr(void __iomem *base, u32 dst_addr,
+			       u32 stall_size, u32 init_offset,
+			       u32 offset_mask);
+void mtk_jpeg_enc_set_config(void __iomem *base, u32 enc_format, bool exif_en,
+			     u32 quality, u32 restart_interval);
+void mtk_jpeg_enc_start(void __iomem *enc_reg_base);
+
+#endif /* _MTK_JPEG_ENC_HW_H */

Regards,

Hans

author
#33 lore raw

Re: [PATCH RESEND v9 18/18] media: platform: Add jpeg enc feature

Quoted 291 lines · Hans Verkuil
Quoted 290 lines · Xia Jiang

Add mtk jpeg encode v4l2 driver based on jpeg decode, because that jpeg decode and encode have great similarities with function operation.

---
v9: add member variable(struct v4l2_rect) in out_q structure for storing
the active crop information.
move the renaming exsting functions/defines/variables to a separate patch.

---
 drivers/media/platform/mtk-jpeg/Makefile      |   5 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 845 +++++++++++++++---
 .../media/platform/mtk-jpeg/mtk_jpeg_core.h   |  44 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c | 193 ++++
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h | 123 +++
 5 files changed, 1084 insertions(+), 126 deletions(-)
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h
drivers/media/platform/mtk-jpeg/Makefile +4 -1
index 48516dcf96e6..76c33aad0f3f 100644
@@ -1,3 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0-only
-mtk_jpeg-objs := mtk_jpeg_core.o mtk_jpeg_dec_hw.o mtk_jpeg_dec_parse.o
+mtk_jpeg-objs := mtk_jpeg_core.o \
+		 mtk_jpeg_dec_hw.o \
+		 mtk_jpeg_dec_parse.o \
+		 mtk_jpeg_enc_hw.o
 obj-$(CONFIG_VIDEO_MEDIATEK_JPEG) += mtk_jpeg.o
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +145 -31
index 29b8b82c606c..d7ef69920530 100644
@@ -3,6 +3,7 @@
  * Copyright (c) 2016 MediaTek Inc.
  * Author: Ming Hsiu Tsai <[email protected]>
  *         Rick Chang <[email protected]>
+ *         Xia Jiang <[email protected]>
  */
 
 #include <linux/clk.h>
@@ -23,10 +24,59 @@
 #include <media/videobuf2-dma-contig.h>
 #include <soc/mediatek/smi.h>
 
+#include "mtk_jpeg_enc_hw.h"
 #include "mtk_jpeg_dec_hw.h"
 #include "mtk_jpeg_core.h"
 #include "mtk_jpeg_dec_parse.h"
 
+static struct mtk_jpeg_fmt mtk_jpeg_enc_formats[] = {
+	{
+		.fourcc		= V4L2_PIX_FMT_JPEG,
+		.colplanes	= 1,
+		.flags		= MTK_JPEG_FMT_FLAG_ENC_CAPTURE,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_NV12M,
+		.hw_format	= JPEG_ENC_YUV_FORMAT_NV12,
+		.h_sample	= {4, 4},
+		.v_sample	= {4, 2},
+		.colplanes	= 2,
+		.h_align	= 4,
+		.v_align	= 4,
+		.flags		= MTK_JPEG_FMT_FLAG_ENC_OUTPUT,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_NV21M,
+		.hw_format	= JEPG_ENC_YUV_FORMAT_NV21,
+		.h_sample	= {4, 4},
+		.v_sample	= {4, 2},
+		.colplanes	= 2,
+		.h_align	= 4,
+		.v_align	= 4,
+		.flags		= MTK_JPEG_FMT_FLAG_ENC_OUTPUT,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_YUYV,
+		.hw_format	= JPEG_ENC_YUV_FORMAT_YUYV,
+		.h_sample	= {8},
+		.v_sample	= {4},
+		.colplanes	= 1,
+		.h_align	= 5,
+		.v_align	= 3,
+		.flags		= MTK_JPEG_FMT_FLAG_ENC_OUTPUT,
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_YVYU,
+		.hw_format	= JPEG_ENC_YUV_FORMAT_YVYU,
+		.h_sample	= {8},
+		.v_sample	= {4},
+		.colplanes	= 1,
+		.h_align	= 5,
+		.v_align	= 3,
+		.flags		= MTK_JPEG_FMT_FLAG_ENC_OUTPUT,
+	},
+};
+
 static struct mtk_jpeg_fmt mtk_jpeg_dec_formats[] = {
 	{
 		.fourcc		= V4L2_PIX_FMT_JPEG,
@@ -53,6 +103,7 @@ static struct mtk_jpeg_fmt mtk_jpeg_dec_formats[] = {
 	},
 };
 
+#define MTK_JPEG_ENC_NUM_FORMATS ARRAY_SIZE(mtk_jpeg_enc_formats)
 #define MTK_JPEG_DEC_NUM_FORMATS ARRAY_SIZE(mtk_jpeg_dec_formats)
 
 struct mtk_jpeg_src_buf {
@@ -64,6 +115,11 @@ struct mtk_jpeg_src_buf {
 static int debug;
 module_param(debug, int, 0644);
 
+static inline struct mtk_jpeg_ctx *ctrl_to_ctx(struct v4l2_ctrl *ctrl)
+{
+	return container_of(ctrl->handler, struct mtk_jpeg_ctx, ctrl_hdl);
+}
+
 static inline struct mtk_jpeg_ctx *mtk_jpeg_fh_to_ctx(struct v4l2_fh *fh)
 {
 	return container_of(fh, struct mtk_jpeg_ctx, fh);
@@ -75,6 +131,19 @@ static inline struct mtk_jpeg_src_buf *mtk_jpeg_vb2_to_srcbuf(
 	return container_of(to_vb2_v4l2_buffer(vb), struct mtk_jpeg_src_buf, b);
 }
 
+static int mtk_jpeg_enc_querycap(struct file *file, void *priv,
+				 struct v4l2_capability *cap)
+{
+	struct mtk_jpeg_dev *jpeg = video_drvdata(file);
+
+	strscpy(cap->driver, MTK_JPEG_NAME, sizeof(cap->driver));
+	strscpy(cap->card, MTK_JPEG_NAME " encoder", sizeof(cap->card));
+	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
+		 dev_name(jpeg->dev));
+
+	return 0;
+}
+
 static int mtk_jpeg_dec_querycap(struct file *file, void *priv,
 				 struct v4l2_capability *cap)
 {
@@ -88,6 +157,54 @@ static int mtk_jpeg_dec_querycap(struct file *file, void *priv,
 	return 0;
 }
 
+static int vidioc_jpeg_enc_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_jpeg_ctx *ctx = ctrl_to_ctx(ctrl);
+
+	switch (ctrl->id) {
+	case V4L2_CID_JPEG_RESTART_INTERVAL:
+		ctx->restart_interval = ctrl->val;
+		break;
+	case V4L2_CID_JPEG_COMPRESSION_QUALITY:
+		ctx->enc_quality = ctrl->val;
+		break;
+	case V4L2_CID_JPEG_ACTIVE_MARKER:
+		ctx->enable_exif = ctrl->val & V4L2_JPEG_ACTIVE_MARKER_APP1 ?
+				   true : false;
+		break;
+	}
+
+	return 0;
+}
+
+static const struct v4l2_ctrl_ops mtk_jpeg_enc_ctrl_ops = {
+	.s_ctrl = vidioc_jpeg_enc_s_ctrl,
+};
+
+static int mtk_jpeg_enc_ctrls_setup(struct mtk_jpeg_ctx *ctx)
+{
+	const struct v4l2_ctrl_ops *ops = &mtk_jpeg_enc_ctrl_ops;
+	struct v4l2_ctrl_handler *handler = &ctx->ctrl_hdl;
+
+	v4l2_ctrl_handler_init(handler, 3);
+
+	v4l2_ctrl_new_std(handler, ops, V4L2_CID_JPEG_RESTART_INTERVAL, 0, 100,
+			  1, 0);
+	v4l2_ctrl_new_std(handler, ops, V4L2_CID_JPEG_COMPRESSION_QUALITY, 48,
+			  100, 1, 90);
+	v4l2_ctrl_new_std(handler, ops, V4L2_CID_JPEG_ACTIVE_MARKER, 0,
+			  V4L2_JPEG_ACTIVE_MARKER_APP1, 0, 0);
+
+	if (handler->error) {
+		v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
+		return handler->error;
+	}
+
+	v4l2_ctrl_handler_setup(&ctx->ctrl_hdl);
+
+	return 0;
+}
+
 static int mtk_jpeg_enum_fmt(struct mtk_jpeg_fmt *mtk_jpeg_formats, int n,
 			     struct v4l2_fmtdesc *f, u32 type)
 {
@@ -109,6 +226,14 @@ static int mtk_jpeg_enum_fmt(struct mtk_jpeg_fmt *mtk_jpeg_formats, int n,
 	return 0;
 }
 
+static int mtk_jpeg_enc_enum_fmt_vid_cap(struct file *file, void *priv,
+					 struct v4l2_fmtdesc *f)
+{
+	return mtk_jpeg_enum_fmt(mtk_jpeg_enc_formats,
+				 MTK_JPEG_ENC_NUM_FORMATS, f,
+				 MTK_JPEG_FMT_FLAG_ENC_CAPTURE);
+}
+
 static int mtk_jpeg_dec_enum_fmt_vid_cap(struct file *file, void *priv,
 					 struct v4l2_fmtdesc *f)
 {
@@ -117,6 +242,14 @@ static int mtk_jpeg_dec_enum_fmt_vid_cap(struct file *file, void *priv,
 				 MTK_JPEG_FMT_FLAG_DEC_CAPTURE);
 }
 
+static int mtk_jpeg_enc_enum_fmt_vid_out(struct file *file, void *priv,
+					 struct v4l2_fmtdesc *f)
+{
+	return mtk_jpeg_enum_fmt(mtk_jpeg_enc_formats,
+				 MTK_JPEG_ENC_NUM_FORMATS, f,
+				 MTK_JPEG_FMT_FLAG_ENC_OUTPUT);
+}
+
 static int mtk_jpeg_dec_enum_fmt_vid_out(struct file *file, void *priv,
 					 struct v4l2_fmtdesc *f)
 {
@@ -132,93 +265,66 @@ mtk_jpeg_get_q_data(struct mtk_jpeg_ctx *ctx, enum v4l2_buf_type type)
 	return &ctx->cap_q;
 }
 
-static struct mtk_jpeg_fmt *mtk_jpeg_find_format(struct mtk_jpeg_ctx *ctx,
-						 u32 pixelformat,
+static struct mtk_jpeg_fmt *mtk_jpeg_find_format(u32 pixelformat,
 						 unsigned int fmt_type)
 {
-	unsigned int k, fmt_flag;
+	unsigned int k;
+	struct mtk_jpeg_fmt *fmt;
 
-	fmt_flag = (fmt_type == MTK_JPEG_FMT_TYPE_OUTPUT) ?
-		   MTK_JPEG_FMT_FLAG_DEC_OUTPUT :
-		   MTK_JPEG_FMT_FLAG_DEC_CAPTURE;
+	for (k = 0; k < MTK_JPEG_ENC_NUM_FORMATS; k++) {
+		fmt = &mtk_jpeg_enc_formats[k];
+
+		if (fmt->fourcc == pixelformat && fmt->flags & fmt_type)
+			return fmt;
+	}
 
 	for (k = 0; k < MTK_JPEG_DEC_NUM_FORMATS; k++) {
-		struct mtk_jpeg_fmt *fmt = &mtk_jpeg_dec_formats[k];
+		fmt = &mtk_jpeg_dec_formats[k];
 
-		if (fmt->fourcc == pixelformat && fmt->flags & fmt_flag)
+		if (fmt->fourcc == pixelformat && fmt->flags & fmt_type)
 			return fmt;
 	}
 
 	return NULL;
 }
 
-static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx,
-				       struct v4l2_format *f)
-{
-	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
-	struct mtk_jpeg_q_data *q_data;
-	int i;
-
-	q_data = mtk_jpeg_get_q_data(ctx, f->type);
-
-	pix_mp->width = q_data->w;
-	pix_mp->height = q_data->h;
-	pix_mp->pixelformat = q_data->fmt->fourcc;
-	pix_mp->num_planes = q_data->fmt->colplanes;
-
-	for (i = 0; i < pix_mp->num_planes; i++) {
-		pix_mp->plane_fmt[i].bytesperline = q_data->bytesperline[i];
-		pix_mp->plane_fmt[i].sizeimage = q_data->sizeimage[i];
-	}
-}
-
-static int mtk_jpeg_try_fmt_mplane(struct v4l2_format *f,
-				   struct mtk_jpeg_fmt *fmt,
-				   struct mtk_jpeg_ctx *ctx, int q_type)
+static int vidioc_try_fmt(struct v4l2_format *f, struct mtk_jpeg_fmt *fmt)
 {
 	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
 	int i;
 
Quoted 7 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c mtk_jpeg_get_q_data(struct mtk_jpeg_ctx *ctx, enum v4l2_buf_type type) · line 185

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +0 -3
 	pix_mp->field = V4L2_FIELD_NONE;
-
-	if (ctx->state != MTK_JPEG_INIT) {
-		mtk_jpeg_adjust_fmt_mplane(ctx, f);

The removal of this function seems unrelated to adding the jpeg enc feature. It's not obvious to me why this is no longer needed and what replaces it.

Dear Hans,
Thank you for your replay.

mtk_jpeg_g_fmt_vid_mplane() has done the same thing, so use it to replace mtk_jpeg_adjust_fmt_mplane().

Quoted 2 lines

I think this is something that should be split off to its own patch.

done.

Quoted 104 lines
Quoted 104 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +45 -29
-		return 0;
-	}
-
 	pix_mp->num_planes = fmt->colplanes;
 	pix_mp->pixelformat = fmt->fourcc;
 
-	if (q_type == MTK_JPEG_FMT_TYPE_OUTPUT) {
-		struct v4l2_plane_pix_format *pfmt = &pix_mp->plane_fmt[0];
-
+	if (fmt->fourcc == V4L2_PIX_FMT_JPEG) {
 		pix_mp->height = clamp(pix_mp->height, MTK_JPEG_MIN_HEIGHT,
 				       MTK_JPEG_MAX_HEIGHT);
 		pix_mp->width = clamp(pix_mp->width, MTK_JPEG_MIN_WIDTH,
 				      MTK_JPEG_MAX_WIDTH);
-
-		pfmt->bytesperline = 0;
-		/* Source size must be aligned to 128 */
-		pfmt->sizeimage = round_up(pfmt->sizeimage, 128);
-		if (pfmt->sizeimage == 0)
-			pfmt->sizeimage = MTK_JPEG_DEFAULT_SIZEIMAGE;
-		return 0;
+		pix_mp->plane_fmt[0].bytesperline = 0;
+		pix_mp->plane_fmt[0].sizeimage =
+				round_up(pix_mp->plane_fmt[0].sizeimage, 128);
+		if (pix_mp->plane_fmt[0].sizeimage == 0)
+			pix_mp->plane_fmt[0].sizeimage =
+				MTK_JPEG_DEFAULT_SIZEIMAGE;
+	} else {
+		pix_mp->height = clamp(round_up(pix_mp->height, fmt->v_align),
+				       MTK_JPEG_MIN_HEIGHT,
+				       MTK_JPEG_MAX_HEIGHT);
+		pix_mp->width = clamp(round_up(pix_mp->width, fmt->h_align),
+				      MTK_JPEG_MIN_WIDTH, MTK_JPEG_MAX_WIDTH);
+		for (i = 0; i < pix_mp->num_planes; i++) {
+			struct v4l2_plane_pix_format *pfmt =
+							&pix_mp->plane_fmt[i];
+			u32 stride = pix_mp->width * fmt->h_sample[i] / 4;
+			u32 h = pix_mp->height * fmt->v_sample[i] / 4;
+
+			pfmt->bytesperline = stride;
+			pfmt->sizeimage = stride * h;
+		}
 	}
 
-	/* type is MTK_JPEG_FMT_TYPE_CAPTURE */
-	pix_mp->height = clamp(round_up(pix_mp->height, fmt->v_align),
-			       MTK_JPEG_MIN_HEIGHT, MTK_JPEG_MAX_HEIGHT);
-	pix_mp->width = clamp(round_up(pix_mp->width, fmt->h_align),
-			      MTK_JPEG_MIN_WIDTH, MTK_JPEG_MAX_WIDTH);
-
-	for (i = 0; i < fmt->colplanes; i++) {
-		struct v4l2_plane_pix_format *pfmt = &pix_mp->plane_fmt[i];
-		u32 stride = pix_mp->width * fmt->h_sample[i] / 4;
-		u32 h = pix_mp->height * fmt->v_sample[i] / 4;
-
-		pfmt->bytesperline = stride;
-		pfmt->sizeimage = stride * h;
-	}
 	return 0;
 }
 
@@ -271,14 +377,35 @@ static int mtk_jpeg_g_fmt_vid_mplane(struct file *file, void *priv,
 	return 0;
 }
 
+static int mtk_jpeg_enc_try_fmt_vid_cap_mplane(struct file *file, void *priv,
+					       struct v4l2_format *f)
+{
+	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+	struct mtk_jpeg_fmt *fmt;
+
+	fmt = mtk_jpeg_find_format(f->fmt.pix_mp.pixelformat,
+				   MTK_JPEG_FMT_FLAG_ENC_CAPTURE);
+	if (!fmt)
+		fmt = ctx->cap_q.fmt;
+
+	v4l2_dbg(2, debug, &ctx->jpeg->v4l2_dev, "(%d) try_fmt:%c%c%c%c\n",
+		 f->type,
+		 (fmt->fourcc & 0xff),
+		 (fmt->fourcc >>  8 & 0xff),
+		 (fmt->fourcc >> 16 & 0xff),
+		 (fmt->fourcc >> 24 & 0xff));
+
+	return vidioc_try_fmt(f, fmt);
+}
+
 static int mtk_jpeg_dec_try_fmt_vid_cap_mplane(struct file *file, void *priv,
 					       struct v4l2_format *f)
 {
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
 	struct mtk_jpeg_fmt *fmt;
 
-	fmt = mtk_jpeg_find_format(ctx, f->fmt.pix_mp.pixelformat,
-				   MTK_JPEG_FMT_TYPE_CAPTURE);
+	fmt = mtk_jpeg_find_format(f->fmt.pix_mp.pixelformat,
+				   MTK_JPEG_FMT_FLAG_DEC_CAPTURE);
 	if (!fmt)
 		fmt = ctx->cap_q.fmt;
 
@@ -289,7 +416,33 @@ static int mtk_jpeg_dec_try_fmt_vid_cap_mplane(struct file *file, void *priv,
 		 (fmt->fourcc >> 16 & 0xff),
 		 (fmt->fourcc >> 24 & 0xff));
 
Quoted 7 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_try_fmt_vid_cap_mplane(struct file *file, void *priv, · line 422

Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -1
-	return mtk_jpeg_try_fmt_mplane(f, fmt, ctx, MTK_JPEG_FMT_TYPE_CAPTURE);
+	if (ctx->state != MTK_JPEG_INIT) {
+		mtk_jpeg_g_fmt_vid_mplane(file, priv, f);
+		return 0;
+	}

This is probably related to the same mtk_jpeg_adjust_fmt_mplane() removal.

yes.

Quoted 42 lines
Quoted 42 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +25 -2
+
+	return vidioc_try_fmt(f, fmt);
+}
+
+static int mtk_jpeg_enc_try_fmt_vid_out_mplane(struct file *file, void *priv,
+					       struct v4l2_format *f)
+{
+	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+	struct mtk_jpeg_fmt *fmt;
+
+	fmt = mtk_jpeg_find_format(f->fmt.pix_mp.pixelformat,
+				   MTK_JPEG_FMT_FLAG_ENC_OUTPUT);
+	if (!fmt)
+		fmt = ctx->out_q.fmt;
+
+	v4l2_dbg(2, debug, &ctx->jpeg->v4l2_dev, "(%d) try_fmt:%c%c%c%c\n",
+		 f->type,
+		 (fmt->fourcc & 0xff),
+		 (fmt->fourcc >>  8 & 0xff),
+		 (fmt->fourcc >> 16 & 0xff),
+		 (fmt->fourcc >> 24 & 0xff));
+
+	return vidioc_try_fmt(f, fmt);
 }
 
 static int mtk_jpeg_dec_try_fmt_vid_out_mplane(struct file *file, void *priv,
@@ -298,8 +451,8 @@ static int mtk_jpeg_dec_try_fmt_vid_out_mplane(struct file *file, void *priv,
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
 	struct mtk_jpeg_fmt *fmt;
 
-	fmt = mtk_jpeg_find_format(ctx, f->fmt.pix_mp.pixelformat,
-				   MTK_JPEG_FMT_TYPE_OUTPUT);
+	fmt = mtk_jpeg_find_format(f->fmt.pix_mp.pixelformat,
+				   MTK_JPEG_FMT_FLAG_DEC_OUTPUT);
 	if (!fmt)
 		fmt = ctx->out_q.fmt;
 
@@ -310,17 +463,21 @@ static int mtk_jpeg_dec_try_fmt_vid_out_mplane(struct file *file, void *priv,
 		 (fmt->fourcc >> 16 & 0xff),
 		 (fmt->fourcc >> 24 & 0xff));
 
Quoted 7 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_try_fmt_vid_out_mplane(struct file *file, void *priv, · line 469

Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -1
-	return mtk_jpeg_try_fmt_mplane(f, fmt, ctx, MTK_JPEG_FMT_TYPE_OUTPUT);
+	if (ctx->state != MTK_JPEG_INIT) {
+		mtk_jpeg_g_fmt_vid_mplane(file, priv, f);
+		return 0;
+	}

Ditto.

done.

Quoted 100 lines
Quoted 100 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +46 -8
+
+	return vidioc_try_fmt(f, fmt);
 }
 
 static int mtk_jpeg_s_fmt_mplane(struct mtk_jpeg_ctx *ctx,
-				 struct v4l2_format *f)
+				 struct v4l2_format *f, unsigned int fmt_type)
 {
 	struct vb2_queue *vq;
 	struct mtk_jpeg_q_data *q_data = NULL;
 	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
 	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
-	unsigned int f_type;
 	int i;
 
 	vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, f->type);
@@ -334,12 +491,11 @@ static int mtk_jpeg_s_fmt_mplane(struct mtk_jpeg_ctx *ctx,
 		return -EBUSY;
 	}
 
-	f_type = V4L2_TYPE_IS_OUTPUT(f->type) ?
-			 MTK_JPEG_FMT_TYPE_OUTPUT : MTK_JPEG_FMT_TYPE_CAPTURE;
-
-	q_data->fmt = mtk_jpeg_find_format(ctx, pix_mp->pixelformat, f_type);
+	q_data->fmt = mtk_jpeg_find_format(pix_mp->pixelformat, fmt_type);
 	q_data->w = pix_mp->width;
 	q_data->h = pix_mp->height;
+	q_data->crop_rect.width = pix_mp->width;
+	q_data->crop_rect.height = pix_mp->height;
 	ctx->colorspace = pix_mp->colorspace;
 	ctx->ycbcr_enc = pix_mp->ycbcr_enc;
 	ctx->xfer_func = pix_mp->xfer_func;
@@ -365,6 +521,19 @@ static int mtk_jpeg_s_fmt_mplane(struct mtk_jpeg_ctx *ctx,
 	return 0;
 }
 
+static int mtk_jpeg_enc_s_fmt_vid_out_mplane(struct file *file, void *priv,
+					     struct v4l2_format *f)
+{
+	int ret;
+
+	ret = mtk_jpeg_enc_try_fmt_vid_out_mplane(file, priv, f);
+	if (ret)
+		return ret;
+
+	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f,
+				     MTK_JPEG_FMT_FLAG_ENC_OUTPUT);
+}
+
 static int mtk_jpeg_dec_s_fmt_vid_out_mplane(struct file *file, void *priv,
 					     struct v4l2_format *f)
 {
@@ -374,7 +543,21 @@ static int mtk_jpeg_dec_s_fmt_vid_out_mplane(struct file *file, void *priv,
 	if (ret)
 		return ret;
 
-	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f);
+	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f,
+				     MTK_JPEG_FMT_FLAG_DEC_OUTPUT);
+}
+
+static int mtk_jpeg_enc_s_fmt_vid_cap_mplane(struct file *file, void *priv,
+					     struct v4l2_format *f)
+{
+	int ret;
+
+	ret = mtk_jpeg_enc_try_fmt_vid_cap_mplane(file, priv, f);
+	if (ret)
+		return ret;
+
+	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f,
+				     MTK_JPEG_FMT_FLAG_ENC_CAPTURE);
 }
 
 static int mtk_jpeg_dec_s_fmt_vid_cap_mplane(struct file *file, void *priv,
@@ -386,7 +569,8 @@ static int mtk_jpeg_dec_s_fmt_vid_cap_mplane(struct file *file, void *priv,
 	if (ret)
 		return ret;
 
-	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f);
+	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f,
+				     MTK_JPEG_FMT_FLAG_DEC_CAPTURE);
 }
 
 static void mtk_jpeg_queue_src_chg_event(struct mtk_jpeg_ctx *ctx)
@@ -411,6 +595,29 @@ static int mtk_jpeg_subscribe_event(struct v4l2_fh *fh,
 	return v4l2_ctrl_subscribe_event(fh, sub);
 }
 
+static int mtk_jpeg_enc_g_selection(struct file *file, void *priv,
+				    struct v4l2_selection *s)
+{
+	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+
+	if (s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)
+		return -EINVAL;
+
+	switch (s->target) {
+	case V4L2_SEL_TGT_CROP:
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_subscribe_event(struct v4l2_fh *fh, · line 611

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -0
+	case V4L2_SEL_TGT_CROP_BOUNDS:
+	case V4L2_SEL_TGT_CROP_DEFAULT:
+		s->r.width = ctx->out_q.w;
+		s->r.height = ctx->out_q.h;

If TGT_CROP can be set, then g_selection should return ctx->out_q.crop_rect for that target, and not ctx->out_q.w/h. That's for CROP_BOUNDS/CROP_DEFAULT only.

done.

Quoted 17 lines
Quoted 17 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +9 -0
+		s->r.left = 0;
+		s->r.top = 0;
+		break;
+	default:
+		return -EINVAL;
+	}
+	return 0;
+}
+
 static int mtk_jpeg_dec_g_selection(struct file *file, void *priv,
 				    struct v4l2_selection *s)
 {
@@ -440,6 +647,29 @@ static int mtk_jpeg_dec_g_selection(struct file *file, void *priv,
 	return 0;
 }
 
Quoted 14 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_g_selection(struct file *file, void *priv, · line 661

Quoted 12 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +12 -0
+static int mtk_jpeg_enc_s_selection(struct file *file, void *priv,
+				    struct v4l2_selection *s)
+{
+	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+
+	if (s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)
+		return -EINVAL;
+
+	switch (s->target) {
+	case V4L2_SEL_TGT_CROP:
+		s->r.left = 0;
+		s->r.top = 0;

Is it a hardware limitation that left/top can only be 0?

yes.

Quoted 6 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_g_selection(struct file *file, void *priv, · line 664

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -0
+		s->r.width = min(s->r.width, ctx->out_q.w);
+		s->r.height = min(s->r.height, ctx->out_q.h);
+		ctx->out_q.crop_rect = s->r;

I'd rename crop_rect to enc_crop_rect.

done.

Quoted 17 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_g_selection(struct file *file, void *priv, · line 675

Quoted 12 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +8 -0
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
 static int mtk_jpeg_dec_s_selection(struct file *file, void *priv,
 				    struct v4l2_selection *s)
 {

I also looked at the current decoder selection code and I see no reason why s_selection is supported at all for the decoder since you can't actually set a compose rectangle, whatever you pass is just ignored.

Quoted 3 lines

I would recommend that mtk_jpeg_dec_s_selection() is just removed in a separate patch. The presence of that ioctl suggests that you can set the compose rectangle when in reality you can't. That can be confusing for applications.

done.

Quoted 347 lines
Quoted 335 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +240 -10
@@ -484,6 +714,33 @@ static int mtk_jpeg_qbuf(struct file *file, void *priv, struct v4l2_buffer *buf)
 	return v4l2_m2m_qbuf(file, fh->m2m_ctx, buf);
 }
 
+static const struct v4l2_ioctl_ops mtk_jpeg_enc_ioctl_ops = {
+	.vidioc_querycap                = mtk_jpeg_enc_querycap,
+	.vidioc_enum_fmt_vid_cap	= mtk_jpeg_enc_enum_fmt_vid_cap,
+	.vidioc_enum_fmt_vid_out	= mtk_jpeg_enc_enum_fmt_vid_out,
+	.vidioc_try_fmt_vid_cap_mplane	= mtk_jpeg_enc_try_fmt_vid_cap_mplane,
+	.vidioc_try_fmt_vid_out_mplane	= mtk_jpeg_enc_try_fmt_vid_out_mplane,
+	.vidioc_g_fmt_vid_cap_mplane    = mtk_jpeg_g_fmt_vid_mplane,
+	.vidioc_g_fmt_vid_out_mplane    = mtk_jpeg_g_fmt_vid_mplane,
+	.vidioc_s_fmt_vid_cap_mplane    = mtk_jpeg_enc_s_fmt_vid_cap_mplane,
+	.vidioc_s_fmt_vid_out_mplane    = mtk_jpeg_enc_s_fmt_vid_out_mplane,
+	.vidioc_qbuf                    = mtk_jpeg_qbuf,
+	.vidioc_subscribe_event         = mtk_jpeg_subscribe_event,
+	.vidioc_g_selection		= mtk_jpeg_enc_g_selection,
+	.vidioc_s_selection		= mtk_jpeg_enc_s_selection,
+
+	.vidioc_create_bufs		= v4l2_m2m_ioctl_create_bufs,
+	.vidioc_prepare_buf		= v4l2_m2m_ioctl_prepare_buf,
+	.vidioc_reqbufs                 = v4l2_m2m_ioctl_reqbufs,
+	.vidioc_querybuf                = v4l2_m2m_ioctl_querybuf,
+	.vidioc_dqbuf                   = v4l2_m2m_ioctl_dqbuf,
+	.vidioc_expbuf                  = v4l2_m2m_ioctl_expbuf,
+	.vidioc_streamon                = v4l2_m2m_ioctl_streamon,
+	.vidioc_streamoff               = v4l2_m2m_ioctl_streamoff,
+
+	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
+};
+
 static const struct v4l2_ioctl_ops mtk_jpeg_dec_ioctl_ops = {
 	.vidioc_querycap                = mtk_jpeg_dec_querycap,
 	.vidioc_enum_fmt_vid_cap	= mtk_jpeg_dec_enum_fmt_vid_cap,
@@ -575,8 +832,9 @@ static bool mtk_jpeg_check_resolution_change(struct mtk_jpeg_ctx *ctx,
 	}
 
 	q_data = &ctx->cap_q;
-	if (q_data->fmt != mtk_jpeg_find_format(ctx, param->dst_fourcc,
-						MTK_JPEG_FMT_TYPE_CAPTURE)) {
+	if (q_data->fmt !=
+	    mtk_jpeg_find_format(param->dst_fourcc,
+				 MTK_JPEG_FMT_FLAG_DEC_CAPTURE)) {
 		v4l2_dbg(1, debug, &jpeg->v4l2_dev, "format change\n");
 		return true;
 	}
@@ -597,9 +855,8 @@ static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx,
 	q_data = &ctx->cap_q;
 	q_data->w = param->dec_w;
 	q_data->h = param->dec_h;
-	q_data->fmt = mtk_jpeg_find_format(ctx,
-					   param->dst_fourcc,
-					   MTK_JPEG_FMT_TYPE_CAPTURE);
+	q_data->fmt = mtk_jpeg_find_format(param->dst_fourcc,
+					   MTK_JPEG_FMT_FLAG_DEC_CAPTURE);
 
 	for (i = 0; i < q_data->fmt->colplanes; i++) {
 		q_data->bytesperline[i] = param->mem_stride[i];
@@ -616,6 +873,17 @@ static void mtk_jpeg_set_queue_data(struct mtk_jpeg_ctx *ctx,
 		 param->dec_w, param->dec_h);
 }
 
+static void mtk_jpeg_enc_buf_queue(struct vb2_buffer *vb)
+{
+	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+
+	v4l2_dbg(2, debug, &jpeg->v4l2_dev, "(%d) buf_q id=%d, vb=%p\n",
+		 vb->vb2_queue->type, vb->index, vb);
+
+	v4l2_m2m_buf_queue(ctx->fh.m2m_ctx, to_vb2_v4l2_buffer(vb));
+}
+
 static void mtk_jpeg_dec_buf_queue(struct vb2_buffer *vb)
 {
 	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
@@ -664,6 +932,15 @@ static struct vb2_v4l2_buffer *mtk_jpeg_buf_remove(struct mtk_jpeg_ctx *ctx,
 		return v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
 }
 
+static void mtk_jpeg_enc_stop_streaming(struct vb2_queue *q)
+{
+	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(q);
+	struct vb2_v4l2_buffer *vb;
+
+	while ((vb = mtk_jpeg_buf_remove(ctx, q->type)))
+		v4l2_m2m_buf_done(vb, VB2_BUF_STATE_ERROR);
+}
+
 static void mtk_jpeg_dec_stop_streaming(struct vb2_queue *q)
 {
 	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(q);
@@ -699,6 +976,15 @@ static const struct vb2_ops mtk_jpeg_dec_qops = {
 	.stop_streaming     = mtk_jpeg_dec_stop_streaming,
 };
 
+static const struct vb2_ops mtk_jpeg_enc_qops = {
+	.queue_setup        = mtk_jpeg_queue_setup,
+	.buf_prepare        = mtk_jpeg_buf_prepare,
+	.buf_queue          = mtk_jpeg_enc_buf_queue,
+	.wait_prepare       = vb2_ops_wait_prepare,
+	.wait_finish        = vb2_ops_wait_finish,
+	.stop_streaming     = mtk_jpeg_enc_stop_streaming,
+};
+
 static void mtk_jpeg_set_dec_src(struct mtk_jpeg_ctx *ctx,
 				 struct vb2_buffer *src_buf,
 				 struct mtk_jpeg_bs *bs)
@@ -736,6 +1022,85 @@ static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx,
 	return 0;
 }
 
+static void mtk_jpeg_set_enc_dst(struct mtk_jpeg_ctx *ctx, void __iomem *base,
+				 struct vb2_buffer *dst_buf,
+				 struct mtk_jpeg_enc_bs *bs)
+{
+	bs->dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
+	bs->dma_addr_offset = ctx->enable_exif ? MTK_JPEG_MAX_EXIF_SIZE : 0;
+	bs->dma_addr_offsetmask = bs->dma_addr & JPEG_ENC_DST_ADDR_OFFSET_MASK;
+	bs->size = vb2_plane_size(dst_buf, 0);
+
+	mtk_jpeg_enc_set_dst_addr(base, bs->dma_addr, bs->size,
+				  bs->dma_addr_offset,
+				  bs->dma_addr_offsetmask);
+}
+
+static void mtk_jpeg_set_enc_src(struct mtk_jpeg_ctx *ctx, void __iomem *base,
+				 struct vb2_buffer *src_buf)
+{
+	int i;
+	dma_addr_t	dma_addr;
+
+	mtk_jpeg_enc_set_img_size(base, ctx->out_q.crop_rect.width,
+				  ctx->out_q.crop_rect.height);
+	mtk_jpeg_enc_set_blk_num(base, ctx->out_q.fmt->fourcc,
+				 ctx->out_q.crop_rect.width,
+				 ctx->out_q.crop_rect.height);
+	mtk_jpeg_enc_set_stride(base, ctx->out_q.fmt->fourcc, ctx->out_q.w,
+				ctx->out_q.h, ctx->out_q.bytesperline[0]);
+
+	for (i = 0; i < src_buf->num_planes; i++) {
+		dma_addr = vb2_dma_contig_plane_dma_addr(src_buf, i) +
+			   src_buf->planes[i].data_offset;
+		mtk_jpeg_enc_set_src_addr(base, dma_addr, i);
+	}
+}
+
+static void mtk_jpeg_enc_device_run(void *priv)
+{
+	struct mtk_jpeg_ctx *ctx = priv;
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+	struct vb2_v4l2_buffer *src_buf, *dst_buf;
+	enum vb2_buffer_state buf_state = VB2_BUF_STATE_ERROR;
+	unsigned long flags;
+	struct mtk_jpeg_src_buf *jpeg_src_buf;
+	struct mtk_jpeg_enc_bs enc_bs;
+	int ret;
+
+	src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
+	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
+	jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(&src_buf->vb2_buf);
+
+	ret = pm_runtime_get_sync(jpeg->dev);
+	if (ret < 0)
+		goto enc_end;
+
+	spin_lock_irqsave(&jpeg->hw_lock, flags);
+
+	/*
+	 * Resetting the hardware every frame is to ensure that all the
+	 * registers are cleared. This is a hardware requirement.
+	 */
+	mtk_jpeg_enc_reset(jpeg->reg_base);
+
+	mtk_jpeg_set_enc_dst(ctx, jpeg->reg_base, &dst_buf->vb2_buf, &enc_bs);
+	mtk_jpeg_set_enc_src(ctx, jpeg->reg_base, &src_buf->vb2_buf);
+	mtk_jpeg_enc_set_config(jpeg->reg_base, ctx->out_q.fmt->hw_format,
+				ctx->enable_exif, ctx->enc_quality,
+				ctx->restart_interval);
+	mtk_jpeg_enc_start(jpeg->reg_base);
+	spin_unlock_irqrestore(&jpeg->hw_lock, flags);
+	return;
+
+enc_end:
+	v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+	v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+	v4l2_m2m_buf_done(src_buf, buf_state);
+	v4l2_m2m_buf_done(dst_buf, buf_state);
+	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
+}
+
 static void mtk_jpeg_dec_device_run(void *priv)
 {
 	struct mtk_jpeg_ctx *ctx = priv;
@@ -785,6 +1150,11 @@ static void mtk_jpeg_dec_device_run(void *priv)
 	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
 }
 
+static int mtk_jpeg_enc_job_ready(void *priv)
+{
+		return 1;
+}
+
 static int mtk_jpeg_dec_job_ready(void *priv)
 {
 	struct mtk_jpeg_ctx *ctx = priv;
@@ -792,6 +1162,11 @@ static int mtk_jpeg_dec_job_ready(void *priv)
 	return (ctx->state == MTK_JPEG_RUNNING) ? 1 : 0;
 }
 
+static const struct v4l2_m2m_ops mtk_jpeg_enc_m2m_ops = {
+	.device_run = mtk_jpeg_enc_device_run,
+	.job_ready  = mtk_jpeg_enc_job_ready,
+};
+
 static const struct v4l2_m2m_ops mtk_jpeg_dec_m2m_ops = {
 	.device_run = mtk_jpeg_dec_device_run,
 	.job_ready  = mtk_jpeg_dec_job_ready,
@@ -830,24 +1205,109 @@ static int mtk_jpeg_dec_queue_init(void *priv, struct vb2_queue *src_vq,
 	return ret;
 }
 
-static void mtk_jpeg_clk_on(struct mtk_jpeg_dev *jpeg)
+static int mtk_jpeg_enc_queue_init(void *priv, struct vb2_queue *src_vq,
+				   struct vb2_queue *dst_vq)
 {
+	struct mtk_jpeg_ctx *ctx = priv;
 	int ret;
 
+	src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+	src_vq->io_modes = VB2_DMABUF | VB2_MMAP;
+	src_vq->drv_priv = ctx;
+	src_vq->buf_struct_size = sizeof(struct mtk_jpeg_src_buf);
+	src_vq->ops = &mtk_jpeg_enc_qops;
+	src_vq->mem_ops = &vb2_dma_contig_memops;
+	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	src_vq->lock = &ctx->jpeg->lock;
+	src_vq->dev = ctx->jpeg->dev;
+	ret = vb2_queue_init(src_vq);
+	if (ret)
+		return ret;
+
+	dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+	dst_vq->io_modes = VB2_DMABUF | VB2_MMAP;
+	dst_vq->drv_priv = ctx;
+	dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
+	dst_vq->ops = &mtk_jpeg_enc_qops;
+	dst_vq->mem_ops = &vb2_dma_contig_memops;
+	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	dst_vq->lock = &ctx->jpeg->lock;
+	dst_vq->dev = ctx->jpeg->dev;
+	ret = vb2_queue_init(dst_vq);
+
+	return ret;
+}
+
+static void mtk_jpeg_clk_on(struct mtk_jpeg_dev *jpeg)
+{
+	int ret, i;
+
 	ret = mtk_smi_larb_get(jpeg->larb);
 	if (ret)
 		dev_err(jpeg->dev, "mtk_smi_larb_get larbvdec fail %d\n", ret);
-	clk_prepare_enable(jpeg->clk_jdec_smi);
-	clk_prepare_enable(jpeg->clk_jdec);
+
+	for (i = 0; i < jpeg->variant->num_clocks; i++) {
+		ret = clk_prepare_enable(jpeg->clocks[i]);
+		if (ret) {
+			while (--i >= 0)
+				clk_disable_unprepare(jpeg->clocks[i]);
+		}
+	}
 }
 
 static void mtk_jpeg_clk_off(struct mtk_jpeg_dev *jpeg)
 {
-	clk_disable_unprepare(jpeg->clk_jdec);
-	clk_disable_unprepare(jpeg->clk_jdec_smi);
+	int i;
+
+	for (i = jpeg->variant->num_clocks - 1; i >= 0; i--)
+		clk_disable_unprepare(jpeg->clocks[i]);
 	mtk_smi_larb_put(jpeg->larb);
 }
 
+static irqreturn_t mtk_jpeg_enc_irq(int irq, void *priv)
+{
+	struct mtk_jpeg_dev *jpeg = priv;
+	struct mtk_jpeg_ctx *ctx;
+	struct vb2_v4l2_buffer *src_buf, *dst_buf;
+	struct mtk_jpeg_src_buf *jpeg_src_buf;
+	enum vb2_buffer_state buf_state = VB2_BUF_STATE_ERROR;
+	u32 enc_irq_ret;
+	u32 enc_ret, result_size;
+
+	ctx = v4l2_m2m_get_curr_priv(jpeg->m2m_dev);
+	if (!ctx) {
+		v4l2_err(&jpeg->v4l2_dev, "Context is NULL\n");
+		return IRQ_HANDLED;
+	}
+
+	src_buf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+	dst_buf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+	jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(&src_buf->vb2_buf);
+
+	enc_ret = mtk_jpeg_enc_get_and_clear_int_status(jpeg->reg_base);
+	enc_irq_ret = mtk_jpeg_enc_enum_result(jpeg->reg_base, enc_ret);
+
+	if (enc_irq_ret >= MTK_JPEG_ENC_RESULT_STALL)
+		mtk_jpeg_enc_reset(jpeg->reg_base);
+
+	if (enc_irq_ret != MTK_JPEG_ENC_RESULT_DONE) {
+		dev_err(jpeg->dev, "encode failed\n");
+		goto enc_end;
+	}
+
+	result_size = mtk_jpeg_enc_get_file_size(jpeg->reg_base);
+	vb2_set_plane_payload(&dst_buf->vb2_buf, 0, result_size);
+
+	buf_state = VB2_BUF_STATE_DONE;
+
+enc_end:
+	v4l2_m2m_buf_done(src_buf, buf_state);
+	v4l2_m2m_buf_done(dst_buf, buf_state);
+	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
+	pm_runtime_put(ctx->jpeg->dev);
+	return IRQ_HANDLED;
+}
+
 static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv)
 {
 	struct mtk_jpeg_dev *jpeg = priv;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1361

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
@@ -893,36 +1353,130 @@ static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv)
 	return IRQ_HANDLED;
 }
 
+static void mtk_jpeg_set_enc_default_params(struct mtk_jpeg_ctx *ctx)
+{
+	struct mtk_jpeg_q_data *q = &ctx->out_q;
+	struct v4l2_pix_format_mplane *pix_mp;
+
+	pix_mp = kmalloc(sizeof(*pix_mp), GFP_KERNEL);

Huh? It's allocated, but not freed?

Quoted 3 lines

This appears to be a temporary struct, so why not just do:

	struct v4l2_pix_format_mplane pix_mp = {};

refactor this function.

Quoted 17 lines
Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -0
+
+	ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+	ctx->colorspace = V4L2_COLORSPACE_JPEG,

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1374

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+	ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
+	ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+	pix_mp->width = MTK_JPEG_MIN_WIDTH;
+	pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+
+	q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_YUYV,
+				      MTK_JPEG_FMT_FLAG_ENC_OUTPUT);
+	vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+				    fmt.pix_mp), q->fmt);

Yuck. Just change the vidioc_try_fmt prototype where the first argument is a v4l2_pix_format_mplane pointer instead of a v4l2_format pointer.

done.

Quoted 27 lines
Quoted 27 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +21 -0
+	q->w = pix_mp->width;
+	q->h = pix_mp->height;
+	q->crop_rect.width = pix_mp->width;
+	q->crop_rect.height = pix_mp->height;
+	q->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
+	q->bytesperline[0] = pix_mp->plane_fmt[0].bytesperline;
+
+	q = &ctx->cap_q;
+	q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_JPEG,
+				      MTK_JPEG_FMT_FLAG_ENC_CAPTURE);
+	pix_mp->width = MTK_JPEG_MIN_WIDTH;
+	pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+	vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+				    fmt.pix_mp), q->fmt);
+	q->w = pix_mp->width;
+	q->h = pix_mp->height;
+	q->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
+	q->bytesperline[0] = pix_mp->plane_fmt[0].bytesperline;
+}
+
 static void mtk_jpeg_set_dec_default_params(struct mtk_jpeg_ctx *ctx)
 {
 	struct mtk_jpeg_q_data *q = &ctx->out_q;
+	struct v4l2_pix_format_mplane *pix_mp;
 	int i;
 
Quoted 3 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1401

Quoted 1 line
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -0
+	pix_mp = kmalloc(sizeof(*pix_mp), GFP_KERNEL);

Ditto.

done.

Quoted 729 lines
Quoted 716 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +143 -31
+
+	ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
 	ctx->colorspace = V4L2_COLORSPACE_JPEG,
 	ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
 	ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
 	ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;
-
-	q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_JPEG,
-					      MTK_JPEG_FMT_TYPE_OUTPUT);
-	q->w = MTK_JPEG_MIN_WIDTH;
-	q->h = MTK_JPEG_MIN_HEIGHT;
-	q->bytesperline[0] = 0;
-	q->sizeimage[0] = MTK_JPEG_DEFAULT_SIZEIMAGE;
+	pix_mp->width = MTK_JPEG_MIN_WIDTH;
+	pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+
+	q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_JPEG,
+				      MTK_JPEG_FMT_FLAG_DEC_OUTPUT);
+	vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+				    fmt.pix_mp), q->fmt);
+	q->w = pix_mp->width;
+	q->h = pix_mp->height;
+	q->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
+	q->bytesperline[0] = pix_mp->plane_fmt[0].bytesperline;
 
 	q = &ctx->cap_q;
-	q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_YUV420M,
-					      MTK_JPEG_FMT_TYPE_CAPTURE);
-	q->w = MTK_JPEG_MIN_WIDTH;
-	q->h = MTK_JPEG_MIN_HEIGHT;
-
+	q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_YUV420M,
+				      MTK_JPEG_FMT_FLAG_DEC_CAPTURE);
+	pix_mp->width = MTK_JPEG_MIN_WIDTH;
+	pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+	vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+				    fmt.pix_mp), q->fmt);
+	q->w = pix_mp->width;
+	q->h = pix_mp->height;
 	for (i = 0; i < q->fmt->colplanes; i++) {
-		u32 stride = q->w * q->fmt->h_sample[i] / 4;
-		u32 h = q->h * q->fmt->v_sample[i] / 4;
+		q->sizeimage[i] = pix_mp->plane_fmt[i].sizeimage;
+		q->bytesperline[i] = pix_mp->plane_fmt[i].bytesperline;
+	}
+}
 
-		q->bytesperline[i] = stride;
-		q->sizeimage[i] = stride * h;
+static int mtk_jpeg_enc_open(struct file *file)
+{
+	struct mtk_jpeg_dev *jpeg = video_drvdata(file);
+	struct video_device *vfd = video_devdata(file);
+	struct mtk_jpeg_ctx *ctx;
+	int ret = 0;
+
+	ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+	if (!ctx)
+		return -ENOMEM;
+
+	if (mutex_lock_interruptible(&jpeg->lock)) {
+		ret = -ERESTARTSYS;
+		goto free;
+	}
+
+	v4l2_fh_init(&ctx->fh, vfd);
+	file->private_data = &ctx->fh;
+	v4l2_fh_add(&ctx->fh);
+
+	ctx->jpeg = jpeg;
+	ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(jpeg->m2m_dev, ctx,
+					    mtk_jpeg_enc_queue_init);
+	if (IS_ERR(ctx->fh.m2m_ctx)) {
+		ret = PTR_ERR(ctx->fh.m2m_ctx);
+		goto error;
 	}
+
+	ret = mtk_jpeg_enc_ctrls_setup(ctx);
+	if (ret) {
+		v4l2_err(&jpeg->v4l2_dev, "Failed to setup jpeg enc controls\n");
+		goto error;
+	}
+	mtk_jpeg_set_enc_default_params(ctx);
+
+	mutex_unlock(&jpeg->lock);
+	return 0;
+
+error:
+	v4l2_fh_del(&ctx->fh);
+	v4l2_fh_exit(&ctx->fh);
+	mutex_unlock(&jpeg->lock);
+free:
+	kfree(ctx);
+	return ret;
 }
 
 static int mtk_jpeg_dec_open(struct file *file)
@@ -953,6 +1507,12 @@ static int mtk_jpeg_dec_open(struct file *file)
 		goto error;
 	}
 
+	v4l2_ctrl_handler_init(&ctx->ctrl_hdl, 0);
+	ret = v4l2_ctrl_handler_setup(&ctx->ctrl_hdl);
+	if (ret) {
+		v4l2_err(&jpeg->v4l2_dev, "Failed to setup jpeg dec controls\n");
+		goto error;
+	}
 	mtk_jpeg_set_dec_default_params(ctx);
 	mutex_unlock(&jpeg->lock);
 	return 0;
@@ -973,6 +1533,7 @@ static int mtk_jpeg_release(struct file *file)
 
 	mutex_lock(&jpeg->lock);
 	v4l2_m2m_ctx_release(ctx->fh.m2m_ctx);
+	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
 	v4l2_fh_del(&ctx->fh);
 	v4l2_fh_exit(&ctx->fh);
 	kfree(ctx);
@@ -980,6 +1541,15 @@ static int mtk_jpeg_release(struct file *file)
 	return 0;
 }
 
+static const struct v4l2_file_operations mtk_jpeg_enc_fops = {
+	.owner          = THIS_MODULE,
+	.open           = mtk_jpeg_enc_open,
+	.release        = mtk_jpeg_release,
+	.poll           = v4l2_m2m_fop_poll,
+	.unlocked_ioctl = video_ioctl2,
+	.mmap           = v4l2_m2m_fop_mmap,
+};
+
 static const struct v4l2_file_operations mtk_jpeg_dec_fops = {
 	.owner          = THIS_MODULE,
 	.open           = mtk_jpeg_dec_open,
@@ -993,6 +1563,7 @@ static int mtk_jpeg_clk_init(struct mtk_jpeg_dev *jpeg)
 {
 	struct device_node *node;
 	struct platform_device *pdev;
+	int i;
 
 	node = of_parse_phandle(jpeg->dev->of_node, "mediatek,larb", 0);
 	if (!node)
@@ -1006,12 +1577,17 @@ static int mtk_jpeg_clk_init(struct mtk_jpeg_dev *jpeg)
 
 	jpeg->larb = &pdev->dev;
 
-	jpeg->clk_jdec = devm_clk_get(jpeg->dev, "jpgdec");
-	if (IS_ERR(jpeg->clk_jdec))
-		return PTR_ERR(jpeg->clk_jdec);
+	for (i = 0; i < jpeg->variant->num_clocks; i++) {
+		jpeg->clocks[i] = devm_clk_get(jpeg->dev,
+					       jpeg->variant->clk_names[i]);
+		if (IS_ERR(jpeg->clocks[i])) {
+			dev_err(&pdev->dev, "failed to get clock: %s\n",
+				jpeg->variant->clk_names[i]);
+			return PTR_ERR(jpeg->clocks[i]);
+		}
+	}
 
-	jpeg->clk_jdec_smi = devm_clk_get(jpeg->dev, "jpgdec-smi");
-	return PTR_ERR_OR_ZERO(jpeg->clk_jdec_smi);
+	return 0;
 }
 
 static int mtk_jpeg_probe(struct platform_device *pdev)
@@ -1028,6 +1604,7 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 	mutex_init(&jpeg->lock);
 	spin_lock_init(&jpeg->hw_lock);
 	jpeg->dev = &pdev->dev;
+	jpeg->variant = of_device_get_match_data(jpeg->dev);
 
 	res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
 	jpeg->reg_base = devm_ioremap_resource(&pdev->dev, res);
@@ -1042,8 +1619,12 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 		return jpeg_irq;
 	}
 
-	ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_dec_irq, 0,
-			       pdev->name, jpeg);
+	if (jpeg->variant->is_encoder)
+		ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_enc_irq,
+				       0, pdev->name, jpeg);
+	else
+		ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_dec_irq,
+				       0, pdev->name, jpeg);
 	if (ret) {
 		dev_err(&pdev->dev, "Failed to request jpeg_irq %d (%d)\n",
 			jpeg_irq, ret);
@@ -1063,7 +1644,10 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 		goto err_dev_register;
 	}
 
-	jpeg->m2m_dev = v4l2_m2m_init(&mtk_jpeg_dec_m2m_ops);
+	if (jpeg->variant->is_encoder)
+		jpeg->m2m_dev = v4l2_m2m_init(&mtk_jpeg_enc_m2m_ops);
+	else
+		jpeg->m2m_dev = v4l2_m2m_init(&mtk_jpeg_dec_m2m_ops);
 	if (IS_ERR(jpeg->m2m_dev)) {
 		v4l2_err(&jpeg->v4l2_dev, "Failed to init mem2mem device\n");
 		ret = PTR_ERR(jpeg->m2m_dev);
@@ -1076,9 +1660,15 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 		goto err_vfd_jpeg_alloc;
 	}
 	snprintf(jpeg->vdev->name, sizeof(jpeg->vdev->name),
-		 "%s-dec", MTK_JPEG_NAME);
-	jpeg->vdev->fops = &mtk_jpeg_dec_fops;
-	jpeg->vdev->ioctl_ops = &mtk_jpeg_dec_ioctl_ops;
+		 "%s-%s", MTK_JPEG_NAME,
+		 jpeg->variant->is_encoder ? "enc" : "dec");
+	if (jpeg->variant->is_encoder) {
+		jpeg->vdev->fops = &mtk_jpeg_enc_fops;
+		jpeg->vdev->ioctl_ops = &mtk_jpeg_enc_ioctl_ops;
+	} else {
+		jpeg->vdev->fops = &mtk_jpeg_dec_fops;
+		jpeg->vdev->ioctl_ops = &mtk_jpeg_dec_ioctl_ops;
+	}
 	jpeg->vdev->minor = -1;
 	jpeg->vdev->release = video_device_release;
 	jpeg->vdev->lock = &jpeg->lock;
@@ -1095,8 +1685,9 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 
 	video_set_drvdata(jpeg->vdev, jpeg);
 	v4l2_info(&jpeg->v4l2_dev,
-		  "decoder device registered as /dev/video%d (%d,%d)\n",
-		  jpeg->vdev->num, VIDEO_MAJOR, jpeg->vdev->minor);
+		  "jpeg %s device registered as /dev/video%d (%d,%d)\n",
+		  jpeg->variant->is_encoder ? "enc" : "dec", jpeg->vdev->num,
+		  VIDEO_MAJOR, jpeg->vdev->minor);
 
 	platform_set_drvdata(pdev, jpeg);
 
@@ -1187,14 +1778,36 @@ static const struct dev_pm_ops mtk_jpeg_pm_ops = {
 	SET_RUNTIME_PM_OPS(mtk_jpeg_pm_suspend, mtk_jpeg_pm_resume, NULL)
 };
 
+static struct mtk_jpeg_variant mt8173_jpeg_drvdata = {
+	.is_encoder	= false,
+	.clk_names	= {"jpgdec-smi", "jpgdec"},
+	.num_clocks	= 2,
+};
+
+static struct mtk_jpeg_variant mt2701_jpeg_drvdata = {
+	.is_encoder	= false,
+	.clk_names	= {"jpgdec-smi", "jpgdec"},
+	.num_clocks	= 2,
+};
+
+static struct mtk_jpeg_variant mtk_jpeg_drvdata = {
+	.is_encoder	= true,
+	.clk_names	= {"jpgenc"},
+	.num_clocks	= 1,
+};
+
 static const struct of_device_id mtk_jpeg_match[] = {
 	{
 		.compatible = "mediatek,mt8173-jpgdec",
-		.data       = NULL,
+		.data = &mt8173_jpeg_drvdata,
 	},
 	{
 		.compatible = "mediatek,mt2701-jpgdec",
-		.data       = NULL,
+		.data = &mt2701_jpeg_drvdata,
+	},
+	{
+		.compatible = "mediatek,mtk-jpgenc",
+		.data = &mtk_jpeg_drvdata,
 	},
 	{},
 };
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +35 -9
index 0b59e48495d5..9ec2c3350a16 100644
@@ -3,6 +3,7 @@
  * Copyright (c) 2016 MediaTek Inc.
  * Author: Ming Hsiu Tsai <[email protected]>
  *         Rick Chang <[email protected]>
+ *         Xia Jiang <[email protected]>
  */
 
 #ifndef _MTK_JPEG_CORE_H
@@ -16,19 +17,21 @@
 #define MTK_JPEG_NAME		"mtk-jpeg"
 
 #define MTK_JPEG_COMP_MAX		3
+#define MTK_JPEG_MAX_CLOCKS		2
+
 
 #define MTK_JPEG_FMT_FLAG_DEC_OUTPUT	BIT(0)
 #define MTK_JPEG_FMT_FLAG_DEC_CAPTURE	BIT(1)
-
-#define MTK_JPEG_FMT_TYPE_OUTPUT	1
-#define MTK_JPEG_FMT_TYPE_CAPTURE	2
+#define MTK_JPEG_FMT_FLAG_ENC_OUTPUT	BIT(2)
+#define MTK_JPEG_FMT_FLAG_ENC_CAPTURE	BIT(3)
 
 #define MTK_JPEG_MIN_WIDTH	32U
 #define MTK_JPEG_MIN_HEIGHT	32U
-#define MTK_JPEG_MAX_WIDTH	8192U
-#define MTK_JPEG_MAX_HEIGHT	8192U
+#define MTK_JPEG_MAX_WIDTH	65535U
+#define MTK_JPEG_MAX_HEIGHT	65535U
 
 #define MTK_JPEG_DEFAULT_SIZEIMAGE	(1 * 1024 * 1024)
+#define MTK_JPEG_MAX_EXIF_SIZE	(64 * 1024)
 
 /**
  * enum mtk_jpeg_ctx_state - states of the context state machine
@@ -42,6 +45,18 @@ enum mtk_jpeg_ctx_state {
 	MTK_JPEG_SOURCE_CHANGE,
 };
 
+/**
+ * mtk_jpeg_variant - mtk jpeg driver variant
+ * @is_encoder:		driver mode is jpeg encoder
+ * @clk_names:		clock names
+ * @num_clocks:		numbers of clock
+ */
+struct mtk_jpeg_variant {
+	bool is_encoder;
+	const char		*clk_names[MTK_JPEG_MAX_CLOCKS];
+	int			num_clocks;
+};
+
 /**
  * struct mt_jpeg - JPEG IP abstraction
  * @lock:		the mutex protecting this structure
@@ -53,9 +68,9 @@ enum mtk_jpeg_ctx_state {
  * @alloc_ctx:		videobuf2 memory allocator's context
  * @vdev:		video device node for jpeg mem2mem mode
  * @reg_base:		JPEG registers mapping
- * @clk_jdec:		JPEG hw working clock
- * @clk_jdec_smi:	JPEG SMI bus clock
  * @larb:		SMI device
+ * @clocks:		JPEG IP clock(s)
+ * @variant:		driver variant to be used
  */
 struct mtk_jpeg_dev {
 	struct mutex		lock;
@@ -67,14 +82,15 @@ struct mtk_jpeg_dev {
 	void			*alloc_ctx;
 	struct video_device	*vdev;
 	void __iomem		*reg_base;
-	struct clk		*clk_jdec;
-	struct clk		*clk_jdec_smi;
 	struct device		*larb;
+	struct clk		*clocks[MTK_JPEG_MAX_CLOCKS];
+	const struct mtk_jpeg_variant *variant;
 };
 
 /**
  * struct jpeg_fmt - driver's internal color format data
  * @fourcc:	the fourcc code, 0 if not applicable
+ * @hw_format:	hardware format value
  * @h_sample:	horizontal sample count of plane in 4 * 4 pixel image
  * @v_sample:	vertical sample count of plane in 4 * 4 pixel image
  * @colplanes:	number of color planes (1 for packed formats)
@@ -84,6 +100,7 @@ struct mtk_jpeg_dev {
  */
 struct mtk_jpeg_fmt {
 	u32	fourcc;
+	u32	hw_format;
 	int	h_sample[VIDEO_MAX_PLANES];
 	int	v_sample[VIDEO_MAX_PLANES];
 	int	colplanes;
@@ -107,6 +124,7 @@ struct mtk_jpeg_q_data {
 	u32			h;
 	u32			bytesperline[VIDEO_MAX_PLANES];
 	u32			sizeimage[VIDEO_MAX_PLANES];
+	struct v4l2_rect	crop_rect;
 };
 
 /**
@@ -116,6 +134,10 @@ struct mtk_jpeg_q_data {
  * @cap_q:		destination (capture) queue queue information
  * @fh:			V4L2 file handle
  * @state:		state of the context
+ * @enable_exif:	enable exif mode of jpeg encoder
+ * @enc_quality:	jpeg encoder quality
+ * @restart_interval:	jpeg encoder restart interval
+ * @ctrl_hdl:		controls handler
  * @colorspace: enum v4l2_colorspace; supplemental to pixelformat
  * @ycbcr_enc: enum v4l2_ycbcr_encoding, Y'CbCr encoding
  * @quantization: enum v4l2_quantization, colorspace quantization
@@ -127,6 +149,10 @@ struct mtk_jpeg_ctx {
 	struct mtk_jpeg_q_data		cap_q;
 	struct v4l2_fh			fh;
 	enum mtk_jpeg_ctx_state		state;
+	bool				enable_exif;
+	u8				enc_quality;
+	u8				restart_interval;
+	struct v4l2_ctrl_handler	ctrl_hdl;
 
 	enum v4l2_colorspace colorspace;
 	enum v4l2_ycbcr_encoding ycbcr_enc;
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +193 -0
new file mode 100644
index 000000000000..7fc1de920a75
@@ -0,0 +1,193 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ * Author: Xia Jiang <[email protected]>
+ *
+ */
+
+#include <linux/io.h>
+#include <linux/kernel.h>
+#include <media/videobuf2-core.h>
+
+#include "mtk_jpeg_enc_hw.h"
+
+static const struct mtk_jpeg_enc_qlt mtk_jpeg_enc_quality[] = {
+	{.quality_param = 34, .hardware_value = JPEG_ENC_QUALITY_Q34},
+	{.quality_param = 39, .hardware_value = JPEG_ENC_QUALITY_Q39},
+	{.quality_param = 48, .hardware_value = JPEG_ENC_QUALITY_Q48},
+	{.quality_param = 60, .hardware_value = JPEG_ENC_QUALITY_Q60},
+	{.quality_param = 64, .hardware_value = JPEG_ENC_QUALITY_Q64},
+	{.quality_param = 68, .hardware_value = JPEG_ENC_QUALITY_Q68},
+	{.quality_param = 74, .hardware_value = JPEG_ENC_QUALITY_Q74},
+	{.quality_param = 80, .hardware_value = JPEG_ENC_QUALITY_Q80},
+	{.quality_param = 82, .hardware_value = JPEG_ENC_QUALITY_Q82},
+	{.quality_param = 84, .hardware_value = JPEG_ENC_QUALITY_Q84},
+	{.quality_param = 87, .hardware_value = JPEG_ENC_QUALITY_Q87},
+	{.quality_param = 90, .hardware_value = JPEG_ENC_QUALITY_Q90},
+	{.quality_param = 92, .hardware_value = JPEG_ENC_QUALITY_Q92},
+	{.quality_param = 95, .hardware_value = JPEG_ENC_QUALITY_Q95},
+	{.quality_param = 97, .hardware_value = JPEG_ENC_QUALITY_Q97},
+};
+
+void mtk_jpeg_enc_reset(void __iomem *base)
+{
+	writel(0x00, base + JPEG_ENC_RSTB);
+	writel(JPEG_ENC_RESET_BIT, base + JPEG_ENC_RSTB);
+	writel(0x00, base + JPEG_ENC_CODEC_SEL);
+}
+
+u32 mtk_jpeg_enc_get_and_clear_int_status(void __iomem *base)
+{
+	u32 ret;
+
+	ret = readl(base + JPEG_ENC_INT_STS) &
+		    JPEG_ENC_INT_STATUS_MASK_ALLIRQ;
+	if (ret)
+		writel(0, base + JPEG_ENC_INT_STS);
+
+	return ret;
+}
+
+u32 mtk_jpeg_enc_get_file_size(void __iomem *base)
+{
+	return readl(base + JPEG_ENC_DMA_ADDR0) -
+	       readl(base + JPEG_ENC_DST_ADDR0);
+}
+
+u32 mtk_jpeg_enc_enum_result(void __iomem *base, u32 irq_status)
+{
+	if (irq_status & JPEG_ENC_INT_STATUS_DONE)
+		return MTK_JPEG_ENC_RESULT_DONE;
+	else if (irq_status & JPEG_ENC_INT_STATUS_STALL)
+		return MTK_JPEG_ENC_RESULT_STALL;
+	else
+		return MTK_JPEG_ENC_RESULT_VCODEC_IRQ;
+}
+
+void mtk_jpeg_enc_set_img_size(void __iomem *base, u32 width, u32 height)
+{
+	u32 value;
+
+	value = width << 16 | height;
+	writel(value, base + JPEG_ENC_IMG_SIZE);
+}
+
+void mtk_jpeg_enc_set_blk_num(void __iomem *base, u32 enc_format, u32 width,
+			      u32 height)
+{
+	u32 blk_num;
+	u32 is_420;
+	u32 padding_width;
+	u32 padding_height;
+	u32 luma_blocks;
+	u32 chroma_blocks;
+
+	is_420 = (enc_format == V4L2_PIX_FMT_NV12M ||
+		  enc_format == V4L2_PIX_FMT_NV21M) ? 1 : 0;
+	padding_width = round_up(width, 16);
+	padding_height = round_up(height, is_420 ? 16 : 8);
+
+	luma_blocks = padding_width / 8 * padding_height / 8;
+	if (is_420)
+		chroma_blocks = luma_blocks / 4;
+	else
+		chroma_blocks = luma_blocks / 2;
+
+	blk_num = luma_blocks + 2 * chroma_blocks - 1;
+
+	writel(blk_num, base + JPEG_ENC_BLK_NUM);
+}
+
+void mtk_jpeg_enc_set_stride(void __iomem *base, u32 enc_format, u32 width,
+			     u32 height, u32 bytesperline)
+{
+	u32 img_stride;
+	u32 mem_stride;
+
+	if (enc_format == V4L2_PIX_FMT_NV12M ||
+	    enc_format == V4L2_PIX_FMT_NV21M) {
+		img_stride = round_up(width, 16);
+		mem_stride = bytesperline;
+	} else {
+		img_stride = round_up(width * 2, 32);
+		mem_stride = img_stride;
+	}
+
+	writel(img_stride, base + JPEG_ENC_IMG_STRIDE);
+	writel(mem_stride, base + JPEG_ENC_STRIDE);
+}
+
+void mtk_jpeg_enc_set_src_addr(void __iomem *base, u32 src_addr,
+			       u32 plane_index)
+{
+	if (!plane_index)
+		writel(src_addr, base + JPEG_ENC_SRC_LUMA_ADDR);
+	else
+		writel(src_addr, base + JPEG_ENC_SRC_CHROMA_ADDR);
+}
+
+void mtk_jpeg_enc_set_dst_addr(void __iomem *base, u32 dst_addr,
+			       u32 stall_size, u32 init_offset,
+			       u32 offset_mask)
+{
+	writel(init_offset & ~0xf, base + JPEG_ENC_OFFSET_ADDR);
+	writel(offset_mask & 0xf, base + JPEG_ENC_BYTE_OFFSET_MASK);
+	writel(dst_addr & ~0xf, base + JPEG_ENC_DST_ADDR0);
+	writel((dst_addr + stall_size) & ~0xf, base + JPEG_ENC_STALL_ADDR0);
+}
+
+static void mtk_jpeg_enc_set_quality(void __iomem *base, u32 quality)
+{
+	u32 value;
+	u32 i, enc_quality;
+
+	enc_quality = mtk_jpeg_enc_quality[0].hardware_value;
+	for (i = 0; i < ARRAY_SIZE(mtk_jpeg_enc_quality); i++) {
+		if (quality <= mtk_jpeg_enc_quality[i].quality_param) {
+			enc_quality = mtk_jpeg_enc_quality[i].hardware_value;
+			break;
+		}
+	}
+
+	value = readl(base + JPEG_ENC_QUALITY);
+	value = (value & JPEG_ENC_QUALITY_MASK) | enc_quality;
+	writel(value, base + JPEG_ENC_QUALITY);
+}
+
+static void mtk_jpeg_enc_set_ctrl(void __iomem *base, u32 enc_format,
+				  bool exif_en, u32 restart_interval)
+{
+	u32 value;
+
+	value = readl(base + JPEG_ENC_CTRL);
+	value &= ~JPEG_ENC_CTRL_YUV_FORMAT_MASK;
+	value |= (enc_format & 3) << 3;
+	if (exif_en)
+		value |= JPEG_ENC_CTRL_FILE_FORMAT_BIT;
+	else
+		value &= ~JPEG_ENC_CTRL_FILE_FORMAT_BIT;
+	if (restart_interval)
+		value |= JPEG_ENC_CTRL_RESTART_EN_BIT;
+	else
+		value &= ~JPEG_ENC_CTRL_RESTART_EN_BIT;
+	writel(value, base + JPEG_ENC_CTRL);
+}
+
+void mtk_jpeg_enc_set_config(void __iomem *base, u32 enc_format, bool exif_en,
+			     u32 quality, u32 restart_interval)
+{
+	mtk_jpeg_enc_set_quality(base, quality);
+
+	mtk_jpeg_enc_set_ctrl(base, enc_format, exif_en, restart_interval);
+
+	writel(restart_interval, base + JPEG_ENC_RST_MCU_NUM);
+}
+
+void mtk_jpeg_enc_start(void __iomem *base)
+{
+	u32 value;
+
+	value = readl(base + JPEG_ENC_CTRL);
+	value |= JPEG_ENC_CTRL_INT_EN_BIT | JPEG_ENC_CTRL_ENABLE_BIT;
+	writel(value, base + JPEG_ENC_CTRL);
+}
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h new file +113 -0
new file mode 100644
index 000000000000..73faf49b667c
@@ -0,0 +1,123 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ * Author: Xia Jiang <[email protected]>
+ *
+ */
+
+#ifndef _MTK_JPEG_ENC_HW_H
+#define _MTK_JPEG_ENC_HW_H
+
+#include <media/videobuf2-core.h>
+
+#include "mtk_jpeg_core.h"
+
+#define JPEG_ENC_INT_STATUS_DONE	BIT(0)
+#define JPEG_ENC_INT_STATUS_STALL	BIT(1)
+#define JPEG_ENC_INT_STATUS_VCODEC_IRQ	BIT(4)
+#define JPEG_ENC_INT_STATUS_MASK_ALLIRQ	0x13
+
+#define JPEG_ENC_DST_ADDR_OFFSET_MASK	GENMASK(3, 0)
+#define JPEG_ENC_QUALITY_MASK		GENMASK(31, 16)
+
+#define JPEG_ENC_CTRL_YUV_FORMAT_MASK	0x18
+#define JPEG_ENC_CTRL_RESTART_EN_BIT	BIT(10)
+#define JPEG_ENC_CTRL_FILE_FORMAT_BIT	BIT(5)
+#define JPEG_ENC_CTRL_INT_EN_BIT	BIT(2)
+#define JPEG_ENC_CTRL_ENABLE_BIT	BIT(0)
+#define JPEG_ENC_RESET_BIT		BIT(0)
+
+#define JPEG_ENC_YUV_FORMAT_YUYV	0
+#define JPEG_ENC_YUV_FORMAT_YVYU	1
+#define JPEG_ENC_YUV_FORMAT_NV12	2
+#define JEPG_ENC_YUV_FORMAT_NV21	3
+
+#define JPEG_ENC_QUALITY_Q60		0x0
+#define JPEG_ENC_QUALITY_Q80		0x1
+#define JPEG_ENC_QUALITY_Q90		0x2
+#define JPEG_ENC_QUALITY_Q95		0x3
+#define JPEG_ENC_QUALITY_Q39		0x4
+#define JPEG_ENC_QUALITY_Q68		0x5
+#define JPEG_ENC_QUALITY_Q84		0x6
+#define JPEG_ENC_QUALITY_Q92		0x7
+#define JPEG_ENC_QUALITY_Q48		0x8
+#define JPEG_ENC_QUALITY_Q74		0xa
+#define JPEG_ENC_QUALITY_Q87		0xb
+#define JPEG_ENC_QUALITY_Q34		0xc
+#define JPEG_ENC_QUALITY_Q64		0xe
+#define JPEG_ENC_QUALITY_Q82		0xf
+#define JPEG_ENC_QUALITY_Q97		0x10
+
+#define JPEG_ENC_RSTB			0x100
+#define JPEG_ENC_CTRL			0x104
+#define JPEG_ENC_QUALITY		0x108
+#define JPEG_ENC_BLK_NUM		0x10C
+#define JPEG_ENC_BLK_CNT		0x110
+#define JPEG_ENC_INT_STS		0x11c
+#define JPEG_ENC_DST_ADDR0		0x120
+#define JPEG_ENC_DMA_ADDR0		0x124
+#define JPEG_ENC_STALL_ADDR0		0x128
+#define JPEG_ENC_OFFSET_ADDR		0x138
+#define JPEG_ENC_RST_MCU_NUM		0x150
+#define JPEG_ENC_IMG_SIZE		0x154
+#define JPEG_ENC_DEBUG_INFO0		0x160
+#define JPEG_ENC_DEBUG_INFO1		0x164
+#define JPEG_ENC_TOTAL_CYCLE		0x168
+#define JPEG_ENC_BYTE_OFFSET_MASK	0x16c
+#define JPEG_ENC_SRC_LUMA_ADDR		0x170
+#define JPEG_ENC_SRC_CHROMA_ADDR	0x174
+#define JPEG_ENC_STRIDE			0x178
+#define JPEG_ENC_IMG_STRIDE		0x17c
+#define JPEG_ENC_DCM_CTRL		0x300
+#define JPEG_ENC_CODEC_SEL		0x314
+#define JPEG_ENC_ULTRA_THRES		0x318
+
+enum {
+	MTK_JPEG_ENC_RESULT_DONE,
+	MTK_JPEG_ENC_RESULT_STALL,
+	MTK_JPEG_ENC_RESULT_VCODEC_IRQ
+};
+
+/**
+ * struct mtk_jpeg_enc_qlt - JPEG encoder quality data
+ * @quality_param:	quality value
+ * @hardware_value:	hardware value of quality
+ */
+struct mtk_jpeg_enc_qlt {
+	u8	quality_param;
+	u8	hardware_value;
+};
+
+/**
+ * struct mt_jpeg_enc_bs - JPEG encoder bitstream  buffer
+ * @dma_addr:			JPEG encoder destination address
+ * @size:			JPEG encoder bistream size
+ * @dma_addr_offset:		JPEG encoder offset address
+ * @dma_addr_offsetmask:	JPEG encoder destination address offset mask
+ */
+struct mtk_jpeg_enc_bs {
+	dma_addr_t	dma_addr;
+	size_t		size;
+	u32		dma_addr_offset;
+	u32		dma_addr_offsetmask;
+};
+
+void mtk_jpeg_enc_reset(void __iomem *base);
+u32 mtk_jpeg_enc_get_and_clear_int_status(void __iomem *base);
+u32 mtk_jpeg_enc_get_file_size(void __iomem *base);
+u32 mtk_jpeg_enc_enum_result(void __iomem *base, u32 irq_status);
+void mtk_jpeg_enc_set_img_size(void __iomem *base, u32 width, u32 height);
+void mtk_jpeg_enc_set_blk_num(void __iomem *base, u32 enc_format, u32 width,
+			      u32 height);
+void mtk_jpeg_enc_set_stride(void __iomem *base, u32 enc_format, u32 width,
+			     u32 height, u32 bytesperline);

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h line 123

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h new file +10 -0
+void mtk_jpeg_enc_set_src_addr(void __iomem *base, u32 src_addr,
+			       u32 plane_index);
+void mtk_jpeg_enc_set_dst_addr(void __iomem *base, u32 dst_addr,
+			       u32 stall_size, u32 init_offset,
+			       u32 offset_mask);
+void mtk_jpeg_enc_set_config(void __iomem *base, u32 enc_format, bool exif_en,
+			     u32 quality, u32 restart_interval);
+void mtk_jpeg_enc_start(void __iomem *enc_reg_base);
+
+#endif /* _MTK_JPEG_ENC_HW_H */

Regards,

Quoted 1 line

Hans

Best regards,
Xia Jiang

_______________________________________________
Linux-mediatek mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-mediatek

#34 lore raw

Re: [PATCH RESEND v9 18/18] media: platform: Add jpeg enc feature

In reply to Xia Jiang

Hi Xia,

Quoted 8 lines · Xia Jiang

Add mtk jpeg encode v4l2 driver based on jpeg decode, because that jpeg decode and encode have great similarities with function operation.

---
v9: add member variable(struct v4l2_rect) in out_q structure for storing
the active crop information.
move the renaming exsting functions/defines/variables to a separate patch.

Quoted 10 lines
---
 drivers/media/platform/mtk-jpeg/Makefile      |   5 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 845 +++++++++++++++---
 .../media/platform/mtk-jpeg/mtk_jpeg_core.h   |  44 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c | 193 ++++
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h | 123 +++
 5 files changed, 1084 insertions(+), 126 deletions(-)
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h

Thank you for the patch. Please see my comments inline.

[snip]

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static inline struct mtk_jpeg_src_buf *mtk_jpeg_vb2_to_srcbuf( · line 145

Quoted 12 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +12 -0
+static int mtk_jpeg_enc_querycap(struct file *file, void *priv,
+				 struct v4l2_capability *cap)
+{
+	struct mtk_jpeg_dev *jpeg = video_drvdata(file);
+
+	strscpy(cap->driver, MTK_JPEG_NAME, sizeof(cap->driver));
+	strscpy(cap->card, MTK_JPEG_NAME " encoder", sizeof(cap->card));
+	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
+		 dev_name(jpeg->dev));
+
+	return 0;
+}

I can see that this function differs from mtk_jpeg_dec_querycap() only with the " encoder" string. Perhaps they could be merged?

Quoted 12 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -0
+
 static int mtk_jpeg_dec_querycap(struct file *file, void *priv,
 				 struct v4l2_capability *cap)
 {
@@ -88,6 +157,54 @@ static int mtk_jpeg_dec_querycap(struct file *file, void *priv,
 	return 0;
 }
 
+static int vidioc_jpeg_enc_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_jpeg_ctx *ctx = ctrl_to_ctx(ctrl);
+

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_querycap(struct file *file, void *priv, · line 173

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+	switch (ctrl->id) {
+	case V4L2_CID_JPEG_RESTART_INTERVAL:
+		ctx->restart_interval = ctrl->val;
+		break;
+	case V4L2_CID_JPEG_COMPRESSION_QUALITY:
+		ctx->enc_quality = ctrl->val;
+		break;
+	case V4L2_CID_JPEG_ACTIVE_MARKER:
+		ctx->enable_exif = ctrl->val & V4L2_JPEG_ACTIVE_MARKER_APP1 ?
+				   true : false;

nit: If ctx->enable_exif is of the bool type, the ternary operator could be removed, because any non-zero value is implicitly turned into 1, as per [1].

[1] https://www.kernel.org/doc/html/v5.6/process/coding-style.html#using-bool

[snip]

Quoted 12 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +8 -0
+static int mtk_jpeg_enc_enum_fmt_vid_cap(struct file *file, void *priv,
+					 struct v4l2_fmtdesc *f)
+{
+	return mtk_jpeg_enum_fmt(mtk_jpeg_enc_formats,
+				 MTK_JPEG_ENC_NUM_FORMATS, f,
+				 MTK_JPEG_FMT_FLAG_ENC_CAPTURE);
+}
+
 static int mtk_jpeg_dec_enum_fmt_vid_cap(struct file *file, void *priv,
 					 struct v4l2_fmtdesc *f)
 {

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_enum_fmt_vid_cap(struct file *file, void *priv, · line 250

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
@@ -117,6 +242,14 @@ static int mtk_jpeg_dec_enum_fmt_vid_cap(struct file *file, void *priv,
 				 MTK_JPEG_FMT_FLAG_DEC_CAPTURE);
 }
 
+static int mtk_jpeg_enc_enum_fmt_vid_out(struct file *file, void *priv,
+					 struct v4l2_fmtdesc *f)
+{
+	return mtk_jpeg_enum_fmt(mtk_jpeg_enc_formats,
+				 MTK_JPEG_ENC_NUM_FORMATS, f,
+				 MTK_JPEG_FMT_FLAG_ENC_OUTPUT);

Do we need separate implementations of these? "formats" and "num_formats" could be specified by the match data struct and used in a generic function.

Also, do we need separate flags for ENC_OUTPUT/CAPTURE and DEC_OUTPUT/CAPTURE?

Quoted 26 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -6
+}
+
 static int mtk_jpeg_dec_enum_fmt_vid_out(struct file *file, void *priv,
 					 struct v4l2_fmtdesc *f)
 {
@@ -132,93 +265,66 @@ mtk_jpeg_get_q_data(struct mtk_jpeg_ctx *ctx, enum v4l2_buf_type type)
 	return &ctx->cap_q;
 }
 
-static struct mtk_jpeg_fmt *mtk_jpeg_find_format(struct mtk_jpeg_ctx *ctx,
-						 u32 pixelformat,
+static struct mtk_jpeg_fmt *mtk_jpeg_find_format(u32 pixelformat,
 						 unsigned int fmt_type)
 {
-	unsigned int k, fmt_flag;
+	unsigned int k;
+	struct mtk_jpeg_fmt *fmt;
 
-	fmt_flag = (fmt_type == MTK_JPEG_FMT_TYPE_OUTPUT) ?
-		   MTK_JPEG_FMT_FLAG_DEC_OUTPUT :
-		   MTK_JPEG_FMT_FLAG_DEC_CAPTURE;
+	for (k = 0; k < MTK_JPEG_ENC_NUM_FORMATS; k++) {
+		fmt = &mtk_jpeg_enc_formats[k];
+
+		if (fmt->fourcc == pixelformat && fmt->flags & fmt_type)
+			return fmt;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c mtk_jpeg_get_q_data(struct mtk_jpeg_ctx *ctx, enum v4l2_buf_type type) · line 286

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -2
+	}
 
 	for (k = 0; k < MTK_JPEG_DEC_NUM_FORMATS; k++) {
-		struct mtk_jpeg_fmt *fmt = &mtk_jpeg_dec_formats[k];
+		fmt = &mtk_jpeg_dec_formats[k];
 
-		if (fmt->fourcc == pixelformat && fmt->flags & fmt_flag)
+		if (fmt->fourcc == pixelformat && fmt->flags & fmt_type)
 			return fmt;
 	}

We should only need to iterate through one of the arrays. My suggestion above about making "formats" and "num_formats" a member of the match data should take care of this one too.

Quoted 18 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +0 -14
 	return NULL;
 }
 
-static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx,
-				       struct v4l2_format *f)
-{
-	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
-	struct mtk_jpeg_q_data *q_data;
-	int i;
-
-	q_data = mtk_jpeg_get_q_data(ctx, f->type);
-
-	pix_mp->width = q_data->w;
-	pix_mp->height = q_data->h;
-	pix_mp->pixelformat = q_data->fmt->fourcc;
-	pix_mp->num_planes = q_data->fmt->colplanes;
-

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c mtk_jpeg_get_q_data(struct mtk_jpeg_ctx *ctx, enum v4l2_buf_type type) · line 291

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -9
-	for (i = 0; i < pix_mp->num_planes; i++) {
-		pix_mp->plane_fmt[i].bytesperline = q_data->bytesperline[i];
-		pix_mp->plane_fmt[i].sizeimage = q_data->sizeimage[i];
-	}
-}
-
-static int mtk_jpeg_try_fmt_mplane(struct v4l2_format *f,
-				   struct mtk_jpeg_fmt *fmt,
-				   struct mtk_jpeg_ctx *ctx, int q_type)
+static int vidioc_try_fmt(struct v4l2_format *f, struct mtk_jpeg_fmt *fmt)

The name is a bit misleading, because it's suggesting that it's the vidioc_try_fmt callback, but it's just an internal helper. I think the original name was fine.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c mtk_jpeg_get_q_data(struct mtk_jpeg_ctx *ctx, enum v4l2_buf_type type) · line 187

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +0 -5
 {
 	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
 	int i;
 
 	pix_mp->field = V4L2_FIELD_NONE;
-
-	if (ctx->state != MTK_JPEG_INIT) {
-		mtk_jpeg_adjust_fmt_mplane(ctx, f);
-		return 0;
-	}

Is it okay to remove this? My understanding is that it prevented changing the format while streaming, which should is as expected.

Quoted 47 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +22 -16
-
 	pix_mp->num_planes = fmt->colplanes;
 	pix_mp->pixelformat = fmt->fourcc;
 
-	if (q_type == MTK_JPEG_FMT_TYPE_OUTPUT) {
-		struct v4l2_plane_pix_format *pfmt = &pix_mp->plane_fmt[0];
-
+	if (fmt->fourcc == V4L2_PIX_FMT_JPEG) {
 		pix_mp->height = clamp(pix_mp->height, MTK_JPEG_MIN_HEIGHT,
 				       MTK_JPEG_MAX_HEIGHT);
 		pix_mp->width = clamp(pix_mp->width, MTK_JPEG_MIN_WIDTH,
 				      MTK_JPEG_MAX_WIDTH);
-
-		pfmt->bytesperline = 0;
-		/* Source size must be aligned to 128 */
-		pfmt->sizeimage = round_up(pfmt->sizeimage, 128);
-		if (pfmt->sizeimage == 0)
-			pfmt->sizeimage = MTK_JPEG_DEFAULT_SIZEIMAGE;
-		return 0;
+		pix_mp->plane_fmt[0].bytesperline = 0;
+		pix_mp->plane_fmt[0].sizeimage =
+				round_up(pix_mp->plane_fmt[0].sizeimage, 128);
+		if (pix_mp->plane_fmt[0].sizeimage == 0)
+			pix_mp->plane_fmt[0].sizeimage =
+				MTK_JPEG_DEFAULT_SIZEIMAGE;
+	} else {
+		pix_mp->height = clamp(round_up(pix_mp->height, fmt->v_align),
+				       MTK_JPEG_MIN_HEIGHT,
+				       MTK_JPEG_MAX_HEIGHT);
+		pix_mp->width = clamp(round_up(pix_mp->width, fmt->h_align),
+				      MTK_JPEG_MIN_WIDTH, MTK_JPEG_MAX_WIDTH);
+		for (i = 0; i < pix_mp->num_planes; i++) {
+			struct v4l2_plane_pix_format *pfmt =
+							&pix_mp->plane_fmt[i];
+			u32 stride = pix_mp->width * fmt->h_sample[i] / 4;
+			u32 h = pix_mp->height * fmt->v_sample[i] / 4;
+
+			pfmt->bytesperline = stride;
+			pfmt->sizeimage = stride * h;
+		}
 	}
 
-	/* type is MTK_JPEG_FMT_TYPE_CAPTURE */
-	pix_mp->height = clamp(round_up(pix_mp->height, fmt->v_align),
-			       MTK_JPEG_MIN_HEIGHT, MTK_JPEG_MAX_HEIGHT);
-	pix_mp->width = clamp(round_up(pix_mp->width, fmt->h_align),
-			      MTK_JPEG_MIN_WIDTH, MTK_JPEG_MAX_WIDTH);

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c mtk_jpeg_get_q_data(struct mtk_jpeg_ctx *ctx, enum v4l2_buf_type type) · line 328

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +0 -9
-
-	for (i = 0; i < fmt->colplanes; i++) {
-		struct v4l2_plane_pix_format *pfmt = &pix_mp->plane_fmt[i];
-		u32 stride = pix_mp->width * fmt->h_sample[i] / 4;
-		u32 h = pix_mp->height * fmt->v_sample[i] / 4;
-
-		pfmt->bytesperline = stride;
-		pfmt->sizeimage = stride * h;
-	}
 	return 0;

Are all the changes above needed for the encoder? If I'm looking correctly, they're mostly refactoring and should be done in a separate patch. However, I don't see any big issue with the existing coding style in this function, so perhaps the refactoring could be avoided.

Quoted 16 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
 }
 
@@ -271,14 +377,35 @@ static int mtk_jpeg_g_fmt_vid_mplane(struct file *file, void *priv,
 	return 0;
 }
 
+static int mtk_jpeg_enc_try_fmt_vid_cap_mplane(struct file *file, void *priv,
+					       struct v4l2_format *f)
+{
+	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+	struct mtk_jpeg_fmt *fmt;
+
+	fmt = mtk_jpeg_find_format(f->fmt.pix_mp.pixelformat,
+				   MTK_JPEG_FMT_FLAG_ENC_CAPTURE);
+	if (!fmt)
+		fmt = ctx->cap_q.fmt;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_g_fmt_vid_mplane(struct file *file, void *priv, · line 399

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+
+	v4l2_dbg(2, debug, &ctx->jpeg->v4l2_dev, "(%d) try_fmt:%c%c%c%c\n",
+		 f->type,
+		 (fmt->fourcc & 0xff),
+		 (fmt->fourcc >>  8 & 0xff),
+		 (fmt->fourcc >> 16 & 0xff),
+		 (fmt->fourcc >> 24 & 0xff));
+
+	return vidioc_try_fmt(f, fmt);
+}

Is there really anything encoder-specific in this function? Could the same function as the decoder be used instead?

Quoted 15 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -2
+
 static int mtk_jpeg_dec_try_fmt_vid_cap_mplane(struct file *file, void *priv,
 					       struct v4l2_format *f)
 {
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
 	struct mtk_jpeg_fmt *fmt;
 
-	fmt = mtk_jpeg_find_format(ctx, f->fmt.pix_mp.pixelformat,
-				   MTK_JPEG_FMT_TYPE_CAPTURE);
+	fmt = mtk_jpeg_find_format(f->fmt.pix_mp.pixelformat,
+				   MTK_JPEG_FMT_FLAG_DEC_CAPTURE);
 	if (!fmt)
 		fmt = ctx->cap_q.fmt;
 

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_try_fmt_vid_cap_mplane(struct file *file, void *priv, · line 422

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -1
@@ -289,7 +416,33 @@ static int mtk_jpeg_dec_try_fmt_vid_cap_mplane(struct file *file, void *priv,
 		 (fmt->fourcc >> 16 & 0xff),
 		 (fmt->fourcc >> 24 & 0xff));
 
-	return mtk_jpeg_try_fmt_mplane(f, fmt, ctx, MTK_JPEG_FMT_TYPE_CAPTURE);
+	if (ctx->state != MTK_JPEG_INIT) {
+		mtk_jpeg_g_fmt_vid_mplane(file, priv, f);
+		return 0;
+	}

Is this really limited to the decoder? In general currently we don't support changing the resolution from the userspace when streaming is started and it applies to both encoder and decoder.

Quoted 14 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +14 -0
+
+	return vidioc_try_fmt(f, fmt);
+}
+
+static int mtk_jpeg_enc_try_fmt_vid_out_mplane(struct file *file, void *priv,
+					       struct v4l2_format *f)
+{
+	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+	struct mtk_jpeg_fmt *fmt;
+
+	fmt = mtk_jpeg_find_format(f->fmt.pix_mp.pixelformat,
+				   MTK_JPEG_FMT_FLAG_ENC_OUTPUT);
+	if (!fmt)
+		fmt = ctx->out_q.fmt;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_try_fmt_vid_cap_mplane(struct file *file, void *priv, · line 446

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +9 -0
+
+	v4l2_dbg(2, debug, &ctx->jpeg->v4l2_dev, "(%d) try_fmt:%c%c%c%c\n",
+		 f->type,
+		 (fmt->fourcc & 0xff),
+		 (fmt->fourcc >>  8 & 0xff),
+		 (fmt->fourcc >> 16 & 0xff),
+		 (fmt->fourcc >> 24 & 0xff));
+
+	return vidioc_try_fmt(f, fmt);
 }

Ditto.

Quoted 14 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -2
 static int mtk_jpeg_dec_try_fmt_vid_out_mplane(struct file *file, void *priv,
@@ -298,8 +451,8 @@ static int mtk_jpeg_dec_try_fmt_vid_out_mplane(struct file *file, void *priv,
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
 	struct mtk_jpeg_fmt *fmt;
 
-	fmt = mtk_jpeg_find_format(ctx, f->fmt.pix_mp.pixelformat,
-				   MTK_JPEG_FMT_TYPE_OUTPUT);
+	fmt = mtk_jpeg_find_format(f->fmt.pix_mp.pixelformat,
+				   MTK_JPEG_FMT_FLAG_DEC_OUTPUT);
 	if (!fmt)
 		fmt = ctx->out_q.fmt;
 

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_try_fmt_vid_out_mplane(struct file *file, void *priv, · line 469

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -1
@@ -310,17 +463,21 @@ static int mtk_jpeg_dec_try_fmt_vid_out_mplane(struct file *file, void *priv,
 		 (fmt->fourcc >> 16 & 0xff),
 		 (fmt->fourcc >> 24 & 0xff));
 
-	return mtk_jpeg_try_fmt_mplane(f, fmt, ctx, MTK_JPEG_FMT_TYPE_OUTPUT);
+	if (ctx->state != MTK_JPEG_INIT) {
+		mtk_jpeg_g_fmt_vid_mplane(file, priv, f);
+		return 0;
+	}

Ditto.

Quoted 76 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +34 -7
+
+	return vidioc_try_fmt(f, fmt);
 }
 
 static int mtk_jpeg_s_fmt_mplane(struct mtk_jpeg_ctx *ctx,
-				 struct v4l2_format *f)
+				 struct v4l2_format *f, unsigned int fmt_type)
 {
 	struct vb2_queue *vq;
 	struct mtk_jpeg_q_data *q_data = NULL;
 	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
 	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
-	unsigned int f_type;
 	int i;
 
 	vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, f->type);
@@ -334,12 +491,11 @@ static int mtk_jpeg_s_fmt_mplane(struct mtk_jpeg_ctx *ctx,
 		return -EBUSY;
 	}
 
-	f_type = V4L2_TYPE_IS_OUTPUT(f->type) ?
-			 MTK_JPEG_FMT_TYPE_OUTPUT : MTK_JPEG_FMT_TYPE_CAPTURE;
-
-	q_data->fmt = mtk_jpeg_find_format(ctx, pix_mp->pixelformat, f_type);
+	q_data->fmt = mtk_jpeg_find_format(pix_mp->pixelformat, fmt_type);
 	q_data->w = pix_mp->width;
 	q_data->h = pix_mp->height;
+	q_data->crop_rect.width = pix_mp->width;
+	q_data->crop_rect.height = pix_mp->height;
 	ctx->colorspace = pix_mp->colorspace;
 	ctx->ycbcr_enc = pix_mp->ycbcr_enc;
 	ctx->xfer_func = pix_mp->xfer_func;
@@ -365,6 +521,19 @@ static int mtk_jpeg_s_fmt_mplane(struct mtk_jpeg_ctx *ctx,
 	return 0;
 }
 
+static int mtk_jpeg_enc_s_fmt_vid_out_mplane(struct file *file, void *priv,
+					     struct v4l2_format *f)
+{
+	int ret;
+
+	ret = mtk_jpeg_enc_try_fmt_vid_out_mplane(file, priv, f);
+	if (ret)
+		return ret;
+
+	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f,
+				     MTK_JPEG_FMT_FLAG_ENC_OUTPUT);
+}
+
 static int mtk_jpeg_dec_s_fmt_vid_out_mplane(struct file *file, void *priv,
 					     struct v4l2_format *f)
 {
@@ -374,7 +543,21 @@ static int mtk_jpeg_dec_s_fmt_vid_out_mplane(struct file *file, void *priv,
 	if (ret)
 		return ret;
 
-	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f);
+	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f,
+				     MTK_JPEG_FMT_FLAG_DEC_OUTPUT);
+}
+
+static int mtk_jpeg_enc_s_fmt_vid_cap_mplane(struct file *file, void *priv,
+					     struct v4l2_format *f)
+{
+	int ret;
+
+	ret = mtk_jpeg_enc_try_fmt_vid_cap_mplane(file, priv, f);
+	if (ret)
+		return ret;
+
+	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f,
+				     MTK_JPEG_FMT_FLAG_ENC_CAPTURE);
 }
 
 static int mtk_jpeg_dec_s_fmt_vid_cap_mplane(struct file *file, void *priv,

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_s_fmt_vid_cap_mplane(struct file *file, void *priv, · line 574

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -1
@@ -386,7 +569,8 @@ static int mtk_jpeg_dec_s_fmt_vid_cap_mplane(struct file *file, void *priv,
 	if (ret)
 		return ret;
 
-	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f);
+	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f,
+				     MTK_JPEG_FMT_FLAG_DEC_CAPTURE);
 }

Is it really necessary to have separate variants of the above functions for decoder and encoder?

Quoted 12 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
 static void mtk_jpeg_queue_src_chg_event(struct mtk_jpeg_ctx *ctx)
@@ -411,6 +595,29 @@ static int mtk_jpeg_subscribe_event(struct v4l2_fh *fh,
 	return v4l2_ctrl_subscribe_event(fh, sub);
 }
 
+static int mtk_jpeg_enc_g_selection(struct file *file, void *priv,
+				    struct v4l2_selection *s)
+{
+	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+
+	if (s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_subscribe_event(struct v4l2_fh *fh, · line 613

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+		return -EINVAL;
+
+	switch (s->target) {
+	case V4L2_SEL_TGT_CROP:
+	case V4L2_SEL_TGT_CROP_BOUNDS:
+	case V4L2_SEL_TGT_CROP_DEFAULT:
+		s->r.width = ctx->out_q.w;
+		s->r.height = ctx->out_q.h;
+		s->r.left = 0;
+		s->r.top = 0;

Is this really correct? The function seems to be returning the full frame size regardless of the selection target. For BOUNDS and DEFAULTS this would be the correct behavior indeed, but CROP should return the active crop rectangle, as set by S_SELECTION.

Quoted 45 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +31 -0
+		break;
+	default:
+		return -EINVAL;
+	}
+	return 0;
+}
+
 static int mtk_jpeg_dec_g_selection(struct file *file, void *priv,
 				    struct v4l2_selection *s)
 {
@@ -440,6 +647,29 @@ static int mtk_jpeg_dec_g_selection(struct file *file, void *priv,
 	return 0;
 }
 
+static int mtk_jpeg_enc_s_selection(struct file *file, void *priv,
+				    struct v4l2_selection *s)
+{
+	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+
+	if (s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)
+		return -EINVAL;
+
+	switch (s->target) {
+	case V4L2_SEL_TGT_CROP:
+		s->r.left = 0;
+		s->r.top = 0;
+		s->r.width = min(s->r.width, ctx->out_q.w);
+		s->r.height = min(s->r.height, ctx->out_q.h);
+		ctx->out_q.crop_rect = s->r;
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
 static int mtk_jpeg_dec_s_selection(struct file *file, void *priv,
 				    struct v4l2_selection *s)
 {
@@ -484,6 +714,33 @@ static int mtk_jpeg_qbuf(struct file *file, void *priv, struct v4l2_buffer *buf)
 	return v4l2_m2m_qbuf(file, fh->m2m_ctx, buf);
 }
 
+static const struct v4l2_ioctl_ops mtk_jpeg_enc_ioctl_ops = {

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_qbuf(struct file *file, void *priv, struct v4l2_buffer *buf) · line 727

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+	.vidioc_querycap                = mtk_jpeg_enc_querycap,
+	.vidioc_enum_fmt_vid_cap	= mtk_jpeg_enc_enum_fmt_vid_cap,
+	.vidioc_enum_fmt_vid_out	= mtk_jpeg_enc_enum_fmt_vid_out,
+	.vidioc_try_fmt_vid_cap_mplane	= mtk_jpeg_enc_try_fmt_vid_cap_mplane,
+	.vidioc_try_fmt_vid_out_mplane	= mtk_jpeg_enc_try_fmt_vid_out_mplane,
+	.vidioc_g_fmt_vid_cap_mplane    = mtk_jpeg_g_fmt_vid_mplane,
+	.vidioc_g_fmt_vid_out_mplane    = mtk_jpeg_g_fmt_vid_mplane,
+	.vidioc_s_fmt_vid_cap_mplane    = mtk_jpeg_enc_s_fmt_vid_cap_mplane,
+	.vidioc_s_fmt_vid_out_mplane    = mtk_jpeg_enc_s_fmt_vid_out_mplane,
+	.vidioc_qbuf                    = mtk_jpeg_qbuf,

Not directly a comment for this patch, but since the previous patch removed the LAST_FRAME handling, wouldn't it be enough to just use v4l2_m2m_qbuf() as this callback?

[snip]

Quoted 48 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +29 -0
+static void mtk_jpeg_enc_buf_queue(struct vb2_buffer *vb)
+{
+	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+
+	v4l2_dbg(2, debug, &jpeg->v4l2_dev, "(%d) buf_q id=%d, vb=%p\n",
+		 vb->vb2_queue->type, vb->index, vb);
+
+	v4l2_m2m_buf_queue(ctx->fh.m2m_ctx, to_vb2_v4l2_buffer(vb));
+}
+
 static void mtk_jpeg_dec_buf_queue(struct vb2_buffer *vb)
 {
 	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
@@ -664,6 +932,15 @@ static struct vb2_v4l2_buffer *mtk_jpeg_buf_remove(struct mtk_jpeg_ctx *ctx,
 		return v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
 }
 
+static void mtk_jpeg_enc_stop_streaming(struct vb2_queue *q)
+{
+	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(q);
+	struct vb2_v4l2_buffer *vb;
+
+	while ((vb = mtk_jpeg_buf_remove(ctx, q->type)))
+		v4l2_m2m_buf_done(vb, VB2_BUF_STATE_ERROR);
+}
+
 static void mtk_jpeg_dec_stop_streaming(struct vb2_queue *q)
 {
 	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(q);
@@ -699,6 +976,15 @@ static const struct vb2_ops mtk_jpeg_dec_qops = {
 	.stop_streaming     = mtk_jpeg_dec_stop_streaming,
 };
 
+static const struct vb2_ops mtk_jpeg_enc_qops = {
+	.queue_setup        = mtk_jpeg_queue_setup,
+	.buf_prepare        = mtk_jpeg_buf_prepare,
+	.buf_queue          = mtk_jpeg_enc_buf_queue,
+	.wait_prepare       = vb2_ops_wait_prepare,
+	.wait_finish        = vb2_ops_wait_finish,
+	.stop_streaming     = mtk_jpeg_enc_stop_streaming,
+};
+
 static void mtk_jpeg_set_dec_src(struct mtk_jpeg_ctx *ctx,
 				 struct vb2_buffer *src_buf,
 				 struct mtk_jpeg_bs *bs)
@@ -736,6 +1022,85 @@ static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx,
 	return 0;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx, · line 1032

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +8 -0
 }
 
+static void mtk_jpeg_set_enc_dst(struct mtk_jpeg_ctx *ctx, void __iomem *base,
+				 struct vb2_buffer *dst_buf,
+				 struct mtk_jpeg_enc_bs *bs)
+{
+	bs->dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
+	bs->dma_addr_offset = ctx->enable_exif ? MTK_JPEG_MAX_EXIF_SIZE : 0;
+	bs->dma_addr_offsetmask = bs->dma_addr & JPEG_ENC_DST_ADDR_OFFSET_MASK;
+	bs->size = vb2_plane_size(dst_buf, 0);

We're computing these values and then writing to the hardware straightaway. Do we need to save these values to the bs struct? If no, do we need the bs struct at all?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx, · line 1043

Quoted 11 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +11 -0
+
+	mtk_jpeg_enc_set_dst_addr(base, bs->dma_addr, bs->size,
+				  bs->dma_addr_offset,
+				  bs->dma_addr_offsetmask);
+}
+
+static void mtk_jpeg_set_enc_src(struct mtk_jpeg_ctx *ctx, void __iomem *base,
+				 struct vb2_buffer *src_buf)
+{
+	int i;
+	dma_addr_t	dma_addr;

nit: Only one space should be between variable type and name.

Quoted 39 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +39 -0
+
+	mtk_jpeg_enc_set_img_size(base, ctx->out_q.crop_rect.width,
+				  ctx->out_q.crop_rect.height);
+	mtk_jpeg_enc_set_blk_num(base, ctx->out_q.fmt->fourcc,
+				 ctx->out_q.crop_rect.width,
+				 ctx->out_q.crop_rect.height);
+	mtk_jpeg_enc_set_stride(base, ctx->out_q.fmt->fourcc, ctx->out_q.w,
+				ctx->out_q.h, ctx->out_q.bytesperline[0]);
+
+	for (i = 0; i < src_buf->num_planes; i++) {
+		dma_addr = vb2_dma_contig_plane_dma_addr(src_buf, i) +
+			   src_buf->planes[i].data_offset;
+		mtk_jpeg_enc_set_src_addr(base, dma_addr, i);
+	}
+}
+
+static void mtk_jpeg_enc_device_run(void *priv)
+{
+	struct mtk_jpeg_ctx *ctx = priv;
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+	struct vb2_v4l2_buffer *src_buf, *dst_buf;
+	enum vb2_buffer_state buf_state = VB2_BUF_STATE_ERROR;
+	unsigned long flags;
+	struct mtk_jpeg_src_buf *jpeg_src_buf;
+	struct mtk_jpeg_enc_bs enc_bs;
+	int ret;
+
+	src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
+	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
+	jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(&src_buf->vb2_buf);
+
+	ret = pm_runtime_get_sync(jpeg->dev);
+	if (ret < 0)
+		goto enc_end;
+
+	spin_lock_irqsave(&jpeg->hw_lock, flags);
+
+	/*
+	 * Resetting the hardware every frame is to ensure that all the

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx, · line 1092

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+	 * registers are cleared. This is a hardware requirement.
+	 */
+	mtk_jpeg_enc_reset(jpeg->reg_base);
+
+	mtk_jpeg_set_enc_dst(ctx, jpeg->reg_base, &dst_buf->vb2_buf, &enc_bs);
+	mtk_jpeg_set_enc_src(ctx, jpeg->reg_base, &src_buf->vb2_buf);
+	mtk_jpeg_enc_set_config(jpeg->reg_base, ctx->out_q.fmt->hw_format,
+				ctx->enable_exif, ctx->enc_quality,
+				ctx->restart_interval);
+	mtk_jpeg_enc_start(jpeg->reg_base);

Could we just move the above 5 functions into one function inside mtk_jpeg_enc_hw.c that takes mtk_jpeg_dev pointer as its argument, let's say mtk_jpeg_enc_hw_run() and simply program all the data to the registers directly, without the extra level of abstractions?

Quoted 15 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +11 -0
+	spin_unlock_irqrestore(&jpeg->hw_lock, flags);
+	return;
+
+enc_end:
+	v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+	v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+	v4l2_m2m_buf_done(src_buf, buf_state);
+	v4l2_m2m_buf_done(dst_buf, buf_state);
+	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
+}
+
 static void mtk_jpeg_dec_device_run(void *priv)
 {
 	struct mtk_jpeg_ctx *ctx = priv;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_dec_device_run(void *priv) · line 1157

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -0
@@ -785,6 +1150,11 @@ static void mtk_jpeg_dec_device_run(void *priv)
 	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
 }
 
+static int mtk_jpeg_enc_job_ready(void *priv)
+{
+		return 1;
+}
+

The callback is optional, so can be just removed if it always returns 1.

Quoted 42 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +24 -1
 static int mtk_jpeg_dec_job_ready(void *priv)
 {
 	struct mtk_jpeg_ctx *ctx = priv;
@@ -792,6 +1162,11 @@ static int mtk_jpeg_dec_job_ready(void *priv)
 	return (ctx->state == MTK_JPEG_RUNNING) ? 1 : 0;
 }
 
+static const struct v4l2_m2m_ops mtk_jpeg_enc_m2m_ops = {
+	.device_run = mtk_jpeg_enc_device_run,
+	.job_ready  = mtk_jpeg_enc_job_ready,
+};
+
 static const struct v4l2_m2m_ops mtk_jpeg_dec_m2m_ops = {
 	.device_run = mtk_jpeg_dec_device_run,
 	.job_ready  = mtk_jpeg_dec_job_ready,
@@ -830,24 +1205,109 @@ static int mtk_jpeg_dec_queue_init(void *priv, struct vb2_queue *src_vq,
 	return ret;
 }
 
-static void mtk_jpeg_clk_on(struct mtk_jpeg_dev *jpeg)
+static int mtk_jpeg_enc_queue_init(void *priv, struct vb2_queue *src_vq,
+				   struct vb2_queue *dst_vq)
 {
+	struct mtk_jpeg_ctx *ctx = priv;
 	int ret;
 
+	src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+	src_vq->io_modes = VB2_DMABUF | VB2_MMAP;
+	src_vq->drv_priv = ctx;
+	src_vq->buf_struct_size = sizeof(struct mtk_jpeg_src_buf);
+	src_vq->ops = &mtk_jpeg_enc_qops;
+	src_vq->mem_ops = &vb2_dma_contig_memops;
+	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	src_vq->lock = &ctx->jpeg->lock;
+	src_vq->dev = ctx->jpeg->dev;
+	ret = vb2_queue_init(src_vq);
+	if (ret)
+		return ret;
+
+	dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+	dst_vq->io_modes = VB2_DMABUF | VB2_MMAP;
+	dst_vq->drv_priv = ctx;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_queue_init(void *priv, struct vb2_queue *src_vq, · line 1239

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+	dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
+	dst_vq->ops = &mtk_jpeg_enc_qops;
+	dst_vq->mem_ops = &vb2_dma_contig_memops;
+	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	dst_vq->lock = &ctx->jpeg->lock;
+	dst_vq->dev = ctx->jpeg->dev;
+	ret = vb2_queue_init(dst_vq);
+
+	return ret;
+}

This only differs in "ops" from the decoder implementation. Perhaps both functions could be merged?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_queue_init(void *priv, struct vb2_queue *src_vq, · line 1250

Quoted 13 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +8 -2
+
+static void mtk_jpeg_clk_on(struct mtk_jpeg_dev *jpeg)
+{
+	int ret, i;
+
 	ret = mtk_smi_larb_get(jpeg->larb);
 	if (ret)
 		dev_err(jpeg->dev, "mtk_smi_larb_get larbvdec fail %d\n", ret);
-	clk_prepare_enable(jpeg->clk_jdec_smi);
-	clk_prepare_enable(jpeg->clk_jdec);
+
+	for (i = 0; i < jpeg->variant->num_clocks; i++) {
+		ret = clk_prepare_enable(jpeg->clocks[i]);

Instead of an open coded loop, could the clk_bulk_*() helpers be used instead? (Look for devm_clk_bulk_get() and devm_clk_bulk_prepare_enable().)

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_queue_init(void *priv, struct vb2_queue *src_vq, · line 1253

Quoted 3 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -0
+		if (ret) {
+			while (--i >= 0)
+				clk_disable_unprepare(jpeg->clocks[i]);

nit: The typical convention is to do error handling in an error path on the bottom of the function.

Also, it would be nice to print an error message.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_queue_init(void *priv, struct vb2_queue *src_vq, · line 1263

Quoted 12 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -2
+		}
+	}
 }
 
 static void mtk_jpeg_clk_off(struct mtk_jpeg_dev *jpeg)
 {
-	clk_disable_unprepare(jpeg->clk_jdec);
-	clk_disable_unprepare(jpeg->clk_jdec_smi);
+	int i;
+
+	for (i = jpeg->variant->num_clocks - 1; i >= 0; i--)
+		clk_disable_unprepare(jpeg->clocks[i]);

Ditto.

Quoted 14 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +11 -0
 	mtk_smi_larb_put(jpeg->larb);
 }
 
+static irqreturn_t mtk_jpeg_enc_irq(int irq, void *priv)
+{
+	struct mtk_jpeg_dev *jpeg = priv;
+	struct mtk_jpeg_ctx *ctx;
+	struct vb2_v4l2_buffer *src_buf, *dst_buf;
+	struct mtk_jpeg_src_buf *jpeg_src_buf;
+	enum vb2_buffer_state buf_state = VB2_BUF_STATE_ERROR;
+	u32 enc_irq_ret;
+	u32 enc_ret, result_size;
+
+	ctx = v4l2_m2m_get_curr_priv(jpeg->m2m_dev);

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_queue_init(void *priv, struct vb2_queue *src_vq, · line 1287

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+	if (!ctx) {
+		v4l2_err(&jpeg->v4l2_dev, "Context is NULL\n");
+		return IRQ_HANDLED;
+	}
+
+	src_buf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+	dst_buf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+	jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(&src_buf->vb2_buf);
+
+	enc_ret = mtk_jpeg_enc_get_and_clear_int_status(jpeg->reg_base);

We should check the interrupt status at the beginning of the function and return IRQ_NONE if there wasn't any flag set.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_queue_init(void *priv, struct vb2_queue *src_vq, · line 1296

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +9 -0
+	enc_irq_ret = mtk_jpeg_enc_enum_result(jpeg->reg_base, enc_ret);
+
+	if (enc_irq_ret >= MTK_JPEG_ENC_RESULT_STALL)
+		mtk_jpeg_enc_reset(jpeg->reg_base);
+
+	if (enc_irq_ret != MTK_JPEG_ENC_RESULT_DONE) {
+		dev_err(jpeg->dev, "encode failed\n");
+		goto enc_end;
+	}

As I suggested before, it would have been much more clear if the interrupt status bits were just directly read and checked in this function, without introducing the additional abstraction.

Quoted 18 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +14 -0
+
+	result_size = mtk_jpeg_enc_get_file_size(jpeg->reg_base);
+	vb2_set_plane_payload(&dst_buf->vb2_buf, 0, result_size);
+
+	buf_state = VB2_BUF_STATE_DONE;
+
+enc_end:
+	v4l2_m2m_buf_done(src_buf, buf_state);
+	v4l2_m2m_buf_done(dst_buf, buf_state);
+	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
+	pm_runtime_put(ctx->jpeg->dev);
+	return IRQ_HANDLED;
+}
+
 static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv)
 {
 	struct mtk_jpeg_dev *jpeg = priv;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1361

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
@@ -893,36 +1353,130 @@ static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv)
 	return IRQ_HANDLED;
 }
 
+static void mtk_jpeg_set_enc_default_params(struct mtk_jpeg_ctx *ctx)
+{
+	struct mtk_jpeg_q_data *q = &ctx->out_q;
+	struct v4l2_pix_format_mplane *pix_mp;
+
+	pix_mp = kmalloc(sizeof(*pix_mp), GFP_KERNEL);

Do we need to allocate this pix_mp? Could we instead just embed it inside ctx?

Also, this is actually a memory leak, because I don't see this structure saved anywhere or freed.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1367

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
+
+	ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+	ctx->colorspace = V4L2_COLORSPACE_JPEG,
+	ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
+	ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;

Since we already have a v4l2_pix_format_mplane struct which has fields for the above 4 values, could we just store them there?

Also, I don't see this driver handling the colorspaces in any way, but it seems to allow changing them from the userspace. This is incorrect, because the userspace has no way to know that the colorspace is not handled. Instead, the try_fmt implementation should always override the userspace-provided colorspace configuration with the ones that the driver assumes.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1380

Quoted 13 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +13 -0
+	pix_mp->width = MTK_JPEG_MIN_WIDTH;
+	pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+
+	q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_YUYV,
+				      MTK_JPEG_FMT_FLAG_ENC_OUTPUT);
+	vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+				    fmt.pix_mp), q->fmt);
+	q->w = pix_mp->width;
+	q->h = pix_mp->height;
+	q->crop_rect.width = pix_mp->width;
+	q->crop_rect.height = pix_mp->height;
+	q->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
+	q->bytesperline[0] = pix_mp->plane_fmt[0].bytesperline;

Actually, do we need this custom mtk_jpeg_q_data struct? Why couldn't we just keep the same values inside the standard v4l2_pix_format_mplane struct?

In general it's preferred to use the standard kernel structures as much as possible and only introduce local driver structures for data that can't be stored in generic ones.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1392

Quoted 12 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +12 -0
+
+	q = &ctx->cap_q;
+	q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_JPEG,
+				      MTK_JPEG_FMT_FLAG_ENC_CAPTURE);
+	pix_mp->width = MTK_JPEG_MIN_WIDTH;
+	pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+	vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+				    fmt.pix_mp), q->fmt);
+	q->w = pix_mp->width;
+	q->h = pix_mp->height;
+	q->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
+	q->bytesperline[0] = pix_mp->plane_fmt[0].bytesperline;

Ditto.

Quoted 46 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +23 -12
+}
+
 static void mtk_jpeg_set_dec_default_params(struct mtk_jpeg_ctx *ctx)
 {
 	struct mtk_jpeg_q_data *q = &ctx->out_q;
+	struct v4l2_pix_format_mplane *pix_mp;
 	int i;
 
+	pix_mp = kmalloc(sizeof(*pix_mp), GFP_KERNEL);
+
+	ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
 	ctx->colorspace = V4L2_COLORSPACE_JPEG,
 	ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
 	ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
 	ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;
-
-	q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_JPEG,
-					      MTK_JPEG_FMT_TYPE_OUTPUT);
-	q->w = MTK_JPEG_MIN_WIDTH;
-	q->h = MTK_JPEG_MIN_HEIGHT;
-	q->bytesperline[0] = 0;
-	q->sizeimage[0] = MTK_JPEG_DEFAULT_SIZEIMAGE;
+	pix_mp->width = MTK_JPEG_MIN_WIDTH;
+	pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+
+	q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_JPEG,
+				      MTK_JPEG_FMT_FLAG_DEC_OUTPUT);
+	vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+				    fmt.pix_mp), q->fmt);
+	q->w = pix_mp->width;
+	q->h = pix_mp->height;
+	q->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
+	q->bytesperline[0] = pix_mp->plane_fmt[0].bytesperline;
 
 	q = &ctx->cap_q;
-	q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_YUV420M,
-					      MTK_JPEG_FMT_TYPE_CAPTURE);
-	q->w = MTK_JPEG_MIN_WIDTH;
-	q->h = MTK_JPEG_MIN_HEIGHT;
-
+	q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_YUV420M,
+				      MTK_JPEG_FMT_FLAG_DEC_CAPTURE);
+	pix_mp->width = MTK_JPEG_MIN_WIDTH;
+	pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+	vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+				    fmt.pix_mp), q->fmt);

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1434

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -2
+	q->w = pix_mp->width;
+	q->h = pix_mp->height;
 	for (i = 0; i < q->fmt->colplanes; i++) {
-		u32 stride = q->w * q->fmt->h_sample[i] / 4;
-		u32 h = q->h * q->fmt->v_sample[i] / 4;
+		q->sizeimage[i] = pix_mp->plane_fmt[i].sizeimage;
+		q->bytesperline[i] = pix_mp->plane_fmt[i].bytesperline;
+	}
+}
 

Same comments as for the encoder version.

On top of that, both functions are almost identical and it should be possible to merge them.

Quoted 38 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +35 -2
-		q->bytesperline[i] = stride;
-		q->sizeimage[i] = stride * h;
+static int mtk_jpeg_enc_open(struct file *file)
+{
+	struct mtk_jpeg_dev *jpeg = video_drvdata(file);
+	struct video_device *vfd = video_devdata(file);
+	struct mtk_jpeg_ctx *ctx;
+	int ret = 0;
+
+	ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+	if (!ctx)
+		return -ENOMEM;
+
+	if (mutex_lock_interruptible(&jpeg->lock)) {
+		ret = -ERESTARTSYS;
+		goto free;
+	}
+
+	v4l2_fh_init(&ctx->fh, vfd);
+	file->private_data = &ctx->fh;
+	v4l2_fh_add(&ctx->fh);
+
+	ctx->jpeg = jpeg;
+	ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(jpeg->m2m_dev, ctx,
+					    mtk_jpeg_enc_queue_init);
+	if (IS_ERR(ctx->fh.m2m_ctx)) {
+		ret = PTR_ERR(ctx->fh.m2m_ctx);
+		goto error;
 	}
+
+	ret = mtk_jpeg_enc_ctrls_setup(ctx);
+	if (ret) {
+		v4l2_err(&jpeg->v4l2_dev, "Failed to setup jpeg enc controls\n");
+		goto error;
+	}
+	mtk_jpeg_set_enc_default_params(ctx);
+
+	mutex_unlock(&jpeg->lock);

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1480

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +9 -0
+	return 0;
+
+error:
+	v4l2_fh_del(&ctx->fh);
+	v4l2_fh_exit(&ctx->fh);
+	mutex_unlock(&jpeg->lock);
+free:
+	kfree(ctx);
+	return ret;
 }

It looks like the queue_init argument to v4l2_m2m_ctx_init() and control handling would be the only differences from the decoder version. Perhaps the functions can be merged?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_open(struct file *file) · line 1515

Quoted 12 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
 static int mtk_jpeg_dec_open(struct file *file)
@@ -953,6 +1507,12 @@ static int mtk_jpeg_dec_open(struct file *file)
 		goto error;
 	}
 
+	v4l2_ctrl_handler_init(&ctx->ctrl_hdl, 0);
+	ret = v4l2_ctrl_handler_setup(&ctx->ctrl_hdl);
+	if (ret) {
+		v4l2_err(&jpeg->v4l2_dev, "Failed to setup jpeg dec controls\n");
+		goto error;
+	}

There are no controls for the decoder, so there should be no need to set up a control handler.

Quoted 14 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -0
 	mtk_jpeg_set_dec_default_params(ctx);
 	mutex_unlock(&jpeg->lock);
 	return 0;
@@ -973,6 +1533,7 @@ static int mtk_jpeg_release(struct file *file)
 
 	mutex_lock(&jpeg->lock);
 	v4l2_m2m_ctx_release(ctx->fh.m2m_ctx);
+	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
 	v4l2_fh_del(&ctx->fh);
 	v4l2_fh_exit(&ctx->fh);
 	kfree(ctx);
@@ -980,6 +1541,15 @@ static int mtk_jpeg_release(struct file *file)
 	return 0;
 }

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_release(struct file *file) · line 1552

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +9 -0
 
+static const struct v4l2_file_operations mtk_jpeg_enc_fops = {
+	.owner          = THIS_MODULE,
+	.open           = mtk_jpeg_enc_open,
+	.release        = mtk_jpeg_release,
+	.poll           = v4l2_m2m_fop_poll,
+	.unlocked_ioctl = video_ioctl2,
+	.mmap           = v4l2_m2m_fop_mmap,
+};
+

If we merge the .open() implementation, the same struct could be used for both decoder and encoder.

[snip]

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_probe(struct platform_device *pdev) · line 1627

Quoted 12 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -2
@@ -1042,8 +1619,12 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 		return jpeg_irq;
 	}
 
-	ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_dec_irq, 0,
-			       pdev->name, jpeg);
+	if (jpeg->variant->is_encoder)
+		ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_enc_irq,
+				       0, pdev->name, jpeg);
+	else
+		ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_dec_irq,
+				       0, pdev->name, jpeg);

Rather than having "is_encoder" in the variant struct, would it make more sense to have "irq_handler" instead? That would avoid the explicit if.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_probe(struct platform_device *pdev) · line 1650

Quoted 12 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -1
 	if (ret) {
 		dev_err(&pdev->dev, "Failed to request jpeg_irq %d (%d)\n",
 			jpeg_irq, ret);
@@ -1063,7 +1644,10 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 		goto err_dev_register;
 	}
 
-	jpeg->m2m_dev = v4l2_m2m_init(&mtk_jpeg_dec_m2m_ops);
+	if (jpeg->variant->is_encoder)
+		jpeg->m2m_dev = v4l2_m2m_init(&mtk_jpeg_enc_m2m_ops);
+	else
+		jpeg->m2m_dev = v4l2_m2m_init(&mtk_jpeg_dec_m2m_ops);

Same here. The variant struct could have a "m2m_ops" pointer.

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +0 -2
 	if (IS_ERR(jpeg->m2m_dev)) {
 		v4l2_err(&jpeg->v4l2_dev, "Failed to init mem2mem device\n");
 		ret = PTR_ERR(jpeg->m2m_dev);
@@ -1076,9 +1660,15 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 		goto err_vfd_jpeg_alloc;
 	}
 	snprintf(jpeg->vdev->name, sizeof(jpeg->vdev->name),
-		 "%s-dec", MTK_JPEG_NAME);
-	jpeg->vdev->fops = &mtk_jpeg_dec_fops;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_probe(struct platform_device *pdev) · line 1671

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +9 -1
-	jpeg->vdev->ioctl_ops = &mtk_jpeg_dec_ioctl_ops;
+		 "%s-%s", MTK_JPEG_NAME,
+		 jpeg->variant->is_encoder ? "enc" : "dec");
+	if (jpeg->variant->is_encoder) {
+		jpeg->vdev->fops = &mtk_jpeg_enc_fops;
+		jpeg->vdev->ioctl_ops = &mtk_jpeg_enc_ioctl_ops;
+	} else {
+		jpeg->vdev->fops = &mtk_jpeg_dec_fops;
+		jpeg->vdev->ioctl_ops = &mtk_jpeg_dec_ioctl_ops;
+	}

Similarly here.

[snip]

Quoted 13 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +1 -0
index 0b59e48495d5..9ec2c3350a16 100644
@@ -3,6 +3,7 @@
  * Copyright (c) 2016 MediaTek Inc.
  * Author: Ming Hsiu Tsai <[email protected]>
  *         Rick Chang <[email protected]>
+ *         Xia Jiang <[email protected]>
  */
 
 #ifndef _MTK_JPEG_CORE_H

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 22

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +2 -0
@@ -16,19 +17,21 @@
 #define MTK_JPEG_NAME		"mtk-jpeg"
 
 #define MTK_JPEG_COMP_MAX		3
+#define MTK_JPEG_MAX_CLOCKS		2
+
 

Duplicate blank line.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 26

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +2 -3
 #define MTK_JPEG_FMT_FLAG_DEC_OUTPUT	BIT(0)
 #define MTK_JPEG_FMT_FLAG_DEC_CAPTURE	BIT(1)
-
-#define MTK_JPEG_FMT_TYPE_OUTPUT	1
-#define MTK_JPEG_FMT_TYPE_CAPTURE	2
+#define MTK_JPEG_FMT_FLAG_ENC_OUTPUT	BIT(2)
+#define MTK_JPEG_FMT_FLAG_ENC_CAPTURE	BIT(3)

Do we need separate bits for decoder and encoder?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 31

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +2 -2
 #define MTK_JPEG_MIN_WIDTH	32U
 #define MTK_JPEG_MIN_HEIGHT	32U
-#define MTK_JPEG_MAX_WIDTH	8192U
-#define MTK_JPEG_MAX_HEIGHT	8192U
+#define MTK_JPEG_MAX_WIDTH	65535U
+#define MTK_JPEG_MAX_HEIGHT	65535U

If this is a change valid for the decoder too, it should be a separate patch.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 34

Quoted 3 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +1 -0
 #define MTK_JPEG_DEFAULT_SIZEIMAGE	(1 * 1024 * 1024)
+#define MTK_JPEG_MAX_EXIF_SIZE	(64 * 1024)

There is one thing that I realized now. If the EXIF mode is enabled, the driver needs to ensure that the buffer is big enough to hold the EXIF data. The vb2 .buf_prepare callback would be the right place to do that.

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +0 -0
 /**
  * enum mtk_jpeg_ctx_state - states of the context state machine
@@ -42,6 +45,18 @@ enum mtk_jpeg_ctx_state {
 	MTK_JPEG_SOURCE_CHANGE,
 };
 

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h enum mtk_jpeg_ctx_state { · line 57

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +10 -0
+/**
+ * mtk_jpeg_variant - mtk jpeg driver variant
+ * @is_encoder:		driver mode is jpeg encoder
+ * @clk_names:		clock names
+ * @num_clocks:		numbers of clock
+ */
+struct mtk_jpeg_variant {
+	bool is_encoder;
+	const char		*clk_names[MTK_JPEG_MAX_CLOCKS];
+	int			num_clocks;

hint: Please avoid tabs between types and names, as it makes it difficult to add new fields later (the number of tabs might need to change for all members).

[snip]

Quoted 28 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +22 -0
new file mode 100644
index 000000000000..7fc1de920a75
@@ -0,0 +1,193 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ * Author: Xia Jiang <[email protected]>
+ *
+ */
+
+#include <linux/io.h>
+#include <linux/kernel.h>
+#include <media/videobuf2-core.h>
+
+#include "mtk_jpeg_enc_hw.h"
+
+static const struct mtk_jpeg_enc_qlt mtk_jpeg_enc_quality[] = {
+	{.quality_param = 34, .hardware_value = JPEG_ENC_QUALITY_Q34},
+	{.quality_param = 39, .hardware_value = JPEG_ENC_QUALITY_Q39},
+	{.quality_param = 48, .hardware_value = JPEG_ENC_QUALITY_Q48},
+	{.quality_param = 60, .hardware_value = JPEG_ENC_QUALITY_Q60},
+	{.quality_param = 64, .hardware_value = JPEG_ENC_QUALITY_Q64},
+	{.quality_param = 68, .hardware_value = JPEG_ENC_QUALITY_Q68},
+	{.quality_param = 74, .hardware_value = JPEG_ENC_QUALITY_Q74},
+	{.quality_param = 80, .hardware_value = JPEG_ENC_QUALITY_Q80},

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 32

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +10 -0
+	{.quality_param = 82, .hardware_value = JPEG_ENC_QUALITY_Q82},
+	{.quality_param = 84, .hardware_value = JPEG_ENC_QUALITY_Q84},
+	{.quality_param = 87, .hardware_value = JPEG_ENC_QUALITY_Q87},
+	{.quality_param = 90, .hardware_value = JPEG_ENC_QUALITY_Q90},
+	{.quality_param = 92, .hardware_value = JPEG_ENC_QUALITY_Q92},
+	{.quality_param = 95, .hardware_value = JPEG_ENC_QUALITY_Q95},
+	{.quality_param = 97, .hardware_value = JPEG_ENC_QUALITY_Q97},
+};
+
+void mtk_jpeg_enc_reset(void __iomem *base)

I'd suggest passing struct mtk_jpeg_dev pointer to all these functions, in case more data about the hardware is needed in the future.

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 34

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +2 -0
+{
+	writel(0x00, base + JPEG_ENC_RSTB);

nit: Just 0 is enough.

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 36

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +2 -0
+	writel(JPEG_ENC_RESET_BIT, base + JPEG_ENC_RSTB);
+	writel(0x00, base + JPEG_ENC_CODEC_SEL);

Ditto.

Quoted 19 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +19 -0
+}
+
+u32 mtk_jpeg_enc_get_and_clear_int_status(void __iomem *base)
+{
+	u32 ret;
+
+	ret = readl(base + JPEG_ENC_INT_STS) &
+		    JPEG_ENC_INT_STATUS_MASK_ALLIRQ;
+	if (ret)
+		writel(0, base + JPEG_ENC_INT_STS);
+
+	return ret;
+}
+
+u32 mtk_jpeg_enc_get_file_size(void __iomem *base)
+{
+	return readl(base + JPEG_ENC_DMA_ADDR0) -
+	       readl(base + JPEG_ENC_DST_ADDR0);
+}

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 65

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +10 -0
+
+u32 mtk_jpeg_enc_enum_result(void __iomem *base, u32 irq_status)
+{
+	if (irq_status & JPEG_ENC_INT_STATUS_DONE)
+		return MTK_JPEG_ENC_RESULT_DONE;
+	else if (irq_status & JPEG_ENC_INT_STATUS_STALL)
+		return MTK_JPEG_ENC_RESULT_STALL;
+	else
+		return MTK_JPEG_ENC_RESULT_VCODEC_IRQ;
+}

It looks like the driver only cares about 2 cases: DONE and anything else. Actually it also cares about a case where no bits are set in irq_status, which could be an interrupt controller misconfiguration or an interrupt generated by another device on the same shared IRQ line, which should be handled by returning IRQ_NONE. Since interrupt handling is a low level hardware operation, I'd suggest another design:
- put the irq_handler_t function here in this file and have it directly access the hardware registers and check the interrupt bits,
- add an mtk_jpeg_enc_done(..., enum vb2_buffer_state state, size_t payload), which would be called from this low level interrupt handler and do the V4L2 buffer and M2M job handling.

WDYT?

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 67

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +2 -0
+
+void mtk_jpeg_enc_set_img_size(void __iomem *base, u32 width, u32 height)

If we pass struct mtk_jpeg_dev to this function, the width and height arguments wouldn't be necessary, as they could be directly retrieved from the driver state.

Similar advice applies to the other functions in this file.

Quoted 22 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +22 -0
+{
+	u32 value;
+
+	value = width << 16 | height;
+	writel(value, base + JPEG_ENC_IMG_SIZE);
+}
+
+void mtk_jpeg_enc_set_blk_num(void __iomem *base, u32 enc_format, u32 width,
+			      u32 height)
+{
+	u32 blk_num;
+	u32 is_420;
+	u32 padding_width;
+	u32 padding_height;
+	u32 luma_blocks;
+	u32 chroma_blocks;
+
+	is_420 = (enc_format == V4L2_PIX_FMT_NV12M ||
+		  enc_format == V4L2_PIX_FMT_NV21M) ? 1 : 0;
+	padding_width = round_up(width, 16);
+	padding_height = round_up(height, is_420 ? 16 : 8);
+

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 99

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +10 -0
+	luma_blocks = padding_width / 8 * padding_height / 8;
+	if (is_420)
+		chroma_blocks = luma_blocks / 4;
+	else
+		chroma_blocks = luma_blocks / 2;
+
+	blk_num = luma_blocks + 2 * chroma_blocks - 1;
+
+	writel(blk_num, base + JPEG_ENC_BLK_NUM);
+}

My comments for this function from v7 haven't been addressed.

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +10 -0
+
+void mtk_jpeg_enc_set_stride(void __iomem *base, u32 enc_format, u32 width,
+			     u32 height, u32 bytesperline)
+{
+	u32 img_stride;
+	u32 mem_stride;
+
+	if (enc_format == V4L2_PIX_FMT_NV12M ||
+	    enc_format == V4L2_PIX_FMT_NV21M) {
+		img_stride = round_up(width, 16);

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 119

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +10 -0
+		mem_stride = bytesperline;
+	} else {
+		img_stride = round_up(width * 2, 32);
+		mem_stride = img_stride;
+	}
+
+	writel(img_stride, base + JPEG_ENC_IMG_STRIDE);
+	writel(mem_stride, base + JPEG_ENC_STRIDE);
+}
+

My comments for this function from v7 haven't been addressed.

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 126

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +7 -0
+void mtk_jpeg_enc_set_src_addr(void __iomem *base, u32 src_addr,
+			       u32 plane_index)
+{
+	if (!plane_index)
+		writel(src_addr, base + JPEG_ENC_SRC_LUMA_ADDR);
+	else
+		writel(src_addr, base + JPEG_ENC_SRC_CHROMA_ADDR);

Do we need this plane_index if we only have 2 planes? Also, for interleaved formats, like YUYV, what should the LUMA and CHROMA registers be set to?

Quoted 16 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +16 -0
+}
+
+void mtk_jpeg_enc_set_dst_addr(void __iomem *base, u32 dst_addr,
+			       u32 stall_size, u32 init_offset,
+			       u32 offset_mask)
+{
+	writel(init_offset & ~0xf, base + JPEG_ENC_OFFSET_ADDR);
+	writel(offset_mask & 0xf, base + JPEG_ENC_BYTE_OFFSET_MASK);
+	writel(dst_addr & ~0xf, base + JPEG_ENC_DST_ADDR0);
+	writel((dst_addr + stall_size) & ~0xf, base + JPEG_ENC_STALL_ADDR0);
+}
+
+static void mtk_jpeg_enc_set_quality(void __iomem *base, u32 quality)
+{
+	u32 value;
+	u32 i, enc_quality;

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 152

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +10 -0
+
+	enc_quality = mtk_jpeg_enc_quality[0].hardware_value;
+	for (i = 0; i < ARRAY_SIZE(mtk_jpeg_enc_quality); i++) {
+		if (quality <= mtk_jpeg_enc_quality[i].quality_param) {
+			enc_quality = mtk_jpeg_enc_quality[i].hardware_value;
+			break;
+		}
+	}
+
+	value = readl(base + JPEG_ENC_QUALITY);

nit: Is it still necessary to read the register here? Typically the reserved bits would be defined as SBZ (should be zero) and it should be safe to just write 0 to them.

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 162

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +10 -0
+	value = (value & JPEG_ENC_QUALITY_MASK) | enc_quality;
+	writel(value, base + JPEG_ENC_QUALITY);
+}
+
+static void mtk_jpeg_enc_set_ctrl(void __iomem *base, u32 enc_format,
+				  bool exif_en, u32 restart_interval)
+{
+	u32 value;
+
+	value = readl(base + JPEG_ENC_CTRL);

nit: Is it still necessary to read the register here?

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 175

Quoted 13 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +13 -0
+	value &= ~JPEG_ENC_CTRL_YUV_FORMAT_MASK;
+	value |= (enc_format & 3) << 3;
+	if (exif_en)
+		value |= JPEG_ENC_CTRL_FILE_FORMAT_BIT;
+	else
+		value &= ~JPEG_ENC_CTRL_FILE_FORMAT_BIT;
+	if (restart_interval)
+		value |= JPEG_ENC_CTRL_RESTART_EN_BIT;
+	else
+		value &= ~JPEG_ENC_CTRL_RESTART_EN_BIT;
+	writel(value, base + JPEG_ENC_CTRL);
+}
+

Best regards,
Tomasz

author
#35 lore raw

Re: [PATCH RESEND v9 18/18] media: platform: Add jpeg enc feature

Quoted 22 lines · Tomasz Figa

Hi Xia,

Quoted 8 lines · Xia Jiang

Add mtk jpeg encode v4l2 driver based on jpeg decode, because that jpeg decode and encode have great similarities with function operation.

---
v9: add member variable(struct v4l2_rect) in out_q structure for storing
the active crop information.
move the renaming exsting functions/defines/variables to a separate patch.

Quoted 10 lines
---
 drivers/media/platform/mtk-jpeg/Makefile      |   5 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 845 +++++++++++++++---
 .../media/platform/mtk-jpeg/mtk_jpeg_core.h   |  44 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c | 193 ++++
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h | 123 +++
 5 files changed, 1084 insertions(+), 126 deletions(-)
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h
Quoted 1 line

Thank you for the patch. Please see my comments inline.

Dear Tomasz,

Thank you for your reply.

I have fixed the driver by following your opinions almostly, but there is some confusion about some of the opinions. I'd like to have a further discussion with you. Please see my reply below.

Quoted 635 lines

[snip]

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static inline struct mtk_jpeg_src_buf *mtk_jpeg_vb2_to_srcbuf( · line 145

Quoted 12 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +12 -0
+static int mtk_jpeg_enc_querycap(struct file *file, void *priv,
+				 struct v4l2_capability *cap)
+{
+	struct mtk_jpeg_dev *jpeg = video_drvdata(file);
+
+	strscpy(cap->driver, MTK_JPEG_NAME, sizeof(cap->driver));
+	strscpy(cap->card, MTK_JPEG_NAME " encoder", sizeof(cap->card));
+	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
+		 dev_name(jpeg->dev));
+
+	return 0;
+}

I can see that this function differs from mtk_jpeg_dec_querycap() only with the " encoder" string. Perhaps they could be merged?

Quoted 12 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -0
+
 static int mtk_jpeg_dec_querycap(struct file *file, void *priv,
 				 struct v4l2_capability *cap)
 {
@@ -88,6 +157,54 @@ static int mtk_jpeg_dec_querycap(struct file *file, void *priv,
 	return 0;
 }
 
+static int vidioc_jpeg_enc_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_jpeg_ctx *ctx = ctrl_to_ctx(ctrl);
+

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_querycap(struct file *file, void *priv, · line 173

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+	switch (ctrl->id) {
+	case V4L2_CID_JPEG_RESTART_INTERVAL:
+		ctx->restart_interval = ctrl->val;
+		break;
+	case V4L2_CID_JPEG_COMPRESSION_QUALITY:
+		ctx->enc_quality = ctrl->val;
+		break;
+	case V4L2_CID_JPEG_ACTIVE_MARKER:
+		ctx->enable_exif = ctrl->val & V4L2_JPEG_ACTIVE_MARKER_APP1 ?
+				   true : false;

nit: If ctx->enable_exif is of the bool type, the ternary operator could be removed, because any non-zero value is implicitly turned into 1, as per [1].

[1] https://www.kernel.org/doc/html/v5.6/process/coding-style.html#using-bool

[snip]

Quoted 12 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +8 -0
+static int mtk_jpeg_enc_enum_fmt_vid_cap(struct file *file, void *priv,
+					 struct v4l2_fmtdesc *f)
+{
+	return mtk_jpeg_enum_fmt(mtk_jpeg_enc_formats,
+				 MTK_JPEG_ENC_NUM_FORMATS, f,
+				 MTK_JPEG_FMT_FLAG_ENC_CAPTURE);
+}
+
 static int mtk_jpeg_dec_enum_fmt_vid_cap(struct file *file, void *priv,
 					 struct v4l2_fmtdesc *f)
 {

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_enum_fmt_vid_cap(struct file *file, void *priv, · line 250

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
@@ -117,6 +242,14 @@ static int mtk_jpeg_dec_enum_fmt_vid_cap(struct file *file, void *priv,
 				 MTK_JPEG_FMT_FLAG_DEC_CAPTURE);
 }
 
+static int mtk_jpeg_enc_enum_fmt_vid_out(struct file *file, void *priv,
+					 struct v4l2_fmtdesc *f)
+{
+	return mtk_jpeg_enum_fmt(mtk_jpeg_enc_formats,
+				 MTK_JPEG_ENC_NUM_FORMATS, f,
+				 MTK_JPEG_FMT_FLAG_ENC_OUTPUT);

Do we need separate implementations of these? "formats" and "num_formats" could be specified by the match data struct and used in a generic function.

Also, do we need separate flags for ENC_OUTPUT/CAPTURE and DEC_OUTPUT/CAPTURE?

Quoted 26 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -6
+}
+
 static int mtk_jpeg_dec_enum_fmt_vid_out(struct file *file, void *priv,
 					 struct v4l2_fmtdesc *f)
 {
@@ -132,93 +265,66 @@ mtk_jpeg_get_q_data(struct mtk_jpeg_ctx *ctx, enum v4l2_buf_type type)
 	return &ctx->cap_q;
 }
 
-static struct mtk_jpeg_fmt *mtk_jpeg_find_format(struct mtk_jpeg_ctx *ctx,
-						 u32 pixelformat,
+static struct mtk_jpeg_fmt *mtk_jpeg_find_format(u32 pixelformat,
 						 unsigned int fmt_type)
 {
-	unsigned int k, fmt_flag;
+	unsigned int k;
+	struct mtk_jpeg_fmt *fmt;
 
-	fmt_flag = (fmt_type == MTK_JPEG_FMT_TYPE_OUTPUT) ?
-		   MTK_JPEG_FMT_FLAG_DEC_OUTPUT :
-		   MTK_JPEG_FMT_FLAG_DEC_CAPTURE;
+	for (k = 0; k < MTK_JPEG_ENC_NUM_FORMATS; k++) {
+		fmt = &mtk_jpeg_enc_formats[k];
+
+		if (fmt->fourcc == pixelformat && fmt->flags & fmt_type)
+			return fmt;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c mtk_jpeg_get_q_data(struct mtk_jpeg_ctx *ctx, enum v4l2_buf_type type) · line 286

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -2
+	}
 
 	for (k = 0; k < MTK_JPEG_DEC_NUM_FORMATS; k++) {
-		struct mtk_jpeg_fmt *fmt = &mtk_jpeg_dec_formats[k];
+		fmt = &mtk_jpeg_dec_formats[k];
 
-		if (fmt->fourcc == pixelformat && fmt->flags & fmt_flag)
+		if (fmt->fourcc == pixelformat && fmt->flags & fmt_type)
 			return fmt;
 	}

We should only need to iterate through one of the arrays. My suggestion above about making "formats" and "num_formats" a member of the match data should take care of this one too.

Quoted 18 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +0 -14
 	return NULL;
 }
 
-static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx,
-				       struct v4l2_format *f)
-{
-	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
-	struct mtk_jpeg_q_data *q_data;
-	int i;
-
-	q_data = mtk_jpeg_get_q_data(ctx, f->type);
-
-	pix_mp->width = q_data->w;
-	pix_mp->height = q_data->h;
-	pix_mp->pixelformat = q_data->fmt->fourcc;
-	pix_mp->num_planes = q_data->fmt->colplanes;
-

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c mtk_jpeg_get_q_data(struct mtk_jpeg_ctx *ctx, enum v4l2_buf_type type) · line 291

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -9
-	for (i = 0; i < pix_mp->num_planes; i++) {
-		pix_mp->plane_fmt[i].bytesperline = q_data->bytesperline[i];
-		pix_mp->plane_fmt[i].sizeimage = q_data->sizeimage[i];
-	}
-}
-
-static int mtk_jpeg_try_fmt_mplane(struct v4l2_format *f,
-				   struct mtk_jpeg_fmt *fmt,
-				   struct mtk_jpeg_ctx *ctx, int q_type)
+static int vidioc_try_fmt(struct v4l2_format *f, struct mtk_jpeg_fmt *fmt)

The name is a bit misleading, because it's suggesting that it's the vidioc_try_fmt callback, but it's just an internal helper. I think the original name was fine.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c mtk_jpeg_get_q_data(struct mtk_jpeg_ctx *ctx, enum v4l2_buf_type type) · line 187

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +0 -5
 {
 	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
 	int i;
 
 	pix_mp->field = V4L2_FIELD_NONE;
-
-	if (ctx->state != MTK_JPEG_INIT) {
-		mtk_jpeg_adjust_fmt_mplane(ctx, f);
-		return 0;
-	}

Is it okay to remove this? My understanding is that it prevented changing the format while streaming, which should is as expected.

Quoted 47 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +22 -16
-
 	pix_mp->num_planes = fmt->colplanes;
 	pix_mp->pixelformat = fmt->fourcc;
 
-	if (q_type == MTK_JPEG_FMT_TYPE_OUTPUT) {
-		struct v4l2_plane_pix_format *pfmt = &pix_mp->plane_fmt[0];
-
+	if (fmt->fourcc == V4L2_PIX_FMT_JPEG) {
 		pix_mp->height = clamp(pix_mp->height, MTK_JPEG_MIN_HEIGHT,
 				       MTK_JPEG_MAX_HEIGHT);
 		pix_mp->width = clamp(pix_mp->width, MTK_JPEG_MIN_WIDTH,
 				      MTK_JPEG_MAX_WIDTH);
-
-		pfmt->bytesperline = 0;
-		/* Source size must be aligned to 128 */
-		pfmt->sizeimage = round_up(pfmt->sizeimage, 128);
-		if (pfmt->sizeimage == 0)
-			pfmt->sizeimage = MTK_JPEG_DEFAULT_SIZEIMAGE;
-		return 0;
+		pix_mp->plane_fmt[0].bytesperline = 0;
+		pix_mp->plane_fmt[0].sizeimage =
+				round_up(pix_mp->plane_fmt[0].sizeimage, 128);
+		if (pix_mp->plane_fmt[0].sizeimage == 0)
+			pix_mp->plane_fmt[0].sizeimage =
+				MTK_JPEG_DEFAULT_SIZEIMAGE;
+	} else {
+		pix_mp->height = clamp(round_up(pix_mp->height, fmt->v_align),
+				       MTK_JPEG_MIN_HEIGHT,
+				       MTK_JPEG_MAX_HEIGHT);
+		pix_mp->width = clamp(round_up(pix_mp->width, fmt->h_align),
+				      MTK_JPEG_MIN_WIDTH, MTK_JPEG_MAX_WIDTH);
+		for (i = 0; i < pix_mp->num_planes; i++) {
+			struct v4l2_plane_pix_format *pfmt =
+							&pix_mp->plane_fmt[i];
+			u32 stride = pix_mp->width * fmt->h_sample[i] / 4;
+			u32 h = pix_mp->height * fmt->v_sample[i] / 4;
+
+			pfmt->bytesperline = stride;
+			pfmt->sizeimage = stride * h;
+		}
 	}
 
-	/* type is MTK_JPEG_FMT_TYPE_CAPTURE */
-	pix_mp->height = clamp(round_up(pix_mp->height, fmt->v_align),
-			       MTK_JPEG_MIN_HEIGHT, MTK_JPEG_MAX_HEIGHT);
-	pix_mp->width = clamp(round_up(pix_mp->width, fmt->h_align),
-			      MTK_JPEG_MIN_WIDTH, MTK_JPEG_MAX_WIDTH);

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c mtk_jpeg_get_q_data(struct mtk_jpeg_ctx *ctx, enum v4l2_buf_type type) · line 328

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +0 -9
-
-	for (i = 0; i < fmt->colplanes; i++) {
-		struct v4l2_plane_pix_format *pfmt = &pix_mp->plane_fmt[i];
-		u32 stride = pix_mp->width * fmt->h_sample[i] / 4;
-		u32 h = pix_mp->height * fmt->v_sample[i] / 4;
-
-		pfmt->bytesperline = stride;
-		pfmt->sizeimage = stride * h;
-	}
 	return 0;

Are all the changes above needed for the encoder? If I'm looking correctly, they're mostly refactoring and should be done in a separate patch. However, I don't see any big issue with the existing coding style in this function, so perhaps the refactoring could be avoided.

Quoted 16 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
 }
 
@@ -271,14 +377,35 @@ static int mtk_jpeg_g_fmt_vid_mplane(struct file *file, void *priv,
 	return 0;
 }
 
+static int mtk_jpeg_enc_try_fmt_vid_cap_mplane(struct file *file, void *priv,
+					       struct v4l2_format *f)
+{
+	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+	struct mtk_jpeg_fmt *fmt;
+
+	fmt = mtk_jpeg_find_format(f->fmt.pix_mp.pixelformat,
+				   MTK_JPEG_FMT_FLAG_ENC_CAPTURE);
+	if (!fmt)
+		fmt = ctx->cap_q.fmt;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_g_fmt_vid_mplane(struct file *file, void *priv, · line 399

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+
+	v4l2_dbg(2, debug, &ctx->jpeg->v4l2_dev, "(%d) try_fmt:%c%c%c%c\n",
+		 f->type,
+		 (fmt->fourcc & 0xff),
+		 (fmt->fourcc >>  8 & 0xff),
+		 (fmt->fourcc >> 16 & 0xff),
+		 (fmt->fourcc >> 24 & 0xff));
+
+	return vidioc_try_fmt(f, fmt);
+}

Is there really anything encoder-specific in this function? Could the same function as the decoder be used instead?

Quoted 15 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -2
+
 static int mtk_jpeg_dec_try_fmt_vid_cap_mplane(struct file *file, void *priv,
 					       struct v4l2_format *f)
 {
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
 	struct mtk_jpeg_fmt *fmt;
 
-	fmt = mtk_jpeg_find_format(ctx, f->fmt.pix_mp.pixelformat,
-				   MTK_JPEG_FMT_TYPE_CAPTURE);
+	fmt = mtk_jpeg_find_format(f->fmt.pix_mp.pixelformat,
+				   MTK_JPEG_FMT_FLAG_DEC_CAPTURE);
 	if (!fmt)
 		fmt = ctx->cap_q.fmt;
 

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_try_fmt_vid_cap_mplane(struct file *file, void *priv, · line 422

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -1
@@ -289,7 +416,33 @@ static int mtk_jpeg_dec_try_fmt_vid_cap_mplane(struct file *file, void *priv,
 		 (fmt->fourcc >> 16 & 0xff),
 		 (fmt->fourcc >> 24 & 0xff));
 
-	return mtk_jpeg_try_fmt_mplane(f, fmt, ctx, MTK_JPEG_FMT_TYPE_CAPTURE);
+	if (ctx->state != MTK_JPEG_INIT) {
+		mtk_jpeg_g_fmt_vid_mplane(file, priv, f);
+		return 0;
+	}

Is this really limited to the decoder? In general currently we don't support changing the resolution from the userspace when streaming is started and it applies to both encoder and decoder.

Quoted 14 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +14 -0
+
+	return vidioc_try_fmt(f, fmt);
+}
+
+static int mtk_jpeg_enc_try_fmt_vid_out_mplane(struct file *file, void *priv,
+					       struct v4l2_format *f)
+{
+	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+	struct mtk_jpeg_fmt *fmt;
+
+	fmt = mtk_jpeg_find_format(f->fmt.pix_mp.pixelformat,
+				   MTK_JPEG_FMT_FLAG_ENC_OUTPUT);
+	if (!fmt)
+		fmt = ctx->out_q.fmt;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_try_fmt_vid_cap_mplane(struct file *file, void *priv, · line 446

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +9 -0
+
+	v4l2_dbg(2, debug, &ctx->jpeg->v4l2_dev, "(%d) try_fmt:%c%c%c%c\n",
+		 f->type,
+		 (fmt->fourcc & 0xff),
+		 (fmt->fourcc >>  8 & 0xff),
+		 (fmt->fourcc >> 16 & 0xff),
+		 (fmt->fourcc >> 24 & 0xff));
+
+	return vidioc_try_fmt(f, fmt);
 }

Ditto.

Quoted 14 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -2
 static int mtk_jpeg_dec_try_fmt_vid_out_mplane(struct file *file, void *priv,
@@ -298,8 +451,8 @@ static int mtk_jpeg_dec_try_fmt_vid_out_mplane(struct file *file, void *priv,
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
 	struct mtk_jpeg_fmt *fmt;
 
-	fmt = mtk_jpeg_find_format(ctx, f->fmt.pix_mp.pixelformat,
-				   MTK_JPEG_FMT_TYPE_OUTPUT);
+	fmt = mtk_jpeg_find_format(f->fmt.pix_mp.pixelformat,
+				   MTK_JPEG_FMT_FLAG_DEC_OUTPUT);
 	if (!fmt)
 		fmt = ctx->out_q.fmt;
 

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_try_fmt_vid_out_mplane(struct file *file, void *priv, · line 469

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -1
@@ -310,17 +463,21 @@ static int mtk_jpeg_dec_try_fmt_vid_out_mplane(struct file *file, void *priv,
 		 (fmt->fourcc >> 16 & 0xff),
 		 (fmt->fourcc >> 24 & 0xff));
 
-	return mtk_jpeg_try_fmt_mplane(f, fmt, ctx, MTK_JPEG_FMT_TYPE_OUTPUT);
+	if (ctx->state != MTK_JPEG_INIT) {
+		mtk_jpeg_g_fmt_vid_mplane(file, priv, f);
+		return 0;
+	}

Ditto.

Quoted 76 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +34 -7
+
+	return vidioc_try_fmt(f, fmt);
 }
 
 static int mtk_jpeg_s_fmt_mplane(struct mtk_jpeg_ctx *ctx,
-				 struct v4l2_format *f)
+				 struct v4l2_format *f, unsigned int fmt_type)
 {
 	struct vb2_queue *vq;
 	struct mtk_jpeg_q_data *q_data = NULL;
 	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
 	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
-	unsigned int f_type;
 	int i;
 
 	vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, f->type);
@@ -334,12 +491,11 @@ static int mtk_jpeg_s_fmt_mplane(struct mtk_jpeg_ctx *ctx,
 		return -EBUSY;
 	}
 
-	f_type = V4L2_TYPE_IS_OUTPUT(f->type) ?
-			 MTK_JPEG_FMT_TYPE_OUTPUT : MTK_JPEG_FMT_TYPE_CAPTURE;
-
-	q_data->fmt = mtk_jpeg_find_format(ctx, pix_mp->pixelformat, f_type);
+	q_data->fmt = mtk_jpeg_find_format(pix_mp->pixelformat, fmt_type);
 	q_data->w = pix_mp->width;
 	q_data->h = pix_mp->height;
+	q_data->crop_rect.width = pix_mp->width;
+	q_data->crop_rect.height = pix_mp->height;
 	ctx->colorspace = pix_mp->colorspace;
 	ctx->ycbcr_enc = pix_mp->ycbcr_enc;
 	ctx->xfer_func = pix_mp->xfer_func;
@@ -365,6 +521,19 @@ static int mtk_jpeg_s_fmt_mplane(struct mtk_jpeg_ctx *ctx,
 	return 0;
 }
 
+static int mtk_jpeg_enc_s_fmt_vid_out_mplane(struct file *file, void *priv,
+					     struct v4l2_format *f)
+{
+	int ret;
+
+	ret = mtk_jpeg_enc_try_fmt_vid_out_mplane(file, priv, f);
+	if (ret)
+		return ret;
+
+	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f,
+				     MTK_JPEG_FMT_FLAG_ENC_OUTPUT);
+}
+
 static int mtk_jpeg_dec_s_fmt_vid_out_mplane(struct file *file, void *priv,
 					     struct v4l2_format *f)
 {
@@ -374,7 +543,21 @@ static int mtk_jpeg_dec_s_fmt_vid_out_mplane(struct file *file, void *priv,
 	if (ret)
 		return ret;
 
-	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f);
+	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f,
+				     MTK_JPEG_FMT_FLAG_DEC_OUTPUT);
+}
+
+static int mtk_jpeg_enc_s_fmt_vid_cap_mplane(struct file *file, void *priv,
+					     struct v4l2_format *f)
+{
+	int ret;
+
+	ret = mtk_jpeg_enc_try_fmt_vid_cap_mplane(file, priv, f);
+	if (ret)
+		return ret;
+
+	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f,
+				     MTK_JPEG_FMT_FLAG_ENC_CAPTURE);
 }
 
 static int mtk_jpeg_dec_s_fmt_vid_cap_mplane(struct file *file, void *priv,

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_s_fmt_vid_cap_mplane(struct file *file, void *priv, · line 574

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -1
@@ -386,7 +569,8 @@ static int mtk_jpeg_dec_s_fmt_vid_cap_mplane(struct file *file, void *priv,
 	if (ret)
 		return ret;
 
-	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f);
+	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f,
+				     MTK_JPEG_FMT_FLAG_DEC_CAPTURE);
 }

Is it really necessary to have separate variants of the above functions for decoder and encoder?

Quoted 12 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
 static void mtk_jpeg_queue_src_chg_event(struct mtk_jpeg_ctx *ctx)
@@ -411,6 +595,29 @@ static int mtk_jpeg_subscribe_event(struct v4l2_fh *fh,
 	return v4l2_ctrl_subscribe_event(fh, sub);
 }
 
+static int mtk_jpeg_enc_g_selection(struct file *file, void *priv,
+				    struct v4l2_selection *s)
+{
+	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+
+	if (s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_subscribe_event(struct v4l2_fh *fh, · line 613

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+		return -EINVAL;
+
+	switch (s->target) {
+	case V4L2_SEL_TGT_CROP:
+	case V4L2_SEL_TGT_CROP_BOUNDS:
+	case V4L2_SEL_TGT_CROP_DEFAULT:
+		s->r.width = ctx->out_q.w;
+		s->r.height = ctx->out_q.h;
+		s->r.left = 0;
+		s->r.top = 0;

Is this really correct? The function seems to be returning the full frame size regardless of the selection target. For BOUNDS and DEFAULTS this would be the correct behavior indeed, but CROP should return the active crop rectangle, as set by S_SELECTION.

Quoted 45 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +31 -0
+		break;
+	default:
+		return -EINVAL;
+	}
+	return 0;
+}
+
 static int mtk_jpeg_dec_g_selection(struct file *file, void *priv,
 				    struct v4l2_selection *s)
 {
@@ -440,6 +647,29 @@ static int mtk_jpeg_dec_g_selection(struct file *file, void *priv,
 	return 0;
 }
 
+static int mtk_jpeg_enc_s_selection(struct file *file, void *priv,
+				    struct v4l2_selection *s)
+{
+	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+
+	if (s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)
+		return -EINVAL;
+
+	switch (s->target) {
+	case V4L2_SEL_TGT_CROP:
+		s->r.left = 0;
+		s->r.top = 0;
+		s->r.width = min(s->r.width, ctx->out_q.w);
+		s->r.height = min(s->r.height, ctx->out_q.h);
+		ctx->out_q.crop_rect = s->r;
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
 static int mtk_jpeg_dec_s_selection(struct file *file, void *priv,
 				    struct v4l2_selection *s)
 {
@@ -484,6 +714,33 @@ static int mtk_jpeg_qbuf(struct file *file, void *priv, struct v4l2_buffer *buf)
 	return v4l2_m2m_qbuf(file, fh->m2m_ctx, buf);
 }
 
+static const struct v4l2_ioctl_ops mtk_jpeg_enc_ioctl_ops = {

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_qbuf(struct file *file, void *priv, struct v4l2_buffer *buf) · line 727

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+	.vidioc_querycap                = mtk_jpeg_enc_querycap,
+	.vidioc_enum_fmt_vid_cap	= mtk_jpeg_enc_enum_fmt_vid_cap,
+	.vidioc_enum_fmt_vid_out	= mtk_jpeg_enc_enum_fmt_vid_out,
+	.vidioc_try_fmt_vid_cap_mplane	= mtk_jpeg_enc_try_fmt_vid_cap_mplane,
+	.vidioc_try_fmt_vid_out_mplane	= mtk_jpeg_enc_try_fmt_vid_out_mplane,
+	.vidioc_g_fmt_vid_cap_mplane    = mtk_jpeg_g_fmt_vid_mplane,
+	.vidioc_g_fmt_vid_out_mplane    = mtk_jpeg_g_fmt_vid_mplane,
+	.vidioc_s_fmt_vid_cap_mplane    = mtk_jpeg_enc_s_fmt_vid_cap_mplane,
+	.vidioc_s_fmt_vid_out_mplane    = mtk_jpeg_enc_s_fmt_vid_out_mplane,
+	.vidioc_qbuf                    = mtk_jpeg_qbuf,

Not directly a comment for this patch, but since the previous patch removed the LAST_FRAME handling, wouldn't it be enough to just use v4l2_m2m_qbuf() as this callback?

[snip]

Quoted 48 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +29 -0
+static void mtk_jpeg_enc_buf_queue(struct vb2_buffer *vb)
+{
+	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+
+	v4l2_dbg(2, debug, &jpeg->v4l2_dev, "(%d) buf_q id=%d, vb=%p\n",
+		 vb->vb2_queue->type, vb->index, vb);
+
+	v4l2_m2m_buf_queue(ctx->fh.m2m_ctx, to_vb2_v4l2_buffer(vb));
+}
+
 static void mtk_jpeg_dec_buf_queue(struct vb2_buffer *vb)
 {
 	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
@@ -664,6 +932,15 @@ static struct vb2_v4l2_buffer *mtk_jpeg_buf_remove(struct mtk_jpeg_ctx *ctx,
 		return v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
 }
 
+static void mtk_jpeg_enc_stop_streaming(struct vb2_queue *q)
+{
+	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(q);
+	struct vb2_v4l2_buffer *vb;
+
+	while ((vb = mtk_jpeg_buf_remove(ctx, q->type)))
+		v4l2_m2m_buf_done(vb, VB2_BUF_STATE_ERROR);
+}
+
 static void mtk_jpeg_dec_stop_streaming(struct vb2_queue *q)
 {
 	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(q);
@@ -699,6 +976,15 @@ static const struct vb2_ops mtk_jpeg_dec_qops = {
 	.stop_streaming     = mtk_jpeg_dec_stop_streaming,
 };
 
+static const struct vb2_ops mtk_jpeg_enc_qops = {
+	.queue_setup        = mtk_jpeg_queue_setup,
+	.buf_prepare        = mtk_jpeg_buf_prepare,
+	.buf_queue          = mtk_jpeg_enc_buf_queue,
+	.wait_prepare       = vb2_ops_wait_prepare,
+	.wait_finish        = vb2_ops_wait_finish,
+	.stop_streaming     = mtk_jpeg_enc_stop_streaming,
+};
+
 static void mtk_jpeg_set_dec_src(struct mtk_jpeg_ctx *ctx,
 				 struct vb2_buffer *src_buf,
 				 struct mtk_jpeg_bs *bs)
@@ -736,6 +1022,85 @@ static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx,
 	return 0;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx, · line 1032

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +8 -0
 }
 
+static void mtk_jpeg_set_enc_dst(struct mtk_jpeg_ctx *ctx, void __iomem *base,
+				 struct vb2_buffer *dst_buf,
+				 struct mtk_jpeg_enc_bs *bs)
+{
+	bs->dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
+	bs->dma_addr_offset = ctx->enable_exif ? MTK_JPEG_MAX_EXIF_SIZE : 0;
+	bs->dma_addr_offsetmask = bs->dma_addr & JPEG_ENC_DST_ADDR_OFFSET_MASK;
+	bs->size = vb2_plane_size(dst_buf, 0);

We're computing these values and then writing to the hardware straightaway. Do we need to save these values to the bs struct? If no, do we need the bs struct at all?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx, · line 1043

Quoted 11 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +11 -0
+
+	mtk_jpeg_enc_set_dst_addr(base, bs->dma_addr, bs->size,
+				  bs->dma_addr_offset,
+				  bs->dma_addr_offsetmask);
+}
+
+static void mtk_jpeg_set_enc_src(struct mtk_jpeg_ctx *ctx, void __iomem *base,
+				 struct vb2_buffer *src_buf)
+{
+	int i;
+	dma_addr_t	dma_addr;

nit: Only one space should be between variable type and name.

Quoted 46 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +46 -0
+
+	mtk_jpeg_enc_set_img_size(base, ctx->out_q.crop_rect.width,
+				  ctx->out_q.crop_rect.height);
+	mtk_jpeg_enc_set_blk_num(base, ctx->out_q.fmt->fourcc,
+				 ctx->out_q.crop_rect.width,
+				 ctx->out_q.crop_rect.height);
+	mtk_jpeg_enc_set_stride(base, ctx->out_q.fmt->fourcc, ctx->out_q.w,
+				ctx->out_q.h, ctx->out_q.bytesperline[0]);
+
+	for (i = 0; i < src_buf->num_planes; i++) {
+		dma_addr = vb2_dma_contig_plane_dma_addr(src_buf, i) +
+			   src_buf->planes[i].data_offset;
+		mtk_jpeg_enc_set_src_addr(base, dma_addr, i);
+	}
+}
+
+static void mtk_jpeg_enc_device_run(void *priv)
+{
+	struct mtk_jpeg_ctx *ctx = priv;
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+	struct vb2_v4l2_buffer *src_buf, *dst_buf;
+	enum vb2_buffer_state buf_state = VB2_BUF_STATE_ERROR;
+	unsigned long flags;
+	struct mtk_jpeg_src_buf *jpeg_src_buf;
+	struct mtk_jpeg_enc_bs enc_bs;
+	int ret;
+
+	src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
+	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
+	jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(&src_buf->vb2_buf);
+
+	ret = pm_runtime_get_sync(jpeg->dev);
+	if (ret < 0)
+		goto enc_end;
+
+	spin_lock_irqsave(&jpeg->hw_lock, flags);
+
+	/*
+	 * Resetting the hardware every frame is to ensure that all the
+	 * registers are cleared. This is a hardware requirement.
+	 */
+	mtk_jpeg_enc_reset(jpeg->reg_base);
+
+	mtk_jpeg_set_enc_dst(ctx, jpeg->reg_base, &dst_buf->vb2_buf, &enc_bs);
+	mtk_jpeg_set_enc_src(ctx, jpeg->reg_base, &src_buf->vb2_buf);
+	mtk_jpeg_enc_set_config(jpeg->reg_base, ctx->out_q.fmt->hw_format,
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx, · line 1092

Quoted 3 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -0
+				ctx->enable_exif, ctx->enc_quality,
+				ctx->restart_interval);
+	mtk_jpeg_enc_start(jpeg->reg_base);

Could we just move the above 5 functions into one function inside mtk_jpeg_enc_hw.c that takes mtk_jpeg_dev pointer as its argument, let's say mtk_jpeg_enc_hw_run() and simply program all the data to the registers directly, without the extra level of abstractions?

I can move the 5 functions into one function(mtk_jpeg_enc_hw_run()), but this function will be very long, because it contains computation code such as setting dst addr, blk_num, quality.
In v4, you have adviced the following architecture:
How about the following model, as used by many other drivers:

mtk_jpeg_enc_set_src()

{
        // Set any registers related to source format and buffer
}

mtk_jpeg_enc_set_dst()

{
        // Set any registers related to destination format and buffer
}

mtk_jpeg_enc_set_params()

{
        // Set any registers related to additional encoding parameters
}

mtk_jpeg_enc_device_run(enc, ctx)

{
        mtk_jpeg_enc_set_src(enc, src_buf, src_fmt);
        mtk_jpeg_enc_set_dst(enc, dst_buf, dst_fmt);
        mtk_jpeg_enc_set_params(enc, ctx);
        // Trigger the hardware run
}

I think that this architecture is more clear(mtk_jpeg_enc_set_config is equivalent to mtk_jpeg_enc_set_params).
Should I keep the original architecture or move 5 functions into mtk_jpeg_enc_hw_run?

Quoted 226 lines
Quoted 16 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +11 -0
+	spin_unlock_irqrestore(&jpeg->hw_lock, flags);
+	return;
+
+enc_end:
+	v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+	v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+	v4l2_m2m_buf_done(src_buf, buf_state);
+	v4l2_m2m_buf_done(dst_buf, buf_state);
+	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
+}
+
 static void mtk_jpeg_dec_device_run(void *priv)
 {
 	struct mtk_jpeg_ctx *ctx = priv;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_dec_device_run(void *priv) · line 1157

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -0
@@ -785,6 +1150,11 @@ static void mtk_jpeg_dec_device_run(void *priv)
 	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
 }
 
+static int mtk_jpeg_enc_job_ready(void *priv)
+{
+		return 1;
+}
+

The callback is optional, so can be just removed if it always returns 1.

Quoted 42 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +24 -1
 static int mtk_jpeg_dec_job_ready(void *priv)
 {
 	struct mtk_jpeg_ctx *ctx = priv;
@@ -792,6 +1162,11 @@ static int mtk_jpeg_dec_job_ready(void *priv)
 	return (ctx->state == MTK_JPEG_RUNNING) ? 1 : 0;
 }
 
+static const struct v4l2_m2m_ops mtk_jpeg_enc_m2m_ops = {
+	.device_run = mtk_jpeg_enc_device_run,
+	.job_ready  = mtk_jpeg_enc_job_ready,
+};
+
 static const struct v4l2_m2m_ops mtk_jpeg_dec_m2m_ops = {
 	.device_run = mtk_jpeg_dec_device_run,
 	.job_ready  = mtk_jpeg_dec_job_ready,
@@ -830,24 +1205,109 @@ static int mtk_jpeg_dec_queue_init(void *priv, struct vb2_queue *src_vq,
 	return ret;
 }
 
-static void mtk_jpeg_clk_on(struct mtk_jpeg_dev *jpeg)
+static int mtk_jpeg_enc_queue_init(void *priv, struct vb2_queue *src_vq,
+				   struct vb2_queue *dst_vq)
 {
+	struct mtk_jpeg_ctx *ctx = priv;
 	int ret;
 
+	src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+	src_vq->io_modes = VB2_DMABUF | VB2_MMAP;
+	src_vq->drv_priv = ctx;
+	src_vq->buf_struct_size = sizeof(struct mtk_jpeg_src_buf);
+	src_vq->ops = &mtk_jpeg_enc_qops;
+	src_vq->mem_ops = &vb2_dma_contig_memops;
+	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	src_vq->lock = &ctx->jpeg->lock;
+	src_vq->dev = ctx->jpeg->dev;
+	ret = vb2_queue_init(src_vq);
+	if (ret)
+		return ret;
+
+	dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+	dst_vq->io_modes = VB2_DMABUF | VB2_MMAP;
+	dst_vq->drv_priv = ctx;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_queue_init(void *priv, struct vb2_queue *src_vq, · line 1239

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+	dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
+	dst_vq->ops = &mtk_jpeg_enc_qops;
+	dst_vq->mem_ops = &vb2_dma_contig_memops;
+	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	dst_vq->lock = &ctx->jpeg->lock;
+	dst_vq->dev = ctx->jpeg->dev;
+	ret = vb2_queue_init(dst_vq);
+
+	return ret;
+}

This only differs in "ops" from the decoder implementation. Perhaps both functions could be merged?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_queue_init(void *priv, struct vb2_queue *src_vq, · line 1250

Quoted 13 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +8 -2
+
+static void mtk_jpeg_clk_on(struct mtk_jpeg_dev *jpeg)
+{
+	int ret, i;
+
 	ret = mtk_smi_larb_get(jpeg->larb);
 	if (ret)
 		dev_err(jpeg->dev, "mtk_smi_larb_get larbvdec fail %d\n", ret);
-	clk_prepare_enable(jpeg->clk_jdec_smi);
-	clk_prepare_enable(jpeg->clk_jdec);
+
+	for (i = 0; i < jpeg->variant->num_clocks; i++) {
+		ret = clk_prepare_enable(jpeg->clocks[i]);

Instead of an open coded loop, could the clk_bulk_*() helpers be used instead? (Look for devm_clk_bulk_get() and devm_clk_bulk_prepare_enable().)

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_queue_init(void *priv, struct vb2_queue *src_vq, · line 1253

Quoted 3 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -0
+		if (ret) {
+			while (--i >= 0)
+				clk_disable_unprepare(jpeg->clocks[i]);

nit: The typical convention is to do error handling in an error path on the bottom of the function.

Also, it would be nice to print an error message.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_queue_init(void *priv, struct vb2_queue *src_vq, · line 1263

Quoted 12 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -2
+		}
+	}
 }
 
 static void mtk_jpeg_clk_off(struct mtk_jpeg_dev *jpeg)
 {
-	clk_disable_unprepare(jpeg->clk_jdec);
-	clk_disable_unprepare(jpeg->clk_jdec_smi);
+	int i;
+
+	for (i = jpeg->variant->num_clocks - 1; i >= 0; i--)
+		clk_disable_unprepare(jpeg->clocks[i]);

Ditto.

Quoted 14 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +11 -0
 	mtk_smi_larb_put(jpeg->larb);
 }
 
+static irqreturn_t mtk_jpeg_enc_irq(int irq, void *priv)
+{
+	struct mtk_jpeg_dev *jpeg = priv;
+	struct mtk_jpeg_ctx *ctx;
+	struct vb2_v4l2_buffer *src_buf, *dst_buf;
+	struct mtk_jpeg_src_buf *jpeg_src_buf;
+	enum vb2_buffer_state buf_state = VB2_BUF_STATE_ERROR;
+	u32 enc_irq_ret;
+	u32 enc_ret, result_size;
+
+	ctx = v4l2_m2m_get_curr_priv(jpeg->m2m_dev);

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_queue_init(void *priv, struct vb2_queue *src_vq, · line 1287

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+	if (!ctx) {
+		v4l2_err(&jpeg->v4l2_dev, "Context is NULL\n");
+		return IRQ_HANDLED;
+	}
+
+	src_buf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+	dst_buf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+	jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(&src_buf->vb2_buf);
+
+	enc_ret = mtk_jpeg_enc_get_and_clear_int_status(jpeg->reg_base);

We should check the interrupt status at the beginning of the function and return IRQ_NONE if there wasn't any flag set.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_queue_init(void *priv, struct vb2_queue *src_vq, · line 1296

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +9 -0
+	enc_irq_ret = mtk_jpeg_enc_enum_result(jpeg->reg_base, enc_ret);
+
+	if (enc_irq_ret >= MTK_JPEG_ENC_RESULT_STALL)
+		mtk_jpeg_enc_reset(jpeg->reg_base);
+
+	if (enc_irq_ret != MTK_JPEG_ENC_RESULT_DONE) {
+		dev_err(jpeg->dev, "encode failed\n");
+		goto enc_end;
+	}

As I suggested before, it would have been much more clear if the interrupt status bits were just directly read and checked in this function, without introducing the additional abstraction.

Quoted 18 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +14 -0
+
+	result_size = mtk_jpeg_enc_get_file_size(jpeg->reg_base);
+	vb2_set_plane_payload(&dst_buf->vb2_buf, 0, result_size);
+
+	buf_state = VB2_BUF_STATE_DONE;
+
+enc_end:
+	v4l2_m2m_buf_done(src_buf, buf_state);
+	v4l2_m2m_buf_done(dst_buf, buf_state);
+	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
+	pm_runtime_put(ctx->jpeg->dev);
+	return IRQ_HANDLED;
+}
+
 static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv)
 {
 	struct mtk_jpeg_dev *jpeg = priv;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1361

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
@@ -893,36 +1353,130 @@ static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv)
 	return IRQ_HANDLED;
 }
 
+static void mtk_jpeg_set_enc_default_params(struct mtk_jpeg_ctx *ctx)
+{
+	struct mtk_jpeg_q_data *q = &ctx->out_q;
+	struct v4l2_pix_format_mplane *pix_mp;
+
+	pix_mp = kmalloc(sizeof(*pix_mp), GFP_KERNEL);

Do we need to allocate this pix_mp? Could we instead just embed it inside ctx?

Also, this is actually a memory leak, because I don't see this structure saved anywhere or freed.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1367

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
+
+	ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+	ctx->colorspace = V4L2_COLORSPACE_JPEG,
+	ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
+	ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;

Since we already have a v4l2_pix_format_mplane struct which has fields for the above 4 values, could we just store them there?

Also, I don't see this driver handling the colorspaces in any way, but it seems to allow changing them from the userspace. This is incorrect, because the userspace has no way to know that the colorspace is not handled. Instead, the try_fmt implementation should always override the userspace-provided colorspace configuration with the ones that the driver assumes.

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +9 -0
+	pix_mp->width = MTK_JPEG_MIN_WIDTH;
+	pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+
+	q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_YUYV,
+				      MTK_JPEG_FMT_FLAG_ENC_OUTPUT);
+	vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+				    fmt.pix_mp), q->fmt);
+	q->w = pix_mp->width;
+	q->h = pix_mp->height;
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1380

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -0
+	q->crop_rect.width = pix_mp->width;
+	q->crop_rect.height = pix_mp->height;
+	q->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
+	q->bytesperline[0] = pix_mp->plane_fmt[0].bytesperline;

Actually, do we need this custom mtk_jpeg_q_data struct? Why couldn't we just keep the same values inside the standard v4l2_pix_format_mplane struct?

I think that we need mtk_jpeg_q_data struct.If we delete it, how can we know these values(w, h, sizeimage, bytesperline, mtk_jpeg_fmt) belong to output or capture(output and capture's sizeimages are different, width and height are differnt too for jpeg dec )?We have s_fmt_vid_out_mplane/cap_mplane function to set these values.

But we can use standard v4l2_pix_format_mplane struct replacing the w, h bytesperline, sizeimage in mtk_jpeg_q_data struct like this:

struct mtk_jpeg_q_data{
	struct mtk_jpeg_fmt *fmt;
	struct v4l2_pix_format_mplane pix_mp;
	struct v4l2_rect enc_crop_rect
}

Then delete ctx->colorspace ctx->ycbcr_enc ctx->quantization ctx->xfer_func, becuase v4l2_pix_format_mplane in q_data has contained them and assign them for out_q and cap_q separately.

WDYT?

Quoted 184 lines

In general it's preferred to use the standard kernel structures as much as possible and only introduce local driver structures for data that can't be stored in generic ones.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1392

Quoted 12 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +12 -0
+
+	q = &ctx->cap_q;
+	q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_JPEG,
+				      MTK_JPEG_FMT_FLAG_ENC_CAPTURE);
+	pix_mp->width = MTK_JPEG_MIN_WIDTH;
+	pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+	vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+				    fmt.pix_mp), q->fmt);
+	q->w = pix_mp->width;
+	q->h = pix_mp->height;
+	q->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
+	q->bytesperline[0] = pix_mp->plane_fmt[0].bytesperline;

Ditto.

Quoted 46 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +23 -12
+}
+
 static void mtk_jpeg_set_dec_default_params(struct mtk_jpeg_ctx *ctx)
 {
 	struct mtk_jpeg_q_data *q = &ctx->out_q;
+	struct v4l2_pix_format_mplane *pix_mp;
 	int i;
 
+	pix_mp = kmalloc(sizeof(*pix_mp), GFP_KERNEL);
+
+	ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
 	ctx->colorspace = V4L2_COLORSPACE_JPEG,
 	ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
 	ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
 	ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;
-
-	q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_JPEG,
-					      MTK_JPEG_FMT_TYPE_OUTPUT);
-	q->w = MTK_JPEG_MIN_WIDTH;
-	q->h = MTK_JPEG_MIN_HEIGHT;
-	q->bytesperline[0] = 0;
-	q->sizeimage[0] = MTK_JPEG_DEFAULT_SIZEIMAGE;
+	pix_mp->width = MTK_JPEG_MIN_WIDTH;
+	pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+
+	q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_JPEG,
+				      MTK_JPEG_FMT_FLAG_DEC_OUTPUT);
+	vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+				    fmt.pix_mp), q->fmt);
+	q->w = pix_mp->width;
+	q->h = pix_mp->height;
+	q->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
+	q->bytesperline[0] = pix_mp->plane_fmt[0].bytesperline;
 
 	q = &ctx->cap_q;
-	q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_YUV420M,
-					      MTK_JPEG_FMT_TYPE_CAPTURE);
-	q->w = MTK_JPEG_MIN_WIDTH;
-	q->h = MTK_JPEG_MIN_HEIGHT;
-
+	q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_YUV420M,
+				      MTK_JPEG_FMT_FLAG_DEC_CAPTURE);
+	pix_mp->width = MTK_JPEG_MIN_WIDTH;
+	pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+	vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+				    fmt.pix_mp), q->fmt);

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1434

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -2
+	q->w = pix_mp->width;
+	q->h = pix_mp->height;
 	for (i = 0; i < q->fmt->colplanes; i++) {
-		u32 stride = q->w * q->fmt->h_sample[i] / 4;
-		u32 h = q->h * q->fmt->v_sample[i] / 4;
+		q->sizeimage[i] = pix_mp->plane_fmt[i].sizeimage;
+		q->bytesperline[i] = pix_mp->plane_fmt[i].bytesperline;
+	}
+}
 

Same comments as for the encoder version.

On top of that, both functions are almost identical and it should be possible to merge them.

Quoted 38 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +35 -2
-		q->bytesperline[i] = stride;
-		q->sizeimage[i] = stride * h;
+static int mtk_jpeg_enc_open(struct file *file)
+{
+	struct mtk_jpeg_dev *jpeg = video_drvdata(file);
+	struct video_device *vfd = video_devdata(file);
+	struct mtk_jpeg_ctx *ctx;
+	int ret = 0;
+
+	ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+	if (!ctx)
+		return -ENOMEM;
+
+	if (mutex_lock_interruptible(&jpeg->lock)) {
+		ret = -ERESTARTSYS;
+		goto free;
+	}
+
+	v4l2_fh_init(&ctx->fh, vfd);
+	file->private_data = &ctx->fh;
+	v4l2_fh_add(&ctx->fh);
+
+	ctx->jpeg = jpeg;
+	ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(jpeg->m2m_dev, ctx,
+					    mtk_jpeg_enc_queue_init);
+	if (IS_ERR(ctx->fh.m2m_ctx)) {
+		ret = PTR_ERR(ctx->fh.m2m_ctx);
+		goto error;
 	}
+
+	ret = mtk_jpeg_enc_ctrls_setup(ctx);
+	if (ret) {
+		v4l2_err(&jpeg->v4l2_dev, "Failed to setup jpeg enc controls\n");
+		goto error;
+	}
+	mtk_jpeg_set_enc_default_params(ctx);
+
+	mutex_unlock(&jpeg->lock);

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1480

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +9 -0
+	return 0;
+
+error:
+	v4l2_fh_del(&ctx->fh);
+	v4l2_fh_exit(&ctx->fh);
+	mutex_unlock(&jpeg->lock);
+free:
+	kfree(ctx);
+	return ret;
 }

It looks like the queue_init argument to v4l2_m2m_ctx_init() and control handling would be the only differences from the decoder version. Perhaps the functions can be merged?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_open(struct file *file) · line 1515

Quoted 12 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
 static int mtk_jpeg_dec_open(struct file *file)
@@ -953,6 +1507,12 @@ static int mtk_jpeg_dec_open(struct file *file)
 		goto error;
 	}
 
+	v4l2_ctrl_handler_init(&ctx->ctrl_hdl, 0);
+	ret = v4l2_ctrl_handler_setup(&ctx->ctrl_hdl);
+	if (ret) {
+		v4l2_err(&jpeg->v4l2_dev, "Failed to setup jpeg dec controls\n");
+		goto error;
+	}

There are no controls for the decoder, so there should be no need to set up a control handler.

Quoted 14 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -0
 	mtk_jpeg_set_dec_default_params(ctx);
 	mutex_unlock(&jpeg->lock);
 	return 0;
@@ -973,6 +1533,7 @@ static int mtk_jpeg_release(struct file *file)
 
 	mutex_lock(&jpeg->lock);
 	v4l2_m2m_ctx_release(ctx->fh.m2m_ctx);
+	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
 	v4l2_fh_del(&ctx->fh);
 	v4l2_fh_exit(&ctx->fh);
 	kfree(ctx);
@@ -980,6 +1541,15 @@ static int mtk_jpeg_release(struct file *file)
 	return 0;
 }

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_release(struct file *file) · line 1552

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +9 -0
 
+static const struct v4l2_file_operations mtk_jpeg_enc_fops = {
+	.owner          = THIS_MODULE,
+	.open           = mtk_jpeg_enc_open,
+	.release        = mtk_jpeg_release,
+	.poll           = v4l2_m2m_fop_poll,
+	.unlocked_ioctl = video_ioctl2,
+	.mmap           = v4l2_m2m_fop_mmap,
+};
+

If we merge the .open() implementation, the same struct could be used for both decoder and encoder.

[snip]

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +0 -0
@@ -1042,8 +1619,12 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 		return jpeg_irq;
 	}
 
Quoted 11 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_probe(struct platform_device *pdev) · line 1627

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -2
-	ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_dec_irq, 0,
-			       pdev->name, jpeg);
+	if (jpeg->variant->is_encoder)
+		ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_enc_irq,
+				       0, pdev->name, jpeg);
+	else
+		ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_dec_irq,
+				       0, pdev->name, jpeg);

Rather than having "is_encoder" in the variant struct, would it make more sense to have "irq_handler" instead? That would avoid the explicit if.

Do you mean to delete "is_encoder"? It is used 8 times in the driver.Should I move them all to the match data?

Best Regards,
Xia Jiang

Quoted 347 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_probe(struct platform_device *pdev) · line 1650

Quoted 13 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -1
 	if (ret) {
 		dev_err(&pdev->dev, "Failed to request jpeg_irq %d (%d)\n",
 			jpeg_irq, ret);
@@ -1063,7 +1644,10 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 		goto err_dev_register;
 	}
 
-	jpeg->m2m_dev = v4l2_m2m_init(&mtk_jpeg_dec_m2m_ops);
+	if (jpeg->variant->is_encoder)
+		jpeg->m2m_dev = v4l2_m2m_init(&mtk_jpeg_enc_m2m_ops);
+	else
+		jpeg->m2m_dev = v4l2_m2m_init(&mtk_jpeg_dec_m2m_ops);

Same here. The variant struct could have a "m2m_ops" pointer.

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +0 -2
 	if (IS_ERR(jpeg->m2m_dev)) {
 		v4l2_err(&jpeg->v4l2_dev, "Failed to init mem2mem device\n");
 		ret = PTR_ERR(jpeg->m2m_dev);
@@ -1076,9 +1660,15 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 		goto err_vfd_jpeg_alloc;
 	}
 	snprintf(jpeg->vdev->name, sizeof(jpeg->vdev->name),
-		 "%s-dec", MTK_JPEG_NAME);
-	jpeg->vdev->fops = &mtk_jpeg_dec_fops;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_probe(struct platform_device *pdev) · line 1671

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +9 -1
-	jpeg->vdev->ioctl_ops = &mtk_jpeg_dec_ioctl_ops;
+		 "%s-%s", MTK_JPEG_NAME,
+		 jpeg->variant->is_encoder ? "enc" : "dec");
+	if (jpeg->variant->is_encoder) {
+		jpeg->vdev->fops = &mtk_jpeg_enc_fops;
+		jpeg->vdev->ioctl_ops = &mtk_jpeg_enc_ioctl_ops;
+	} else {
+		jpeg->vdev->fops = &mtk_jpeg_dec_fops;
+		jpeg->vdev->ioctl_ops = &mtk_jpeg_dec_ioctl_ops;
+	}

Similarly here.

[snip]

Quoted 13 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +1 -0
index 0b59e48495d5..9ec2c3350a16 100644
@@ -3,6 +3,7 @@
  * Copyright (c) 2016 MediaTek Inc.
  * Author: Ming Hsiu Tsai <[email protected]>
  *         Rick Chang <[email protected]>
+ *         Xia Jiang <[email protected]>
  */
 
 #ifndef _MTK_JPEG_CORE_H

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 22

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +2 -0
@@ -16,19 +17,21 @@
 #define MTK_JPEG_NAME		"mtk-jpeg"
 
 #define MTK_JPEG_COMP_MAX		3
+#define MTK_JPEG_MAX_CLOCKS		2
+
 

Duplicate blank line.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 26

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +2 -3
 #define MTK_JPEG_FMT_FLAG_DEC_OUTPUT	BIT(0)
 #define MTK_JPEG_FMT_FLAG_DEC_CAPTURE	BIT(1)
-
-#define MTK_JPEG_FMT_TYPE_OUTPUT	1
-#define MTK_JPEG_FMT_TYPE_CAPTURE	2
+#define MTK_JPEG_FMT_FLAG_ENC_OUTPUT	BIT(2)
+#define MTK_JPEG_FMT_FLAG_ENC_CAPTURE	BIT(3)

Do we need separate bits for decoder and encoder?

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 31

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +2 -2
 #define MTK_JPEG_MIN_WIDTH	32U
 #define MTK_JPEG_MIN_HEIGHT	32U
-#define MTK_JPEG_MAX_WIDTH	8192U
-#define MTK_JPEG_MAX_HEIGHT	8192U
+#define MTK_JPEG_MAX_WIDTH	65535U
+#define MTK_JPEG_MAX_HEIGHT	65535U

If this is a change valid for the decoder too, it should be a separate patch.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 34

Quoted 3 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +1 -0
 #define MTK_JPEG_DEFAULT_SIZEIMAGE	(1 * 1024 * 1024)
+#define MTK_JPEG_MAX_EXIF_SIZE	(64 * 1024)

There is one thing that I realized now. If the EXIF mode is enabled, the driver needs to ensure that the buffer is big enough to hold the EXIF data. The vb2 .buf_prepare callback would be the right place to do that.

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +0 -0
 /**
  * enum mtk_jpeg_ctx_state - states of the context state machine
@@ -42,6 +45,18 @@ enum mtk_jpeg_ctx_state {
 	MTK_JPEG_SOURCE_CHANGE,
 };
 

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h enum mtk_jpeg_ctx_state { · line 57

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +10 -0
+/**
+ * mtk_jpeg_variant - mtk jpeg driver variant
+ * @is_encoder:		driver mode is jpeg encoder
+ * @clk_names:		clock names
+ * @num_clocks:		numbers of clock
+ */
+struct mtk_jpeg_variant {
+	bool is_encoder;
+	const char		*clk_names[MTK_JPEG_MAX_CLOCKS];
+	int			num_clocks;

hint: Please avoid tabs between types and names, as it makes it difficult to add new fields later (the number of tabs might need to change for all members).

[snip]

Quoted 28 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +22 -0
new file mode 100644
index 000000000000..7fc1de920a75
@@ -0,0 +1,193 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ * Author: Xia Jiang <[email protected]>
+ *
+ */
+
+#include <linux/io.h>
+#include <linux/kernel.h>
+#include <media/videobuf2-core.h>
+
+#include "mtk_jpeg_enc_hw.h"
+
+static const struct mtk_jpeg_enc_qlt mtk_jpeg_enc_quality[] = {
+	{.quality_param = 34, .hardware_value = JPEG_ENC_QUALITY_Q34},
+	{.quality_param = 39, .hardware_value = JPEG_ENC_QUALITY_Q39},
+	{.quality_param = 48, .hardware_value = JPEG_ENC_QUALITY_Q48},
+	{.quality_param = 60, .hardware_value = JPEG_ENC_QUALITY_Q60},
+	{.quality_param = 64, .hardware_value = JPEG_ENC_QUALITY_Q64},
+	{.quality_param = 68, .hardware_value = JPEG_ENC_QUALITY_Q68},
+	{.quality_param = 74, .hardware_value = JPEG_ENC_QUALITY_Q74},
+	{.quality_param = 80, .hardware_value = JPEG_ENC_QUALITY_Q80},

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 32

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +10 -0
+	{.quality_param = 82, .hardware_value = JPEG_ENC_QUALITY_Q82},
+	{.quality_param = 84, .hardware_value = JPEG_ENC_QUALITY_Q84},
+	{.quality_param = 87, .hardware_value = JPEG_ENC_QUALITY_Q87},
+	{.quality_param = 90, .hardware_value = JPEG_ENC_QUALITY_Q90},
+	{.quality_param = 92, .hardware_value = JPEG_ENC_QUALITY_Q92},
+	{.quality_param = 95, .hardware_value = JPEG_ENC_QUALITY_Q95},
+	{.quality_param = 97, .hardware_value = JPEG_ENC_QUALITY_Q97},
+};
+
+void mtk_jpeg_enc_reset(void __iomem *base)

I'd suggest passing struct mtk_jpeg_dev pointer to all these functions, in case more data about the hardware is needed in the future.

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 34

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +2 -0
+{
+	writel(0x00, base + JPEG_ENC_RSTB);

nit: Just 0 is enough.

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 36

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +2 -0
+	writel(JPEG_ENC_RESET_BIT, base + JPEG_ENC_RSTB);
+	writel(0x00, base + JPEG_ENC_CODEC_SEL);

Ditto.

Quoted 19 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +19 -0
+}
+
+u32 mtk_jpeg_enc_get_and_clear_int_status(void __iomem *base)
+{
+	u32 ret;
+
+	ret = readl(base + JPEG_ENC_INT_STS) &
+		    JPEG_ENC_INT_STATUS_MASK_ALLIRQ;
+	if (ret)
+		writel(0, base + JPEG_ENC_INT_STS);
+
+	return ret;
+}
+
+u32 mtk_jpeg_enc_get_file_size(void __iomem *base)
+{
+	return readl(base + JPEG_ENC_DMA_ADDR0) -
+	       readl(base + JPEG_ENC_DST_ADDR0);
+}

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 65

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +10 -0
+
+u32 mtk_jpeg_enc_enum_result(void __iomem *base, u32 irq_status)
+{
+	if (irq_status & JPEG_ENC_INT_STATUS_DONE)
+		return MTK_JPEG_ENC_RESULT_DONE;
+	else if (irq_status & JPEG_ENC_INT_STATUS_STALL)
+		return MTK_JPEG_ENC_RESULT_STALL;
+	else
+		return MTK_JPEG_ENC_RESULT_VCODEC_IRQ;
+}

It looks like the driver only cares about 2 cases: DONE and anything else. Actually it also cares about a case where no bits are set in irq_status, which could be an interrupt controller misconfiguration or an interrupt generated by another device on the same shared IRQ line, which should be handled by returning IRQ_NONE. Since interrupt handling is a low level hardware operation, I'd suggest another design:
- put the irq_handler_t function here in this file and have it directly access the hardware registers and check the interrupt bits,
- add an mtk_jpeg_enc_done(..., enum vb2_buffer_state state, size_t payload), which would be called from this low level interrupt handler and do the V4L2 buffer and M2M job handling.

WDYT?

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 67

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +2 -0
+
+void mtk_jpeg_enc_set_img_size(void __iomem *base, u32 width, u32 height)

If we pass struct mtk_jpeg_dev to this function, the width and height arguments wouldn't be necessary, as they could be directly retrieved from the driver state.

Similar advice applies to the other functions in this file.

Quoted 22 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +22 -0
+{
+	u32 value;
+
+	value = width << 16 | height;
+	writel(value, base + JPEG_ENC_IMG_SIZE);
+}
+
+void mtk_jpeg_enc_set_blk_num(void __iomem *base, u32 enc_format, u32 width,
+			      u32 height)
+{
+	u32 blk_num;
+	u32 is_420;
+	u32 padding_width;
+	u32 padding_height;
+	u32 luma_blocks;
+	u32 chroma_blocks;
+
+	is_420 = (enc_format == V4L2_PIX_FMT_NV12M ||
+		  enc_format == V4L2_PIX_FMT_NV21M) ? 1 : 0;
+	padding_width = round_up(width, 16);
+	padding_height = round_up(height, is_420 ? 16 : 8);
+

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 99

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +10 -0
+	luma_blocks = padding_width / 8 * padding_height / 8;
+	if (is_420)
+		chroma_blocks = luma_blocks / 4;
+	else
+		chroma_blocks = luma_blocks / 2;
+
+	blk_num = luma_blocks + 2 * chroma_blocks - 1;
+
+	writel(blk_num, base + JPEG_ENC_BLK_NUM);
+}

My comments for this function from v7 haven't been addressed.

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +10 -0
+
+void mtk_jpeg_enc_set_stride(void __iomem *base, u32 enc_format, u32 width,
+			     u32 height, u32 bytesperline)
+{
+	u32 img_stride;
+	u32 mem_stride;
+
+	if (enc_format == V4L2_PIX_FMT_NV12M ||
+	    enc_format == V4L2_PIX_FMT_NV21M) {
+		img_stride = round_up(width, 16);

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 119

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +10 -0
+		mem_stride = bytesperline;
+	} else {
+		img_stride = round_up(width * 2, 32);
+		mem_stride = img_stride;
+	}
+
+	writel(img_stride, base + JPEG_ENC_IMG_STRIDE);
+	writel(mem_stride, base + JPEG_ENC_STRIDE);
+}
+

My comments for this function from v7 haven't been addressed.

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 126

Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +7 -0
+void mtk_jpeg_enc_set_src_addr(void __iomem *base, u32 src_addr,
+			       u32 plane_index)
+{
+	if (!plane_index)
+		writel(src_addr, base + JPEG_ENC_SRC_LUMA_ADDR);
+	else
+		writel(src_addr, base + JPEG_ENC_SRC_CHROMA_ADDR);

Do we need this plane_index if we only have 2 planes? Also, for interleaved formats, like YUYV, what should the LUMA and CHROMA registers be set to?

Quoted 16 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +16 -0
+}
+
+void mtk_jpeg_enc_set_dst_addr(void __iomem *base, u32 dst_addr,
+			       u32 stall_size, u32 init_offset,
+			       u32 offset_mask)
+{
+	writel(init_offset & ~0xf, base + JPEG_ENC_OFFSET_ADDR);
+	writel(offset_mask & 0xf, base + JPEG_ENC_BYTE_OFFSET_MASK);
+	writel(dst_addr & ~0xf, base + JPEG_ENC_DST_ADDR0);
+	writel((dst_addr + stall_size) & ~0xf, base + JPEG_ENC_STALL_ADDR0);
+}
+
+static void mtk_jpeg_enc_set_quality(void __iomem *base, u32 quality)
+{
+	u32 value;
+	u32 i, enc_quality;

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 152

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +10 -0
+
+	enc_quality = mtk_jpeg_enc_quality[0].hardware_value;
+	for (i = 0; i < ARRAY_SIZE(mtk_jpeg_enc_quality); i++) {
+		if (quality <= mtk_jpeg_enc_quality[i].quality_param) {
+			enc_quality = mtk_jpeg_enc_quality[i].hardware_value;
+			break;
+		}
+	}
+
+	value = readl(base + JPEG_ENC_QUALITY);

nit: Is it still necessary to read the register here? Typically the reserved bits would be defined as SBZ (should be zero) and it should be safe to just write 0 to them.

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 162

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +10 -0
+	value = (value & JPEG_ENC_QUALITY_MASK) | enc_quality;
+	writel(value, base + JPEG_ENC_QUALITY);
+}
+
+static void mtk_jpeg_enc_set_ctrl(void __iomem *base, u32 enc_format,
+				  bool exif_en, u32 restart_interval)
+{
+	u32 value;
+
+	value = readl(base + JPEG_ENC_CTRL);

nit: Is it still necessary to read the register here?

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 175

Quoted 13 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +13 -0
+	value &= ~JPEG_ENC_CTRL_YUV_FORMAT_MASK;
+	value |= (enc_format & 3) << 3;
+	if (exif_en)
+		value |= JPEG_ENC_CTRL_FILE_FORMAT_BIT;
+	else
+		value &= ~JPEG_ENC_CTRL_FILE_FORMAT_BIT;
+	if (restart_interval)
+		value |= JPEG_ENC_CTRL_RESTART_EN_BIT;
+	else
+		value &= ~JPEG_ENC_CTRL_RESTART_EN_BIT;
+	writel(value, base + JPEG_ENC_CTRL);
+}
+

Best regards,
Tomasz

#36 lore raw

Re: [PATCH RESEND v9 18/18] media: platform: Add jpeg enc feature

Hi Xia,

Quoted 4 lines · Xia Jiang
Quoted 3 lines · Tomasz Figa

Hi Xia,

On Thu, Jun 04, 2020 at 05:05:53PM +0800, Xia Jiang wrote:

[snip]

Quoted 66 lines
Quoted 30 lines
Quoted 30 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +30 -0
+static void mtk_jpeg_enc_device_run(void *priv)
+{
+	struct mtk_jpeg_ctx *ctx = priv;
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+	struct vb2_v4l2_buffer *src_buf, *dst_buf;
+	enum vb2_buffer_state buf_state = VB2_BUF_STATE_ERROR;
+	unsigned long flags;
+	struct mtk_jpeg_src_buf *jpeg_src_buf;
+	struct mtk_jpeg_enc_bs enc_bs;
+	int ret;
+
+	src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
+	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
+	jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(&src_buf->vb2_buf);
+
+	ret = pm_runtime_get_sync(jpeg->dev);
+	if (ret < 0)
+		goto enc_end;
+
+	spin_lock_irqsave(&jpeg->hw_lock, flags);
+
+	/*
+	 * Resetting the hardware every frame is to ensure that all the
+	 * registers are cleared. This is a hardware requirement.
+	 */
+	mtk_jpeg_enc_reset(jpeg->reg_base);
+
+	mtk_jpeg_set_enc_dst(ctx, jpeg->reg_base, &dst_buf->vb2_buf, &enc_bs);
+	mtk_jpeg_set_enc_src(ctx, jpeg->reg_base, &src_buf->vb2_buf);
+	mtk_jpeg_enc_set_config(jpeg->reg_base, ctx->out_q.fmt->hw_format,
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx, · line 1092

Quoted 3 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -0
+				ctx->enable_exif, ctx->enc_quality,
+				ctx->restart_interval);
+	mtk_jpeg_enc_start(jpeg->reg_base);

Could we just move the above 5 functions into one function inside mtk_jpeg_enc_hw.c that takes mtk_jpeg_dev pointer as its argument, let's say mtk_jpeg_enc_hw_run() and simply program all the data to the registers directly, without the extra level of abstractions?

I can move the 5 functions into one function(mtk_jpeg_enc_hw_run()), but this function will be very long, because it contains computation code such as setting dst addr, blk_num, quality.
In v4, you have adviced the following architecture:
How about the following model, as used by many other drivers:

mtk_jpeg_enc_set_src()

{
        // Set any registers related to source format and buffer
}

mtk_jpeg_enc_set_dst()

{
        // Set any registers related to destination format and buffer
}

mtk_jpeg_enc_set_params()

{
        // Set any registers related to additional encoding parameters
}

mtk_jpeg_enc_device_run(enc, ctx)

{
        mtk_jpeg_enc_set_src(enc, src_buf, src_fmt);
        mtk_jpeg_enc_set_dst(enc, dst_buf, dst_fmt);
        mtk_jpeg_enc_set_params(enc, ctx);
        // Trigger the hardware run
}
Quoted 4 lines

I think that this architecture is more clear(mtk_jpeg_enc_set_config is equivalent to mtk_jpeg_enc_set_params).
Should I keep the original architecture or move 5 functions into mtk_jpeg_enc_hw_run?

Sounds good to me.

My biggest issue with the code that it ends up introducing one more level of abstraction, but with the approach you suggested, the arguments just accept standard structs, which avoids that problem.

[snip]

Quoted 47 lines
Quoted 26 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1367

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
+
+	ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+	ctx->colorspace = V4L2_COLORSPACE_JPEG,
+	ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
+	ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;

Since we already have a v4l2_pix_format_mplane struct which has fields for the above 4 values, could we just store them there?

Also, I don't see this driver handling the colorspaces in any way, but it seems to allow changing them from the userspace. This is incorrect, because the userspace has no way to know that the colorspace is not handled. Instead, the try_fmt implementation should always override the userspace-provided colorspace configuration with the ones that the driver assumes.

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +9 -0
+	pix_mp->width = MTK_JPEG_MIN_WIDTH;
+	pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+
+	q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_YUYV,
+				      MTK_JPEG_FMT_FLAG_ENC_OUTPUT);
+	vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+				    fmt.pix_mp), q->fmt);
+	q->w = pix_mp->width;
+	q->h = pix_mp->height;
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1380

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -0
+	q->crop_rect.width = pix_mp->width;
+	q->crop_rect.height = pix_mp->height;
+	q->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
+	q->bytesperline[0] = pix_mp->plane_fmt[0].bytesperline;

Actually, do we need this custom mtk_jpeg_q_data struct? Why couldn't we just keep the same values inside the standard v4l2_pix_format_mplane struct?

I think that we need mtk_jpeg_q_data struct.If we delete it, how can we know these values(w, h, sizeimage, bytesperline, mtk_jpeg_fmt) belong to output or capture(output and capture's sizeimages are different, width and height are differnt too for jpeg dec )?We have s_fmt_vid_out_mplane/cap_mplane function to set these values.

But we can use standard v4l2_pix_format_mplane struct replacing the w, h bytesperline, sizeimage in mtk_jpeg_q_data struct like this:

struct mtk_jpeg_q_data{
	struct mtk_jpeg_fmt *fmt;
	struct v4l2_pix_format_mplane pix_mp;
	struct v4l2_rect enc_crop_rect
}
Quoted 5 lines

Then delete ctx->colorspace ctx->ycbcr_enc ctx->quantization ctx->xfer_func, becuase v4l2_pix_format_mplane in q_data has contained them and assign them for out_q and cap_q separately.

WDYT?

Sounds good to me. I was considering just making it like

struct mtk_jpeg_ctx {
	struct mtk_jpeg_fmt *src_fmt;
	struct v4l2_pix_format_mplane src_pix_mp;
	struct v4l2_rect src_crop;

	struct mtk_jpeg_fmt *dst_fmt;
	struct v4l2_pix_format_mplane dst_pix_mp;
	struct v4l2_rect dst_crop;
};

but I like your suggestion as well, as long as custom data structures are not used to store standard information.
[snip]

Quoted 15 lines
Quoted 4 lines
Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +0 -0
@@ -1042,8 +1619,12 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 		return jpeg_irq;
 	}
 
Quoted 11 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_probe(struct platform_device *pdev) · line 1627

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -2
-	ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_dec_irq, 0,
-			       pdev->name, jpeg);
+	if (jpeg->variant->is_encoder)
+		ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_enc_irq,
+				       0, pdev->name, jpeg);
+	else
+		ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_dec_irq,
+				       0, pdev->name, jpeg);

Rather than having "is_encoder" in the variant struct, would it make more sense to have "irq_handler" instead? That would avoid the explicit if.

Quoted 2 lines

Do you mean to delete "is_encoder"? It is used 8 times in the driver.Should I move them all to the match data?

Yes. It would make the code linear and the varability between the decoder and encoder would be self-contained in the variant struct.

Best regards,
Tomasz

author
#37 lore raw

Re: [PATCH RESEND v9 18/18] media: platform: Add jpeg enc feature

Quoted 79 lines · Tomasz Figa

Hi Xia,

Quoted 4 lines · Xia Jiang
Quoted 3 lines · Tomasz Figa

Hi Xia,

On Thu, Jun 04, 2020 at 05:05:53PM +0800, Xia Jiang wrote:

[snip]

Quoted 66 lines
Quoted 30 lines
Quoted 30 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +30 -0
+static void mtk_jpeg_enc_device_run(void *priv)
+{
+	struct mtk_jpeg_ctx *ctx = priv;
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+	struct vb2_v4l2_buffer *src_buf, *dst_buf;
+	enum vb2_buffer_state buf_state = VB2_BUF_STATE_ERROR;
+	unsigned long flags;
+	struct mtk_jpeg_src_buf *jpeg_src_buf;
+	struct mtk_jpeg_enc_bs enc_bs;
+	int ret;
+
+	src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
+	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
+	jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(&src_buf->vb2_buf);
+
+	ret = pm_runtime_get_sync(jpeg->dev);
+	if (ret < 0)
+		goto enc_end;
+
+	spin_lock_irqsave(&jpeg->hw_lock, flags);
+
+	/*
+	 * Resetting the hardware every frame is to ensure that all the
+	 * registers are cleared. This is a hardware requirement.
+	 */
+	mtk_jpeg_enc_reset(jpeg->reg_base);
+
+	mtk_jpeg_set_enc_dst(ctx, jpeg->reg_base, &dst_buf->vb2_buf, &enc_bs);
+	mtk_jpeg_set_enc_src(ctx, jpeg->reg_base, &src_buf->vb2_buf);
+	mtk_jpeg_enc_set_config(jpeg->reg_base, ctx->out_q.fmt->hw_format,
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx, · line 1092

Quoted 3 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -0
+				ctx->enable_exif, ctx->enc_quality,
+				ctx->restart_interval);
+	mtk_jpeg_enc_start(jpeg->reg_base);

Could we just move the above 5 functions into one function inside mtk_jpeg_enc_hw.c that takes mtk_jpeg_dev pointer as its argument, let's say mtk_jpeg_enc_hw_run() and simply program all the data to the registers directly, without the extra level of abstractions?

I can move the 5 functions into one function(mtk_jpeg_enc_hw_run()), but this function will be very long, because it contains computation code such as setting dst addr, blk_num, quality.
In v4, you have adviced the following architecture:
How about the following model, as used by many other drivers:

mtk_jpeg_enc_set_src()

{
        // Set any registers related to source format and buffer
}

mtk_jpeg_enc_set_dst()

{
        // Set any registers related to destination format and buffer
}

mtk_jpeg_enc_set_params()

{
        // Set any registers related to additional encoding parameters
}

mtk_jpeg_enc_device_run(enc, ctx)

{
        mtk_jpeg_enc_set_src(enc, src_buf, src_fmt);
        mtk_jpeg_enc_set_dst(enc, dst_buf, dst_fmt);
        mtk_jpeg_enc_set_params(enc, ctx);
        // Trigger the hardware run
}
Quoted 4 lines

I think that this architecture is more clear(mtk_jpeg_enc_set_config is equivalent to mtk_jpeg_enc_set_params).
Should I keep the original architecture or move 5 functions into mtk_jpeg_enc_hw_run?

Quoted 5 lines

Sounds good to me.

My biggest issue with the code that it ends up introducing one more level of abstraction, but with the approach you suggested, the arguments just accept standard structs, which avoids that problem.

Dear Tomasz,

Sorry for that I didn't understand your final preference.

As you mentioned, using mtk_jpeg_dev pointer as its argument, but some arguments come from mtk_jpeg_ctx pointer, such as ctx->enable_exif/ ctx->enc_quality/ctx->restart_interval. Should we use mtk_jpeg_ctx pointer as its argument? Should we use src_dma_addr/dst_dma_addr as its arguments too? Because that src_dma_addr/dst_dma_addr need to be getted by v4l2 interfaces(
src_buf=v4l2_m2m_next_src_buf(); src_dma_ddr=vb2_dma_contig_plane_dma_addr();).
Using V4L2 interfaces in mtk_jpeg_enc_hw.c doesn't seem reasonable.

solution 1:
mtk_jpeg_enc_hw_run(ctx, src_dma_addr, dst_dma_addr)

{
	//Set all the registers

without one more level of abstraction
}

solution 2:
mtk_jpeg_enc_reset(jpeg)

{
	//set the reset register
}

mtk_jpeg_set_enc_dst(ctx, dst_dma_addr)

{
	
	//Set any registers related to destination format and buffer

without one more level of abstraction
}
mtk_jpeg_set_enc_src(ctx, src_dma_addr)
{

	//Set any registers related to source format and buffer     without one
more level of abstraction
}
mtk_jpeg_enc_set_config(ctx)
{
	// Set any registers related to additional encoding parameters
without one more level of abstraction
}
mtk_jpeg_enc_start(jpeg)
{
	//set the trigger register
}

Solution 1 or Solution 2?

Best Regards,
Xia Jiang

Quoted 93 lines

[snip]

Quoted 47 lines
Quoted 26 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1367

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
+
+	ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+	ctx->colorspace = V4L2_COLORSPACE_JPEG,
+	ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
+	ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;

Since we already have a v4l2_pix_format_mplane struct which has fields for the above 4 values, could we just store them there?

Also, I don't see this driver handling the colorspaces in any way, but it seems to allow changing them from the userspace. This is incorrect, because the userspace has no way to know that the colorspace is not handled. Instead, the try_fmt implementation should always override the userspace-provided colorspace configuration with the ones that the driver assumes.

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +9 -0
+	pix_mp->width = MTK_JPEG_MIN_WIDTH;
+	pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+
+	q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_YUYV,
+				      MTK_JPEG_FMT_FLAG_ENC_OUTPUT);
+	vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+				    fmt.pix_mp), q->fmt);
+	q->w = pix_mp->width;
+	q->h = pix_mp->height;
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1380

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -0
+	q->crop_rect.width = pix_mp->width;
+	q->crop_rect.height = pix_mp->height;
+	q->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
+	q->bytesperline[0] = pix_mp->plane_fmt[0].bytesperline;

Actually, do we need this custom mtk_jpeg_q_data struct? Why couldn't we just keep the same values inside the standard v4l2_pix_format_mplane struct?

I think that we need mtk_jpeg_q_data struct.If we delete it, how can we know these values(w, h, sizeimage, bytesperline, mtk_jpeg_fmt) belong to output or capture(output and capture's sizeimages are different, width and height are differnt too for jpeg dec )?We have s_fmt_vid_out_mplane/cap_mplane function to set these values.

But we can use standard v4l2_pix_format_mplane struct replacing the w, h bytesperline, sizeimage in mtk_jpeg_q_data struct like this:

struct mtk_jpeg_q_data{
	struct mtk_jpeg_fmt *fmt;
	struct v4l2_pix_format_mplane pix_mp;
	struct v4l2_rect enc_crop_rect
}
Quoted 5 lines

Then delete ctx->colorspace ctx->ycbcr_enc ctx->quantization ctx->xfer_func, becuase v4l2_pix_format_mplane in q_data has contained them and assign them for out_q and cap_q separately.

WDYT?

Sounds good to me. I was considering just making it like

struct mtk_jpeg_ctx {
	struct mtk_jpeg_fmt *src_fmt;
	struct v4l2_pix_format_mplane src_pix_mp;
	struct v4l2_rect src_crop;

	struct mtk_jpeg_fmt *dst_fmt;
	struct v4l2_pix_format_mplane dst_pix_mp;
	struct v4l2_rect dst_crop;
};

but I like your suggestion as well, as long as custom data structures are not used to store standard information.
[snip]

Quoted 15 lines
Quoted 4 lines
Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +0 -0
@@ -1042,8 +1619,12 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 		return jpeg_irq;
 	}
 
Quoted 11 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_probe(struct platform_device *pdev) · line 1627

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -2
-	ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_dec_irq, 0,
-			       pdev->name, jpeg);
+	if (jpeg->variant->is_encoder)
+		ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_enc_irq,
+				       0, pdev->name, jpeg);
+	else
+		ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_dec_irq,
+				       0, pdev->name, jpeg);

Rather than having "is_encoder" in the variant struct, would it make more sense to have "irq_handler" instead? That would avoid the explicit if.

Quoted 2 lines

Do you mean to delete "is_encoder"? It is used 8 times in the driver.Should I move them all to the match data?

Yes. It would make the code linear and the varability between the decoder and encoder would be self-contained in the variant struct.

Best regards,
Tomasz

#38 lore raw

Re: [PATCH RESEND v9 18/18] media: platform: Add jpeg enc feature

Quoted 135 lines · Xia Jiang
Quoted 79 lines · Tomasz Figa

Hi Xia,

Quoted 4 lines · Xia Jiang
Quoted 3 lines · Tomasz Figa

Hi Xia,

On Thu, Jun 04, 2020 at 05:05:53PM +0800, Xia Jiang wrote:

[snip]

Quoted 66 lines
Quoted 30 lines
Quoted 30 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +30 -0
+static void mtk_jpeg_enc_device_run(void *priv)
+{
+       struct mtk_jpeg_ctx *ctx = priv;
+       struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+       struct vb2_v4l2_buffer *src_buf, *dst_buf;
+       enum vb2_buffer_state buf_state = VB2_BUF_STATE_ERROR;
+       unsigned long flags;
+       struct mtk_jpeg_src_buf *jpeg_src_buf;
+       struct mtk_jpeg_enc_bs enc_bs;
+       int ret;
+
+       src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
+       dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
+       jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(&src_buf->vb2_buf);
+
+       ret = pm_runtime_get_sync(jpeg->dev);
+       if (ret < 0)
+               goto enc_end;
+
+       spin_lock_irqsave(&jpeg->hw_lock, flags);
+
+       /*
+        * Resetting the hardware every frame is to ensure that all the
+        * registers are cleared. This is a hardware requirement.
+        */
+       mtk_jpeg_enc_reset(jpeg->reg_base);
+
+       mtk_jpeg_set_enc_dst(ctx, jpeg->reg_base, &dst_buf->vb2_buf, &enc_bs);
+       mtk_jpeg_set_enc_src(ctx, jpeg->reg_base, &src_buf->vb2_buf);
+       mtk_jpeg_enc_set_config(jpeg->reg_base, ctx->out_q.fmt->hw_format,
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx, · line 1092

Quoted 3 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -0
+                               ctx->enable_exif, ctx->enc_quality,
+                               ctx->restart_interval);
+       mtk_jpeg_enc_start(jpeg->reg_base);

Could we just move the above 5 functions into one function inside mtk_jpeg_enc_hw.c that takes mtk_jpeg_dev pointer as its argument, let's say mtk_jpeg_enc_hw_run() and simply program all the data to the registers directly, without the extra level of abstractions?

I can move the 5 functions into one function(mtk_jpeg_enc_hw_run()), but this function will be very long, because it contains computation code such as setting dst addr, blk_num, quality.
In v4, you have adviced the following architecture:
How about the following model, as used by many other drivers:

mtk_jpeg_enc_set_src()

{
        // Set any registers related to source format and buffer
}

mtk_jpeg_enc_set_dst()

{
        // Set any registers related to destination format and buffer
}

mtk_jpeg_enc_set_params()

{
        // Set any registers related to additional encoding parameters
}

mtk_jpeg_enc_device_run(enc, ctx)

{
        mtk_jpeg_enc_set_src(enc, src_buf, src_fmt);
        mtk_jpeg_enc_set_dst(enc, dst_buf, dst_fmt);
        mtk_jpeg_enc_set_params(enc, ctx);
        // Trigger the hardware run
}
Quoted 4 lines

I think that this architecture is more clear(mtk_jpeg_enc_set_config is equivalent to mtk_jpeg_enc_set_params).
Should I keep the original architecture or move 5 functions into mtk_jpeg_enc_hw_run?

Quoted 5 lines

Sounds good to me.

My biggest issue with the code that it ends up introducing one more level of abstraction, but with the approach you suggested, the arguments just accept standard structs, which avoids that problem.

Dear Tomasz,

Sorry for that I didn't understand your final preference.

As you mentioned, using mtk_jpeg_dev pointer as its argument, but some arguments come from mtk_jpeg_ctx pointer, such as ctx->enable_exif/ ctx->enc_quality/ctx->restart_interval. Should we use mtk_jpeg_ctx pointer as its argument? Should we use src_dma_addr/dst_dma_addr as its arguments too? Because that src_dma_addr/dst_dma_addr need to be getted by v4l2 interfaces(
src_buf=v4l2_m2m_next_src_buf(); src_dma_ddr=vb2_dma_contig_plane_dma_addr();).
Using V4L2 interfaces in mtk_jpeg_enc_hw.c doesn't seem reasonable.

solution 1:
mtk_jpeg_enc_hw_run(ctx, src_dma_addr, dst_dma_addr)

{
        //Set all the registers

without one more level of abstraction
}

solution 2:
mtk_jpeg_enc_reset(jpeg)

{
        //set the reset register
}

mtk_jpeg_set_enc_dst(ctx, dst_dma_addr)
{

        //Set any registers related to destination format and buffer

without one more level of abstraction
}
mtk_jpeg_set_enc_src(ctx, src_dma_addr)
{

        //Set any registers related to source format and buffer     without one
more level of abstraction
}
mtk_jpeg_enc_set_config(ctx)
{
        // Set any registers related to additional encoding parameters
without one more level of abstraction
}
mtk_jpeg_enc_start(jpeg)
{
        //set the trigger register
}
Quoted 1 line

Solution 1 or Solution 2?

I like your previous proposal the most. Let me quote it below:

mtk_jpeg_enc_set_src()

{
        // Set any registers related to source format and buffer
}

mtk_jpeg_enc_set_dst()

{
        // Set any registers related to destination format and buffer
}

mtk_jpeg_enc_set_params()

{
        // Set any registers related to additional encoding parameters
}

mtk_jpeg_enc_device_run(enc, ctx)

{
        mtk_jpeg_enc_set_src(enc, src_buf, src_fmt);
        mtk_jpeg_enc_set_dst(enc, dst_buf, dst_fmt);
        mtk_jpeg_enc_set_params(enc, ctx);
        // Trigger the hardware run
}

That is assuming src/dst_buf would be vb2_buffer and src/dst_fmt v4l2_pix_format_mplane. Does it make sense?

Best regards,
Tomasz

Quoted 97 lines

Best Regards,
Xia Jiang

Quoted 93 lines

[snip]

Quoted 47 lines
Quoted 31 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1367

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
+
+       ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+       ctx->colorspace = V4L2_COLORSPACE_JPEG,
+       ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+       ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
+       ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;

Since we already have a v4l2_pix_format_mplane struct which has fields for the above 4 values, could we just store them there?

Also, I don't see this driver handling the colorspaces in any way, but it seems to allow changing them from the userspace. This is incorrect, because the userspace has no way to know that the colorspace is not handled. Instead, the try_fmt implementation should always override the userspace-provided colorspace configuration with the ones that the driver assumes.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1380

Quoted 13 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +13 -0
+       pix_mp->width = MTK_JPEG_MIN_WIDTH;
+       pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+
+       q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_YUYV,
+                                     MTK_JPEG_FMT_FLAG_ENC_OUTPUT);
+       vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+                                   fmt.pix_mp), q->fmt);
+       q->w = pix_mp->width;
+       q->h = pix_mp->height;
+       q->crop_rect.width = pix_mp->width;
+       q->crop_rect.height = pix_mp->height;
+       q->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
+       q->bytesperline[0] = pix_mp->plane_fmt[0].bytesperline;
Quoted 3 lines

Actually, do we need this custom mtk_jpeg_q_data struct? Why couldn't we just keep the same values inside the standard v4l2_pix_format_mplane struct?

I think that we need mtk_jpeg_q_data struct.If we delete it, how can we know these values(w, h, sizeimage, bytesperline, mtk_jpeg_fmt) belong to output or capture(output and capture's sizeimages are different, width and height are differnt too for jpeg dec )?We have s_fmt_vid_out_mplane/cap_mplane function to set these values.

But we can use standard v4l2_pix_format_mplane struct replacing the w, h bytesperline, sizeimage in mtk_jpeg_q_data struct like this:

struct mtk_jpeg_q_data{
    struct mtk_jpeg_fmt *fmt;
    struct v4l2_pix_format_mplane pix_mp;
    struct v4l2_rect enc_crop_rect
}
Quoted 5 lines

Then delete ctx->colorspace ctx->ycbcr_enc ctx->quantization ctx->xfer_func, becuase v4l2_pix_format_mplane in q_data has contained them and assign them for out_q and cap_q separately.

WDYT?

Sounds good to me. I was considering just making it like

struct mtk_jpeg_ctx {
      struct mtk_jpeg_fmt *src_fmt;
      struct v4l2_pix_format_mplane src_pix_mp;
      struct v4l2_rect src_crop;

      struct mtk_jpeg_fmt *dst_fmt;
      struct v4l2_pix_format_mplane dst_pix_mp;
      struct v4l2_rect dst_crop;
};

but I like your suggestion as well, as long as custom data structures are not used to store standard information.
[snip]

Quoted 15 lines
Quoted 4 lines
Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +0 -0
@@ -1042,8 +1619,12 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
                return jpeg_irq;
        }
 
Quoted 11 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_probe(struct platform_device *pdev) · line 1627

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -2
-       ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_dec_irq, 0,
-                              pdev->name, jpeg);
+       if (jpeg->variant->is_encoder)
+               ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_enc_irq,
+                                      0, pdev->name, jpeg);
+       else
+               ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_dec_irq,
+                                      0, pdev->name, jpeg);

Rather than having "is_encoder" in the variant struct, would it make more sense to have "irq_handler" instead? That would avoid the explicit if.

Quoted 2 lines

Do you mean to delete "is_encoder"? It is used 8 times in the driver.Should I move them all to the match data?

Yes. It would make the code linear and the varability between the decoder and encoder would be self-contained in the variant struct.

Best regards,
Tomasz

author
#39 lore raw

Re: [PATCH RESEND v9 18/18] media: platform: Add jpeg enc feature

In reply to Tomasz Figa
Quoted 151 lines · Tomasz Figa

Hi Xia,

Quoted 4 lines · Xia Jiang
Quoted 3 lines · Tomasz Figa

Hi Xia,

On Thu, Jun 04, 2020 at 05:05:53PM +0800, Xia Jiang wrote:

[snip]

Quoted 66 lines
Quoted 30 lines
Quoted 30 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +30 -0
+static void mtk_jpeg_enc_device_run(void *priv)
+{
+	struct mtk_jpeg_ctx *ctx = priv;
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+	struct vb2_v4l2_buffer *src_buf, *dst_buf;
+	enum vb2_buffer_state buf_state = VB2_BUF_STATE_ERROR;
+	unsigned long flags;
+	struct mtk_jpeg_src_buf *jpeg_src_buf;
+	struct mtk_jpeg_enc_bs enc_bs;
+	int ret;
+
+	src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
+	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
+	jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(&src_buf->vb2_buf);
+
+	ret = pm_runtime_get_sync(jpeg->dev);
+	if (ret < 0)
+		goto enc_end;
+
+	spin_lock_irqsave(&jpeg->hw_lock, flags);
+
+	/*
+	 * Resetting the hardware every frame is to ensure that all the
+	 * registers are cleared. This is a hardware requirement.
+	 */
+	mtk_jpeg_enc_reset(jpeg->reg_base);
+
+	mtk_jpeg_set_enc_dst(ctx, jpeg->reg_base, &dst_buf->vb2_buf, &enc_bs);
+	mtk_jpeg_set_enc_src(ctx, jpeg->reg_base, &src_buf->vb2_buf);
+	mtk_jpeg_enc_set_config(jpeg->reg_base, ctx->out_q.fmt->hw_format,
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx, · line 1092

Quoted 3 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -0
+				ctx->enable_exif, ctx->enc_quality,
+				ctx->restart_interval);
+	mtk_jpeg_enc_start(jpeg->reg_base);

Could we just move the above 5 functions into one function inside mtk_jpeg_enc_hw.c that takes mtk_jpeg_dev pointer as its argument, let's say mtk_jpeg_enc_hw_run() and simply program all the data to the registers directly, without the extra level of abstractions?

I can move the 5 functions into one function(mtk_jpeg_enc_hw_run()), but this function will be very long, because it contains computation code such as setting dst addr, blk_num, quality.
In v4, you have adviced the following architecture:
How about the following model, as used by many other drivers:

mtk_jpeg_enc_set_src()

{
        // Set any registers related to source format and buffer
}

mtk_jpeg_enc_set_dst()

{
        // Set any registers related to destination format and buffer
}

mtk_jpeg_enc_set_params()

{
        // Set any registers related to additional encoding parameters
}

mtk_jpeg_enc_device_run(enc, ctx)

{
        mtk_jpeg_enc_set_src(enc, src_buf, src_fmt);
        mtk_jpeg_enc_set_dst(enc, dst_buf, dst_fmt);
        mtk_jpeg_enc_set_params(enc, ctx);
        // Trigger the hardware run
}
Quoted 4 lines

I think that this architecture is more clear(mtk_jpeg_enc_set_config is equivalent to mtk_jpeg_enc_set_params).
Should I keep the original architecture or move 5 functions into mtk_jpeg_enc_hw_run?

Sounds good to me.

My biggest issue with the code that it ends up introducing one more level of abstraction, but with the approach you suggested, the arguments just accept standard structs, which avoids that problem.

[snip]

Quoted 47 lines
Quoted 26 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1367

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
+
+	ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+	ctx->colorspace = V4L2_COLORSPACE_JPEG,
+	ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
+	ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;

Since we already have a v4l2_pix_format_mplane struct which has fields for the above 4 values, could we just store them there?

Also, I don't see this driver handling the colorspaces in any way, but it seems to allow changing them from the userspace. This is incorrect, because the userspace has no way to know that the colorspace is not handled. Instead, the try_fmt implementation should always override the userspace-provided colorspace configuration with the ones that the driver assumes.

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +9 -0
+	pix_mp->width = MTK_JPEG_MIN_WIDTH;
+	pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+
+	q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_YUYV,
+				      MTK_JPEG_FMT_FLAG_ENC_OUTPUT);
+	vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+				    fmt.pix_mp), q->fmt);
+	q->w = pix_mp->width;
+	q->h = pix_mp->height;
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1380

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -0
+	q->crop_rect.width = pix_mp->width;
+	q->crop_rect.height = pix_mp->height;
+	q->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
+	q->bytesperline[0] = pix_mp->plane_fmt[0].bytesperline;

Actually, do we need this custom mtk_jpeg_q_data struct? Why couldn't we just keep the same values inside the standard v4l2_pix_format_mplane struct?

I think that we need mtk_jpeg_q_data struct.If we delete it, how can we know these values(w, h, sizeimage, bytesperline, mtk_jpeg_fmt) belong to output or capture(output and capture's sizeimages are different, width and height are differnt too for jpeg dec )?We have s_fmt_vid_out_mplane/cap_mplane function to set these values.

But we can use standard v4l2_pix_format_mplane struct replacing the w, h bytesperline, sizeimage in mtk_jpeg_q_data struct like this:

struct mtk_jpeg_q_data{
	struct mtk_jpeg_fmt *fmt;
	struct v4l2_pix_format_mplane pix_mp;
	struct v4l2_rect enc_crop_rect
}
Quoted 5 lines

Then delete ctx->colorspace ctx->ycbcr_enc ctx->quantization ctx->xfer_func, becuase v4l2_pix_format_mplane in q_data has contained them and assign them for out_q and cap_q separately.

WDYT?

Sounds good to me. I was considering just making it like

struct mtk_jpeg_ctx {
	struct mtk_jpeg_fmt *src_fmt;
	struct v4l2_pix_format_mplane src_pix_mp;
	struct v4l2_rect src_crop;

	struct mtk_jpeg_fmt *dst_fmt;
	struct v4l2_pix_format_mplane dst_pix_mp;
	struct v4l2_rect dst_crop;
};
Quoted 2 lines

but I like your suggestion as well, as long as custom data structures are not used to store standard information.

Dear Tomasz,

I used the structure like below:

 struct mtk_jpeg_q_data{
	struct mtk_jpeg_fmt *fmt;
	struct v4l2_pix_format_mplane pix_mp;
	struct v4l2_rect enc_crop_rect
}

Then delete ctx->colorspace ctx->ycbcr_enc ctx->quantization ctx->xfer_func, becuase v4l2_pix_format_mplane in q_data has contained them and assign them for out_q and cap_q separately.

Then the v4l2_compliance test will fail, the fail log as below:
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
fail: v4l2-test-formats.cpp(836):
fmt_cap.g_colorspace() != col
test VIDIOC_S_FMT: FAIL
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)

The source code of v4l2-test-formats.cpp as below:

static int testM2MFormats(struct node *node)

{
	cv4l_fmt fmt_out;
	cv4l_fmt fmt;
	cv4l_fmt fmt_cap;
	__u32 cap_type = node->g_type();
	__u32 out_type = v4l_type_invert(cap_type);
	__u32 col, ycbcr_enc, quant, xfer_func;

	fail_on_test(node->g_fmt(fmt_out, out_type));
	node->g_fmt(fmt_cap, cap_type);
	fail_on_test(fmt_cap.g_colorspace() != fmt_out.g_colorspace());
	fail_on_test(fmt_cap.g_ycbcr_enc() != fmt_out.g_ycbcr_enc());
	fail_on_test(fmt_cap.g_quantization() != fmt_out.g_quantization());
	fail_on_test(fmt_cap.g_xfer_func() != fmt_out.g_xfer_func());
	col = fmt_out.g_colorspace() == V4L2_COLORSPACE_SMPTE170M ?
		V4L2_COLORSPACE_REC709 : V4L2_COLORSPACE_SMPTE170M;
	ycbcr_enc = fmt_out.g_ycbcr_enc() == V4L2_YCBCR_ENC_601 ?
		V4L2_YCBCR_ENC_709 : V4L2_YCBCR_ENC_601;
	quant = fmt_out.g_quantization() == V4L2_QUANTIZATION_LIM_RANGE ?
		V4L2_QUANTIZATION_FULL_RANGE : V4L2_QUANTIZATION_LIM_RANGE;
	xfer_func = fmt_out.g_xfer_func() == V4L2_XFER_FUNC_SRGB ?
		V4L2_XFER_FUNC_709 : V4L2_XFER_FUNC_SRGB;
	fmt_out.s_colorspace(col);
	fmt_out.s_xfer_func(xfer_func);
	fmt_out.s_ycbcr_enc(ycbcr_enc);
	fmt_out.s_quantization(quant);
	node->s_fmt(fmt_out);
	fail_on_test(fmt_out.g_colorspace() != col);
	fail_on_test(fmt_out.g_xfer_func() != xfer_func);
	fail_on_test(fmt_out.g_ycbcr_enc() != ycbcr_enc);
	fail_on_test(fmt_out.g_quantization() != quant);
	node->g_fmt(fmt_cap);
	fail_on_test(fmt_cap.g_colorspace() != col);  // line 836
	fail_on_test(fmt_cap.g_xfer_func() != xfer_func);
	fail_on_test(fmt_cap.g_ycbcr_enc() != ycbcr_enc);
	fail_on_test(fmt_cap.g_quantization() != quant);
}

It needs that cap's colorspace equals out's colorspace when userspace just set out's colorspace and then get cap's colorspace. However, cap's colorspace which getted from driver equals V4L2_COLORSPACE_JPEG, because the code in g_fmt() like this:
pix_mp->colorspace = q_data->pix_mp.colorspace; pix_mp->ycbcr_enc = q_data->pix_mp.ycbcr_enc; pix_mp->xfer_func = q_data->pix_mp.xfer_func; pix_mp->quantization = q_data->pix_mp.quantization;

How should I handle this case? Should I store them(colorspace, sfer_func, ycbcr_enc, quatization) in ctx as the orinal desin? Then I can get them from g_fmt() like this:
pix_mp->colorspace = ctx->colorspace;
pix_mp->ycbcr_enc = ctx->ycbcr_enc;
pix_mp->xfer_func = ctx->xfer_func;
pix_mp->quantization = ctx->quantization;

Best Regards,
Xia Jiang

Quoted 24 lines

[snip]

Quoted 15 lines
Quoted 4 lines
Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +0 -0
@@ -1042,8 +1619,12 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 		return jpeg_irq;
 	}
 
Quoted 11 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_probe(struct platform_device *pdev) · line 1627

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -2
-	ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_dec_irq, 0,
-			       pdev->name, jpeg);
+	if (jpeg->variant->is_encoder)
+		ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_enc_irq,
+				       0, pdev->name, jpeg);
+	else
+		ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_dec_irq,
+				       0, pdev->name, jpeg);

Rather than having "is_encoder" in the variant struct, would it make more sense to have "irq_handler" instead? That would avoid the explicit if.

Quoted 2 lines

Do you mean to delete "is_encoder"? It is used 8 times in the driver.Should I move them all to the match data?

Yes. It would make the code linear and the varability between the decoder and encoder would be self-contained in the variant struct.

Best regards,
Tomasz

#40 lore raw

Re: [PATCH RESEND v9 18/18] media: platform: Add jpeg enc feature

Quoted 233 lines · Xia Jiang
Quoted 151 lines · Tomasz Figa

Hi Xia,

Quoted 4 lines · Xia Jiang
Quoted 3 lines · Tomasz Figa

Hi Xia,

On Thu, Jun 04, 2020 at 05:05:53PM +0800, Xia Jiang wrote:

[snip]

Quoted 66 lines
Quoted 30 lines
Quoted 30 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +30 -0
+static void mtk_jpeg_enc_device_run(void *priv)
+{
+       struct mtk_jpeg_ctx *ctx = priv;
+       struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+       struct vb2_v4l2_buffer *src_buf, *dst_buf;
+       enum vb2_buffer_state buf_state = VB2_BUF_STATE_ERROR;
+       unsigned long flags;
+       struct mtk_jpeg_src_buf *jpeg_src_buf;
+       struct mtk_jpeg_enc_bs enc_bs;
+       int ret;
+
+       src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
+       dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
+       jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(&src_buf->vb2_buf);
+
+       ret = pm_runtime_get_sync(jpeg->dev);
+       if (ret < 0)
+               goto enc_end;
+
+       spin_lock_irqsave(&jpeg->hw_lock, flags);
+
+       /*
+        * Resetting the hardware every frame is to ensure that all the
+        * registers are cleared. This is a hardware requirement.
+        */
+       mtk_jpeg_enc_reset(jpeg->reg_base);
+
+       mtk_jpeg_set_enc_dst(ctx, jpeg->reg_base, &dst_buf->vb2_buf, &enc_bs);
+       mtk_jpeg_set_enc_src(ctx, jpeg->reg_base, &src_buf->vb2_buf);
+       mtk_jpeg_enc_set_config(jpeg->reg_base, ctx->out_q.fmt->hw_format,
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx, · line 1092

Quoted 3 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -0
+                               ctx->enable_exif, ctx->enc_quality,
+                               ctx->restart_interval);
+       mtk_jpeg_enc_start(jpeg->reg_base);

Could we just move the above 5 functions into one function inside mtk_jpeg_enc_hw.c that takes mtk_jpeg_dev pointer as its argument, let's say mtk_jpeg_enc_hw_run() and simply program all the data to the registers directly, without the extra level of abstractions?

I can move the 5 functions into one function(mtk_jpeg_enc_hw_run()), but this function will be very long, because it contains computation code such as setting dst addr, blk_num, quality.
In v4, you have adviced the following architecture:
How about the following model, as used by many other drivers:

mtk_jpeg_enc_set_src()

{
        // Set any registers related to source format and buffer
}

mtk_jpeg_enc_set_dst()

{
        // Set any registers related to destination format and buffer
}

mtk_jpeg_enc_set_params()

{
        // Set any registers related to additional encoding parameters
}

mtk_jpeg_enc_device_run(enc, ctx)

{
        mtk_jpeg_enc_set_src(enc, src_buf, src_fmt);
        mtk_jpeg_enc_set_dst(enc, dst_buf, dst_fmt);
        mtk_jpeg_enc_set_params(enc, ctx);
        // Trigger the hardware run
}
Quoted 4 lines

I think that this architecture is more clear(mtk_jpeg_enc_set_config is equivalent to mtk_jpeg_enc_set_params).
Should I keep the original architecture or move 5 functions into mtk_jpeg_enc_hw_run?

Sounds good to me.

My biggest issue with the code that it ends up introducing one more level of abstraction, but with the approach you suggested, the arguments just accept standard structs, which avoids that problem.

[snip]

Quoted 47 lines
Quoted 31 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1367

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
+
+       ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+       ctx->colorspace = V4L2_COLORSPACE_JPEG,
+       ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+       ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
+       ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;

Since we already have a v4l2_pix_format_mplane struct which has fields for the above 4 values, could we just store them there?

Also, I don't see this driver handling the colorspaces in any way, but it seems to allow changing them from the userspace. This is incorrect, because the userspace has no way to know that the colorspace is not handled. Instead, the try_fmt implementation should always override the userspace-provided colorspace configuration with the ones that the driver assumes.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1380

Quoted 13 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +13 -0
+       pix_mp->width = MTK_JPEG_MIN_WIDTH;
+       pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+
+       q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_YUYV,
+                                     MTK_JPEG_FMT_FLAG_ENC_OUTPUT);
+       vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+                                   fmt.pix_mp), q->fmt);
+       q->w = pix_mp->width;
+       q->h = pix_mp->height;
+       q->crop_rect.width = pix_mp->width;
+       q->crop_rect.height = pix_mp->height;
+       q->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
+       q->bytesperline[0] = pix_mp->plane_fmt[0].bytesperline;
Quoted 3 lines

Actually, do we need this custom mtk_jpeg_q_data struct? Why couldn't we just keep the same values inside the standard v4l2_pix_format_mplane struct?

I think that we need mtk_jpeg_q_data struct.If we delete it, how can we know these values(w, h, sizeimage, bytesperline, mtk_jpeg_fmt) belong to output or capture(output and capture's sizeimages are different, width and height are differnt too for jpeg dec )?We have s_fmt_vid_out_mplane/cap_mplane function to set these values.

But we can use standard v4l2_pix_format_mplane struct replacing the w, h bytesperline, sizeimage in mtk_jpeg_q_data struct like this:

struct mtk_jpeg_q_data{
    struct mtk_jpeg_fmt *fmt;
    struct v4l2_pix_format_mplane pix_mp;
    struct v4l2_rect enc_crop_rect
}
Quoted 5 lines

Then delete ctx->colorspace ctx->ycbcr_enc ctx->quantization ctx->xfer_func, becuase v4l2_pix_format_mplane in q_data has contained them and assign them for out_q and cap_q separately.

WDYT?

Sounds good to me. I was considering just making it like

struct mtk_jpeg_ctx {
      struct mtk_jpeg_fmt *src_fmt;
      struct v4l2_pix_format_mplane src_pix_mp;
      struct v4l2_rect src_crop;

      struct mtk_jpeg_fmt *dst_fmt;
      struct v4l2_pix_format_mplane dst_pix_mp;
      struct v4l2_rect dst_crop;
};
Quoted 2 lines

but I like your suggestion as well, as long as custom data structures are not used to store standard information.

Dear Tomasz,

I used the structure like below:

 struct mtk_jpeg_q_data{
        struct mtk_jpeg_fmt *fmt;
        struct v4l2_pix_format_mplane pix_mp;
        struct v4l2_rect enc_crop_rect
}

Then delete ctx->colorspace ctx->ycbcr_enc ctx->quantization ctx->xfer_func, becuase v4l2_pix_format_mplane in q_data has contained them and assign them for out_q and cap_q separately.

Then the v4l2_compliance test will fail, the fail log as below:
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
fail: v4l2-test-formats.cpp(836):
fmt_cap.g_colorspace() != col
test VIDIOC_S_FMT: FAIL
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)

The source code of v4l2-test-formats.cpp as below:

static int testM2MFormats(struct node *node)

{
        cv4l_fmt fmt_out;
        cv4l_fmt fmt;
        cv4l_fmt fmt_cap;
        __u32 cap_type = node->g_type();
        __u32 out_type = v4l_type_invert(cap_type);
        __u32 col, ycbcr_enc, quant, xfer_func;

        fail_on_test(node->g_fmt(fmt_out, out_type));
        node->g_fmt(fmt_cap, cap_type);
        fail_on_test(fmt_cap.g_colorspace() != fmt_out.g_colorspace());
        fail_on_test(fmt_cap.g_ycbcr_enc() != fmt_out.g_ycbcr_enc());
        fail_on_test(fmt_cap.g_quantization() != fmt_out.g_quantization());
        fail_on_test(fmt_cap.g_xfer_func() != fmt_out.g_xfer_func());
        col = fmt_out.g_colorspace() == V4L2_COLORSPACE_SMPTE170M ?
                V4L2_COLORSPACE_REC709 : V4L2_COLORSPACE_SMPTE170M;
        ycbcr_enc = fmt_out.g_ycbcr_enc() == V4L2_YCBCR_ENC_601 ?
                V4L2_YCBCR_ENC_709 : V4L2_YCBCR_ENC_601;
        quant = fmt_out.g_quantization() == V4L2_QUANTIZATION_LIM_RANGE ?
                V4L2_QUANTIZATION_FULL_RANGE : V4L2_QUANTIZATION_LIM_RANGE;
        xfer_func = fmt_out.g_xfer_func() == V4L2_XFER_FUNC_SRGB ?
                V4L2_XFER_FUNC_709 : V4L2_XFER_FUNC_SRGB;
        fmt_out.s_colorspace(col);
        fmt_out.s_xfer_func(xfer_func);
        fmt_out.s_ycbcr_enc(ycbcr_enc);
        fmt_out.s_quantization(quant);
        node->s_fmt(fmt_out);
        fail_on_test(fmt_out.g_colorspace() != col);
        fail_on_test(fmt_out.g_xfer_func() != xfer_func);
        fail_on_test(fmt_out.g_ycbcr_enc() != ycbcr_enc);
        fail_on_test(fmt_out.g_quantization() != quant);
        node->g_fmt(fmt_cap);
        fail_on_test(fmt_cap.g_colorspace() != col);  // line 836
        fail_on_test(fmt_cap.g_xfer_func() != xfer_func);
        fail_on_test(fmt_cap.g_ycbcr_enc() != ycbcr_enc);
        fail_on_test(fmt_cap.g_quantization() != quant);
}

It needs that cap's colorspace equals out's colorspace when userspace just set out's colorspace and then get cap's colorspace. However, cap's colorspace which getted from driver equals V4L2_COLORSPACE_JPEG, because the code in g_fmt() like this:
pix_mp->colorspace = q_data->pix_mp.colorspace; pix_mp->ycbcr_enc = q_data->pix_mp.ycbcr_enc; pix_mp->xfer_func = q_data->pix_mp.xfer_func; pix_mp->quantization = q_data->pix_mp.quantization;

Quoted 7 lines

How should I handle this case? Should I store them(colorspace, sfer_func, ycbcr_enc, quatization) in ctx as the orinal desin? Then I can get them from g_fmt() like this:
pix_mp->colorspace = ctx->colorspace;
pix_mp->ycbcr_enc = ctx->ycbcr_enc;
pix_mp->xfer_func = ctx->xfer_func;
pix_mp->quantization = ctx->quantization;

Why would there be any other colorspace accepted? I suppose that the hardware only supports the JPEG color space, so it shouldn't accept any other colorspace in TRY_FMT (and thus S_FMT) anyway.

Still, for correctness, I would suggest propagating the colorspace (and related) information from OUTPUT format to CAPTURE format in S_FMT(OUTPUT).

Best regards,
Tomasz

Quoted 28 lines

Best Regards,
Xia Jiang

Quoted 24 lines

[snip]

Quoted 15 lines
Quoted 4 lines
Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +0 -0
@@ -1042,8 +1619,12 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
                return jpeg_irq;
        }
 
Quoted 11 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_probe(struct platform_device *pdev) · line 1627

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -2
-       ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_dec_irq, 0,
-                              pdev->name, jpeg);
+       if (jpeg->variant->is_encoder)
+               ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_enc_irq,
+                                      0, pdev->name, jpeg);
+       else
+               ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_dec_irq,
+                                      0, pdev->name, jpeg);

Rather than having "is_encoder" in the variant struct, would it make more sense to have "irq_handler" instead? That would avoid the explicit if.

Quoted 2 lines

Do you mean to delete "is_encoder"? It is used 8 times in the driver.Should I move them all to the match data?

Yes. It would make the code linear and the varability between the decoder and encoder would be self-contained in the variant struct.

Best regards,
Tomasz

author
#41 lore raw

Re: [PATCH RESEND v9 18/18] media: platform: Add jpeg enc feature

Quoted 242 lines · Tomasz Figa
Quoted 233 lines · Xia Jiang
Quoted 151 lines · Tomasz Figa

Hi Xia,

Quoted 4 lines · Xia Jiang
Quoted 3 lines · Tomasz Figa

Hi Xia,

On Thu, Jun 04, 2020 at 05:05:53PM +0800, Xia Jiang wrote:

[snip]

Quoted 66 lines
Quoted 30 lines
Quoted 30 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +30 -0
+static void mtk_jpeg_enc_device_run(void *priv)
+{
+       struct mtk_jpeg_ctx *ctx = priv;
+       struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+       struct vb2_v4l2_buffer *src_buf, *dst_buf;
+       enum vb2_buffer_state buf_state = VB2_BUF_STATE_ERROR;
+       unsigned long flags;
+       struct mtk_jpeg_src_buf *jpeg_src_buf;
+       struct mtk_jpeg_enc_bs enc_bs;
+       int ret;
+
+       src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
+       dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
+       jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(&src_buf->vb2_buf);
+
+       ret = pm_runtime_get_sync(jpeg->dev);
+       if (ret < 0)
+               goto enc_end;
+
+       spin_lock_irqsave(&jpeg->hw_lock, flags);
+
+       /*
+        * Resetting the hardware every frame is to ensure that all the
+        * registers are cleared. This is a hardware requirement.
+        */
+       mtk_jpeg_enc_reset(jpeg->reg_base);
+
+       mtk_jpeg_set_enc_dst(ctx, jpeg->reg_base, &dst_buf->vb2_buf, &enc_bs);
+       mtk_jpeg_set_enc_src(ctx, jpeg->reg_base, &src_buf->vb2_buf);
+       mtk_jpeg_enc_set_config(jpeg->reg_base, ctx->out_q.fmt->hw_format,
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx, · line 1092

Quoted 3 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -0
+                               ctx->enable_exif, ctx->enc_quality,
+                               ctx->restart_interval);
+       mtk_jpeg_enc_start(jpeg->reg_base);

Could we just move the above 5 functions into one function inside mtk_jpeg_enc_hw.c that takes mtk_jpeg_dev pointer as its argument, let's say mtk_jpeg_enc_hw_run() and simply program all the data to the registers directly, without the extra level of abstractions?

I can move the 5 functions into one function(mtk_jpeg_enc_hw_run()), but this function will be very long, because it contains computation code such as setting dst addr, blk_num, quality.
In v4, you have adviced the following architecture:
How about the following model, as used by many other drivers:

mtk_jpeg_enc_set_src()

{
        // Set any registers related to source format and buffer
}

mtk_jpeg_enc_set_dst()

{
        // Set any registers related to destination format and buffer
}

mtk_jpeg_enc_set_params()

{
        // Set any registers related to additional encoding parameters
}

mtk_jpeg_enc_device_run(enc, ctx)

{
        mtk_jpeg_enc_set_src(enc, src_buf, src_fmt);
        mtk_jpeg_enc_set_dst(enc, dst_buf, dst_fmt);
        mtk_jpeg_enc_set_params(enc, ctx);
        // Trigger the hardware run
}
Quoted 4 lines

I think that this architecture is more clear(mtk_jpeg_enc_set_config is equivalent to mtk_jpeg_enc_set_params).
Should I keep the original architecture or move 5 functions into mtk_jpeg_enc_hw_run?

Sounds good to me.

My biggest issue with the code that it ends up introducing one more level of abstraction, but with the approach you suggested, the arguments just accept standard structs, which avoids that problem.

[snip]

Quoted 47 lines
Quoted 31 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1367

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
+
+       ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+       ctx->colorspace = V4L2_COLORSPACE_JPEG,
+       ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+       ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
+       ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;

Since we already have a v4l2_pix_format_mplane struct which has fields for the above 4 values, could we just store them there?

Also, I don't see this driver handling the colorspaces in any way, but it seems to allow changing them from the userspace. This is incorrect, because the userspace has no way to know that the colorspace is not handled. Instead, the try_fmt implementation should always override the userspace-provided colorspace configuration with the ones that the driver assumes.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1380

Quoted 13 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +13 -0
+       pix_mp->width = MTK_JPEG_MIN_WIDTH;
+       pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+
+       q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_YUYV,
+                                     MTK_JPEG_FMT_FLAG_ENC_OUTPUT);
+       vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+                                   fmt.pix_mp), q->fmt);
+       q->w = pix_mp->width;
+       q->h = pix_mp->height;
+       q->crop_rect.width = pix_mp->width;
+       q->crop_rect.height = pix_mp->height;
+       q->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
+       q->bytesperline[0] = pix_mp->plane_fmt[0].bytesperline;
Quoted 3 lines

Actually, do we need this custom mtk_jpeg_q_data struct? Why couldn't we just keep the same values inside the standard v4l2_pix_format_mplane struct?

I think that we need mtk_jpeg_q_data struct.If we delete it, how can we know these values(w, h, sizeimage, bytesperline, mtk_jpeg_fmt) belong to output or capture(output and capture's sizeimages are different, width and height are differnt too for jpeg dec )?We have s_fmt_vid_out_mplane/cap_mplane function to set these values.

But we can use standard v4l2_pix_format_mplane struct replacing the w, h bytesperline, sizeimage in mtk_jpeg_q_data struct like this:

struct mtk_jpeg_q_data{
    struct mtk_jpeg_fmt *fmt;
    struct v4l2_pix_format_mplane pix_mp;
    struct v4l2_rect enc_crop_rect
}
Quoted 5 lines

Then delete ctx->colorspace ctx->ycbcr_enc ctx->quantization ctx->xfer_func, becuase v4l2_pix_format_mplane in q_data has contained them and assign them for out_q and cap_q separately.

WDYT?

Sounds good to me. I was considering just making it like

struct mtk_jpeg_ctx {
      struct mtk_jpeg_fmt *src_fmt;
      struct v4l2_pix_format_mplane src_pix_mp;
      struct v4l2_rect src_crop;

      struct mtk_jpeg_fmt *dst_fmt;
      struct v4l2_pix_format_mplane dst_pix_mp;
      struct v4l2_rect dst_crop;
};
Quoted 2 lines

but I like your suggestion as well, as long as custom data structures are not used to store standard information.

Dear Tomasz,

I used the structure like below:

 struct mtk_jpeg_q_data{
        struct mtk_jpeg_fmt *fmt;
        struct v4l2_pix_format_mplane pix_mp;
        struct v4l2_rect enc_crop_rect
}

Then delete ctx->colorspace ctx->ycbcr_enc ctx->quantization ctx->xfer_func, becuase v4l2_pix_format_mplane in q_data has contained them and assign them for out_q and cap_q separately.

Then the v4l2_compliance test will fail, the fail log as below:
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
fail: v4l2-test-formats.cpp(836):
fmt_cap.g_colorspace() != col
test VIDIOC_S_FMT: FAIL
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)

The source code of v4l2-test-formats.cpp as below:

static int testM2MFormats(struct node *node)

{
        cv4l_fmt fmt_out;
        cv4l_fmt fmt;
        cv4l_fmt fmt_cap;
        __u32 cap_type = node->g_type();
        __u32 out_type = v4l_type_invert(cap_type);
        __u32 col, ycbcr_enc, quant, xfer_func;

        fail_on_test(node->g_fmt(fmt_out, out_type));
        node->g_fmt(fmt_cap, cap_type);
        fail_on_test(fmt_cap.g_colorspace() != fmt_out.g_colorspace());
        fail_on_test(fmt_cap.g_ycbcr_enc() != fmt_out.g_ycbcr_enc());
        fail_on_test(fmt_cap.g_quantization() != fmt_out.g_quantization());
        fail_on_test(fmt_cap.g_xfer_func() != fmt_out.g_xfer_func());
        col = fmt_out.g_colorspace() == V4L2_COLORSPACE_SMPTE170M ?
                V4L2_COLORSPACE_REC709 : V4L2_COLORSPACE_SMPTE170M;
        ycbcr_enc = fmt_out.g_ycbcr_enc() == V4L2_YCBCR_ENC_601 ?
                V4L2_YCBCR_ENC_709 : V4L2_YCBCR_ENC_601;
        quant = fmt_out.g_quantization() == V4L2_QUANTIZATION_LIM_RANGE ?
                V4L2_QUANTIZATION_FULL_RANGE : V4L2_QUANTIZATION_LIM_RANGE;
        xfer_func = fmt_out.g_xfer_func() == V4L2_XFER_FUNC_SRGB ?
                V4L2_XFER_FUNC_709 : V4L2_XFER_FUNC_SRGB;
        fmt_out.s_colorspace(col);
        fmt_out.s_xfer_func(xfer_func);
        fmt_out.s_ycbcr_enc(ycbcr_enc);
        fmt_out.s_quantization(quant);
        node->s_fmt(fmt_out);
        fail_on_test(fmt_out.g_colorspace() != col);
        fail_on_test(fmt_out.g_xfer_func() != xfer_func);
        fail_on_test(fmt_out.g_ycbcr_enc() != ycbcr_enc);
        fail_on_test(fmt_out.g_quantization() != quant);
        node->g_fmt(fmt_cap);
        fail_on_test(fmt_cap.g_colorspace() != col);  // line 836
        fail_on_test(fmt_cap.g_xfer_func() != xfer_func);
        fail_on_test(fmt_cap.g_ycbcr_enc() != ycbcr_enc);
        fail_on_test(fmt_cap.g_quantization() != quant);
}

It needs that cap's colorspace equals out's colorspace when userspace just set out's colorspace and then get cap's colorspace. However, cap's colorspace which getted from driver equals V4L2_COLORSPACE_JPEG, because the code in g_fmt() like this:
pix_mp->colorspace = q_data->pix_mp.colorspace; pix_mp->ycbcr_enc = q_data->pix_mp.ycbcr_enc; pix_mp->xfer_func = q_data->pix_mp.xfer_func; pix_mp->quantization = q_data->pix_mp.quantization;

Quoted 7 lines

How should I handle this case? Should I store them(colorspace, sfer_func, ycbcr_enc, quatization) in ctx as the orinal desin? Then I can get them from g_fmt() like this:
pix_mp->colorspace = ctx->colorspace;
pix_mp->ycbcr_enc = ctx->ycbcr_enc;
pix_mp->xfer_func = ctx->xfer_func;
pix_mp->quantization = ctx->quantization;

Quoted 7 lines

Why would there be any other colorspace accepted? I suppose that the hardware only supports the JPEG color space, so it shouldn't accept any other colorspace in TRY_FMT (and thus S_FMT) anyway.

Still, for correctness, I would suggest propagating the colorspace (and related) information from OUTPUT format to CAPTURE format in S_FMT(OUTPUT).

Dear Tomasz,
If the driver doesn't accept any other colorspace from userspace(means the colorspace always equals V4L2_COLORSPACE_JPEG on my understanding), the v4l2_compliance will fail at line 831:
fail_on_test(fmt_out.g_colorspace() != col); // line 831

The v4l2_compliance needs driver can accept other colorspace and capture colorspace equals output colorspace.

I try to propagate the colorspace from OUTPUT format to CAPTURE foramt in S_FMT, the compliance test succeed.

Should the jpeg driver can accept other colorspace?

Best Regards,
Xia Jiang

Quoted 32 lines

Best regards,
Tomasz

Quoted 28 lines

Best Regards,
Xia Jiang

Quoted 24 lines

[snip]

Quoted 15 lines
Quoted 4 lines
Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +0 -0
@@ -1042,8 +1619,12 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
                return jpeg_irq;
        }
 
Quoted 11 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_probe(struct platform_device *pdev) · line 1627

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -2
-       ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_dec_irq, 0,
-                              pdev->name, jpeg);
+       if (jpeg->variant->is_encoder)
+               ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_enc_irq,
+                                      0, pdev->name, jpeg);
+       else
+               ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_dec_irq,
+                                      0, pdev->name, jpeg);

Rather than having "is_encoder" in the variant struct, would it make more sense to have "irq_handler" instead? That would avoid the explicit if.

Quoted 2 lines

Do you mean to delete "is_encoder"? It is used 8 times in the driver.Should I move them all to the match data?

Yes. It would make the code linear and the varability between the decoder and encoder would be self-contained in the variant struct.

Best regards,
Tomasz

#42 lore raw

Re: [PATCH RESEND v9 18/18] media: platform: Add jpeg enc feature

Quoted 257 lines · Xia Jiang
Quoted 242 lines · Tomasz Figa
Quoted 233 lines · Xia Jiang
Quoted 151 lines · Tomasz Figa

Hi Xia,

Quoted 4 lines · Xia Jiang
Quoted 3 lines · Tomasz Figa

Hi Xia,

On Thu, Jun 04, 2020 at 05:05:53PM +0800, Xia Jiang wrote:

[snip]

Quoted 66 lines
Quoted 30 lines
Quoted 30 lines
(quoted diff) +30 -0
+static void mtk_jpeg_enc_device_run(void *priv)
+{
+       struct mtk_jpeg_ctx *ctx = priv;
+       struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+       struct vb2_v4l2_buffer *src_buf, *dst_buf;
+       enum vb2_buffer_state buf_state = VB2_BUF_STATE_ERROR;
+       unsigned long flags;
+       struct mtk_jpeg_src_buf *jpeg_src_buf;
+       struct mtk_jpeg_enc_bs enc_bs;
+       int ret;
+
+       src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
+       dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
+       jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(&src_buf->vb2_buf);
+
+       ret = pm_runtime_get_sync(jpeg->dev);
+       if (ret < 0)
+               goto enc_end;
+
+       spin_lock_irqsave(&jpeg->hw_lock, flags);
+
+       /*
+        * Resetting the hardware every frame is to ensure that all the
+        * registers are cleared. This is a hardware requirement.
+        */
+       mtk_jpeg_enc_reset(jpeg->reg_base);
+
+       mtk_jpeg_set_enc_dst(ctx, jpeg->reg_base, &dst_buf->vb2_buf, &enc_bs);
+       mtk_jpeg_set_enc_src(ctx, jpeg->reg_base, &src_buf->vb2_buf);
+       mtk_jpeg_enc_set_config(jpeg->reg_base, ctx->out_q.fmt->hw_format,
Quoted 8 lines

Review

Quoted 3 lines
(quoted diff) +3 -0
+                               ctx->enable_exif, ctx->enc_quality,
+                               ctx->restart_interval);
+       mtk_jpeg_enc_start(jpeg->reg_base);

Could we just move the above 5 functions into one function inside mtk_jpeg_enc_hw.c that takes mtk_jpeg_dev pointer as its argument, let's say mtk_jpeg_enc_hw_run() and simply program all the data to the registers directly, without the extra level of abstractions?

I can move the 5 functions into one function(mtk_jpeg_enc_hw_run()), but this function will be very long, because it contains computation code such as setting dst addr, blk_num, quality.
In v4, you have adviced the following architecture:
How about the following model, as used by many other drivers:

mtk_jpeg_enc_set_src()

{
        // Set any registers related to source format and buffer
}

mtk_jpeg_enc_set_dst()

{
        // Set any registers related to destination format and buffer
}

mtk_jpeg_enc_set_params()

{
        // Set any registers related to additional encoding parameters
}

mtk_jpeg_enc_device_run(enc, ctx)

{
        mtk_jpeg_enc_set_src(enc, src_buf, src_fmt);
        mtk_jpeg_enc_set_dst(enc, dst_buf, dst_fmt);
        mtk_jpeg_enc_set_params(enc, ctx);
        // Trigger the hardware run
}
Quoted 4 lines

I think that this architecture is more clear(mtk_jpeg_enc_set_config is equivalent to mtk_jpeg_enc_set_params).
Should I keep the original architecture or move 5 functions into mtk_jpeg_enc_hw_run?

Sounds good to me.

My biggest issue with the code that it ends up introducing one more level of abstraction, but with the approach you suggested, the arguments just accept standard structs, which avoids that problem.

[snip]

Quoted 47 lines
Quoted 31 lines

Review

Quoted 6 lines
(quoted diff) +6 -0
+
+       ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+       ctx->colorspace = V4L2_COLORSPACE_JPEG,
+       ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+       ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
+       ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;

Since we already have a v4l2_pix_format_mplane struct which has fields for the above 4 values, could we just store them there?

Also, I don't see this driver handling the colorspaces in any way, but it seems to allow changing them from the userspace. This is incorrect, because the userspace has no way to know that the colorspace is not handled. Instead, the try_fmt implementation should always override the userspace-provided colorspace configuration with the ones that the driver assumes.

Review

Quoted 13 lines
(quoted diff) +13 -0
+       pix_mp->width = MTK_JPEG_MIN_WIDTH;
+       pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+
+       q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_YUYV,
+                                     MTK_JPEG_FMT_FLAG_ENC_OUTPUT);
+       vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+                                   fmt.pix_mp), q->fmt);
+       q->w = pix_mp->width;
+       q->h = pix_mp->height;
+       q->crop_rect.width = pix_mp->width;
+       q->crop_rect.height = pix_mp->height;
+       q->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
+       q->bytesperline[0] = pix_mp->plane_fmt[0].bytesperline;
Quoted 3 lines

Actually, do we need this custom mtk_jpeg_q_data struct? Why couldn't we just keep the same values inside the standard v4l2_pix_format_mplane struct?

I think that we need mtk_jpeg_q_data struct.If we delete it, how can we know these values(w, h, sizeimage, bytesperline, mtk_jpeg_fmt) belong to output or capture(output and capture's sizeimages are different, width and height are differnt too for jpeg dec )?We have s_fmt_vid_out_mplane/cap_mplane function to set these values.

But we can use standard v4l2_pix_format_mplane struct replacing the w, h bytesperline, sizeimage in mtk_jpeg_q_data struct like this:

struct mtk_jpeg_q_data{
    struct mtk_jpeg_fmt *fmt;
    struct v4l2_pix_format_mplane pix_mp;
    struct v4l2_rect enc_crop_rect
}
Quoted 5 lines

Then delete ctx->colorspace ctx->ycbcr_enc ctx->quantization ctx->xfer_func, becuase v4l2_pix_format_mplane in q_data has contained them and assign them for out_q and cap_q separately.

WDYT?

Sounds good to me. I was considering just making it like

struct mtk_jpeg_ctx {
      struct mtk_jpeg_fmt *src_fmt;
      struct v4l2_pix_format_mplane src_pix_mp;
      struct v4l2_rect src_crop;

      struct mtk_jpeg_fmt *dst_fmt;
      struct v4l2_pix_format_mplane dst_pix_mp;
      struct v4l2_rect dst_crop;
};
Quoted 2 lines

but I like your suggestion as well, as long as custom data structures are not used to store standard information.

Dear Tomasz,

I used the structure like below:

 struct mtk_jpeg_q_data{
        struct mtk_jpeg_fmt *fmt;
        struct v4l2_pix_format_mplane pix_mp;
        struct v4l2_rect enc_crop_rect
}

Then delete ctx->colorspace ctx->ycbcr_enc ctx->quantization ctx->xfer_func, becuase v4l2_pix_format_mplane in q_data has contained them and assign them for out_q and cap_q separately.

Then the v4l2_compliance test will fail, the fail log as below:
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
fail: v4l2-test-formats.cpp(836):
fmt_cap.g_colorspace() != col
test VIDIOC_S_FMT: FAIL
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)

The source code of v4l2-test-formats.cpp as below:

static int testM2MFormats(struct node *node)

{
        cv4l_fmt fmt_out;
        cv4l_fmt fmt;
        cv4l_fmt fmt_cap;
        __u32 cap_type = node->g_type();
        __u32 out_type = v4l_type_invert(cap_type);
        __u32 col, ycbcr_enc, quant, xfer_func;

        fail_on_test(node->g_fmt(fmt_out, out_type));
        node->g_fmt(fmt_cap, cap_type);
        fail_on_test(fmt_cap.g_colorspace() != fmt_out.g_colorspace());
        fail_on_test(fmt_cap.g_ycbcr_enc() != fmt_out.g_ycbcr_enc());
        fail_on_test(fmt_cap.g_quantization() != fmt_out.g_quantization());
        fail_on_test(fmt_cap.g_xfer_func() != fmt_out.g_xfer_func());
        col = fmt_out.g_colorspace() == V4L2_COLORSPACE_SMPTE170M ?
                V4L2_COLORSPACE_REC709 : V4L2_COLORSPACE_SMPTE170M;
        ycbcr_enc = fmt_out.g_ycbcr_enc() == V4L2_YCBCR_ENC_601 ?
                V4L2_YCBCR_ENC_709 : V4L2_YCBCR_ENC_601;
        quant = fmt_out.g_quantization() == V4L2_QUANTIZATION_LIM_RANGE ?
                V4L2_QUANTIZATION_FULL_RANGE : V4L2_QUANTIZATION_LIM_RANGE;
        xfer_func = fmt_out.g_xfer_func() == V4L2_XFER_FUNC_SRGB ?
                V4L2_XFER_FUNC_709 : V4L2_XFER_FUNC_SRGB;
        fmt_out.s_colorspace(col);
        fmt_out.s_xfer_func(xfer_func);
        fmt_out.s_ycbcr_enc(ycbcr_enc);
        fmt_out.s_quantization(quant);
        node->s_fmt(fmt_out);
        fail_on_test(fmt_out.g_colorspace() != col);
        fail_on_test(fmt_out.g_xfer_func() != xfer_func);
        fail_on_test(fmt_out.g_ycbcr_enc() != ycbcr_enc);
        fail_on_test(fmt_out.g_quantization() != quant);
        node->g_fmt(fmt_cap);
        fail_on_test(fmt_cap.g_colorspace() != col);  // line 836
        fail_on_test(fmt_cap.g_xfer_func() != xfer_func);
        fail_on_test(fmt_cap.g_ycbcr_enc() != ycbcr_enc);
        fail_on_test(fmt_cap.g_quantization() != quant);
}

It needs that cap's colorspace equals out's colorspace when userspace just set out's colorspace and then get cap's colorspace. However, cap's colorspace which getted from driver equals V4L2_COLORSPACE_JPEG, because the code in g_fmt() like this:
pix_mp->colorspace = q_data->pix_mp.colorspace; pix_mp->ycbcr_enc = q_data->pix_mp.ycbcr_enc; pix_mp->xfer_func = q_data->pix_mp.xfer_func; pix_mp->quantization = q_data->pix_mp.quantization;

Quoted 7 lines

How should I handle this case? Should I store them(colorspace, sfer_func, ycbcr_enc, quatization) in ctx as the orinal desin? Then I can get them from g_fmt() like this:
pix_mp->colorspace = ctx->colorspace;
pix_mp->ycbcr_enc = ctx->ycbcr_enc;
pix_mp->xfer_func = ctx->xfer_func;
pix_mp->quantization = ctx->quantization;

Quoted 7 lines

Why would there be any other colorspace accepted? I suppose that the hardware only supports the JPEG color space, so it shouldn't accept any other colorspace in TRY_FMT (and thus S_FMT) anyway.

Still, for correctness, I would suggest propagating the colorspace (and related) information from OUTPUT format to CAPTURE format in S_FMT(OUTPUT).

Dear Tomasz,
If the driver doesn't accept any other colorspace from userspace(means the colorspace always equals V4L2_COLORSPACE_JPEG on my understanding), the v4l2_compliance will fail at line 831:
fail_on_test(fmt_out.g_colorspace() != col); // line 831

Quoted 7 lines

The v4l2_compliance needs driver can accept other colorspace and capture colorspace equals output colorspace.

I try to propagate the colorspace from OUTPUT format to CAPTURE foramt in S_FMT, the compliance test succeed.

Should the jpeg driver can accept other colorspace?

Hans, could you advise on what is the proper behavior regarding colorimetry support?

Best regards,
Tomasz

#43 lore raw

Re: [PATCH RESEND v9 18/18] media: platform: Add jpeg enc feature

Quoted 261 lines · Tomasz Figa
Quoted 260 lines · Xia Jiang
Quoted 242 lines · Tomasz Figa
Quoted 233 lines · Xia Jiang
Quoted 151 lines · Tomasz Figa

Hi Xia,

Quoted 4 lines · Xia Jiang
Quoted 3 lines · Tomasz Figa

Hi Xia,

On Thu, Jun 04, 2020 at 05:05:53PM +0800, Xia Jiang wrote:

[snip]

Quoted 66 lines
Quoted 30 lines
Quoted 30 lines
(quoted diff) +30 -0
+static void mtk_jpeg_enc_device_run(void *priv)
+{
+       struct mtk_jpeg_ctx *ctx = priv;
+       struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+       struct vb2_v4l2_buffer *src_buf, *dst_buf;
+       enum vb2_buffer_state buf_state = VB2_BUF_STATE_ERROR;
+       unsigned long flags;
+       struct mtk_jpeg_src_buf *jpeg_src_buf;
+       struct mtk_jpeg_enc_bs enc_bs;
+       int ret;
+
+       src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
+       dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
+       jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(&src_buf->vb2_buf);
+
+       ret = pm_runtime_get_sync(jpeg->dev);
+       if (ret < 0)
+               goto enc_end;
+
+       spin_lock_irqsave(&jpeg->hw_lock, flags);
+
+       /*
+        * Resetting the hardware every frame is to ensure that all the
+        * registers are cleared. This is a hardware requirement.
+        */
+       mtk_jpeg_enc_reset(jpeg->reg_base);
+
+       mtk_jpeg_set_enc_dst(ctx, jpeg->reg_base, &dst_buf->vb2_buf, &enc_bs);
+       mtk_jpeg_set_enc_src(ctx, jpeg->reg_base, &src_buf->vb2_buf);
+       mtk_jpeg_enc_set_config(jpeg->reg_base, ctx->out_q.fmt->hw_format,
Quoted 8 lines

Review

Quoted 3 lines
(quoted diff) +3 -0
+                               ctx->enable_exif, ctx->enc_quality,
+                               ctx->restart_interval);
+       mtk_jpeg_enc_start(jpeg->reg_base);

Could we just move the above 5 functions into one function inside mtk_jpeg_enc_hw.c that takes mtk_jpeg_dev pointer as its argument, let's say mtk_jpeg_enc_hw_run() and simply program all the data to the registers directly, without the extra level of abstractions?

I can move the 5 functions into one function(mtk_jpeg_enc_hw_run()), but this function will be very long, because it contains computation code such as setting dst addr, blk_num, quality.
In v4, you have adviced the following architecture:
How about the following model, as used by many other drivers:

mtk_jpeg_enc_set_src()

{
        // Set any registers related to source format and buffer
}

mtk_jpeg_enc_set_dst()

{
        // Set any registers related to destination format and buffer
}

mtk_jpeg_enc_set_params()

{
        // Set any registers related to additional encoding parameters
}

mtk_jpeg_enc_device_run(enc, ctx)

{
        mtk_jpeg_enc_set_src(enc, src_buf, src_fmt);
        mtk_jpeg_enc_set_dst(enc, dst_buf, dst_fmt);
        mtk_jpeg_enc_set_params(enc, ctx);
        // Trigger the hardware run
}
Quoted 4 lines

I think that this architecture is more clear(mtk_jpeg_enc_set_config is equivalent to mtk_jpeg_enc_set_params).
Should I keep the original architecture or move 5 functions into mtk_jpeg_enc_hw_run?

Sounds good to me.

My biggest issue with the code that it ends up introducing one more level of abstraction, but with the approach you suggested, the arguments just accept standard structs, which avoids that problem.

[snip]

Quoted 47 lines
Quoted 31 lines

Review

Quoted 6 lines
(quoted diff) +6 -0
+
+       ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+       ctx->colorspace = V4L2_COLORSPACE_JPEG,
+       ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+       ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
+       ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;

Since we already have a v4l2_pix_format_mplane struct which has fields for the above 4 values, could we just store them there?

Also, I don't see this driver handling the colorspaces in any way, but it seems to allow changing them from the userspace. This is incorrect, because the userspace has no way to know that the colorspace is not handled. Instead, the try_fmt implementation should always override the userspace-provided colorspace configuration with the ones that the driver assumes.

Review

Quoted 13 lines
(quoted diff) +13 -0
+       pix_mp->width = MTK_JPEG_MIN_WIDTH;
+       pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+
+       q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_YUYV,
+                                     MTK_JPEG_FMT_FLAG_ENC_OUTPUT);
+       vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+                                   fmt.pix_mp), q->fmt);
+       q->w = pix_mp->width;
+       q->h = pix_mp->height;
+       q->crop_rect.width = pix_mp->width;
+       q->crop_rect.height = pix_mp->height;
+       q->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
+       q->bytesperline[0] = pix_mp->plane_fmt[0].bytesperline;
Quoted 3 lines

Actually, do we need this custom mtk_jpeg_q_data struct? Why couldn't we just keep the same values inside the standard v4l2_pix_format_mplane struct?

I think that we need mtk_jpeg_q_data struct.If we delete it, how can we know these values(w, h, sizeimage, bytesperline, mtk_jpeg_fmt) belong to output or capture(output and capture's sizeimages are different, width and height are differnt too for jpeg dec )?We have s_fmt_vid_out_mplane/cap_mplane function to set these values.

But we can use standard v4l2_pix_format_mplane struct replacing the w, h bytesperline, sizeimage in mtk_jpeg_q_data struct like this:

struct mtk_jpeg_q_data{
    struct mtk_jpeg_fmt *fmt;
    struct v4l2_pix_format_mplane pix_mp;
    struct v4l2_rect enc_crop_rect
}
Quoted 5 lines

Then delete ctx->colorspace ctx->ycbcr_enc ctx->quantization ctx->xfer_func, becuase v4l2_pix_format_mplane in q_data has contained them and assign them for out_q and cap_q separately.

WDYT?

Sounds good to me. I was considering just making it like

struct mtk_jpeg_ctx {
      struct mtk_jpeg_fmt *src_fmt;
      struct v4l2_pix_format_mplane src_pix_mp;
      struct v4l2_rect src_crop;

      struct mtk_jpeg_fmt *dst_fmt;
      struct v4l2_pix_format_mplane dst_pix_mp;
      struct v4l2_rect dst_crop;
};
Quoted 2 lines

but I like your suggestion as well, as long as custom data structures are not used to store standard information.

Dear Tomasz,

I used the structure like below:

 struct mtk_jpeg_q_data{
        struct mtk_jpeg_fmt *fmt;
        struct v4l2_pix_format_mplane pix_mp;
        struct v4l2_rect enc_crop_rect
}

Then delete ctx->colorspace ctx->ycbcr_enc ctx->quantization ctx->xfer_func, becuase v4l2_pix_format_mplane in q_data has contained them and assign them for out_q and cap_q separately.

Then the v4l2_compliance test will fail, the fail log as below:
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
fail: v4l2-test-formats.cpp(836):
fmt_cap.g_colorspace() != col
test VIDIOC_S_FMT: FAIL
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)

The source code of v4l2-test-formats.cpp as below:

static int testM2MFormats(struct node *node)

{
        cv4l_fmt fmt_out;
        cv4l_fmt fmt;
        cv4l_fmt fmt_cap;
        __u32 cap_type = node->g_type();
        __u32 out_type = v4l_type_invert(cap_type);
        __u32 col, ycbcr_enc, quant, xfer_func;

        fail_on_test(node->g_fmt(fmt_out, out_type));
        node->g_fmt(fmt_cap, cap_type);
        fail_on_test(fmt_cap.g_colorspace() != fmt_out.g_colorspace());
        fail_on_test(fmt_cap.g_ycbcr_enc() != fmt_out.g_ycbcr_enc());
        fail_on_test(fmt_cap.g_quantization() != fmt_out.g_quantization());
        fail_on_test(fmt_cap.g_xfer_func() != fmt_out.g_xfer_func());
        col = fmt_out.g_colorspace() == V4L2_COLORSPACE_SMPTE170M ?
                V4L2_COLORSPACE_REC709 : V4L2_COLORSPACE_SMPTE170M;
        ycbcr_enc = fmt_out.g_ycbcr_enc() == V4L2_YCBCR_ENC_601 ?
                V4L2_YCBCR_ENC_709 : V4L2_YCBCR_ENC_601;
        quant = fmt_out.g_quantization() == V4L2_QUANTIZATION_LIM_RANGE ?
                V4L2_QUANTIZATION_FULL_RANGE : V4L2_QUANTIZATION_LIM_RANGE;
        xfer_func = fmt_out.g_xfer_func() == V4L2_XFER_FUNC_SRGB ?
                V4L2_XFER_FUNC_709 : V4L2_XFER_FUNC_SRGB;
        fmt_out.s_colorspace(col);
        fmt_out.s_xfer_func(xfer_func);
        fmt_out.s_ycbcr_enc(ycbcr_enc);
        fmt_out.s_quantization(quant);
        node->s_fmt(fmt_out);
        fail_on_test(fmt_out.g_colorspace() != col);
        fail_on_test(fmt_out.g_xfer_func() != xfer_func);
        fail_on_test(fmt_out.g_ycbcr_enc() != ycbcr_enc);
        fail_on_test(fmt_out.g_quantization() != quant);
        node->g_fmt(fmt_cap);
        fail_on_test(fmt_cap.g_colorspace() != col);  // line 836
        fail_on_test(fmt_cap.g_xfer_func() != xfer_func);
        fail_on_test(fmt_cap.g_ycbcr_enc() != ycbcr_enc);
        fail_on_test(fmt_cap.g_quantization() != quant);
}

It needs that cap's colorspace equals out's colorspace when userspace just set out's colorspace and then get cap's colorspace. However, cap's colorspace which getted from driver equals V4L2_COLORSPACE_JPEG, because the code in g_fmt() like this:
pix_mp->colorspace = q_data->pix_mp.colorspace; pix_mp->ycbcr_enc = q_data->pix_mp.ycbcr_enc; pix_mp->xfer_func = q_data->pix_mp.xfer_func; pix_mp->quantization = q_data->pix_mp.quantization;

Quoted 7 lines

How should I handle this case? Should I store them(colorspace, sfer_func, ycbcr_enc, quatization) in ctx as the orinal desin? Then I can get them from g_fmt() like this:
pix_mp->colorspace = ctx->colorspace;
pix_mp->ycbcr_enc = ctx->ycbcr_enc;
pix_mp->xfer_func = ctx->xfer_func;
pix_mp->quantization = ctx->quantization;

Quoted 7 lines

Why would there be any other colorspace accepted? I suppose that the hardware only supports the JPEG color space, so it shouldn't accept any other colorspace in TRY_FMT (and thus S_FMT) anyway.

Still, for correctness, I would suggest propagating the colorspace (and related) information from OUTPUT format to CAPTURE format in S_FMT(OUTPUT).

Dear Tomasz,
If the driver doesn't accept any other colorspace from userspace(means the colorspace always equals V4L2_COLORSPACE_JPEG on my understanding), the v4l2_compliance will fail at line 831:
fail_on_test(fmt_out.g_colorspace() != col); // line 831

The v4l2_compliance needs driver can accept other colorspace and capture colorspace equals output colorspace.

Quoted 7 lines
Quoted 4 lines

I try to propagate the colorspace from OUTPUT format to CAPTURE foramt in S_FMT, the compliance test succeed.

Should the jpeg driver can accept other colorspace?

Hans, could you advise on what is the proper behavior regarding colorimetry support?

The JPEG encoder doesn't do any colorspace conversion as far as I know, so to get a valid JPEG out of the encoder the colorspace of the OUTPUT format has to be the same as how the captured JPEG data is interpreted.

So when userspace sets the output format, then the driver should replace the colorspace field by V4L2_COLORSPACE_SRGB, xfer_func by V4L2_XFER_FUNC and ycbcr_enc to V4L2_YCBCR_ENC_601.

What I don't know is if this encoder driver supports both limited and full range for the YUV data. If so, then it supports both values for that field. If it supports only limited or full range, then it should replace the quantization field for the output format with a fixed value as well.

The compliance test doesn't handle JPEG codecs very well when it comes to colorspace. I'll try to fix this.

Regards,

Hans

author
#44 lore raw

Re: [PATCH RESEND v9 18/18] media: platform: Add jpeg enc feature

In reply to Tomasz Figa
Quoted 22 lines · Tomasz Figa

Hi Xia,

Quoted 8 lines · Xia Jiang

Add mtk jpeg encode v4l2 driver based on jpeg decode, because that jpeg decode and encode have great similarities with function operation.

---
v9: add member variable(struct v4l2_rect) in out_q structure for storing
the active crop information.
move the renaming exsting functions/defines/variables to a separate patch.

Quoted 10 lines
---
 drivers/media/platform/mtk-jpeg/Makefile      |   5 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_core.c   | 845 +++++++++++++++---
 .../media/platform/mtk-jpeg/mtk_jpeg_core.h   |  44 +-
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c | 193 ++++
 .../media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h | 123 +++
 5 files changed, 1084 insertions(+), 126 deletions(-)
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.h
Quoted 1 line

Thank you for the patch. Please see my comments inline.

Dear Tomasz,
Thank you for your comments.
I have uploaded the v10 version which contains the changes you mentioned.

Quoted 17 lines

[snip]

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static inline struct mtk_jpeg_src_buf *mtk_jpeg_vb2_to_srcbuf( · line 145

Quoted 12 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +12 -0
+static int mtk_jpeg_enc_querycap(struct file *file, void *priv,
+				 struct v4l2_capability *cap)
+{
+	struct mtk_jpeg_dev *jpeg = video_drvdata(file);
+
+	strscpy(cap->driver, MTK_JPEG_NAME, sizeof(cap->driver));
+	strscpy(cap->card, MTK_JPEG_NAME " encoder", sizeof(cap->card));
+	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
+		 dev_name(jpeg->dev));
+
+	return 0;
+}

I can see that this function differs from mtk_jpeg_dec_querycap() only with the " encoder" string. Perhaps they could be merged?

done.

Quoted 27 lines
Quoted 13 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -0
+
 static int mtk_jpeg_dec_querycap(struct file *file, void *priv,
 				 struct v4l2_capability *cap)
 {
@@ -88,6 +157,54 @@ static int mtk_jpeg_dec_querycap(struct file *file, void *priv,
 	return 0;
 }
 
+static int vidioc_jpeg_enc_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_jpeg_ctx *ctx = ctrl_to_ctx(ctrl);
+

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_querycap(struct file *file, void *priv, · line 173

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+	switch (ctrl->id) {
+	case V4L2_CID_JPEG_RESTART_INTERVAL:
+		ctx->restart_interval = ctrl->val;
+		break;
+	case V4L2_CID_JPEG_COMPRESSION_QUALITY:
+		ctx->enc_quality = ctrl->val;
+		break;
+	case V4L2_CID_JPEG_ACTIVE_MARKER:
+		ctx->enable_exif = ctrl->val & V4L2_JPEG_ACTIVE_MARKER_APP1 ?
+				   true : false;

nit: If ctx->enable_exif is of the bool type, the ternary operator could be removed, because any non-zero value is implicitly turned into 1, as per [1].

Quoted 1 line

[1] https://www.kernel.org/doc/html/v5.6/process/coding-style.html#using-bool

done.

Quoted 17 lines

[snip]

Quoted 15 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +8 -0
+static int mtk_jpeg_enc_enum_fmt_vid_cap(struct file *file, void *priv,
+					 struct v4l2_fmtdesc *f)
+{
+	return mtk_jpeg_enum_fmt(mtk_jpeg_enc_formats,
+				 MTK_JPEG_ENC_NUM_FORMATS, f,
+				 MTK_JPEG_FMT_FLAG_ENC_CAPTURE);
+}
+
 static int mtk_jpeg_dec_enum_fmt_vid_cap(struct file *file, void *priv,
 					 struct v4l2_fmtdesc *f)
 {
@@ -117,6 +242,14 @@ static int mtk_jpeg_dec_enum_fmt_vid_cap(struct file *file, void *priv,
 				 MTK_JPEG_FMT_FLAG_DEC_CAPTURE);
 }
 
Quoted 9 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_enum_fmt_vid_cap(struct file *file, void *priv, · line 250

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
+static int mtk_jpeg_enc_enum_fmt_vid_out(struct file *file, void *priv,
+					 struct v4l2_fmtdesc *f)
+{
+	return mtk_jpeg_enum_fmt(mtk_jpeg_enc_formats,
+				 MTK_JPEG_ENC_NUM_FORMATS, f,
+				 MTK_JPEG_FMT_FLAG_ENC_OUTPUT);

Do we need separate implementations of these? "formats" and "num_formats" could be specified by the match data struct and used in a generic function.

done.

Quoted 3 lines

Also, do we need separate flags for ENC_OUTPUT/CAPTURE and DEC_OUTPUT/CAPTURE?

done

Quoted 33 lines
Quoted 33 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +12 -7
+}
+
 static int mtk_jpeg_dec_enum_fmt_vid_out(struct file *file, void *priv,
 					 struct v4l2_fmtdesc *f)
 {
@@ -132,93 +265,66 @@ mtk_jpeg_get_q_data(struct mtk_jpeg_ctx *ctx, enum v4l2_buf_type type)
 	return &ctx->cap_q;
 }
 
-static struct mtk_jpeg_fmt *mtk_jpeg_find_format(struct mtk_jpeg_ctx *ctx,
-						 u32 pixelformat,
+static struct mtk_jpeg_fmt *mtk_jpeg_find_format(u32 pixelformat,
 						 unsigned int fmt_type)
 {
-	unsigned int k, fmt_flag;
+	unsigned int k;
+	struct mtk_jpeg_fmt *fmt;
 
-	fmt_flag = (fmt_type == MTK_JPEG_FMT_TYPE_OUTPUT) ?
-		   MTK_JPEG_FMT_FLAG_DEC_OUTPUT :
-		   MTK_JPEG_FMT_FLAG_DEC_CAPTURE;
+	for (k = 0; k < MTK_JPEG_ENC_NUM_FORMATS; k++) {
+		fmt = &mtk_jpeg_enc_formats[k];
+
+		if (fmt->fourcc == pixelformat && fmt->flags & fmt_type)
+			return fmt;
+	}
 
 	for (k = 0; k < MTK_JPEG_DEC_NUM_FORMATS; k++) {
-		struct mtk_jpeg_fmt *fmt = &mtk_jpeg_dec_formats[k];
+		fmt = &mtk_jpeg_dec_formats[k];
 
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c mtk_jpeg_get_q_data(struct mtk_jpeg_ctx *ctx, enum v4l2_buf_type type) · line 286

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -1
-		if (fmt->fourcc == pixelformat && fmt->flags & fmt_flag)
+		if (fmt->fourcc == pixelformat && fmt->flags & fmt_type)
 			return fmt;
 	}

We should only need to iterate through one of the arrays. My suggestion above about making "formats" and "num_formats" a member of the match data should take care of this one too.

done.

Quoted 25 lines
Quoted 25 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +0 -20
 	return NULL;
 }
 
-static void mtk_jpeg_adjust_fmt_mplane(struct mtk_jpeg_ctx *ctx,
-				       struct v4l2_format *f)
-{
-	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
-	struct mtk_jpeg_q_data *q_data;
-	int i;
-
-	q_data = mtk_jpeg_get_q_data(ctx, f->type);
-
-	pix_mp->width = q_data->w;
-	pix_mp->height = q_data->h;
-	pix_mp->pixelformat = q_data->fmt->fourcc;
-	pix_mp->num_planes = q_data->fmt->colplanes;
-
-	for (i = 0; i < pix_mp->num_planes; i++) {
-		pix_mp->plane_fmt[i].bytesperline = q_data->bytesperline[i];
-		pix_mp->plane_fmt[i].sizeimage = q_data->sizeimage[i];
-	}
-}
-
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c mtk_jpeg_get_q_data(struct mtk_jpeg_ctx *ctx, enum v4l2_buf_type type) · line 291

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -3
-static int mtk_jpeg_try_fmt_mplane(struct v4l2_format *f,
-				   struct mtk_jpeg_fmt *fmt,
-				   struct mtk_jpeg_ctx *ctx, int q_type)
+static int vidioc_try_fmt(struct v4l2_format *f, struct mtk_jpeg_fmt *fmt)

The name is a bit misleading, because it's suggesting that it's the vidioc_try_fmt callback, but it's just an internal helper. I think the original name was fine.

changed it to original name.

Quoted 5 lines
Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +0 -0
 {
 	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
 	int i;
 
Quoted 9 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c mtk_jpeg_get_q_data(struct mtk_jpeg_ctx *ctx, enum v4l2_buf_type type) · line 187

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +0 -5
 	pix_mp->field = V4L2_FIELD_NONE;
-
-	if (ctx->state != MTK_JPEG_INIT) {
-		mtk_jpeg_adjust_fmt_mplane(ctx, f);
-		return 0;
-	}

Is it okay to remove this? My understanding is that it prevented changing the format while streaming, which should is as expected.

mtk_jpeg_g_fmt_vid_mplane() does the same thing, so use it to replace mtk_jpeg_adjust_fmt_mplane.

Quoted 55 lines
Quoted 55 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +22 -23
-
 	pix_mp->num_planes = fmt->colplanes;
 	pix_mp->pixelformat = fmt->fourcc;
 
-	if (q_type == MTK_JPEG_FMT_TYPE_OUTPUT) {
-		struct v4l2_plane_pix_format *pfmt = &pix_mp->plane_fmt[0];
-
+	if (fmt->fourcc == V4L2_PIX_FMT_JPEG) {
 		pix_mp->height = clamp(pix_mp->height, MTK_JPEG_MIN_HEIGHT,
 				       MTK_JPEG_MAX_HEIGHT);
 		pix_mp->width = clamp(pix_mp->width, MTK_JPEG_MIN_WIDTH,
 				      MTK_JPEG_MAX_WIDTH);
-
-		pfmt->bytesperline = 0;
-		/* Source size must be aligned to 128 */
-		pfmt->sizeimage = round_up(pfmt->sizeimage, 128);
-		if (pfmt->sizeimage == 0)
-			pfmt->sizeimage = MTK_JPEG_DEFAULT_SIZEIMAGE;
-		return 0;
+		pix_mp->plane_fmt[0].bytesperline = 0;
+		pix_mp->plane_fmt[0].sizeimage =
+				round_up(pix_mp->plane_fmt[0].sizeimage, 128);
+		if (pix_mp->plane_fmt[0].sizeimage == 0)
+			pix_mp->plane_fmt[0].sizeimage =
+				MTK_JPEG_DEFAULT_SIZEIMAGE;
+	} else {
+		pix_mp->height = clamp(round_up(pix_mp->height, fmt->v_align),
+				       MTK_JPEG_MIN_HEIGHT,
+				       MTK_JPEG_MAX_HEIGHT);
+		pix_mp->width = clamp(round_up(pix_mp->width, fmt->h_align),
+				      MTK_JPEG_MIN_WIDTH, MTK_JPEG_MAX_WIDTH);
+		for (i = 0; i < pix_mp->num_planes; i++) {
+			struct v4l2_plane_pix_format *pfmt =
+							&pix_mp->plane_fmt[i];
+			u32 stride = pix_mp->width * fmt->h_sample[i] / 4;
+			u32 h = pix_mp->height * fmt->v_sample[i] / 4;
+
+			pfmt->bytesperline = stride;
+			pfmt->sizeimage = stride * h;
+		}
 	}
 
-	/* type is MTK_JPEG_FMT_TYPE_CAPTURE */
-	pix_mp->height = clamp(round_up(pix_mp->height, fmt->v_align),
-			       MTK_JPEG_MIN_HEIGHT, MTK_JPEG_MAX_HEIGHT);
-	pix_mp->width = clamp(round_up(pix_mp->width, fmt->h_align),
-			      MTK_JPEG_MIN_WIDTH, MTK_JPEG_MAX_WIDTH);
-
-	for (i = 0; i < fmt->colplanes; i++) {
-		struct v4l2_plane_pix_format *pfmt = &pix_mp->plane_fmt[i];
-		u32 stride = pix_mp->width * fmt->h_sample[i] / 4;
-		u32 h = pix_mp->height * fmt->v_sample[i] / 4;
-
-		pfmt->bytesperline = stride;
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c mtk_jpeg_get_q_data(struct mtk_jpeg_ctx *ctx, enum v4l2_buf_type type) · line 328

Quoted 3 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +0 -2
-		pfmt->sizeimage = stride * h;
-	}
 	return 0;

Are all the changes above needed for the encoder? If I'm looking correctly, they're mostly refactoring and should be done in a separate patch. However, I don't see any big issue with the existing coding style in this function, so perhaps the refactoring could be avoided.

In order to compatible jpeg encoder, use the fourcc to distinguish different types as you mentioned before.

Quoted 22 lines
Quoted 22 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +15 -0
 }
 
@@ -271,14 +377,35 @@ static int mtk_jpeg_g_fmt_vid_mplane(struct file *file, void *priv,
 	return 0;
 }
 
+static int mtk_jpeg_enc_try_fmt_vid_cap_mplane(struct file *file, void *priv,
+					       struct v4l2_format *f)
+{
+	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+	struct mtk_jpeg_fmt *fmt;
+
+	fmt = mtk_jpeg_find_format(f->fmt.pix_mp.pixelformat,
+				   MTK_JPEG_FMT_FLAG_ENC_CAPTURE);
+	if (!fmt)
+		fmt = ctx->cap_q.fmt;
+
+	v4l2_dbg(2, debug, &ctx->jpeg->v4l2_dev, "(%d) try_fmt:%c%c%c%c\n",
+		 f->type,
+		 (fmt->fourcc & 0xff),
+		 (fmt->fourcc >>  8 & 0xff),
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_g_fmt_vid_mplane(struct file *file, void *priv, · line 399

Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -0
+		 (fmt->fourcc >> 16 & 0xff),
+		 (fmt->fourcc >> 24 & 0xff));
+
+	return vidioc_try_fmt(f, fmt);
+}

Is there really anything encoder-specific in this function? Could the same function as the decoder be used instead?

done.

Quoted 19 lines
Quoted 19 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -2
+
 static int mtk_jpeg_dec_try_fmt_vid_cap_mplane(struct file *file, void *priv,
 					       struct v4l2_format *f)
 {
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
 	struct mtk_jpeg_fmt *fmt;
 
-	fmt = mtk_jpeg_find_format(ctx, f->fmt.pix_mp.pixelformat,
-				   MTK_JPEG_FMT_TYPE_CAPTURE);
+	fmt = mtk_jpeg_find_format(f->fmt.pix_mp.pixelformat,
+				   MTK_JPEG_FMT_FLAG_DEC_CAPTURE);
 	if (!fmt)
 		fmt = ctx->cap_q.fmt;
 
@@ -289,7 +416,33 @@ static int mtk_jpeg_dec_try_fmt_vid_cap_mplane(struct file *file, void *priv,
 		 (fmt->fourcc >> 16 & 0xff),
 		 (fmt->fourcc >> 24 & 0xff));
 
Quoted 9 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_try_fmt_vid_cap_mplane(struct file *file, void *priv, · line 422

Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -1
-	return mtk_jpeg_try_fmt_mplane(f, fmt, ctx, MTK_JPEG_FMT_TYPE_CAPTURE);
+	if (ctx->state != MTK_JPEG_INIT) {
+		mtk_jpeg_g_fmt_vid_mplane(file, priv, f);
+		return 0;
+	}

Is this really limited to the decoder? In general currently we don't support changing the resolution from the userspace when streaming is started and it applies to both encoder and decoder.

done.

Quoted 19 lines
Quoted 19 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +18 -0
+
+	return vidioc_try_fmt(f, fmt);
+}
+
+static int mtk_jpeg_enc_try_fmt_vid_out_mplane(struct file *file, void *priv,
+					       struct v4l2_format *f)
+{
+	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+	struct mtk_jpeg_fmt *fmt;
+
+	fmt = mtk_jpeg_find_format(f->fmt.pix_mp.pixelformat,
+				   MTK_JPEG_FMT_FLAG_ENC_OUTPUT);
+	if (!fmt)
+		fmt = ctx->out_q.fmt;
+
+	v4l2_dbg(2, debug, &ctx->jpeg->v4l2_dev, "(%d) try_fmt:%c%c%c%c\n",
+		 f->type,
+		 (fmt->fourcc & 0xff),
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_try_fmt_vid_cap_mplane(struct file *file, void *priv, · line 446

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -0
+		 (fmt->fourcc >>  8 & 0xff),
+		 (fmt->fourcc >> 16 & 0xff),
+		 (fmt->fourcc >> 24 & 0xff));
+
+	return vidioc_try_fmt(f, fmt);
 }

Ditto.

done.

Quoted 18 lines
Quoted 18 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -2
 static int mtk_jpeg_dec_try_fmt_vid_out_mplane(struct file *file, void *priv,
@@ -298,8 +451,8 @@ static int mtk_jpeg_dec_try_fmt_vid_out_mplane(struct file *file, void *priv,
 	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
 	struct mtk_jpeg_fmt *fmt;
 
-	fmt = mtk_jpeg_find_format(ctx, f->fmt.pix_mp.pixelformat,
-				   MTK_JPEG_FMT_TYPE_OUTPUT);
+	fmt = mtk_jpeg_find_format(f->fmt.pix_mp.pixelformat,
+				   MTK_JPEG_FMT_FLAG_DEC_OUTPUT);
 	if (!fmt)
 		fmt = ctx->out_q.fmt;
 
@@ -310,17 +463,21 @@ static int mtk_jpeg_dec_try_fmt_vid_out_mplane(struct file *file, void *priv,
 		 (fmt->fourcc >> 16 & 0xff),
 		 (fmt->fourcc >> 24 & 0xff));
 
Quoted 7 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_try_fmt_vid_out_mplane(struct file *file, void *priv, · line 469

Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -1
-	return mtk_jpeg_try_fmt_mplane(f, fmt, ctx, MTK_JPEG_FMT_TYPE_OUTPUT);
+	if (ctx->state != MTK_JPEG_INIT) {
+		mtk_jpeg_g_fmt_vid_mplane(file, priv, f);
+		return 0;
+	}

Ditto.

done.

Quoted 80 lines
Quoted 80 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +34 -7
+
+	return vidioc_try_fmt(f, fmt);
 }
 
 static int mtk_jpeg_s_fmt_mplane(struct mtk_jpeg_ctx *ctx,
-				 struct v4l2_format *f)
+				 struct v4l2_format *f, unsigned int fmt_type)
 {
 	struct vb2_queue *vq;
 	struct mtk_jpeg_q_data *q_data = NULL;
 	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
 	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
-	unsigned int f_type;
 	int i;
 
 	vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, f->type);
@@ -334,12 +491,11 @@ static int mtk_jpeg_s_fmt_mplane(struct mtk_jpeg_ctx *ctx,
 		return -EBUSY;
 	}
 
-	f_type = V4L2_TYPE_IS_OUTPUT(f->type) ?
-			 MTK_JPEG_FMT_TYPE_OUTPUT : MTK_JPEG_FMT_TYPE_CAPTURE;
-
-	q_data->fmt = mtk_jpeg_find_format(ctx, pix_mp->pixelformat, f_type);
+	q_data->fmt = mtk_jpeg_find_format(pix_mp->pixelformat, fmt_type);
 	q_data->w = pix_mp->width;
 	q_data->h = pix_mp->height;
+	q_data->crop_rect.width = pix_mp->width;
+	q_data->crop_rect.height = pix_mp->height;
 	ctx->colorspace = pix_mp->colorspace;
 	ctx->ycbcr_enc = pix_mp->ycbcr_enc;
 	ctx->xfer_func = pix_mp->xfer_func;
@@ -365,6 +521,19 @@ static int mtk_jpeg_s_fmt_mplane(struct mtk_jpeg_ctx *ctx,
 	return 0;
 }
 
+static int mtk_jpeg_enc_s_fmt_vid_out_mplane(struct file *file, void *priv,
+					     struct v4l2_format *f)
+{
+	int ret;
+
+	ret = mtk_jpeg_enc_try_fmt_vid_out_mplane(file, priv, f);
+	if (ret)
+		return ret;
+
+	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f,
+				     MTK_JPEG_FMT_FLAG_ENC_OUTPUT);
+}
+
 static int mtk_jpeg_dec_s_fmt_vid_out_mplane(struct file *file, void *priv,
 					     struct v4l2_format *f)
 {
@@ -374,7 +543,21 @@ static int mtk_jpeg_dec_s_fmt_vid_out_mplane(struct file *file, void *priv,
 	if (ret)
 		return ret;
 
-	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f);
+	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f,
+				     MTK_JPEG_FMT_FLAG_DEC_OUTPUT);
+}
+
+static int mtk_jpeg_enc_s_fmt_vid_cap_mplane(struct file *file, void *priv,
+					     struct v4l2_format *f)
+{
+	int ret;
+
+	ret = mtk_jpeg_enc_try_fmt_vid_cap_mplane(file, priv, f);
+	if (ret)
+		return ret;
+
+	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f,
+				     MTK_JPEG_FMT_FLAG_ENC_CAPTURE);
 }
 
 static int mtk_jpeg_dec_s_fmt_vid_cap_mplane(struct file *file, void *priv,
@@ -386,7 +569,8 @@ static int mtk_jpeg_dec_s_fmt_vid_cap_mplane(struct file *file, void *priv,
 	if (ret)
 		return ret;
 
Quoted 7 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_s_fmt_vid_cap_mplane(struct file *file, void *priv, · line 574

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -1
-	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f);
+	return mtk_jpeg_s_fmt_mplane(mtk_jpeg_fh_to_ctx(priv), f,
+				     MTK_JPEG_FMT_FLAG_DEC_CAPTURE);
 }

Is it really necessary to have separate variants of the above functions for decoder and encoder?

done.

Quoted 20 lines
Quoted 20 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +13 -0
 static void mtk_jpeg_queue_src_chg_event(struct mtk_jpeg_ctx *ctx)
@@ -411,6 +595,29 @@ static int mtk_jpeg_subscribe_event(struct v4l2_fh *fh,
 	return v4l2_ctrl_subscribe_event(fh, sub);
 }
 
+static int mtk_jpeg_enc_g_selection(struct file *file, void *priv,
+				    struct v4l2_selection *s)
+{
+	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+
+	if (s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)
+		return -EINVAL;
+
+	switch (s->target) {
+	case V4L2_SEL_TGT_CROP:
+	case V4L2_SEL_TGT_CROP_BOUNDS:
+	case V4L2_SEL_TGT_CROP_DEFAULT:
+		s->r.width = ctx->out_q.w;
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_subscribe_event(struct v4l2_fh *fh, · line 613

Quoted 3 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -0
+		s->r.height = ctx->out_q.h;
+		s->r.left = 0;
+		s->r.top = 0;

Is this really correct? The function seems to be returning the full frame size regardless of the selection target. For BOUNDS and DEFAULTS this would be the correct behavior indeed, but CROP should return the active crop rectangle, as set by S_SELECTION.

done.

Quoted 45 lines
Quoted 45 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +30 -0
+		break;
+	default:
+		return -EINVAL;
+	}
+	return 0;
+}
+
 static int mtk_jpeg_dec_g_selection(struct file *file, void *priv,
 				    struct v4l2_selection *s)
 {
@@ -440,6 +647,29 @@ static int mtk_jpeg_dec_g_selection(struct file *file, void *priv,
 	return 0;
 }
 
+static int mtk_jpeg_enc_s_selection(struct file *file, void *priv,
+				    struct v4l2_selection *s)
+{
+	struct mtk_jpeg_ctx *ctx = mtk_jpeg_fh_to_ctx(priv);
+
+	if (s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)
+		return -EINVAL;
+
+	switch (s->target) {
+	case V4L2_SEL_TGT_CROP:
+		s->r.left = 0;
+		s->r.top = 0;
+		s->r.width = min(s->r.width, ctx->out_q.w);
+		s->r.height = min(s->r.height, ctx->out_q.h);
+		ctx->out_q.crop_rect = s->r;
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
 static int mtk_jpeg_dec_s_selection(struct file *file, void *priv,
 				    struct v4l2_selection *s)
 {
@@ -484,6 +714,33 @@ static int mtk_jpeg_qbuf(struct file *file, void *priv, struct v4l2_buffer *buf)
 	return v4l2_m2m_qbuf(file, fh->m2m_ctx, buf);
 }
 
Quoted 15 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_qbuf(struct file *file, void *priv, struct v4l2_buffer *buf) · line 727

Quoted 11 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +11 -0
+static const struct v4l2_ioctl_ops mtk_jpeg_enc_ioctl_ops = {
+	.vidioc_querycap                = mtk_jpeg_enc_querycap,
+	.vidioc_enum_fmt_vid_cap	= mtk_jpeg_enc_enum_fmt_vid_cap,
+	.vidioc_enum_fmt_vid_out	= mtk_jpeg_enc_enum_fmt_vid_out,
+	.vidioc_try_fmt_vid_cap_mplane	= mtk_jpeg_enc_try_fmt_vid_cap_mplane,
+	.vidioc_try_fmt_vid_out_mplane	= mtk_jpeg_enc_try_fmt_vid_out_mplane,
+	.vidioc_g_fmt_vid_cap_mplane    = mtk_jpeg_g_fmt_vid_mplane,
+	.vidioc_g_fmt_vid_out_mplane    = mtk_jpeg_g_fmt_vid_mplane,
+	.vidioc_s_fmt_vid_cap_mplane    = mtk_jpeg_enc_s_fmt_vid_cap_mplane,
+	.vidioc_s_fmt_vid_out_mplane    = mtk_jpeg_enc_s_fmt_vid_out_mplane,
+	.vidioc_qbuf                    = mtk_jpeg_qbuf,

Not directly a comment for this patch, but since the previous patch removed the LAST_FRAME handling, wouldn't it be enough to just use v4l2_m2m_qbuf() as this callback?

yes,done.

Quoted 52 lines

[snip]

Quoted 50 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +29 -0
+static void mtk_jpeg_enc_buf_queue(struct vb2_buffer *vb)
+{
+	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+
+	v4l2_dbg(2, debug, &jpeg->v4l2_dev, "(%d) buf_q id=%d, vb=%p\n",
+		 vb->vb2_queue->type, vb->index, vb);
+
+	v4l2_m2m_buf_queue(ctx->fh.m2m_ctx, to_vb2_v4l2_buffer(vb));
+}
+
 static void mtk_jpeg_dec_buf_queue(struct vb2_buffer *vb)
 {
 	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
@@ -664,6 +932,15 @@ static struct vb2_v4l2_buffer *mtk_jpeg_buf_remove(struct mtk_jpeg_ctx *ctx,
 		return v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
 }
 
+static void mtk_jpeg_enc_stop_streaming(struct vb2_queue *q)
+{
+	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(q);
+	struct vb2_v4l2_buffer *vb;
+
+	while ((vb = mtk_jpeg_buf_remove(ctx, q->type)))
+		v4l2_m2m_buf_done(vb, VB2_BUF_STATE_ERROR);
+}
+
 static void mtk_jpeg_dec_stop_streaming(struct vb2_queue *q)
 {
 	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(q);
@@ -699,6 +976,15 @@ static const struct vb2_ops mtk_jpeg_dec_qops = {
 	.stop_streaming     = mtk_jpeg_dec_stop_streaming,
 };
 
+static const struct vb2_ops mtk_jpeg_enc_qops = {
+	.queue_setup        = mtk_jpeg_queue_setup,
+	.buf_prepare        = mtk_jpeg_buf_prepare,
+	.buf_queue          = mtk_jpeg_enc_buf_queue,
+	.wait_prepare       = vb2_ops_wait_prepare,
+	.wait_finish        = vb2_ops_wait_finish,
+	.stop_streaming     = mtk_jpeg_enc_stop_streaming,
+};
+
 static void mtk_jpeg_set_dec_src(struct mtk_jpeg_ctx *ctx,
 				 struct vb2_buffer *src_buf,
 				 struct mtk_jpeg_bs *bs)
@@ -736,6 +1022,85 @@ static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx,
 	return 0;
 }
 
Quoted 12 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx, · line 1032

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +8 -0
+static void mtk_jpeg_set_enc_dst(struct mtk_jpeg_ctx *ctx, void __iomem *base,
+				 struct vb2_buffer *dst_buf,
+				 struct mtk_jpeg_enc_bs *bs)
+{
+	bs->dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
+	bs->dma_addr_offset = ctx->enable_exif ? MTK_JPEG_MAX_EXIF_SIZE : 0;
+	bs->dma_addr_offsetmask = bs->dma_addr & JPEG_ENC_DST_ADDR_OFFSET_MASK;
+	bs->size = vb2_plane_size(dst_buf, 0);

We're computing these values and then writing to the hardware straightaway. Do we need to save these values to the bs struct? If no, do we need the bs struct at all?

yes, removed this structure.

Quoted 14 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx, · line 1043

Quoted 12 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +11 -0
+
+	mtk_jpeg_enc_set_dst_addr(base, bs->dma_addr, bs->size,
+				  bs->dma_addr_offset,
+				  bs->dma_addr_offsetmask);
+}
+
+static void mtk_jpeg_set_enc_src(struct mtk_jpeg_ctx *ctx, void __iomem *base,
+				 struct vb2_buffer *src_buf)
+{
+	int i;
+	dma_addr_t	dma_addr;

nit: Only one space should be between variable type and name.

done.

Quoted 47 lines
Quoted 47 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +46 -0
+
+	mtk_jpeg_enc_set_img_size(base, ctx->out_q.crop_rect.width,
+				  ctx->out_q.crop_rect.height);
+	mtk_jpeg_enc_set_blk_num(base, ctx->out_q.fmt->fourcc,
+				 ctx->out_q.crop_rect.width,
+				 ctx->out_q.crop_rect.height);
+	mtk_jpeg_enc_set_stride(base, ctx->out_q.fmt->fourcc, ctx->out_q.w,
+				ctx->out_q.h, ctx->out_q.bytesperline[0]);
+
+	for (i = 0; i < src_buf->num_planes; i++) {
+		dma_addr = vb2_dma_contig_plane_dma_addr(src_buf, i) +
+			   src_buf->planes[i].data_offset;
+		mtk_jpeg_enc_set_src_addr(base, dma_addr, i);
+	}
+}
+
+static void mtk_jpeg_enc_device_run(void *priv)
+{
+	struct mtk_jpeg_ctx *ctx = priv;
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
+	struct vb2_v4l2_buffer *src_buf, *dst_buf;
+	enum vb2_buffer_state buf_state = VB2_BUF_STATE_ERROR;
+	unsigned long flags;
+	struct mtk_jpeg_src_buf *jpeg_src_buf;
+	struct mtk_jpeg_enc_bs enc_bs;
+	int ret;
+
+	src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
+	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
+	jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(&src_buf->vb2_buf);
+
+	ret = pm_runtime_get_sync(jpeg->dev);
+	if (ret < 0)
+		goto enc_end;
+
+	spin_lock_irqsave(&jpeg->hw_lock, flags);
+
+	/*
+	 * Resetting the hardware every frame is to ensure that all the
+	 * registers are cleared. This is a hardware requirement.
+	 */
+	mtk_jpeg_enc_reset(jpeg->reg_base);
+
+	mtk_jpeg_set_enc_dst(ctx, jpeg->reg_base, &dst_buf->vb2_buf, &enc_bs);
+	mtk_jpeg_set_enc_src(ctx, jpeg->reg_base, &src_buf->vb2_buf);
+	mtk_jpeg_enc_set_config(jpeg->reg_base, ctx->out_q.fmt->hw_format,
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_set_dec_dst(struct mtk_jpeg_ctx *ctx, · line 1092

Quoted 3 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -0
+				ctx->enable_exif, ctx->enc_quality,
+				ctx->restart_interval);
+	mtk_jpeg_enc_start(jpeg->reg_base);

Could we just move the above 5 functions into one function inside mtk_jpeg_enc_hw.c that takes mtk_jpeg_dev pointer as its argument, let's say mtk_jpeg_enc_hw_run() and simply program all the data to the registers directly, without the extra level of abstractions?

done.

Quoted 19 lines
Quoted 19 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +11 -0
+	spin_unlock_irqrestore(&jpeg->hw_lock, flags);
+	return;
+
+enc_end:
+	v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+	v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+	v4l2_m2m_buf_done(src_buf, buf_state);
+	v4l2_m2m_buf_done(dst_buf, buf_state);
+	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
+}
+
 static void mtk_jpeg_dec_device_run(void *priv)
 {
 	struct mtk_jpeg_ctx *ctx = priv;
@@ -785,6 +1150,11 @@ static void mtk_jpeg_dec_device_run(void *priv)
 	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
 }
 
Quoted 7 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static void mtk_jpeg_dec_device_run(void *priv) · line 1157

Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -0
+static int mtk_jpeg_enc_job_ready(void *priv)
+{
+		return 1;
+}
+

The callback is optional, so can be just removed if it always returns 1.

done.

Quoted 48 lines
Quoted 48 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +29 -1
 static int mtk_jpeg_dec_job_ready(void *priv)
 {
 	struct mtk_jpeg_ctx *ctx = priv;
@@ -792,6 +1162,11 @@ static int mtk_jpeg_dec_job_ready(void *priv)
 	return (ctx->state == MTK_JPEG_RUNNING) ? 1 : 0;
 }
 
+static const struct v4l2_m2m_ops mtk_jpeg_enc_m2m_ops = {
+	.device_run = mtk_jpeg_enc_device_run,
+	.job_ready  = mtk_jpeg_enc_job_ready,
+};
+
 static const struct v4l2_m2m_ops mtk_jpeg_dec_m2m_ops = {
 	.device_run = mtk_jpeg_dec_device_run,
 	.job_ready  = mtk_jpeg_dec_job_ready,
@@ -830,24 +1205,109 @@ static int mtk_jpeg_dec_queue_init(void *priv, struct vb2_queue *src_vq,
 	return ret;
 }
 
-static void mtk_jpeg_clk_on(struct mtk_jpeg_dev *jpeg)
+static int mtk_jpeg_enc_queue_init(void *priv, struct vb2_queue *src_vq,
+				   struct vb2_queue *dst_vq)
 {
+	struct mtk_jpeg_ctx *ctx = priv;
 	int ret;
 
+	src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+	src_vq->io_modes = VB2_DMABUF | VB2_MMAP;
+	src_vq->drv_priv = ctx;
+	src_vq->buf_struct_size = sizeof(struct mtk_jpeg_src_buf);
+	src_vq->ops = &mtk_jpeg_enc_qops;
+	src_vq->mem_ops = &vb2_dma_contig_memops;
+	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	src_vq->lock = &ctx->jpeg->lock;
+	src_vq->dev = ctx->jpeg->dev;
+	ret = vb2_queue_init(src_vq);
+	if (ret)
+		return ret;
+
+	dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+	dst_vq->io_modes = VB2_DMABUF | VB2_MMAP;
+	dst_vq->drv_priv = ctx;
+	dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
+	dst_vq->ops = &mtk_jpeg_enc_qops;
+	dst_vq->mem_ops = &vb2_dma_contig_memops;
+	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	dst_vq->lock = &ctx->jpeg->lock;
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_queue_init(void *priv, struct vb2_queue *src_vq, · line 1239

Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -0
+	dst_vq->dev = ctx->jpeg->dev;
+	ret = vb2_queue_init(dst_vq);
+
+	return ret;
+}

This only differs in "ops" from the decoder implementation. Perhaps both functions could be merged?

done.

Quoted 18 lines
Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -0
+
+static void mtk_jpeg_clk_on(struct mtk_jpeg_dev *jpeg)
+{

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_queue_init(void *priv, struct vb2_queue *src_vq, · line 1250

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -2
+	int ret, i;
+
 	ret = mtk_smi_larb_get(jpeg->larb);
 	if (ret)
 		dev_err(jpeg->dev, "mtk_smi_larb_get larbvdec fail %d\n", ret);
-	clk_prepare_enable(jpeg->clk_jdec_smi);
-	clk_prepare_enable(jpeg->clk_jdec);
+
+	for (i = 0; i < jpeg->variant->num_clocks; i++) {
+		ret = clk_prepare_enable(jpeg->clocks[i]);

Instead of an open coded loop, could the clk_bulk_*() helpers be used instead? (Look for devm_clk_bulk_get() and devm_clk_bulk_prepare_enable().)

Quoted 6 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_queue_init(void *priv, struct vb2_queue *src_vq, · line 1253

Quoted 3 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -0
+		if (ret) {
+			while (--i >= 0)
+				clk_disable_unprepare(jpeg->clocks[i]);

nit: The typical convention is to do error handling in an error path on the bottom of the function.

done

Quoted 2 lines

Also, it would be nice to print an error message.

done.

Quoted 5 lines
Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +2 -0
+		}
+	}
 }
 
Quoted 10 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_queue_init(void *priv, struct vb2_queue *src_vq, · line 1263

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -2
 static void mtk_jpeg_clk_off(struct mtk_jpeg_dev *jpeg)
 {
-	clk_disable_unprepare(jpeg->clk_jdec);
-	clk_disable_unprepare(jpeg->clk_jdec_smi);
+	int i;
+
+	for (i = jpeg->variant->num_clocks - 1; i >= 0; i--)
+		clk_disable_unprepare(jpeg->clocks[i]);

Ditto.

done.

Quoted 20 lines
Quoted 20 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +16 -0
 	mtk_smi_larb_put(jpeg->larb);
 }
 
+static irqreturn_t mtk_jpeg_enc_irq(int irq, void *priv)
+{
+	struct mtk_jpeg_dev *jpeg = priv;
+	struct mtk_jpeg_ctx *ctx;
+	struct vb2_v4l2_buffer *src_buf, *dst_buf;
+	struct mtk_jpeg_src_buf *jpeg_src_buf;
+	enum vb2_buffer_state buf_state = VB2_BUF_STATE_ERROR;
+	u32 enc_irq_ret;
+	u32 enc_ret, result_size;
+
+	ctx = v4l2_m2m_get_curr_priv(jpeg->m2m_dev);
+	if (!ctx) {
+		v4l2_err(&jpeg->v4l2_dev, "Context is NULL\n");
+		return IRQ_HANDLED;
+	}
+
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_queue_init(void *priv, struct vb2_queue *src_vq, · line 1287

Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +5 -0
+	src_buf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+	dst_buf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+	jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(&src_buf->vb2_buf);
+
+	enc_ret = mtk_jpeg_enc_get_and_clear_int_status(jpeg->reg_base);

We should check the interrupt status at the beginning of the function and return IRQ_NONE if there wasn't any flag set.

done.

Quoted 14 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_queue_init(void *priv, struct vb2_queue *src_vq, · line 1296

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +9 -0
+	enc_irq_ret = mtk_jpeg_enc_enum_result(jpeg->reg_base, enc_ret);
+
+	if (enc_irq_ret >= MTK_JPEG_ENC_RESULT_STALL)
+		mtk_jpeg_enc_reset(jpeg->reg_base);
+
+	if (enc_irq_ret != MTK_JPEG_ENC_RESULT_DONE) {
+		dev_err(jpeg->dev, "encode failed\n");
+		goto enc_end;
+	}

As I suggested before, it would have been much more clear if the interrupt status bits were just directly read and checked in this function, without introducing the additional abstraction.

done.

Quoted 45 lines
Quoted 19 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +14 -0
+
+	result_size = mtk_jpeg_enc_get_file_size(jpeg->reg_base);
+	vb2_set_plane_payload(&dst_buf->vb2_buf, 0, result_size);
+
+	buf_state = VB2_BUF_STATE_DONE;
+
+enc_end:
+	v4l2_m2m_buf_done(src_buf, buf_state);
+	v4l2_m2m_buf_done(dst_buf, buf_state);
+	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
+	pm_runtime_put(ctx->jpeg->dev);
+	return IRQ_HANDLED;
+}
+
 static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv)
 {
 	struct mtk_jpeg_dev *jpeg = priv;

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1361

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
@@ -893,36 +1353,130 @@ static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv)
 	return IRQ_HANDLED;
 }
 
+static void mtk_jpeg_set_enc_default_params(struct mtk_jpeg_ctx *ctx)
+{
+	struct mtk_jpeg_q_data *q = &ctx->out_q;
+	struct v4l2_pix_format_mplane *pix_mp;
+
+	pix_mp = kmalloc(sizeof(*pix_mp), GFP_KERNEL);

Do we need to allocate this pix_mp? Could we instead just embed it inside ctx?

Also, this is actually a memory leak, because I don't see this structure saved anywhere or freed.

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1367

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
+
+	ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+	ctx->colorspace = V4L2_COLORSPACE_JPEG,
+	ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
+	ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;

Since we already have a v4l2_pix_format_mplane struct which has fields for the above 4 values, could we just store them there?

Quoted 6 lines

Also, I don't see this driver handling the colorspaces in any way, but it seems to allow changing them from the userspace. This is incorrect, because the userspace has no way to know that the colorspace is not handled. Instead, the try_fmt implementation should always override the userspace-provided colorspace configuration with the ones that the driver assumes.

did as what we discussed earlier.

Quoted 19 lines
Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -0
+	pix_mp->width = MTK_JPEG_MIN_WIDTH;
+	pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1380

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +10 -0
+	q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_YUYV,
+				      MTK_JPEG_FMT_FLAG_ENC_OUTPUT);
+	vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+				    fmt.pix_mp), q->fmt);
+	q->w = pix_mp->width;
+	q->h = pix_mp->height;
+	q->crop_rect.width = pix_mp->width;
+	q->crop_rect.height = pix_mp->height;
+	q->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
+	q->bytesperline[0] = pix_mp->plane_fmt[0].bytesperline;

Actually, do we need this custom mtk_jpeg_q_data struct? Why couldn't we just keep the same values inside the standard v4l2_pix_format_mplane struct?

Quoted 3 lines

In general it's preferred to use the standard kernel structures as much as possible and only introduce local driver structures for data that can't be stored in generic ones.

did as what we discussed earlier.

Quoted 15 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1392

Quoted 13 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +12 -0
+
+	q = &ctx->cap_q;
+	q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_JPEG,
+				      MTK_JPEG_FMT_FLAG_ENC_CAPTURE);
+	pix_mp->width = MTK_JPEG_MIN_WIDTH;
+	pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+	vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+				    fmt.pix_mp), q->fmt);
+	q->w = pix_mp->width;
+	q->h = pix_mp->height;
+	q->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
+	q->bytesperline[0] = pix_mp->plane_fmt[0].bytesperline;

Ditto.

done.

Quoted 60 lines
Quoted 47 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +23 -12
+}
+
 static void mtk_jpeg_set_dec_default_params(struct mtk_jpeg_ctx *ctx)
 {
 	struct mtk_jpeg_q_data *q = &ctx->out_q;
+	struct v4l2_pix_format_mplane *pix_mp;
 	int i;
 
+	pix_mp = kmalloc(sizeof(*pix_mp), GFP_KERNEL);
+
+	ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
 	ctx->colorspace = V4L2_COLORSPACE_JPEG,
 	ctx->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
 	ctx->quantization = V4L2_QUANTIZATION_DEFAULT;
 	ctx->xfer_func = V4L2_XFER_FUNC_DEFAULT;
-
-	q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_JPEG,
-					      MTK_JPEG_FMT_TYPE_OUTPUT);
-	q->w = MTK_JPEG_MIN_WIDTH;
-	q->h = MTK_JPEG_MIN_HEIGHT;
-	q->bytesperline[0] = 0;
-	q->sizeimage[0] = MTK_JPEG_DEFAULT_SIZEIMAGE;
+	pix_mp->width = MTK_JPEG_MIN_WIDTH;
+	pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+
+	q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_JPEG,
+				      MTK_JPEG_FMT_FLAG_DEC_OUTPUT);
+	vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+				    fmt.pix_mp), q->fmt);
+	q->w = pix_mp->width;
+	q->h = pix_mp->height;
+	q->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
+	q->bytesperline[0] = pix_mp->plane_fmt[0].bytesperline;
 
 	q = &ctx->cap_q;
-	q->fmt = mtk_jpeg_find_format(ctx, V4L2_PIX_FMT_YUV420M,
-					      MTK_JPEG_FMT_TYPE_CAPTURE);
-	q->w = MTK_JPEG_MIN_WIDTH;
-	q->h = MTK_JPEG_MIN_HEIGHT;
-
+	q->fmt = mtk_jpeg_find_format(V4L2_PIX_FMT_YUV420M,
+				      MTK_JPEG_FMT_FLAG_DEC_CAPTURE);
+	pix_mp->width = MTK_JPEG_MIN_WIDTH;
+	pix_mp->height = MTK_JPEG_MIN_HEIGHT;
+	vidioc_try_fmt(container_of(pix_mp, struct v4l2_format,
+				    fmt.pix_mp), q->fmt);

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1434

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -2
+	q->w = pix_mp->width;
+	q->h = pix_mp->height;
 	for (i = 0; i < q->fmt->colplanes; i++) {
-		u32 stride = q->w * q->fmt->h_sample[i] / 4;
-		u32 h = q->h * q->fmt->v_sample[i] / 4;
+		q->sizeimage[i] = pix_mp->plane_fmt[i].sizeimage;
+		q->bytesperline[i] = pix_mp->plane_fmt[i].bytesperline;
+	}
+}
 

Same comments as for the encoder version.

Quoted 2 lines

On top of that, both functions are almost identical and it should be possible to merge them.

done.

Quoted 45 lines
Quoted 45 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +41 -2
-		q->bytesperline[i] = stride;
-		q->sizeimage[i] = stride * h;
+static int mtk_jpeg_enc_open(struct file *file)
+{
+	struct mtk_jpeg_dev *jpeg = video_drvdata(file);
+	struct video_device *vfd = video_devdata(file);
+	struct mtk_jpeg_ctx *ctx;
+	int ret = 0;
+
+	ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+	if (!ctx)
+		return -ENOMEM;
+
+	if (mutex_lock_interruptible(&jpeg->lock)) {
+		ret = -ERESTARTSYS;
+		goto free;
+	}
+
+	v4l2_fh_init(&ctx->fh, vfd);
+	file->private_data = &ctx->fh;
+	v4l2_fh_add(&ctx->fh);
+
+	ctx->jpeg = jpeg;
+	ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(jpeg->m2m_dev, ctx,
+					    mtk_jpeg_enc_queue_init);
+	if (IS_ERR(ctx->fh.m2m_ctx)) {
+		ret = PTR_ERR(ctx->fh.m2m_ctx);
+		goto error;
 	}
+
+	ret = mtk_jpeg_enc_ctrls_setup(ctx);
+	if (ret) {
+		v4l2_err(&jpeg->v4l2_dev, "Failed to setup jpeg enc controls\n");
+		goto error;
+	}
+	mtk_jpeg_set_enc_default_params(ctx);
+
+	mutex_unlock(&jpeg->lock);
+	return 0;
+
+error:
+	v4l2_fh_del(&ctx->fh);
+	v4l2_fh_exit(&ctx->fh);
+	mutex_unlock(&jpeg->lock);
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) · line 1480

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -0
+free:
+	kfree(ctx);
+	return ret;
 }

It looks like the queue_init argument to v4l2_m2m_ctx_init() and control handling would be the only differences from the decoder version. Perhaps the functions can be merged?

done.

Quoted 7 lines
Quoted 7 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +0 -0
 static int mtk_jpeg_dec_open(struct file *file)
@@ -953,6 +1507,12 @@ static int mtk_jpeg_dec_open(struct file *file)
 		goto error;
 	}
 
Quoted 9 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_dec_open(struct file *file) · line 1515

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
+	v4l2_ctrl_handler_init(&ctx->ctrl_hdl, 0);
+	ret = v4l2_ctrl_handler_setup(&ctx->ctrl_hdl);
+	if (ret) {
+		v4l2_err(&jpeg->v4l2_dev, "Failed to setup jpeg dec controls\n");
+		goto error;
+	}

There are no controls for the decoder, so there should be no need to set up a control handler.

done.

Quoted 16 lines
Quoted 16 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +1 -0
 	mtk_jpeg_set_dec_default_params(ctx);
 	mutex_unlock(&jpeg->lock);
 	return 0;
@@ -973,6 +1533,7 @@ static int mtk_jpeg_release(struct file *file)
 
 	mutex_lock(&jpeg->lock);
 	v4l2_m2m_ctx_release(ctx->fh.m2m_ctx);
+	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
 	v4l2_fh_del(&ctx->fh);
 	v4l2_fh_exit(&ctx->fh);
 	kfree(ctx);
@@ -980,6 +1541,15 @@ static int mtk_jpeg_release(struct file *file)
 	return 0;
 }
 
Quoted 12 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_release(struct file *file) · line 1552

Quoted 9 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +9 -0
+static const struct v4l2_file_operations mtk_jpeg_enc_fops = {
+	.owner          = THIS_MODULE,
+	.open           = mtk_jpeg_enc_open,
+	.release        = mtk_jpeg_release,
+	.poll           = v4l2_m2m_fop_poll,
+	.unlocked_ioctl = video_ioctl2,
+	.mmap           = v4l2_m2m_fop_mmap,
+};
+

If we merge the .open() implementation, the same struct could be used for both decoder and encoder.

done.

Quoted 6 lines

[snip]

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +0 -0
@@ -1042,8 +1619,12 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 		return jpeg_irq;
 	}
 
Quoted 11 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_probe(struct platform_device *pdev) · line 1627

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -2
-	ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_dec_irq, 0,
-			       pdev->name, jpeg);
+	if (jpeg->variant->is_encoder)
+		ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_enc_irq,
+				       0, pdev->name, jpeg);
+	else
+		ret = devm_request_irq(&pdev->dev, jpeg_irq, mtk_jpeg_dec_irq,
+				       0, pdev->name, jpeg);

Rather than having "is_encoder" in the variant struct, would it make more sense to have "irq_handler" instead? That would avoid the explicit if.

done.

Quoted 8 lines
Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +0 -0
 	if (ret) {
 		dev_err(&pdev->dev, "Failed to request jpeg_irq %d (%d)\n",
 			jpeg_irq, ret);
@@ -1063,7 +1644,10 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 		goto err_dev_register;
 	}
 
Quoted 7 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_probe(struct platform_device *pdev) · line 1650

Quoted 5 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +4 -1
-	jpeg->m2m_dev = v4l2_m2m_init(&mtk_jpeg_dec_m2m_ops);
+	if (jpeg->variant->is_encoder)
+		jpeg->m2m_dev = v4l2_m2m_init(&mtk_jpeg_enc_m2m_ops);
+	else
+		jpeg->m2m_dev = v4l2_m2m_init(&mtk_jpeg_dec_m2m_ops);

Same here. The variant struct could have a "m2m_ops" pointer.

done.

Quoted 14 lines
Quoted 14 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +3 -3
 	if (IS_ERR(jpeg->m2m_dev)) {
 		v4l2_err(&jpeg->v4l2_dev, "Failed to init mem2mem device\n");
 		ret = PTR_ERR(jpeg->m2m_dev);
@@ -1076,9 +1660,15 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
 		goto err_vfd_jpeg_alloc;
 	}
 	snprintf(jpeg->vdev->name, sizeof(jpeg->vdev->name),
-		 "%s-dec", MTK_JPEG_NAME);
-	jpeg->vdev->fops = &mtk_jpeg_dec_fops;
-	jpeg->vdev->ioctl_ops = &mtk_jpeg_dec_ioctl_ops;
+		 "%s-%s", MTK_JPEG_NAME,
+		 jpeg->variant->is_encoder ? "enc" : "dec");
+	if (jpeg->variant->is_encoder) {
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c static int mtk_jpeg_probe(struct platform_device *pdev) · line 1671

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c +6 -0
+		jpeg->vdev->fops = &mtk_jpeg_enc_fops;
+		jpeg->vdev->ioctl_ops = &mtk_jpeg_enc_ioctl_ops;
+	} else {
+		jpeg->vdev->fops = &mtk_jpeg_dec_fops;
+		jpeg->vdev->ioctl_ops = &mtk_jpeg_dec_ioctl_ops;
+	}

Similarly here.

done.

Quoted 17 lines

[snip]

Quoted 15 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +1 -0
index 0b59e48495d5..9ec2c3350a16 100644
@@ -3,6 +3,7 @@
  * Copyright (c) 2016 MediaTek Inc.
  * Author: Ming Hsiu Tsai <[email protected]>
  *         Rick Chang <[email protected]>
+ *         Xia Jiang <[email protected]>
  */
 
 #ifndef _MTK_JPEG_CORE_H
@@ -16,19 +17,21 @@
 #define MTK_JPEG_NAME		"mtk-jpeg"
 
Quoted 6 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 22

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +2 -0
 #define MTK_JPEG_COMP_MAX		3
+#define MTK_JPEG_MAX_CLOCKS		2
+
 

Duplicate blank line.

removed.

Quoted 10 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 26

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +2 -3
 #define MTK_JPEG_FMT_FLAG_DEC_OUTPUT	BIT(0)
 #define MTK_JPEG_FMT_FLAG_DEC_CAPTURE	BIT(1)
-
-#define MTK_JPEG_FMT_TYPE_OUTPUT	1
-#define MTK_JPEG_FMT_TYPE_CAPTURE	2
+#define MTK_JPEG_FMT_FLAG_ENC_OUTPUT	BIT(2)
+#define MTK_JPEG_FMT_FLAG_ENC_CAPTURE	BIT(3)

Do we need separate bits for decoder and encoder?

no need, I have removed it.

Quoted 11 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 31

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +2 -2
 #define MTK_JPEG_MIN_WIDTH	32U
 #define MTK_JPEG_MIN_HEIGHT	32U
-#define MTK_JPEG_MAX_WIDTH	8192U
-#define MTK_JPEG_MAX_HEIGHT	8192U
+#define MTK_JPEG_MAX_WIDTH	65535U
+#define MTK_JPEG_MAX_HEIGHT	65535U

If this is a change valid for the decoder too, it should be a separate patch.

done.

Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h line 34

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +1 -0
 #define MTK_JPEG_DEFAULT_SIZEIMAGE	(1 * 1024 * 1024)
+#define MTK_JPEG_MAX_EXIF_SIZE	(64 * 1024)

There is one thing that I realized now. If the EXIF mode is enabled, the driver needs to ensure that the buffer is big enough to hold the EXIF data. The vb2 .buf_prepare callback would be the right place to do that.

done.

Quoted 8 lines
Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +0 -0
 /**
  * enum mtk_jpeg_ctx_state - states of the context state machine
@@ -42,6 +45,18 @@ enum mtk_jpeg_ctx_state {
 	MTK_JPEG_SOURCE_CHANGE,
 };
 
Quoted 14 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h enum mtk_jpeg_ctx_state { · line 57

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h +10 -0
+/**
+ * mtk_jpeg_variant - mtk jpeg driver variant
+ * @is_encoder:		driver mode is jpeg encoder
+ * @clk_names:		clock names
+ * @num_clocks:		numbers of clock
+ */
+struct mtk_jpeg_variant {
+	bool is_encoder;
+	const char		*clk_names[MTK_JPEG_MAX_CLOCKS];
+	int			num_clocks;

hint: Please avoid tabs between types and names, as it makes it difficult to add new fields later (the number of tabs might need to change for all members).

done.

Quoted 44 lines

[snip]

Quoted 28 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +22 -0
new file mode 100644
index 000000000000..7fc1de920a75
@@ -0,0 +1,193 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ * Author: Xia Jiang <[email protected]>
+ *
+ */
+
+#include <linux/io.h>
+#include <linux/kernel.h>
+#include <media/videobuf2-core.h>
+
+#include "mtk_jpeg_enc_hw.h"
+
+static const struct mtk_jpeg_enc_qlt mtk_jpeg_enc_quality[] = {
+	{.quality_param = 34, .hardware_value = JPEG_ENC_QUALITY_Q34},
+	{.quality_param = 39, .hardware_value = JPEG_ENC_QUALITY_Q39},
+	{.quality_param = 48, .hardware_value = JPEG_ENC_QUALITY_Q48},
+	{.quality_param = 60, .hardware_value = JPEG_ENC_QUALITY_Q60},
+	{.quality_param = 64, .hardware_value = JPEG_ENC_QUALITY_Q64},
+	{.quality_param = 68, .hardware_value = JPEG_ENC_QUALITY_Q68},
+	{.quality_param = 74, .hardware_value = JPEG_ENC_QUALITY_Q74},
+	{.quality_param = 80, .hardware_value = JPEG_ENC_QUALITY_Q80},

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 32

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +10 -0
+	{.quality_param = 82, .hardware_value = JPEG_ENC_QUALITY_Q82},
+	{.quality_param = 84, .hardware_value = JPEG_ENC_QUALITY_Q84},
+	{.quality_param = 87, .hardware_value = JPEG_ENC_QUALITY_Q87},
+	{.quality_param = 90, .hardware_value = JPEG_ENC_QUALITY_Q90},
+	{.quality_param = 92, .hardware_value = JPEG_ENC_QUALITY_Q92},
+	{.quality_param = 95, .hardware_value = JPEG_ENC_QUALITY_Q95},
+	{.quality_param = 97, .hardware_value = JPEG_ENC_QUALITY_Q97},
+};
+
+void mtk_jpeg_enc_reset(void __iomem *base)

I'd suggest passing struct mtk_jpeg_dev pointer to all these functions, in case more data about the hardware is needed in the future.

Quoted 4 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 34

Quoted 2 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +2 -0
+{
+	writel(0x00, base + JPEG_ENC_RSTB);

nit: Just 0 is enough.

done.

Quoted 5 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 36

Quoted 3 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +2 -0
+	writel(JPEG_ENC_RESET_BIT, base + JPEG_ENC_RSTB);
+	writel(0x00, base + JPEG_ENC_CODEC_SEL);

Ditto.

done.

Quoted 43 lines
Quoted 20 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +19 -0
+}
+
+u32 mtk_jpeg_enc_get_and_clear_int_status(void __iomem *base)
+{
+	u32 ret;
+
+	ret = readl(base + JPEG_ENC_INT_STS) &
+		    JPEG_ENC_INT_STATUS_MASK_ALLIRQ;
+	if (ret)
+		writel(0, base + JPEG_ENC_INT_STS);
+
+	return ret;
+}
+
+u32 mtk_jpeg_enc_get_file_size(void __iomem *base)
+{
+	return readl(base + JPEG_ENC_DMA_ADDR0) -
+	       readl(base + JPEG_ENC_DST_ADDR0);
+}

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 65

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +10 -0
+
+u32 mtk_jpeg_enc_enum_result(void __iomem *base, u32 irq_status)
+{
+	if (irq_status & JPEG_ENC_INT_STATUS_DONE)
+		return MTK_JPEG_ENC_RESULT_DONE;
+	else if (irq_status & JPEG_ENC_INT_STATUS_STALL)
+		return MTK_JPEG_ENC_RESULT_STALL;
+	else
+		return MTK_JPEG_ENC_RESULT_VCODEC_IRQ;
+}

It looks like the driver only cares about 2 cases: DONE and anything else. Actually it also cares about a case where no bits are set in irq_status, which could be an interrupt controller misconfiguration or an interrupt generated by another device on the same shared IRQ line, which should be handled by returning IRQ_NONE. Since interrupt handling is a low level hardware operation, I'd suggest another design:
- put the irq_handler_t function here in this file and have it directly access the hardware registers and check the interrupt bits,
- add an mtk_jpeg_enc_done(..., enum vb2_buffer_state state, size_t payload), which would be called from this low level interrupt handler and do the V4L2 buffer and M2M job handling.

Quoted 1 line

WDYT?

done.

Quoted 9 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 67

Quoted 3 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +2 -0
+
+void mtk_jpeg_enc_set_img_size(void __iomem *base, u32 width, u32 height)

If we pass struct mtk_jpeg_dev to this function, the width and height arguments wouldn't be necessary, as they could be directly retrieved from the driver state.

Similar advice applies to the other functions in this file.

done.

Quoted 27 lines
Quoted 27 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +26 -0
+{
+	u32 value;
+
+	value = width << 16 | height;
+	writel(value, base + JPEG_ENC_IMG_SIZE);
+}
+
+void mtk_jpeg_enc_set_blk_num(void __iomem *base, u32 enc_format, u32 width,
+			      u32 height)
+{
+	u32 blk_num;
+	u32 is_420;
+	u32 padding_width;
+	u32 padding_height;
+	u32 luma_blocks;
+	u32 chroma_blocks;
+
+	is_420 = (enc_format == V4L2_PIX_FMT_NV12M ||
+		  enc_format == V4L2_PIX_FMT_NV21M) ? 1 : 0;
+	padding_width = round_up(width, 16);
+	padding_height = round_up(height, is_420 ? 16 : 8);
+
+	luma_blocks = padding_width / 8 * padding_height / 8;
+	if (is_420)
+		chroma_blocks = luma_blocks / 4;
+	else
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 99

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +6 -0
+		chroma_blocks = luma_blocks / 2;
+
+	blk_num = luma_blocks + 2 * chroma_blocks - 1;
+
+	writel(blk_num, base + JPEG_ENC_BLK_NUM);
+}

My comments for this function from v7 haven't been addressed.

done.

Quoted 15 lines
Quoted 15 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +14 -0
+
+void mtk_jpeg_enc_set_stride(void __iomem *base, u32 enc_format, u32 width,
+			     u32 height, u32 bytesperline)
+{
+	u32 img_stride;
+	u32 mem_stride;
+
+	if (enc_format == V4L2_PIX_FMT_NV12M ||
+	    enc_format == V4L2_PIX_FMT_NV21M) {
+		img_stride = round_up(width, 16);
+		mem_stride = bytesperline;
+	} else {
+		img_stride = round_up(width * 2, 32);
+		mem_stride = img_stride;
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 119

Quoted 6 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +6 -0
+	}
+
+	writel(img_stride, base + JPEG_ENC_IMG_STRIDE);
+	writel(mem_stride, base + JPEG_ENC_STRIDE);
+}
+

My comments for this function from v7 haven't been addressed.

done.

Quoted 11 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 126

Quoted 8 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +7 -0
+void mtk_jpeg_enc_set_src_addr(void __iomem *base, u32 src_addr,
+			       u32 plane_index)
+{
+	if (!plane_index)
+		writel(src_addr, base + JPEG_ENC_SRC_LUMA_ADDR);
+	else
+		writel(src_addr, base + JPEG_ENC_SRC_CHROMA_ADDR);

Do we need this plane_index if we only have 2 planes? Also, for interleaved formats, like YUYV, what should the LUMA and CHROMA registers be set to?

If the format is YUYV, there is no need to set CHROMA register. Hardware can compute the CHROMA address from the LUMA address itself.

Quoted 23 lines
Quoted 23 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +22 -0
+}
+
+void mtk_jpeg_enc_set_dst_addr(void __iomem *base, u32 dst_addr,
+			       u32 stall_size, u32 init_offset,
+			       u32 offset_mask)
+{
+	writel(init_offset & ~0xf, base + JPEG_ENC_OFFSET_ADDR);
+	writel(offset_mask & 0xf, base + JPEG_ENC_BYTE_OFFSET_MASK);
+	writel(dst_addr & ~0xf, base + JPEG_ENC_DST_ADDR0);
+	writel((dst_addr + stall_size) & ~0xf, base + JPEG_ENC_STALL_ADDR0);
+}
+
+static void mtk_jpeg_enc_set_quality(void __iomem *base, u32 quality)
+{
+	u32 value;
+	u32 i, enc_quality;
+
+	enc_quality = mtk_jpeg_enc_quality[0].hardware_value;
+	for (i = 0; i < ARRAY_SIZE(mtk_jpeg_enc_quality); i++) {
+		if (quality <= mtk_jpeg_enc_quality[i].quality_param) {
+			enc_quality = mtk_jpeg_enc_quality[i].hardware_value;
+			break;
Quoted 8 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 152

Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +4 -0
+		}
+	}
+
+	value = readl(base + JPEG_ENC_QUALITY);

nit: Is it still necessary to read the register here? Typically the reserved bits would be defined as SBZ (should be zero) and it should be safe to just write 0 to them.

yes,removed it.

Quoted 13 lines

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 162

Quoted 11 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +10 -0
+	value = (value & JPEG_ENC_QUALITY_MASK) | enc_quality;
+	writel(value, base + JPEG_ENC_QUALITY);
+}
+
+static void mtk_jpeg_enc_set_ctrl(void __iomem *base, u32 enc_format,
+				  bool exif_en, u32 restart_interval)
+{
+	u32 value;
+
+	value = readl(base + JPEG_ENC_CTRL);

nit: Is it still necessary to read the register here?

I think it is necessary to read this register.Because that the default value of JPEG_ENC_CTRL's bit[2](enable interrupt) is 1.

Quoted 17 lines
Quoted 4 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +3 -0
+	value &= ~JPEG_ENC_CTRL_YUV_FORMAT_MASK;
+	value |= (enc_format & 3) << 3;
+	if (exif_en)

drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c line 175

Quoted 10 lines
drivers/media/platform/mtk-jpeg/mtk_jpeg_enc_hw.c new file +10 -0
+		value |= JPEG_ENC_CTRL_FILE_FORMAT_BIT;
+	else
+		value &= ~JPEG_ENC_CTRL_FILE_FORMAT_BIT;
+	if (restart_interval)
+		value |= JPEG_ENC_CTRL_RESTART_EN_BIT;
+	else
+		value &= ~JPEG_ENC_CTRL_RESTART_EN_BIT;
+	writel(value, base + JPEG_ENC_CTRL);
+}
+

Best regards,
Tomasz

Best Regards,
Xia Jiang