media: amphion: add support for contiguous format

PATCH Superseded Ming Qian 3 participants 20 messages Aug 9, 2022 – Aug 22, 2022

Subsystems (2)

Media (V4L/DVB) Amphion VPU codec V4L2 driver

lore

Share & tools

lore

Ming Qian

author maintainer

2022-08-09 06:51 UTC #0 lore raw

v4 cover media: amphion: add support for contiguous format

amphion vpu support non contiguous planes yuv format, but when we make pipeline with other device, the device may only support contiguous format, for example, amphion encoder support NV12M, but the capture device may only support NV12. we can't connect them directly.

So to improve compatibility, driver can add support for contiguous format in the same time.

Then encoder can support NV12 and NV12M,
decoder can support NV12_8L128 and NV12M_8L128.

v4
- set bytesperline of compressed formats to 0

v3
- negotiate bytesperline with user
- zero the reserved of v4l2_format

v2
-- fix an uninitialized array index read error
-- remove an used variable

Ming Qian (4):
media: add nv12_8l128 and nv12_10be_8l128 video format.
media: amphion: tell and handle contiguous and non contiguous format
media: amphion: decoder add support for contiguous planes
media: amphion: encoder add support for contiguous planes

 .../media/v4l/pixfmt-yuv-planar.rst           |   8 +
 drivers/media/platform/amphion/vdec.c         | 203 ++++++++++--------
 drivers/media/platform/amphion/venc.c         |  41 ++--
 drivers/media/platform/amphion/vpu.h          |   4 +-
 drivers/media/platform/amphion/vpu_dbg.c      |   8 +-
 drivers/media/platform/amphion/vpu_helpers.c  |  48 ++++-
 drivers/media/platform/amphion/vpu_helpers.h  |   2 +
 drivers/media/platform/amphion/vpu_malone.c   |   3 +-
 drivers/media/platform/amphion/vpu_v4l2.c     | 168 +++++++++++----
 drivers/media/platform/amphion/vpu_v4l2.h     |   3 +-
 drivers/media/platform/amphion/vpu_windsor.c  |   8 +-
 drivers/media/v4l2-core/v4l2-ioctl.c          |   2 +
 include/uapi/linux/videodev2.h                |   2 +
 13 files changed, 335 insertions(+), 165 deletions(-)

Signature · 1 line

2.37.1

Ming Qian

author maintainer

2022-08-09 06:51 UTC #1 lore raw

PATCH v4 1/4 media: add nv12_8l128 and nv12_10be_8l128 video format.

add contiguous nv12 tiled format nv12_8l128 and nv12_10be_8l128

Signed-off-by Ming Qian <[email protected]>

Reviewed-by Nicolas Dufresne <[email protected]>

---
 .../userspace-api/media/v4l/pixfmt-yuv-planar.rst         | 8 ++++++++
 drivers/media/v4l2-core/v4l2-ioctl.c                      | 2 ++
 include/uapi/linux/videodev2.h                            | 2 ++
 3 files changed, 12 insertions(+)

Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst +8 -0

index 10b1feeb0b57..f1d5bb7b806d 100644
@@ -273,7 +273,9 @@ of the luma plane.
 .. _V4L2-PIX-FMT-NV12-16L16:
 .. _V4L2-PIX-FMT-NV12-32L32:
 .. _V4L2-PIX-FMT-NV12M-8L128:
+.. _V4L2-PIX-FMT-NV12-8L128:
 .. _V4L2-PIX-FMT-NV12M-10BE-8L128:
+.. _V4L2-PIX-FMT-NV12-10BE-8L128:
 .. _V4L2-PIX-FMT-MM21:
 
 Tiled NV12

@@ -319,6 +321,9 @@ pixels in 2D 8x128 tiles, and stores tiles linearly in memory.
 The image height must be aligned to a multiple of 128.
 The layouts of the luma and chroma planes are identical.
 
+``V4L2_PIX_FMT_NV12_8L128`` is similar to ``V4L2_PIX_FMT_NV12M_8L128`` but stores
+two planes in one memory.
+
 ``V4L2_PIX_FMT_NV12M_10BE_8L128`` is similar to ``V4L2_PIX_FMT_NV12M`` but stores
 10 bits pixels in 2D 8x128 tiles, and stores tiles linearly in memory.
 the data is arranged in big endian order.

@@ -334,6 +339,9 @@ byte 2: Y1(bits 3-0) Y2(bits 9-6)
 byte 3: Y2(bits 5-0) Y3(bits 9-8)
 byte 4: Y3(bits 7-0)
 
+``V4L2_PIX_FMT_NV12_10BE_8L128`` is similar to ``V4L2_PIX_FMT_NV12M_10BE_8L128`` but stores
+two planes in one memory.
+
 ``V4L2_PIX_FMT_MM21`` store luma pixel in 16x32 tiles, and chroma pixels
 in 16x16 tiles. The line stride must be aligned to a multiple of 16 and the
 image height must be aligned to a multiple of 32. The number of luma and chroma

drivers/media/v4l2-core/v4l2-ioctl.c +2 -0

index c314025d977e..d973bd2ff750 100644
@@ -1444,7 +1444,9 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_META_FMT_VIVID:       descr = "Vivid Metadata"; break;
 	case V4L2_META_FMT_RK_ISP1_PARAMS:	descr = "Rockchip ISP1 3A Parameters"; break;
 	case V4L2_META_FMT_RK_ISP1_STAT_3A:	descr = "Rockchip ISP1 3A Statistics"; break;
+	case V4L2_PIX_FMT_NV12_8L128:	descr = "NV12 (8x128 Linear)"; break;
 	case V4L2_PIX_FMT_NV12M_8L128:	descr = "NV12M (8x128 Linear)"; break;
+	case V4L2_PIX_FMT_NV12_10BE_8L128:	descr = "10-bit NV12 (8x128 Linear, BE)"; break;
 	case V4L2_PIX_FMT_NV12M_10BE_8L128:	descr = "10-bit NV12M (8x128 Linear, BE)"; break;
 
 	default:

include/uapi/linux/videodev2.h +2 -0

index cd66e01ed3c3..64f16490dd2b 100644
@@ -655,6 +655,8 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_NV12_16L16 v4l2_fourcc('H', 'M', '1', '2') /* 12  Y/CbCr 4:2:0 16x16 tiles */
 #define V4L2_PIX_FMT_NV12_32L32 v4l2_fourcc('S', 'T', '1', '2') /* 12  Y/CbCr 4:2:0 32x32 tiles */
 #define V4L2_PIX_FMT_P010_4L4 v4l2_fourcc('T', '0', '1', '0') /* 12  Y/CbCr 4:2:0 10-bit 4x4 macroblocks */
+#define V4L2_PIX_FMT_NV12_8L128       v4l2_fourcc('A', 'T', '1', '2') /* Y/CbCr 4:2:0 8x128 tiles */
+#define V4L2_PIX_FMT_NV12_10BE_8L128  v4l2_fourcc_be('A', 'X', '1', '2') /* Y/CbCr 4:2:0 10-bit 8x128 tiles */
 
 /* Tiled YUV formats, non contiguous planes */
 #define V4L2_PIX_FMT_NV12MT  v4l2_fourcc('T', 'M', '1', '2') /* 12  Y/CbCr 4:2:0 64x32 tiles */

Signature · 1 line

2.37.1

Tommaso Merciai

Reviewed-by on v4 1/4

2022-08-11 15:19 UTC #2 lore raw

Re: [PATCH v4 1/4] media: add nv12_8l128 and nv12_10be_8l128 video format.

Hi Ming,

Quoted 25 lines · Ming Qian

add contiguous nv12 tiled format nv12_8l128 and nv12_10be_8l128

Signed-off-by Ming Qian <[email protected]>

Reviewed-by Nicolas Dufresne <[email protected]>

---
 .../userspace-api/media/v4l/pixfmt-yuv-planar.rst         | 8 ++++++++
 drivers/media/v4l2-core/v4l2-ioctl.c                      | 2 ++
 include/uapi/linux/videodev2.h                            | 2 ++
 3 files changed, 12 insertions(+)

Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst +2 -0

index 10b1feeb0b57..f1d5bb7b806d 100644
@@ -273,7 +273,9 @@ of the luma plane.
 .. _V4L2-PIX-FMT-NV12-16L16:
 .. _V4L2-PIX-FMT-NV12-32L32:
 .. _V4L2-PIX-FMT-NV12M-8L128:
+.. _V4L2-PIX-FMT-NV12-8L128:
 .. _V4L2-PIX-FMT-NV12M-10BE-8L128:
+.. _V4L2-PIX-FMT-NV12-10BE-8L128:
 .. _V4L2-PIX-FMT-MM21:
 
 Tiled NV12

Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst pixels in 2D 8x128 tiles, and stores tiles linearly in memory. · line 326

Quoted 6 lines

Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst +3 -0

@@ -319,6 +321,9 @@ pixels in 2D 8x128 tiles, and stores tiles linearly in memory.
 The image height must be aligned to a multiple of 128.
 The layouts of the luma and chroma planes are identical.
 
+``V4L2_PIX_FMT_NV12_8L128`` is similar to ``V4L2_PIX_FMT_NV12M_8L128`` but stores
+two planes in one memory.
+

Don't know, maybe we need more details here?

Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst byte 2: Y1(bits 3-0) Y2(bits 9-6) · line 344

Quoted 10 lines

Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst +3 -0

 ``V4L2_PIX_FMT_NV12M_10BE_8L128`` is similar to ``V4L2_PIX_FMT_NV12M`` but stores
 10 bits pixels in 2D 8x128 tiles, and stores tiles linearly in memory.
 the data is arranged in big endian order.

@@ -334,6 +339,9 @@ byte 2: Y1(bits 3-0) Y2(bits 9-6)
 byte 3: Y2(bits 5-0) Y3(bits 9-8)
 byte 4: Y3(bits 7-0)
 
+``V4L2_PIX_FMT_NV12_10BE_8L128`` is similar to ``V4L2_PIX_FMT_NV12M_10BE_8L128`` but stores
+two planes in one memory.
+

here also?

Quoted 33 lines

Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst +0 -0

 ``V4L2_PIX_FMT_MM21`` store luma pixel in 16x32 tiles, and chroma pixels
 in 16x16 tiles. The line stride must be aligned to a multiple of 16 and the
 image height must be aligned to a multiple of 32. The number of luma and chroma

drivers/media/v4l2-core/v4l2-ioctl.c +2 -0

index c314025d977e..d973bd2ff750 100644
@@ -1444,7 +1444,9 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_META_FMT_VIVID:       descr = "Vivid Metadata"; break;
 	case V4L2_META_FMT_RK_ISP1_PARAMS:	descr = "Rockchip ISP1 3A Parameters"; break;
 	case V4L2_META_FMT_RK_ISP1_STAT_3A:	descr = "Rockchip ISP1 3A Statistics"; break;
+	case V4L2_PIX_FMT_NV12_8L128:	descr = "NV12 (8x128 Linear)"; break;
 	case V4L2_PIX_FMT_NV12M_8L128:	descr = "NV12M (8x128 Linear)"; break;
+	case V4L2_PIX_FMT_NV12_10BE_8L128:	descr = "10-bit NV12 (8x128 Linear, BE)"; break;
 	case V4L2_PIX_FMT_NV12M_10BE_8L128:	descr = "10-bit NV12M (8x128 Linear, BE)"; break;
 
 	default:

include/uapi/linux/videodev2.h +2 -0

index cd66e01ed3c3..64f16490dd2b 100644
@@ -655,6 +655,8 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_NV12_16L16 v4l2_fourcc('H', 'M', '1', '2') /* 12  Y/CbCr 4:2:0 16x16 tiles */
 #define V4L2_PIX_FMT_NV12_32L32 v4l2_fourcc('S', 'T', '1', '2') /* 12  Y/CbCr 4:2:0 32x32 tiles */
 #define V4L2_PIX_FMT_P010_4L4 v4l2_fourcc('T', '0', '1', '0') /* 12  Y/CbCr 4:2:0 10-bit 4x4 macroblocks */
+#define V4L2_PIX_FMT_NV12_8L128       v4l2_fourcc('A', 'T', '1', '2') /* Y/CbCr 4:2:0 8x128 tiles */
+#define V4L2_PIX_FMT_NV12_10BE_8L128  v4l2_fourcc_be('A', 'X', '1', '2') /* Y/CbCr 4:2:0 10-bit 8x128 tiles */
 
 /* Tiled YUV formats, non contiguous planes */
 #define V4L2_PIX_FMT_NV12MT  v4l2_fourcc('T', 'M', '1', '2') /* 12  Y/CbCr 4:2:0 64x32 tiles */

Signature · 1 line

2.37.1

For the other parts look's good to me.

Reviewed-by Tommaso Merciai <[email protected]>

Regards,
Tommaso

Signature · 10 lines

Tommaso Merciai
Embedded Linux Engineer
[email protected]
__________________________________

Amarula Solutions SRL
Via Le Canevare 30, 31100 Treviso, Veneto, IT
T. +39 042 243 5310
[email protected]
www.amarulasolutions.com

Ming Qian

author maintainer

2022-08-12 01:25 UTC #3 lore raw

RE: [EXT] Re: [PATCH v4 1/4] media: add nv12_8l128 and nv12_10be_8l128 video format.

Quoted 44 lines

From: Tommaso Merciai <[email protected]>
Sent: 2022年8月11日 23:19
To: Ming Qian <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; dl-linux-imx <[email protected]>; X.H. Bao <[email protected]>; [email protected]; [email protected]; [email protected]
Subject: [EXT] Re: [PATCH v4 1/4] media: add nv12_8l128 and nv12_10be_8l128 video format.
Caution: EXT Email
Hi Ming,
Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst pixels in 2D 8x128 tiles, and stores tiles linearly in
Quoted 27 lines · Ming Qian
add contiguous nv12 tiled format nv12_8l128 and nv12_10be_8l128
Signed-off-by Ming Qian <[email protected]>
Reviewed-by Nicolas Dufresne <[email protected]>
---
 .../userspace-api/media/v4l/pixfmt-yuv-planar.rst         | 8 ++++++++
 drivers/media/v4l2-core/v4l2-ioctl.c                      | 2 ++
 include/uapi/linux/videodev2.h                            | 2 ++
 3 files changed, 12 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst b/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst index 10b1feeb0b57..f1d5bb7b806d 100644
Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst +2 -0
@@ -273,7 +273,9 @@ of the luma plane.
 .. _V4L2-PIX-FMT-NV12-16L16:
 .. _V4L2-PIX-FMT-NV12-32L32:
 .. _V4L2-PIX-FMT-NV12M-8L128:
+.. _V4L2-PIX-FMT-NV12-8L128:
 .. _V4L2-PIX-FMT-NV12M-10BE-8L128:
+.. _V4L2-PIX-FMT-NV12-10BE-8L128:
 .. _V4L2-PIX-FMT-MM21:
 
 Tiled NV12
@@ -319,6 +321,9 @@ pixels in 2D 8x128 tiles, and stores tiles linearly in
memory.

Quoted 8 lines

Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst pixels in 2D 8x128 tiles, and stores tiles linearly in memory. · line 326
Quoted 6 lines
Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst +3 -0
 The image height must be aligned to a multiple of 128.
 The layouts of the luma and chroma planes are identical.
 
+``V4L2_PIX_FMT_NV12_8L128`` is similar to
+``V4L2_PIX_FMT_NV12M_8L128`` but stores two planes in one memory.
+
Don't know, maybe we need more details here?

Hi Tommaso,
I have added the non contiguous format V4L2_PIX_FMT_NV12M_8L128, they're almost same, but store two planes in contiguous memory or non contiguous memory, just like NV12 and NV12M, the descriptions are almost the same, so I just say V4L2_PIX_FMT_NV12_8L128 is similar to V4L2_PIX_FMT_NV12M_8L128.
Do I need to copy the full description from the V4L2_PIX_FMT_NV12M_8L128 to here?

Ming

Quoted 8 lines

Quoted 8 lines
Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst +0 -0
 ``V4L2_PIX_FMT_NV12M_10BE_8L128`` is similar to
``V4L2_PIX_FMT_NV12M`` but stores
10 bits pixels in 2D 8x128 tiles, and stores tiles linearly in memory. the data is arranged in big endian order.
Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst +0 -0
@@ -334,6 +339,9 @@ byte 2: Y1(bits 3-0) Y2(bits 9-6)  byte 3: Y2(bits
 5-0) Y3(bits 9-8)  byte 4: Y3(bits 7-0)
 

Quoted 6 lines

Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst byte 2: Y1(bits 3-0) Y2(bits 9-6) byte 3: Y2(bits · line 342
Quoted 2 lines
Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst +2 -0
+``V4L2_PIX_FMT_NV12_10BE_8L128`` is similar to
+``V4L2_PIX_FMT_NV12M_10BE_8L128`` but stores two planes in one
memory.
Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst byte 2: Y1(bits 3-0) Y2(bits 9-6) byte 3: Y2(bits · line 343
Quoted 1 line
Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst +1 -0
+
here also?

It's the same issue.

Quoted 73 lines

drivers/media/v4l2-core/v4l2-ioctl.c static void v4l_fill_fmtdesc(struct v4l2_fmtdesc
Quoted 11 lines
Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst +0 -0
 ``V4L2_PIX_FMT_MM21`` store luma pixel in 16x32 tiles, and chroma
pixels in 16x16 tiles. The line stride must be aligned to a multiple of 16 and the image height must be aligned to a multiple of 32. The number of luma and chroma diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index c314025d977e..d973bd2ff750 100644
drivers/media/v4l2-core/v4l2-ioctl.c +0 -0
@@ -1444,7 +1444,9 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc
*fmt)
drivers/media/v4l2-core/v4l2-ioctl.c static void v4l_fill_fmtdesc(struct v4l2_fmtdesc · line 1445
Quoted 2 lines
drivers/media/v4l2-core/v4l2-ioctl.c +0 -0
      case V4L2_META_FMT_VIVID:       descr = "Vivid Metadata"; break;
      case V4L2_META_FMT_RK_ISP1_PARAMS:      descr = "Rockchip
ISP1 3A Parameters"; break;
drivers/media/v4l2-core/v4l2-ioctl.c static void v4l_fill_fmtdesc(struct v4l2_fmtdesc · line 1446
Quoted 1 line
drivers/media/v4l2-core/v4l2-ioctl.c +0 -0
      case V4L2_META_FMT_RK_ISP1_STAT_3A:     descr = "Rockchip ISP1
3A Statistics"; break;
drivers/media/v4l2-core/v4l2-ioctl.c static void v4l_fill_fmtdesc(struct v4l2_fmtdesc · line 1447
Quoted 1 line
drivers/media/v4l2-core/v4l2-ioctl.c +1 -0
+     case V4L2_PIX_FMT_NV12_8L128:   descr = "NV12 (8x128 Linear)";
break;
drivers/media/v4l2-core/v4l2-ioctl.c static void v4l_fill_fmtdesc(struct v4l2_fmtdesc · line 1448
Quoted 1 line
drivers/media/v4l2-core/v4l2-ioctl.c +0 -0
      case V4L2_PIX_FMT_NV12M_8L128:  descr = "NV12M (8x128
Linear)";
drivers/media/v4l2-core/v4l2-ioctl.c static void v4l_fill_fmtdesc(struct v4l2_fmtdesc · line 1450
Quoted 2 lines
drivers/media/v4l2-core/v4l2-ioctl.c +1 -0
 break;
+     case V4L2_PIX_FMT_NV12_10BE_8L128:      descr = "10-bit NV12
(8x128 Linear, BE)"; break;
drivers/media/v4l2-core/v4l2-ioctl.c static void v4l_fill_fmtdesc(struct v4l2_fmtdesc · line 1451
Quoted 1 line
drivers/media/v4l2-core/v4l2-ioctl.c +0 -0
      case V4L2_PIX_FMT_NV12M_10BE_8L128:     descr = "10-bit
NV12M (8x128 Linear, BE)"; break;
Quoted 8 lines
drivers/media/v4l2-core/v4l2-ioctl.c +0 -0
      default:
(quoted diff) +0 -0
b/include/uapi/linux/videodev2.h index cd66e01ed3c3..64f16490dd2b 100644
include/uapi/linux/videodev2.h +0 -0
@@ -655,6 +655,8 @@ struct v4l2_pix_format {  #define
Quoted 6 lines
V4L2_PIX_FMT_NV12_16L16 v4l2_fourcc('H', 'M', '1', '2') /* 12 Y/CbCr 4:2:0 16x16 tiles */ #define V4L2_PIX_FMT_NV12_32L32 v4l2_fourcc('S', 'T', '1', '2') /* 12 Y/CbCr 4:2:0 32x32 tiles */ #define V4L2_PIX_FMT_P010_4L4 v4l2_fourcc('T', '0', '1', '0') /* 12 Y/CbCr 4:2:0 10-bit 4x4 macroblocks */
+#define V4L2_PIX_FMT_NV12_8L128 v4l2_fourcc('A', 'T', '1', '2') /*
Y/CbCr 4:2:0 8x128 tiles */
Quoted 9 lines
+#define V4L2_PIX_FMT_NV12_10BE_8L128 v4l2_fourcc_be('A', 'X', '1', +'2') /* Y/CbCr 4:2:0 10-bit 8x128 tiles */
/* Tiled YUV formats, non contiguous planes */ #define
V4L2_PIX_FMT_NV12MT v4l2_fourcc('T', 'M', '1', '2') /* 12 Y/CbCr 4:2:0 64x32 tiles */
--
2.37.1
For the other parts look's good to me.
Reviewed-by Tommaso Merciai <[email protected]>
Regards,
Tommaso
--
Tommaso Merciai
Embedded Linux Engineer
[email protected]
__________________________________
Amarula Solutions SRL
Via Le Canevare 30, 31100 Treviso, Veneto, IT T. +39 042 243 5310 [email protected]
https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.am
arulasolutions.com%2F&data=05%7C01%7Cming.qian%40nxp.com%7Cb6 16d089dcbb47195b1308da7baccf05%7C686ea1d3bc2b4c6fa92cd99c5c30163 5%7C0%7C0%7C637958279379165164%7CUnknown%7CTWFpbGZsb3d8eyJW IjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C30 00%7C%7C%7C&sdata=ReeppP8PqkPQN5uQClb1CdRcNupMFHybRrbQoM %2B2Wkg%3D&reserved=0

Tommaso Merciai

2022-08-12 07:11 UTC #4 lore raw

Re: [EXT] Re: [PATCH v4 1/4] media: add nv12_8l128 and nv12_10be_8l128 video format.

Hi Ming,

Quoted 51 lines · Ming Qian

Quoted 51 lines
From: Tommaso Merciai <[email protected]>
Sent: 2022年8月11日 23:19
To: Ming Qian <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; dl-linux-imx <[email protected]>; X.H. Bao <[email protected]>; [email protected]; [email protected]; [email protected]
Subject: [EXT] Re: [PATCH v4 1/4] media: add nv12_8l128 and nv12_10be_8l128 video format.
Caution: EXT Email
Hi Ming,
Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst pixels in 2D 8x128 tiles, and stores tiles linearly in
Quoted 27 lines · Ming Qian
add contiguous nv12 tiled format nv12_8l128 and nv12_10be_8l128
Signed-off-by Ming Qian <[email protected]>
Reviewed-by Nicolas Dufresne <[email protected]>
---
 .../userspace-api/media/v4l/pixfmt-yuv-planar.rst         | 8 ++++++++
 drivers/media/v4l2-core/v4l2-ioctl.c                      | 2 ++
 include/uapi/linux/videodev2.h                            | 2 ++
 3 files changed, 12 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst b/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst index 10b1feeb0b57..f1d5bb7b806d 100644
Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst +2 -0
@@ -273,7 +273,9 @@ of the luma plane.
 .. _V4L2-PIX-FMT-NV12-16L16:
 .. _V4L2-PIX-FMT-NV12-32L32:
 .. _V4L2-PIX-FMT-NV12M-8L128:
+.. _V4L2-PIX-FMT-NV12-8L128:
 .. _V4L2-PIX-FMT-NV12M-10BE-8L128:
+.. _V4L2-PIX-FMT-NV12-10BE-8L128:
 .. _V4L2-PIX-FMT-MM21:
 
 Tiled NV12
@@ -319,6 +321,9 @@ pixels in 2D 8x128 tiles, and stores tiles linearly in
memory.
Quoted 6 lines
Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst +3 -0
 The image height must be aligned to a multiple of 128.
 The layouts of the luma and chroma planes are identical.
 
+``V4L2_PIX_FMT_NV12_8L128`` is similar to
+``V4L2_PIX_FMT_NV12M_8L128`` but stores two planes in one memory.
+

Quoted 8 lines

Quoted 1 line
Don't know, maybe we need more details here?
Hi Tommaso,
I have added the non contiguous format V4L2_PIX_FMT_NV12M_8L128, they're almost same, but store two planes in contiguous memory or non contiguous memory, just like NV12 and NV12M, the descriptions are almost the same, so I just say V4L2_PIX_FMT_NV12_8L128 is similar to V4L2_PIX_FMT_NV12M_8L128.
Do I need to copy the full description from the V4L2_PIX_FMT_NV12M_8L128 to here?
Ming

You are right the only differences is in which way the planes are stored in memory, sorry. It's clear :)

Thanks for clarifications,
Tommaso

Quoted 91 lines

Quoted 9 lines
Quoted 9 lines
Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst +0 -0
 ``V4L2_PIX_FMT_NV12M_10BE_8L128`` is similar to
``V4L2_PIX_FMT_NV12M`` but stores
10 bits pixels in 2D 8x128 tiles, and stores tiles linearly in memory. the data is arranged in big endian order.
Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst +0 -0
@@ -334,6 +339,9 @@ byte 2: Y1(bits 3-0) Y2(bits 9-6)  byte 3: Y2(bits
 5-0) Y3(bits 9-8)  byte 4: Y3(bits 7-0)
 
Quoted 6 lines
Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst byte 2: Y1(bits 3-0) Y2(bits 9-6) byte 3: Y2(bits · line 342
Quoted 2 lines
Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst +2 -0
+``V4L2_PIX_FMT_NV12_10BE_8L128`` is similar to
+``V4L2_PIX_FMT_NV12M_10BE_8L128`` but stores two planes in one
memory.
Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst byte 2: Y1(bits 3-0) Y2(bits 9-6) byte 3: Y2(bits · line 343
Quoted 1 line
Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst +1 -0
+
here also?
It's the same issue.
Quoted 73 lines
drivers/media/v4l2-core/v4l2-ioctl.c static void v4l_fill_fmtdesc(struct v4l2_fmtdesc
Quoted 11 lines
Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst +0 -0
 ``V4L2_PIX_FMT_MM21`` store luma pixel in 16x32 tiles, and chroma
pixels in 16x16 tiles. The line stride must be aligned to a multiple of 16 and the image height must be aligned to a multiple of 32. The number of luma and chroma diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index c314025d977e..d973bd2ff750 100644
drivers/media/v4l2-core/v4l2-ioctl.c +0 -0
@@ -1444,7 +1444,9 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc
*fmt)
drivers/media/v4l2-core/v4l2-ioctl.c static void v4l_fill_fmtdesc(struct v4l2_fmtdesc · line 1445
Quoted 2 lines
drivers/media/v4l2-core/v4l2-ioctl.c +0 -0
      case V4L2_META_FMT_VIVID:       descr = "Vivid Metadata"; break;
      case V4L2_META_FMT_RK_ISP1_PARAMS:      descr = "Rockchip
ISP1 3A Parameters"; break;
drivers/media/v4l2-core/v4l2-ioctl.c static void v4l_fill_fmtdesc(struct v4l2_fmtdesc · line 1446
Quoted 1 line
drivers/media/v4l2-core/v4l2-ioctl.c +0 -0
      case V4L2_META_FMT_RK_ISP1_STAT_3A:     descr = "Rockchip ISP1
3A Statistics"; break;
drivers/media/v4l2-core/v4l2-ioctl.c static void v4l_fill_fmtdesc(struct v4l2_fmtdesc · line 1447
Quoted 1 line
drivers/media/v4l2-core/v4l2-ioctl.c +1 -0
+     case V4L2_PIX_FMT_NV12_8L128:   descr = "NV12 (8x128 Linear)";
break;
drivers/media/v4l2-core/v4l2-ioctl.c static void v4l_fill_fmtdesc(struct v4l2_fmtdesc · line 1448
Quoted 1 line
drivers/media/v4l2-core/v4l2-ioctl.c +0 -0
      case V4L2_PIX_FMT_NV12M_8L128:  descr = "NV12M (8x128
Linear)";
drivers/media/v4l2-core/v4l2-ioctl.c static void v4l_fill_fmtdesc(struct v4l2_fmtdesc · line 1450
Quoted 2 lines
drivers/media/v4l2-core/v4l2-ioctl.c +1 -0
 break;
+     case V4L2_PIX_FMT_NV12_10BE_8L128:      descr = "10-bit NV12
(8x128 Linear, BE)"; break;
drivers/media/v4l2-core/v4l2-ioctl.c static void v4l_fill_fmtdesc(struct v4l2_fmtdesc · line 1451
Quoted 1 line
drivers/media/v4l2-core/v4l2-ioctl.c +0 -0
      case V4L2_PIX_FMT_NV12M_10BE_8L128:     descr = "10-bit
NV12M (8x128 Linear, BE)"; break;
Quoted 8 lines
drivers/media/v4l2-core/v4l2-ioctl.c +0 -0
      default:
(quoted diff) +0 -0
b/include/uapi/linux/videodev2.h index cd66e01ed3c3..64f16490dd2b 100644
include/uapi/linux/videodev2.h +0 -0
@@ -655,6 +655,8 @@ struct v4l2_pix_format {  #define
Quoted 6 lines
V4L2_PIX_FMT_NV12_16L16 v4l2_fourcc('H', 'M', '1', '2') /* 12 Y/CbCr 4:2:0 16x16 tiles */ #define V4L2_PIX_FMT_NV12_32L32 v4l2_fourcc('S', 'T', '1', '2') /* 12 Y/CbCr 4:2:0 32x32 tiles */ #define V4L2_PIX_FMT_P010_4L4 v4l2_fourcc('T', '0', '1', '0') /* 12 Y/CbCr 4:2:0 10-bit 4x4 macroblocks */
+#define V4L2_PIX_FMT_NV12_8L128 v4l2_fourcc('A', 'T', '1', '2') /*
Y/CbCr 4:2:0 8x128 tiles */
Quoted 9 lines
+#define V4L2_PIX_FMT_NV12_10BE_8L128 v4l2_fourcc_be('A', 'X', '1', +'2') /* Y/CbCr 4:2:0 10-bit 8x128 tiles */
/* Tiled YUV formats, non contiguous planes */ #define
V4L2_PIX_FMT_NV12MT v4l2_fourcc('T', 'M', '1', '2') /* 12 Y/CbCr 4:2:0 64x32 tiles */
--
2.37.1
For the other parts look's good to me.
Reviewed-by Tommaso Merciai <[email protected]>
Regards,
Tommaso
--
Tommaso Merciai
Embedded Linux Engineer
[email protected]
__________________________________
Amarula Solutions SRL
Via Le Canevare 30, 31100 Treviso, Veneto, IT T. +39 042 243 5310 [email protected]
https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.am
arulasolutions.com%2F&data=05%7C01%7Cming.qian%40nxp.com%7Cb6 16d089dcbb47195b1308da7baccf05%7C686ea1d3bc2b4c6fa92cd99c5c30163 5%7C0%7C0%7C637958279379165164%7CUnknown%7CTWFpbGZsb3d8eyJW IjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C30 00%7C%7C%7C&sdata=ReeppP8PqkPQN5uQClb1CdRcNupMFHybRrbQoM %2B2Wkg%3D&reserved=0

Signature · 10 lines

Tommaso Merciai
Embedded Linux Engineer
[email protected]
__________________________________

Amarula Solutions SRL
Via Le Canevare 30, 31100 Treviso, Veneto, IT
T. +39 042 243 5310
[email protected]
www.amarulasolutions.com

Nicolas Dufresne

2022-08-16 14:42 UTC #5 lore raw

Re: [PATCH v4 1/4] media: add nv12_8l128 and nv12_10be_8l128 video format.

In reply to Tommaso Merciai

Le jeudi 11 août 2022 à 17:18 +0200, Tommaso Merciai a écrit :

Quoted 44 lines

Hi Ming,
Quoted 25 lines · Ming Qian
add contiguous nv12 tiled format nv12_8l128 and nv12_10be_8l128
Signed-off-by Ming Qian <[email protected]>
Reviewed-by Nicolas Dufresne <[email protected]>
---
 .../userspace-api/media/v4l/pixfmt-yuv-planar.rst         | 8 ++++++++
 drivers/media/v4l2-core/v4l2-ioctl.c                      | 2 ++
 include/uapi/linux/videodev2.h                            | 2 ++
 3 files changed, 12 insertions(+)
Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst +2 -0
index 10b1feeb0b57..f1d5bb7b806d 100644
@@ -273,7 +273,9 @@ of the luma plane.
 .. _V4L2-PIX-FMT-NV12-16L16:
 .. _V4L2-PIX-FMT-NV12-32L32:
 .. _V4L2-PIX-FMT-NV12M-8L128:
+.. _V4L2-PIX-FMT-NV12-8L128:
 .. _V4L2-PIX-FMT-NV12M-10BE-8L128:
+.. _V4L2-PIX-FMT-NV12-10BE-8L128:
 .. _V4L2-PIX-FMT-MM21:
 
 Tiled NV12
Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst pixels in 2D 8x128 tiles, and stores tiles linearly in memory. · line 326
Quoted 6 lines
Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst +3 -0
@@ -319,6 +321,9 @@ pixels in 2D 8x128 tiles, and stores tiles linearly in memory.
 The image height must be aligned to a multiple of 128.
 The layouts of the luma and chroma planes are identical.
 
+``V4L2_PIX_FMT_NV12_8L128`` is similar to ``V4L2_PIX_FMT_NV12M_8L128`` but stores
+two planes in one memory.
+
Don't know, maybe we need more details here?
Quoted 7 lines
Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst +0 -0
 ``V4L2_PIX_FMT_NV12M_10BE_8L128`` is similar to ``V4L2_PIX_FMT_NV12M`` but stores
 10 bits pixels in 2D 8x128 tiles, and stores tiles linearly in memory.
 the data is arranged in big endian order.
@@ -334,6 +339,9 @@ byte 2: Y1(bits 3-0) Y2(bits 9-6)
 byte 3: Y2(bits 5-0) Y3(bits 9-8)
 byte 4: Y3(bits 7-0)
 

Quoted 5 lines

Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst byte 2: Y1(bits 3-0) Y2(bits 9-6) · line 344
Quoted 3 lines
Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst +3 -0
+``V4L2_PIX_FMT_NV12_10BE_8L128`` is similar to ``V4L2_PIX_FMT_NV12M_10BE_8L128`` but stores
+two planes in one memory.
+
here also?

Perhaps I'm too much into it, but it felt clear to me. Note that the duplication of pixel formats based on the number of allocation is pretty unique to V4L2, but its also generalized to all planar formats. So adding a lot of documentation for each specific format will be heavy. Note that one improvement I see, would be s/memory/allocation/ , or memory allocation.

Quoted 41 lines

Quoted 34 lines

Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst +0 -0

 ``V4L2_PIX_FMT_MM21`` store luma pixel in 16x32 tiles, and chroma pixels
 in 16x16 tiles. The line stride must be aligned to a multiple of 16 and the
 image height must be aligned to a multiple of 32. The number of luma and chroma

drivers/media/v4l2-core/v4l2-ioctl.c +2 -0

index c314025d977e..d973bd2ff750 100644
@@ -1444,7 +1444,9 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_META_FMT_VIVID:       descr = "Vivid Metadata"; break;
 	case V4L2_META_FMT_RK_ISP1_PARAMS:	descr = "Rockchip ISP1 3A Parameters"; break;
 	case V4L2_META_FMT_RK_ISP1_STAT_3A:	descr = "Rockchip ISP1 3A Statistics"; break;
+	case V4L2_PIX_FMT_NV12_8L128:	descr = "NV12 (8x128 Linear)"; break;
 	case V4L2_PIX_FMT_NV12M_8L128:	descr = "NV12M (8x128 Linear)"; break;
+	case V4L2_PIX_FMT_NV12_10BE_8L128:	descr = "10-bit NV12 (8x128 Linear, BE)"; break;
 	case V4L2_PIX_FMT_NV12M_10BE_8L128:	descr = "10-bit NV12M (8x128 Linear, BE)"; break;
 
 	default:

include/uapi/linux/videodev2.h +2 -0

index cd66e01ed3c3..64f16490dd2b 100644
@@ -655,6 +655,8 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_NV12_16L16 v4l2_fourcc('H', 'M', '1', '2') /* 12  Y/CbCr 4:2:0 16x16 tiles */
 #define V4L2_PIX_FMT_NV12_32L32 v4l2_fourcc('S', 'T', '1', '2') /* 12  Y/CbCr 4:2:0 32x32 tiles */
 #define V4L2_PIX_FMT_P010_4L4 v4l2_fourcc('T', '0', '1', '0') /* 12  Y/CbCr 4:2:0 10-bit 4x4 macroblocks */
+#define V4L2_PIX_FMT_NV12_8L128       v4l2_fourcc('A', 'T', '1', '2') /* Y/CbCr 4:2:0 8x128 tiles */
+#define V4L2_PIX_FMT_NV12_10BE_8L128  v4l2_fourcc_be('A', 'X', '1', '2') /* Y/CbCr 4:2:0 10-bit 8x128 tiles */
 
 /* Tiled YUV formats, non contiguous planes */
 #define V4L2_PIX_FMT_NV12MT  v4l2_fourcc('T', 'M', '1', '2') /* 12  Y/CbCr 4:2:0 64x32 tiles */

Signature · 1 line

2.37.1

For the other parts look's good to me.

Reviewed-by Tommaso Merciai <[email protected]>

Regards,
Tommaso

Ming Qian

author maintainer

2022-08-09 06:51 UTC #6 lore raw

PATCH v4 2/4 media: amphion: tell and handle contiguous and non contiguous format

In reply to Ming Qian

Driver should tell the number of memory planes and component planes. the amphion vpu support non contiguous planes,
but for compatibility with other device
that only support contiguous planes.
driver can add support for contiguous planes in the same time. Then the mem_planes can be different from the comp_planes. driver need to handle buffer according mem_planes and comp_planes.

So driver can support NV12 and NV12M.

Signed-off-by Ming Qian <[email protected]>

---
 drivers/media/platform/amphion/vdec.c        | 187 ++++++++++---------
 drivers/media/platform/amphion/venc.c        |  33 ++--
 drivers/media/platform/amphion/vpu.h         |   4 +-
 drivers/media/platform/amphion/vpu_dbg.c     |   8 +-
 drivers/media/platform/amphion/vpu_helpers.c |  48 ++++-
 drivers/media/platform/amphion/vpu_helpers.h |   2 +
 drivers/media/platform/amphion/vpu_malone.c  |   3 +-
 drivers/media/platform/amphion/vpu_v4l2.c    | 168 ++++++++++++-----
 drivers/media/platform/amphion/vpu_v4l2.h    |   3 +-
 drivers/media/platform/amphion/vpu_windsor.c |   8 +-
 10 files changed, 299 insertions(+), 165 deletions(-)

drivers/media/platform/amphion/vdec.c +98 ~~-89~~

index feb75dc204de..48ab664fa7ef 100644
@@ -69,72 +69,85 @@ struct vdec_t {
 static const struct vpu_format vdec_formats[] = {
 	{
 		.pixfmt = V4L2_PIX_FMT_NV12M_8L128,
-		.num_planes = 2,
+		.mem_planes = 2,
+		.comp_planes = 2,
 		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_NV12M_10BE_8L128,
-		.num_planes = 2,
+		.mem_planes = 2,
+		.comp_planes = 2,
 		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_H264,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_H264_MVC,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_HEVC,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_VC1_ANNEX_G,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_VC1_ANNEX_L,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+		.flags = V4L2_FMT_FLAG_COMPRESSED
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_MPEG2,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_MPEG4,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_XVID,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_VP8,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_H263,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
 	},
 	{0, 0, 0, 0},
 };

@@ -256,23 +269,22 @@ static int vdec_enum_fmt(struct file *file, void *fh, struct v4l2_fmtdesc *f)
 	int ret = -EINVAL;
 
 	vpu_inst_lock(inst);
-	if (!V4L2_TYPE_IS_OUTPUT(f->type) && vdec->fixed_fmt) {
-		if (f->index == 0) {
-			f->pixelformat = inst->cap_format.pixfmt;
-			f->flags = inst->cap_format.flags;
-			ret = 0;
-		}
+	if (V4L2_TYPE_IS_CAPTURE(f->type) && vdec->fixed_fmt) {
+		fmt = vpu_get_format(inst, f->type);
+		if (f->index == 1)
+			fmt = vpu_helper_find_sibling(inst, f->type, fmt->pixfmt);
+		if (f->index > 1)
+			fmt = NULL;
 	} else {
 		fmt = vpu_helper_enum_format(inst, f->type, f->index);
-		memset(f->reserved, 0, sizeof(f->reserved));
-		if (!fmt)
-			goto exit;
-
-		f->pixelformat = fmt->pixfmt;
-		f->flags = fmt->flags;
-		ret = 0;
 	}
+	if (!fmt)
+		goto exit;
 
+	memset(f->reserved, 0, sizeof(f->reserved));
+	f->pixelformat = fmt->pixfmt;
+	f->flags = fmt->flags;
+	ret = 0;
 exit:
 	vpu_inst_unlock(inst);
 	return ret;

@@ -289,14 +301,14 @@ static int vdec_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
 	cur_fmt = vpu_get_format(inst, f->type);
 
 	pixmp->pixelformat = cur_fmt->pixfmt;
-	pixmp->num_planes = cur_fmt->num_planes;
+	pixmp->num_planes = cur_fmt->mem_planes;
 	pixmp->width = cur_fmt->width;
 	pixmp->height = cur_fmt->height;
 	pixmp->field = cur_fmt->field;
 	pixmp->flags = cur_fmt->flags;
 	for (i = 0; i < pixmp->num_planes; i++) {
 		pixmp->plane_fmt[i].bytesperline = cur_fmt->bytesperline[i];
-		pixmp->plane_fmt[i].sizeimage = cur_fmt->sizeimage[i];
+		pixmp->plane_fmt[i].sizeimage = vpu_get_fmt_plane_size(cur_fmt, i);
 	}
 
 	f->fmt.pix_mp.colorspace = vdec->codec_info.color_primaries;

@@ -311,10 +323,19 @@ static int vdec_try_fmt(struct file *file, void *fh, struct v4l2_format *f)
 {
 	struct vpu_inst *inst = to_inst(file);
 	struct vdec_t *vdec = inst->priv;
-
-	vpu_try_fmt_common(inst, f);
+	struct vpu_format fmt;
 
 	vpu_inst_lock(inst);
+	if (V4L2_TYPE_IS_CAPTURE(f->type) && vdec->fixed_fmt) {
+		struct vpu_format *cap_fmt = vpu_get_format(inst, f->type);
+
+		if (!vpu_helper_match_format(inst, cap_fmt->type, cap_fmt->pixfmt,
+					     f->fmt.pix_mp.pixelformat))
+			f->fmt.pix_mp.pixelformat = cap_fmt->pixfmt;
+	}
+
+	vpu_try_fmt_common(inst, f, &fmt);
+
 	if (vdec->fixed_fmt) {
 		f->fmt.pix_mp.colorspace = vdec->codec_info.color_primaries;
 		f->fmt.pix_mp.xfer_func = vdec->codec_info.transfer_chars;

@@ -334,7 +355,7 @@ static int vdec_try_fmt(struct file *file, void *fh, struct v4l2_format *f)
 static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)
 {
 	struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
-	const struct vpu_format *fmt;
+	struct vpu_format fmt;
 	struct vpu_format *cur_fmt;
 	struct vb2_queue *q;
 	struct vdec_t *vdec = inst->priv;

@@ -349,36 +370,30 @@ static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)
 	if (vb2_is_busy(q))
 		return -EBUSY;
 
-	fmt = vpu_try_fmt_common(inst, f);
-	if (!fmt)
+	if (vpu_try_fmt_common(inst, f, &fmt))
 		return -EINVAL;
 
 	cur_fmt = vpu_get_format(inst, f->type);
 	if (V4L2_TYPE_IS_OUTPUT(f->type) && inst->state != VPU_CODEC_STATE_DEINIT) {
-		if (cur_fmt->pixfmt != fmt->pixfmt) {
+		if (cur_fmt->pixfmt != fmt.pixfmt) {
 			vdec->reset_codec = true;
 			vdec->fixed_fmt = false;
 		}
 	}
-	cur_fmt->pixfmt = fmt->pixfmt;
 	if (V4L2_TYPE_IS_OUTPUT(f->type) || !vdec->fixed_fmt) {
-		cur_fmt->num_planes = fmt->num_planes;
-		cur_fmt->flags = fmt->flags;
-		cur_fmt->width = pixmp->width;
-		cur_fmt->height = pixmp->height;
-		for (i = 0; i < fmt->num_planes; i++) {
-			cur_fmt->sizeimage[i] = pixmp->plane_fmt[i].sizeimage;
-			cur_fmt->bytesperline[i] = pixmp->plane_fmt[i].bytesperline;
-		}
-		if (pixmp->field != V4L2_FIELD_ANY)
-			cur_fmt->field = pixmp->field;
+		memcpy(cur_fmt, &fmt, sizeof(*cur_fmt));
 	} else {
-		pixmp->num_planes = cur_fmt->num_planes;
+		if (vpu_helper_match_format(inst, f->type, cur_fmt->pixfmt, pixmp->pixelformat)) {
+			cur_fmt->pixfmt = fmt.pixfmt;
+			cur_fmt->mem_planes = fmt.mem_planes;
+		}
+		pixmp->pixelformat = cur_fmt->pixfmt;
+		pixmp->num_planes = cur_fmt->mem_planes;
 		pixmp->width = cur_fmt->width;
 		pixmp->height = cur_fmt->height;
 		for (i = 0; i < pixmp->num_planes; i++) {
 			pixmp->plane_fmt[i].bytesperline = cur_fmt->bytesperline[i];
-			pixmp->plane_fmt[i].sizeimage = cur_fmt->sizeimage[i];
+			pixmp->plane_fmt[i].sizeimage = vpu_get_fmt_plane_size(cur_fmt, i);
 		}
 		pixmp->field = cur_fmt->field;
 	}

@@ -678,9 +693,11 @@ static struct vpu_vb2_buffer *vdec_find_buffer(struct vpu_inst *inst, u32 luma)
 static void vdec_buf_done(struct vpu_inst *inst, struct vpu_frame_info *frame)
 {
 	struct vdec_t *vdec = inst->priv;
+	struct vpu_format *cur_fmt;
 	struct vpu_vb2_buffer *vpu_buf;
 	struct vb2_v4l2_buffer *vbuf;
 	u32 sequence;
+	int i;
 
 	if (!frame)
 		return;

@@ -699,6 +716,7 @@ static void vdec_buf_done(struct vpu_inst *inst, struct vpu_frame_info *frame)
 		return;
 	}
 
+	cur_fmt = vpu_get_format(inst, inst->cap_format.type);
 	vbuf = &vpu_buf->m2m_buf.vb;
 	if (vbuf->vb2_buf.index != frame->id)
 		dev_err(inst->dev, "[%d] buffer id(%d, %d) dismatch\n",

@@ -707,9 +725,9 @@ static void vdec_buf_done(struct vpu_inst *inst, struct vpu_frame_info *frame)
 	if (vpu_get_buffer_state(vbuf) != VPU_BUF_STATE_DECODED)
 		dev_err(inst->dev, "[%d] buffer(%d) ready without decoded\n", inst->id, frame->id);
 	vpu_set_buffer_state(vbuf, VPU_BUF_STATE_READY);
-	vb2_set_plane_payload(&vbuf->vb2_buf, 0, inst->cap_format.sizeimage[0]);
-	vb2_set_plane_payload(&vbuf->vb2_buf, 1, inst->cap_format.sizeimage[1]);
-	vbuf->field = inst->cap_format.field;
+	for (i = 0; i < vbuf->vb2_buf.num_planes; i++)
+		vb2_set_plane_payload(&vbuf->vb2_buf, i, vpu_get_fmt_plane_size(cur_fmt, i));
+	vbuf->field = cur_fmt->field;
 	vbuf->sequence = sequence;
 	dev_dbg(inst->dev, "[%d][OUTPUT TS]%32lld\n", inst->id, vbuf->vb2_buf.timestamp);

@@ -747,15 +765,17 @@ static void vdec_stop_done(struct vpu_inst *inst)
 static bool vdec_check_source_change(struct vpu_inst *inst)
 {
 	struct vdec_t *vdec = inst->priv;
-	const struct vpu_format *fmt;
-	int i;
+	const struct vpu_format *sibling;
 
 	if (!inst->fh.m2m_ctx)
 		return false;
 
+	sibling = vpu_helper_find_sibling(inst, inst->cap_format.type, inst->cap_format.pixfmt);
+	if (sibling && vdec->codec_info.pixfmt == sibling->pixfmt)
+		vdec->codec_info.pixfmt = inst->cap_format.pixfmt;
+
 	if (!vb2_is_streaming(v4l2_m2m_get_dst_vq(inst->fh.m2m_ctx)))
 		return true;
-	fmt = vpu_helper_find_format(inst, inst->cap_format.type, vdec->codec_info.pixfmt);
 	if (inst->cap_format.pixfmt != vdec->codec_info.pixfmt)
 		return true;
 	if (inst->cap_format.width != vdec->codec_info.decoded_width)

@@ -772,14 +792,6 @@ static bool vdec_check_source_change(struct vpu_inst *inst)
 		return true;
 	if (inst->crop.height != vdec->codec_info.height)
 		return true;
-	if (fmt && inst->cap_format.num_planes != fmt->num_planes)
-		return true;
-	for (i = 0; i < inst->cap_format.num_planes; i++) {
-		if (inst->cap_format.bytesperline[i] != vdec->codec_info.bytesperline[i])
-			return true;
-		if (inst->cap_format.sizeimage[i] != vdec->codec_info.sizeimage[i])
-			return true;
-	}
 
 	return false;
 }

@@ -787,27 +799,21 @@ static bool vdec_check_source_change(struct vpu_inst *inst)
 static void vdec_init_fmt(struct vpu_inst *inst)
 {
 	struct vdec_t *vdec = inst->priv;
-	const struct vpu_format *fmt;
-	int i;
+	struct v4l2_format f;
 
-	fmt = vpu_helper_find_format(inst, inst->cap_format.type, vdec->codec_info.pixfmt);
-	inst->out_format.width = vdec->codec_info.width;
-	inst->out_format.height = vdec->codec_info.height;
-	inst->cap_format.width = vdec->codec_info.decoded_width;
-	inst->cap_format.height = vdec->codec_info.decoded_height;
-	inst->cap_format.pixfmt = vdec->codec_info.pixfmt;
-	if (fmt) {
-		inst->cap_format.num_planes = fmt->num_planes;
-		inst->cap_format.flags = fmt->flags;
-	}
-	for (i = 0; i < inst->cap_format.num_planes; i++) {
-		inst->cap_format.bytesperline[i] = vdec->codec_info.bytesperline[i];
-		inst->cap_format.sizeimage[i] = vdec->codec_info.sizeimage[i];
-	}
+	memset(&f, 0, sizeof(f));
+	f.type = inst->cap_format.type;
+	f.fmt.pix_mp.pixelformat = vdec->codec_info.pixfmt;
+	f.fmt.pix_mp.width = vdec->codec_info.decoded_width;
+	f.fmt.pix_mp.height = vdec->codec_info.decoded_height;
 	if (vdec->codec_info.progressive)
-		inst->cap_format.field = V4L2_FIELD_NONE;
+		f.fmt.pix_mp.field = V4L2_FIELD_NONE;
 	else
-		inst->cap_format.field = V4L2_FIELD_SEQ_TB;
+		f.fmt.pix_mp.field = V4L2_FIELD_SEQ_TB;
+	vpu_try_fmt_common(inst, &f, &inst->cap_format);
+
+	inst->out_format.width = vdec->codec_info.width;
+	inst->out_format.height = vdec->codec_info.height;
 }
 
 static void vdec_init_crop(struct vpu_inst *inst)

@@ -966,7 +972,10 @@ static int vdec_response_frame(struct vpu_inst *inst, struct vb2_v4l2_buffer *vb
 	info.tag = vdec->seq_tag;
 	info.luma_addr = vpu_get_vb_phy_addr(&vbuf->vb2_buf, 0);
 	info.luma_size = inst->cap_format.sizeimage[0];
-	info.chroma_addr = vpu_get_vb_phy_addr(&vbuf->vb2_buf, 1);
+	if (vbuf->vb2_buf.num_planes > 1)
+		info.chroma_addr = vpu_get_vb_phy_addr(&vbuf->vb2_buf, 1);
+	else
+		info.chroma_addr = info.luma_addr + info.luma_size;
 	info.chromau_size = inst->cap_format.sizeimage[1];
 	info.bytesperline = inst->cap_format.bytesperline[0];
 	ret = vpu_session_alloc_fs(inst, &info);

@@ -975,7 +984,7 @@ static int vdec_response_frame(struct vpu_inst *inst, struct vb2_v4l2_buffer *vb
 
 	vpu_buf->tag = info.tag;
 	vpu_buf->luma = info.luma_addr;
-	vpu_buf->chroma_u = info.chromau_size;
+	vpu_buf->chroma_u = info.chroma_addr;
 	vpu_buf->chroma_v = 0;
 	vpu_set_buffer_state(vbuf, VPU_BUF_STATE_INUSE);
 	vdec->slots[info.id] = vpu_buf;

drivers/media/platform/amphion/venc.c +12 ~~-21~~

index 37212f087fdd..060a1ee78b17 100644
@@ -69,13 +69,16 @@ struct venc_frame_t {
 static const struct vpu_format venc_formats[] = {
 	{
 		.pixfmt = V4L2_PIX_FMT_NV12M,
-		.num_planes = 2,
+		.mem_planes = 2,
+		.comp_planes = 2,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_H264,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.flags = V4L2_FMT_FLAG_COMPRESSED
 	},
 	{0, 0, 0, 0},
 };

@@ -173,14 +176,14 @@ static int venc_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
 	cur_fmt = vpu_get_format(inst, f->type);
 
 	pixmp->pixelformat = cur_fmt->pixfmt;
-	pixmp->num_planes = cur_fmt->num_planes;
+	pixmp->num_planes = cur_fmt->mem_planes;
 	pixmp->width = cur_fmt->width;
 	pixmp->height = cur_fmt->height;
 	pixmp->field = cur_fmt->field;
 	pixmp->flags = cur_fmt->flags;
 	for (i = 0; i < pixmp->num_planes; i++) {
 		pixmp->plane_fmt[i].bytesperline = cur_fmt->bytesperline[i];
-		pixmp->plane_fmt[i].sizeimage = cur_fmt->sizeimage[i];
+		pixmp->plane_fmt[i].sizeimage = vpu_get_fmt_plane_size(cur_fmt, i);
 	}
 
 	f->fmt.pix_mp.colorspace = venc->params.color.primaries;

@@ -194,8 +197,9 @@ static int venc_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
 static int venc_try_fmt(struct file *file, void *fh, struct v4l2_format *f)
 {
 	struct vpu_inst *inst = to_inst(file);
+	struct vpu_format fmt;
 
-	vpu_try_fmt_common(inst, f);
+	vpu_try_fmt_common(inst, f, &fmt);
 
 	return 0;
 }

@@ -203,12 +207,11 @@ static int venc_try_fmt(struct file *file, void *fh, struct v4l2_format *f)
 static int venc_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
 {
 	struct vpu_inst *inst = to_inst(file);
-	const struct vpu_format *fmt;
+	struct vpu_format fmt;
 	struct vpu_format *cur_fmt;
 	struct vb2_queue *q;
 	struct venc_t *venc = inst->priv;
 	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
-	int i;
 
 	q = v4l2_m2m_get_vq(inst->fh.m2m_ctx, f->type);
 	if (!q)

@@ -216,24 +219,12 @@ static int venc_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
 	if (vb2_is_busy(q))
 		return -EBUSY;
 
-	fmt = vpu_try_fmt_common(inst, f);
-	if (!fmt)
+	if (vpu_try_fmt_common(inst, f, &fmt))
 		return -EINVAL;
 
 	cur_fmt = vpu_get_format(inst, f->type);
 
-	cur_fmt->pixfmt = fmt->pixfmt;
-	cur_fmt->num_planes = fmt->num_planes;
-	cur_fmt->flags = fmt->flags;
-	cur_fmt->width = pix_mp->width;
-	cur_fmt->height = pix_mp->height;
-	for (i = 0; i < fmt->num_planes; i++) {
-		cur_fmt->sizeimage[i] = pix_mp->plane_fmt[i].sizeimage;
-		cur_fmt->bytesperline[i] = pix_mp->plane_fmt[i].bytesperline;
-	}
-
-	if (pix_mp->field != V4L2_FIELD_ANY)
-		cur_fmt->field = pix_mp->field;
+	memcpy(cur_fmt, &fmt, sizeof(*cur_fmt));
 
 	if (V4L2_TYPE_IS_OUTPUT(f->type)) {
 		venc->params.input_format = cur_fmt->pixfmt;

drivers/media/platform/amphion/vpu.h +3 -1

index f914de6ed81e..998591fa5b85 100644
@@ -84,7 +84,8 @@ struct vpu_dev {
 
 struct vpu_format {
 	u32 pixfmt;
-	unsigned int num_planes;
+	u32 mem_planes;
+	u32 comp_planes;
 	u32 type;
 	u32 flags;
 	u32 width;

@@ -92,6 +93,7 @@ struct vpu_format {
 	u32 sizeimage[VIDEO_MAX_PLANES];
 	u32 bytesperline[VIDEO_MAX_PLANES];
 	u32 field;
+	u32 sibling;
 };
 
 struct vpu_core_resources {

drivers/media/platform/amphion/vpu_dbg.c +4 -4

index f72c8a506b22..c41c9896912c 100644
@@ -89,9 +89,9 @@ static int vpu_dbg_instance(struct seq_file *s, void *data)
 			vq->last_buffer_dequeued);
 	if (seq_write(s, str, num))
 		return 0;
-	for (i = 0; i < inst->out_format.num_planes; i++) {
+	for (i = 0; i < inst->out_format.mem_planes; i++) {
 		num = scnprintf(str, sizeof(str), " %d(%d)",
-				inst->out_format.sizeimage[i],
+				vpu_get_fmt_plane_size(&inst->out_format, i),
 				inst->out_format.bytesperline[i]);
 		if (seq_write(s, str, num))
 			return 0;

@@ -113,9 +113,9 @@ static int vpu_dbg_instance(struct seq_file *s, void *data)
 			vq->last_buffer_dequeued);
 	if (seq_write(s, str, num))
 		return 0;
-	for (i = 0; i < inst->cap_format.num_planes; i++) {
+	for (i = 0; i < inst->cap_format.mem_planes; i++) {
 		num = scnprintf(str, sizeof(str), " %d(%d)",
-				inst->cap_format.sizeimage[i],
+				vpu_get_fmt_plane_size(&inst->cap_format, i),
 				inst->cap_format.bytesperline[i]);
 		if (seq_write(s, str, num))
 			return 0;

drivers/media/platform/amphion/vpu_helpers.c +43 -5

89 lines not on this page · raw message

drivers/media/platform/amphion/vpu_helpers.h +2 -0

index bc28350958be..0eaddb07190d 100644
@@ -14,6 +14,8 @@ struct vpu_pair {
 int vpu_helper_find_in_array_u8(const u8 *array, u32 size, u32 x);
 bool vpu_helper_check_type(struct vpu_inst *inst, u32 type);
 const struct vpu_format *vpu_helper_find_format(struct vpu_inst *inst, u32 type, u32 pixelfmt);
+const struct vpu_format *vpu_helper_find_sibling(struct vpu_inst *inst, u32 type, u32 pixelfmt);
+bool vpu_helper_match_format(struct vpu_inst *inst, u32 type, u32 fmta, u32 fmtb);
 const struct vpu_format *vpu_helper_enum_format(struct vpu_inst *inst, u32 type, int index);
 u32 vpu_helper_valid_frame_width(struct vpu_inst *inst, u32 width);
 u32 vpu_helper_valid_frame_height(struct vpu_inst *inst, u32 height);

drivers/media/platform/amphion/vpu_malone.c +2 -1

index 51e0702f9ae1..69d9a2269fce 100644
@@ -583,7 +583,8 @@ bool vpu_malone_check_fmt(enum vpu_core_type type, u32 pixelfmt)
 	if (!vpu_imx8q_check_fmt(type, pixelfmt))
 		return false;
 
-	if (pixelfmt == V4L2_PIX_FMT_NV12M_8L128 || pixelfmt == V4L2_PIX_FMT_NV12M_10BE_8L128)
+	if (pixelfmt == V4L2_PIX_FMT_NV12_8L128 || pixelfmt == V4L2_PIX_FMT_NV12_10BE_8L128 ||
+	    pixelfmt == V4L2_PIX_FMT_NV12M_8L128 || pixelfmt == V4L2_PIX_FMT_NV12M_10BE_8L128)
 		return true;
 	if (vpu_malone_format_remap(pixelfmt) == MALONE_FMT_NULL)
 		return false;

drivers/media/platform/amphion/vpu_v4l2.c +126 ~~-42~~

204 lines not on this page · raw message

drivers/media/platform/amphion/vpu_v4l2.h +2 -1

index 795ca33a6a50..ef5de6b66e47 100644
@@ -16,7 +16,8 @@ unsigned int vpu_get_buffer_state(struct vb2_v4l2_buffer *vbuf);
 int vpu_v4l2_open(struct file *file, struct vpu_inst *inst);
 int vpu_v4l2_close(struct file *file);
 
-const struct vpu_format *vpu_try_fmt_common(struct vpu_inst *inst, struct v4l2_format *f);
+u32 vpu_get_fmt_plane_size(struct vpu_format *fmt, u32 plane_no);
+int vpu_try_fmt_common(struct vpu_inst *inst, struct v4l2_format *f, struct vpu_format *fmt);
 int vpu_process_output_buffer(struct vpu_inst *inst);
 int vpu_process_capture_buffer(struct vpu_inst *inst);
 struct vb2_v4l2_buffer *vpu_next_src_buf(struct vpu_inst *inst);

drivers/media/platform/amphion/vpu_windsor.c +7 -1

index 1526af2ef9da..a454f142ae17 100644
@@ -775,6 +775,8 @@ static int vpu_windsor_fill_yuv_frame(struct vpu_shared_addr *shared,
 				      u32 instance,
 				      struct vb2_buffer *vb)
 {
+	struct vpu_inst *inst = vb2_get_drv_priv(vb->vb2_queue);
+	struct vpu_format *out_fmt;
 	struct vpu_enc_yuv_desc *desc;
 	struct vb2_v4l2_buffer *vbuf;

@@ -782,6 +784,7 @@ static int vpu_windsor_fill_yuv_frame(struct vpu_shared_addr *shared,
 		return -EINVAL;
 
 	desc = get_yuv_desc(shared, instance);
+	out_fmt = vpu_get_format(inst, vb->type);
 
 	vbuf = to_vb2_v4l2_buffer(vb);
 	desc->frame_id = vbuf->sequence;

@@ -790,7 +793,10 @@ static int vpu_windsor_fill_yuv_frame(struct vpu_shared_addr *shared,
 	else
 		desc->key_frame = 0;
 	desc->luma_base = vpu_get_vb_phy_addr(vb, 0);
-	desc->chroma_base = vpu_get_vb_phy_addr(vb, 1);
+	if (vb->num_planes > 1)
+		desc->chroma_base = vpu_get_vb_phy_addr(vb, 1);
+	else
+		desc->chroma_base = desc->luma_base + out_fmt->sizeimage[0];
 
 	return 0;
 }

Signature · 1 line

2.37.1

Tommaso Merciai

2022-08-12 07:15 UTC #7 lore raw

Re: [PATCH v4 2/4] media: amphion: tell and handle contiguous and non contiguous format

Hi Ming,
Just an info. I'm planning to help to review your series. On which Linux tree are you working?

Thanks in advance,
Tommaso

Quoted 925 lines · Ming Qian

So driver can support NV12 and NV12M.

Signed-off-by Ming Qian <[email protected]>

---
 drivers/media/platform/amphion/vdec.c        | 187 ++++++++++---------
 drivers/media/platform/amphion/venc.c        |  33 ++--
 drivers/media/platform/amphion/vpu.h         |   4 +-
 drivers/media/platform/amphion/vpu_dbg.c     |   8 +-
 drivers/media/platform/amphion/vpu_helpers.c |  48 ++++-
 drivers/media/platform/amphion/vpu_helpers.h |   2 +
 drivers/media/platform/amphion/vpu_malone.c  |   3 +-
 drivers/media/platform/amphion/vpu_v4l2.c    | 168 ++++++++++++-----
 drivers/media/platform/amphion/vpu_v4l2.h    |   3 +-
 drivers/media/platform/amphion/vpu_windsor.c |   8 +-
 10 files changed, 299 insertions(+), 165 deletions(-)

drivers/media/platform/amphion/vdec.c +98 ~~-89~~

index feb75dc204de..48ab664fa7ef 100644
@@ -69,72 +69,85 @@ struct vdec_t {
 static const struct vpu_format vdec_formats[] = {
 	{
 		.pixfmt = V4L2_PIX_FMT_NV12M_8L128,
-		.num_planes = 2,
+		.mem_planes = 2,
+		.comp_planes = 2,
 		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_NV12M_10BE_8L128,
-		.num_planes = 2,
+		.mem_planes = 2,
+		.comp_planes = 2,
 		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_H264,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_H264_MVC,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_HEVC,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_VC1_ANNEX_G,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_VC1_ANNEX_L,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+		.flags = V4L2_FMT_FLAG_COMPRESSED
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_MPEG2,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_MPEG4,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_XVID,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_VP8,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_H263,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
 	},
 	{0, 0, 0, 0},
 };

@@ -256,23 +269,22 @@ static int vdec_enum_fmt(struct file *file, void *fh, struct v4l2_fmtdesc *f)
 	int ret = -EINVAL;
 
 	vpu_inst_lock(inst);
-	if (!V4L2_TYPE_IS_OUTPUT(f->type) && vdec->fixed_fmt) {
-		if (f->index == 0) {
-			f->pixelformat = inst->cap_format.pixfmt;
-			f->flags = inst->cap_format.flags;
-			ret = 0;
-		}
+	if (V4L2_TYPE_IS_CAPTURE(f->type) && vdec->fixed_fmt) {
+		fmt = vpu_get_format(inst, f->type);
+		if (f->index == 1)
+			fmt = vpu_helper_find_sibling(inst, f->type, fmt->pixfmt);
+		if (f->index > 1)
+			fmt = NULL;
 	} else {
 		fmt = vpu_helper_enum_format(inst, f->type, f->index);
-		memset(f->reserved, 0, sizeof(f->reserved));
-		if (!fmt)
-			goto exit;
-
-		f->pixelformat = fmt->pixfmt;
-		f->flags = fmt->flags;
-		ret = 0;
 	}
+	if (!fmt)
+		goto exit;
 
+	memset(f->reserved, 0, sizeof(f->reserved));
+	f->pixelformat = fmt->pixfmt;
+	f->flags = fmt->flags;
+	ret = 0;
 exit:
 	vpu_inst_unlock(inst);
 	return ret;

@@ -289,14 +301,14 @@ static int vdec_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
 	cur_fmt = vpu_get_format(inst, f->type);
 
 	pixmp->pixelformat = cur_fmt->pixfmt;
-	pixmp->num_planes = cur_fmt->num_planes;
+	pixmp->num_planes = cur_fmt->mem_planes;
 	pixmp->width = cur_fmt->width;
 	pixmp->height = cur_fmt->height;
 	pixmp->field = cur_fmt->field;
 	pixmp->flags = cur_fmt->flags;
 	for (i = 0; i < pixmp->num_planes; i++) {
 		pixmp->plane_fmt[i].bytesperline = cur_fmt->bytesperline[i];
-		pixmp->plane_fmt[i].sizeimage = cur_fmt->sizeimage[i];
+		pixmp->plane_fmt[i].sizeimage = vpu_get_fmt_plane_size(cur_fmt, i);
 	}
 
 	f->fmt.pix_mp.colorspace = vdec->codec_info.color_primaries;

@@ -311,10 +323,19 @@ static int vdec_try_fmt(struct file *file, void *fh, struct v4l2_format *f)
 {
 	struct vpu_inst *inst = to_inst(file);
 	struct vdec_t *vdec = inst->priv;
-
-	vpu_try_fmt_common(inst, f);
+	struct vpu_format fmt;
 
 	vpu_inst_lock(inst);
+	if (V4L2_TYPE_IS_CAPTURE(f->type) && vdec->fixed_fmt) {
+		struct vpu_format *cap_fmt = vpu_get_format(inst, f->type);
+
+		if (!vpu_helper_match_format(inst, cap_fmt->type, cap_fmt->pixfmt,
+					     f->fmt.pix_mp.pixelformat))
+			f->fmt.pix_mp.pixelformat = cap_fmt->pixfmt;
+	}
+
+	vpu_try_fmt_common(inst, f, &fmt);
+
 	if (vdec->fixed_fmt) {
 		f->fmt.pix_mp.colorspace = vdec->codec_info.color_primaries;
 		f->fmt.pix_mp.xfer_func = vdec->codec_info.transfer_chars;

@@ -334,7 +355,7 @@ static int vdec_try_fmt(struct file *file, void *fh, struct v4l2_format *f)
 static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)
 {
 	struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
-	const struct vpu_format *fmt;
+	struct vpu_format fmt;
 	struct vpu_format *cur_fmt;
 	struct vb2_queue *q;
 	struct vdec_t *vdec = inst->priv;

@@ -349,36 +370,30 @@ static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)
 	if (vb2_is_busy(q))
 		return -EBUSY;
 
-	fmt = vpu_try_fmt_common(inst, f);
-	if (!fmt)
+	if (vpu_try_fmt_common(inst, f, &fmt))
 		return -EINVAL;
 
 	cur_fmt = vpu_get_format(inst, f->type);
 	if (V4L2_TYPE_IS_OUTPUT(f->type) && inst->state != VPU_CODEC_STATE_DEINIT) {
-		if (cur_fmt->pixfmt != fmt->pixfmt) {
+		if (cur_fmt->pixfmt != fmt.pixfmt) {
 			vdec->reset_codec = true;
 			vdec->fixed_fmt = false;
 		}
 	}
-	cur_fmt->pixfmt = fmt->pixfmt;
 	if (V4L2_TYPE_IS_OUTPUT(f->type) || !vdec->fixed_fmt) {
-		cur_fmt->num_planes = fmt->num_planes;
-		cur_fmt->flags = fmt->flags;
-		cur_fmt->width = pixmp->width;
-		cur_fmt->height = pixmp->height;
-		for (i = 0; i < fmt->num_planes; i++) {
-			cur_fmt->sizeimage[i] = pixmp->plane_fmt[i].sizeimage;
-			cur_fmt->bytesperline[i] = pixmp->plane_fmt[i].bytesperline;
-		}
-		if (pixmp->field != V4L2_FIELD_ANY)
-			cur_fmt->field = pixmp->field;
+		memcpy(cur_fmt, &fmt, sizeof(*cur_fmt));
 	} else {
-		pixmp->num_planes = cur_fmt->num_planes;
+		if (vpu_helper_match_format(inst, f->type, cur_fmt->pixfmt, pixmp->pixelformat)) {
+			cur_fmt->pixfmt = fmt.pixfmt;
+			cur_fmt->mem_planes = fmt.mem_planes;
+		}
+		pixmp->pixelformat = cur_fmt->pixfmt;
+		pixmp->num_planes = cur_fmt->mem_planes;
 		pixmp->width = cur_fmt->width;
 		pixmp->height = cur_fmt->height;
 		for (i = 0; i < pixmp->num_planes; i++) {
 			pixmp->plane_fmt[i].bytesperline = cur_fmt->bytesperline[i];
-			pixmp->plane_fmt[i].sizeimage = cur_fmt->sizeimage[i];
+			pixmp->plane_fmt[i].sizeimage = vpu_get_fmt_plane_size(cur_fmt, i);
 		}
 		pixmp->field = cur_fmt->field;
 	}

@@ -678,9 +693,11 @@ static struct vpu_vb2_buffer *vdec_find_buffer(struct vpu_inst *inst, u32 luma)
 static void vdec_buf_done(struct vpu_inst *inst, struct vpu_frame_info *frame)
 {
 	struct vdec_t *vdec = inst->priv;
+	struct vpu_format *cur_fmt;
 	struct vpu_vb2_buffer *vpu_buf;
 	struct vb2_v4l2_buffer *vbuf;
 	u32 sequence;
+	int i;
 
 	if (!frame)
 		return;

@@ -699,6 +716,7 @@ static void vdec_buf_done(struct vpu_inst *inst, struct vpu_frame_info *frame)
 		return;
 	}
 
+	cur_fmt = vpu_get_format(inst, inst->cap_format.type);
 	vbuf = &vpu_buf->m2m_buf.vb;
 	if (vbuf->vb2_buf.index != frame->id)
 		dev_err(inst->dev, "[%d] buffer id(%d, %d) dismatch\n",

@@ -707,9 +725,9 @@ static void vdec_buf_done(struct vpu_inst *inst, struct vpu_frame_info *frame)
 	if (vpu_get_buffer_state(vbuf) != VPU_BUF_STATE_DECODED)
 		dev_err(inst->dev, "[%d] buffer(%d) ready without decoded\n", inst->id, frame->id);
 	vpu_set_buffer_state(vbuf, VPU_BUF_STATE_READY);
-	vb2_set_plane_payload(&vbuf->vb2_buf, 0, inst->cap_format.sizeimage[0]);
-	vb2_set_plane_payload(&vbuf->vb2_buf, 1, inst->cap_format.sizeimage[1]);
-	vbuf->field = inst->cap_format.field;
+	for (i = 0; i < vbuf->vb2_buf.num_planes; i++)
+		vb2_set_plane_payload(&vbuf->vb2_buf, i, vpu_get_fmt_plane_size(cur_fmt, i));
+	vbuf->field = cur_fmt->field;
 	vbuf->sequence = sequence;
 	dev_dbg(inst->dev, "[%d][OUTPUT TS]%32lld\n", inst->id, vbuf->vb2_buf.timestamp);

@@ -747,15 +765,17 @@ static void vdec_stop_done(struct vpu_inst *inst)
 static bool vdec_check_source_change(struct vpu_inst *inst)
 {
 	struct vdec_t *vdec = inst->priv;
-	const struct vpu_format *fmt;
-	int i;
+	const struct vpu_format *sibling;
 
 	if (!inst->fh.m2m_ctx)
 		return false;
 
+	sibling = vpu_helper_find_sibling(inst, inst->cap_format.type, inst->cap_format.pixfmt);
+	if (sibling && vdec->codec_info.pixfmt == sibling->pixfmt)
+		vdec->codec_info.pixfmt = inst->cap_format.pixfmt;
+
 	if (!vb2_is_streaming(v4l2_m2m_get_dst_vq(inst->fh.m2m_ctx)))
 		return true;
-	fmt = vpu_helper_find_format(inst, inst->cap_format.type, vdec->codec_info.pixfmt);
 	if (inst->cap_format.pixfmt != vdec->codec_info.pixfmt)
 		return true;
 	if (inst->cap_format.width != vdec->codec_info.decoded_width)

@@ -772,14 +792,6 @@ static bool vdec_check_source_change(struct vpu_inst *inst)
 		return true;
 	if (inst->crop.height != vdec->codec_info.height)
 		return true;
-	if (fmt && inst->cap_format.num_planes != fmt->num_planes)
-		return true;
-	for (i = 0; i < inst->cap_format.num_planes; i++) {
-		if (inst->cap_format.bytesperline[i] != vdec->codec_info.bytesperline[i])
-			return true;
-		if (inst->cap_format.sizeimage[i] != vdec->codec_info.sizeimage[i])
-			return true;
-	}
 
 	return false;
 }

@@ -787,27 +799,21 @@ static bool vdec_check_source_change(struct vpu_inst *inst)
 static void vdec_init_fmt(struct vpu_inst *inst)
 {
 	struct vdec_t *vdec = inst->priv;
-	const struct vpu_format *fmt;
-	int i;
+	struct v4l2_format f;
 
-	fmt = vpu_helper_find_format(inst, inst->cap_format.type, vdec->codec_info.pixfmt);
-	inst->out_format.width = vdec->codec_info.width;
-	inst->out_format.height = vdec->codec_info.height;
-	inst->cap_format.width = vdec->codec_info.decoded_width;
-	inst->cap_format.height = vdec->codec_info.decoded_height;
-	inst->cap_format.pixfmt = vdec->codec_info.pixfmt;
-	if (fmt) {
-		inst->cap_format.num_planes = fmt->num_planes;
-		inst->cap_format.flags = fmt->flags;
-	}
-	for (i = 0; i < inst->cap_format.num_planes; i++) {
-		inst->cap_format.bytesperline[i] = vdec->codec_info.bytesperline[i];
-		inst->cap_format.sizeimage[i] = vdec->codec_info.sizeimage[i];
-	}
+	memset(&f, 0, sizeof(f));
+	f.type = inst->cap_format.type;
+	f.fmt.pix_mp.pixelformat = vdec->codec_info.pixfmt;
+	f.fmt.pix_mp.width = vdec->codec_info.decoded_width;
+	f.fmt.pix_mp.height = vdec->codec_info.decoded_height;
 	if (vdec->codec_info.progressive)
-		inst->cap_format.field = V4L2_FIELD_NONE;
+		f.fmt.pix_mp.field = V4L2_FIELD_NONE;
 	else
-		inst->cap_format.field = V4L2_FIELD_SEQ_TB;
+		f.fmt.pix_mp.field = V4L2_FIELD_SEQ_TB;
+	vpu_try_fmt_common(inst, &f, &inst->cap_format);
+
+	inst->out_format.width = vdec->codec_info.width;
+	inst->out_format.height = vdec->codec_info.height;
 }
 
 static void vdec_init_crop(struct vpu_inst *inst)

@@ -966,7 +972,10 @@ static int vdec_response_frame(struct vpu_inst *inst, struct vb2_v4l2_buffer *vb
 	info.tag = vdec->seq_tag;
 	info.luma_addr = vpu_get_vb_phy_addr(&vbuf->vb2_buf, 0);
 	info.luma_size = inst->cap_format.sizeimage[0];
-	info.chroma_addr = vpu_get_vb_phy_addr(&vbuf->vb2_buf, 1);
+	if (vbuf->vb2_buf.num_planes > 1)
+		info.chroma_addr = vpu_get_vb_phy_addr(&vbuf->vb2_buf, 1);
+	else
+		info.chroma_addr = info.luma_addr + info.luma_size;
 	info.chromau_size = inst->cap_format.sizeimage[1];
 	info.bytesperline = inst->cap_format.bytesperline[0];
 	ret = vpu_session_alloc_fs(inst, &info);

@@ -975,7 +984,7 @@ static int vdec_response_frame(struct vpu_inst *inst, struct vb2_v4l2_buffer *vb
 
 	vpu_buf->tag = info.tag;
 	vpu_buf->luma = info.luma_addr;
-	vpu_buf->chroma_u = info.chromau_size;
+	vpu_buf->chroma_u = info.chroma_addr;
 	vpu_buf->chroma_v = 0;
 	vpu_set_buffer_state(vbuf, VPU_BUF_STATE_INUSE);
 	vdec->slots[info.id] = vpu_buf;

drivers/media/platform/amphion/venc.c +12 ~~-21~~

index 37212f087fdd..060a1ee78b17 100644
@@ -69,13 +69,16 @@ struct venc_frame_t {
 static const struct vpu_format venc_formats[] = {
 	{
 		.pixfmt = V4L2_PIX_FMT_NV12M,
-		.num_planes = 2,
+		.mem_planes = 2,
+		.comp_planes = 2,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_H264,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.flags = V4L2_FMT_FLAG_COMPRESSED
 	},
 	{0, 0, 0, 0},
 };

@@ -173,14 +176,14 @@ static int venc_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
 	cur_fmt = vpu_get_format(inst, f->type);
 
 	pixmp->pixelformat = cur_fmt->pixfmt;
-	pixmp->num_planes = cur_fmt->num_planes;
+	pixmp->num_planes = cur_fmt->mem_planes;
 	pixmp->width = cur_fmt->width;
 	pixmp->height = cur_fmt->height;
 	pixmp->field = cur_fmt->field;
 	pixmp->flags = cur_fmt->flags;
 	for (i = 0; i < pixmp->num_planes; i++) {
 		pixmp->plane_fmt[i].bytesperline = cur_fmt->bytesperline[i];
-		pixmp->plane_fmt[i].sizeimage = cur_fmt->sizeimage[i];
+		pixmp->plane_fmt[i].sizeimage = vpu_get_fmt_plane_size(cur_fmt, i);
 	}
 
 	f->fmt.pix_mp.colorspace = venc->params.color.primaries;

@@ -194,8 +197,9 @@ static int venc_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
 static int venc_try_fmt(struct file *file, void *fh, struct v4l2_format *f)
 {
 	struct vpu_inst *inst = to_inst(file);
+	struct vpu_format fmt;
 
-	vpu_try_fmt_common(inst, f);
+	vpu_try_fmt_common(inst, f, &fmt);
 
 	return 0;
 }

@@ -203,12 +207,11 @@ static int venc_try_fmt(struct file *file, void *fh, struct v4l2_format *f)
 static int venc_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
 {
 	struct vpu_inst *inst = to_inst(file);
-	const struct vpu_format *fmt;
+	struct vpu_format fmt;
 	struct vpu_format *cur_fmt;
 	struct vb2_queue *q;
 	struct venc_t *venc = inst->priv;
 	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
-	int i;
 
 	q = v4l2_m2m_get_vq(inst->fh.m2m_ctx, f->type);
 	if (!q)

@@ -216,24 +219,12 @@ static int venc_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
 	if (vb2_is_busy(q))
 		return -EBUSY;
 
-	fmt = vpu_try_fmt_common(inst, f);
-	if (!fmt)
+	if (vpu_try_fmt_common(inst, f, &fmt))
 		return -EINVAL;
 
 	cur_fmt = vpu_get_format(inst, f->type);
 
-	cur_fmt->pixfmt = fmt->pixfmt;
-	cur_fmt->num_planes = fmt->num_planes;
-	cur_fmt->flags = fmt->flags;
-	cur_fmt->width = pix_mp->width;
-	cur_fmt->height = pix_mp->height;
-	for (i = 0; i < fmt->num_planes; i++) {
-		cur_fmt->sizeimage[i] = pix_mp->plane_fmt[i].sizeimage;
-		cur_fmt->bytesperline[i] = pix_mp->plane_fmt[i].bytesperline;
-	}
-
-	if (pix_mp->field != V4L2_FIELD_ANY)
-		cur_fmt->field = pix_mp->field;
+	memcpy(cur_fmt, &fmt, sizeof(*cur_fmt));
 
 	if (V4L2_TYPE_IS_OUTPUT(f->type)) {
 		venc->params.input_format = cur_fmt->pixfmt;

drivers/media/platform/amphion/vpu.h +3 -1

index f914de6ed81e..998591fa5b85 100644
@@ -84,7 +84,8 @@ struct vpu_dev {
 
 struct vpu_format {
 	u32 pixfmt;
-	unsigned int num_planes;
+	u32 mem_planes;
+	u32 comp_planes;
 	u32 type;
 	u32 flags;
 	u32 width;

@@ -92,6 +93,7 @@ struct vpu_format {
 	u32 sizeimage[VIDEO_MAX_PLANES];
 	u32 bytesperline[VIDEO_MAX_PLANES];
 	u32 field;
+	u32 sibling;
 };
 
 struct vpu_core_resources {

drivers/media/platform/amphion/vpu_dbg.c +4 -4

index f72c8a506b22..c41c9896912c 100644
@@ -89,9 +89,9 @@ static int vpu_dbg_instance(struct seq_file *s, void *data)
 			vq->last_buffer_dequeued);
 	if (seq_write(s, str, num))
 		return 0;
-	for (i = 0; i < inst->out_format.num_planes; i++) {
+	for (i = 0; i < inst->out_format.mem_planes; i++) {
 		num = scnprintf(str, sizeof(str), " %d(%d)",
-				inst->out_format.sizeimage[i],
+				vpu_get_fmt_plane_size(&inst->out_format, i),
 				inst->out_format.bytesperline[i]);
 		if (seq_write(s, str, num))
 			return 0;

@@ -113,9 +113,9 @@ static int vpu_dbg_instance(struct seq_file *s, void *data)
 			vq->last_buffer_dequeued);
 	if (seq_write(s, str, num))
 		return 0;
-	for (i = 0; i < inst->cap_format.num_planes; i++) {
+	for (i = 0; i < inst->cap_format.mem_planes; i++) {
 		num = scnprintf(str, sizeof(str), " %d(%d)",
-				inst->cap_format.sizeimage[i],
+				vpu_get_fmt_plane_size(&inst->cap_format, i),
 				inst->cap_format.bytesperline[i]);
 		if (seq_write(s, str, num))
 			return 0;

drivers/media/platform/amphion/vpu_helpers.c +43 -5

index e9aeb3453dfc..db8f471b9187 100644
@@ -59,6 +59,39 @@ const struct vpu_format *vpu_helper_find_format(struct vpu_inst *inst, u32 type,
 	return NULL;
 }
 
+const struct vpu_format *vpu_helper_find_sibling(struct vpu_inst *inst, u32 type, u32 pixelfmt)
+{
+	const struct vpu_format *fmt;
+	const struct vpu_format *sibling;
+
+	fmt = vpu_helper_find_format(inst, type, pixelfmt);
+	if (!fmt)
+		return NULL;
+	if (!fmt->sibling)
+		return NULL;
+	sibling = vpu_helper_find_format(inst, type, fmt->sibling);
+	if (!sibling)
+		return NULL;
+	if (sibling->sibling != fmt->pixfmt)
+		return NULL;
+	if (sibling->comp_planes != fmt->comp_planes)
+		return NULL;
+	return sibling;
+}
+
+bool vpu_helper_match_format(struct vpu_inst *inst, u32 type, u32 fmta, u32 fmtb)
+{
+	const struct vpu_format *sibling;
+
+	if (fmta == fmtb)
+		return true;
+
+	sibling = vpu_helper_find_sibling(inst, type, fmta);
+	if (sibling && sibling->pixfmt == fmtb)
+		return true;
+	return false;
+}
+
 const struct vpu_format *vpu_helper_enum_format(struct vpu_inst *inst, u32 type, int index)
 {
 	const struct vpu_format *pfmt;

@@ -123,9 +156,10 @@ static u32 get_nv12_plane_size(u32 width, u32 height, int plane_no,
 	u32 bytesperline;
 	u32 size = 0;
 
-	bytesperline = ALIGN(width, stride);
+	bytesperline = width;
 	if (pbl)
 		bytesperline = max(bytesperline, *pbl);
+	bytesperline = ALIGN(bytesperline, stride);
 	height = ALIGN(height, 2);
 	if (plane_no == 0)
 		size = bytesperline * height;

@@ -148,13 +182,13 @@ static u32 get_tiled_8l128_plane_size(u32 fmt, u32 width, u32 height, int plane_
 
 	if (interlaced)
 		hs++;
-	if (fmt == V4L2_PIX_FMT_NV12M_10BE_8L128)
+	if (fmt == V4L2_PIX_FMT_NV12M_10BE_8L128 || fmt == V4L2_PIX_FMT_NV12_10BE_8L128)
 		bitdepth = 10;
 	bytesperline = DIV_ROUND_UP(width * bitdepth, BITS_PER_BYTE);
-	bytesperline = ALIGN(bytesperline, 1 << ws);
-	bytesperline = ALIGN(bytesperline, stride);
 	if (pbl)
 		bytesperline = max(bytesperline, *pbl);
+	bytesperline = ALIGN(bytesperline, 1 << ws);
+	bytesperline = ALIGN(bytesperline, stride);
 	height = ALIGN(height, 1 << hs);
 	if (plane_no == 0)
 		size = bytesperline * height;

@@ -172,9 +206,10 @@ static u32 get_default_plane_size(u32 width, u32 height, int plane_no,
 	u32 bytesperline;
 	u32 size = 0;
 
-	bytesperline = ALIGN(width, stride);
+	bytesperline = width;
 	if (pbl)
 		bytesperline = max(bytesperline, *pbl);
+	bytesperline = ALIGN(bytesperline, stride);
 	if (plane_no == 0)
 		size = bytesperline * height;
 	if (pbl)

@@ -187,9 +222,12 @@ u32 vpu_helper_get_plane_size(u32 fmt, u32 w, u32 h, int plane_no,
 			      u32 stride, u32 interlaced, u32 *pbl)
 {
 	switch (fmt) {
+	case V4L2_PIX_FMT_NV12:
 	case V4L2_PIX_FMT_NV12M:
 		return get_nv12_plane_size(w, h, plane_no, stride, interlaced, pbl);
+	case V4L2_PIX_FMT_NV12_8L128:
 	case V4L2_PIX_FMT_NV12M_8L128:
+	case V4L2_PIX_FMT_NV12_10BE_8L128:
 	case V4L2_PIX_FMT_NV12M_10BE_8L128:
 		return get_tiled_8l128_plane_size(fmt, w, h, plane_no, stride, interlaced, pbl);
 	default:

drivers/media/platform/amphion/vpu_helpers.h +2 -0

index bc28350958be..0eaddb07190d 100644
@@ -14,6 +14,8 @@ struct vpu_pair {
 int vpu_helper_find_in_array_u8(const u8 *array, u32 size, u32 x);
 bool vpu_helper_check_type(struct vpu_inst *inst, u32 type);
 const struct vpu_format *vpu_helper_find_format(struct vpu_inst *inst, u32 type, u32 pixelfmt);
+const struct vpu_format *vpu_helper_find_sibling(struct vpu_inst *inst, u32 type, u32 pixelfmt);
+bool vpu_helper_match_format(struct vpu_inst *inst, u32 type, u32 fmta, u32 fmtb);
 const struct vpu_format *vpu_helper_enum_format(struct vpu_inst *inst, u32 type, int index);
 u32 vpu_helper_valid_frame_width(struct vpu_inst *inst, u32 width);
 u32 vpu_helper_valid_frame_height(struct vpu_inst *inst, u32 height);

drivers/media/platform/amphion/vpu_malone.c +2 -1

index 51e0702f9ae1..69d9a2269fce 100644
@@ -583,7 +583,8 @@ bool vpu_malone_check_fmt(enum vpu_core_type type, u32 pixelfmt)
 	if (!vpu_imx8q_check_fmt(type, pixelfmt))
 		return false;
 
-	if (pixelfmt == V4L2_PIX_FMT_NV12M_8L128 || pixelfmt == V4L2_PIX_FMT_NV12M_10BE_8L128)
+	if (pixelfmt == V4L2_PIX_FMT_NV12_8L128 || pixelfmt == V4L2_PIX_FMT_NV12_10BE_8L128 ||
+	    pixelfmt == V4L2_PIX_FMT_NV12M_8L128 || pixelfmt == V4L2_PIX_FMT_NV12M_10BE_8L128)
 		return true;
 	if (vpu_malone_format_remap(pixelfmt) == MALONE_FMT_NULL)
 		return false;

drivers/media/platform/amphion/vpu_v4l2.c +126 ~~-42~~

index 8a3eed957ae6..87cce552e906 100644
@@ -140,51 +140,135 @@ bool vpu_is_source_empty(struct vpu_inst *inst)
 	return true;
 }
 
-const struct vpu_format *vpu_try_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)
+static int vpu_init_format(struct vpu_inst *inst, struct vpu_format *fmt)
+{
+	const struct vpu_format *info;
+
+	info = vpu_helper_find_format(inst, fmt->type, fmt->pixfmt);
+	if (!info) {
+		info = vpu_helper_enum_format(inst, fmt->type, 0);
+		if (!info)
+			return -EINVAL;
+	}
+	memcpy(fmt, info, sizeof(*fmt));
+
+	return 0;
+}
+
+static int vpu_calc_fmt_bytesperline(struct v4l2_format *f, struct vpu_format *fmt)
 {
 	struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
-	u32 type = f->type;
+	int i;
+
+	if (fmt->flags & V4L2_FMT_FLAG_COMPRESSED) {
+		for (i = 0; i < fmt->comp_planes; i++)
+			fmt->bytesperline[i] = 0;
+		return 0;
+	}
+	if (pixmp->num_planes == fmt->comp_planes) {
+		for (i = 0; i < fmt->comp_planes; i++)
+			fmt->bytesperline[i] = pixmp->plane_fmt[i].bytesperline;
+		return 0;
+	}
+	if (pixmp->num_planes > 1)
+		return -EINVAL;
+
+	/*amphion vpu only support nv12 and nv12 tiled,
+	 * so the bytesperline of luma and chroma should be same
+	 */
+	for (i = 0; i < fmt->comp_planes; i++)
+		fmt->bytesperline[i] = pixmp->plane_fmt[0].bytesperline;
+
+	return 0;
+}
+
+static int vpu_calc_fmt_sizeimage(struct vpu_inst *inst, struct vpu_format *fmt)
+{
 	u32 stride = 1;
-	u32 bytesperline;
-	u32 sizeimage;
-	const struct vpu_format *fmt;
-	const struct vpu_core_resources *res;
 	int i;
 
-	fmt = vpu_helper_find_format(inst, type, pixmp->pixelformat);
-	if (!fmt) {
-		fmt = vpu_helper_enum_format(inst, type, 0);
-		if (!fmt)
-			return NULL;
-		pixmp->pixelformat = fmt->pixfmt;
+	if (!(fmt->flags & V4L2_FMT_FLAG_COMPRESSED)) {
+		const struct vpu_core_resources *res = vpu_get_resource(inst);
+
+		if (res)
+			stride = res->stride;
 	}
 
-	res = vpu_get_resource(inst);
-	if (res)
-		stride = res->stride;
-	if (pixmp->width)
-		pixmp->width = vpu_helper_valid_frame_width(inst, pixmp->width);
-	if (pixmp->height)
-		pixmp->height = vpu_helper_valid_frame_height(inst, pixmp->height);
+	for (i = 0; i < fmt->comp_planes; i++) {
+		fmt->sizeimage[i] = vpu_helper_get_plane_size(fmt->pixfmt,
+							      fmt->width,
+							      fmt->height,
+							      i,
+							      stride,
+							      fmt->field != V4L2_FIELD_NONE ? 1 : 0,
+							      &fmt->bytesperline[i]);
+		fmt->sizeimage[i] = max_t(u32, fmt->sizeimage[i], PAGE_SIZE);
+		if (fmt->flags & V4L2_FMT_FLAG_COMPRESSED) {
+			fmt->sizeimage[i] = clamp_val(fmt->sizeimage[i], SZ_128K, SZ_8M);
+			fmt->bytesperline[i] = 0;
+		}
+	}
+
+	return 0;
+}
+
+u32 vpu_get_fmt_plane_size(struct vpu_format *fmt, u32 plane_no)
+{
+	u32 size;
+	int i;
+
+	if (plane_no >= fmt->mem_planes)
+		return 0;
+
+	if (fmt->comp_planes == fmt->mem_planes)
+		return fmt->sizeimage[plane_no];
+	if (plane_no < fmt->mem_planes - 1)
+		return fmt->sizeimage[plane_no];
+	size = fmt->sizeimage[plane_no];
+	for (i = fmt->mem_planes; i < fmt->comp_planes; i++)
+		size += fmt->sizeimage[i];
+
+	return size;
+}
+
+int vpu_try_fmt_common(struct vpu_inst *inst, struct v4l2_format *f, struct vpu_format *fmt)
+{
+	struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
+	int i;
+	int ret;
+
+	fmt->pixfmt = pixmp->pixelformat;
+	fmt->type = f->type;
+	ret = vpu_init_format(inst, fmt);
+	if (ret < 0)
+		return ret;
+
+	fmt->width = pixmp->width;
+	fmt->height = pixmp->height;
+	if (fmt->width)
+		fmt->width = vpu_helper_valid_frame_width(inst, fmt->width);
+	if (fmt->height)
+		fmt->height = vpu_helper_valid_frame_height(inst, fmt->height);
+	fmt->field = pixmp->field == V4L2_FIELD_ANY ? V4L2_FIELD_NONE : pixmp->field;
+	vpu_calc_fmt_bytesperline(f, fmt);
+	vpu_calc_fmt_sizeimage(inst, fmt);
+	if ((fmt->flags & V4L2_FMT_FLAG_COMPRESSED) && pixmp->plane_fmt[0].sizeimage)
+		fmt->sizeimage[0] = clamp_val(pixmp->plane_fmt[0].sizeimage, SZ_128K, SZ_8M);
+
+	pixmp->pixelformat = fmt->pixfmt;
+	pixmp->width = fmt->width;
+	pixmp->height = fmt->height;
 	pixmp->flags = fmt->flags;
-	pixmp->num_planes = fmt->num_planes;
-	if (pixmp->field == V4L2_FIELD_ANY)
-		pixmp->field = V4L2_FIELD_NONE;
+	pixmp->num_planes = fmt->mem_planes;
+	pixmp->field = fmt->field;
+	memset(pixmp->reserved, 0, sizeof(pixmp->reserved));
 	for (i = 0; i < pixmp->num_planes; i++) {
-		bytesperline = max_t(s32, pixmp->plane_fmt[i].bytesperline, 0);
-		sizeimage = vpu_helper_get_plane_size(pixmp->pixelformat,
-						      pixmp->width,
-						      pixmp->height,
-						      i,
-						      stride,
-						      pixmp->field > V4L2_FIELD_NONE ? 1 : 0,
-						      &bytesperline);
-		sizeimage = max_t(s32, pixmp->plane_fmt[i].sizeimage, sizeimage);
-		pixmp->plane_fmt[i].bytesperline = bytesperline;
-		pixmp->plane_fmt[i].sizeimage = sizeimage;
+		pixmp->plane_fmt[i].bytesperline = fmt->bytesperline[i];
+		pixmp->plane_fmt[i].sizeimage = vpu_get_fmt_plane_size(fmt, i);
+		memset(pixmp->plane_fmt[i].reserved, 0, sizeof(pixmp->plane_fmt[i].reserved));
 	}
 
-	return fmt;
+	return 0;
 }
 
 static bool vpu_check_ready(struct vpu_inst *inst, u32 type)

@@ -389,10 +473,10 @@ static int vpu_vb2_queue_setup(struct vb2_queue *vq,
 	cur_fmt = vpu_get_format(inst, vq->type);
 
 	if (*plane_count) {
-		if (*plane_count != cur_fmt->num_planes)
+		if (*plane_count != cur_fmt->mem_planes)
 			return -EINVAL;
-		for (i = 0; i < cur_fmt->num_planes; i++) {
-			if (psize[i] < cur_fmt->sizeimage[i])
+		for (i = 0; i < cur_fmt->mem_planes; i++) {
+			if (psize[i] < vpu_get_fmt_plane_size(cur_fmt, i))
 				return -EINVAL;
 		}
 		return 0;

@@ -402,9 +486,9 @@ static int vpu_vb2_queue_setup(struct vb2_queue *vq,
 		*buf_count = max_t(unsigned int, *buf_count, inst->min_buffer_out);
 	else
 		*buf_count = max_t(unsigned int, *buf_count, inst->min_buffer_cap);
-	*plane_count = cur_fmt->num_planes;
-	for (i = 0; i < cur_fmt->num_planes; i++)
-		psize[i] = cur_fmt->sizeimage[i];
+	*plane_count = cur_fmt->mem_planes;
+	for (i = 0; i < cur_fmt->mem_planes; i++)
+		psize[i] = vpu_get_fmt_plane_size(cur_fmt, i);
 
 	return 0;
 }

@@ -434,8 +518,8 @@ static int vpu_vb2_buf_prepare(struct vb2_buffer *vb)
 	u32 i;
 
 	cur_fmt = vpu_get_format(inst, vb->type);
-	for (i = 0; i < cur_fmt->num_planes; i++) {
-		if (vpu_get_vb_length(vb, i) < cur_fmt->sizeimage[i]) {
+	for (i = 0; i < cur_fmt->mem_planes; i++) {
+		if (vpu_get_vb_length(vb, i) < vpu_get_fmt_plane_size(cur_fmt, i)) {
 			dev_dbg(inst->dev, "[%d] %s buf[%d] is invalid\n",
 				inst->id, vpu_type_name(vb->type), vb->index);
 			vpu_set_buffer_state(vbuf, VPU_BUF_STATE_ERROR);

drivers/media/platform/amphion/vpu_v4l2.h +2 -1

index 795ca33a6a50..ef5de6b66e47 100644
@@ -16,7 +16,8 @@ unsigned int vpu_get_buffer_state(struct vb2_v4l2_buffer *vbuf);
 int vpu_v4l2_open(struct file *file, struct vpu_inst *inst);
 int vpu_v4l2_close(struct file *file);
 
-const struct vpu_format *vpu_try_fmt_common(struct vpu_inst *inst, struct v4l2_format *f);
+u32 vpu_get_fmt_plane_size(struct vpu_format *fmt, u32 plane_no);
+int vpu_try_fmt_common(struct vpu_inst *inst, struct v4l2_format *f, struct vpu_format *fmt);
 int vpu_process_output_buffer(struct vpu_inst *inst);
 int vpu_process_capture_buffer(struct vpu_inst *inst);
 struct vb2_v4l2_buffer *vpu_next_src_buf(struct vpu_inst *inst);

drivers/media/platform/amphion/vpu_windsor.c +7 -1

index 1526af2ef9da..a454f142ae17 100644
@@ -775,6 +775,8 @@ static int vpu_windsor_fill_yuv_frame(struct vpu_shared_addr *shared,
 				      u32 instance,
 				      struct vb2_buffer *vb)
 {
+	struct vpu_inst *inst = vb2_get_drv_priv(vb->vb2_queue);
+	struct vpu_format *out_fmt;
 	struct vpu_enc_yuv_desc *desc;
 	struct vb2_v4l2_buffer *vbuf;

@@ -782,6 +784,7 @@ static int vpu_windsor_fill_yuv_frame(struct vpu_shared_addr *shared,
 		return -EINVAL;
 
 	desc = get_yuv_desc(shared, instance);
+	out_fmt = vpu_get_format(inst, vb->type);
 
 	vbuf = to_vb2_v4l2_buffer(vb);
 	desc->frame_id = vbuf->sequence;

@@ -790,7 +793,10 @@ static int vpu_windsor_fill_yuv_frame(struct vpu_shared_addr *shared,
 	else
 		desc->key_frame = 0;
 	desc->luma_base = vpu_get_vb_phy_addr(vb, 0);
-	desc->chroma_base = vpu_get_vb_phy_addr(vb, 1);
+	if (vb->num_planes > 1)
+		desc->chroma_base = vpu_get_vb_phy_addr(vb, 1);
+	else
+		desc->chroma_base = desc->luma_base + out_fmt->sizeimage[0];
 
 	return 0;
 }

Signature · 1 line

2.37.1

Signature · 10 lines

Tommaso Merciai
Embedded Linux Engineer
[email protected]
__________________________________

Amarula Solutions SRL
Via Le Canevare 30, 31100 Treviso, Veneto, IT
T. +39 042 243 5310
[email protected]
www.amarulasolutions.com

Ming Qian

author maintainer

2022-08-12 07:20 UTC #8 lore raw

RE: [EXT] Re: [PATCH v4 2/4] media: amphion: tell and handle contiguous and non contiguous format

Quoted 11 lines

From: Tommaso Merciai <[email protected]>
Sent: 2022年8月12日 15:15
To: Ming Qian <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; dl-linux-imx <[email protected]>; X.H. Bao <[email protected]>; [email protected]; [email protected]; [email protected]
Subject: [EXT] Re: [PATCH v4 2/4] media: amphion: tell and handle contiguous and non contiguous format

Quoted 5 lines

Caution: EXT Email
Hi Ming,
Just an info. I'm planning to help to review your series. On which Linux tree are you working?

Hi Tommaso,
Thanks for your help, and I'm working on https://git.linuxtv.org/media_tree.git

Ming

Quoted 1,046 lines

Thanks in advance,
Tommaso

drivers/media/platform/amphion/vdec.c static int vdec_enum_fmt(struct file *file, void *fh,

Quoted 145 lines · Ming Qian

Driver should tell the number of memory planes and component planes. the amphion vpu support non contiguous planes, but for compatibility with other device that only support contiguous planes. driver can add support for contiguous planes in the same time. Then the mem_planes can be different from the comp_planes. driver need to handle buffer according mem_planes and comp_planes.

So driver can support NV12 and NV12M.

Signed-off-by Ming Qian <[email protected]>

---
 drivers/media/platform/amphion/vdec.c        | 187 ++++++++++---------
 drivers/media/platform/amphion/venc.c        |  33 ++--
 drivers/media/platform/amphion/vpu.h         |   4 +-
 drivers/media/platform/amphion/vpu_dbg.c     |   8 +-
 drivers/media/platform/amphion/vpu_helpers.c |  48 ++++-
 drivers/media/platform/amphion/vpu_helpers.h |   2 +
 drivers/media/platform/amphion/vpu_malone.c  |   3 +-
 drivers/media/platform/amphion/vpu_v4l2.c    | 168 ++++++++++++-----
 drivers/media/platform/amphion/vpu_v4l2.h    |   3 +-
 drivers/media/platform/amphion/vpu_windsor.c |   8 +-
 10 files changed, 299 insertions(+), 165 deletions(-)

drivers/media/platform/amphion/vdec.c +43 ~~-21~~

index feb75dc204de..48ab664fa7ef 100644
@@ -69,72 +69,85 @@ struct vdec_t {
 static const struct vpu_format vdec_formats[] = {
      {
              .pixfmt = V4L2_PIX_FMT_NV12M_8L128,
-             .num_planes = 2,
+             .mem_planes = 2,
+             .comp_planes = 2,
              .type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
      },
      {
              .pixfmt = V4L2_PIX_FMT_NV12M_10BE_8L128,
-             .num_planes = 2,
+             .mem_planes = 2,
+             .comp_planes = 2,
              .type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
      },
      {
              .pixfmt = V4L2_PIX_FMT_H264,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_H264_MVC,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_HEVC,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_VC1_ANNEX_G,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_VC1_ANNEX_L,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+             .flags = V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_MPEG2,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_MPEG4,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_XVID,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_VP8,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_H263,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {0, 0, 0, 0},
 };

@@ -256,23 +269,22 @@ static int vdec_enum_fmt(struct file *file, void *fh,

struct v4l2_fmtdesc *f)

drivers/media/platform/amphion/vdec.c static int vdec_enum_fmt(struct file *file, void *fh, struct v4l2_fmtdesc *f)

Quoted 13 lines

drivers/media/platform/amphion/vdec.c +4 -6

      int ret = -EINVAL;
 
      vpu_inst_lock(inst);
-     if (!V4L2_TYPE_IS_OUTPUT(f->type) && vdec->fixed_fmt) {
-             if (f->index == 0) {
-                     f->pixelformat = inst->cap_format.pixfmt;
-                     f->flags = inst->cap_format.flags;
-                     ret = 0;
-             }
+     if (V4L2_TYPE_IS_CAPTURE(f->type) && vdec->fixed_fmt) {
+             fmt = vpu_get_format(inst, f->type);
+             if (f->index == 1)
+                     fmt = vpu_helper_find_sibling(inst, f->type,

fmt->pixfmt);

drivers/media/platform/amphion/vdec.c static int vdec_g_fmt(struct file *file, void *fh,

Quoted 23 lines

drivers/media/platform/amphion/vdec.c +8 -7

+             if (f->index > 1)
+                     fmt = NULL;
      } else {
              fmt = vpu_helper_enum_format(inst, f->type, f->index);
-             memset(f->reserved, 0, sizeof(f->reserved));
-             if (!fmt)
-                     goto exit;
-
-             f->pixelformat = fmt->pixfmt;
-             f->flags = fmt->flags;
-             ret = 0;
      }
+     if (!fmt)
+             goto exit;
 
+     memset(f->reserved, 0, sizeof(f->reserved));
+     f->pixelformat = fmt->pixfmt;
+     f->flags = fmt->flags;
+     ret = 0;
 exit:
      vpu_inst_unlock(inst);
      return ret;

@@ -289,14 +301,14 @@ static int vdec_g_fmt(struct file *file, void *fh,

struct v4l2_format *f)

Quoted 21 lines

drivers/media/platform/amphion/vdec.c +3 -3

      cur_fmt = vpu_get_format(inst, f->type);
 
      pixmp->pixelformat = cur_fmt->pixfmt;
-     pixmp->num_planes = cur_fmt->num_planes;
+     pixmp->num_planes = cur_fmt->mem_planes;
      pixmp->width = cur_fmt->width;
      pixmp->height = cur_fmt->height;
      pixmp->field = cur_fmt->field;
      pixmp->flags = cur_fmt->flags;
      for (i = 0; i < pixmp->num_planes; i++) {
              pixmp->plane_fmt[i].bytesperline = cur_fmt->bytesperline[i];
-             pixmp->plane_fmt[i].sizeimage = cur_fmt->sizeimage[i];
+             pixmp->plane_fmt[i].sizeimage =
+ vpu_get_fmt_plane_size(cur_fmt, i);
      }
 
      f->fmt.pix_mp.colorspace = vdec->codec_info.color_primaries; @@
-311,10 +323,19 @@ static int vdec_try_fmt(struct file *file, void

*fh, struct v4l2_format *f)  {
      struct vpu_inst *inst = to_inst(file);
      struct vdec_t *vdec = inst->priv;

drivers/media/platform/amphion/vdec.c static int vdec_try_fmt(struct file *file, void *fh, struct v4l2_format *f)

Quoted 10 lines

drivers/media/platform/amphion/vdec.c +6 -2

-
-     vpu_try_fmt_common(inst, f);
+     struct vpu_format fmt;
 
      vpu_inst_lock(inst);
+     if (V4L2_TYPE_IS_CAPTURE(f->type) && vdec->fixed_fmt) {
+             struct vpu_format *cap_fmt = vpu_get_format(inst,
+ f->type);
+
+             if (!vpu_helper_match_format(inst, cap_fmt->type,

cap_fmt->pixfmt,

drivers/media/platform/amphion/vdec.c static int vdec_try_fmt(struct file *file, void *fh, struct v4l2_format *f)

Quoted 8 lines

drivers/media/platform/amphion/vdec.c +6 -0

+                                          f->fmt.pix_mp.pixelformat))
+                     f->fmt.pix_mp.pixelformat = cap_fmt->pixfmt;
+     }
+
+     vpu_try_fmt_common(inst, f, &fmt);
+
      if (vdec->fixed_fmt) {
              f->fmt.pix_mp.colorspace =

vdec->codec_info.color_primaries;

Quoted 14 lines

              f->fmt.pix_mp.xfer_func =
vdec->codec_info.transfer_chars; @@ -334,7 +355,7 @@ static int
vdec_try_fmt(struct file *file, void *fh, struct v4l2_format *f)
static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)  {
      struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;

drivers/media/platform/amphion/vdec.c +1 -1

-     const struct vpu_format *fmt;
+     struct vpu_format fmt;
      struct vpu_format *cur_fmt;
      struct vb2_queue *q;
      struct vdec_t *vdec = inst->priv; @@ -349,36 +370,30 @@ static

int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)

      if (vb2_is_busy(q))
              return -EBUSY;

drivers/media/platform/amphion/vdec.c static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)

Quoted 7 lines

drivers/media/platform/amphion/vdec.c +1 -2

-     fmt = vpu_try_fmt_common(inst, f);
-     if (!fmt)
+     if (vpu_try_fmt_common(inst, f, &fmt))
              return -EINVAL;
 
      cur_fmt = vpu_get_format(inst, f->type);
      if (V4L2_TYPE_IS_OUTPUT(f->type) && inst->state !=

VPU_CODEC_STATE_DEINIT) {

Quoted 4 lines

drivers/media/platform/amphion/vdec.c +1 -1

-             if (cur_fmt->pixfmt != fmt->pixfmt) {
+             if (cur_fmt->pixfmt != fmt.pixfmt) {
                      vdec->reset_codec = true;
                      vdec->fixed_fmt = false;

drivers/media/platform/amphion/vdec.c static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)

Quoted 10 lines

drivers/media/platform/amphion/vdec.c +0 -7

              }
      }
-     cur_fmt->pixfmt = fmt->pixfmt;
      if (V4L2_TYPE_IS_OUTPUT(f->type) || !vdec->fixed_fmt) {
-             cur_fmt->num_planes = fmt->num_planes;
-             cur_fmt->flags = fmt->flags;
-             cur_fmt->width = pixmp->width;
-             cur_fmt->height = pixmp->height;
-             for (i = 0; i < fmt->num_planes; i++) {
-                     cur_fmt->sizeimage[i] =

pixmp->plane_fmt[i].sizeimage;

Quoted 1 line

- cur_fmt->bytesperline[i] =

pixmp->plane_fmt[i].bytesperline;

drivers/media/platform/amphion/vdec.c static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)

Quoted 7 lines

drivers/media/platform/amphion/vdec.c +2 -4

-             }
-             if (pixmp->field != V4L2_FIELD_ANY)
-                     cur_fmt->field = pixmp->field;
+             memcpy(cur_fmt, &fmt, sizeof(*cur_fmt));
      } else {
-             pixmp->num_planes = cur_fmt->num_planes;
+             if (vpu_helper_match_format(inst, f->type, cur_fmt->pixfmt,

pixmp->pixelformat)) {

drivers/media/platform/amphion/vdec.c static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)

Quoted 9 lines

drivers/media/platform/amphion/vdec.c +5 -0

+                     cur_fmt->pixfmt = fmt.pixfmt;
+                     cur_fmt->mem_planes = fmt.mem_planes;
+             }
+             pixmp->pixelformat = cur_fmt->pixfmt;
+             pixmp->num_planes = cur_fmt->mem_planes;
              pixmp->width = cur_fmt->width;
              pixmp->height = cur_fmt->height;
              for (i = 0; i < pixmp->num_planes; i++) {
                      pixmp->plane_fmt[i].bytesperline =

cur_fmt->bytesperline[i];

Quoted 1 line

- pixmp->plane_fmt[i].sizeimage =

cur_fmt->sizeimage[i];

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst,

Quoted 18 lines

drivers/media/platform/amphion/vdec.c +4 -0

+                     pixmp->plane_fmt[i].sizeimage =
+ vpu_get_fmt_plane_size(cur_fmt, i);
              }
              pixmp->field = cur_fmt->field;
      }

@@ -678,9 +693,11 @@ static struct vpu_vb2_buffer
 *vdec_find_buffer(struct vpu_inst *inst, u32 luma)  static void
 vdec_buf_done(struct vpu_inst *inst, struct vpu_frame_info *frame)  {
      struct vdec_t *vdec = inst->priv;
+     struct vpu_format *cur_fmt;
      struct vpu_vb2_buffer *vpu_buf;
      struct vb2_v4l2_buffer *vbuf;
      u32 sequence;
+     int i;
 
      if (!frame)
              return;

@@ -699,6 +716,7 @@ static void vdec_buf_done(struct vpu_inst *inst,

struct vpu_frame_info *frame)

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst,

Quoted 8 lines

drivers/media/platform/amphion/vdec.c +1 -0

              return;
      }
 
+     cur_fmt = vpu_get_format(inst, inst->cap_format.type);
      vbuf = &vpu_buf->m2m_buf.vb;
      if (vbuf->vb2_buf.index != frame->id)
              dev_err(inst->dev, "[%d] buffer id(%d, %d) dismatch\n",

@@ -707,9 +725,9 @@ static void vdec_buf_done(struct vpu_inst *inst,

struct vpu_frame_info *frame)

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst, · line 726

Quoted 2 lines

drivers/media/platform/amphion/vdec.c +0 -0

      if (vpu_get_buffer_state(vbuf) != VPU_BUF_STATE_DECODED)
              dev_err(inst->dev, "[%d] buffer(%d) ready without

decoded\n", inst->id, frame->id);

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst, · line 710

Quoted 2 lines

drivers/media/platform/amphion/vdec.c +0 -1

      vpu_set_buffer_state(vbuf, VPU_BUF_STATE_READY);
-     vb2_set_plane_payload(&vbuf->vb2_buf, 0,

inst->cap_format.sizeimage[0]);

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst, · line 711

Quoted 1 line

drivers/media/platform/amphion/vdec.c +0 -1
-     vb2_set_plane_payload(&vbuf->vb2_buf, 1,

inst->cap_format.sizeimage[1]);

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst, struct vpu_frame_info *frame)

Quoted 3 lines

drivers/media/platform/amphion/vdec.c +2 -1

-     vbuf->field = inst->cap_format.field;
+     for (i = 0; i < vbuf->vb2_buf.num_planes; i++)
+             vb2_set_plane_payload(&vbuf->vb2_buf, i,

vpu_get_fmt_plane_size(cur_fmt, i));

Quoted 6 lines

drivers/media/platform/amphion/vdec.c +1 -0

+     vbuf->field = cur_fmt->field;
      vbuf->sequence = sequence;
      dev_dbg(inst->dev, "[%d][OUTPUT TS]%32lld\n", inst->id,
 vbuf->vb2_buf.timestamp);

drivers/media/platform/amphion/vdec.c static void vdec_stop_done(struct vpu_inst · line 772

Quoted 10 lines

drivers/media/platform/amphion/vdec.c +2 -2

@@ -747,15 +765,17 @@ static void vdec_stop_done(struct vpu_inst
 *inst)  static bool vdec_check_source_change(struct vpu_inst *inst)  {
      struct vdec_t *vdec = inst->priv;
-     const struct vpu_format *fmt;
-     int i;
+     const struct vpu_format *sibling;
 
      if (!inst->fh.m2m_ctx)
              return false;
 
+     sibling = vpu_helper_find_sibling(inst, inst->cap_format.type,

inst->cap_format.pixfmt);

drivers/media/platform/amphion/vdec.c static void vdec_stop_done(struct vpu_inst *inst)

Quoted 6 lines

drivers/media/platform/amphion/vdec.c +3 -1

+     if (sibling && vdec->codec_info.pixfmt == sibling->pixfmt)
+             vdec->codec_info.pixfmt = inst->cap_format.pixfmt;
+
      if (!vb2_is_streaming(v4l2_m2m_get_dst_vq(inst->fh.m2m_ctx)))
              return true;
-     fmt = vpu_helper_find_format(inst, inst->cap_format.type,

vdec->codec_info.pixfmt);

drivers/media/platform/amphion/vdec.c static void vdec_stop_done(struct vpu_inst *inst)

Quoted 4 lines

drivers/media/platform/amphion/vdec.c +0 -1

      if (inst->cap_format.pixfmt != vdec->codec_info.pixfmt)
              return true;
      if (inst->cap_format.width != vdec->codec_info.decoded_width) @@
-772,14 +792,6 @@ static bool vdec_check_source_change(struct vpu_inst

*inst)

drivers/media/platform/amphion/vdec.c static bool vdec_check_source_change(struct vpu_inst *inst)

Quoted 7 lines

drivers/media/platform/amphion/vdec.c +0 -4

              return true;
      if (inst->crop.height != vdec->codec_info.height)
              return true;
-     if (fmt && inst->cap_format.num_planes != fmt->num_planes)
-             return true;
-     for (i = 0; i < inst->cap_format.num_planes; i++) {
-             if (inst->cap_format.bytesperline[i] !=

vdec->codec_info.bytesperline[i])

drivers/media/platform/amphion/vdec.c

Quoted 2 lines

drivers/media/platform/amphion/vdec.c +0 -2

-                     return true;
-             if (inst->cap_format.sizeimage[i] !=

vdec->codec_info.sizeimage[i])

drivers/media/platform/amphion/vdec.c static bool vdec_check_source_change(struct · line 792

Quoted 13 lines

drivers/media/platform/amphion/vdec.c +1 -5

-                     return true;
-     }
 
      return false;
 }

@@ -787,27 +799,21 @@ static bool vdec_check_source_change(struct
 vpu_inst *inst)  static void vdec_init_fmt(struct vpu_inst *inst)  {
      struct vdec_t *vdec = inst->priv;
-     const struct vpu_format *fmt;
-     int i;
+     struct v4l2_format f;
 
-     fmt = vpu_helper_find_format(inst, inst->cap_format.type,

vdec->codec_info.pixfmt);

drivers/media/platform/amphion/vdec.c static bool vdec_check_source_change(struct vpu_inst *inst)

Quoted 11 lines

drivers/media/platform/amphion/vdec.c +0 ~~-11~~

-     inst->out_format.width = vdec->codec_info.width;
-     inst->out_format.height = vdec->codec_info.height;
-     inst->cap_format.width = vdec->codec_info.decoded_width;
-     inst->cap_format.height = vdec->codec_info.decoded_height;
-     inst->cap_format.pixfmt = vdec->codec_info.pixfmt;
-     if (fmt) {
-             inst->cap_format.num_planes = fmt->num_planes;
-             inst->cap_format.flags = fmt->flags;
-     }
-     for (i = 0; i < inst->cap_format.num_planes; i++) {
-             inst->cap_format.bytesperline[i] =

vdec->codec_info.bytesperline[i];

Quoted 20 lines

drivers/media/platform/amphion/vdec.c +11 -4

-             inst->cap_format.sizeimage[i] = vdec->codec_info.sizeimage[i];
-     }
+     memset(&f, 0, sizeof(f));
+     f.type = inst->cap_format.type;
+     f.fmt.pix_mp.pixelformat = vdec->codec_info.pixfmt;
+     f.fmt.pix_mp.width = vdec->codec_info.decoded_width;
+     f.fmt.pix_mp.height = vdec->codec_info.decoded_height;
      if (vdec->codec_info.progressive)
-             inst->cap_format.field = V4L2_FIELD_NONE;
+             f.fmt.pix_mp.field = V4L2_FIELD_NONE;
      else
-             inst->cap_format.field = V4L2_FIELD_SEQ_TB;
+             f.fmt.pix_mp.field = V4L2_FIELD_SEQ_TB;
+     vpu_try_fmt_common(inst, &f, &inst->cap_format);
+
+     inst->out_format.width = vdec->codec_info.width;
+     inst->out_format.height = vdec->codec_info.height;
 }
 
 static void vdec_init_crop(struct vpu_inst *inst) @@ -966,7 +972,10

Quoted 1 line

@@ static int vdec_response_frame(struct vpu_inst *inst, struct

vb2_v4l2_buffer *vb

drivers/media/platform/amphion/vdec.c static int vdec_response_frame(struct vpu_inst *inst, struct vb2_v4l2_buffer *vb

Quoted 6 lines

      info.tag = vdec->seq_tag;
      info.luma_addr = vpu_get_vb_phy_addr(&vbuf->vb2_buf, 0);
      info.luma_size = inst->cap_format.sizeimage[0];

drivers/media/platform/amphion/vdec.c +2 -1

-     info.chroma_addr = vpu_get_vb_phy_addr(&vbuf->vb2_buf, 1);
+     if (vbuf->vb2_buf.num_planes > 1)
+             info.chroma_addr = vpu_get_vb_phy_addr(&vbuf->vb2_buf,

1);

drivers/media/platform/amphion/venc.c static int venc_g_fmt(struct file *file, void *fh,

Quoted 40 lines

drivers/media/platform/amphion/vdec.c +2 -0

+     else
+             info.chroma_addr = info.luma_addr + info.luma_size;
      info.chromau_size = inst->cap_format.sizeimage[1];
      info.bytesperline = inst->cap_format.bytesperline[0];
      ret = vpu_session_alloc_fs(inst, &info); @@ -975,7 +984,7 @@

static int vdec_response_frame(struct vpu_inst *inst, struct vb2_v4l2_buffer *vb

      vpu_buf->tag = info.tag;
      vpu_buf->luma = info.luma_addr;

drivers/media/platform/amphion/vdec.c +1 -1

-     vpu_buf->chroma_u = info.chromau_size;
+     vpu_buf->chroma_u = info.chroma_addr;
      vpu_buf->chroma_v = 0;
      vpu_set_buffer_state(vbuf, VPU_BUF_STATE_INUSE);
      vdec->slots[info.id] = vpu_buf;

drivers/media/platform/amphion/venc.c +5 -2

index 37212f087fdd..060a1ee78b17 100644
@@ -69,13 +69,16 @@ struct venc_frame_t {  static const struct
 vpu_format venc_formats[] = {
      {
              .pixfmt = V4L2_PIX_FMT_NV12M,
-             .num_planes = 2,
+             .mem_planes = 2,
+             .comp_planes = 2,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
      },
      {
              .pixfmt = V4L2_PIX_FMT_H264,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+             .flags = V4L2_FMT_FLAG_COMPRESSED
      },
      {0, 0, 0, 0},
 };

@@ -173,14 +176,14 @@ static int venc_g_fmt(struct file *file, void *fh,

struct v4l2_format *f)

drivers/media/platform/amphion/venc.c static int venc_s_fmt(struct file *file, void *fh,

Quoted 43 lines

drivers/media/platform/amphion/venc.c +3 -3

      cur_fmt = vpu_get_format(inst, f->type);
 
      pixmp->pixelformat = cur_fmt->pixfmt;
-     pixmp->num_planes = cur_fmt->num_planes;
+     pixmp->num_planes = cur_fmt->mem_planes;
      pixmp->width = cur_fmt->width;
      pixmp->height = cur_fmt->height;
      pixmp->field = cur_fmt->field;
      pixmp->flags = cur_fmt->flags;
      for (i = 0; i < pixmp->num_planes; i++) {
              pixmp->plane_fmt[i].bytesperline = cur_fmt->bytesperline[i];
-             pixmp->plane_fmt[i].sizeimage = cur_fmt->sizeimage[i];
+             pixmp->plane_fmt[i].sizeimage =
+ vpu_get_fmt_plane_size(cur_fmt, i);
      }
 
      f->fmt.pix_mp.colorspace = venc->params.color.primaries; @@
-194,8 +197,9 @@ static int venc_g_fmt(struct file *file, void *fh,

struct v4l2_format *f) static int venc_try_fmt(struct file *file,

void *fh, struct v4l2_format *f)  {
      struct vpu_inst *inst = to_inst(file);

drivers/media/platform/amphion/venc.c +3 -3

+     struct vpu_format fmt;
 
-     vpu_try_fmt_common(inst, f);
+     vpu_try_fmt_common(inst, f, &fmt);
 
      return 0;
 }

@@ -203,12 +207,11 @@ static int venc_try_fmt(struct file *file, void
 *fh, struct v4l2_format *f)  static int venc_s_fmt(struct file *file,
 void *fh, struct v4l2_format *f)  {
      struct vpu_inst *inst = to_inst(file);
-     const struct vpu_format *fmt;
+     struct vpu_format fmt;
      struct vpu_format *cur_fmt;
      struct vb2_queue *q;
      struct venc_t *venc = inst->priv;
      struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
-     int i;
 
      q = v4l2_m2m_get_vq(inst->fh.m2m_ctx, f->type);
      if (!q)

@@ -216,24 +219,12 @@ static int venc_s_fmt(struct file *file, void *fh,

struct v4l2_format *f)

drivers/media/platform/amphion/vpu_dbg.c static int vpu_dbg_instance(struct seq_file *s, void

Quoted 55 lines

drivers/media/platform/amphion/venc.c +2 ~~-14~~

      if (vb2_is_busy(q))
              return -EBUSY;
 
-     fmt = vpu_try_fmt_common(inst, f);
-     if (!fmt)
+     if (vpu_try_fmt_common(inst, f, &fmt))
              return -EINVAL;
 
      cur_fmt = vpu_get_format(inst, f->type);
 
-     cur_fmt->pixfmt = fmt->pixfmt;
-     cur_fmt->num_planes = fmt->num_planes;
-     cur_fmt->flags = fmt->flags;
-     cur_fmt->width = pix_mp->width;
-     cur_fmt->height = pix_mp->height;
-     for (i = 0; i < fmt->num_planes; i++) {
-             cur_fmt->sizeimage[i] = pix_mp->plane_fmt[i].sizeimage;
-             cur_fmt->bytesperline[i] = pix_mp->plane_fmt[i].bytesperline;
-     }
-
-     if (pix_mp->field != V4L2_FIELD_ANY)
-             cur_fmt->field = pix_mp->field;
+     memcpy(cur_fmt, &fmt, sizeof(*cur_fmt));
 
      if (V4L2_TYPE_IS_OUTPUT(f->type)) {
              venc->params.input_format = cur_fmt->pixfmt; diff --git

a/drivers/media/platform/amphion/vpu.h
b/drivers/media/platform/amphion/vpu.h
index f914de6ed81e..998591fa5b85 100644

drivers/media/platform/amphion/vpu.h +3 -1

@@ -84,7 +84,8 @@ struct vpu_dev {
 
 struct vpu_format {
      u32 pixfmt;
-     unsigned int num_planes;
+     u32 mem_planes;
+     u32 comp_planes;
      u32 type;
      u32 flags;
      u32 width;

@@ -92,6 +93,7 @@ struct vpu_format {
      u32 sizeimage[VIDEO_MAX_PLANES];
      u32 bytesperline[VIDEO_MAX_PLANES];
      u32 field;
+     u32 sibling;
 };
 
 struct vpu_core_resources {

drivers/media/platform/amphion/vpu_dbg.c +0 -0

index f72c8a506b22..c41c9896912c 100644
@@ -89,9 +89,9 @@ static int vpu_dbg_instance(struct seq_file *s, void

*data)

drivers/media/platform/amphion/vpu_dbg.c static int vpu_dbg_instance(struct seq_file *s, void

Quoted 13 lines

drivers/media/platform/amphion/vpu_dbg.c +3 -2

                      vq->last_buffer_dequeued);
      if (seq_write(s, str, num))
              return 0;
-     for (i = 0; i < inst->out_format.num_planes; i++) {
+     for (i = 0; i < inst->out_format.mem_planes; i++) {
              num = scnprintf(str, sizeof(str), " %d(%d)",
-                             inst->out_format.sizeimage[i],
+
+ vpu_get_fmt_plane_size(&inst->out_format, i),
                              inst->out_format.bytesperline[i]);
              if (seq_write(s, str, num))
                      return 0;

@@ -113,9 +113,9 @@ static int vpu_dbg_instance(struct seq_file *s, void

*data)

drivers/media/platform/amphion/vpu_helpers.c const struct vpu_format

Quoted 18 lines

drivers/media/platform/amphion/vpu_dbg.c +3 -2

                      vq->last_buffer_dequeued);
      if (seq_write(s, str, num))
              return 0;
-     for (i = 0; i < inst->cap_format.num_planes; i++) {
+     for (i = 0; i < inst->cap_format.mem_planes; i++) {
              num = scnprintf(str, sizeof(str), " %d(%d)",
-                             inst->cap_format.sizeimage[i],
+
+ vpu_get_fmt_plane_size(&inst->cap_format, i),
                              inst->cap_format.bytesperline[i]);
              if (seq_write(s, str, num))
                      return 0;

drivers/media/platform/amphion/vpu_helpers.c +0 -0

index e9aeb3453dfc..db8f471b9187 100644
@@ -59,6 +59,39 @@ const struct vpu_format

*vpu_helper_find_format(struct vpu_inst *inst, u32 type,

drivers/media/platform/amphion/vpu_helpers.c static u32 get_nv12_plane_size(u32 width, u32

Quoted 40 lines

drivers/media/platform/amphion/vpu_helpers.c +33 -0

      return NULL;
 }
 
+const struct vpu_format *vpu_helper_find_sibling(struct vpu_inst
+*inst, u32 type, u32 pixelfmt) {
+     const struct vpu_format *fmt;
+     const struct vpu_format *sibling;
+
+     fmt = vpu_helper_find_format(inst, type, pixelfmt);
+     if (!fmt)
+             return NULL;
+     if (!fmt->sibling)
+             return NULL;
+     sibling = vpu_helper_find_format(inst, type, fmt->sibling);
+     if (!sibling)
+             return NULL;
+     if (sibling->sibling != fmt->pixfmt)
+             return NULL;
+     if (sibling->comp_planes != fmt->comp_planes)
+             return NULL;
+     return sibling;
+}
+
+bool vpu_helper_match_format(struct vpu_inst *inst, u32 type, u32
+fmta, u32 fmtb) {
+     const struct vpu_format *sibling;
+
+     if (fmta == fmtb)
+             return true;
+
+     sibling = vpu_helper_find_sibling(inst, type, fmta);
+     if (sibling && sibling->pixfmt == fmtb)
+             return true;
+     return false;
+}
+
 const struct vpu_format *vpu_helper_enum_format(struct vpu_inst
 *inst, u32 type, int index)  {
      const struct vpu_format *pfmt;

@@ -123,9 +156,10 @@ static u32 get_nv12_plane_size(u32 width, u32

height, int plane_no,

drivers/media/platform/amphion/vpu_helpers.c u32 vpu_helper_get_plane_size(u32 fmt, u32 w, u32

Quoted 43 lines

drivers/media/platform/amphion/vpu_helpers.c +2 -1

      u32 bytesperline;
      u32 size = 0;
 
-     bytesperline = ALIGN(width, stride);
+     bytesperline = width;
      if (pbl)
              bytesperline = max(bytesperline, *pbl);
+     bytesperline = ALIGN(bytesperline, stride);
      height = ALIGN(height, 2);
      if (plane_no == 0)
              size = bytesperline * height; @@ -148,13 +182,13 @@

static u32 get_tiled_8l128_plane_size(u32 fmt, u32 width, u32 height, int plane_

      if (interlaced)
              hs++;

drivers/media/platform/amphion/vpu_helpers.c +4 -3

-     if (fmt == V4L2_PIX_FMT_NV12M_10BE_8L128)
+     if (fmt == V4L2_PIX_FMT_NV12M_10BE_8L128 || fmt ==
+ V4L2_PIX_FMT_NV12_10BE_8L128)
              bitdepth = 10;
      bytesperline = DIV_ROUND_UP(width * bitdepth, BITS_PER_BYTE);
-     bytesperline = ALIGN(bytesperline, 1 << ws);
-     bytesperline = ALIGN(bytesperline, stride);
      if (pbl)
              bytesperline = max(bytesperline, *pbl);
+     bytesperline = ALIGN(bytesperline, 1 << ws);
+     bytesperline = ALIGN(bytesperline, stride);
      height = ALIGN(height, 1 << hs);
      if (plane_no == 0)
              size = bytesperline * height; @@ -172,9 +206,10 @@

static u32 get_default_plane_size(u32 width, u32 height, int plane_no,

      u32 bytesperline;
      u32 size = 0;

drivers/media/platform/amphion/vpu_helpers.c +2 -1

-     bytesperline = ALIGN(width, stride);
+     bytesperline = width;
      if (pbl)
              bytesperline = max(bytesperline, *pbl);
+     bytesperline = ALIGN(bytesperline, stride);
      if (plane_no == 0)
              size = bytesperline * height;
      if (pbl)

@@ -187,9 +222,12 @@ u32 vpu_helper_get_plane_size(u32 fmt, u32 w, u32

h, int plane_no,

drivers/media/platform/amphion/vpu_helpers.c u32 vpu_helper_get_plane_size(u32 fmt, u32 w, u32 h, int plane_no,

Quoted 11 lines

drivers/media/platform/amphion/vpu_helpers.c +3 -0

                            u32 stride, u32 interlaced, u32 *pbl)  {
      switch (fmt) {
+     case V4L2_PIX_FMT_NV12:
      case V4L2_PIX_FMT_NV12M:
              return get_nv12_plane_size(w, h, plane_no, stride,
 interlaced, pbl);
+     case V4L2_PIX_FMT_NV12_8L128:
      case V4L2_PIX_FMT_NV12M_8L128:
+     case V4L2_PIX_FMT_NV12_10BE_8L128:
      case V4L2_PIX_FMT_NV12M_10BE_8L128:
              return get_tiled_8l128_plane_size(fmt, w, h, plane_no, stride,

interlaced, pbl);

drivers/media/platform/amphion/vpu_malone.c bool vpu_malone_check_fmt(enum vpu_core_type

Quoted 24 lines

drivers/media/platform/amphion/vpu_helpers.c +0 -0
      default:
drivers/media/platform/amphion/vpu_helpers.h +0 -0
index bc28350958be..0eaddb07190d 100644
@@ -14,6 +14,8 @@ struct vpu_pair {
 int vpu_helper_find_in_array_u8(const u8 *array, u32 size, u32 x);
bool vpu_helper_check_type(struct vpu_inst *inst, u32 type); const struct vpu_format *vpu_helper_find_format(struct vpu_inst *inst, u32 type, u32 pixelfmt);
drivers/media/platform/amphion/vpu_helpers.h +3 -0
+const struct vpu_format *vpu_helper_find_sibling(struct vpu_inst
+*inst, u32 type, u32 pixelfmt); bool vpu_helper_match_format(struct
+vpu_inst *inst, u32 type, u32 fmta, u32 fmtb);
 const struct vpu_format *vpu_helper_enum_format(struct vpu_inst
*inst, u32 type, int index);
u32 vpu_helper_valid_frame_width(struct vpu_inst *inst, u32 width); u32 vpu_helper_valid_frame_height(struct vpu_inst *inst, u32 height);
drivers/media/platform/amphion/vpu_malone.c +0 -0
index 51e0702f9ae1..69d9a2269fce 100644
@@ -583,7 +583,8 @@ bool vpu_malone_check_fmt(enum vpu_core_type

type, u32 pixelfmt)

drivers/media/platform/amphion/vpu_malone.c bool vpu_malone_check_fmt(enum vpu_core_type type, u32 pixelfmt)

Quoted 4 lines

drivers/media/platform/amphion/vpu_malone.c +0 -1

      if (!vpu_imx8q_check_fmt(type, pixelfmt))
              return false;
 
-     if (pixelfmt == V4L2_PIX_FMT_NV12M_8L128 || pixelfmt ==

V4L2_PIX_FMT_NV12M_10BE_8L128)

drivers/media/platform/amphion/vpu_malone.c bool vpu_malone_check_fmt(enum vpu_core_type · line 586

Quoted 1 line

drivers/media/platform/amphion/vpu_malone.c +1 -0
+     if (pixelfmt == V4L2_PIX_FMT_NV12_8L128 || pixelfmt ==

V4L2_PIX_FMT_NV12_10BE_8L128 ||

Quoted 36 lines

drivers/media/platform/amphion/vpu_malone.c +2 -0

+         pixelfmt == V4L2_PIX_FMT_NV12M_8L128 || pixelfmt ==
+ V4L2_PIX_FMT_NV12M_10BE_8L128)
              return true;
      if (vpu_malone_format_remap(pixelfmt) == MALONE_FMT_NULL)
              return false;

drivers/media/platform/amphion/vpu_v4l2.c +17 -2

index 8a3eed957ae6..87cce552e906 100644
@@ -140,51 +140,135 @@ bool vpu_is_source_empty(struct vpu_inst *inst)
      return true;
 }
 
-const struct vpu_format *vpu_try_fmt_common(struct vpu_inst *inst,
 struct v4l2_format *f)
+static int vpu_init_format(struct vpu_inst *inst, struct vpu_format
+*fmt) {
+     const struct vpu_format *info;
+
+     info = vpu_helper_find_format(inst, fmt->type, fmt->pixfmt);
+     if (!info) {
+             info = vpu_helper_enum_format(inst, fmt->type, 0);
+             if (!info)
+                     return -EINVAL;
+     }
+     memcpy(fmt, info, sizeof(*fmt));
+
+     return 0;
+}
+
+static int vpu_calc_fmt_bytesperline(struct v4l2_format *f, struct
+vpu_format *fmt)
 {
      struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
-     u32 type = f->type;

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 172

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +10 -0

+     int i;
+
+     if (fmt->flags & V4L2_FMT_FLAG_COMPRESSED) {
+             for (i = 0; i < fmt->comp_planes; i++)
+                     fmt->bytesperline[i] = 0;
+             return 0;
+     }
+     if (pixmp->num_planes == fmt->comp_planes) {
+             for (i = 0; i < fmt->comp_planes; i++)
+                     fmt->bytesperline[i] =

pixmp->plane_fmt[i].bytesperline;

Quoted 32 lines

drivers/media/platform/amphion/vpu_v4l2.c +19 ~~-10~~

+             return 0;
+     }
+     if (pixmp->num_planes > 1)
+             return -EINVAL;
+
+     /*amphion vpu only support nv12 and nv12 tiled,
+      * so the bytesperline of luma and chroma should be same
+      */
+     for (i = 0; i < fmt->comp_planes; i++)
+             fmt->bytesperline[i] = pixmp->plane_fmt[0].bytesperline;
+
+     return 0;
+}
+
+static int vpu_calc_fmt_sizeimage(struct vpu_inst *inst, struct
+vpu_format *fmt) {
      u32 stride = 1;
-     u32 bytesperline;
-     u32 sizeimage;
-     const struct vpu_format *fmt;
-     const struct vpu_core_resources *res;
      int i;
 
-     fmt = vpu_helper_find_format(inst, type, pixmp->pixelformat);
-     if (!fmt) {
-             fmt = vpu_helper_enum_format(inst, type, 0);
-             if (!fmt)
-                     return NULL;
-             pixmp->pixelformat = fmt->pixfmt;
+     if (!(fmt->flags & V4L2_FMT_FLAG_COMPRESSED)) {
+             const struct vpu_core_resources *res =
+ vpu_get_resource(inst);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -5

+
+             if (res)
+                     stride = res->stride;
      }
 
-     res = vpu_get_resource(inst);
-     if (res)
-             stride = res->stride;
-     if (pixmp->width)
-             pixmp->width = vpu_helper_valid_frame_width(inst,

pixmp->width);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 169

Quoted 2 lines

drivers/media/platform/amphion/vpu_v4l2.c +0 -2

-     if (pixmp->height)
-             pixmp->height = vpu_helper_valid_frame_height(inst,

pixmp->height);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 211

Quoted 3 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -0

+     for (i = 0; i < fmt->comp_planes; i++) {
+             fmt->sizeimage[i] = vpu_helper_get_plane_size(fmt->pixfmt,
+

fmt->width,

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 203

Quoted 1 line

drivers/media/platform/amphion/vpu_v4l2.c +1 -0
+

fmt->height,

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 211

Quoted 3 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -0

+                                                           i,
+                                                           stride,
+

fmt->field != V4L2_FIELD_NONE ? 1 : 0,

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 207

Quoted 1 line

drivers/media/platform/amphion/vpu_v4l2.c +1 -0
+

&fmt->bytesperline[i]);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 3 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -0

+             fmt->sizeimage[i] = max_t(u32, fmt->sizeimage[i], PAGE_SIZE);
+             if (fmt->flags & V4L2_FMT_FLAG_COMPRESSED) {
+                     fmt->sizeimage[i] = clamp_val(fmt->sizeimage[i],

SZ_128K, SZ_8M);

Quoted 31 lines

drivers/media/platform/amphion/vpu_v4l2.c +31 -0

+                     fmt->bytesperline[i] = 0;
+             }
+     }
+
+     return 0;
+}
+
+u32 vpu_get_fmt_plane_size(struct vpu_format *fmt, u32 plane_no) {
+     u32 size;
+     int i;
+
+     if (plane_no >= fmt->mem_planes)
+             return 0;
+
+     if (fmt->comp_planes == fmt->mem_planes)
+             return fmt->sizeimage[plane_no];
+     if (plane_no < fmt->mem_planes - 1)
+             return fmt->sizeimage[plane_no];
+     size = fmt->sizeimage[plane_no];
+     for (i = fmt->mem_planes; i < fmt->comp_planes; i++)
+             size += fmt->sizeimage[i];
+
+     return size;
+}
+
+int vpu_try_fmt_common(struct vpu_inst *inst, struct v4l2_format *f,
+struct vpu_format *fmt) {
+     struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
+     int i;
+     int ret;
+

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +10 -0

+     fmt->pixfmt = pixmp->pixelformat;
+     fmt->type = f->type;
+     ret = vpu_init_format(inst, fmt);
+     if (ret < 0)
+             return ret;
+
+     fmt->width = pixmp->width;
+     fmt->height = pixmp->height;
+     if (fmt->width)
+             fmt->width = vpu_helper_valid_frame_width(inst,

fmt->width);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 253

Quoted 2 lines

drivers/media/platform/amphion/vpu_v4l2.c +2 -0

+     if (fmt->height)
+             fmt->height = vpu_helper_valid_frame_height(inst,

fmt->height);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 254

Quoted 1 line

drivers/media/platform/amphion/vpu_v4l2.c +1 -0
+     fmt->field = pixmp->field == V4L2_FIELD_ANY ? V4L2_FIELD_NONE :

pixmp->field;

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 3 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -0

+     vpu_calc_fmt_bytesperline(f, fmt);
+     vpu_calc_fmt_sizeimage(inst, fmt);
+     if ((fmt->flags & V4L2_FMT_FLAG_COMPRESSED) &&

pixmp->plane_fmt[0].sizeimage)

Quoted 5 lines

drivers/media/platform/amphion/vpu_v4l2.c +5 -0

+             fmt->sizeimage[0] =
+ clamp_val(pixmp->plane_fmt[0].sizeimage, SZ_128K, SZ_8M);
+
+     pixmp->pixelformat = fmt->pixfmt;
+     pixmp->width = fmt->width;

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +4 -4

+     pixmp->height = fmt->height;
      pixmp->flags = fmt->flags;
-     pixmp->num_planes = fmt->num_planes;
-     if (pixmp->field == V4L2_FIELD_ANY)
-             pixmp->field = V4L2_FIELD_NONE;
+     pixmp->num_planes = fmt->mem_planes;
+     pixmp->field = fmt->field;
+     memset(pixmp->reserved, 0, sizeof(pixmp->reserved));
      for (i = 0; i < pixmp->num_planes; i++) {
-             bytesperline = max_t(s32, pixmp->plane_fmt[i].bytesperline,

0);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 6 lines

drivers/media/platform/amphion/vpu_v4l2.c +0 -6

-             sizeimage = vpu_helper_get_plane_size(pixmp->pixelformat,
-                                                   pixmp->width,
-                                                   pixmp->height,
-                                                   i,
-                                                   stride,
-                                                   pixmp->field >

V4L2_FIELD_NONE ? 1 : 0,

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 183

Quoted 2 lines

drivers/media/platform/amphion/vpu_v4l2.c +0 -2

-                                                   &bytesperline);
-             sizeimage = max_t(s32, pixmp->plane_fmt[i].sizeimage,

sizeimage);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 4 lines

drivers/media/platform/amphion/vpu_v4l2.c +2 -2

-             pixmp->plane_fmt[i].bytesperline = bytesperline;
-             pixmp->plane_fmt[i].sizeimage = sizeimage;
+             pixmp->plane_fmt[i].bytesperline = fmt->bytesperline[i];
+             pixmp->plane_fmt[i].sizeimage = vpu_get_fmt_plane_size(fmt,

i);

drivers/media/platform/amphion/vpu_v4l2.c static int vpu_vb2_queue_setup(struct vb2_queue

Quoted 25 lines

drivers/media/platform/amphion/vpu_v4l2.c +7 -5

+             memset(pixmp->plane_fmt[i].reserved, 0,
+ sizeof(pixmp->plane_fmt[i].reserved));
      }
 
-     return fmt;
+     return 0;
 }
 
 static bool vpu_check_ready(struct vpu_inst *inst, u32 type) @@
-389,10 +473,10 @@ static int vpu_vb2_queue_setup(struct vb2_queue *vq,
      cur_fmt = vpu_get_format(inst, vq->type);
 
      if (*plane_count) {
-             if (*plane_count != cur_fmt->num_planes)
+             if (*plane_count != cur_fmt->mem_planes)
                      return -EINVAL;
-             for (i = 0; i < cur_fmt->num_planes; i++) {
-                     if (psize[i] < cur_fmt->sizeimage[i])
+             for (i = 0; i < cur_fmt->mem_planes; i++) {
+                     if (psize[i] < vpu_get_fmt_plane_size(cur_fmt,
+ i))
                              return -EINVAL;
              }
              return 0;

@@ -402,9 +486,9 @@ static int vpu_vb2_queue_setup(struct vb2_queue

*vq,

drivers/media/platform/amphion/vpu_v4l2.c static int vpu_vb2_queue_setup(struct vb2_queue · line 486

Quoted 1 line

drivers/media/platform/amphion/vpu_v4l2.c +0 -0
              *buf_count = max_t(unsigned int, *buf_count,

inst->min_buffer_out);

drivers/media/platform/amphion/vpu_v4l2.c static int vpu_vb2_queue_setup(struct vb2_queue · line 488

Quoted 2 lines

drivers/media/platform/amphion/vpu_v4l2.c +0 -0

      else
              *buf_count = max_t(unsigned int, *buf_count,

inst->min_buffer_cap);

drivers/media/platform/amphion/vpu_v4l2.c static int vpu_vb2_buf_prepare(struct vb2_buffer

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -3

-     *plane_count = cur_fmt->num_planes;
-     for (i = 0; i < cur_fmt->num_planes; i++)
-             psize[i] = cur_fmt->sizeimage[i];
+     *plane_count = cur_fmt->mem_planes;
+     for (i = 0; i < cur_fmt->mem_planes; i++)
+             psize[i] = vpu_get_fmt_plane_size(cur_fmt, i);
 
      return 0;
 }

@@ -434,8 +518,8 @@ static int vpu_vb2_buf_prepare(struct vb2_buffer

*vb)

drivers/media/platform/amphion/vpu_v4l2.c static int vpu_vb2_buf_prepare(struct vb2_buffer *vb)

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -2

      u32 i;
 
      cur_fmt = vpu_get_format(inst, vb->type);
-     for (i = 0; i < cur_fmt->num_planes; i++) {
-             if (vpu_get_vb_length(vb, i) < cur_fmt->sizeimage[i]) {
+     for (i = 0; i < cur_fmt->mem_planes; i++) {
+             if (vpu_get_vb_length(vb, i) <
+ vpu_get_fmt_plane_size(cur_fmt, i)) {
                      dev_dbg(inst->dev, "[%d] %s buf[%d] is invalid\n",
                              inst->id, vpu_type_name(vb->type),

vb->index);

Quoted 1 line

vpu_set_buffer_state(vbuf,

VPU_BUF_STATE_ERROR);

drivers/media/platform/amphion/vpu_windsor.c static int vpu_windsor_fill_yuv_frame(struct

Quoted 23 lines

drivers/media/platform/amphion/vpu_v4l2.h +0 -0
index 795ca33a6a50..ef5de6b66e47 100644
@@ -16,7 +16,8 @@ unsigned int vpu_get_buffer_state(struct
vb2_v4l2_buffer *vbuf); int vpu_v4l2_open(struct file *file, struct vpu_inst *inst); int vpu_v4l2_close(struct file *file);
-const struct vpu_format *vpu_try_fmt_common(struct vpu_inst *inst, struct v4l2_format *f);
drivers/media/platform/amphion/vpu_v4l2.h +3 -0
+u32 vpu_get_fmt_plane_size(struct vpu_format *fmt, u32 plane_no); int
+vpu_try_fmt_common(struct vpu_inst *inst, struct v4l2_format *f,
+struct vpu_format *fmt);
 int vpu_process_output_buffer(struct vpu_inst *inst);  int
vpu_process_capture_buffer(struct vpu_inst *inst); struct vb2_v4l2_buffer *vpu_next_src_buf(struct vpu_inst *inst); diff --git a/drivers/media/platform/amphion/vpu_windsor.c b/drivers/media/platform/amphion/vpu_windsor.c
index 1526af2ef9da..a454f142ae17 100644
drivers/media/platform/amphion/vpu_windsor.c +0 -0
@@ -775,6 +775,8 @@ static int vpu_windsor_fill_yuv_frame(struct

vpu_shared_addr *shared,

drivers/media/platform/amphion/vpu_windsor.c static int vpu_windsor_fill_yuv_frame(struct

Quoted 8 lines

drivers/media/platform/amphion/vpu_windsor.c +2 -0

                                    u32 instance,
                                    struct vb2_buffer *vb)  {
+     struct vpu_inst *inst = vb2_get_drv_priv(vb->vb2_queue);
+     struct vpu_format *out_fmt;
      struct vpu_enc_yuv_desc *desc;
      struct vb2_v4l2_buffer *vbuf;

@@ -782,6 +784,7 @@ static int vpu_windsor_fill_yuv_frame(struct

vpu_shared_addr *shared,

Quoted 21 lines

drivers/media/platform/amphion/vpu_windsor.c +1 -0

              return -EINVAL;
 
      desc = get_yuv_desc(shared, instance);
+     out_fmt = vpu_get_format(inst, vb->type);
 
      vbuf = to_vb2_v4l2_buffer(vb);
      desc->frame_id = vbuf->sequence; @@ -790,7 +793,10 @@ static int

vpu_windsor_fill_yuv_frame(struct vpu_shared_addr *shared,

      else
              desc->key_frame = 0;
      desc->luma_base = vpu_get_vb_phy_addr(vb, 0);

drivers/media/platform/amphion/vpu_windsor.c +5 -2

-     desc->chroma_base = vpu_get_vb_phy_addr(vb, 1);
+     if (vb->num_planes > 1)
+             desc->chroma_base = vpu_get_vb_phy_addr(vb, 1);
+     else
+             desc->chroma_base = desc->luma_base +
+ out_fmt->sizeimage[0];
 
      return 0;
 }
--

Quoted 2 lines

2.37.1

--
Tommaso Merciai
Embedded Linux Engineer
[email protected]
__________________________________

Amarula Solutions SRL
Via Le Canevare 30, 31100 Treviso, Veneto, IT T. +39 042 243 5310 [email protected]
https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.am
arulasolutions.com%2F&data=05%7C01%7Cming.qian%40nxp.com%7C39 82ab247a5043fd1f6608da7c325e21%7C686ea1d3bc2b4c6fa92cd99c5c30163 5%7C0%7C0%7C637958853015002900%7CUnknown%7CTWFpbGZsb3d8eyJW IjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C30 00%7C%7C%7C&sdata=Ig20cUBTsVAOcR7blwbR0mWrGhZp7iQYUO6ZaH8 L00I%3D&reserved=0

Tommaso Merciai

2022-08-12 09:42 UTC #9 lore raw

Re: [EXT] Re: [PATCH v4 2/4] media: amphion: tell and handle contiguous and non contiguous format

Quoted 11 lines · Ming Qian

Quoted 11 lines
From: Tommaso Merciai <[email protected]>
Sent: 2022年8月12日 15:15
To: Ming Qian <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; dl-linux-imx <[email protected]>; X.H. Bao <[email protected]>; [email protected]; [email protected]; [email protected]
Subject: [EXT] Re: [PATCH v4 2/4] media: amphion: tell and handle contiguous and non contiguous format

Quoted 8 lines

Quoted 5 lines
Caution: EXT Email
Hi Ming,
Just an info. I'm planning to help to review your series. On which Linux tree are you working?
Hi Tommaso,
Thanks for your help, and I'm working on https://git.linuxtv.org/media_tree.git

Thanks for your time.

Tommaso

Quoted 1,049 lines

Ming

Quoted 1,046 lines

Thanks in advance,
Tommaso

drivers/media/platform/amphion/vdec.c static int vdec_enum_fmt(struct file *file, void *fh,

Quoted 145 lines · Ming Qian

Driver should tell the number of memory planes and component planes. the amphion vpu support non contiguous planes, but for compatibility with other device that only support contiguous planes. driver can add support for contiguous planes in the same time. Then the mem_planes can be different from the comp_planes. driver need to handle buffer according mem_planes and comp_planes.

So driver can support NV12 and NV12M.

Signed-off-by Ming Qian <[email protected]>

---
 drivers/media/platform/amphion/vdec.c        | 187 ++++++++++---------
 drivers/media/platform/amphion/venc.c        |  33 ++--
 drivers/media/platform/amphion/vpu.h         |   4 +-
 drivers/media/platform/amphion/vpu_dbg.c     |   8 +-
 drivers/media/platform/amphion/vpu_helpers.c |  48 ++++-
 drivers/media/platform/amphion/vpu_helpers.h |   2 +
 drivers/media/platform/amphion/vpu_malone.c  |   3 +-
 drivers/media/platform/amphion/vpu_v4l2.c    | 168 ++++++++++++-----
 drivers/media/platform/amphion/vpu_v4l2.h    |   3 +-
 drivers/media/platform/amphion/vpu_windsor.c |   8 +-
 10 files changed, 299 insertions(+), 165 deletions(-)

drivers/media/platform/amphion/vdec.c +43 ~~-21~~

index feb75dc204de..48ab664fa7ef 100644
@@ -69,72 +69,85 @@ struct vdec_t {
 static const struct vpu_format vdec_formats[] = {
      {
              .pixfmt = V4L2_PIX_FMT_NV12M_8L128,
-             .num_planes = 2,
+             .mem_planes = 2,
+             .comp_planes = 2,
              .type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
      },
      {
              .pixfmt = V4L2_PIX_FMT_NV12M_10BE_8L128,
-             .num_planes = 2,
+             .mem_planes = 2,
+             .comp_planes = 2,
              .type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
      },
      {
              .pixfmt = V4L2_PIX_FMT_H264,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_H264_MVC,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_HEVC,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_VC1_ANNEX_G,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_VC1_ANNEX_L,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+             .flags = V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_MPEG2,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_MPEG4,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_XVID,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_VP8,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_H263,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {0, 0, 0, 0},
 };

@@ -256,23 +269,22 @@ static int vdec_enum_fmt(struct file *file, void *fh,

struct v4l2_fmtdesc *f)

drivers/media/platform/amphion/vdec.c static int vdec_enum_fmt(struct file *file, void *fh, struct v4l2_fmtdesc *f)

Quoted 13 lines

drivers/media/platform/amphion/vdec.c +4 -6

      int ret = -EINVAL;
 
      vpu_inst_lock(inst);
-     if (!V4L2_TYPE_IS_OUTPUT(f->type) && vdec->fixed_fmt) {
-             if (f->index == 0) {
-                     f->pixelformat = inst->cap_format.pixfmt;
-                     f->flags = inst->cap_format.flags;
-                     ret = 0;
-             }
+     if (V4L2_TYPE_IS_CAPTURE(f->type) && vdec->fixed_fmt) {
+             fmt = vpu_get_format(inst, f->type);
+             if (f->index == 1)
+                     fmt = vpu_helper_find_sibling(inst, f->type,

fmt->pixfmt);

drivers/media/platform/amphion/vdec.c static int vdec_g_fmt(struct file *file, void *fh,

Quoted 23 lines

drivers/media/platform/amphion/vdec.c +8 -7

+             if (f->index > 1)
+                     fmt = NULL;
      } else {
              fmt = vpu_helper_enum_format(inst, f->type, f->index);
-             memset(f->reserved, 0, sizeof(f->reserved));
-             if (!fmt)
-                     goto exit;
-
-             f->pixelformat = fmt->pixfmt;
-             f->flags = fmt->flags;
-             ret = 0;
      }
+     if (!fmt)
+             goto exit;
 
+     memset(f->reserved, 0, sizeof(f->reserved));
+     f->pixelformat = fmt->pixfmt;
+     f->flags = fmt->flags;
+     ret = 0;
 exit:
      vpu_inst_unlock(inst);
      return ret;

@@ -289,14 +301,14 @@ static int vdec_g_fmt(struct file *file, void *fh,

struct v4l2_format *f)

Quoted 21 lines

drivers/media/platform/amphion/vdec.c +3 -3

      cur_fmt = vpu_get_format(inst, f->type);
 
      pixmp->pixelformat = cur_fmt->pixfmt;
-     pixmp->num_planes = cur_fmt->num_planes;
+     pixmp->num_planes = cur_fmt->mem_planes;
      pixmp->width = cur_fmt->width;
      pixmp->height = cur_fmt->height;
      pixmp->field = cur_fmt->field;
      pixmp->flags = cur_fmt->flags;
      for (i = 0; i < pixmp->num_planes; i++) {
              pixmp->plane_fmt[i].bytesperline = cur_fmt->bytesperline[i];
-             pixmp->plane_fmt[i].sizeimage = cur_fmt->sizeimage[i];
+             pixmp->plane_fmt[i].sizeimage =
+ vpu_get_fmt_plane_size(cur_fmt, i);
      }
 
      f->fmt.pix_mp.colorspace = vdec->codec_info.color_primaries; @@
-311,10 +323,19 @@ static int vdec_try_fmt(struct file *file, void

*fh, struct v4l2_format *f)  {
      struct vpu_inst *inst = to_inst(file);
      struct vdec_t *vdec = inst->priv;

drivers/media/platform/amphion/vdec.c static int vdec_try_fmt(struct file *file, void *fh, struct v4l2_format *f)

Quoted 10 lines

drivers/media/platform/amphion/vdec.c +6 -2

-
-     vpu_try_fmt_common(inst, f);
+     struct vpu_format fmt;
 
      vpu_inst_lock(inst);
+     if (V4L2_TYPE_IS_CAPTURE(f->type) && vdec->fixed_fmt) {
+             struct vpu_format *cap_fmt = vpu_get_format(inst,
+ f->type);
+
+             if (!vpu_helper_match_format(inst, cap_fmt->type,

cap_fmt->pixfmt,

drivers/media/platform/amphion/vdec.c static int vdec_try_fmt(struct file *file, void *fh, struct v4l2_format *f)

Quoted 8 lines

drivers/media/platform/amphion/vdec.c +6 -0

+                                          f->fmt.pix_mp.pixelformat))
+                     f->fmt.pix_mp.pixelformat = cap_fmt->pixfmt;
+     }
+
+     vpu_try_fmt_common(inst, f, &fmt);
+
      if (vdec->fixed_fmt) {
              f->fmt.pix_mp.colorspace =

vdec->codec_info.color_primaries;

Quoted 14 lines

              f->fmt.pix_mp.xfer_func =
vdec->codec_info.transfer_chars; @@ -334,7 +355,7 @@ static int
vdec_try_fmt(struct file *file, void *fh, struct v4l2_format *f)
static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)  {
      struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;

drivers/media/platform/amphion/vdec.c +1 -1

-     const struct vpu_format *fmt;
+     struct vpu_format fmt;
      struct vpu_format *cur_fmt;
      struct vb2_queue *q;
      struct vdec_t *vdec = inst->priv; @@ -349,36 +370,30 @@ static

int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)

      if (vb2_is_busy(q))
              return -EBUSY;

drivers/media/platform/amphion/vdec.c static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)

Quoted 7 lines

drivers/media/platform/amphion/vdec.c +1 -2

-     fmt = vpu_try_fmt_common(inst, f);
-     if (!fmt)
+     if (vpu_try_fmt_common(inst, f, &fmt))
              return -EINVAL;
 
      cur_fmt = vpu_get_format(inst, f->type);
      if (V4L2_TYPE_IS_OUTPUT(f->type) && inst->state !=

VPU_CODEC_STATE_DEINIT) {

Quoted 4 lines

drivers/media/platform/amphion/vdec.c +1 -1

-             if (cur_fmt->pixfmt != fmt->pixfmt) {
+             if (cur_fmt->pixfmt != fmt.pixfmt) {
                      vdec->reset_codec = true;
                      vdec->fixed_fmt = false;

drivers/media/platform/amphion/vdec.c static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)

Quoted 10 lines

drivers/media/platform/amphion/vdec.c +0 -7

              }
      }
-     cur_fmt->pixfmt = fmt->pixfmt;
      if (V4L2_TYPE_IS_OUTPUT(f->type) || !vdec->fixed_fmt) {
-             cur_fmt->num_planes = fmt->num_planes;
-             cur_fmt->flags = fmt->flags;
-             cur_fmt->width = pixmp->width;
-             cur_fmt->height = pixmp->height;
-             for (i = 0; i < fmt->num_planes; i++) {
-                     cur_fmt->sizeimage[i] =

pixmp->plane_fmt[i].sizeimage;

Quoted 1 line

- cur_fmt->bytesperline[i] =

pixmp->plane_fmt[i].bytesperline;

drivers/media/platform/amphion/vdec.c static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)

Quoted 7 lines

drivers/media/platform/amphion/vdec.c +2 -4

-             }
-             if (pixmp->field != V4L2_FIELD_ANY)
-                     cur_fmt->field = pixmp->field;
+             memcpy(cur_fmt, &fmt, sizeof(*cur_fmt));
      } else {
-             pixmp->num_planes = cur_fmt->num_planes;
+             if (vpu_helper_match_format(inst, f->type, cur_fmt->pixfmt,

pixmp->pixelformat)) {

drivers/media/platform/amphion/vdec.c static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)

Quoted 9 lines

drivers/media/platform/amphion/vdec.c +5 -0

+                     cur_fmt->pixfmt = fmt.pixfmt;
+                     cur_fmt->mem_planes = fmt.mem_planes;
+             }
+             pixmp->pixelformat = cur_fmt->pixfmt;
+             pixmp->num_planes = cur_fmt->mem_planes;
              pixmp->width = cur_fmt->width;
              pixmp->height = cur_fmt->height;
              for (i = 0; i < pixmp->num_planes; i++) {
                      pixmp->plane_fmt[i].bytesperline =

cur_fmt->bytesperline[i];

Quoted 1 line

- pixmp->plane_fmt[i].sizeimage =

cur_fmt->sizeimage[i];

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst,

Quoted 18 lines

drivers/media/platform/amphion/vdec.c +4 -0

+                     pixmp->plane_fmt[i].sizeimage =
+ vpu_get_fmt_plane_size(cur_fmt, i);
              }
              pixmp->field = cur_fmt->field;
      }

@@ -678,9 +693,11 @@ static struct vpu_vb2_buffer
 *vdec_find_buffer(struct vpu_inst *inst, u32 luma)  static void
 vdec_buf_done(struct vpu_inst *inst, struct vpu_frame_info *frame)  {
      struct vdec_t *vdec = inst->priv;
+     struct vpu_format *cur_fmt;
      struct vpu_vb2_buffer *vpu_buf;
      struct vb2_v4l2_buffer *vbuf;
      u32 sequence;
+     int i;
 
      if (!frame)
              return;

@@ -699,6 +716,7 @@ static void vdec_buf_done(struct vpu_inst *inst,

struct vpu_frame_info *frame)

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst,

Quoted 8 lines

drivers/media/platform/amphion/vdec.c +1 -0

              return;
      }
 
+     cur_fmt = vpu_get_format(inst, inst->cap_format.type);
      vbuf = &vpu_buf->m2m_buf.vb;
      if (vbuf->vb2_buf.index != frame->id)
              dev_err(inst->dev, "[%d] buffer id(%d, %d) dismatch\n",

@@ -707,9 +725,9 @@ static void vdec_buf_done(struct vpu_inst *inst,

struct vpu_frame_info *frame)

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst, · line 726

Quoted 2 lines

drivers/media/platform/amphion/vdec.c +0 -0

      if (vpu_get_buffer_state(vbuf) != VPU_BUF_STATE_DECODED)
              dev_err(inst->dev, "[%d] buffer(%d) ready without

decoded\n", inst->id, frame->id);

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst, · line 710

Quoted 2 lines

drivers/media/platform/amphion/vdec.c +0 -1

      vpu_set_buffer_state(vbuf, VPU_BUF_STATE_READY);
-     vb2_set_plane_payload(&vbuf->vb2_buf, 0,

inst->cap_format.sizeimage[0]);

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst, · line 711

Quoted 1 line

drivers/media/platform/amphion/vdec.c +0 -1
-     vb2_set_plane_payload(&vbuf->vb2_buf, 1,

inst->cap_format.sizeimage[1]);

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst, struct vpu_frame_info *frame)

Quoted 3 lines

drivers/media/platform/amphion/vdec.c +2 -1

-     vbuf->field = inst->cap_format.field;
+     for (i = 0; i < vbuf->vb2_buf.num_planes; i++)
+             vb2_set_plane_payload(&vbuf->vb2_buf, i,

vpu_get_fmt_plane_size(cur_fmt, i));

Quoted 6 lines

drivers/media/platform/amphion/vdec.c +1 -0

+     vbuf->field = cur_fmt->field;
      vbuf->sequence = sequence;
      dev_dbg(inst->dev, "[%d][OUTPUT TS]%32lld\n", inst->id,
 vbuf->vb2_buf.timestamp);

drivers/media/platform/amphion/vdec.c static void vdec_stop_done(struct vpu_inst · line 772

Quoted 10 lines

drivers/media/platform/amphion/vdec.c +2 -2

@@ -747,15 +765,17 @@ static void vdec_stop_done(struct vpu_inst
 *inst)  static bool vdec_check_source_change(struct vpu_inst *inst)  {
      struct vdec_t *vdec = inst->priv;
-     const struct vpu_format *fmt;
-     int i;
+     const struct vpu_format *sibling;
 
      if (!inst->fh.m2m_ctx)
              return false;
 
+     sibling = vpu_helper_find_sibling(inst, inst->cap_format.type,

inst->cap_format.pixfmt);

drivers/media/platform/amphion/vdec.c static void vdec_stop_done(struct vpu_inst *inst)

Quoted 6 lines

drivers/media/platform/amphion/vdec.c +3 -1

+     if (sibling && vdec->codec_info.pixfmt == sibling->pixfmt)
+             vdec->codec_info.pixfmt = inst->cap_format.pixfmt;
+
      if (!vb2_is_streaming(v4l2_m2m_get_dst_vq(inst->fh.m2m_ctx)))
              return true;
-     fmt = vpu_helper_find_format(inst, inst->cap_format.type,

vdec->codec_info.pixfmt);

drivers/media/platform/amphion/vdec.c static void vdec_stop_done(struct vpu_inst *inst)

Quoted 4 lines

drivers/media/platform/amphion/vdec.c +0 -1

      if (inst->cap_format.pixfmt != vdec->codec_info.pixfmt)
              return true;
      if (inst->cap_format.width != vdec->codec_info.decoded_width) @@
-772,14 +792,6 @@ static bool vdec_check_source_change(struct vpu_inst

*inst)

drivers/media/platform/amphion/vdec.c static bool vdec_check_source_change(struct vpu_inst *inst)

Quoted 7 lines

drivers/media/platform/amphion/vdec.c +0 -4

              return true;
      if (inst->crop.height != vdec->codec_info.height)
              return true;
-     if (fmt && inst->cap_format.num_planes != fmt->num_planes)
-             return true;
-     for (i = 0; i < inst->cap_format.num_planes; i++) {
-             if (inst->cap_format.bytesperline[i] !=

vdec->codec_info.bytesperline[i])

drivers/media/platform/amphion/vdec.c

Quoted 2 lines

drivers/media/platform/amphion/vdec.c +0 -2

-                     return true;
-             if (inst->cap_format.sizeimage[i] !=

vdec->codec_info.sizeimage[i])

drivers/media/platform/amphion/vdec.c static bool vdec_check_source_change(struct · line 792

Quoted 13 lines

drivers/media/platform/amphion/vdec.c +1 -5

-                     return true;
-     }
 
      return false;
 }

@@ -787,27 +799,21 @@ static bool vdec_check_source_change(struct
 vpu_inst *inst)  static void vdec_init_fmt(struct vpu_inst *inst)  {
      struct vdec_t *vdec = inst->priv;
-     const struct vpu_format *fmt;
-     int i;
+     struct v4l2_format f;
 
-     fmt = vpu_helper_find_format(inst, inst->cap_format.type,

vdec->codec_info.pixfmt);

drivers/media/platform/amphion/vdec.c static bool vdec_check_source_change(struct vpu_inst *inst)

Quoted 11 lines

drivers/media/platform/amphion/vdec.c +0 ~~-11~~

-     inst->out_format.width = vdec->codec_info.width;
-     inst->out_format.height = vdec->codec_info.height;
-     inst->cap_format.width = vdec->codec_info.decoded_width;
-     inst->cap_format.height = vdec->codec_info.decoded_height;
-     inst->cap_format.pixfmt = vdec->codec_info.pixfmt;
-     if (fmt) {
-             inst->cap_format.num_planes = fmt->num_planes;
-             inst->cap_format.flags = fmt->flags;
-     }
-     for (i = 0; i < inst->cap_format.num_planes; i++) {
-             inst->cap_format.bytesperline[i] =

vdec->codec_info.bytesperline[i];

Quoted 20 lines

drivers/media/platform/amphion/vdec.c +11 -4

-             inst->cap_format.sizeimage[i] = vdec->codec_info.sizeimage[i];
-     }
+     memset(&f, 0, sizeof(f));
+     f.type = inst->cap_format.type;
+     f.fmt.pix_mp.pixelformat = vdec->codec_info.pixfmt;
+     f.fmt.pix_mp.width = vdec->codec_info.decoded_width;
+     f.fmt.pix_mp.height = vdec->codec_info.decoded_height;
      if (vdec->codec_info.progressive)
-             inst->cap_format.field = V4L2_FIELD_NONE;
+             f.fmt.pix_mp.field = V4L2_FIELD_NONE;
      else
-             inst->cap_format.field = V4L2_FIELD_SEQ_TB;
+             f.fmt.pix_mp.field = V4L2_FIELD_SEQ_TB;
+     vpu_try_fmt_common(inst, &f, &inst->cap_format);
+
+     inst->out_format.width = vdec->codec_info.width;
+     inst->out_format.height = vdec->codec_info.height;
 }
 
 static void vdec_init_crop(struct vpu_inst *inst) @@ -966,7 +972,10

Quoted 1 line

@@ static int vdec_response_frame(struct vpu_inst *inst, struct

vb2_v4l2_buffer *vb

drivers/media/platform/amphion/vdec.c static int vdec_response_frame(struct vpu_inst *inst, struct vb2_v4l2_buffer *vb

Quoted 6 lines

      info.tag = vdec->seq_tag;
      info.luma_addr = vpu_get_vb_phy_addr(&vbuf->vb2_buf, 0);
      info.luma_size = inst->cap_format.sizeimage[0];

drivers/media/platform/amphion/vdec.c +2 -1

-     info.chroma_addr = vpu_get_vb_phy_addr(&vbuf->vb2_buf, 1);
+     if (vbuf->vb2_buf.num_planes > 1)
+             info.chroma_addr = vpu_get_vb_phy_addr(&vbuf->vb2_buf,

1);

drivers/media/platform/amphion/venc.c static int venc_g_fmt(struct file *file, void *fh,

Quoted 40 lines

drivers/media/platform/amphion/vdec.c +2 -0

+     else
+             info.chroma_addr = info.luma_addr + info.luma_size;
      info.chromau_size = inst->cap_format.sizeimage[1];
      info.bytesperline = inst->cap_format.bytesperline[0];
      ret = vpu_session_alloc_fs(inst, &info); @@ -975,7 +984,7 @@

static int vdec_response_frame(struct vpu_inst *inst, struct vb2_v4l2_buffer *vb

      vpu_buf->tag = info.tag;
      vpu_buf->luma = info.luma_addr;

drivers/media/platform/amphion/vdec.c +1 -1

-     vpu_buf->chroma_u = info.chromau_size;
+     vpu_buf->chroma_u = info.chroma_addr;
      vpu_buf->chroma_v = 0;
      vpu_set_buffer_state(vbuf, VPU_BUF_STATE_INUSE);
      vdec->slots[info.id] = vpu_buf;

drivers/media/platform/amphion/venc.c +5 -2

index 37212f087fdd..060a1ee78b17 100644
@@ -69,13 +69,16 @@ struct venc_frame_t {  static const struct
 vpu_format venc_formats[] = {
      {
              .pixfmt = V4L2_PIX_FMT_NV12M,
-             .num_planes = 2,
+             .mem_planes = 2,
+             .comp_planes = 2,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
      },
      {
              .pixfmt = V4L2_PIX_FMT_H264,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+             .flags = V4L2_FMT_FLAG_COMPRESSED
      },
      {0, 0, 0, 0},
 };

@@ -173,14 +176,14 @@ static int venc_g_fmt(struct file *file, void *fh,

struct v4l2_format *f)

drivers/media/platform/amphion/venc.c static int venc_s_fmt(struct file *file, void *fh,

Quoted 43 lines

drivers/media/platform/amphion/venc.c +3 -3

      cur_fmt = vpu_get_format(inst, f->type);
 
      pixmp->pixelformat = cur_fmt->pixfmt;
-     pixmp->num_planes = cur_fmt->num_planes;
+     pixmp->num_planes = cur_fmt->mem_planes;
      pixmp->width = cur_fmt->width;
      pixmp->height = cur_fmt->height;
      pixmp->field = cur_fmt->field;
      pixmp->flags = cur_fmt->flags;
      for (i = 0; i < pixmp->num_planes; i++) {
              pixmp->plane_fmt[i].bytesperline = cur_fmt->bytesperline[i];
-             pixmp->plane_fmt[i].sizeimage = cur_fmt->sizeimage[i];
+             pixmp->plane_fmt[i].sizeimage =
+ vpu_get_fmt_plane_size(cur_fmt, i);
      }
 
      f->fmt.pix_mp.colorspace = venc->params.color.primaries; @@
-194,8 +197,9 @@ static int venc_g_fmt(struct file *file, void *fh,

struct v4l2_format *f) static int venc_try_fmt(struct file *file,

void *fh, struct v4l2_format *f)  {
      struct vpu_inst *inst = to_inst(file);

drivers/media/platform/amphion/venc.c +3 -3

+     struct vpu_format fmt;
 
-     vpu_try_fmt_common(inst, f);
+     vpu_try_fmt_common(inst, f, &fmt);
 
      return 0;
 }

@@ -203,12 +207,11 @@ static int venc_try_fmt(struct file *file, void
 *fh, struct v4l2_format *f)  static int venc_s_fmt(struct file *file,
 void *fh, struct v4l2_format *f)  {
      struct vpu_inst *inst = to_inst(file);
-     const struct vpu_format *fmt;
+     struct vpu_format fmt;
      struct vpu_format *cur_fmt;
      struct vb2_queue *q;
      struct venc_t *venc = inst->priv;
      struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
-     int i;
 
      q = v4l2_m2m_get_vq(inst->fh.m2m_ctx, f->type);
      if (!q)

@@ -216,24 +219,12 @@ static int venc_s_fmt(struct file *file, void *fh,

struct v4l2_format *f)

drivers/media/platform/amphion/vpu_dbg.c static int vpu_dbg_instance(struct seq_file *s, void

Quoted 55 lines

drivers/media/platform/amphion/venc.c +2 ~~-14~~

      if (vb2_is_busy(q))
              return -EBUSY;
 
-     fmt = vpu_try_fmt_common(inst, f);
-     if (!fmt)
+     if (vpu_try_fmt_common(inst, f, &fmt))
              return -EINVAL;
 
      cur_fmt = vpu_get_format(inst, f->type);
 
-     cur_fmt->pixfmt = fmt->pixfmt;
-     cur_fmt->num_planes = fmt->num_planes;
-     cur_fmt->flags = fmt->flags;
-     cur_fmt->width = pix_mp->width;
-     cur_fmt->height = pix_mp->height;
-     for (i = 0; i < fmt->num_planes; i++) {
-             cur_fmt->sizeimage[i] = pix_mp->plane_fmt[i].sizeimage;
-             cur_fmt->bytesperline[i] = pix_mp->plane_fmt[i].bytesperline;
-     }
-
-     if (pix_mp->field != V4L2_FIELD_ANY)
-             cur_fmt->field = pix_mp->field;
+     memcpy(cur_fmt, &fmt, sizeof(*cur_fmt));
 
      if (V4L2_TYPE_IS_OUTPUT(f->type)) {
              venc->params.input_format = cur_fmt->pixfmt; diff --git

a/drivers/media/platform/amphion/vpu.h
b/drivers/media/platform/amphion/vpu.h
index f914de6ed81e..998591fa5b85 100644

drivers/media/platform/amphion/vpu.h +3 -1

@@ -84,7 +84,8 @@ struct vpu_dev {
 
 struct vpu_format {
      u32 pixfmt;
-     unsigned int num_planes;
+     u32 mem_planes;
+     u32 comp_planes;
      u32 type;
      u32 flags;
      u32 width;

@@ -92,6 +93,7 @@ struct vpu_format {
      u32 sizeimage[VIDEO_MAX_PLANES];
      u32 bytesperline[VIDEO_MAX_PLANES];
      u32 field;
+     u32 sibling;
 };
 
 struct vpu_core_resources {

drivers/media/platform/amphion/vpu_dbg.c +0 -0

index f72c8a506b22..c41c9896912c 100644
@@ -89,9 +89,9 @@ static int vpu_dbg_instance(struct seq_file *s, void

*data)

drivers/media/platform/amphion/vpu_dbg.c static int vpu_dbg_instance(struct seq_file *s, void

Quoted 13 lines

drivers/media/platform/amphion/vpu_dbg.c +3 -2

                      vq->last_buffer_dequeued);
      if (seq_write(s, str, num))
              return 0;
-     for (i = 0; i < inst->out_format.num_planes; i++) {
+     for (i = 0; i < inst->out_format.mem_planes; i++) {
              num = scnprintf(str, sizeof(str), " %d(%d)",
-                             inst->out_format.sizeimage[i],
+
+ vpu_get_fmt_plane_size(&inst->out_format, i),
                              inst->out_format.bytesperline[i]);
              if (seq_write(s, str, num))
                      return 0;

@@ -113,9 +113,9 @@ static int vpu_dbg_instance(struct seq_file *s, void

*data)

drivers/media/platform/amphion/vpu_helpers.c const struct vpu_format

Quoted 18 lines

drivers/media/platform/amphion/vpu_dbg.c +3 -2

                      vq->last_buffer_dequeued);
      if (seq_write(s, str, num))
              return 0;
-     for (i = 0; i < inst->cap_format.num_planes; i++) {
+     for (i = 0; i < inst->cap_format.mem_planes; i++) {
              num = scnprintf(str, sizeof(str), " %d(%d)",
-                             inst->cap_format.sizeimage[i],
+
+ vpu_get_fmt_plane_size(&inst->cap_format, i),
                              inst->cap_format.bytesperline[i]);
              if (seq_write(s, str, num))
                      return 0;

drivers/media/platform/amphion/vpu_helpers.c +0 -0

index e9aeb3453dfc..db8f471b9187 100644
@@ -59,6 +59,39 @@ const struct vpu_format

*vpu_helper_find_format(struct vpu_inst *inst, u32 type,

drivers/media/platform/amphion/vpu_helpers.c static u32 get_nv12_plane_size(u32 width, u32

Quoted 40 lines

drivers/media/platform/amphion/vpu_helpers.c +33 -0

      return NULL;
 }
 
+const struct vpu_format *vpu_helper_find_sibling(struct vpu_inst
+*inst, u32 type, u32 pixelfmt) {
+     const struct vpu_format *fmt;
+     const struct vpu_format *sibling;
+
+     fmt = vpu_helper_find_format(inst, type, pixelfmt);
+     if (!fmt)
+             return NULL;
+     if (!fmt->sibling)
+             return NULL;
+     sibling = vpu_helper_find_format(inst, type, fmt->sibling);
+     if (!sibling)
+             return NULL;
+     if (sibling->sibling != fmt->pixfmt)
+             return NULL;
+     if (sibling->comp_planes != fmt->comp_planes)
+             return NULL;
+     return sibling;
+}
+
+bool vpu_helper_match_format(struct vpu_inst *inst, u32 type, u32
+fmta, u32 fmtb) {
+     const struct vpu_format *sibling;
+
+     if (fmta == fmtb)
+             return true;
+
+     sibling = vpu_helper_find_sibling(inst, type, fmta);
+     if (sibling && sibling->pixfmt == fmtb)
+             return true;
+     return false;
+}
+
 const struct vpu_format *vpu_helper_enum_format(struct vpu_inst
 *inst, u32 type, int index)  {
      const struct vpu_format *pfmt;

@@ -123,9 +156,10 @@ static u32 get_nv12_plane_size(u32 width, u32

height, int plane_no,

drivers/media/platform/amphion/vpu_helpers.c u32 vpu_helper_get_plane_size(u32 fmt, u32 w, u32

Quoted 43 lines

drivers/media/platform/amphion/vpu_helpers.c +2 -1

      u32 bytesperline;
      u32 size = 0;
 
-     bytesperline = ALIGN(width, stride);
+     bytesperline = width;
      if (pbl)
              bytesperline = max(bytesperline, *pbl);
+     bytesperline = ALIGN(bytesperline, stride);
      height = ALIGN(height, 2);
      if (plane_no == 0)
              size = bytesperline * height; @@ -148,13 +182,13 @@

static u32 get_tiled_8l128_plane_size(u32 fmt, u32 width, u32 height, int plane_

      if (interlaced)
              hs++;

drivers/media/platform/amphion/vpu_helpers.c +4 -3

-     if (fmt == V4L2_PIX_FMT_NV12M_10BE_8L128)
+     if (fmt == V4L2_PIX_FMT_NV12M_10BE_8L128 || fmt ==
+ V4L2_PIX_FMT_NV12_10BE_8L128)
              bitdepth = 10;
      bytesperline = DIV_ROUND_UP(width * bitdepth, BITS_PER_BYTE);
-     bytesperline = ALIGN(bytesperline, 1 << ws);
-     bytesperline = ALIGN(bytesperline, stride);
      if (pbl)
              bytesperline = max(bytesperline, *pbl);
+     bytesperline = ALIGN(bytesperline, 1 << ws);
+     bytesperline = ALIGN(bytesperline, stride);
      height = ALIGN(height, 1 << hs);
      if (plane_no == 0)
              size = bytesperline * height; @@ -172,9 +206,10 @@

static u32 get_default_plane_size(u32 width, u32 height, int plane_no,

      u32 bytesperline;
      u32 size = 0;

drivers/media/platform/amphion/vpu_helpers.c +2 -1

-     bytesperline = ALIGN(width, stride);
+     bytesperline = width;
      if (pbl)
              bytesperline = max(bytesperline, *pbl);
+     bytesperline = ALIGN(bytesperline, stride);
      if (plane_no == 0)
              size = bytesperline * height;
      if (pbl)

@@ -187,9 +222,12 @@ u32 vpu_helper_get_plane_size(u32 fmt, u32 w, u32

h, int plane_no,

drivers/media/platform/amphion/vpu_helpers.c u32 vpu_helper_get_plane_size(u32 fmt, u32 w, u32 h, int plane_no,

Quoted 11 lines

drivers/media/platform/amphion/vpu_helpers.c +3 -0

                            u32 stride, u32 interlaced, u32 *pbl)  {
      switch (fmt) {
+     case V4L2_PIX_FMT_NV12:
      case V4L2_PIX_FMT_NV12M:
              return get_nv12_plane_size(w, h, plane_no, stride,
 interlaced, pbl);
+     case V4L2_PIX_FMT_NV12_8L128:
      case V4L2_PIX_FMT_NV12M_8L128:
+     case V4L2_PIX_FMT_NV12_10BE_8L128:
      case V4L2_PIX_FMT_NV12M_10BE_8L128:
              return get_tiled_8l128_plane_size(fmt, w, h, plane_no, stride,

interlaced, pbl);

drivers/media/platform/amphion/vpu_malone.c bool vpu_malone_check_fmt(enum vpu_core_type

Quoted 24 lines

drivers/media/platform/amphion/vpu_helpers.c +0 -0
      default:
drivers/media/platform/amphion/vpu_helpers.h +0 -0
index bc28350958be..0eaddb07190d 100644
@@ -14,6 +14,8 @@ struct vpu_pair {
 int vpu_helper_find_in_array_u8(const u8 *array, u32 size, u32 x);
bool vpu_helper_check_type(struct vpu_inst *inst, u32 type); const struct vpu_format *vpu_helper_find_format(struct vpu_inst *inst, u32 type, u32 pixelfmt);
drivers/media/platform/amphion/vpu_helpers.h +3 -0
+const struct vpu_format *vpu_helper_find_sibling(struct vpu_inst
+*inst, u32 type, u32 pixelfmt); bool vpu_helper_match_format(struct
+vpu_inst *inst, u32 type, u32 fmta, u32 fmtb);
 const struct vpu_format *vpu_helper_enum_format(struct vpu_inst
*inst, u32 type, int index);
u32 vpu_helper_valid_frame_width(struct vpu_inst *inst, u32 width); u32 vpu_helper_valid_frame_height(struct vpu_inst *inst, u32 height);
drivers/media/platform/amphion/vpu_malone.c +0 -0
index 51e0702f9ae1..69d9a2269fce 100644
@@ -583,7 +583,8 @@ bool vpu_malone_check_fmt(enum vpu_core_type

type, u32 pixelfmt)

drivers/media/platform/amphion/vpu_malone.c bool vpu_malone_check_fmt(enum vpu_core_type type, u32 pixelfmt)

Quoted 4 lines

drivers/media/platform/amphion/vpu_malone.c +0 -1

      if (!vpu_imx8q_check_fmt(type, pixelfmt))
              return false;
 
-     if (pixelfmt == V4L2_PIX_FMT_NV12M_8L128 || pixelfmt ==

V4L2_PIX_FMT_NV12M_10BE_8L128)

drivers/media/platform/amphion/vpu_malone.c bool vpu_malone_check_fmt(enum vpu_core_type · line 586

Quoted 1 line

drivers/media/platform/amphion/vpu_malone.c +1 -0
+     if (pixelfmt == V4L2_PIX_FMT_NV12_8L128 || pixelfmt ==

V4L2_PIX_FMT_NV12_10BE_8L128 ||

Quoted 36 lines

drivers/media/platform/amphion/vpu_malone.c +2 -0

+         pixelfmt == V4L2_PIX_FMT_NV12M_8L128 || pixelfmt ==
+ V4L2_PIX_FMT_NV12M_10BE_8L128)
              return true;
      if (vpu_malone_format_remap(pixelfmt) == MALONE_FMT_NULL)
              return false;

drivers/media/platform/amphion/vpu_v4l2.c +17 -2

index 8a3eed957ae6..87cce552e906 100644
@@ -140,51 +140,135 @@ bool vpu_is_source_empty(struct vpu_inst *inst)
      return true;
 }
 
-const struct vpu_format *vpu_try_fmt_common(struct vpu_inst *inst,
 struct v4l2_format *f)
+static int vpu_init_format(struct vpu_inst *inst, struct vpu_format
+*fmt) {
+     const struct vpu_format *info;
+
+     info = vpu_helper_find_format(inst, fmt->type, fmt->pixfmt);
+     if (!info) {
+             info = vpu_helper_enum_format(inst, fmt->type, 0);
+             if (!info)
+                     return -EINVAL;
+     }
+     memcpy(fmt, info, sizeof(*fmt));
+
+     return 0;
+}
+
+static int vpu_calc_fmt_bytesperline(struct v4l2_format *f, struct
+vpu_format *fmt)
 {
      struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
-     u32 type = f->type;

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 172

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +10 -0

+     int i;
+
+     if (fmt->flags & V4L2_FMT_FLAG_COMPRESSED) {
+             for (i = 0; i < fmt->comp_planes; i++)
+                     fmt->bytesperline[i] = 0;
+             return 0;
+     }
+     if (pixmp->num_planes == fmt->comp_planes) {
+             for (i = 0; i < fmt->comp_planes; i++)
+                     fmt->bytesperline[i] =

pixmp->plane_fmt[i].bytesperline;

Quoted 32 lines

drivers/media/platform/amphion/vpu_v4l2.c +19 ~~-10~~

+             return 0;
+     }
+     if (pixmp->num_planes > 1)
+             return -EINVAL;
+
+     /*amphion vpu only support nv12 and nv12 tiled,
+      * so the bytesperline of luma and chroma should be same
+      */
+     for (i = 0; i < fmt->comp_planes; i++)
+             fmt->bytesperline[i] = pixmp->plane_fmt[0].bytesperline;
+
+     return 0;
+}
+
+static int vpu_calc_fmt_sizeimage(struct vpu_inst *inst, struct
+vpu_format *fmt) {
      u32 stride = 1;
-     u32 bytesperline;
-     u32 sizeimage;
-     const struct vpu_format *fmt;
-     const struct vpu_core_resources *res;
      int i;
 
-     fmt = vpu_helper_find_format(inst, type, pixmp->pixelformat);
-     if (!fmt) {
-             fmt = vpu_helper_enum_format(inst, type, 0);
-             if (!fmt)
-                     return NULL;
-             pixmp->pixelformat = fmt->pixfmt;
+     if (!(fmt->flags & V4L2_FMT_FLAG_COMPRESSED)) {
+             const struct vpu_core_resources *res =
+ vpu_get_resource(inst);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -5

+
+             if (res)
+                     stride = res->stride;
      }
 
-     res = vpu_get_resource(inst);
-     if (res)
-             stride = res->stride;
-     if (pixmp->width)
-             pixmp->width = vpu_helper_valid_frame_width(inst,

pixmp->width);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 169

Quoted 2 lines

drivers/media/platform/amphion/vpu_v4l2.c +0 -2

-     if (pixmp->height)
-             pixmp->height = vpu_helper_valid_frame_height(inst,

pixmp->height);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 211

Quoted 3 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -0

+     for (i = 0; i < fmt->comp_planes; i++) {
+             fmt->sizeimage[i] = vpu_helper_get_plane_size(fmt->pixfmt,
+

fmt->width,

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 203

Quoted 1 line

drivers/media/platform/amphion/vpu_v4l2.c +1 -0
+

fmt->height,

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 211

Quoted 3 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -0

+                                                           i,
+                                                           stride,
+

fmt->field != V4L2_FIELD_NONE ? 1 : 0,

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 207

Quoted 1 line

drivers/media/platform/amphion/vpu_v4l2.c +1 -0
+

&fmt->bytesperline[i]);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 3 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -0

+             fmt->sizeimage[i] = max_t(u32, fmt->sizeimage[i], PAGE_SIZE);
+             if (fmt->flags & V4L2_FMT_FLAG_COMPRESSED) {
+                     fmt->sizeimage[i] = clamp_val(fmt->sizeimage[i],

SZ_128K, SZ_8M);

Quoted 31 lines

drivers/media/platform/amphion/vpu_v4l2.c +31 -0

+                     fmt->bytesperline[i] = 0;
+             }
+     }
+
+     return 0;
+}
+
+u32 vpu_get_fmt_plane_size(struct vpu_format *fmt, u32 plane_no) {
+     u32 size;
+     int i;
+
+     if (plane_no >= fmt->mem_planes)
+             return 0;
+
+     if (fmt->comp_planes == fmt->mem_planes)
+             return fmt->sizeimage[plane_no];
+     if (plane_no < fmt->mem_planes - 1)
+             return fmt->sizeimage[plane_no];
+     size = fmt->sizeimage[plane_no];
+     for (i = fmt->mem_planes; i < fmt->comp_planes; i++)
+             size += fmt->sizeimage[i];
+
+     return size;
+}
+
+int vpu_try_fmt_common(struct vpu_inst *inst, struct v4l2_format *f,
+struct vpu_format *fmt) {
+     struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
+     int i;
+     int ret;
+

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +10 -0

+     fmt->pixfmt = pixmp->pixelformat;
+     fmt->type = f->type;
+     ret = vpu_init_format(inst, fmt);
+     if (ret < 0)
+             return ret;
+
+     fmt->width = pixmp->width;
+     fmt->height = pixmp->height;
+     if (fmt->width)
+             fmt->width = vpu_helper_valid_frame_width(inst,

fmt->width);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 253

Quoted 2 lines

drivers/media/platform/amphion/vpu_v4l2.c +2 -0

+     if (fmt->height)
+             fmt->height = vpu_helper_valid_frame_height(inst,

fmt->height);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 254

Quoted 1 line

drivers/media/platform/amphion/vpu_v4l2.c +1 -0
+     fmt->field = pixmp->field == V4L2_FIELD_ANY ? V4L2_FIELD_NONE :

pixmp->field;

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 3 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -0

+     vpu_calc_fmt_bytesperline(f, fmt);
+     vpu_calc_fmt_sizeimage(inst, fmt);
+     if ((fmt->flags & V4L2_FMT_FLAG_COMPRESSED) &&

pixmp->plane_fmt[0].sizeimage)

Quoted 5 lines

drivers/media/platform/amphion/vpu_v4l2.c +5 -0

+             fmt->sizeimage[0] =
+ clamp_val(pixmp->plane_fmt[0].sizeimage, SZ_128K, SZ_8M);
+
+     pixmp->pixelformat = fmt->pixfmt;
+     pixmp->width = fmt->width;

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +4 -4

+     pixmp->height = fmt->height;
      pixmp->flags = fmt->flags;
-     pixmp->num_planes = fmt->num_planes;
-     if (pixmp->field == V4L2_FIELD_ANY)
-             pixmp->field = V4L2_FIELD_NONE;
+     pixmp->num_planes = fmt->mem_planes;
+     pixmp->field = fmt->field;
+     memset(pixmp->reserved, 0, sizeof(pixmp->reserved));
      for (i = 0; i < pixmp->num_planes; i++) {
-             bytesperline = max_t(s32, pixmp->plane_fmt[i].bytesperline,

0);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 6 lines

drivers/media/platform/amphion/vpu_v4l2.c +0 -6

-             sizeimage = vpu_helper_get_plane_size(pixmp->pixelformat,
-                                                   pixmp->width,
-                                                   pixmp->height,
-                                                   i,
-                                                   stride,
-                                                   pixmp->field >

V4L2_FIELD_NONE ? 1 : 0,

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 183

Quoted 2 lines

drivers/media/platform/amphion/vpu_v4l2.c +0 -2

-                                                   &bytesperline);
-             sizeimage = max_t(s32, pixmp->plane_fmt[i].sizeimage,

sizeimage);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 4 lines

drivers/media/platform/amphion/vpu_v4l2.c +2 -2

-             pixmp->plane_fmt[i].bytesperline = bytesperline;
-             pixmp->plane_fmt[i].sizeimage = sizeimage;
+             pixmp->plane_fmt[i].bytesperline = fmt->bytesperline[i];
+             pixmp->plane_fmt[i].sizeimage = vpu_get_fmt_plane_size(fmt,

i);

drivers/media/platform/amphion/vpu_v4l2.c static int vpu_vb2_queue_setup(struct vb2_queue

Quoted 25 lines

drivers/media/platform/amphion/vpu_v4l2.c +7 -5

+             memset(pixmp->plane_fmt[i].reserved, 0,
+ sizeof(pixmp->plane_fmt[i].reserved));
      }
 
-     return fmt;
+     return 0;
 }
 
 static bool vpu_check_ready(struct vpu_inst *inst, u32 type) @@
-389,10 +473,10 @@ static int vpu_vb2_queue_setup(struct vb2_queue *vq,
      cur_fmt = vpu_get_format(inst, vq->type);
 
      if (*plane_count) {
-             if (*plane_count != cur_fmt->num_planes)
+             if (*plane_count != cur_fmt->mem_planes)
                      return -EINVAL;
-             for (i = 0; i < cur_fmt->num_planes; i++) {
-                     if (psize[i] < cur_fmt->sizeimage[i])
+             for (i = 0; i < cur_fmt->mem_planes; i++) {
+                     if (psize[i] < vpu_get_fmt_plane_size(cur_fmt,
+ i))
                              return -EINVAL;
              }
              return 0;

@@ -402,9 +486,9 @@ static int vpu_vb2_queue_setup(struct vb2_queue

*vq,

drivers/media/platform/amphion/vpu_v4l2.c static int vpu_vb2_queue_setup(struct vb2_queue · line 486

Quoted 1 line

drivers/media/platform/amphion/vpu_v4l2.c +0 -0
              *buf_count = max_t(unsigned int, *buf_count,

inst->min_buffer_out);

drivers/media/platform/amphion/vpu_v4l2.c static int vpu_vb2_queue_setup(struct vb2_queue · line 488

Quoted 2 lines

drivers/media/platform/amphion/vpu_v4l2.c +0 -0

      else
              *buf_count = max_t(unsigned int, *buf_count,

inst->min_buffer_cap);

drivers/media/platform/amphion/vpu_v4l2.c static int vpu_vb2_buf_prepare(struct vb2_buffer

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -3

-     *plane_count = cur_fmt->num_planes;
-     for (i = 0; i < cur_fmt->num_planes; i++)
-             psize[i] = cur_fmt->sizeimage[i];
+     *plane_count = cur_fmt->mem_planes;
+     for (i = 0; i < cur_fmt->mem_planes; i++)
+             psize[i] = vpu_get_fmt_plane_size(cur_fmt, i);
 
      return 0;
 }

@@ -434,8 +518,8 @@ static int vpu_vb2_buf_prepare(struct vb2_buffer

*vb)

drivers/media/platform/amphion/vpu_v4l2.c static int vpu_vb2_buf_prepare(struct vb2_buffer *vb)

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -2

      u32 i;
 
      cur_fmt = vpu_get_format(inst, vb->type);
-     for (i = 0; i < cur_fmt->num_planes; i++) {
-             if (vpu_get_vb_length(vb, i) < cur_fmt->sizeimage[i]) {
+     for (i = 0; i < cur_fmt->mem_planes; i++) {
+             if (vpu_get_vb_length(vb, i) <
+ vpu_get_fmt_plane_size(cur_fmt, i)) {
                      dev_dbg(inst->dev, "[%d] %s buf[%d] is invalid\n",
                              inst->id, vpu_type_name(vb->type),

vb->index);

Quoted 1 line

vpu_set_buffer_state(vbuf,

VPU_BUF_STATE_ERROR);

drivers/media/platform/amphion/vpu_windsor.c static int vpu_windsor_fill_yuv_frame(struct

Quoted 23 lines

drivers/media/platform/amphion/vpu_v4l2.h +0 -0
index 795ca33a6a50..ef5de6b66e47 100644
@@ -16,7 +16,8 @@ unsigned int vpu_get_buffer_state(struct
vb2_v4l2_buffer *vbuf); int vpu_v4l2_open(struct file *file, struct vpu_inst *inst); int vpu_v4l2_close(struct file *file);
-const struct vpu_format *vpu_try_fmt_common(struct vpu_inst *inst, struct v4l2_format *f);
drivers/media/platform/amphion/vpu_v4l2.h +3 -0
+u32 vpu_get_fmt_plane_size(struct vpu_format *fmt, u32 plane_no); int
+vpu_try_fmt_common(struct vpu_inst *inst, struct v4l2_format *f,
+struct vpu_format *fmt);
 int vpu_process_output_buffer(struct vpu_inst *inst);  int
vpu_process_capture_buffer(struct vpu_inst *inst); struct vb2_v4l2_buffer *vpu_next_src_buf(struct vpu_inst *inst); diff --git a/drivers/media/platform/amphion/vpu_windsor.c b/drivers/media/platform/amphion/vpu_windsor.c
index 1526af2ef9da..a454f142ae17 100644
drivers/media/platform/amphion/vpu_windsor.c +0 -0
@@ -775,6 +775,8 @@ static int vpu_windsor_fill_yuv_frame(struct

vpu_shared_addr *shared,

drivers/media/platform/amphion/vpu_windsor.c static int vpu_windsor_fill_yuv_frame(struct

Quoted 8 lines

drivers/media/platform/amphion/vpu_windsor.c +2 -0

                                    u32 instance,
                                    struct vb2_buffer *vb)  {
+     struct vpu_inst *inst = vb2_get_drv_priv(vb->vb2_queue);
+     struct vpu_format *out_fmt;
      struct vpu_enc_yuv_desc *desc;
      struct vb2_v4l2_buffer *vbuf;

@@ -782,6 +784,7 @@ static int vpu_windsor_fill_yuv_frame(struct

vpu_shared_addr *shared,

Quoted 21 lines

drivers/media/platform/amphion/vpu_windsor.c +1 -0

              return -EINVAL;
 
      desc = get_yuv_desc(shared, instance);
+     out_fmt = vpu_get_format(inst, vb->type);
 
      vbuf = to_vb2_v4l2_buffer(vb);
      desc->frame_id = vbuf->sequence; @@ -790,7 +793,10 @@ static int

vpu_windsor_fill_yuv_frame(struct vpu_shared_addr *shared,

      else
              desc->key_frame = 0;
      desc->luma_base = vpu_get_vb_phy_addr(vb, 0);

drivers/media/platform/amphion/vpu_windsor.c +5 -2

-     desc->chroma_base = vpu_get_vb_phy_addr(vb, 1);
+     if (vb->num_planes > 1)
+             desc->chroma_base = vpu_get_vb_phy_addr(vb, 1);
+     else
+             desc->chroma_base = desc->luma_base +
+ out_fmt->sizeimage[0];
 
      return 0;
 }
--

Quoted 2 lines

2.37.1

--
Tommaso Merciai
Embedded Linux Engineer
[email protected]
__________________________________

Signature · 10 lines

Tommaso Merciai
Embedded Linux Engineer
[email protected]
__________________________________

Amarula Solutions SRL
Via Le Canevare 30, 31100 Treviso, Veneto, IT
T. +39 042 243 5310
[email protected]
www.amarulasolutions.com

Tommaso Merciai

2022-08-12 09:49 UTC #10 lore raw

Re: [EXT] Re: [PATCH v4 2/4] media: amphion: tell and handle contiguous and non contiguous format

In reply to Ming Qian

Quoted 13 lines · Ming Qian

Quoted 13 lines
From: Tommaso Merciai <[email protected]>
Sent: 2022年8月12日 15:15
To: Ming Qian <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; dl-linux-imx <[email protected]>; X.H. Bao <[email protected]>; [email protected]; [email protected]; [email protected]
Subject: [EXT] Re: [PATCH v4 2/4] media: amphion: tell and handle contiguous and non contiguous format
Caution: EXT Email

Quoted 8 lines

Quoted 3 lines
Hi Ming,
Just an info. I'm planning to help to review your series. On which Linux tree are you working?
Hi Tommaso,
Thanks for your help, and I'm working on https://git.linuxtv.org/media_tree.git
Ming

Hi Ming,
Are you working on top of master branch? I have some issue on apply this.

Let me know.
Thanks,

Tommaso

Quoted 1,047 lines

Thanks in advance,
Tommaso

drivers/media/platform/amphion/vdec.c static int vdec_enum_fmt(struct file *file, void *fh,

Quoted 145 lines · Ming Qian

Driver should tell the number of memory planes and component planes. the amphion vpu support non contiguous planes, but for compatibility with other device that only support contiguous planes. driver can add support for contiguous planes in the same time. Then the mem_planes can be different from the comp_planes. driver need to handle buffer according mem_planes and comp_planes.

So driver can support NV12 and NV12M.

Signed-off-by Ming Qian <[email protected]>

---
 drivers/media/platform/amphion/vdec.c        | 187 ++++++++++---------
 drivers/media/platform/amphion/venc.c        |  33 ++--
 drivers/media/platform/amphion/vpu.h         |   4 +-
 drivers/media/platform/amphion/vpu_dbg.c     |   8 +-
 drivers/media/platform/amphion/vpu_helpers.c |  48 ++++-
 drivers/media/platform/amphion/vpu_helpers.h |   2 +
 drivers/media/platform/amphion/vpu_malone.c  |   3 +-
 drivers/media/platform/amphion/vpu_v4l2.c    | 168 ++++++++++++-----
 drivers/media/platform/amphion/vpu_v4l2.h    |   3 +-
 drivers/media/platform/amphion/vpu_windsor.c |   8 +-
 10 files changed, 299 insertions(+), 165 deletions(-)

drivers/media/platform/amphion/vdec.c +43 ~~-21~~

index feb75dc204de..48ab664fa7ef 100644
@@ -69,72 +69,85 @@ struct vdec_t {
 static const struct vpu_format vdec_formats[] = {
      {
              .pixfmt = V4L2_PIX_FMT_NV12M_8L128,
-             .num_planes = 2,
+             .mem_planes = 2,
+             .comp_planes = 2,
              .type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
      },
      {
              .pixfmt = V4L2_PIX_FMT_NV12M_10BE_8L128,
-             .num_planes = 2,
+             .mem_planes = 2,
+             .comp_planes = 2,
              .type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
      },
      {
              .pixfmt = V4L2_PIX_FMT_H264,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_H264_MVC,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_HEVC,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_VC1_ANNEX_G,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_VC1_ANNEX_L,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+             .flags = V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_MPEG2,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_MPEG4,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_XVID,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_VP8,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_H263,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {0, 0, 0, 0},
 };

@@ -256,23 +269,22 @@ static int vdec_enum_fmt(struct file *file, void *fh,

struct v4l2_fmtdesc *f)

drivers/media/platform/amphion/vdec.c static int vdec_enum_fmt(struct file *file, void *fh, struct v4l2_fmtdesc *f)

Quoted 13 lines

drivers/media/platform/amphion/vdec.c +4 -6

      int ret = -EINVAL;
 
      vpu_inst_lock(inst);
-     if (!V4L2_TYPE_IS_OUTPUT(f->type) && vdec->fixed_fmt) {
-             if (f->index == 0) {
-                     f->pixelformat = inst->cap_format.pixfmt;
-                     f->flags = inst->cap_format.flags;
-                     ret = 0;
-             }
+     if (V4L2_TYPE_IS_CAPTURE(f->type) && vdec->fixed_fmt) {
+             fmt = vpu_get_format(inst, f->type);
+             if (f->index == 1)
+                     fmt = vpu_helper_find_sibling(inst, f->type,

fmt->pixfmt);

drivers/media/platform/amphion/vdec.c static int vdec_g_fmt(struct file *file, void *fh,

Quoted 23 lines

drivers/media/platform/amphion/vdec.c +8 -7

+             if (f->index > 1)
+                     fmt = NULL;
      } else {
              fmt = vpu_helper_enum_format(inst, f->type, f->index);
-             memset(f->reserved, 0, sizeof(f->reserved));
-             if (!fmt)
-                     goto exit;
-
-             f->pixelformat = fmt->pixfmt;
-             f->flags = fmt->flags;
-             ret = 0;
      }
+     if (!fmt)
+             goto exit;
 
+     memset(f->reserved, 0, sizeof(f->reserved));
+     f->pixelformat = fmt->pixfmt;
+     f->flags = fmt->flags;
+     ret = 0;
 exit:
      vpu_inst_unlock(inst);
      return ret;

@@ -289,14 +301,14 @@ static int vdec_g_fmt(struct file *file, void *fh,

struct v4l2_format *f)

Quoted 21 lines

drivers/media/platform/amphion/vdec.c +3 -3

      cur_fmt = vpu_get_format(inst, f->type);
 
      pixmp->pixelformat = cur_fmt->pixfmt;
-     pixmp->num_planes = cur_fmt->num_planes;
+     pixmp->num_planes = cur_fmt->mem_planes;
      pixmp->width = cur_fmt->width;
      pixmp->height = cur_fmt->height;
      pixmp->field = cur_fmt->field;
      pixmp->flags = cur_fmt->flags;
      for (i = 0; i < pixmp->num_planes; i++) {
              pixmp->plane_fmt[i].bytesperline = cur_fmt->bytesperline[i];
-             pixmp->plane_fmt[i].sizeimage = cur_fmt->sizeimage[i];
+             pixmp->plane_fmt[i].sizeimage =
+ vpu_get_fmt_plane_size(cur_fmt, i);
      }
 
      f->fmt.pix_mp.colorspace = vdec->codec_info.color_primaries; @@
-311,10 +323,19 @@ static int vdec_try_fmt(struct file *file, void

*fh, struct v4l2_format *f)  {
      struct vpu_inst *inst = to_inst(file);
      struct vdec_t *vdec = inst->priv;

drivers/media/platform/amphion/vdec.c static int vdec_try_fmt(struct file *file, void *fh, struct v4l2_format *f)

Quoted 10 lines

drivers/media/platform/amphion/vdec.c +6 -2

-
-     vpu_try_fmt_common(inst, f);
+     struct vpu_format fmt;
 
      vpu_inst_lock(inst);
+     if (V4L2_TYPE_IS_CAPTURE(f->type) && vdec->fixed_fmt) {
+             struct vpu_format *cap_fmt = vpu_get_format(inst,
+ f->type);
+
+             if (!vpu_helper_match_format(inst, cap_fmt->type,

cap_fmt->pixfmt,

drivers/media/platform/amphion/vdec.c static int vdec_try_fmt(struct file *file, void *fh, struct v4l2_format *f)

Quoted 8 lines

drivers/media/platform/amphion/vdec.c +6 -0

+                                          f->fmt.pix_mp.pixelformat))
+                     f->fmt.pix_mp.pixelformat = cap_fmt->pixfmt;
+     }
+
+     vpu_try_fmt_common(inst, f, &fmt);
+
      if (vdec->fixed_fmt) {
              f->fmt.pix_mp.colorspace =

vdec->codec_info.color_primaries;

Quoted 14 lines

              f->fmt.pix_mp.xfer_func =
vdec->codec_info.transfer_chars; @@ -334,7 +355,7 @@ static int
vdec_try_fmt(struct file *file, void *fh, struct v4l2_format *f)
static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)  {
      struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;

drivers/media/platform/amphion/vdec.c +1 -1

-     const struct vpu_format *fmt;
+     struct vpu_format fmt;
      struct vpu_format *cur_fmt;
      struct vb2_queue *q;
      struct vdec_t *vdec = inst->priv; @@ -349,36 +370,30 @@ static

int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)

      if (vb2_is_busy(q))
              return -EBUSY;

drivers/media/platform/amphion/vdec.c static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)

Quoted 7 lines

drivers/media/platform/amphion/vdec.c +1 -2

-     fmt = vpu_try_fmt_common(inst, f);
-     if (!fmt)
+     if (vpu_try_fmt_common(inst, f, &fmt))
              return -EINVAL;
 
      cur_fmt = vpu_get_format(inst, f->type);
      if (V4L2_TYPE_IS_OUTPUT(f->type) && inst->state !=

VPU_CODEC_STATE_DEINIT) {

Quoted 4 lines

drivers/media/platform/amphion/vdec.c +1 -1

-             if (cur_fmt->pixfmt != fmt->pixfmt) {
+             if (cur_fmt->pixfmt != fmt.pixfmt) {
                      vdec->reset_codec = true;
                      vdec->fixed_fmt = false;

drivers/media/platform/amphion/vdec.c static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)

Quoted 10 lines

drivers/media/platform/amphion/vdec.c +0 -7

              }
      }
-     cur_fmt->pixfmt = fmt->pixfmt;
      if (V4L2_TYPE_IS_OUTPUT(f->type) || !vdec->fixed_fmt) {
-             cur_fmt->num_planes = fmt->num_planes;
-             cur_fmt->flags = fmt->flags;
-             cur_fmt->width = pixmp->width;
-             cur_fmt->height = pixmp->height;
-             for (i = 0; i < fmt->num_planes; i++) {
-                     cur_fmt->sizeimage[i] =

pixmp->plane_fmt[i].sizeimage;

Quoted 1 line

- cur_fmt->bytesperline[i] =

pixmp->plane_fmt[i].bytesperline;

drivers/media/platform/amphion/vdec.c static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)

Quoted 7 lines

drivers/media/platform/amphion/vdec.c +2 -4

-             }
-             if (pixmp->field != V4L2_FIELD_ANY)
-                     cur_fmt->field = pixmp->field;
+             memcpy(cur_fmt, &fmt, sizeof(*cur_fmt));
      } else {
-             pixmp->num_planes = cur_fmt->num_planes;
+             if (vpu_helper_match_format(inst, f->type, cur_fmt->pixfmt,

pixmp->pixelformat)) {

drivers/media/platform/amphion/vdec.c static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)

Quoted 9 lines

drivers/media/platform/amphion/vdec.c +5 -0

+                     cur_fmt->pixfmt = fmt.pixfmt;
+                     cur_fmt->mem_planes = fmt.mem_planes;
+             }
+             pixmp->pixelformat = cur_fmt->pixfmt;
+             pixmp->num_planes = cur_fmt->mem_planes;
              pixmp->width = cur_fmt->width;
              pixmp->height = cur_fmt->height;
              for (i = 0; i < pixmp->num_planes; i++) {
                      pixmp->plane_fmt[i].bytesperline =

cur_fmt->bytesperline[i];

Quoted 1 line

- pixmp->plane_fmt[i].sizeimage =

cur_fmt->sizeimage[i];

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst,

Quoted 18 lines

drivers/media/platform/amphion/vdec.c +4 -0

+                     pixmp->plane_fmt[i].sizeimage =
+ vpu_get_fmt_plane_size(cur_fmt, i);
              }
              pixmp->field = cur_fmt->field;
      }

@@ -678,9 +693,11 @@ static struct vpu_vb2_buffer
 *vdec_find_buffer(struct vpu_inst *inst, u32 luma)  static void
 vdec_buf_done(struct vpu_inst *inst, struct vpu_frame_info *frame)  {
      struct vdec_t *vdec = inst->priv;
+     struct vpu_format *cur_fmt;
      struct vpu_vb2_buffer *vpu_buf;
      struct vb2_v4l2_buffer *vbuf;
      u32 sequence;
+     int i;
 
      if (!frame)
              return;

@@ -699,6 +716,7 @@ static void vdec_buf_done(struct vpu_inst *inst,

struct vpu_frame_info *frame)

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst,

Quoted 8 lines

drivers/media/platform/amphion/vdec.c +1 -0

              return;
      }
 
+     cur_fmt = vpu_get_format(inst, inst->cap_format.type);
      vbuf = &vpu_buf->m2m_buf.vb;
      if (vbuf->vb2_buf.index != frame->id)
              dev_err(inst->dev, "[%d] buffer id(%d, %d) dismatch\n",

@@ -707,9 +725,9 @@ static void vdec_buf_done(struct vpu_inst *inst,

struct vpu_frame_info *frame)

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst, · line 726

Quoted 2 lines

drivers/media/platform/amphion/vdec.c +0 -0

      if (vpu_get_buffer_state(vbuf) != VPU_BUF_STATE_DECODED)
              dev_err(inst->dev, "[%d] buffer(%d) ready without

decoded\n", inst->id, frame->id);

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst, · line 710

Quoted 2 lines

drivers/media/platform/amphion/vdec.c +0 -1

      vpu_set_buffer_state(vbuf, VPU_BUF_STATE_READY);
-     vb2_set_plane_payload(&vbuf->vb2_buf, 0,

inst->cap_format.sizeimage[0]);

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst, · line 711

Quoted 1 line

drivers/media/platform/amphion/vdec.c +0 -1
-     vb2_set_plane_payload(&vbuf->vb2_buf, 1,

inst->cap_format.sizeimage[1]);

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst, struct vpu_frame_info *frame)

Quoted 3 lines

drivers/media/platform/amphion/vdec.c +2 -1

-     vbuf->field = inst->cap_format.field;
+     for (i = 0; i < vbuf->vb2_buf.num_planes; i++)
+             vb2_set_plane_payload(&vbuf->vb2_buf, i,

vpu_get_fmt_plane_size(cur_fmt, i));

Quoted 6 lines

drivers/media/platform/amphion/vdec.c +1 -0

+     vbuf->field = cur_fmt->field;
      vbuf->sequence = sequence;
      dev_dbg(inst->dev, "[%d][OUTPUT TS]%32lld\n", inst->id,
 vbuf->vb2_buf.timestamp);

drivers/media/platform/amphion/vdec.c static void vdec_stop_done(struct vpu_inst · line 772

Quoted 10 lines

drivers/media/platform/amphion/vdec.c +2 -2

@@ -747,15 +765,17 @@ static void vdec_stop_done(struct vpu_inst
 *inst)  static bool vdec_check_source_change(struct vpu_inst *inst)  {
      struct vdec_t *vdec = inst->priv;
-     const struct vpu_format *fmt;
-     int i;
+     const struct vpu_format *sibling;
 
      if (!inst->fh.m2m_ctx)
              return false;
 
+     sibling = vpu_helper_find_sibling(inst, inst->cap_format.type,

inst->cap_format.pixfmt);

drivers/media/platform/amphion/vdec.c static void vdec_stop_done(struct vpu_inst *inst)

Quoted 6 lines

drivers/media/platform/amphion/vdec.c +3 -1

+     if (sibling && vdec->codec_info.pixfmt == sibling->pixfmt)
+             vdec->codec_info.pixfmt = inst->cap_format.pixfmt;
+
      if (!vb2_is_streaming(v4l2_m2m_get_dst_vq(inst->fh.m2m_ctx)))
              return true;
-     fmt = vpu_helper_find_format(inst, inst->cap_format.type,

vdec->codec_info.pixfmt);

drivers/media/platform/amphion/vdec.c static void vdec_stop_done(struct vpu_inst *inst)

Quoted 4 lines

drivers/media/platform/amphion/vdec.c +0 -1

      if (inst->cap_format.pixfmt != vdec->codec_info.pixfmt)
              return true;
      if (inst->cap_format.width != vdec->codec_info.decoded_width) @@
-772,14 +792,6 @@ static bool vdec_check_source_change(struct vpu_inst

*inst)

drivers/media/platform/amphion/vdec.c static bool vdec_check_source_change(struct vpu_inst *inst)

Quoted 7 lines

drivers/media/platform/amphion/vdec.c +0 -4

              return true;
      if (inst->crop.height != vdec->codec_info.height)
              return true;
-     if (fmt && inst->cap_format.num_planes != fmt->num_planes)
-             return true;
-     for (i = 0; i < inst->cap_format.num_planes; i++) {
-             if (inst->cap_format.bytesperline[i] !=

vdec->codec_info.bytesperline[i])

drivers/media/platform/amphion/vdec.c

Quoted 2 lines

drivers/media/platform/amphion/vdec.c +0 -2

-                     return true;
-             if (inst->cap_format.sizeimage[i] !=

vdec->codec_info.sizeimage[i])

drivers/media/platform/amphion/vdec.c static bool vdec_check_source_change(struct · line 792

Quoted 13 lines

drivers/media/platform/amphion/vdec.c +1 -5

-                     return true;
-     }
 
      return false;
 }

@@ -787,27 +799,21 @@ static bool vdec_check_source_change(struct
 vpu_inst *inst)  static void vdec_init_fmt(struct vpu_inst *inst)  {
      struct vdec_t *vdec = inst->priv;
-     const struct vpu_format *fmt;
-     int i;
+     struct v4l2_format f;
 
-     fmt = vpu_helper_find_format(inst, inst->cap_format.type,

vdec->codec_info.pixfmt);

drivers/media/platform/amphion/vdec.c static bool vdec_check_source_change(struct vpu_inst *inst)

Quoted 11 lines

drivers/media/platform/amphion/vdec.c +0 ~~-11~~

-     inst->out_format.width = vdec->codec_info.width;
-     inst->out_format.height = vdec->codec_info.height;
-     inst->cap_format.width = vdec->codec_info.decoded_width;
-     inst->cap_format.height = vdec->codec_info.decoded_height;
-     inst->cap_format.pixfmt = vdec->codec_info.pixfmt;
-     if (fmt) {
-             inst->cap_format.num_planes = fmt->num_planes;
-             inst->cap_format.flags = fmt->flags;
-     }
-     for (i = 0; i < inst->cap_format.num_planes; i++) {
-             inst->cap_format.bytesperline[i] =

vdec->codec_info.bytesperline[i];

Quoted 20 lines

drivers/media/platform/amphion/vdec.c +11 -4

-             inst->cap_format.sizeimage[i] = vdec->codec_info.sizeimage[i];
-     }
+     memset(&f, 0, sizeof(f));
+     f.type = inst->cap_format.type;
+     f.fmt.pix_mp.pixelformat = vdec->codec_info.pixfmt;
+     f.fmt.pix_mp.width = vdec->codec_info.decoded_width;
+     f.fmt.pix_mp.height = vdec->codec_info.decoded_height;
      if (vdec->codec_info.progressive)
-             inst->cap_format.field = V4L2_FIELD_NONE;
+             f.fmt.pix_mp.field = V4L2_FIELD_NONE;
      else
-             inst->cap_format.field = V4L2_FIELD_SEQ_TB;
+             f.fmt.pix_mp.field = V4L2_FIELD_SEQ_TB;
+     vpu_try_fmt_common(inst, &f, &inst->cap_format);
+
+     inst->out_format.width = vdec->codec_info.width;
+     inst->out_format.height = vdec->codec_info.height;
 }
 
 static void vdec_init_crop(struct vpu_inst *inst) @@ -966,7 +972,10

Quoted 1 line

@@ static int vdec_response_frame(struct vpu_inst *inst, struct

vb2_v4l2_buffer *vb

drivers/media/platform/amphion/vdec.c static int vdec_response_frame(struct vpu_inst *inst, struct vb2_v4l2_buffer *vb

Quoted 6 lines

      info.tag = vdec->seq_tag;
      info.luma_addr = vpu_get_vb_phy_addr(&vbuf->vb2_buf, 0);
      info.luma_size = inst->cap_format.sizeimage[0];

drivers/media/platform/amphion/vdec.c +2 -1

-     info.chroma_addr = vpu_get_vb_phy_addr(&vbuf->vb2_buf, 1);
+     if (vbuf->vb2_buf.num_planes > 1)
+             info.chroma_addr = vpu_get_vb_phy_addr(&vbuf->vb2_buf,

1);

drivers/media/platform/amphion/venc.c static int venc_g_fmt(struct file *file, void *fh,

Quoted 40 lines

drivers/media/platform/amphion/vdec.c +2 -0

+     else
+             info.chroma_addr = info.luma_addr + info.luma_size;
      info.chromau_size = inst->cap_format.sizeimage[1];
      info.bytesperline = inst->cap_format.bytesperline[0];
      ret = vpu_session_alloc_fs(inst, &info); @@ -975,7 +984,7 @@

static int vdec_response_frame(struct vpu_inst *inst, struct vb2_v4l2_buffer *vb

      vpu_buf->tag = info.tag;
      vpu_buf->luma = info.luma_addr;

drivers/media/platform/amphion/vdec.c +1 -1

-     vpu_buf->chroma_u = info.chromau_size;
+     vpu_buf->chroma_u = info.chroma_addr;
      vpu_buf->chroma_v = 0;
      vpu_set_buffer_state(vbuf, VPU_BUF_STATE_INUSE);
      vdec->slots[info.id] = vpu_buf;

drivers/media/platform/amphion/venc.c +5 -2

index 37212f087fdd..060a1ee78b17 100644
@@ -69,13 +69,16 @@ struct venc_frame_t {  static const struct
 vpu_format venc_formats[] = {
      {
              .pixfmt = V4L2_PIX_FMT_NV12M,
-             .num_planes = 2,
+             .mem_planes = 2,
+             .comp_planes = 2,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
      },
      {
              .pixfmt = V4L2_PIX_FMT_H264,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+             .flags = V4L2_FMT_FLAG_COMPRESSED
      },
      {0, 0, 0, 0},
 };

@@ -173,14 +176,14 @@ static int venc_g_fmt(struct file *file, void *fh,

struct v4l2_format *f)

drivers/media/platform/amphion/venc.c static int venc_s_fmt(struct file *file, void *fh,

Quoted 43 lines

drivers/media/platform/amphion/venc.c +3 -3

      cur_fmt = vpu_get_format(inst, f->type);
 
      pixmp->pixelformat = cur_fmt->pixfmt;
-     pixmp->num_planes = cur_fmt->num_planes;
+     pixmp->num_planes = cur_fmt->mem_planes;
      pixmp->width = cur_fmt->width;
      pixmp->height = cur_fmt->height;
      pixmp->field = cur_fmt->field;
      pixmp->flags = cur_fmt->flags;
      for (i = 0; i < pixmp->num_planes; i++) {
              pixmp->plane_fmt[i].bytesperline = cur_fmt->bytesperline[i];
-             pixmp->plane_fmt[i].sizeimage = cur_fmt->sizeimage[i];
+             pixmp->plane_fmt[i].sizeimage =
+ vpu_get_fmt_plane_size(cur_fmt, i);
      }
 
      f->fmt.pix_mp.colorspace = venc->params.color.primaries; @@
-194,8 +197,9 @@ static int venc_g_fmt(struct file *file, void *fh,

struct v4l2_format *f) static int venc_try_fmt(struct file *file,

void *fh, struct v4l2_format *f)  {
      struct vpu_inst *inst = to_inst(file);

drivers/media/platform/amphion/venc.c +3 -3

+     struct vpu_format fmt;
 
-     vpu_try_fmt_common(inst, f);
+     vpu_try_fmt_common(inst, f, &fmt);
 
      return 0;
 }

@@ -203,12 +207,11 @@ static int venc_try_fmt(struct file *file, void
 *fh, struct v4l2_format *f)  static int venc_s_fmt(struct file *file,
 void *fh, struct v4l2_format *f)  {
      struct vpu_inst *inst = to_inst(file);
-     const struct vpu_format *fmt;
+     struct vpu_format fmt;
      struct vpu_format *cur_fmt;
      struct vb2_queue *q;
      struct venc_t *venc = inst->priv;
      struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
-     int i;
 
      q = v4l2_m2m_get_vq(inst->fh.m2m_ctx, f->type);
      if (!q)

@@ -216,24 +219,12 @@ static int venc_s_fmt(struct file *file, void *fh,

struct v4l2_format *f)

drivers/media/platform/amphion/vpu_dbg.c static int vpu_dbg_instance(struct seq_file *s, void

Quoted 55 lines

drivers/media/platform/amphion/venc.c +2 ~~-14~~

      if (vb2_is_busy(q))
              return -EBUSY;
 
-     fmt = vpu_try_fmt_common(inst, f);
-     if (!fmt)
+     if (vpu_try_fmt_common(inst, f, &fmt))
              return -EINVAL;
 
      cur_fmt = vpu_get_format(inst, f->type);
 
-     cur_fmt->pixfmt = fmt->pixfmt;
-     cur_fmt->num_planes = fmt->num_planes;
-     cur_fmt->flags = fmt->flags;
-     cur_fmt->width = pix_mp->width;
-     cur_fmt->height = pix_mp->height;
-     for (i = 0; i < fmt->num_planes; i++) {
-             cur_fmt->sizeimage[i] = pix_mp->plane_fmt[i].sizeimage;
-             cur_fmt->bytesperline[i] = pix_mp->plane_fmt[i].bytesperline;
-     }
-
-     if (pix_mp->field != V4L2_FIELD_ANY)
-             cur_fmt->field = pix_mp->field;
+     memcpy(cur_fmt, &fmt, sizeof(*cur_fmt));
 
      if (V4L2_TYPE_IS_OUTPUT(f->type)) {
              venc->params.input_format = cur_fmt->pixfmt; diff --git

a/drivers/media/platform/amphion/vpu.h
b/drivers/media/platform/amphion/vpu.h
index f914de6ed81e..998591fa5b85 100644

drivers/media/platform/amphion/vpu.h +3 -1

@@ -84,7 +84,8 @@ struct vpu_dev {
 
 struct vpu_format {
      u32 pixfmt;
-     unsigned int num_planes;
+     u32 mem_planes;
+     u32 comp_planes;
      u32 type;
      u32 flags;
      u32 width;

@@ -92,6 +93,7 @@ struct vpu_format {
      u32 sizeimage[VIDEO_MAX_PLANES];
      u32 bytesperline[VIDEO_MAX_PLANES];
      u32 field;
+     u32 sibling;
 };
 
 struct vpu_core_resources {

drivers/media/platform/amphion/vpu_dbg.c +0 -0

index f72c8a506b22..c41c9896912c 100644
@@ -89,9 +89,9 @@ static int vpu_dbg_instance(struct seq_file *s, void

*data)

drivers/media/platform/amphion/vpu_dbg.c static int vpu_dbg_instance(struct seq_file *s, void

Quoted 13 lines

drivers/media/platform/amphion/vpu_dbg.c +3 -2

                      vq->last_buffer_dequeued);
      if (seq_write(s, str, num))
              return 0;
-     for (i = 0; i < inst->out_format.num_planes; i++) {
+     for (i = 0; i < inst->out_format.mem_planes; i++) {
              num = scnprintf(str, sizeof(str), " %d(%d)",
-                             inst->out_format.sizeimage[i],
+
+ vpu_get_fmt_plane_size(&inst->out_format, i),
                              inst->out_format.bytesperline[i]);
              if (seq_write(s, str, num))
                      return 0;

@@ -113,9 +113,9 @@ static int vpu_dbg_instance(struct seq_file *s, void

*data)

drivers/media/platform/amphion/vpu_helpers.c const struct vpu_format

Quoted 18 lines

drivers/media/platform/amphion/vpu_dbg.c +3 -2

                      vq->last_buffer_dequeued);
      if (seq_write(s, str, num))
              return 0;
-     for (i = 0; i < inst->cap_format.num_planes; i++) {
+     for (i = 0; i < inst->cap_format.mem_planes; i++) {
              num = scnprintf(str, sizeof(str), " %d(%d)",
-                             inst->cap_format.sizeimage[i],
+
+ vpu_get_fmt_plane_size(&inst->cap_format, i),
                              inst->cap_format.bytesperline[i]);
              if (seq_write(s, str, num))
                      return 0;

drivers/media/platform/amphion/vpu_helpers.c +0 -0

index e9aeb3453dfc..db8f471b9187 100644
@@ -59,6 +59,39 @@ const struct vpu_format

*vpu_helper_find_format(struct vpu_inst *inst, u32 type,

drivers/media/platform/amphion/vpu_helpers.c static u32 get_nv12_plane_size(u32 width, u32

Quoted 40 lines

drivers/media/platform/amphion/vpu_helpers.c +33 -0

      return NULL;
 }
 
+const struct vpu_format *vpu_helper_find_sibling(struct vpu_inst
+*inst, u32 type, u32 pixelfmt) {
+     const struct vpu_format *fmt;
+     const struct vpu_format *sibling;
+
+     fmt = vpu_helper_find_format(inst, type, pixelfmt);
+     if (!fmt)
+             return NULL;
+     if (!fmt->sibling)
+             return NULL;
+     sibling = vpu_helper_find_format(inst, type, fmt->sibling);
+     if (!sibling)
+             return NULL;
+     if (sibling->sibling != fmt->pixfmt)
+             return NULL;
+     if (sibling->comp_planes != fmt->comp_planes)
+             return NULL;
+     return sibling;
+}
+
+bool vpu_helper_match_format(struct vpu_inst *inst, u32 type, u32
+fmta, u32 fmtb) {
+     const struct vpu_format *sibling;
+
+     if (fmta == fmtb)
+             return true;
+
+     sibling = vpu_helper_find_sibling(inst, type, fmta);
+     if (sibling && sibling->pixfmt == fmtb)
+             return true;
+     return false;
+}
+
 const struct vpu_format *vpu_helper_enum_format(struct vpu_inst
 *inst, u32 type, int index)  {
      const struct vpu_format *pfmt;

@@ -123,9 +156,10 @@ static u32 get_nv12_plane_size(u32 width, u32

height, int plane_no,

drivers/media/platform/amphion/vpu_helpers.c u32 vpu_helper_get_plane_size(u32 fmt, u32 w, u32

Quoted 43 lines

drivers/media/platform/amphion/vpu_helpers.c +2 -1

      u32 bytesperline;
      u32 size = 0;
 
-     bytesperline = ALIGN(width, stride);
+     bytesperline = width;
      if (pbl)
              bytesperline = max(bytesperline, *pbl);
+     bytesperline = ALIGN(bytesperline, stride);
      height = ALIGN(height, 2);
      if (plane_no == 0)
              size = bytesperline * height; @@ -148,13 +182,13 @@

static u32 get_tiled_8l128_plane_size(u32 fmt, u32 width, u32 height, int plane_

      if (interlaced)
              hs++;

drivers/media/platform/amphion/vpu_helpers.c +4 -3

-     if (fmt == V4L2_PIX_FMT_NV12M_10BE_8L128)
+     if (fmt == V4L2_PIX_FMT_NV12M_10BE_8L128 || fmt ==
+ V4L2_PIX_FMT_NV12_10BE_8L128)
              bitdepth = 10;
      bytesperline = DIV_ROUND_UP(width * bitdepth, BITS_PER_BYTE);
-     bytesperline = ALIGN(bytesperline, 1 << ws);
-     bytesperline = ALIGN(bytesperline, stride);
      if (pbl)
              bytesperline = max(bytesperline, *pbl);
+     bytesperline = ALIGN(bytesperline, 1 << ws);
+     bytesperline = ALIGN(bytesperline, stride);
      height = ALIGN(height, 1 << hs);
      if (plane_no == 0)
              size = bytesperline * height; @@ -172,9 +206,10 @@

static u32 get_default_plane_size(u32 width, u32 height, int plane_no,

      u32 bytesperline;
      u32 size = 0;

drivers/media/platform/amphion/vpu_helpers.c +2 -1

-     bytesperline = ALIGN(width, stride);
+     bytesperline = width;
      if (pbl)
              bytesperline = max(bytesperline, *pbl);
+     bytesperline = ALIGN(bytesperline, stride);
      if (plane_no == 0)
              size = bytesperline * height;
      if (pbl)

@@ -187,9 +222,12 @@ u32 vpu_helper_get_plane_size(u32 fmt, u32 w, u32

h, int plane_no,

drivers/media/platform/amphion/vpu_helpers.c u32 vpu_helper_get_plane_size(u32 fmt, u32 w, u32 h, int plane_no,

Quoted 11 lines

drivers/media/platform/amphion/vpu_helpers.c +3 -0

                            u32 stride, u32 interlaced, u32 *pbl)  {
      switch (fmt) {
+     case V4L2_PIX_FMT_NV12:
      case V4L2_PIX_FMT_NV12M:
              return get_nv12_plane_size(w, h, plane_no, stride,
 interlaced, pbl);
+     case V4L2_PIX_FMT_NV12_8L128:
      case V4L2_PIX_FMT_NV12M_8L128:
+     case V4L2_PIX_FMT_NV12_10BE_8L128:
      case V4L2_PIX_FMT_NV12M_10BE_8L128:
              return get_tiled_8l128_plane_size(fmt, w, h, plane_no, stride,

interlaced, pbl);

drivers/media/platform/amphion/vpu_malone.c bool vpu_malone_check_fmt(enum vpu_core_type

Quoted 24 lines

drivers/media/platform/amphion/vpu_helpers.c +0 -0
      default:
drivers/media/platform/amphion/vpu_helpers.h +0 -0
index bc28350958be..0eaddb07190d 100644
@@ -14,6 +14,8 @@ struct vpu_pair {
 int vpu_helper_find_in_array_u8(const u8 *array, u32 size, u32 x);
bool vpu_helper_check_type(struct vpu_inst *inst, u32 type); const struct vpu_format *vpu_helper_find_format(struct vpu_inst *inst, u32 type, u32 pixelfmt);
drivers/media/platform/amphion/vpu_helpers.h +3 -0
+const struct vpu_format *vpu_helper_find_sibling(struct vpu_inst
+*inst, u32 type, u32 pixelfmt); bool vpu_helper_match_format(struct
+vpu_inst *inst, u32 type, u32 fmta, u32 fmtb);
 const struct vpu_format *vpu_helper_enum_format(struct vpu_inst
*inst, u32 type, int index);
u32 vpu_helper_valid_frame_width(struct vpu_inst *inst, u32 width); u32 vpu_helper_valid_frame_height(struct vpu_inst *inst, u32 height);
drivers/media/platform/amphion/vpu_malone.c +0 -0
index 51e0702f9ae1..69d9a2269fce 100644
@@ -583,7 +583,8 @@ bool vpu_malone_check_fmt(enum vpu_core_type

type, u32 pixelfmt)

drivers/media/platform/amphion/vpu_malone.c bool vpu_malone_check_fmt(enum vpu_core_type type, u32 pixelfmt)

Quoted 4 lines

drivers/media/platform/amphion/vpu_malone.c +0 -1

      if (!vpu_imx8q_check_fmt(type, pixelfmt))
              return false;
 
-     if (pixelfmt == V4L2_PIX_FMT_NV12M_8L128 || pixelfmt ==

V4L2_PIX_FMT_NV12M_10BE_8L128)

drivers/media/platform/amphion/vpu_malone.c bool vpu_malone_check_fmt(enum vpu_core_type · line 586

Quoted 1 line

drivers/media/platform/amphion/vpu_malone.c +1 -0
+     if (pixelfmt == V4L2_PIX_FMT_NV12_8L128 || pixelfmt ==

V4L2_PIX_FMT_NV12_10BE_8L128 ||

Quoted 36 lines

drivers/media/platform/amphion/vpu_malone.c +2 -0

+         pixelfmt == V4L2_PIX_FMT_NV12M_8L128 || pixelfmt ==
+ V4L2_PIX_FMT_NV12M_10BE_8L128)
              return true;
      if (vpu_malone_format_remap(pixelfmt) == MALONE_FMT_NULL)
              return false;

drivers/media/platform/amphion/vpu_v4l2.c +17 -2

index 8a3eed957ae6..87cce552e906 100644
@@ -140,51 +140,135 @@ bool vpu_is_source_empty(struct vpu_inst *inst)
      return true;
 }
 
-const struct vpu_format *vpu_try_fmt_common(struct vpu_inst *inst,
 struct v4l2_format *f)
+static int vpu_init_format(struct vpu_inst *inst, struct vpu_format
+*fmt) {
+     const struct vpu_format *info;
+
+     info = vpu_helper_find_format(inst, fmt->type, fmt->pixfmt);
+     if (!info) {
+             info = vpu_helper_enum_format(inst, fmt->type, 0);
+             if (!info)
+                     return -EINVAL;
+     }
+     memcpy(fmt, info, sizeof(*fmt));
+
+     return 0;
+}
+
+static int vpu_calc_fmt_bytesperline(struct v4l2_format *f, struct
+vpu_format *fmt)
 {
      struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
-     u32 type = f->type;

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 172

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +10 -0

+     int i;
+
+     if (fmt->flags & V4L2_FMT_FLAG_COMPRESSED) {
+             for (i = 0; i < fmt->comp_planes; i++)
+                     fmt->bytesperline[i] = 0;
+             return 0;
+     }
+     if (pixmp->num_planes == fmt->comp_planes) {
+             for (i = 0; i < fmt->comp_planes; i++)
+                     fmt->bytesperline[i] =

pixmp->plane_fmt[i].bytesperline;

Quoted 32 lines

drivers/media/platform/amphion/vpu_v4l2.c +19 ~~-10~~

+             return 0;
+     }
+     if (pixmp->num_planes > 1)
+             return -EINVAL;
+
+     /*amphion vpu only support nv12 and nv12 tiled,
+      * so the bytesperline of luma and chroma should be same
+      */
+     for (i = 0; i < fmt->comp_planes; i++)
+             fmt->bytesperline[i] = pixmp->plane_fmt[0].bytesperline;
+
+     return 0;
+}
+
+static int vpu_calc_fmt_sizeimage(struct vpu_inst *inst, struct
+vpu_format *fmt) {
      u32 stride = 1;
-     u32 bytesperline;
-     u32 sizeimage;
-     const struct vpu_format *fmt;
-     const struct vpu_core_resources *res;
      int i;
 
-     fmt = vpu_helper_find_format(inst, type, pixmp->pixelformat);
-     if (!fmt) {
-             fmt = vpu_helper_enum_format(inst, type, 0);
-             if (!fmt)
-                     return NULL;
-             pixmp->pixelformat = fmt->pixfmt;
+     if (!(fmt->flags & V4L2_FMT_FLAG_COMPRESSED)) {
+             const struct vpu_core_resources *res =
+ vpu_get_resource(inst);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -5

+
+             if (res)
+                     stride = res->stride;
      }
 
-     res = vpu_get_resource(inst);
-     if (res)
-             stride = res->stride;
-     if (pixmp->width)
-             pixmp->width = vpu_helper_valid_frame_width(inst,

pixmp->width);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 169

Quoted 2 lines

drivers/media/platform/amphion/vpu_v4l2.c +0 -2

-     if (pixmp->height)
-             pixmp->height = vpu_helper_valid_frame_height(inst,

pixmp->height);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 211

Quoted 3 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -0

+     for (i = 0; i < fmt->comp_planes; i++) {
+             fmt->sizeimage[i] = vpu_helper_get_plane_size(fmt->pixfmt,
+

fmt->width,

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 203

Quoted 1 line

drivers/media/platform/amphion/vpu_v4l2.c +1 -0
+

fmt->height,

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 211

Quoted 3 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -0

+                                                           i,
+                                                           stride,
+

fmt->field != V4L2_FIELD_NONE ? 1 : 0,

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 207

Quoted 1 line

drivers/media/platform/amphion/vpu_v4l2.c +1 -0
+

&fmt->bytesperline[i]);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 3 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -0

+             fmt->sizeimage[i] = max_t(u32, fmt->sizeimage[i], PAGE_SIZE);
+             if (fmt->flags & V4L2_FMT_FLAG_COMPRESSED) {
+                     fmt->sizeimage[i] = clamp_val(fmt->sizeimage[i],

SZ_128K, SZ_8M);

Quoted 31 lines

drivers/media/platform/amphion/vpu_v4l2.c +31 -0

+                     fmt->bytesperline[i] = 0;
+             }
+     }
+
+     return 0;
+}
+
+u32 vpu_get_fmt_plane_size(struct vpu_format *fmt, u32 plane_no) {
+     u32 size;
+     int i;
+
+     if (plane_no >= fmt->mem_planes)
+             return 0;
+
+     if (fmt->comp_planes == fmt->mem_planes)
+             return fmt->sizeimage[plane_no];
+     if (plane_no < fmt->mem_planes - 1)
+             return fmt->sizeimage[plane_no];
+     size = fmt->sizeimage[plane_no];
+     for (i = fmt->mem_planes; i < fmt->comp_planes; i++)
+             size += fmt->sizeimage[i];
+
+     return size;
+}
+
+int vpu_try_fmt_common(struct vpu_inst *inst, struct v4l2_format *f,
+struct vpu_format *fmt) {
+     struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
+     int i;
+     int ret;
+

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +10 -0

+     fmt->pixfmt = pixmp->pixelformat;
+     fmt->type = f->type;
+     ret = vpu_init_format(inst, fmt);
+     if (ret < 0)
+             return ret;
+
+     fmt->width = pixmp->width;
+     fmt->height = pixmp->height;
+     if (fmt->width)
+             fmt->width = vpu_helper_valid_frame_width(inst,

fmt->width);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 253

Quoted 2 lines

drivers/media/platform/amphion/vpu_v4l2.c +2 -0

+     if (fmt->height)
+             fmt->height = vpu_helper_valid_frame_height(inst,

fmt->height);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 254

Quoted 1 line

drivers/media/platform/amphion/vpu_v4l2.c +1 -0
+     fmt->field = pixmp->field == V4L2_FIELD_ANY ? V4L2_FIELD_NONE :

pixmp->field;

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 3 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -0

+     vpu_calc_fmt_bytesperline(f, fmt);
+     vpu_calc_fmt_sizeimage(inst, fmt);
+     if ((fmt->flags & V4L2_FMT_FLAG_COMPRESSED) &&

pixmp->plane_fmt[0].sizeimage)

Quoted 5 lines

drivers/media/platform/amphion/vpu_v4l2.c +5 -0

+             fmt->sizeimage[0] =
+ clamp_val(pixmp->plane_fmt[0].sizeimage, SZ_128K, SZ_8M);
+
+     pixmp->pixelformat = fmt->pixfmt;
+     pixmp->width = fmt->width;

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +4 -4

+     pixmp->height = fmt->height;
      pixmp->flags = fmt->flags;
-     pixmp->num_planes = fmt->num_planes;
-     if (pixmp->field == V4L2_FIELD_ANY)
-             pixmp->field = V4L2_FIELD_NONE;
+     pixmp->num_planes = fmt->mem_planes;
+     pixmp->field = fmt->field;
+     memset(pixmp->reserved, 0, sizeof(pixmp->reserved));
      for (i = 0; i < pixmp->num_planes; i++) {
-             bytesperline = max_t(s32, pixmp->plane_fmt[i].bytesperline,

0);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 6 lines

drivers/media/platform/amphion/vpu_v4l2.c +0 -6

-             sizeimage = vpu_helper_get_plane_size(pixmp->pixelformat,
-                                                   pixmp->width,
-                                                   pixmp->height,
-                                                   i,
-                                                   stride,
-                                                   pixmp->field >

V4L2_FIELD_NONE ? 1 : 0,

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 183

Quoted 2 lines

drivers/media/platform/amphion/vpu_v4l2.c +0 -2

-                                                   &bytesperline);
-             sizeimage = max_t(s32, pixmp->plane_fmt[i].sizeimage,

sizeimage);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 4 lines

drivers/media/platform/amphion/vpu_v4l2.c +2 -2

-             pixmp->plane_fmt[i].bytesperline = bytesperline;
-             pixmp->plane_fmt[i].sizeimage = sizeimage;
+             pixmp->plane_fmt[i].bytesperline = fmt->bytesperline[i];
+             pixmp->plane_fmt[i].sizeimage = vpu_get_fmt_plane_size(fmt,

i);

drivers/media/platform/amphion/vpu_v4l2.c static int vpu_vb2_queue_setup(struct vb2_queue

Quoted 25 lines

drivers/media/platform/amphion/vpu_v4l2.c +7 -5

+             memset(pixmp->plane_fmt[i].reserved, 0,
+ sizeof(pixmp->plane_fmt[i].reserved));
      }
 
-     return fmt;
+     return 0;
 }
 
 static bool vpu_check_ready(struct vpu_inst *inst, u32 type) @@
-389,10 +473,10 @@ static int vpu_vb2_queue_setup(struct vb2_queue *vq,
      cur_fmt = vpu_get_format(inst, vq->type);
 
      if (*plane_count) {
-             if (*plane_count != cur_fmt->num_planes)
+             if (*plane_count != cur_fmt->mem_planes)
                      return -EINVAL;
-             for (i = 0; i < cur_fmt->num_planes; i++) {
-                     if (psize[i] < cur_fmt->sizeimage[i])
+             for (i = 0; i < cur_fmt->mem_planes; i++) {
+                     if (psize[i] < vpu_get_fmt_plane_size(cur_fmt,
+ i))
                              return -EINVAL;
              }
              return 0;

@@ -402,9 +486,9 @@ static int vpu_vb2_queue_setup(struct vb2_queue

*vq,

drivers/media/platform/amphion/vpu_v4l2.c static int vpu_vb2_queue_setup(struct vb2_queue · line 486

Quoted 1 line

drivers/media/platform/amphion/vpu_v4l2.c +0 -0
              *buf_count = max_t(unsigned int, *buf_count,

inst->min_buffer_out);

drivers/media/platform/amphion/vpu_v4l2.c static int vpu_vb2_queue_setup(struct vb2_queue · line 488

Quoted 2 lines

drivers/media/platform/amphion/vpu_v4l2.c +0 -0

      else
              *buf_count = max_t(unsigned int, *buf_count,

inst->min_buffer_cap);

drivers/media/platform/amphion/vpu_v4l2.c static int vpu_vb2_buf_prepare(struct vb2_buffer

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -3

-     *plane_count = cur_fmt->num_planes;
-     for (i = 0; i < cur_fmt->num_planes; i++)
-             psize[i] = cur_fmt->sizeimage[i];
+     *plane_count = cur_fmt->mem_planes;
+     for (i = 0; i < cur_fmt->mem_planes; i++)
+             psize[i] = vpu_get_fmt_plane_size(cur_fmt, i);
 
      return 0;
 }

@@ -434,8 +518,8 @@ static int vpu_vb2_buf_prepare(struct vb2_buffer

*vb)

drivers/media/platform/amphion/vpu_v4l2.c static int vpu_vb2_buf_prepare(struct vb2_buffer *vb)

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -2

      u32 i;
 
      cur_fmt = vpu_get_format(inst, vb->type);
-     for (i = 0; i < cur_fmt->num_planes; i++) {
-             if (vpu_get_vb_length(vb, i) < cur_fmt->sizeimage[i]) {
+     for (i = 0; i < cur_fmt->mem_planes; i++) {
+             if (vpu_get_vb_length(vb, i) <
+ vpu_get_fmt_plane_size(cur_fmt, i)) {
                      dev_dbg(inst->dev, "[%d] %s buf[%d] is invalid\n",
                              inst->id, vpu_type_name(vb->type),

vb->index);

Quoted 1 line

vpu_set_buffer_state(vbuf,

VPU_BUF_STATE_ERROR);

drivers/media/platform/amphion/vpu_windsor.c static int vpu_windsor_fill_yuv_frame(struct

Quoted 23 lines

drivers/media/platform/amphion/vpu_v4l2.h +0 -0
index 795ca33a6a50..ef5de6b66e47 100644
@@ -16,7 +16,8 @@ unsigned int vpu_get_buffer_state(struct
vb2_v4l2_buffer *vbuf); int vpu_v4l2_open(struct file *file, struct vpu_inst *inst); int vpu_v4l2_close(struct file *file);
-const struct vpu_format *vpu_try_fmt_common(struct vpu_inst *inst, struct v4l2_format *f);
drivers/media/platform/amphion/vpu_v4l2.h +3 -0
+u32 vpu_get_fmt_plane_size(struct vpu_format *fmt, u32 plane_no); int
+vpu_try_fmt_common(struct vpu_inst *inst, struct v4l2_format *f,
+struct vpu_format *fmt);
 int vpu_process_output_buffer(struct vpu_inst *inst);  int
vpu_process_capture_buffer(struct vpu_inst *inst); struct vb2_v4l2_buffer *vpu_next_src_buf(struct vpu_inst *inst); diff --git a/drivers/media/platform/amphion/vpu_windsor.c b/drivers/media/platform/amphion/vpu_windsor.c
index 1526af2ef9da..a454f142ae17 100644
drivers/media/platform/amphion/vpu_windsor.c +0 -0
@@ -775,6 +775,8 @@ static int vpu_windsor_fill_yuv_frame(struct

vpu_shared_addr *shared,

drivers/media/platform/amphion/vpu_windsor.c static int vpu_windsor_fill_yuv_frame(struct

Quoted 8 lines

drivers/media/platform/amphion/vpu_windsor.c +2 -0

                                    u32 instance,
                                    struct vb2_buffer *vb)  {
+     struct vpu_inst *inst = vb2_get_drv_priv(vb->vb2_queue);
+     struct vpu_format *out_fmt;
      struct vpu_enc_yuv_desc *desc;
      struct vb2_v4l2_buffer *vbuf;

@@ -782,6 +784,7 @@ static int vpu_windsor_fill_yuv_frame(struct

vpu_shared_addr *shared,

Quoted 21 lines

drivers/media/platform/amphion/vpu_windsor.c +1 -0

              return -EINVAL;
 
      desc = get_yuv_desc(shared, instance);
+     out_fmt = vpu_get_format(inst, vb->type);
 
      vbuf = to_vb2_v4l2_buffer(vb);
      desc->frame_id = vbuf->sequence; @@ -790,7 +793,10 @@ static int

vpu_windsor_fill_yuv_frame(struct vpu_shared_addr *shared,

      else
              desc->key_frame = 0;
      desc->luma_base = vpu_get_vb_phy_addr(vb, 0);

drivers/media/platform/amphion/vpu_windsor.c +5 -2

-     desc->chroma_base = vpu_get_vb_phy_addr(vb, 1);
+     if (vb->num_planes > 1)
+             desc->chroma_base = vpu_get_vb_phy_addr(vb, 1);
+     else
+             desc->chroma_base = desc->luma_base +
+ out_fmt->sizeimage[0];
 
      return 0;
 }
--

Quoted 2 lines

2.37.1

--
Tommaso Merciai
Embedded Linux Engineer
[email protected]
__________________________________

Signature · 10 lines

Tommaso Merciai
Embedded Linux Engineer
[email protected]
__________________________________

Amarula Solutions SRL
Via Le Canevare 30, 31100 Treviso, Veneto, IT
T. +39 042 243 5310
[email protected]
www.amarulasolutions.com

Ming Qian

author maintainer

2022-08-12 23:26 UTC #11 lore raw

RE: [EXT] Re: [PATCH v4 2/4] media: amphion: tell and handle contiguous and non contiguous format

In reply to Tommaso Merciai

Hi Tommaso,

I'm sorry that I didn't notice this patch set is dependent on the following patch:

85069 New media: amphion: don't change the colorspace reported by decoder.

Ming

Quoted 1,049 lines

Hi Ming,
Just an info. I'm planning to help to review your series. On which Linux tree are you working?

Thanks in advance,
Tommaso

drivers/media/platform/amphion/vdec.c static int vdec_enum_fmt(struct file *file, void *fh,

Quoted 145 lines · Ming Qian

Driver should tell the number of memory planes and component planes. the amphion vpu support non contiguous planes, but for compatibility with other device that only support contiguous planes. driver can add support for contiguous planes in the same time. Then the mem_planes can be different from the comp_planes. driver need to handle buffer according mem_planes and comp_planes.

So driver can support NV12 and NV12M.

Signed-off-by Ming Qian <[email protected]>

---
 drivers/media/platform/amphion/vdec.c        | 187 ++++++++++---------
 drivers/media/platform/amphion/venc.c        |  33 ++--
 drivers/media/platform/amphion/vpu.h         |   4 +-
 drivers/media/platform/amphion/vpu_dbg.c     |   8 +-
 drivers/media/platform/amphion/vpu_helpers.c |  48 ++++-
 drivers/media/platform/amphion/vpu_helpers.h |   2 +
 drivers/media/platform/amphion/vpu_malone.c  |   3 +-
 drivers/media/platform/amphion/vpu_v4l2.c    | 168 ++++++++++++-----
 drivers/media/platform/amphion/vpu_v4l2.h    |   3 +-
 drivers/media/platform/amphion/vpu_windsor.c |   8 +-
 10 files changed, 299 insertions(+), 165 deletions(-)

drivers/media/platform/amphion/vdec.c +43 ~~-21~~

index feb75dc204de..48ab664fa7ef 100644
@@ -69,72 +69,85 @@ struct vdec_t {
 static const struct vpu_format vdec_formats[] = {
      {
              .pixfmt = V4L2_PIX_FMT_NV12M_8L128,
-             .num_planes = 2,
+             .mem_planes = 2,
+             .comp_planes = 2,
              .type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
      },
      {
              .pixfmt = V4L2_PIX_FMT_NV12M_10BE_8L128,
-             .num_planes = 2,
+             .mem_planes = 2,
+             .comp_planes = 2,
              .type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
      },
      {
              .pixfmt = V4L2_PIX_FMT_H264,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_H264_MVC,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_HEVC,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_VC1_ANNEX_G,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_VC1_ANNEX_L,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+             .flags = V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_MPEG2,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_MPEG4,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_XVID,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_VP8,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_H263,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {0, 0, 0, 0},
 };

@@ -256,23 +269,22 @@ static int vdec_enum_fmt(struct file *file, void *fh,

struct v4l2_fmtdesc *f)

drivers/media/platform/amphion/vdec.c static int vdec_enum_fmt(struct file *file, void *fh, struct v4l2_fmtdesc *f)

Quoted 13 lines

drivers/media/platform/amphion/vdec.c +4 -6

      int ret = -EINVAL;
 
      vpu_inst_lock(inst);
-     if (!V4L2_TYPE_IS_OUTPUT(f->type) && vdec->fixed_fmt) {
-             if (f->index == 0) {
-                     f->pixelformat = inst->cap_format.pixfmt;
-                     f->flags = inst->cap_format.flags;
-                     ret = 0;
-             }
+     if (V4L2_TYPE_IS_CAPTURE(f->type) && vdec->fixed_fmt) {
+             fmt = vpu_get_format(inst, f->type);
+             if (f->index == 1)
+                     fmt = vpu_helper_find_sibling(inst, f->type,

fmt->pixfmt);

drivers/media/platform/amphion/vdec.c static int vdec_g_fmt(struct file *file, void *fh,

Quoted 23 lines

drivers/media/platform/amphion/vdec.c +8 -7

+             if (f->index > 1)
+                     fmt = NULL;
      } else {
              fmt = vpu_helper_enum_format(inst, f->type, f->index);
-             memset(f->reserved, 0, sizeof(f->reserved));
-             if (!fmt)
-                     goto exit;
-
-             f->pixelformat = fmt->pixfmt;
-             f->flags = fmt->flags;
-             ret = 0;
      }
+     if (!fmt)
+             goto exit;
 
+     memset(f->reserved, 0, sizeof(f->reserved));
+     f->pixelformat = fmt->pixfmt;
+     f->flags = fmt->flags;
+     ret = 0;
 exit:
      vpu_inst_unlock(inst);
      return ret;

@@ -289,14 +301,14 @@ static int vdec_g_fmt(struct file *file, void *fh,

struct v4l2_format *f)

Quoted 21 lines

drivers/media/platform/amphion/vdec.c +3 -3

      cur_fmt = vpu_get_format(inst, f->type);
 
      pixmp->pixelformat = cur_fmt->pixfmt;
-     pixmp->num_planes = cur_fmt->num_planes;
+     pixmp->num_planes = cur_fmt->mem_planes;
      pixmp->width = cur_fmt->width;
      pixmp->height = cur_fmt->height;
      pixmp->field = cur_fmt->field;
      pixmp->flags = cur_fmt->flags;
      for (i = 0; i < pixmp->num_planes; i++) {
              pixmp->plane_fmt[i].bytesperline = cur_fmt->bytesperline[i];
-             pixmp->plane_fmt[i].sizeimage = cur_fmt->sizeimage[i];
+             pixmp->plane_fmt[i].sizeimage =
+ vpu_get_fmt_plane_size(cur_fmt, i);
      }
 
      f->fmt.pix_mp.colorspace = vdec->codec_info.color_primaries; @@
-311,10 +323,19 @@ static int vdec_try_fmt(struct file *file, void

*fh, struct v4l2_format *f)  {
      struct vpu_inst *inst = to_inst(file);
      struct vdec_t *vdec = inst->priv;

drivers/media/platform/amphion/vdec.c static int vdec_try_fmt(struct file *file, void *fh, struct v4l2_format *f)

Quoted 10 lines

drivers/media/platform/amphion/vdec.c +6 -2

-
-     vpu_try_fmt_common(inst, f);
+     struct vpu_format fmt;
 
      vpu_inst_lock(inst);
+     if (V4L2_TYPE_IS_CAPTURE(f->type) && vdec->fixed_fmt) {
+             struct vpu_format *cap_fmt = vpu_get_format(inst,
+ f->type);
+
+             if (!vpu_helper_match_format(inst, cap_fmt->type,

cap_fmt->pixfmt,

drivers/media/platform/amphion/vdec.c static int vdec_try_fmt(struct file *file, void *fh, struct v4l2_format *f)

Quoted 8 lines

drivers/media/platform/amphion/vdec.c +6 -0

+                                          f->fmt.pix_mp.pixelformat))
+                     f->fmt.pix_mp.pixelformat = cap_fmt->pixfmt;
+     }
+
+     vpu_try_fmt_common(inst, f, &fmt);
+
      if (vdec->fixed_fmt) {
              f->fmt.pix_mp.colorspace =

vdec->codec_info.color_primaries;

Quoted 14 lines

              f->fmt.pix_mp.xfer_func =
vdec->codec_info.transfer_chars; @@ -334,7 +355,7 @@ static int
vdec_try_fmt(struct file *file, void *fh, struct v4l2_format *f)
static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)  {
      struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;

drivers/media/platform/amphion/vdec.c +1 -1

-     const struct vpu_format *fmt;
+     struct vpu_format fmt;
      struct vpu_format *cur_fmt;
      struct vb2_queue *q;
      struct vdec_t *vdec = inst->priv; @@ -349,36 +370,30 @@ static

int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)

      if (vb2_is_busy(q))
              return -EBUSY;

drivers/media/platform/amphion/vdec.c static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)

Quoted 7 lines

drivers/media/platform/amphion/vdec.c +1 -2

-     fmt = vpu_try_fmt_common(inst, f);
-     if (!fmt)
+     if (vpu_try_fmt_common(inst, f, &fmt))
              return -EINVAL;
 
      cur_fmt = vpu_get_format(inst, f->type);
      if (V4L2_TYPE_IS_OUTPUT(f->type) && inst->state !=

VPU_CODEC_STATE_DEINIT) {

Quoted 4 lines

drivers/media/platform/amphion/vdec.c +1 -1

-             if (cur_fmt->pixfmt != fmt->pixfmt) {
+             if (cur_fmt->pixfmt != fmt.pixfmt) {
                      vdec->reset_codec = true;
                      vdec->fixed_fmt = false;

drivers/media/platform/amphion/vdec.c static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)

Quoted 10 lines

drivers/media/platform/amphion/vdec.c +0 -7

              }
      }
-     cur_fmt->pixfmt = fmt->pixfmt;
      if (V4L2_TYPE_IS_OUTPUT(f->type) || !vdec->fixed_fmt) {
-             cur_fmt->num_planes = fmt->num_planes;
-             cur_fmt->flags = fmt->flags;
-             cur_fmt->width = pixmp->width;
-             cur_fmt->height = pixmp->height;
-             for (i = 0; i < fmt->num_planes; i++) {
-                     cur_fmt->sizeimage[i] =

pixmp->plane_fmt[i].sizeimage;

Quoted 1 line

- cur_fmt->bytesperline[i] =

pixmp->plane_fmt[i].bytesperline;

drivers/media/platform/amphion/vdec.c static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)

Quoted 7 lines

drivers/media/platform/amphion/vdec.c +2 -4

-             }
-             if (pixmp->field != V4L2_FIELD_ANY)
-                     cur_fmt->field = pixmp->field;
+             memcpy(cur_fmt, &fmt, sizeof(*cur_fmt));
      } else {
-             pixmp->num_planes = cur_fmt->num_planes;
+             if (vpu_helper_match_format(inst, f->type, cur_fmt->pixfmt,

pixmp->pixelformat)) {

drivers/media/platform/amphion/vdec.c static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)

Quoted 9 lines

drivers/media/platform/amphion/vdec.c +5 -0

+                     cur_fmt->pixfmt = fmt.pixfmt;
+                     cur_fmt->mem_planes = fmt.mem_planes;
+             }
+             pixmp->pixelformat = cur_fmt->pixfmt;
+             pixmp->num_planes = cur_fmt->mem_planes;
              pixmp->width = cur_fmt->width;
              pixmp->height = cur_fmt->height;
              for (i = 0; i < pixmp->num_planes; i++) {
                      pixmp->plane_fmt[i].bytesperline =

cur_fmt->bytesperline[i];

Quoted 1 line

- pixmp->plane_fmt[i].sizeimage =

cur_fmt->sizeimage[i];

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst,

Quoted 18 lines

drivers/media/platform/amphion/vdec.c +4 -0

+                     pixmp->plane_fmt[i].sizeimage =
+ vpu_get_fmt_plane_size(cur_fmt, i);
              }
              pixmp->field = cur_fmt->field;
      }

@@ -678,9 +693,11 @@ static struct vpu_vb2_buffer
 *vdec_find_buffer(struct vpu_inst *inst, u32 luma)  static void
 vdec_buf_done(struct vpu_inst *inst, struct vpu_frame_info *frame)  {
      struct vdec_t *vdec = inst->priv;
+     struct vpu_format *cur_fmt;
      struct vpu_vb2_buffer *vpu_buf;
      struct vb2_v4l2_buffer *vbuf;
      u32 sequence;
+     int i;
 
      if (!frame)
              return;

@@ -699,6 +716,7 @@ static void vdec_buf_done(struct vpu_inst *inst,

struct vpu_frame_info *frame)

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst,

Quoted 8 lines

drivers/media/platform/amphion/vdec.c +1 -0

              return;
      }
 
+     cur_fmt = vpu_get_format(inst, inst->cap_format.type);
      vbuf = &vpu_buf->m2m_buf.vb;
      if (vbuf->vb2_buf.index != frame->id)
              dev_err(inst->dev, "[%d] buffer id(%d, %d) dismatch\n",

@@ -707,9 +725,9 @@ static void vdec_buf_done(struct vpu_inst *inst,

struct vpu_frame_info *frame)

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst, · line 726

Quoted 2 lines

drivers/media/platform/amphion/vdec.c +0 -0

      if (vpu_get_buffer_state(vbuf) != VPU_BUF_STATE_DECODED)
              dev_err(inst->dev, "[%d] buffer(%d) ready without

decoded\n", inst->id, frame->id);

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst, · line 710

Quoted 2 lines

drivers/media/platform/amphion/vdec.c +0 -1

      vpu_set_buffer_state(vbuf, VPU_BUF_STATE_READY);
-     vb2_set_plane_payload(&vbuf->vb2_buf, 0,

inst->cap_format.sizeimage[0]);

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst, · line 711

Quoted 1 line

drivers/media/platform/amphion/vdec.c +0 -1
-     vb2_set_plane_payload(&vbuf->vb2_buf, 1,

inst->cap_format.sizeimage[1]);

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst, struct vpu_frame_info *frame)

Quoted 3 lines

drivers/media/platform/amphion/vdec.c +2 -1

-     vbuf->field = inst->cap_format.field;
+     for (i = 0; i < vbuf->vb2_buf.num_planes; i++)
+             vb2_set_plane_payload(&vbuf->vb2_buf, i,

vpu_get_fmt_plane_size(cur_fmt, i));

Quoted 6 lines

drivers/media/platform/amphion/vdec.c +1 -0

+     vbuf->field = cur_fmt->field;
      vbuf->sequence = sequence;
      dev_dbg(inst->dev, "[%d][OUTPUT TS]%32lld\n", inst->id,
 vbuf->vb2_buf.timestamp);

drivers/media/platform/amphion/vdec.c static void vdec_stop_done(struct vpu_inst · line 772

Quoted 10 lines

drivers/media/platform/amphion/vdec.c +2 -2

@@ -747,15 +765,17 @@ static void vdec_stop_done(struct vpu_inst
 *inst)  static bool vdec_check_source_change(struct vpu_inst *inst)  {
      struct vdec_t *vdec = inst->priv;
-     const struct vpu_format *fmt;
-     int i;
+     const struct vpu_format *sibling;
 
      if (!inst->fh.m2m_ctx)
              return false;
 
+     sibling = vpu_helper_find_sibling(inst, inst->cap_format.type,

inst->cap_format.pixfmt);

drivers/media/platform/amphion/vdec.c static void vdec_stop_done(struct vpu_inst *inst)

Quoted 6 lines

drivers/media/platform/amphion/vdec.c +3 -1

+     if (sibling && vdec->codec_info.pixfmt == sibling->pixfmt)
+             vdec->codec_info.pixfmt = inst->cap_format.pixfmt;
+
      if (!vb2_is_streaming(v4l2_m2m_get_dst_vq(inst->fh.m2m_ctx)))
              return true;
-     fmt = vpu_helper_find_format(inst, inst->cap_format.type,

vdec->codec_info.pixfmt);

drivers/media/platform/amphion/vdec.c static void vdec_stop_done(struct vpu_inst *inst)

Quoted 4 lines

drivers/media/platform/amphion/vdec.c +0 -1

      if (inst->cap_format.pixfmt != vdec->codec_info.pixfmt)
              return true;
      if (inst->cap_format.width != vdec->codec_info.decoded_width) @@
-772,14 +792,6 @@ static bool vdec_check_source_change(struct vpu_inst

*inst)

drivers/media/platform/amphion/vdec.c static bool vdec_check_source_change(struct vpu_inst *inst)

Quoted 7 lines

drivers/media/platform/amphion/vdec.c +0 -4

              return true;
      if (inst->crop.height != vdec->codec_info.height)
              return true;
-     if (fmt && inst->cap_format.num_planes != fmt->num_planes)
-             return true;
-     for (i = 0; i < inst->cap_format.num_planes; i++) {
-             if (inst->cap_format.bytesperline[i] !=

vdec->codec_info.bytesperline[i])

drivers/media/platform/amphion/vdec.c

Quoted 2 lines

drivers/media/platform/amphion/vdec.c +0 -2

-                     return true;
-             if (inst->cap_format.sizeimage[i] !=

vdec->codec_info.sizeimage[i])

drivers/media/platform/amphion/vdec.c static bool vdec_check_source_change(struct · line 792

Quoted 13 lines

drivers/media/platform/amphion/vdec.c +1 -5

-                     return true;
-     }
 
      return false;
 }

@@ -787,27 +799,21 @@ static bool vdec_check_source_change(struct
 vpu_inst *inst)  static void vdec_init_fmt(struct vpu_inst *inst)  {
      struct vdec_t *vdec = inst->priv;
-     const struct vpu_format *fmt;
-     int i;
+     struct v4l2_format f;
 
-     fmt = vpu_helper_find_format(inst, inst->cap_format.type,

vdec->codec_info.pixfmt);

drivers/media/platform/amphion/vdec.c static bool vdec_check_source_change(struct vpu_inst *inst)

Quoted 11 lines

drivers/media/platform/amphion/vdec.c +0 ~~-11~~

-     inst->out_format.width = vdec->codec_info.width;
-     inst->out_format.height = vdec->codec_info.height;
-     inst->cap_format.width = vdec->codec_info.decoded_width;
-     inst->cap_format.height = vdec->codec_info.decoded_height;
-     inst->cap_format.pixfmt = vdec->codec_info.pixfmt;
-     if (fmt) {
-             inst->cap_format.num_planes = fmt->num_planes;
-             inst->cap_format.flags = fmt->flags;
-     }
-     for (i = 0; i < inst->cap_format.num_planes; i++) {
-             inst->cap_format.bytesperline[i] =

vdec->codec_info.bytesperline[i];

Quoted 20 lines

drivers/media/platform/amphion/vdec.c +11 -4

-             inst->cap_format.sizeimage[i] = vdec->codec_info.sizeimage[i];
-     }
+     memset(&f, 0, sizeof(f));
+     f.type = inst->cap_format.type;
+     f.fmt.pix_mp.pixelformat = vdec->codec_info.pixfmt;
+     f.fmt.pix_mp.width = vdec->codec_info.decoded_width;
+     f.fmt.pix_mp.height = vdec->codec_info.decoded_height;
      if (vdec->codec_info.progressive)
-             inst->cap_format.field = V4L2_FIELD_NONE;
+             f.fmt.pix_mp.field = V4L2_FIELD_NONE;
      else
-             inst->cap_format.field = V4L2_FIELD_SEQ_TB;
+             f.fmt.pix_mp.field = V4L2_FIELD_SEQ_TB;
+     vpu_try_fmt_common(inst, &f, &inst->cap_format);
+
+     inst->out_format.width = vdec->codec_info.width;
+     inst->out_format.height = vdec->codec_info.height;
 }
 
 static void vdec_init_crop(struct vpu_inst *inst) @@ -966,7 +972,10

Quoted 1 line

@@ static int vdec_response_frame(struct vpu_inst *inst, struct

vb2_v4l2_buffer *vb

drivers/media/platform/amphion/vdec.c static int vdec_response_frame(struct vpu_inst *inst, struct vb2_v4l2_buffer *vb

Quoted 6 lines

      info.tag = vdec->seq_tag;
      info.luma_addr = vpu_get_vb_phy_addr(&vbuf->vb2_buf, 0);
      info.luma_size = inst->cap_format.sizeimage[0];

drivers/media/platform/amphion/vdec.c +2 -1

-     info.chroma_addr = vpu_get_vb_phy_addr(&vbuf->vb2_buf, 1);
+     if (vbuf->vb2_buf.num_planes > 1)
+             info.chroma_addr = vpu_get_vb_phy_addr(&vbuf->vb2_buf,

1);

drivers/media/platform/amphion/venc.c static int venc_g_fmt(struct file *file, void *fh,

Quoted 40 lines

drivers/media/platform/amphion/vdec.c +2 -0

+     else
+             info.chroma_addr = info.luma_addr + info.luma_size;
      info.chromau_size = inst->cap_format.sizeimage[1];
      info.bytesperline = inst->cap_format.bytesperline[0];
      ret = vpu_session_alloc_fs(inst, &info); @@ -975,7 +984,7 @@

static int vdec_response_frame(struct vpu_inst *inst, struct vb2_v4l2_buffer *vb

      vpu_buf->tag = info.tag;
      vpu_buf->luma = info.luma_addr;

drivers/media/platform/amphion/vdec.c +1 -1

-     vpu_buf->chroma_u = info.chromau_size;
+     vpu_buf->chroma_u = info.chroma_addr;
      vpu_buf->chroma_v = 0;
      vpu_set_buffer_state(vbuf, VPU_BUF_STATE_INUSE);
      vdec->slots[info.id] = vpu_buf;

drivers/media/platform/amphion/venc.c +5 -2

index 37212f087fdd..060a1ee78b17 100644
@@ -69,13 +69,16 @@ struct venc_frame_t {  static const struct
 vpu_format venc_formats[] = {
      {
              .pixfmt = V4L2_PIX_FMT_NV12M,
-             .num_planes = 2,
+             .mem_planes = 2,
+             .comp_planes = 2,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
      },
      {
              .pixfmt = V4L2_PIX_FMT_H264,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+             .flags = V4L2_FMT_FLAG_COMPRESSED
      },
      {0, 0, 0, 0},
 };

@@ -173,14 +176,14 @@ static int venc_g_fmt(struct file *file, void *fh,

struct v4l2_format *f)

drivers/media/platform/amphion/venc.c static int venc_s_fmt(struct file *file, void *fh,

Quoted 43 lines

drivers/media/platform/amphion/venc.c +3 -3

      cur_fmt = vpu_get_format(inst, f->type);
 
      pixmp->pixelformat = cur_fmt->pixfmt;
-     pixmp->num_planes = cur_fmt->num_planes;
+     pixmp->num_planes = cur_fmt->mem_planes;
      pixmp->width = cur_fmt->width;
      pixmp->height = cur_fmt->height;
      pixmp->field = cur_fmt->field;
      pixmp->flags = cur_fmt->flags;
      for (i = 0; i < pixmp->num_planes; i++) {
              pixmp->plane_fmt[i].bytesperline = cur_fmt->bytesperline[i];
-             pixmp->plane_fmt[i].sizeimage = cur_fmt->sizeimage[i];
+             pixmp->plane_fmt[i].sizeimage =
+ vpu_get_fmt_plane_size(cur_fmt, i);
      }
 
      f->fmt.pix_mp.colorspace = venc->params.color.primaries; @@
-194,8 +197,9 @@ static int venc_g_fmt(struct file *file, void *fh,

struct v4l2_format *f) static int venc_try_fmt(struct file *file,

void *fh, struct v4l2_format *f)  {
      struct vpu_inst *inst = to_inst(file);

drivers/media/platform/amphion/venc.c +3 -3

+     struct vpu_format fmt;
 
-     vpu_try_fmt_common(inst, f);
+     vpu_try_fmt_common(inst, f, &fmt);
 
      return 0;
 }

@@ -203,12 +207,11 @@ static int venc_try_fmt(struct file *file, void
 *fh, struct v4l2_format *f)  static int venc_s_fmt(struct file *file,
 void *fh, struct v4l2_format *f)  {
      struct vpu_inst *inst = to_inst(file);
-     const struct vpu_format *fmt;
+     struct vpu_format fmt;
      struct vpu_format *cur_fmt;
      struct vb2_queue *q;
      struct venc_t *venc = inst->priv;
      struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
-     int i;
 
      q = v4l2_m2m_get_vq(inst->fh.m2m_ctx, f->type);
      if (!q)

@@ -216,24 +219,12 @@ static int venc_s_fmt(struct file *file, void *fh,

struct v4l2_format *f)

drivers/media/platform/amphion/vpu_dbg.c static int vpu_dbg_instance(struct seq_file *s, void

Quoted 55 lines

drivers/media/platform/amphion/venc.c +2 ~~-14~~

      if (vb2_is_busy(q))
              return -EBUSY;
 
-     fmt = vpu_try_fmt_common(inst, f);
-     if (!fmt)
+     if (vpu_try_fmt_common(inst, f, &fmt))
              return -EINVAL;
 
      cur_fmt = vpu_get_format(inst, f->type);
 
-     cur_fmt->pixfmt = fmt->pixfmt;
-     cur_fmt->num_planes = fmt->num_planes;
-     cur_fmt->flags = fmt->flags;
-     cur_fmt->width = pix_mp->width;
-     cur_fmt->height = pix_mp->height;
-     for (i = 0; i < fmt->num_planes; i++) {
-             cur_fmt->sizeimage[i] = pix_mp->plane_fmt[i].sizeimage;
-             cur_fmt->bytesperline[i] = pix_mp->plane_fmt[i].bytesperline;
-     }
-
-     if (pix_mp->field != V4L2_FIELD_ANY)
-             cur_fmt->field = pix_mp->field;
+     memcpy(cur_fmt, &fmt, sizeof(*cur_fmt));
 
      if (V4L2_TYPE_IS_OUTPUT(f->type)) {
              venc->params.input_format = cur_fmt->pixfmt; diff --git

a/drivers/media/platform/amphion/vpu.h
b/drivers/media/platform/amphion/vpu.h
index f914de6ed81e..998591fa5b85 100644

drivers/media/platform/amphion/vpu.h +3 -1

@@ -84,7 +84,8 @@ struct vpu_dev {
 
 struct vpu_format {
      u32 pixfmt;
-     unsigned int num_planes;
+     u32 mem_planes;
+     u32 comp_planes;
      u32 type;
      u32 flags;
      u32 width;

@@ -92,6 +93,7 @@ struct vpu_format {
      u32 sizeimage[VIDEO_MAX_PLANES];
      u32 bytesperline[VIDEO_MAX_PLANES];
      u32 field;
+     u32 sibling;
 };
 
 struct vpu_core_resources {

drivers/media/platform/amphion/vpu_dbg.c +0 -0

index f72c8a506b22..c41c9896912c 100644
@@ -89,9 +89,9 @@ static int vpu_dbg_instance(struct seq_file *s, void

*data)

drivers/media/platform/amphion/vpu_dbg.c static int vpu_dbg_instance(struct seq_file *s, void

Quoted 13 lines

drivers/media/platform/amphion/vpu_dbg.c +3 -2

                      vq->last_buffer_dequeued);
      if (seq_write(s, str, num))
              return 0;
-     for (i = 0; i < inst->out_format.num_planes; i++) {
+     for (i = 0; i < inst->out_format.mem_planes; i++) {
              num = scnprintf(str, sizeof(str), " %d(%d)",
-                             inst->out_format.sizeimage[i],
+
+ vpu_get_fmt_plane_size(&inst->out_format, i),
                              inst->out_format.bytesperline[i]);
              if (seq_write(s, str, num))
                      return 0;

@@ -113,9 +113,9 @@ static int vpu_dbg_instance(struct seq_file *s, void

*data)

drivers/media/platform/amphion/vpu_helpers.c const struct vpu_format

Quoted 18 lines

drivers/media/platform/amphion/vpu_dbg.c +3 -2

                      vq->last_buffer_dequeued);
      if (seq_write(s, str, num))
              return 0;
-     for (i = 0; i < inst->cap_format.num_planes; i++) {
+     for (i = 0; i < inst->cap_format.mem_planes; i++) {
              num = scnprintf(str, sizeof(str), " %d(%d)",
-                             inst->cap_format.sizeimage[i],
+
+ vpu_get_fmt_plane_size(&inst->cap_format, i),
                              inst->cap_format.bytesperline[i]);
              if (seq_write(s, str, num))
                      return 0;

drivers/media/platform/amphion/vpu_helpers.c +0 -0

index e9aeb3453dfc..db8f471b9187 100644
@@ -59,6 +59,39 @@ const struct vpu_format

*vpu_helper_find_format(struct vpu_inst *inst, u32 type,

drivers/media/platform/amphion/vpu_helpers.c static u32 get_nv12_plane_size(u32 width, u32

Quoted 40 lines

drivers/media/platform/amphion/vpu_helpers.c +33 -0

      return NULL;
 }
 
+const struct vpu_format *vpu_helper_find_sibling(struct vpu_inst
+*inst, u32 type, u32 pixelfmt) {
+     const struct vpu_format *fmt;
+     const struct vpu_format *sibling;
+
+     fmt = vpu_helper_find_format(inst, type, pixelfmt);
+     if (!fmt)
+             return NULL;
+     if (!fmt->sibling)
+             return NULL;
+     sibling = vpu_helper_find_format(inst, type, fmt->sibling);
+     if (!sibling)
+             return NULL;
+     if (sibling->sibling != fmt->pixfmt)
+             return NULL;
+     if (sibling->comp_planes != fmt->comp_planes)
+             return NULL;
+     return sibling;
+}
+
+bool vpu_helper_match_format(struct vpu_inst *inst, u32 type, u32
+fmta, u32 fmtb) {
+     const struct vpu_format *sibling;
+
+     if (fmta == fmtb)
+             return true;
+
+     sibling = vpu_helper_find_sibling(inst, type, fmta);
+     if (sibling && sibling->pixfmt == fmtb)
+             return true;
+     return false;
+}
+
 const struct vpu_format *vpu_helper_enum_format(struct vpu_inst
 *inst, u32 type, int index)  {
      const struct vpu_format *pfmt;

@@ -123,9 +156,10 @@ static u32 get_nv12_plane_size(u32 width, u32

height, int plane_no,

drivers/media/platform/amphion/vpu_helpers.c u32 vpu_helper_get_plane_size(u32 fmt, u32 w, u32

Quoted 43 lines

drivers/media/platform/amphion/vpu_helpers.c +2 -1

      u32 bytesperline;
      u32 size = 0;
 
-     bytesperline = ALIGN(width, stride);
+     bytesperline = width;
      if (pbl)
              bytesperline = max(bytesperline, *pbl);
+     bytesperline = ALIGN(bytesperline, stride);
      height = ALIGN(height, 2);
      if (plane_no == 0)
              size = bytesperline * height; @@ -148,13 +182,13 @@

static u32 get_tiled_8l128_plane_size(u32 fmt, u32 width, u32 height, int plane_

      if (interlaced)
              hs++;

drivers/media/platform/amphion/vpu_helpers.c +4 -3

-     if (fmt == V4L2_PIX_FMT_NV12M_10BE_8L128)
+     if (fmt == V4L2_PIX_FMT_NV12M_10BE_8L128 || fmt ==
+ V4L2_PIX_FMT_NV12_10BE_8L128)
              bitdepth = 10;
      bytesperline = DIV_ROUND_UP(width * bitdepth, BITS_PER_BYTE);
-     bytesperline = ALIGN(bytesperline, 1 << ws);
-     bytesperline = ALIGN(bytesperline, stride);
      if (pbl)
              bytesperline = max(bytesperline, *pbl);
+     bytesperline = ALIGN(bytesperline, 1 << ws);
+     bytesperline = ALIGN(bytesperline, stride);
      height = ALIGN(height, 1 << hs);
      if (plane_no == 0)
              size = bytesperline * height; @@ -172,9 +206,10 @@

static u32 get_default_plane_size(u32 width, u32 height, int plane_no,

      u32 bytesperline;
      u32 size = 0;

drivers/media/platform/amphion/vpu_helpers.c +2 -1

-     bytesperline = ALIGN(width, stride);
+     bytesperline = width;
      if (pbl)
              bytesperline = max(bytesperline, *pbl);
+     bytesperline = ALIGN(bytesperline, stride);
      if (plane_no == 0)
              size = bytesperline * height;
      if (pbl)

@@ -187,9 +222,12 @@ u32 vpu_helper_get_plane_size(u32 fmt, u32 w, u32

h, int plane_no,

drivers/media/platform/amphion/vpu_helpers.c u32 vpu_helper_get_plane_size(u32 fmt, u32 w, u32 h, int plane_no,

Quoted 11 lines

drivers/media/platform/amphion/vpu_helpers.c +3 -0

                            u32 stride, u32 interlaced, u32 *pbl)  {
      switch (fmt) {
+     case V4L2_PIX_FMT_NV12:
      case V4L2_PIX_FMT_NV12M:
              return get_nv12_plane_size(w, h, plane_no, stride,
 interlaced, pbl);
+     case V4L2_PIX_FMT_NV12_8L128:
      case V4L2_PIX_FMT_NV12M_8L128:
+     case V4L2_PIX_FMT_NV12_10BE_8L128:
      case V4L2_PIX_FMT_NV12M_10BE_8L128:
              return get_tiled_8l128_plane_size(fmt, w, h, plane_no, stride,

interlaced, pbl);

drivers/media/platform/amphion/vpu_malone.c bool vpu_malone_check_fmt(enum vpu_core_type

Quoted 24 lines

drivers/media/platform/amphion/vpu_helpers.c +0 -0
      default:
drivers/media/platform/amphion/vpu_helpers.h +0 -0
index bc28350958be..0eaddb07190d 100644
@@ -14,6 +14,8 @@ struct vpu_pair {
 int vpu_helper_find_in_array_u8(const u8 *array, u32 size, u32 x);
bool vpu_helper_check_type(struct vpu_inst *inst, u32 type); const struct vpu_format *vpu_helper_find_format(struct vpu_inst *inst, u32 type, u32 pixelfmt);
drivers/media/platform/amphion/vpu_helpers.h +3 -0
+const struct vpu_format *vpu_helper_find_sibling(struct vpu_inst
+*inst, u32 type, u32 pixelfmt); bool vpu_helper_match_format(struct
+vpu_inst *inst, u32 type, u32 fmta, u32 fmtb);
 const struct vpu_format *vpu_helper_enum_format(struct vpu_inst
*inst, u32 type, int index);
u32 vpu_helper_valid_frame_width(struct vpu_inst *inst, u32 width); u32 vpu_helper_valid_frame_height(struct vpu_inst *inst, u32 height);
drivers/media/platform/amphion/vpu_malone.c +0 -0
index 51e0702f9ae1..69d9a2269fce 100644
@@ -583,7 +583,8 @@ bool vpu_malone_check_fmt(enum vpu_core_type

type, u32 pixelfmt)

drivers/media/platform/amphion/vpu_malone.c bool vpu_malone_check_fmt(enum vpu_core_type type, u32 pixelfmt)

Quoted 4 lines

drivers/media/platform/amphion/vpu_malone.c +0 -1

      if (!vpu_imx8q_check_fmt(type, pixelfmt))
              return false;
 
-     if (pixelfmt == V4L2_PIX_FMT_NV12M_8L128 || pixelfmt ==

V4L2_PIX_FMT_NV12M_10BE_8L128)

drivers/media/platform/amphion/vpu_malone.c bool vpu_malone_check_fmt(enum vpu_core_type · line 586

Quoted 1 line

drivers/media/platform/amphion/vpu_malone.c +1 -0
+     if (pixelfmt == V4L2_PIX_FMT_NV12_8L128 || pixelfmt ==

V4L2_PIX_FMT_NV12_10BE_8L128 ||

Quoted 36 lines

drivers/media/platform/amphion/vpu_malone.c +2 -0

+         pixelfmt == V4L2_PIX_FMT_NV12M_8L128 || pixelfmt ==
+ V4L2_PIX_FMT_NV12M_10BE_8L128)
              return true;
      if (vpu_malone_format_remap(pixelfmt) == MALONE_FMT_NULL)
              return false;

drivers/media/platform/amphion/vpu_v4l2.c +17 -2

index 8a3eed957ae6..87cce552e906 100644
@@ -140,51 +140,135 @@ bool vpu_is_source_empty(struct vpu_inst *inst)
      return true;
 }
 
-const struct vpu_format *vpu_try_fmt_common(struct vpu_inst *inst,
 struct v4l2_format *f)
+static int vpu_init_format(struct vpu_inst *inst, struct vpu_format
+*fmt) {
+     const struct vpu_format *info;
+
+     info = vpu_helper_find_format(inst, fmt->type, fmt->pixfmt);
+     if (!info) {
+             info = vpu_helper_enum_format(inst, fmt->type, 0);
+             if (!info)
+                     return -EINVAL;
+     }
+     memcpy(fmt, info, sizeof(*fmt));
+
+     return 0;
+}
+
+static int vpu_calc_fmt_bytesperline(struct v4l2_format *f, struct
+vpu_format *fmt)
 {
      struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
-     u32 type = f->type;

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 172

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +10 -0

+     int i;
+
+     if (fmt->flags & V4L2_FMT_FLAG_COMPRESSED) {
+             for (i = 0; i < fmt->comp_planes; i++)
+                     fmt->bytesperline[i] = 0;
+             return 0;
+     }
+     if (pixmp->num_planes == fmt->comp_planes) {
+             for (i = 0; i < fmt->comp_planes; i++)
+                     fmt->bytesperline[i] =

pixmp->plane_fmt[i].bytesperline;

Quoted 32 lines

drivers/media/platform/amphion/vpu_v4l2.c +19 ~~-10~~

+             return 0;
+     }
+     if (pixmp->num_planes > 1)
+             return -EINVAL;
+
+     /*amphion vpu only support nv12 and nv12 tiled,
+      * so the bytesperline of luma and chroma should be same
+      */
+     for (i = 0; i < fmt->comp_planes; i++)
+             fmt->bytesperline[i] = pixmp->plane_fmt[0].bytesperline;
+
+     return 0;
+}
+
+static int vpu_calc_fmt_sizeimage(struct vpu_inst *inst, struct
+vpu_format *fmt) {
      u32 stride = 1;
-     u32 bytesperline;
-     u32 sizeimage;
-     const struct vpu_format *fmt;
-     const struct vpu_core_resources *res;
      int i;
 
-     fmt = vpu_helper_find_format(inst, type, pixmp->pixelformat);
-     if (!fmt) {
-             fmt = vpu_helper_enum_format(inst, type, 0);
-             if (!fmt)
-                     return NULL;
-             pixmp->pixelformat = fmt->pixfmt;
+     if (!(fmt->flags & V4L2_FMT_FLAG_COMPRESSED)) {
+             const struct vpu_core_resources *res =
+ vpu_get_resource(inst);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -5

+
+             if (res)
+                     stride = res->stride;
      }
 
-     res = vpu_get_resource(inst);
-     if (res)
-             stride = res->stride;
-     if (pixmp->width)
-             pixmp->width = vpu_helper_valid_frame_width(inst,

pixmp->width);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 169

Quoted 2 lines

drivers/media/platform/amphion/vpu_v4l2.c +0 -2

-     if (pixmp->height)
-             pixmp->height = vpu_helper_valid_frame_height(inst,

pixmp->height);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 211

Quoted 3 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -0

+     for (i = 0; i < fmt->comp_planes; i++) {
+             fmt->sizeimage[i] = vpu_helper_get_plane_size(fmt->pixfmt,
+

fmt->width,

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 203

Quoted 1 line

drivers/media/platform/amphion/vpu_v4l2.c +1 -0
+

fmt->height,

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 211

Quoted 3 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -0

+                                                           i,
+                                                           stride,
+

fmt->field != V4L2_FIELD_NONE ? 1 : 0,

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 207

Quoted 1 line

drivers/media/platform/amphion/vpu_v4l2.c +1 -0
+

&fmt->bytesperline[i]);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 3 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -0

+             fmt->sizeimage[i] = max_t(u32, fmt->sizeimage[i], PAGE_SIZE);
+             if (fmt->flags & V4L2_FMT_FLAG_COMPRESSED) {
+                     fmt->sizeimage[i] = clamp_val(fmt->sizeimage[i],

SZ_128K, SZ_8M);

Quoted 31 lines

drivers/media/platform/amphion/vpu_v4l2.c +31 -0

+                     fmt->bytesperline[i] = 0;
+             }
+     }
+
+     return 0;
+}
+
+u32 vpu_get_fmt_plane_size(struct vpu_format *fmt, u32 plane_no) {
+     u32 size;
+     int i;
+
+     if (plane_no >= fmt->mem_planes)
+             return 0;
+
+     if (fmt->comp_planes == fmt->mem_planes)
+             return fmt->sizeimage[plane_no];
+     if (plane_no < fmt->mem_planes - 1)
+             return fmt->sizeimage[plane_no];
+     size = fmt->sizeimage[plane_no];
+     for (i = fmt->mem_planes; i < fmt->comp_planes; i++)
+             size += fmt->sizeimage[i];
+
+     return size;
+}
+
+int vpu_try_fmt_common(struct vpu_inst *inst, struct v4l2_format *f,
+struct vpu_format *fmt) {
+     struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
+     int i;
+     int ret;
+

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +10 -0

+     fmt->pixfmt = pixmp->pixelformat;
+     fmt->type = f->type;
+     ret = vpu_init_format(inst, fmt);
+     if (ret < 0)
+             return ret;
+
+     fmt->width = pixmp->width;
+     fmt->height = pixmp->height;
+     if (fmt->width)
+             fmt->width = vpu_helper_valid_frame_width(inst,

fmt->width);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 253

Quoted 2 lines

drivers/media/platform/amphion/vpu_v4l2.c +2 -0

+     if (fmt->height)
+             fmt->height = vpu_helper_valid_frame_height(inst,

fmt->height);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 254

Quoted 1 line

drivers/media/platform/amphion/vpu_v4l2.c +1 -0
+     fmt->field = pixmp->field == V4L2_FIELD_ANY ? V4L2_FIELD_NONE :

pixmp->field;

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 3 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -0

+     vpu_calc_fmt_bytesperline(f, fmt);
+     vpu_calc_fmt_sizeimage(inst, fmt);
+     if ((fmt->flags & V4L2_FMT_FLAG_COMPRESSED) &&

pixmp->plane_fmt[0].sizeimage)

Quoted 5 lines

drivers/media/platform/amphion/vpu_v4l2.c +5 -0

+             fmt->sizeimage[0] =
+ clamp_val(pixmp->plane_fmt[0].sizeimage, SZ_128K, SZ_8M);
+
+     pixmp->pixelformat = fmt->pixfmt;
+     pixmp->width = fmt->width;

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +4 -4

+     pixmp->height = fmt->height;
      pixmp->flags = fmt->flags;
-     pixmp->num_planes = fmt->num_planes;
-     if (pixmp->field == V4L2_FIELD_ANY)
-             pixmp->field = V4L2_FIELD_NONE;
+     pixmp->num_planes = fmt->mem_planes;
+     pixmp->field = fmt->field;
+     memset(pixmp->reserved, 0, sizeof(pixmp->reserved));
      for (i = 0; i < pixmp->num_planes; i++) {
-             bytesperline = max_t(s32, pixmp->plane_fmt[i].bytesperline,

0);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 6 lines

drivers/media/platform/amphion/vpu_v4l2.c +0 -6

-             sizeimage = vpu_helper_get_plane_size(pixmp->pixelformat,
-                                                   pixmp->width,
-                                                   pixmp->height,
-                                                   i,
-                                                   stride,
-                                                   pixmp->field >

V4L2_FIELD_NONE ? 1 : 0,

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 183

Quoted 2 lines

drivers/media/platform/amphion/vpu_v4l2.c +0 -2

-                                                   &bytesperline);
-             sizeimage = max_t(s32, pixmp->plane_fmt[i].sizeimage,

sizeimage);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 4 lines

drivers/media/platform/amphion/vpu_v4l2.c +2 -2

-             pixmp->plane_fmt[i].bytesperline = bytesperline;
-             pixmp->plane_fmt[i].sizeimage = sizeimage;
+             pixmp->plane_fmt[i].bytesperline = fmt->bytesperline[i];
+             pixmp->plane_fmt[i].sizeimage = vpu_get_fmt_plane_size(fmt,

i);

drivers/media/platform/amphion/vpu_v4l2.c static int vpu_vb2_queue_setup(struct vb2_queue

Quoted 25 lines

drivers/media/platform/amphion/vpu_v4l2.c +7 -5

+             memset(pixmp->plane_fmt[i].reserved, 0,
+ sizeof(pixmp->plane_fmt[i].reserved));
      }
 
-     return fmt;
+     return 0;
 }
 
 static bool vpu_check_ready(struct vpu_inst *inst, u32 type) @@
-389,10 +473,10 @@ static int vpu_vb2_queue_setup(struct vb2_queue *vq,
      cur_fmt = vpu_get_format(inst, vq->type);
 
      if (*plane_count) {
-             if (*plane_count != cur_fmt->num_planes)
+             if (*plane_count != cur_fmt->mem_planes)
                      return -EINVAL;
-             for (i = 0; i < cur_fmt->num_planes; i++) {
-                     if (psize[i] < cur_fmt->sizeimage[i])
+             for (i = 0; i < cur_fmt->mem_planes; i++) {
+                     if (psize[i] < vpu_get_fmt_plane_size(cur_fmt,
+ i))
                              return -EINVAL;
              }
              return 0;

@@ -402,9 +486,9 @@ static int vpu_vb2_queue_setup(struct vb2_queue

*vq,

drivers/media/platform/amphion/vpu_v4l2.c static int vpu_vb2_queue_setup(struct vb2_queue · line 486

Quoted 1 line

drivers/media/platform/amphion/vpu_v4l2.c +0 -0
              *buf_count = max_t(unsigned int, *buf_count,

inst->min_buffer_out);

drivers/media/platform/amphion/vpu_v4l2.c static int vpu_vb2_queue_setup(struct vb2_queue · line 488

Quoted 2 lines

drivers/media/platform/amphion/vpu_v4l2.c +0 -0

      else
              *buf_count = max_t(unsigned int, *buf_count,

inst->min_buffer_cap);

drivers/media/platform/amphion/vpu_v4l2.c static int vpu_vb2_buf_prepare(struct vb2_buffer

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -3

-     *plane_count = cur_fmt->num_planes;
-     for (i = 0; i < cur_fmt->num_planes; i++)
-             psize[i] = cur_fmt->sizeimage[i];
+     *plane_count = cur_fmt->mem_planes;
+     for (i = 0; i < cur_fmt->mem_planes; i++)
+             psize[i] = vpu_get_fmt_plane_size(cur_fmt, i);
 
      return 0;
 }

@@ -434,8 +518,8 @@ static int vpu_vb2_buf_prepare(struct vb2_buffer

*vb)

drivers/media/platform/amphion/vpu_v4l2.c static int vpu_vb2_buf_prepare(struct vb2_buffer *vb)

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -2

      u32 i;
 
      cur_fmt = vpu_get_format(inst, vb->type);
-     for (i = 0; i < cur_fmt->num_planes; i++) {
-             if (vpu_get_vb_length(vb, i) < cur_fmt->sizeimage[i]) {
+     for (i = 0; i < cur_fmt->mem_planes; i++) {
+             if (vpu_get_vb_length(vb, i) <
+ vpu_get_fmt_plane_size(cur_fmt, i)) {
                      dev_dbg(inst->dev, "[%d] %s buf[%d] is invalid\n",
                              inst->id, vpu_type_name(vb->type),

vb->index);

Quoted 1 line

vpu_set_buffer_state(vbuf,

VPU_BUF_STATE_ERROR);

drivers/media/platform/amphion/vpu_windsor.c static int vpu_windsor_fill_yuv_frame(struct

Quoted 23 lines

drivers/media/platform/amphion/vpu_v4l2.h +0 -0
index 795ca33a6a50..ef5de6b66e47 100644
@@ -16,7 +16,8 @@ unsigned int vpu_get_buffer_state(struct
vb2_v4l2_buffer *vbuf); int vpu_v4l2_open(struct file *file, struct vpu_inst *inst); int vpu_v4l2_close(struct file *file);
-const struct vpu_format *vpu_try_fmt_common(struct vpu_inst *inst, struct v4l2_format *f);
drivers/media/platform/amphion/vpu_v4l2.h +3 -0
+u32 vpu_get_fmt_plane_size(struct vpu_format *fmt, u32 plane_no); int
+vpu_try_fmt_common(struct vpu_inst *inst, struct v4l2_format *f,
+struct vpu_format *fmt);
 int vpu_process_output_buffer(struct vpu_inst *inst);  int
vpu_process_capture_buffer(struct vpu_inst *inst); struct vb2_v4l2_buffer *vpu_next_src_buf(struct vpu_inst *inst); diff --git a/drivers/media/platform/amphion/vpu_windsor.c b/drivers/media/platform/amphion/vpu_windsor.c
index 1526af2ef9da..a454f142ae17 100644
drivers/media/platform/amphion/vpu_windsor.c +0 -0
@@ -775,6 +775,8 @@ static int vpu_windsor_fill_yuv_frame(struct

vpu_shared_addr *shared,

drivers/media/platform/amphion/vpu_windsor.c static int vpu_windsor_fill_yuv_frame(struct

Quoted 8 lines

drivers/media/platform/amphion/vpu_windsor.c +2 -0

                                    u32 instance,
                                    struct vb2_buffer *vb)  {
+     struct vpu_inst *inst = vb2_get_drv_priv(vb->vb2_queue);
+     struct vpu_format *out_fmt;
      struct vpu_enc_yuv_desc *desc;
      struct vb2_v4l2_buffer *vbuf;

@@ -782,6 +784,7 @@ static int vpu_windsor_fill_yuv_frame(struct

vpu_shared_addr *shared,

Quoted 21 lines

drivers/media/platform/amphion/vpu_windsor.c +1 -0

              return -EINVAL;
 
      desc = get_yuv_desc(shared, instance);
+     out_fmt = vpu_get_format(inst, vb->type);
 
      vbuf = to_vb2_v4l2_buffer(vb);
      desc->frame_id = vbuf->sequence; @@ -790,7 +793,10 @@ static int

vpu_windsor_fill_yuv_frame(struct vpu_shared_addr *shared,

      else
              desc->key_frame = 0;
      desc->luma_base = vpu_get_vb_phy_addr(vb, 0);

drivers/media/platform/amphion/vpu_windsor.c +5 -2

-     desc->chroma_base = vpu_get_vb_phy_addr(vb, 1);
+     if (vb->num_planes > 1)
+             desc->chroma_base = vpu_get_vb_phy_addr(vb, 1);
+     else
+             desc->chroma_base = desc->luma_base +
+ out_fmt->sizeimage[0];
 
      return 0;
 }
--

Quoted 2 lines

2.37.1

--
Tommaso Merciai
Embedded Linux Engineer
[email protected]
__________________________________

Tommaso Merciai

2022-08-22 07:25 UTC #12 lore raw

Re: [EXT] Re: [PATCH v4 2/4] media: amphion: tell and handle contiguous and non contiguous format

Quoted 5 lines · Ming Qian

Hi Tommaso,
I'm sorry that I didn't notice this patch set is dependent on the following patch:
85069 New media: amphion: don't change the colorspace reported by decoder.

Hi Ming,
Sorry for delay, thanks for the info.

Regards,
Tommaso

Quoted 1,052 lines

Ming

Quoted 1,049 lines

Hi Ming,
Just an info. I'm planning to help to review your series. On which Linux tree are you working?

Thanks in advance,
Tommaso

drivers/media/platform/amphion/vdec.c static int vdec_enum_fmt(struct file *file, void *fh,

Quoted 145 lines · Ming Qian

Driver should tell the number of memory planes and component planes. the amphion vpu support non contiguous planes, but for compatibility with other device that only support contiguous planes. driver can add support for contiguous planes in the same time. Then the mem_planes can be different from the comp_planes. driver need to handle buffer according mem_planes and comp_planes.

So driver can support NV12 and NV12M.

Signed-off-by Ming Qian <[email protected]>

---
 drivers/media/platform/amphion/vdec.c        | 187 ++++++++++---------
 drivers/media/platform/amphion/venc.c        |  33 ++--
 drivers/media/platform/amphion/vpu.h         |   4 +-
 drivers/media/platform/amphion/vpu_dbg.c     |   8 +-
 drivers/media/platform/amphion/vpu_helpers.c |  48 ++++-
 drivers/media/platform/amphion/vpu_helpers.h |   2 +
 drivers/media/platform/amphion/vpu_malone.c  |   3 +-
 drivers/media/platform/amphion/vpu_v4l2.c    | 168 ++++++++++++-----
 drivers/media/platform/amphion/vpu_v4l2.h    |   3 +-
 drivers/media/platform/amphion/vpu_windsor.c |   8 +-
 10 files changed, 299 insertions(+), 165 deletions(-)

drivers/media/platform/amphion/vdec.c +43 ~~-21~~

index feb75dc204de..48ab664fa7ef 100644
@@ -69,72 +69,85 @@ struct vdec_t {
 static const struct vpu_format vdec_formats[] = {
      {
              .pixfmt = V4L2_PIX_FMT_NV12M_8L128,
-             .num_planes = 2,
+             .mem_planes = 2,
+             .comp_planes = 2,
              .type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
      },
      {
              .pixfmt = V4L2_PIX_FMT_NV12M_10BE_8L128,
-             .num_planes = 2,
+             .mem_planes = 2,
+             .comp_planes = 2,
              .type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
      },
      {
              .pixfmt = V4L2_PIX_FMT_H264,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_H264_MVC,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_HEVC,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_VC1_ANNEX_G,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_VC1_ANNEX_L,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+             .flags = V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_MPEG2,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_MPEG4,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_XVID,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_VP8,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {
              .pixfmt = V4L2_PIX_FMT_H263,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+             .flags = V4L2_FMT_FLAG_DYN_RESOLUTION |
+ V4L2_FMT_FLAG_COMPRESSED
      },
      {0, 0, 0, 0},
 };

@@ -256,23 +269,22 @@ static int vdec_enum_fmt(struct file *file, void *fh,

struct v4l2_fmtdesc *f)

drivers/media/platform/amphion/vdec.c static int vdec_enum_fmt(struct file *file, void *fh, struct v4l2_fmtdesc *f)

Quoted 13 lines

drivers/media/platform/amphion/vdec.c +4 -6

      int ret = -EINVAL;
 
      vpu_inst_lock(inst);
-     if (!V4L2_TYPE_IS_OUTPUT(f->type) && vdec->fixed_fmt) {
-             if (f->index == 0) {
-                     f->pixelformat = inst->cap_format.pixfmt;
-                     f->flags = inst->cap_format.flags;
-                     ret = 0;
-             }
+     if (V4L2_TYPE_IS_CAPTURE(f->type) && vdec->fixed_fmt) {
+             fmt = vpu_get_format(inst, f->type);
+             if (f->index == 1)
+                     fmt = vpu_helper_find_sibling(inst, f->type,

fmt->pixfmt);

drivers/media/platform/amphion/vdec.c static int vdec_g_fmt(struct file *file, void *fh,

Quoted 23 lines

drivers/media/platform/amphion/vdec.c +8 -7

+             if (f->index > 1)
+                     fmt = NULL;
      } else {
              fmt = vpu_helper_enum_format(inst, f->type, f->index);
-             memset(f->reserved, 0, sizeof(f->reserved));
-             if (!fmt)
-                     goto exit;
-
-             f->pixelformat = fmt->pixfmt;
-             f->flags = fmt->flags;
-             ret = 0;
      }
+     if (!fmt)
+             goto exit;
 
+     memset(f->reserved, 0, sizeof(f->reserved));
+     f->pixelformat = fmt->pixfmt;
+     f->flags = fmt->flags;
+     ret = 0;
 exit:
      vpu_inst_unlock(inst);
      return ret;

@@ -289,14 +301,14 @@ static int vdec_g_fmt(struct file *file, void *fh,

struct v4l2_format *f)

Quoted 21 lines

drivers/media/platform/amphion/vdec.c +3 -3

      cur_fmt = vpu_get_format(inst, f->type);
 
      pixmp->pixelformat = cur_fmt->pixfmt;
-     pixmp->num_planes = cur_fmt->num_planes;
+     pixmp->num_planes = cur_fmt->mem_planes;
      pixmp->width = cur_fmt->width;
      pixmp->height = cur_fmt->height;
      pixmp->field = cur_fmt->field;
      pixmp->flags = cur_fmt->flags;
      for (i = 0; i < pixmp->num_planes; i++) {
              pixmp->plane_fmt[i].bytesperline = cur_fmt->bytesperline[i];
-             pixmp->plane_fmt[i].sizeimage = cur_fmt->sizeimage[i];
+             pixmp->plane_fmt[i].sizeimage =
+ vpu_get_fmt_plane_size(cur_fmt, i);
      }
 
      f->fmt.pix_mp.colorspace = vdec->codec_info.color_primaries; @@
-311,10 +323,19 @@ static int vdec_try_fmt(struct file *file, void

*fh, struct v4l2_format *f)  {
      struct vpu_inst *inst = to_inst(file);
      struct vdec_t *vdec = inst->priv;

drivers/media/platform/amphion/vdec.c static int vdec_try_fmt(struct file *file, void *fh, struct v4l2_format *f)

Quoted 10 lines

drivers/media/platform/amphion/vdec.c +6 -2

-
-     vpu_try_fmt_common(inst, f);
+     struct vpu_format fmt;
 
      vpu_inst_lock(inst);
+     if (V4L2_TYPE_IS_CAPTURE(f->type) && vdec->fixed_fmt) {
+             struct vpu_format *cap_fmt = vpu_get_format(inst,
+ f->type);
+
+             if (!vpu_helper_match_format(inst, cap_fmt->type,

cap_fmt->pixfmt,

drivers/media/platform/amphion/vdec.c static int vdec_try_fmt(struct file *file, void *fh, struct v4l2_format *f)

Quoted 8 lines

drivers/media/platform/amphion/vdec.c +6 -0

+                                          f->fmt.pix_mp.pixelformat))
+                     f->fmt.pix_mp.pixelformat = cap_fmt->pixfmt;
+     }
+
+     vpu_try_fmt_common(inst, f, &fmt);
+
      if (vdec->fixed_fmt) {
              f->fmt.pix_mp.colorspace =

vdec->codec_info.color_primaries;

Quoted 14 lines

              f->fmt.pix_mp.xfer_func =
vdec->codec_info.transfer_chars; @@ -334,7 +355,7 @@ static int
vdec_try_fmt(struct file *file, void *fh, struct v4l2_format *f)
static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)  {
      struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;

drivers/media/platform/amphion/vdec.c +1 -1

-     const struct vpu_format *fmt;
+     struct vpu_format fmt;
      struct vpu_format *cur_fmt;
      struct vb2_queue *q;
      struct vdec_t *vdec = inst->priv; @@ -349,36 +370,30 @@ static

int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)

      if (vb2_is_busy(q))
              return -EBUSY;

drivers/media/platform/amphion/vdec.c static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)

Quoted 7 lines

drivers/media/platform/amphion/vdec.c +1 -2

-     fmt = vpu_try_fmt_common(inst, f);
-     if (!fmt)
+     if (vpu_try_fmt_common(inst, f, &fmt))
              return -EINVAL;
 
      cur_fmt = vpu_get_format(inst, f->type);
      if (V4L2_TYPE_IS_OUTPUT(f->type) && inst->state !=

VPU_CODEC_STATE_DEINIT) {

Quoted 4 lines

drivers/media/platform/amphion/vdec.c +1 -1

-             if (cur_fmt->pixfmt != fmt->pixfmt) {
+             if (cur_fmt->pixfmt != fmt.pixfmt) {
                      vdec->reset_codec = true;
                      vdec->fixed_fmt = false;

drivers/media/platform/amphion/vdec.c static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)

Quoted 10 lines

drivers/media/platform/amphion/vdec.c +0 -7

              }
      }
-     cur_fmt->pixfmt = fmt->pixfmt;
      if (V4L2_TYPE_IS_OUTPUT(f->type) || !vdec->fixed_fmt) {
-             cur_fmt->num_planes = fmt->num_planes;
-             cur_fmt->flags = fmt->flags;
-             cur_fmt->width = pixmp->width;
-             cur_fmt->height = pixmp->height;
-             for (i = 0; i < fmt->num_planes; i++) {
-                     cur_fmt->sizeimage[i] =

pixmp->plane_fmt[i].sizeimage;

Quoted 1 line

- cur_fmt->bytesperline[i] =

pixmp->plane_fmt[i].bytesperline;

drivers/media/platform/amphion/vdec.c static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)

Quoted 7 lines

drivers/media/platform/amphion/vdec.c +2 -4

-             }
-             if (pixmp->field != V4L2_FIELD_ANY)
-                     cur_fmt->field = pixmp->field;
+             memcpy(cur_fmt, &fmt, sizeof(*cur_fmt));
      } else {
-             pixmp->num_planes = cur_fmt->num_planes;
+             if (vpu_helper_match_format(inst, f->type, cur_fmt->pixfmt,

pixmp->pixelformat)) {

drivers/media/platform/amphion/vdec.c static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)

Quoted 9 lines

drivers/media/platform/amphion/vdec.c +5 -0

+                     cur_fmt->pixfmt = fmt.pixfmt;
+                     cur_fmt->mem_planes = fmt.mem_planes;
+             }
+             pixmp->pixelformat = cur_fmt->pixfmt;
+             pixmp->num_planes = cur_fmt->mem_planes;
              pixmp->width = cur_fmt->width;
              pixmp->height = cur_fmt->height;
              for (i = 0; i < pixmp->num_planes; i++) {
                      pixmp->plane_fmt[i].bytesperline =

cur_fmt->bytesperline[i];

Quoted 1 line

- pixmp->plane_fmt[i].sizeimage =

cur_fmt->sizeimage[i];

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst,

Quoted 18 lines

drivers/media/platform/amphion/vdec.c +4 -0

+                     pixmp->plane_fmt[i].sizeimage =
+ vpu_get_fmt_plane_size(cur_fmt, i);
              }
              pixmp->field = cur_fmt->field;
      }

@@ -678,9 +693,11 @@ static struct vpu_vb2_buffer
 *vdec_find_buffer(struct vpu_inst *inst, u32 luma)  static void
 vdec_buf_done(struct vpu_inst *inst, struct vpu_frame_info *frame)  {
      struct vdec_t *vdec = inst->priv;
+     struct vpu_format *cur_fmt;
      struct vpu_vb2_buffer *vpu_buf;
      struct vb2_v4l2_buffer *vbuf;
      u32 sequence;
+     int i;
 
      if (!frame)
              return;

@@ -699,6 +716,7 @@ static void vdec_buf_done(struct vpu_inst *inst,

struct vpu_frame_info *frame)

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst,

Quoted 8 lines

drivers/media/platform/amphion/vdec.c +1 -0

              return;
      }
 
+     cur_fmt = vpu_get_format(inst, inst->cap_format.type);
      vbuf = &vpu_buf->m2m_buf.vb;
      if (vbuf->vb2_buf.index != frame->id)
              dev_err(inst->dev, "[%d] buffer id(%d, %d) dismatch\n",

@@ -707,9 +725,9 @@ static void vdec_buf_done(struct vpu_inst *inst,

struct vpu_frame_info *frame)

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst, · line 726

Quoted 2 lines

drivers/media/platform/amphion/vdec.c +0 -0

      if (vpu_get_buffer_state(vbuf) != VPU_BUF_STATE_DECODED)
              dev_err(inst->dev, "[%d] buffer(%d) ready without

decoded\n", inst->id, frame->id);

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst, · line 710

Quoted 2 lines

drivers/media/platform/amphion/vdec.c +0 -1

      vpu_set_buffer_state(vbuf, VPU_BUF_STATE_READY);
-     vb2_set_plane_payload(&vbuf->vb2_buf, 0,

inst->cap_format.sizeimage[0]);

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst, · line 711

Quoted 1 line

drivers/media/platform/amphion/vdec.c +0 -1
-     vb2_set_plane_payload(&vbuf->vb2_buf, 1,

inst->cap_format.sizeimage[1]);

drivers/media/platform/amphion/vdec.c static void vdec_buf_done(struct vpu_inst *inst, struct vpu_frame_info *frame)

Quoted 3 lines

drivers/media/platform/amphion/vdec.c +2 -1

-     vbuf->field = inst->cap_format.field;
+     for (i = 0; i < vbuf->vb2_buf.num_planes; i++)
+             vb2_set_plane_payload(&vbuf->vb2_buf, i,

vpu_get_fmt_plane_size(cur_fmt, i));

Quoted 6 lines

drivers/media/platform/amphion/vdec.c +1 -0

+     vbuf->field = cur_fmt->field;
      vbuf->sequence = sequence;
      dev_dbg(inst->dev, "[%d][OUTPUT TS]%32lld\n", inst->id,
 vbuf->vb2_buf.timestamp);

drivers/media/platform/amphion/vdec.c static void vdec_stop_done(struct vpu_inst · line 772

Quoted 10 lines

drivers/media/platform/amphion/vdec.c +2 -2

@@ -747,15 +765,17 @@ static void vdec_stop_done(struct vpu_inst
 *inst)  static bool vdec_check_source_change(struct vpu_inst *inst)  {
      struct vdec_t *vdec = inst->priv;
-     const struct vpu_format *fmt;
-     int i;
+     const struct vpu_format *sibling;
 
      if (!inst->fh.m2m_ctx)
              return false;
 
+     sibling = vpu_helper_find_sibling(inst, inst->cap_format.type,

inst->cap_format.pixfmt);

drivers/media/platform/amphion/vdec.c static void vdec_stop_done(struct vpu_inst *inst)

Quoted 6 lines

drivers/media/platform/amphion/vdec.c +3 -1

+     if (sibling && vdec->codec_info.pixfmt == sibling->pixfmt)
+             vdec->codec_info.pixfmt = inst->cap_format.pixfmt;
+
      if (!vb2_is_streaming(v4l2_m2m_get_dst_vq(inst->fh.m2m_ctx)))
              return true;
-     fmt = vpu_helper_find_format(inst, inst->cap_format.type,

vdec->codec_info.pixfmt);

drivers/media/platform/amphion/vdec.c static void vdec_stop_done(struct vpu_inst *inst)

Quoted 4 lines

drivers/media/platform/amphion/vdec.c +0 -1

      if (inst->cap_format.pixfmt != vdec->codec_info.pixfmt)
              return true;
      if (inst->cap_format.width != vdec->codec_info.decoded_width) @@
-772,14 +792,6 @@ static bool vdec_check_source_change(struct vpu_inst

*inst)

drivers/media/platform/amphion/vdec.c static bool vdec_check_source_change(struct vpu_inst *inst)

Quoted 7 lines

drivers/media/platform/amphion/vdec.c +0 -4

              return true;
      if (inst->crop.height != vdec->codec_info.height)
              return true;
-     if (fmt && inst->cap_format.num_planes != fmt->num_planes)
-             return true;
-     for (i = 0; i < inst->cap_format.num_planes; i++) {
-             if (inst->cap_format.bytesperline[i] !=

vdec->codec_info.bytesperline[i])

drivers/media/platform/amphion/vdec.c

Quoted 2 lines

drivers/media/platform/amphion/vdec.c +0 -2

-                     return true;
-             if (inst->cap_format.sizeimage[i] !=

vdec->codec_info.sizeimage[i])

drivers/media/platform/amphion/vdec.c static bool vdec_check_source_change(struct · line 792

Quoted 13 lines

drivers/media/platform/amphion/vdec.c +1 -5

-                     return true;
-     }
 
      return false;
 }

@@ -787,27 +799,21 @@ static bool vdec_check_source_change(struct
 vpu_inst *inst)  static void vdec_init_fmt(struct vpu_inst *inst)  {
      struct vdec_t *vdec = inst->priv;
-     const struct vpu_format *fmt;
-     int i;
+     struct v4l2_format f;
 
-     fmt = vpu_helper_find_format(inst, inst->cap_format.type,

vdec->codec_info.pixfmt);

drivers/media/platform/amphion/vdec.c static bool vdec_check_source_change(struct vpu_inst *inst)

Quoted 11 lines

drivers/media/platform/amphion/vdec.c +0 ~~-11~~

-     inst->out_format.width = vdec->codec_info.width;
-     inst->out_format.height = vdec->codec_info.height;
-     inst->cap_format.width = vdec->codec_info.decoded_width;
-     inst->cap_format.height = vdec->codec_info.decoded_height;
-     inst->cap_format.pixfmt = vdec->codec_info.pixfmt;
-     if (fmt) {
-             inst->cap_format.num_planes = fmt->num_planes;
-             inst->cap_format.flags = fmt->flags;
-     }
-     for (i = 0; i < inst->cap_format.num_planes; i++) {
-             inst->cap_format.bytesperline[i] =

vdec->codec_info.bytesperline[i];

Quoted 20 lines

drivers/media/platform/amphion/vdec.c +11 -4

-             inst->cap_format.sizeimage[i] = vdec->codec_info.sizeimage[i];
-     }
+     memset(&f, 0, sizeof(f));
+     f.type = inst->cap_format.type;
+     f.fmt.pix_mp.pixelformat = vdec->codec_info.pixfmt;
+     f.fmt.pix_mp.width = vdec->codec_info.decoded_width;
+     f.fmt.pix_mp.height = vdec->codec_info.decoded_height;
      if (vdec->codec_info.progressive)
-             inst->cap_format.field = V4L2_FIELD_NONE;
+             f.fmt.pix_mp.field = V4L2_FIELD_NONE;
      else
-             inst->cap_format.field = V4L2_FIELD_SEQ_TB;
+             f.fmt.pix_mp.field = V4L2_FIELD_SEQ_TB;
+     vpu_try_fmt_common(inst, &f, &inst->cap_format);
+
+     inst->out_format.width = vdec->codec_info.width;
+     inst->out_format.height = vdec->codec_info.height;
 }
 
 static void vdec_init_crop(struct vpu_inst *inst) @@ -966,7 +972,10

Quoted 1 line

@@ static int vdec_response_frame(struct vpu_inst *inst, struct

vb2_v4l2_buffer *vb

drivers/media/platform/amphion/vdec.c static int vdec_response_frame(struct vpu_inst *inst, struct vb2_v4l2_buffer *vb

Quoted 6 lines

      info.tag = vdec->seq_tag;
      info.luma_addr = vpu_get_vb_phy_addr(&vbuf->vb2_buf, 0);
      info.luma_size = inst->cap_format.sizeimage[0];

drivers/media/platform/amphion/vdec.c +2 -1

-     info.chroma_addr = vpu_get_vb_phy_addr(&vbuf->vb2_buf, 1);
+     if (vbuf->vb2_buf.num_planes > 1)
+             info.chroma_addr = vpu_get_vb_phy_addr(&vbuf->vb2_buf,

1);

drivers/media/platform/amphion/venc.c static int venc_g_fmt(struct file *file, void *fh,

Quoted 40 lines

drivers/media/platform/amphion/vdec.c +2 -0

+     else
+             info.chroma_addr = info.luma_addr + info.luma_size;
      info.chromau_size = inst->cap_format.sizeimage[1];
      info.bytesperline = inst->cap_format.bytesperline[0];
      ret = vpu_session_alloc_fs(inst, &info); @@ -975,7 +984,7 @@

static int vdec_response_frame(struct vpu_inst *inst, struct vb2_v4l2_buffer *vb

      vpu_buf->tag = info.tag;
      vpu_buf->luma = info.luma_addr;

drivers/media/platform/amphion/vdec.c +1 -1

-     vpu_buf->chroma_u = info.chromau_size;
+     vpu_buf->chroma_u = info.chroma_addr;
      vpu_buf->chroma_v = 0;
      vpu_set_buffer_state(vbuf, VPU_BUF_STATE_INUSE);
      vdec->slots[info.id] = vpu_buf;

drivers/media/platform/amphion/venc.c +5 -2

index 37212f087fdd..060a1ee78b17 100644
@@ -69,13 +69,16 @@ struct venc_frame_t {  static const struct
 vpu_format venc_formats[] = {
      {
              .pixfmt = V4L2_PIX_FMT_NV12M,
-             .num_planes = 2,
+             .mem_planes = 2,
+             .comp_planes = 2,
              .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
      },
      {
              .pixfmt = V4L2_PIX_FMT_H264,
-             .num_planes = 1,
+             .mem_planes = 1,
+             .comp_planes = 1,
              .type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+             .flags = V4L2_FMT_FLAG_COMPRESSED
      },
      {0, 0, 0, 0},
 };

@@ -173,14 +176,14 @@ static int venc_g_fmt(struct file *file, void *fh,

struct v4l2_format *f)

drivers/media/platform/amphion/venc.c static int venc_s_fmt(struct file *file, void *fh,

Quoted 43 lines

drivers/media/platform/amphion/venc.c +3 -3

      cur_fmt = vpu_get_format(inst, f->type);
 
      pixmp->pixelformat = cur_fmt->pixfmt;
-     pixmp->num_planes = cur_fmt->num_planes;
+     pixmp->num_planes = cur_fmt->mem_planes;
      pixmp->width = cur_fmt->width;
      pixmp->height = cur_fmt->height;
      pixmp->field = cur_fmt->field;
      pixmp->flags = cur_fmt->flags;
      for (i = 0; i < pixmp->num_planes; i++) {
              pixmp->plane_fmt[i].bytesperline = cur_fmt->bytesperline[i];
-             pixmp->plane_fmt[i].sizeimage = cur_fmt->sizeimage[i];
+             pixmp->plane_fmt[i].sizeimage =
+ vpu_get_fmt_plane_size(cur_fmt, i);
      }
 
      f->fmt.pix_mp.colorspace = venc->params.color.primaries; @@
-194,8 +197,9 @@ static int venc_g_fmt(struct file *file, void *fh,

struct v4l2_format *f) static int venc_try_fmt(struct file *file,

void *fh, struct v4l2_format *f)  {
      struct vpu_inst *inst = to_inst(file);

drivers/media/platform/amphion/venc.c +3 -3

+     struct vpu_format fmt;
 
-     vpu_try_fmt_common(inst, f);
+     vpu_try_fmt_common(inst, f, &fmt);
 
      return 0;
 }

@@ -203,12 +207,11 @@ static int venc_try_fmt(struct file *file, void
 *fh, struct v4l2_format *f)  static int venc_s_fmt(struct file *file,
 void *fh, struct v4l2_format *f)  {
      struct vpu_inst *inst = to_inst(file);
-     const struct vpu_format *fmt;
+     struct vpu_format fmt;
      struct vpu_format *cur_fmt;
      struct vb2_queue *q;
      struct venc_t *venc = inst->priv;
      struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
-     int i;
 
      q = v4l2_m2m_get_vq(inst->fh.m2m_ctx, f->type);
      if (!q)

@@ -216,24 +219,12 @@ static int venc_s_fmt(struct file *file, void *fh,

struct v4l2_format *f)

drivers/media/platform/amphion/vpu_dbg.c static int vpu_dbg_instance(struct seq_file *s, void

Quoted 55 lines

drivers/media/platform/amphion/venc.c +2 ~~-14~~

      if (vb2_is_busy(q))
              return -EBUSY;
 
-     fmt = vpu_try_fmt_common(inst, f);
-     if (!fmt)
+     if (vpu_try_fmt_common(inst, f, &fmt))
              return -EINVAL;
 
      cur_fmt = vpu_get_format(inst, f->type);
 
-     cur_fmt->pixfmt = fmt->pixfmt;
-     cur_fmt->num_planes = fmt->num_planes;
-     cur_fmt->flags = fmt->flags;
-     cur_fmt->width = pix_mp->width;
-     cur_fmt->height = pix_mp->height;
-     for (i = 0; i < fmt->num_planes; i++) {
-             cur_fmt->sizeimage[i] = pix_mp->plane_fmt[i].sizeimage;
-             cur_fmt->bytesperline[i] = pix_mp->plane_fmt[i].bytesperline;
-     }
-
-     if (pix_mp->field != V4L2_FIELD_ANY)
-             cur_fmt->field = pix_mp->field;
+     memcpy(cur_fmt, &fmt, sizeof(*cur_fmt));
 
      if (V4L2_TYPE_IS_OUTPUT(f->type)) {
              venc->params.input_format = cur_fmt->pixfmt; diff --git

a/drivers/media/platform/amphion/vpu.h
b/drivers/media/platform/amphion/vpu.h
index f914de6ed81e..998591fa5b85 100644

drivers/media/platform/amphion/vpu.h +3 -1

@@ -84,7 +84,8 @@ struct vpu_dev {
 
 struct vpu_format {
      u32 pixfmt;
-     unsigned int num_planes;
+     u32 mem_planes;
+     u32 comp_planes;
      u32 type;
      u32 flags;
      u32 width;

@@ -92,6 +93,7 @@ struct vpu_format {
      u32 sizeimage[VIDEO_MAX_PLANES];
      u32 bytesperline[VIDEO_MAX_PLANES];
      u32 field;
+     u32 sibling;
 };
 
 struct vpu_core_resources {

drivers/media/platform/amphion/vpu_dbg.c +0 -0

index f72c8a506b22..c41c9896912c 100644
@@ -89,9 +89,9 @@ static int vpu_dbg_instance(struct seq_file *s, void

*data)

drivers/media/platform/amphion/vpu_dbg.c static int vpu_dbg_instance(struct seq_file *s, void

Quoted 13 lines

drivers/media/platform/amphion/vpu_dbg.c +3 -2

                      vq->last_buffer_dequeued);
      if (seq_write(s, str, num))
              return 0;
-     for (i = 0; i < inst->out_format.num_planes; i++) {
+     for (i = 0; i < inst->out_format.mem_planes; i++) {
              num = scnprintf(str, sizeof(str), " %d(%d)",
-                             inst->out_format.sizeimage[i],
+
+ vpu_get_fmt_plane_size(&inst->out_format, i),
                              inst->out_format.bytesperline[i]);
              if (seq_write(s, str, num))
                      return 0;

@@ -113,9 +113,9 @@ static int vpu_dbg_instance(struct seq_file *s, void

*data)

drivers/media/platform/amphion/vpu_helpers.c const struct vpu_format

Quoted 18 lines

drivers/media/platform/amphion/vpu_dbg.c +3 -2

                      vq->last_buffer_dequeued);
      if (seq_write(s, str, num))
              return 0;
-     for (i = 0; i < inst->cap_format.num_planes; i++) {
+     for (i = 0; i < inst->cap_format.mem_planes; i++) {
              num = scnprintf(str, sizeof(str), " %d(%d)",
-                             inst->cap_format.sizeimage[i],
+
+ vpu_get_fmt_plane_size(&inst->cap_format, i),
                              inst->cap_format.bytesperline[i]);
              if (seq_write(s, str, num))
                      return 0;

drivers/media/platform/amphion/vpu_helpers.c +0 -0

index e9aeb3453dfc..db8f471b9187 100644
@@ -59,6 +59,39 @@ const struct vpu_format

*vpu_helper_find_format(struct vpu_inst *inst, u32 type,

drivers/media/platform/amphion/vpu_helpers.c static u32 get_nv12_plane_size(u32 width, u32

Quoted 40 lines

drivers/media/platform/amphion/vpu_helpers.c +33 -0

      return NULL;
 }
 
+const struct vpu_format *vpu_helper_find_sibling(struct vpu_inst
+*inst, u32 type, u32 pixelfmt) {
+     const struct vpu_format *fmt;
+     const struct vpu_format *sibling;
+
+     fmt = vpu_helper_find_format(inst, type, pixelfmt);
+     if (!fmt)
+             return NULL;
+     if (!fmt->sibling)
+             return NULL;
+     sibling = vpu_helper_find_format(inst, type, fmt->sibling);
+     if (!sibling)
+             return NULL;
+     if (sibling->sibling != fmt->pixfmt)
+             return NULL;
+     if (sibling->comp_planes != fmt->comp_planes)
+             return NULL;
+     return sibling;
+}
+
+bool vpu_helper_match_format(struct vpu_inst *inst, u32 type, u32
+fmta, u32 fmtb) {
+     const struct vpu_format *sibling;
+
+     if (fmta == fmtb)
+             return true;
+
+     sibling = vpu_helper_find_sibling(inst, type, fmta);
+     if (sibling && sibling->pixfmt == fmtb)
+             return true;
+     return false;
+}
+
 const struct vpu_format *vpu_helper_enum_format(struct vpu_inst
 *inst, u32 type, int index)  {
      const struct vpu_format *pfmt;

@@ -123,9 +156,10 @@ static u32 get_nv12_plane_size(u32 width, u32

height, int plane_no,

drivers/media/platform/amphion/vpu_helpers.c u32 vpu_helper_get_plane_size(u32 fmt, u32 w, u32

Quoted 43 lines

drivers/media/platform/amphion/vpu_helpers.c +2 -1

      u32 bytesperline;
      u32 size = 0;
 
-     bytesperline = ALIGN(width, stride);
+     bytesperline = width;
      if (pbl)
              bytesperline = max(bytesperline, *pbl);
+     bytesperline = ALIGN(bytesperline, stride);
      height = ALIGN(height, 2);
      if (plane_no == 0)
              size = bytesperline * height; @@ -148,13 +182,13 @@

static u32 get_tiled_8l128_plane_size(u32 fmt, u32 width, u32 height, int plane_

      if (interlaced)
              hs++;

drivers/media/platform/amphion/vpu_helpers.c +4 -3

-     if (fmt == V4L2_PIX_FMT_NV12M_10BE_8L128)
+     if (fmt == V4L2_PIX_FMT_NV12M_10BE_8L128 || fmt ==
+ V4L2_PIX_FMT_NV12_10BE_8L128)
              bitdepth = 10;
      bytesperline = DIV_ROUND_UP(width * bitdepth, BITS_PER_BYTE);
-     bytesperline = ALIGN(bytesperline, 1 << ws);
-     bytesperline = ALIGN(bytesperline, stride);
      if (pbl)
              bytesperline = max(bytesperline, *pbl);
+     bytesperline = ALIGN(bytesperline, 1 << ws);
+     bytesperline = ALIGN(bytesperline, stride);
      height = ALIGN(height, 1 << hs);
      if (plane_no == 0)
              size = bytesperline * height; @@ -172,9 +206,10 @@

static u32 get_default_plane_size(u32 width, u32 height, int plane_no,

      u32 bytesperline;
      u32 size = 0;

drivers/media/platform/amphion/vpu_helpers.c +2 -1

-     bytesperline = ALIGN(width, stride);
+     bytesperline = width;
      if (pbl)
              bytesperline = max(bytesperline, *pbl);
+     bytesperline = ALIGN(bytesperline, stride);
      if (plane_no == 0)
              size = bytesperline * height;
      if (pbl)

@@ -187,9 +222,12 @@ u32 vpu_helper_get_plane_size(u32 fmt, u32 w, u32

h, int plane_no,

drivers/media/platform/amphion/vpu_helpers.c u32 vpu_helper_get_plane_size(u32 fmt, u32 w, u32 h, int plane_no,

Quoted 11 lines

drivers/media/platform/amphion/vpu_helpers.c +3 -0

                            u32 stride, u32 interlaced, u32 *pbl)  {
      switch (fmt) {
+     case V4L2_PIX_FMT_NV12:
      case V4L2_PIX_FMT_NV12M:
              return get_nv12_plane_size(w, h, plane_no, stride,
 interlaced, pbl);
+     case V4L2_PIX_FMT_NV12_8L128:
      case V4L2_PIX_FMT_NV12M_8L128:
+     case V4L2_PIX_FMT_NV12_10BE_8L128:
      case V4L2_PIX_FMT_NV12M_10BE_8L128:
              return get_tiled_8l128_plane_size(fmt, w, h, plane_no, stride,

interlaced, pbl);

drivers/media/platform/amphion/vpu_malone.c bool vpu_malone_check_fmt(enum vpu_core_type

Quoted 24 lines

drivers/media/platform/amphion/vpu_helpers.c +0 -0
      default:
drivers/media/platform/amphion/vpu_helpers.h +0 -0
index bc28350958be..0eaddb07190d 100644
@@ -14,6 +14,8 @@ struct vpu_pair {
 int vpu_helper_find_in_array_u8(const u8 *array, u32 size, u32 x);
bool vpu_helper_check_type(struct vpu_inst *inst, u32 type); const struct vpu_format *vpu_helper_find_format(struct vpu_inst *inst, u32 type, u32 pixelfmt);
drivers/media/platform/amphion/vpu_helpers.h +3 -0
+const struct vpu_format *vpu_helper_find_sibling(struct vpu_inst
+*inst, u32 type, u32 pixelfmt); bool vpu_helper_match_format(struct
+vpu_inst *inst, u32 type, u32 fmta, u32 fmtb);
 const struct vpu_format *vpu_helper_enum_format(struct vpu_inst
*inst, u32 type, int index);
u32 vpu_helper_valid_frame_width(struct vpu_inst *inst, u32 width); u32 vpu_helper_valid_frame_height(struct vpu_inst *inst, u32 height);
drivers/media/platform/amphion/vpu_malone.c +0 -0
index 51e0702f9ae1..69d9a2269fce 100644
@@ -583,7 +583,8 @@ bool vpu_malone_check_fmt(enum vpu_core_type

type, u32 pixelfmt)

drivers/media/platform/amphion/vpu_malone.c bool vpu_malone_check_fmt(enum vpu_core_type type, u32 pixelfmt)

Quoted 4 lines

drivers/media/platform/amphion/vpu_malone.c +0 -1

      if (!vpu_imx8q_check_fmt(type, pixelfmt))
              return false;
 
-     if (pixelfmt == V4L2_PIX_FMT_NV12M_8L128 || pixelfmt ==

V4L2_PIX_FMT_NV12M_10BE_8L128)

drivers/media/platform/amphion/vpu_malone.c bool vpu_malone_check_fmt(enum vpu_core_type · line 586

Quoted 1 line

drivers/media/platform/amphion/vpu_malone.c +1 -0
+     if (pixelfmt == V4L2_PIX_FMT_NV12_8L128 || pixelfmt ==

V4L2_PIX_FMT_NV12_10BE_8L128 ||

Quoted 36 lines

drivers/media/platform/amphion/vpu_malone.c +2 -0

+         pixelfmt == V4L2_PIX_FMT_NV12M_8L128 || pixelfmt ==
+ V4L2_PIX_FMT_NV12M_10BE_8L128)
              return true;
      if (vpu_malone_format_remap(pixelfmt) == MALONE_FMT_NULL)
              return false;

drivers/media/platform/amphion/vpu_v4l2.c +17 -2

index 8a3eed957ae6..87cce552e906 100644
@@ -140,51 +140,135 @@ bool vpu_is_source_empty(struct vpu_inst *inst)
      return true;
 }
 
-const struct vpu_format *vpu_try_fmt_common(struct vpu_inst *inst,
 struct v4l2_format *f)
+static int vpu_init_format(struct vpu_inst *inst, struct vpu_format
+*fmt) {
+     const struct vpu_format *info;
+
+     info = vpu_helper_find_format(inst, fmt->type, fmt->pixfmt);
+     if (!info) {
+             info = vpu_helper_enum_format(inst, fmt->type, 0);
+             if (!info)
+                     return -EINVAL;
+     }
+     memcpy(fmt, info, sizeof(*fmt));
+
+     return 0;
+}
+
+static int vpu_calc_fmt_bytesperline(struct v4l2_format *f, struct
+vpu_format *fmt)
 {
      struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
-     u32 type = f->type;

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 172

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +10 -0

+     int i;
+
+     if (fmt->flags & V4L2_FMT_FLAG_COMPRESSED) {
+             for (i = 0; i < fmt->comp_planes; i++)
+                     fmt->bytesperline[i] = 0;
+             return 0;
+     }
+     if (pixmp->num_planes == fmt->comp_planes) {
+             for (i = 0; i < fmt->comp_planes; i++)
+                     fmt->bytesperline[i] =

pixmp->plane_fmt[i].bytesperline;

Quoted 32 lines

drivers/media/platform/amphion/vpu_v4l2.c +19 ~~-10~~

+             return 0;
+     }
+     if (pixmp->num_planes > 1)
+             return -EINVAL;
+
+     /*amphion vpu only support nv12 and nv12 tiled,
+      * so the bytesperline of luma and chroma should be same
+      */
+     for (i = 0; i < fmt->comp_planes; i++)
+             fmt->bytesperline[i] = pixmp->plane_fmt[0].bytesperline;
+
+     return 0;
+}
+
+static int vpu_calc_fmt_sizeimage(struct vpu_inst *inst, struct
+vpu_format *fmt) {
      u32 stride = 1;
-     u32 bytesperline;
-     u32 sizeimage;
-     const struct vpu_format *fmt;
-     const struct vpu_core_resources *res;
      int i;
 
-     fmt = vpu_helper_find_format(inst, type, pixmp->pixelformat);
-     if (!fmt) {
-             fmt = vpu_helper_enum_format(inst, type, 0);
-             if (!fmt)
-                     return NULL;
-             pixmp->pixelformat = fmt->pixfmt;
+     if (!(fmt->flags & V4L2_FMT_FLAG_COMPRESSED)) {
+             const struct vpu_core_resources *res =
+ vpu_get_resource(inst);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -5

+
+             if (res)
+                     stride = res->stride;
      }
 
-     res = vpu_get_resource(inst);
-     if (res)
-             stride = res->stride;
-     if (pixmp->width)
-             pixmp->width = vpu_helper_valid_frame_width(inst,

pixmp->width);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 169

Quoted 2 lines

drivers/media/platform/amphion/vpu_v4l2.c +0 -2

-     if (pixmp->height)
-             pixmp->height = vpu_helper_valid_frame_height(inst,

pixmp->height);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 211

Quoted 3 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -0

+     for (i = 0; i < fmt->comp_planes; i++) {
+             fmt->sizeimage[i] = vpu_helper_get_plane_size(fmt->pixfmt,
+

fmt->width,

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 203

Quoted 1 line

drivers/media/platform/amphion/vpu_v4l2.c +1 -0
+

fmt->height,

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 211

Quoted 3 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -0

+                                                           i,
+                                                           stride,
+

fmt->field != V4L2_FIELD_NONE ? 1 : 0,

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 207

Quoted 1 line

drivers/media/platform/amphion/vpu_v4l2.c +1 -0
+

&fmt->bytesperline[i]);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 3 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -0

+             fmt->sizeimage[i] = max_t(u32, fmt->sizeimage[i], PAGE_SIZE);
+             if (fmt->flags & V4L2_FMT_FLAG_COMPRESSED) {
+                     fmt->sizeimage[i] = clamp_val(fmt->sizeimage[i],

SZ_128K, SZ_8M);

Quoted 31 lines

drivers/media/platform/amphion/vpu_v4l2.c +31 -0

+                     fmt->bytesperline[i] = 0;
+             }
+     }
+
+     return 0;
+}
+
+u32 vpu_get_fmt_plane_size(struct vpu_format *fmt, u32 plane_no) {
+     u32 size;
+     int i;
+
+     if (plane_no >= fmt->mem_planes)
+             return 0;
+
+     if (fmt->comp_planes == fmt->mem_planes)
+             return fmt->sizeimage[plane_no];
+     if (plane_no < fmt->mem_planes - 1)
+             return fmt->sizeimage[plane_no];
+     size = fmt->sizeimage[plane_no];
+     for (i = fmt->mem_planes; i < fmt->comp_planes; i++)
+             size += fmt->sizeimage[i];
+
+     return size;
+}
+
+int vpu_try_fmt_common(struct vpu_inst *inst, struct v4l2_format *f,
+struct vpu_format *fmt) {
+     struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
+     int i;
+     int ret;
+

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +10 -0

+     fmt->pixfmt = pixmp->pixelformat;
+     fmt->type = f->type;
+     ret = vpu_init_format(inst, fmt);
+     if (ret < 0)
+             return ret;
+
+     fmt->width = pixmp->width;
+     fmt->height = pixmp->height;
+     if (fmt->width)
+             fmt->width = vpu_helper_valid_frame_width(inst,

fmt->width);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 253

Quoted 2 lines

drivers/media/platform/amphion/vpu_v4l2.c +2 -0

+     if (fmt->height)
+             fmt->height = vpu_helper_valid_frame_height(inst,

fmt->height);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 254

Quoted 1 line

drivers/media/platform/amphion/vpu_v4l2.c +1 -0
+     fmt->field = pixmp->field == V4L2_FIELD_ANY ? V4L2_FIELD_NONE :

pixmp->field;

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 3 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -0

+     vpu_calc_fmt_bytesperline(f, fmt);
+     vpu_calc_fmt_sizeimage(inst, fmt);
+     if ((fmt->flags & V4L2_FMT_FLAG_COMPRESSED) &&

pixmp->plane_fmt[0].sizeimage)

Quoted 5 lines

drivers/media/platform/amphion/vpu_v4l2.c +5 -0

+             fmt->sizeimage[0] =
+ clamp_val(pixmp->plane_fmt[0].sizeimage, SZ_128K, SZ_8M);
+
+     pixmp->pixelformat = fmt->pixfmt;
+     pixmp->width = fmt->width;

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +4 -4

+     pixmp->height = fmt->height;
      pixmp->flags = fmt->flags;
-     pixmp->num_planes = fmt->num_planes;
-     if (pixmp->field == V4L2_FIELD_ANY)
-             pixmp->field = V4L2_FIELD_NONE;
+     pixmp->num_planes = fmt->mem_planes;
+     pixmp->field = fmt->field;
+     memset(pixmp->reserved, 0, sizeof(pixmp->reserved));
      for (i = 0; i < pixmp->num_planes; i++) {
-             bytesperline = max_t(s32, pixmp->plane_fmt[i].bytesperline,

0);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 6 lines

drivers/media/platform/amphion/vpu_v4l2.c +0 -6

-             sizeimage = vpu_helper_get_plane_size(pixmp->pixelformat,
-                                                   pixmp->width,
-                                                   pixmp->height,
-                                                   i,
-                                                   stride,
-                                                   pixmp->field >

V4L2_FIELD_NONE ? 1 : 0,

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 183

Quoted 2 lines

drivers/media/platform/amphion/vpu_v4l2.c +0 -2

-                                                   &bytesperline);
-             sizeimage = max_t(s32, pixmp->plane_fmt[i].sizeimage,

sizeimage);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst)

Quoted 4 lines

drivers/media/platform/amphion/vpu_v4l2.c +2 -2

-             pixmp->plane_fmt[i].bytesperline = bytesperline;
-             pixmp->plane_fmt[i].sizeimage = sizeimage;
+             pixmp->plane_fmt[i].bytesperline = fmt->bytesperline[i];
+             pixmp->plane_fmt[i].sizeimage = vpu_get_fmt_plane_size(fmt,

i);

drivers/media/platform/amphion/vpu_v4l2.c static int vpu_vb2_queue_setup(struct vb2_queue

Quoted 25 lines

drivers/media/platform/amphion/vpu_v4l2.c +7 -5

+             memset(pixmp->plane_fmt[i].reserved, 0,
+ sizeof(pixmp->plane_fmt[i].reserved));
      }
 
-     return fmt;
+     return 0;
 }
 
 static bool vpu_check_ready(struct vpu_inst *inst, u32 type) @@
-389,10 +473,10 @@ static int vpu_vb2_queue_setup(struct vb2_queue *vq,
      cur_fmt = vpu_get_format(inst, vq->type);
 
      if (*plane_count) {
-             if (*plane_count != cur_fmt->num_planes)
+             if (*plane_count != cur_fmt->mem_planes)
                      return -EINVAL;
-             for (i = 0; i < cur_fmt->num_planes; i++) {
-                     if (psize[i] < cur_fmt->sizeimage[i])
+             for (i = 0; i < cur_fmt->mem_planes; i++) {
+                     if (psize[i] < vpu_get_fmt_plane_size(cur_fmt,
+ i))
                              return -EINVAL;
              }
              return 0;

@@ -402,9 +486,9 @@ static int vpu_vb2_queue_setup(struct vb2_queue

*vq,

drivers/media/platform/amphion/vpu_v4l2.c static int vpu_vb2_queue_setup(struct vb2_queue · line 486

Quoted 1 line

drivers/media/platform/amphion/vpu_v4l2.c +0 -0
              *buf_count = max_t(unsigned int, *buf_count,

inst->min_buffer_out);

drivers/media/platform/amphion/vpu_v4l2.c static int vpu_vb2_queue_setup(struct vb2_queue · line 488

Quoted 2 lines

drivers/media/platform/amphion/vpu_v4l2.c +0 -0

      else
              *buf_count = max_t(unsigned int, *buf_count,

inst->min_buffer_cap);

drivers/media/platform/amphion/vpu_v4l2.c static int vpu_vb2_buf_prepare(struct vb2_buffer

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -3

-     *plane_count = cur_fmt->num_planes;
-     for (i = 0; i < cur_fmt->num_planes; i++)
-             psize[i] = cur_fmt->sizeimage[i];
+     *plane_count = cur_fmt->mem_planes;
+     for (i = 0; i < cur_fmt->mem_planes; i++)
+             psize[i] = vpu_get_fmt_plane_size(cur_fmt, i);
 
      return 0;
 }

@@ -434,8 +518,8 @@ static int vpu_vb2_buf_prepare(struct vb2_buffer

*vb)

drivers/media/platform/amphion/vpu_v4l2.c static int vpu_vb2_buf_prepare(struct vb2_buffer *vb)

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +3 -2

      u32 i;
 
      cur_fmt = vpu_get_format(inst, vb->type);
-     for (i = 0; i < cur_fmt->num_planes; i++) {
-             if (vpu_get_vb_length(vb, i) < cur_fmt->sizeimage[i]) {
+     for (i = 0; i < cur_fmt->mem_planes; i++) {
+             if (vpu_get_vb_length(vb, i) <
+ vpu_get_fmt_plane_size(cur_fmt, i)) {
                      dev_dbg(inst->dev, "[%d] %s buf[%d] is invalid\n",
                              inst->id, vpu_type_name(vb->type),

vb->index);

Quoted 1 line

vpu_set_buffer_state(vbuf,

VPU_BUF_STATE_ERROR);

drivers/media/platform/amphion/vpu_windsor.c static int vpu_windsor_fill_yuv_frame(struct

Quoted 23 lines

drivers/media/platform/amphion/vpu_v4l2.h +0 -0
index 795ca33a6a50..ef5de6b66e47 100644
@@ -16,7 +16,8 @@ unsigned int vpu_get_buffer_state(struct
vb2_v4l2_buffer *vbuf); int vpu_v4l2_open(struct file *file, struct vpu_inst *inst); int vpu_v4l2_close(struct file *file);
-const struct vpu_format *vpu_try_fmt_common(struct vpu_inst *inst, struct v4l2_format *f);
drivers/media/platform/amphion/vpu_v4l2.h +3 -0
+u32 vpu_get_fmt_plane_size(struct vpu_format *fmt, u32 plane_no); int
+vpu_try_fmt_common(struct vpu_inst *inst, struct v4l2_format *f,
+struct vpu_format *fmt);
 int vpu_process_output_buffer(struct vpu_inst *inst);  int
vpu_process_capture_buffer(struct vpu_inst *inst); struct vb2_v4l2_buffer *vpu_next_src_buf(struct vpu_inst *inst); diff --git a/drivers/media/platform/amphion/vpu_windsor.c b/drivers/media/platform/amphion/vpu_windsor.c
index 1526af2ef9da..a454f142ae17 100644
drivers/media/platform/amphion/vpu_windsor.c +0 -0
@@ -775,6 +775,8 @@ static int vpu_windsor_fill_yuv_frame(struct

vpu_shared_addr *shared,

drivers/media/platform/amphion/vpu_windsor.c static int vpu_windsor_fill_yuv_frame(struct

Quoted 8 lines

drivers/media/platform/amphion/vpu_windsor.c +2 -0

                                    u32 instance,
                                    struct vb2_buffer *vb)  {
+     struct vpu_inst *inst = vb2_get_drv_priv(vb->vb2_queue);
+     struct vpu_format *out_fmt;
      struct vpu_enc_yuv_desc *desc;
      struct vb2_v4l2_buffer *vbuf;

@@ -782,6 +784,7 @@ static int vpu_windsor_fill_yuv_frame(struct

vpu_shared_addr *shared,

Quoted 21 lines

drivers/media/platform/amphion/vpu_windsor.c +1 -0

              return -EINVAL;
 
      desc = get_yuv_desc(shared, instance);
+     out_fmt = vpu_get_format(inst, vb->type);
 
      vbuf = to_vb2_v4l2_buffer(vb);
      desc->frame_id = vbuf->sequence; @@ -790,7 +793,10 @@ static int

vpu_windsor_fill_yuv_frame(struct vpu_shared_addr *shared,

      else
              desc->key_frame = 0;
      desc->luma_base = vpu_get_vb_phy_addr(vb, 0);

drivers/media/platform/amphion/vpu_windsor.c +5 -2

-     desc->chroma_base = vpu_get_vb_phy_addr(vb, 1);
+     if (vb->num_planes > 1)
+             desc->chroma_base = vpu_get_vb_phy_addr(vb, 1);
+     else
+             desc->chroma_base = desc->luma_base +
+ out_fmt->sizeimage[0];
 
      return 0;
 }
--

Quoted 2 lines

2.37.1

--
Tommaso Merciai
Embedded Linux Engineer
[email protected]
__________________________________

Signature · 10 lines

Tommaso Merciai
Embedded Linux Engineer
[email protected]
__________________________________

Amarula Solutions SRL
Via Le Canevare 30, 31100 Treviso, Veneto, IT
T. +39 042 243 5310
[email protected]
www.amarulasolutions.com

Tommaso Merciai

2022-08-22 11:17 UTC #13 lore raw

Re: [PATCH v4 2/4] media: amphion: tell and handle contiguous and non contiguous format

In reply to Ming Qian

Hi Ming,
Sorry for delay. I'm on vacation last week :)

Quoted 555 lines · Ming Qian

So driver can support NV12 and NV12M.

Signed-off-by Ming Qian <[email protected]>

---
 drivers/media/platform/amphion/vdec.c        | 187 ++++++++++---------
 drivers/media/platform/amphion/venc.c        |  33 ++--
 drivers/media/platform/amphion/vpu.h         |   4 +-
 drivers/media/platform/amphion/vpu_dbg.c     |   8 +-
 drivers/media/platform/amphion/vpu_helpers.c |  48 ++++-
 drivers/media/platform/amphion/vpu_helpers.h |   2 +
 drivers/media/platform/amphion/vpu_malone.c  |   3 +-
 drivers/media/platform/amphion/vpu_v4l2.c    | 168 ++++++++++++-----
 drivers/media/platform/amphion/vpu_v4l2.h    |   3 +-
 drivers/media/platform/amphion/vpu_windsor.c |   8 +-
 10 files changed, 299 insertions(+), 165 deletions(-)

drivers/media/platform/amphion/vdec.c +98 ~~-89~~

index feb75dc204de..48ab664fa7ef 100644
@@ -69,72 +69,85 @@ struct vdec_t {
 static const struct vpu_format vdec_formats[] = {
 	{
 		.pixfmt = V4L2_PIX_FMT_NV12M_8L128,
-		.num_planes = 2,
+		.mem_planes = 2,
+		.comp_planes = 2,
 		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_NV12M_10BE_8L128,
-		.num_planes = 2,
+		.mem_planes = 2,
+		.comp_planes = 2,
 		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_H264,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_H264_MVC,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_HEVC,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_VC1_ANNEX_G,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_VC1_ANNEX_L,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+		.flags = V4L2_FMT_FLAG_COMPRESSED
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_MPEG2,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_MPEG4,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_XVID,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_VP8,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_H263,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION
+		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
 	},
 	{0, 0, 0, 0},
 };

@@ -256,23 +269,22 @@ static int vdec_enum_fmt(struct file *file, void *fh, struct v4l2_fmtdesc *f)
 	int ret = -EINVAL;
 
 	vpu_inst_lock(inst);
-	if (!V4L2_TYPE_IS_OUTPUT(f->type) && vdec->fixed_fmt) {
-		if (f->index == 0) {
-			f->pixelformat = inst->cap_format.pixfmt;
-			f->flags = inst->cap_format.flags;
-			ret = 0;
-		}
+	if (V4L2_TYPE_IS_CAPTURE(f->type) && vdec->fixed_fmt) {
+		fmt = vpu_get_format(inst, f->type);
+		if (f->index == 1)
+			fmt = vpu_helper_find_sibling(inst, f->type, fmt->pixfmt);
+		if (f->index > 1)
+			fmt = NULL;
 	} else {
 		fmt = vpu_helper_enum_format(inst, f->type, f->index);
-		memset(f->reserved, 0, sizeof(f->reserved));
-		if (!fmt)
-			goto exit;
-
-		f->pixelformat = fmt->pixfmt;
-		f->flags = fmt->flags;
-		ret = 0;
 	}
+	if (!fmt)
+		goto exit;
 
+	memset(f->reserved, 0, sizeof(f->reserved));
+	f->pixelformat = fmt->pixfmt;
+	f->flags = fmt->flags;
+	ret = 0;
 exit:
 	vpu_inst_unlock(inst);
 	return ret;

@@ -289,14 +301,14 @@ static int vdec_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
 	cur_fmt = vpu_get_format(inst, f->type);
 
 	pixmp->pixelformat = cur_fmt->pixfmt;
-	pixmp->num_planes = cur_fmt->num_planes;
+	pixmp->num_planes = cur_fmt->mem_planes;
 	pixmp->width = cur_fmt->width;
 	pixmp->height = cur_fmt->height;
 	pixmp->field = cur_fmt->field;
 	pixmp->flags = cur_fmt->flags;
 	for (i = 0; i < pixmp->num_planes; i++) {
 		pixmp->plane_fmt[i].bytesperline = cur_fmt->bytesperline[i];
-		pixmp->plane_fmt[i].sizeimage = cur_fmt->sizeimage[i];
+		pixmp->plane_fmt[i].sizeimage = vpu_get_fmt_plane_size(cur_fmt, i);
 	}
 
 	f->fmt.pix_mp.colorspace = vdec->codec_info.color_primaries;

@@ -311,10 +323,19 @@ static int vdec_try_fmt(struct file *file, void *fh, struct v4l2_format *f)
 {
 	struct vpu_inst *inst = to_inst(file);
 	struct vdec_t *vdec = inst->priv;
-
-	vpu_try_fmt_common(inst, f);
+	struct vpu_format fmt;
 
 	vpu_inst_lock(inst);
+	if (V4L2_TYPE_IS_CAPTURE(f->type) && vdec->fixed_fmt) {
+		struct vpu_format *cap_fmt = vpu_get_format(inst, f->type);
+
+		if (!vpu_helper_match_format(inst, cap_fmt->type, cap_fmt->pixfmt,
+					     f->fmt.pix_mp.pixelformat))
+			f->fmt.pix_mp.pixelformat = cap_fmt->pixfmt;
+	}
+
+	vpu_try_fmt_common(inst, f, &fmt);
+
 	if (vdec->fixed_fmt) {
 		f->fmt.pix_mp.colorspace = vdec->codec_info.color_primaries;
 		f->fmt.pix_mp.xfer_func = vdec->codec_info.transfer_chars;

@@ -334,7 +355,7 @@ static int vdec_try_fmt(struct file *file, void *fh, struct v4l2_format *f)
 static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)
 {
 	struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
-	const struct vpu_format *fmt;
+	struct vpu_format fmt;
 	struct vpu_format *cur_fmt;
 	struct vb2_queue *q;
 	struct vdec_t *vdec = inst->priv;

@@ -349,36 +370,30 @@ static int vdec_s_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)
 	if (vb2_is_busy(q))
 		return -EBUSY;
 
-	fmt = vpu_try_fmt_common(inst, f);
-	if (!fmt)
+	if (vpu_try_fmt_common(inst, f, &fmt))
 		return -EINVAL;
 
 	cur_fmt = vpu_get_format(inst, f->type);
 	if (V4L2_TYPE_IS_OUTPUT(f->type) && inst->state != VPU_CODEC_STATE_DEINIT) {
-		if (cur_fmt->pixfmt != fmt->pixfmt) {
+		if (cur_fmt->pixfmt != fmt.pixfmt) {
 			vdec->reset_codec = true;
 			vdec->fixed_fmt = false;
 		}
 	}
-	cur_fmt->pixfmt = fmt->pixfmt;
 	if (V4L2_TYPE_IS_OUTPUT(f->type) || !vdec->fixed_fmt) {
-		cur_fmt->num_planes = fmt->num_planes;
-		cur_fmt->flags = fmt->flags;
-		cur_fmt->width = pixmp->width;
-		cur_fmt->height = pixmp->height;
-		for (i = 0; i < fmt->num_planes; i++) {
-			cur_fmt->sizeimage[i] = pixmp->plane_fmt[i].sizeimage;
-			cur_fmt->bytesperline[i] = pixmp->plane_fmt[i].bytesperline;
-		}
-		if (pixmp->field != V4L2_FIELD_ANY)
-			cur_fmt->field = pixmp->field;
+		memcpy(cur_fmt, &fmt, sizeof(*cur_fmt));
 	} else {
-		pixmp->num_planes = cur_fmt->num_planes;
+		if (vpu_helper_match_format(inst, f->type, cur_fmt->pixfmt, pixmp->pixelformat)) {
+			cur_fmt->pixfmt = fmt.pixfmt;
+			cur_fmt->mem_planes = fmt.mem_planes;
+		}
+		pixmp->pixelformat = cur_fmt->pixfmt;
+		pixmp->num_planes = cur_fmt->mem_planes;
 		pixmp->width = cur_fmt->width;
 		pixmp->height = cur_fmt->height;
 		for (i = 0; i < pixmp->num_planes; i++) {
 			pixmp->plane_fmt[i].bytesperline = cur_fmt->bytesperline[i];
-			pixmp->plane_fmt[i].sizeimage = cur_fmt->sizeimage[i];
+			pixmp->plane_fmt[i].sizeimage = vpu_get_fmt_plane_size(cur_fmt, i);
 		}
 		pixmp->field = cur_fmt->field;
 	}

@@ -678,9 +693,11 @@ static struct vpu_vb2_buffer *vdec_find_buffer(struct vpu_inst *inst, u32 luma)
 static void vdec_buf_done(struct vpu_inst *inst, struct vpu_frame_info *frame)
 {
 	struct vdec_t *vdec = inst->priv;
+	struct vpu_format *cur_fmt;
 	struct vpu_vb2_buffer *vpu_buf;
 	struct vb2_v4l2_buffer *vbuf;
 	u32 sequence;
+	int i;
 
 	if (!frame)
 		return;

@@ -699,6 +716,7 @@ static void vdec_buf_done(struct vpu_inst *inst, struct vpu_frame_info *frame)
 		return;
 	}
 
+	cur_fmt = vpu_get_format(inst, inst->cap_format.type);
 	vbuf = &vpu_buf->m2m_buf.vb;
 	if (vbuf->vb2_buf.index != frame->id)
 		dev_err(inst->dev, "[%d] buffer id(%d, %d) dismatch\n",

@@ -707,9 +725,9 @@ static void vdec_buf_done(struct vpu_inst *inst, struct vpu_frame_info *frame)
 	if (vpu_get_buffer_state(vbuf) != VPU_BUF_STATE_DECODED)
 		dev_err(inst->dev, "[%d] buffer(%d) ready without decoded\n", inst->id, frame->id);
 	vpu_set_buffer_state(vbuf, VPU_BUF_STATE_READY);
-	vb2_set_plane_payload(&vbuf->vb2_buf, 0, inst->cap_format.sizeimage[0]);
-	vb2_set_plane_payload(&vbuf->vb2_buf, 1, inst->cap_format.sizeimage[1]);
-	vbuf->field = inst->cap_format.field;
+	for (i = 0; i < vbuf->vb2_buf.num_planes; i++)
+		vb2_set_plane_payload(&vbuf->vb2_buf, i, vpu_get_fmt_plane_size(cur_fmt, i));
+	vbuf->field = cur_fmt->field;
 	vbuf->sequence = sequence;
 	dev_dbg(inst->dev, "[%d][OUTPUT TS]%32lld\n", inst->id, vbuf->vb2_buf.timestamp);

@@ -747,15 +765,17 @@ static void vdec_stop_done(struct vpu_inst *inst)
 static bool vdec_check_source_change(struct vpu_inst *inst)
 {
 	struct vdec_t *vdec = inst->priv;
-	const struct vpu_format *fmt;
-	int i;
+	const struct vpu_format *sibling;
 
 	if (!inst->fh.m2m_ctx)
 		return false;
 
+	sibling = vpu_helper_find_sibling(inst, inst->cap_format.type, inst->cap_format.pixfmt);
+	if (sibling && vdec->codec_info.pixfmt == sibling->pixfmt)
+		vdec->codec_info.pixfmt = inst->cap_format.pixfmt;
+
 	if (!vb2_is_streaming(v4l2_m2m_get_dst_vq(inst->fh.m2m_ctx)))
 		return true;
-	fmt = vpu_helper_find_format(inst, inst->cap_format.type, vdec->codec_info.pixfmt);
 	if (inst->cap_format.pixfmt != vdec->codec_info.pixfmt)
 		return true;
 	if (inst->cap_format.width != vdec->codec_info.decoded_width)

@@ -772,14 +792,6 @@ static bool vdec_check_source_change(struct vpu_inst *inst)
 		return true;
 	if (inst->crop.height != vdec->codec_info.height)
 		return true;
-	if (fmt && inst->cap_format.num_planes != fmt->num_planes)
-		return true;
-	for (i = 0; i < inst->cap_format.num_planes; i++) {
-		if (inst->cap_format.bytesperline[i] != vdec->codec_info.bytesperline[i])
-			return true;
-		if (inst->cap_format.sizeimage[i] != vdec->codec_info.sizeimage[i])
-			return true;
-	}
 
 	return false;
 }

@@ -787,27 +799,21 @@ static bool vdec_check_source_change(struct vpu_inst *inst)
 static void vdec_init_fmt(struct vpu_inst *inst)
 {
 	struct vdec_t *vdec = inst->priv;
-	const struct vpu_format *fmt;
-	int i;
+	struct v4l2_format f;
 
-	fmt = vpu_helper_find_format(inst, inst->cap_format.type, vdec->codec_info.pixfmt);
-	inst->out_format.width = vdec->codec_info.width;
-	inst->out_format.height = vdec->codec_info.height;
-	inst->cap_format.width = vdec->codec_info.decoded_width;
-	inst->cap_format.height = vdec->codec_info.decoded_height;
-	inst->cap_format.pixfmt = vdec->codec_info.pixfmt;
-	if (fmt) {
-		inst->cap_format.num_planes = fmt->num_planes;
-		inst->cap_format.flags = fmt->flags;
-	}
-	for (i = 0; i < inst->cap_format.num_planes; i++) {
-		inst->cap_format.bytesperline[i] = vdec->codec_info.bytesperline[i];
-		inst->cap_format.sizeimage[i] = vdec->codec_info.sizeimage[i];
-	}
+	memset(&f, 0, sizeof(f));
+	f.type = inst->cap_format.type;
+	f.fmt.pix_mp.pixelformat = vdec->codec_info.pixfmt;
+	f.fmt.pix_mp.width = vdec->codec_info.decoded_width;
+	f.fmt.pix_mp.height = vdec->codec_info.decoded_height;
 	if (vdec->codec_info.progressive)
-		inst->cap_format.field = V4L2_FIELD_NONE;
+		f.fmt.pix_mp.field = V4L2_FIELD_NONE;
 	else
-		inst->cap_format.field = V4L2_FIELD_SEQ_TB;
+		f.fmt.pix_mp.field = V4L2_FIELD_SEQ_TB;
+	vpu_try_fmt_common(inst, &f, &inst->cap_format);
+
+	inst->out_format.width = vdec->codec_info.width;
+	inst->out_format.height = vdec->codec_info.height;
 }
 
 static void vdec_init_crop(struct vpu_inst *inst)

@@ -966,7 +972,10 @@ static int vdec_response_frame(struct vpu_inst *inst, struct vb2_v4l2_buffer *vb
 	info.tag = vdec->seq_tag;
 	info.luma_addr = vpu_get_vb_phy_addr(&vbuf->vb2_buf, 0);
 	info.luma_size = inst->cap_format.sizeimage[0];
-	info.chroma_addr = vpu_get_vb_phy_addr(&vbuf->vb2_buf, 1);
+	if (vbuf->vb2_buf.num_planes > 1)
+		info.chroma_addr = vpu_get_vb_phy_addr(&vbuf->vb2_buf, 1);
+	else
+		info.chroma_addr = info.luma_addr + info.luma_size;
 	info.chromau_size = inst->cap_format.sizeimage[1];
 	info.bytesperline = inst->cap_format.bytesperline[0];
 	ret = vpu_session_alloc_fs(inst, &info);

@@ -975,7 +984,7 @@ static int vdec_response_frame(struct vpu_inst *inst, struct vb2_v4l2_buffer *vb
 
 	vpu_buf->tag = info.tag;
 	vpu_buf->luma = info.luma_addr;
-	vpu_buf->chroma_u = info.chromau_size;
+	vpu_buf->chroma_u = info.chroma_addr;
 	vpu_buf->chroma_v = 0;
 	vpu_set_buffer_state(vbuf, VPU_BUF_STATE_INUSE);
 	vdec->slots[info.id] = vpu_buf;

drivers/media/platform/amphion/venc.c +12 ~~-21~~

index 37212f087fdd..060a1ee78b17 100644
@@ -69,13 +69,16 @@ struct venc_frame_t {
 static const struct vpu_format venc_formats[] = {
 	{
 		.pixfmt = V4L2_PIX_FMT_NV12M,
-		.num_planes = 2,
+		.mem_planes = 2,
+		.comp_planes = 2,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_H264,
-		.num_planes = 1,
+		.mem_planes = 1,
+		.comp_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.flags = V4L2_FMT_FLAG_COMPRESSED
 	},
 	{0, 0, 0, 0},
 };

@@ -173,14 +176,14 @@ static int venc_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
 	cur_fmt = vpu_get_format(inst, f->type);
 
 	pixmp->pixelformat = cur_fmt->pixfmt;
-	pixmp->num_planes = cur_fmt->num_planes;
+	pixmp->num_planes = cur_fmt->mem_planes;
 	pixmp->width = cur_fmt->width;
 	pixmp->height = cur_fmt->height;
 	pixmp->field = cur_fmt->field;
 	pixmp->flags = cur_fmt->flags;
 	for (i = 0; i < pixmp->num_planes; i++) {
 		pixmp->plane_fmt[i].bytesperline = cur_fmt->bytesperline[i];
-		pixmp->plane_fmt[i].sizeimage = cur_fmt->sizeimage[i];
+		pixmp->plane_fmt[i].sizeimage = vpu_get_fmt_plane_size(cur_fmt, i);
 	}
 
 	f->fmt.pix_mp.colorspace = venc->params.color.primaries;

@@ -194,8 +197,9 @@ static int venc_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
 static int venc_try_fmt(struct file *file, void *fh, struct v4l2_format *f)
 {
 	struct vpu_inst *inst = to_inst(file);
+	struct vpu_format fmt;
 
-	vpu_try_fmt_common(inst, f);
+	vpu_try_fmt_common(inst, f, &fmt);
 
 	return 0;
 }

@@ -203,12 +207,11 @@ static int venc_try_fmt(struct file *file, void *fh, struct v4l2_format *f)
 static int venc_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
 {
 	struct vpu_inst *inst = to_inst(file);
-	const struct vpu_format *fmt;
+	struct vpu_format fmt;
 	struct vpu_format *cur_fmt;
 	struct vb2_queue *q;
 	struct venc_t *venc = inst->priv;
 	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
-	int i;
 
 	q = v4l2_m2m_get_vq(inst->fh.m2m_ctx, f->type);
 	if (!q)

@@ -216,24 +219,12 @@ static int venc_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
 	if (vb2_is_busy(q))
 		return -EBUSY;
 
-	fmt = vpu_try_fmt_common(inst, f);
-	if (!fmt)
+	if (vpu_try_fmt_common(inst, f, &fmt))
 		return -EINVAL;
 
 	cur_fmt = vpu_get_format(inst, f->type);
 
-	cur_fmt->pixfmt = fmt->pixfmt;
-	cur_fmt->num_planes = fmt->num_planes;
-	cur_fmt->flags = fmt->flags;
-	cur_fmt->width = pix_mp->width;
-	cur_fmt->height = pix_mp->height;
-	for (i = 0; i < fmt->num_planes; i++) {
-		cur_fmt->sizeimage[i] = pix_mp->plane_fmt[i].sizeimage;
-		cur_fmt->bytesperline[i] = pix_mp->plane_fmt[i].bytesperline;
-	}
-
-	if (pix_mp->field != V4L2_FIELD_ANY)
-		cur_fmt->field = pix_mp->field;
+	memcpy(cur_fmt, &fmt, sizeof(*cur_fmt));
 
 	if (V4L2_TYPE_IS_OUTPUT(f->type)) {
 		venc->params.input_format = cur_fmt->pixfmt;

drivers/media/platform/amphion/vpu.h +3 -1

index f914de6ed81e..998591fa5b85 100644
@@ -84,7 +84,8 @@ struct vpu_dev {
 
 struct vpu_format {
 	u32 pixfmt;
-	unsigned int num_planes;
+	u32 mem_planes;
+	u32 comp_planes;
 	u32 type;
 	u32 flags;
 	u32 width;

@@ -92,6 +93,7 @@ struct vpu_format {
 	u32 sizeimage[VIDEO_MAX_PLANES];
 	u32 bytesperline[VIDEO_MAX_PLANES];
 	u32 field;
+	u32 sibling;
 };
 
 struct vpu_core_resources {

drivers/media/platform/amphion/vpu_dbg.c +4 -4

index f72c8a506b22..c41c9896912c 100644
@@ -89,9 +89,9 @@ static int vpu_dbg_instance(struct seq_file *s, void *data)
 			vq->last_buffer_dequeued);
 	if (seq_write(s, str, num))
 		return 0;
-	for (i = 0; i < inst->out_format.num_planes; i++) {
+	for (i = 0; i < inst->out_format.mem_planes; i++) {
 		num = scnprintf(str, sizeof(str), " %d(%d)",
-				inst->out_format.sizeimage[i],
+				vpu_get_fmt_plane_size(&inst->out_format, i),
 				inst->out_format.bytesperline[i]);
 		if (seq_write(s, str, num))
 			return 0;

@@ -113,9 +113,9 @@ static int vpu_dbg_instance(struct seq_file *s, void *data)
 			vq->last_buffer_dequeued);
 	if (seq_write(s, str, num))
 		return 0;
-	for (i = 0; i < inst->cap_format.num_planes; i++) {
+	for (i = 0; i < inst->cap_format.mem_planes; i++) {
 		num = scnprintf(str, sizeof(str), " %d(%d)",
-				inst->cap_format.sizeimage[i],
+				vpu_get_fmt_plane_size(&inst->cap_format, i),
 				inst->cap_format.bytesperline[i]);
 		if (seq_write(s, str, num))
 			return 0;

drivers/media/platform/amphion/vpu_helpers.c +9 -0

index e9aeb3453dfc..db8f471b9187 100644
@@ -59,6 +59,39 @@ const struct vpu_format *vpu_helper_find_format(struct vpu_inst *inst, u32 type,
 	return NULL;
 }
 
+const struct vpu_format *vpu_helper_find_sibling(struct vpu_inst *inst, u32 type, u32 pixelfmt)
+{
+	const struct vpu_format *fmt;
+	const struct vpu_format *sibling;
+
+	fmt = vpu_helper_find_format(inst, type, pixelfmt);
+	if (!fmt)
+		return NULL;
+	if (!fmt->sibling)

drivers/media/platform/amphion/vpu_helpers.c const struct vpu_format *vpu_helper_find_format(struct vpu_inst *inst, u32 type, · line 80

Quoted 10 lines

drivers/media/platform/amphion/vpu_helpers.c +10 -0

+		return NULL;
+	sibling = vpu_helper_find_format(inst, type, fmt->sibling);
+	if (!sibling)
+		return NULL;
+	if (sibling->sibling != fmt->pixfmt)
+		return NULL;
+	if (sibling->comp_planes != fmt->comp_planes)
+		return NULL;
+	return sibling;
+}

I think we can limit the use of "if" statement here. What about this?

const struct vpu_format *vpu_helper_find_sibling(struct vpu_inst *inst, u32 type, u32 pixelfmt)

{
	const struct vpu_format *fmt;
	const struct vpu_format *sibling;

	fmt = vpu_helper_find_format(inst, type, pixelfmt);
	if (!fmt || !fmt->sibling)
		return NULL;

	sibling = vpu_helper_find_format(inst, type, fmt->sibling);
	if (!sibling || (sibling->sibling != fmt->pixfmt) ||
		(sibling->comp_planes != fmt->comp_planes))
		return NULL;

	return sibling;
}

Quoted 89 lines

drivers/media/platform/amphion/vpu_helpers.c +24 -5

+
+bool vpu_helper_match_format(struct vpu_inst *inst, u32 type, u32 fmta, u32 fmtb)
+{
+	const struct vpu_format *sibling;
+
+	if (fmta == fmtb)
+		return true;
+
+	sibling = vpu_helper_find_sibling(inst, type, fmta);
+	if (sibling && sibling->pixfmt == fmtb)
+		return true;
+	return false;
+}
+
 const struct vpu_format *vpu_helper_enum_format(struct vpu_inst *inst, u32 type, int index)
 {
 	const struct vpu_format *pfmt;

@@ -123,9 +156,10 @@ static u32 get_nv12_plane_size(u32 width, u32 height, int plane_no,
 	u32 bytesperline;
 	u32 size = 0;
 
-	bytesperline = ALIGN(width, stride);
+	bytesperline = width;
 	if (pbl)
 		bytesperline = max(bytesperline, *pbl);
+	bytesperline = ALIGN(bytesperline, stride);
 	height = ALIGN(height, 2);
 	if (plane_no == 0)
 		size = bytesperline * height;

@@ -148,13 +182,13 @@ static u32 get_tiled_8l128_plane_size(u32 fmt, u32 width, u32 height, int plane_
 
 	if (interlaced)
 		hs++;
-	if (fmt == V4L2_PIX_FMT_NV12M_10BE_8L128)
+	if (fmt == V4L2_PIX_FMT_NV12M_10BE_8L128 || fmt == V4L2_PIX_FMT_NV12_10BE_8L128)
 		bitdepth = 10;
 	bytesperline = DIV_ROUND_UP(width * bitdepth, BITS_PER_BYTE);
-	bytesperline = ALIGN(bytesperline, 1 << ws);
-	bytesperline = ALIGN(bytesperline, stride);
 	if (pbl)
 		bytesperline = max(bytesperline, *pbl);
+	bytesperline = ALIGN(bytesperline, 1 << ws);
+	bytesperline = ALIGN(bytesperline, stride);
 	height = ALIGN(height, 1 << hs);
 	if (plane_no == 0)
 		size = bytesperline * height;

@@ -172,9 +206,10 @@ static u32 get_default_plane_size(u32 width, u32 height, int plane_no,
 	u32 bytesperline;
 	u32 size = 0;
 
-	bytesperline = ALIGN(width, stride);
+	bytesperline = width;
 	if (pbl)
 		bytesperline = max(bytesperline, *pbl);
+	bytesperline = ALIGN(bytesperline, stride);
 	if (plane_no == 0)
 		size = bytesperline * height;
 	if (pbl)

@@ -187,9 +222,12 @@ u32 vpu_helper_get_plane_size(u32 fmt, u32 w, u32 h, int plane_no,
 			      u32 stride, u32 interlaced, u32 *pbl)
 {
 	switch (fmt) {
+	case V4L2_PIX_FMT_NV12:
 	case V4L2_PIX_FMT_NV12M:
 		return get_nv12_plane_size(w, h, plane_no, stride, interlaced, pbl);
+	case V4L2_PIX_FMT_NV12_8L128:
 	case V4L2_PIX_FMT_NV12M_8L128:
+	case V4L2_PIX_FMT_NV12_10BE_8L128:
 	case V4L2_PIX_FMT_NV12M_10BE_8L128:
 		return get_tiled_8l128_plane_size(fmt, w, h, plane_no, stride, interlaced, pbl);
 	default:

drivers/media/platform/amphion/vpu_helpers.h +2 -0

index bc28350958be..0eaddb07190d 100644
@@ -14,6 +14,8 @@ struct vpu_pair {
 int vpu_helper_find_in_array_u8(const u8 *array, u32 size, u32 x);
 bool vpu_helper_check_type(struct vpu_inst *inst, u32 type);
 const struct vpu_format *vpu_helper_find_format(struct vpu_inst *inst, u32 type, u32 pixelfmt);
+const struct vpu_format *vpu_helper_find_sibling(struct vpu_inst *inst, u32 type, u32 pixelfmt);
+bool vpu_helper_match_format(struct vpu_inst *inst, u32 type, u32 fmta, u32 fmtb);
 const struct vpu_format *vpu_helper_enum_format(struct vpu_inst *inst, u32 type, int index);
 u32 vpu_helper_valid_frame_width(struct vpu_inst *inst, u32 width);
 u32 vpu_helper_valid_frame_height(struct vpu_inst *inst, u32 height);

drivers/media/platform/amphion/vpu_malone.c +0 -0

index 51e0702f9ae1..69d9a2269fce 100644

drivers/media/platform/amphion/vpu_malone.c bool vpu_malone_check_fmt(enum vpu_core_type type, u32 pixelfmt) · line 587

Quoted 6 lines

drivers/media/platform/amphion/vpu_malone.c +2 -1

@@ -583,7 +583,8 @@ bool vpu_malone_check_fmt(enum vpu_core_type type, u32 pixelfmt)
 	if (!vpu_imx8q_check_fmt(type, pixelfmt))
 		return false;
 
-	if (pixelfmt == V4L2_PIX_FMT_NV12M_8L128 || pixelfmt == V4L2_PIX_FMT_NV12M_10BE_8L128)
+	if (pixelfmt == V4L2_PIX_FMT_NV12_8L128 || pixelfmt == V4L2_PIX_FMT_NV12_10BE_8L128 ||
+	    pixelfmt == V4L2_PIX_FMT_NV12M_8L128 || pixelfmt == V4L2_PIX_FMT_NV12M_10BE_8L128)

^Here are we using spaces instead of tab or I'm wrong?

Quoted 65 lines

drivers/media/platform/amphion/vpu_malone.c +0 -0

 		return true;
 	if (vpu_malone_format_remap(pixelfmt) == MALONE_FMT_NULL)
 		return false;

drivers/media/platform/amphion/vpu_v4l2.c +42 -7

index 8a3eed957ae6..87cce552e906 100644
@@ -140,51 +140,135 @@ bool vpu_is_source_empty(struct vpu_inst *inst)
 	return true;
 }
 
-const struct vpu_format *vpu_try_fmt_common(struct vpu_inst *inst, struct v4l2_format *f)
+static int vpu_init_format(struct vpu_inst *inst, struct vpu_format *fmt)
+{
+	const struct vpu_format *info;
+
+	info = vpu_helper_find_format(inst, fmt->type, fmt->pixfmt);
+	if (!info) {
+		info = vpu_helper_enum_format(inst, fmt->type, 0);
+		if (!info)
+			return -EINVAL;
+	}
+	memcpy(fmt, info, sizeof(*fmt));
+
+	return 0;
+}
+
+static int vpu_calc_fmt_bytesperline(struct v4l2_format *f, struct vpu_format *fmt)
 {
 	struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
-	u32 type = f->type;
+	int i;
+
+	if (fmt->flags & V4L2_FMT_FLAG_COMPRESSED) {
+		for (i = 0; i < fmt->comp_planes; i++)
+			fmt->bytesperline[i] = 0;
+		return 0;
+	}
+	if (pixmp->num_planes == fmt->comp_planes) {
+		for (i = 0; i < fmt->comp_planes; i++)
+			fmt->bytesperline[i] = pixmp->plane_fmt[i].bytesperline;
+		return 0;
+	}
+	if (pixmp->num_planes > 1)
+		return -EINVAL;
+
+	/*amphion vpu only support nv12 and nv12 tiled,
+	 * so the bytesperline of luma and chroma should be same
+	 */
+	for (i = 0; i < fmt->comp_planes; i++)
+		fmt->bytesperline[i] = pixmp->plane_fmt[0].bytesperline;
+
+	return 0;
+}
+
+static int vpu_calc_fmt_sizeimage(struct vpu_inst *inst, struct vpu_format *fmt)
+{
 	u32 stride = 1;
-	u32 bytesperline;
-	u32 sizeimage;
-	const struct vpu_format *fmt;
-	const struct vpu_core_resources *res;
 	int i;
 
-	fmt = vpu_helper_find_format(inst, type, pixmp->pixelformat);

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 194

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +5 -5

-	if (!fmt) {
-		fmt = vpu_helper_enum_format(inst, type, 0);
-		if (!fmt)
-			return NULL;
-		pixmp->pixelformat = fmt->pixfmt;
+	if (!(fmt->flags & V4L2_FMT_FLAG_COMPRESSED)) {
+		const struct vpu_core_resources *res = vpu_get_resource(inst);
+
+		if (res)
+			stride = res->stride;

If res=NULL stride=1 it is ok? Or we need to return some error?

Quoted 29 lines

drivers/media/platform/amphion/vpu_v4l2.c +20 -7

 	}
 
-	res = vpu_get_resource(inst);
-	if (res)
-		stride = res->stride;
-	if (pixmp->width)
-		pixmp->width = vpu_helper_valid_frame_width(inst, pixmp->width);
-	if (pixmp->height)
-		pixmp->height = vpu_helper_valid_frame_height(inst, pixmp->height);
+	for (i = 0; i < fmt->comp_planes; i++) {
+		fmt->sizeimage[i] = vpu_helper_get_plane_size(fmt->pixfmt,
+							      fmt->width,
+							      fmt->height,
+							      i,
+							      stride,
+							      fmt->field != V4L2_FIELD_NONE ? 1 : 0,
+							      &fmt->bytesperline[i]);
+		fmt->sizeimage[i] = max_t(u32, fmt->sizeimage[i], PAGE_SIZE);
+		if (fmt->flags & V4L2_FMT_FLAG_COMPRESSED) {
+			fmt->sizeimage[i] = clamp_val(fmt->sizeimage[i], SZ_128K, SZ_8M);
+			fmt->bytesperline[i] = 0;
+		}
+	}
+
+	return 0;
+}
+
+u32 vpu_get_fmt_plane_size(struct vpu_format *fmt, u32 plane_no)
+{

drivers/media/platform/amphion/vpu_v4l2.c bool vpu_is_source_empty(struct vpu_inst *inst) · line 226

Quoted 10 lines

drivers/media/platform/amphion/vpu_v4l2.c +10 -0

+	u32 size;
+	int i;
+
+	if (plane_no >= fmt->mem_planes)
+		return 0;
+
+	if (fmt->comp_planes == fmt->mem_planes)
+		return fmt->sizeimage[plane_no];
+	if (plane_no < fmt->mem_planes - 1)
+		return fmt->sizeimage[plane_no];

I like a space here but is my personal opinion :)

Quoted 151 lines

drivers/media/platform/amphion/vpu_v4l2.c +49 ~~-23~~

+	size = fmt->sizeimage[plane_no];
+	for (i = fmt->mem_planes; i < fmt->comp_planes; i++)
+		size += fmt->sizeimage[i];
+
+	return size;
+}
+
+int vpu_try_fmt_common(struct vpu_inst *inst, struct v4l2_format *f, struct vpu_format *fmt)
+{
+	struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
+	int i;
+	int ret;
+
+	fmt->pixfmt = pixmp->pixelformat;
+	fmt->type = f->type;
+	ret = vpu_init_format(inst, fmt);
+	if (ret < 0)
+		return ret;
+
+	fmt->width = pixmp->width;
+	fmt->height = pixmp->height;
+	if (fmt->width)
+		fmt->width = vpu_helper_valid_frame_width(inst, fmt->width);
+	if (fmt->height)
+		fmt->height = vpu_helper_valid_frame_height(inst, fmt->height);
+	fmt->field = pixmp->field == V4L2_FIELD_ANY ? V4L2_FIELD_NONE : pixmp->field;
+	vpu_calc_fmt_bytesperline(f, fmt);
+	vpu_calc_fmt_sizeimage(inst, fmt);
+	if ((fmt->flags & V4L2_FMT_FLAG_COMPRESSED) && pixmp->plane_fmt[0].sizeimage)
+		fmt->sizeimage[0] = clamp_val(pixmp->plane_fmt[0].sizeimage, SZ_128K, SZ_8M);
+
+	pixmp->pixelformat = fmt->pixfmt;
+	pixmp->width = fmt->width;
+	pixmp->height = fmt->height;
 	pixmp->flags = fmt->flags;
-	pixmp->num_planes = fmt->num_planes;
-	if (pixmp->field == V4L2_FIELD_ANY)
-		pixmp->field = V4L2_FIELD_NONE;
+	pixmp->num_planes = fmt->mem_planes;
+	pixmp->field = fmt->field;
+	memset(pixmp->reserved, 0, sizeof(pixmp->reserved));
 	for (i = 0; i < pixmp->num_planes; i++) {
-		bytesperline = max_t(s32, pixmp->plane_fmt[i].bytesperline, 0);
-		sizeimage = vpu_helper_get_plane_size(pixmp->pixelformat,
-						      pixmp->width,
-						      pixmp->height,
-						      i,
-						      stride,
-						      pixmp->field > V4L2_FIELD_NONE ? 1 : 0,
-						      &bytesperline);
-		sizeimage = max_t(s32, pixmp->plane_fmt[i].sizeimage, sizeimage);
-		pixmp->plane_fmt[i].bytesperline = bytesperline;
-		pixmp->plane_fmt[i].sizeimage = sizeimage;
+		pixmp->plane_fmt[i].bytesperline = fmt->bytesperline[i];
+		pixmp->plane_fmt[i].sizeimage = vpu_get_fmt_plane_size(fmt, i);
+		memset(pixmp->plane_fmt[i].reserved, 0, sizeof(pixmp->plane_fmt[i].reserved));
 	}
 
-	return fmt;
+	return 0;
 }
 
 static bool vpu_check_ready(struct vpu_inst *inst, u32 type)

@@ -389,10 +473,10 @@ static int vpu_vb2_queue_setup(struct vb2_queue *vq,
 	cur_fmt = vpu_get_format(inst, vq->type);
 
 	if (*plane_count) {
-		if (*plane_count != cur_fmt->num_planes)
+		if (*plane_count != cur_fmt->mem_planes)
 			return -EINVAL;
-		for (i = 0; i < cur_fmt->num_planes; i++) {
-			if (psize[i] < cur_fmt->sizeimage[i])
+		for (i = 0; i < cur_fmt->mem_planes; i++) {
+			if (psize[i] < vpu_get_fmt_plane_size(cur_fmt, i))
 				return -EINVAL;
 		}
 		return 0;

@@ -402,9 +486,9 @@ static int vpu_vb2_queue_setup(struct vb2_queue *vq,
 		*buf_count = max_t(unsigned int, *buf_count, inst->min_buffer_out);
 	else
 		*buf_count = max_t(unsigned int, *buf_count, inst->min_buffer_cap);
-	*plane_count = cur_fmt->num_planes;
-	for (i = 0; i < cur_fmt->num_planes; i++)
-		psize[i] = cur_fmt->sizeimage[i];
+	*plane_count = cur_fmt->mem_planes;
+	for (i = 0; i < cur_fmt->mem_planes; i++)
+		psize[i] = vpu_get_fmt_plane_size(cur_fmt, i);
 
 	return 0;
 }

@@ -434,8 +518,8 @@ static int vpu_vb2_buf_prepare(struct vb2_buffer *vb)
 	u32 i;
 
 	cur_fmt = vpu_get_format(inst, vb->type);
-	for (i = 0; i < cur_fmt->num_planes; i++) {
-		if (vpu_get_vb_length(vb, i) < cur_fmt->sizeimage[i]) {
+	for (i = 0; i < cur_fmt->mem_planes; i++) {
+		if (vpu_get_vb_length(vb, i) < vpu_get_fmt_plane_size(cur_fmt, i)) {
 			dev_dbg(inst->dev, "[%d] %s buf[%d] is invalid\n",
 				inst->id, vpu_type_name(vb->type), vb->index);
 			vpu_set_buffer_state(vbuf, VPU_BUF_STATE_ERROR);

drivers/media/platform/amphion/vpu_v4l2.h +2 -1

index 795ca33a6a50..ef5de6b66e47 100644
@@ -16,7 +16,8 @@ unsigned int vpu_get_buffer_state(struct vb2_v4l2_buffer *vbuf);
 int vpu_v4l2_open(struct file *file, struct vpu_inst *inst);
 int vpu_v4l2_close(struct file *file);
 
-const struct vpu_format *vpu_try_fmt_common(struct vpu_inst *inst, struct v4l2_format *f);
+u32 vpu_get_fmt_plane_size(struct vpu_format *fmt, u32 plane_no);
+int vpu_try_fmt_common(struct vpu_inst *inst, struct v4l2_format *f, struct vpu_format *fmt);
 int vpu_process_output_buffer(struct vpu_inst *inst);
 int vpu_process_capture_buffer(struct vpu_inst *inst);
 struct vb2_v4l2_buffer *vpu_next_src_buf(struct vpu_inst *inst);

drivers/media/platform/amphion/vpu_windsor.c +7 -1

index 1526af2ef9da..a454f142ae17 100644
@@ -775,6 +775,8 @@ static int vpu_windsor_fill_yuv_frame(struct vpu_shared_addr *shared,
 				      u32 instance,
 				      struct vb2_buffer *vb)
 {
+	struct vpu_inst *inst = vb2_get_drv_priv(vb->vb2_queue);
+	struct vpu_format *out_fmt;
 	struct vpu_enc_yuv_desc *desc;
 	struct vb2_v4l2_buffer *vbuf;

@@ -782,6 +784,7 @@ static int vpu_windsor_fill_yuv_frame(struct vpu_shared_addr *shared,
 		return -EINVAL;
 
 	desc = get_yuv_desc(shared, instance);
+	out_fmt = vpu_get_format(inst, vb->type);
 
 	vbuf = to_vb2_v4l2_buffer(vb);
 	desc->frame_id = vbuf->sequence;

@@ -790,7 +793,10 @@ static int vpu_windsor_fill_yuv_frame(struct vpu_shared_addr *shared,
 	else
 		desc->key_frame = 0;
 	desc->luma_base = vpu_get_vb_phy_addr(vb, 0);
-	desc->chroma_base = vpu_get_vb_phy_addr(vb, 1);
+	if (vb->num_planes > 1)
+		desc->chroma_base = vpu_get_vb_phy_addr(vb, 1);
+	else
+		desc->chroma_base = desc->luma_base + out_fmt->sizeimage[0];
 
 	return 0;
 }

Signature · 1 line

2.37.1

Regards,
Tommaso

Signature · 10 lines

Tommaso Merciai
Embedded Linux Engineer
[email protected]
__________________________________

Amarula Solutions SRL
Via Le Canevare 30, 31100 Treviso, Veneto, IT
T. +39 042 243 5310
[email protected]
www.amarulasolutions.com

Ming Qian

author maintainer

2022-08-22 11:56 UTC #14 lore raw

RE: [EXT] Re: [PATCH v4 2/4] media: amphion: tell and handle contiguous and non contiguous format

Quoted 27 lines

Hi Ming,
Sorry for delay. I'm on vacation last week :)
Quoted 23 lines · Ming Qian
Driver should tell the number of memory planes and component planes. the amphion vpu support non contiguous planes, but for compatibility with other device that only support contiguous planes. driver can add support for contiguous planes in the same time. Then the mem_planes can be different from the comp_planes. driver need to handle buffer according mem_planes and comp_planes.
So driver can support NV12 and NV12M.
Signed-off-by Ming Qian <[email protected]>
---
 drivers/media/platform/amphion/vdec.c        | 187 ++++++++++---------
 drivers/media/platform/amphion/venc.c        |  33 ++--
 drivers/media/platform/amphion/vpu.h         |   4 +-
 drivers/media/platform/amphion/vpu_dbg.c     |   8 +-
 drivers/media/platform/amphion/vpu_helpers.c |  48 ++++-
 drivers/media/platform/amphion/vpu_helpers.h |   2 +
 drivers/media/platform/amphion/vpu_malone.c  |   3 +-
 drivers/media/platform/amphion/vpu_v4l2.c    | 168 ++++++++++++-----
 drivers/media/platform/amphion/vpu_v4l2.h    |   3 +-
 drivers/media/platform/amphion/vpu_windsor.c |   8 +-
 10 files changed, 299 insertions(+), 165 deletions(-)

[snip]

Quoted 23 lines

Quoted 9 lines

drivers/media/platform/amphion/vpu_helpers.c +9 -0

+const struct vpu_format *vpu_helper_find_sibling(struct vpu_inst
+*inst, u32 type, u32 pixelfmt) {
+     const struct vpu_format *fmt;
+     const struct vpu_format *sibling;
+
+     fmt = vpu_helper_find_format(inst, type, pixelfmt);
+     if (!fmt)
+             return NULL;
+     if (!fmt->sibling)

drivers/media/platform/amphion/vpu_helpers.c const struct vpu_format *vpu_helper_find_format(struct vpu_inst *inst, u32 type, · line 80

Quoted 10 lines

drivers/media/platform/amphion/vpu_helpers.c +10 -0

+             return NULL;
+     sibling = vpu_helper_find_format(inst, type, fmt->sibling);
+     if (!sibling)
+             return NULL;
+     if (sibling->sibling != fmt->pixfmt)
+             return NULL;
+     if (sibling->comp_planes != fmt->comp_planes)
+             return NULL;
+     return sibling;
+}

I think we can limit the use of "if" statement here. What about this?

const struct vpu_format *vpu_helper_find_sibling(struct vpu_inst *inst, u32

Quoted 16 lines

type, u32 pixelfmt) {
       const struct vpu_format *fmt;
       const struct vpu_format *sibling;

       fmt = vpu_helper_find_format(inst, type, pixelfmt);
       if (!fmt || !fmt->sibling)
               return NULL;

       sibling = vpu_helper_find_format(inst, type, fmt->sibling);
       if (!sibling || (sibling->sibling != fmt->pixfmt) ||
               (sibling->comp_planes != fmt->comp_planes))
               return NULL;

       return sibling;
}

OK, I'll apply this change in v5

Quoted 14 lines

drivers/media/platform/amphion/vpu_helpers.c +14 -0

+
+bool vpu_helper_match_format(struct vpu_inst *inst, u32 type, u32
+fmta, u32 fmtb) {
+     const struct vpu_format *sibling;
+
+     if (fmta == fmtb)
+             return true;
+
+     sibling = vpu_helper_find_sibling(inst, type, fmta);
+     if (sibling && sibling->pixfmt == fmtb)
+             return true;
+     return false;
+}
+

[snip]

Quoted 7 lines

drivers/media/platform/amphion/vpu_malone.c bool vpu_malone_check_fmt(enum vpu_core_type
Quoted 3 lines
drivers/media/platform/amphion/vpu_malone.c +0 -0
@@ -583,7 +583,8 @@ bool vpu_malone_check_fmt(enum vpu_core_type
type, u32 pixelfmt)
Quoted 3 lines
drivers/media/platform/amphion/vpu_malone.c +0 -0
      if (!vpu_imx8q_check_fmt(type, pixelfmt))
              return false;
 

Quoted 9 lines

drivers/media/platform/amphion/vpu_malone.c bool vpu_malone_check_fmt(enum vpu_core_type · line 586
Quoted 1 line
drivers/media/platform/amphion/vpu_malone.c +0 -1
-     if (pixelfmt == V4L2_PIX_FMT_NV12M_8L128 || pixelfmt ==
V4L2_PIX_FMT_NV12M_10BE_8L128)
drivers/media/platform/amphion/vpu_malone.c bool vpu_malone_check_fmt(enum vpu_core_type · line 586
Quoted 1 line
drivers/media/platform/amphion/vpu_malone.c +1 -0
+     if (pixelfmt == V4L2_PIX_FMT_NV12_8L128 || pixelfmt ==
V4L2_PIX_FMT_NV12_10BE_8L128 ||
drivers/media/platform/amphion/vpu_malone.c bool vpu_malone_check_fmt(enum vpu_core_type · line 588
Quoted 2 lines
drivers/media/platform/amphion/vpu_malone.c +2 -0
+         pixelfmt == V4L2_PIX_FMT_NV12M_8L128 || pixelfmt ==
+ V4L2_PIX_FMT_NV12M_10BE_8L128)
^Here are we using spaces instead of tab or I'm wrong?

It's following the rule of checkpatch.pl

Quoted 3 lines

drivers/media/platform/amphion/vpu_malone.c +0 -0

              return true;
      if (vpu_malone_format_remap(pixelfmt) == MALONE_FMT_NULL)
              return false;

[snip]

Quoted 9 lines

drivers/media/platform/amphion/vpu_malone.c +2 -4

+static int vpu_calc_fmt_sizeimage(struct vpu_inst *inst, struct
+vpu_format *fmt) {
      u32 stride = 1;
-     u32 bytesperline;
-     u32 sizeimage;
-     const struct vpu_format *fmt;
-     const struct vpu_core_resources *res;
      int i;

Quoted 15 lines

drivers/media/platform/amphion/vpu_malone.c

Quoted 12 lines

drivers/media/platform/amphion/vpu_malone.c +6 -6

-     fmt = vpu_helper_find_format(inst, type, pixmp->pixelformat);
-     if (!fmt) {
-             fmt = vpu_helper_enum_format(inst, type, 0);
-             if (!fmt)
-                     return NULL;
-             pixmp->pixelformat = fmt->pixfmt;
+     if (!(fmt->flags & V4L2_FMT_FLAG_COMPRESSED)) {
+             const struct vpu_core_resources *res =
+ vpu_get_resource(inst);
+
+             if (res)
+                     stride = res->stride;

If res=NULL stride=1 it is ok? Or we need to return some error?

If res is NULL, it means there is no additional alignment constraints So it's ok to set stride to 1 in this case.

Quoted 39 lines

drivers/media/platform/amphion/vpu_malone.c
Quoted 7 lines
      }
drivers/media/platform/amphion/vpu_malone.c +0 -5
-     res = vpu_get_resource(inst);
-     if (res)
-             stride = res->stride;
-     if (pixmp->width)
-             pixmp->width = vpu_helper_valid_frame_width(inst,
pixmp->width);
drivers/media/platform/amphion/vpu_malone.c
Quoted 2 lines
drivers/media/platform/amphion/vpu_malone.c +0 -2
-     if (pixmp->height)
-             pixmp->height = vpu_helper_valid_frame_height(inst,
pixmp->height);
drivers/media/platform/amphion/vpu_malone.c
Quoted 3 lines
drivers/media/platform/amphion/vpu_malone.c +3 -0
+     for (i = 0; i < fmt->comp_planes; i++) {
+             fmt->sizeimage[i] = vpu_helper_get_plane_size(fmt->pixfmt,
+
fmt->width,
Quoted 1 line
+
fmt->height,
drivers/media/platform/amphion/vpu_malone.c
Quoted 3 lines
drivers/media/platform/amphion/vpu_malone.c +3 -0
+                                                           i,
+                                                           stride,
+
fmt->field != V4L2_FIELD_NONE ? 1 : 0,
Quoted 1 line
+
&fmt->bytesperline[i]);
drivers/media/platform/amphion/vpu_malone.c
Quoted 3 lines
drivers/media/platform/amphion/vpu_malone.c +3 -0
+             fmt->sizeimage[i] = max_t(u32, fmt->sizeimage[i], PAGE_SIZE);
+             if (fmt->flags & V4L2_FMT_FLAG_COMPRESSED) {
+                     fmt->sizeimage[i] = clamp_val(fmt->sizeimage[i],
SZ_128K, SZ_8M);
Quoted 12 lines
drivers/media/platform/amphion/vpu_malone.c +12 -0
+                     fmt->bytesperline[i] = 0;
+             }
+     }
+
+     return 0;
+}
+
+u32 vpu_get_fmt_plane_size(struct vpu_format *fmt, u32 plane_no) {
+     u32 size;
+     int i;
+
+     if (plane_no >= fmt->mem_planes)

Quoted 9 lines

drivers/media/platform/amphion/vpu_malone.c
Quoted 6 lines
drivers/media/platform/amphion/vpu_malone.c +6 -0
+             return 0;
+
+     if (fmt->comp_planes == fmt->mem_planes)
+             return fmt->sizeimage[plane_no];
+     if (plane_no < fmt->mem_planes - 1)
+             return fmt->sizeimage[plane_no];
I like a space here but is my personal opinion :)

OK, I'll add a space line here in v5

[snip]

Tommaso Merciai

2022-08-22 12:03 UTC #15 lore raw

Re: [EXT] Re: [PATCH v4 2/4] media: amphion: tell and handle contiguous and non contiguous format

Hi Ming,

Quoted 191 lines · Ming Qian

Quoted 27 lines
Hi Ming,
Sorry for delay. I'm on vacation last week :)
Quoted 23 lines · Ming Qian
Driver should tell the number of memory planes and component planes. the amphion vpu support non contiguous planes, but for compatibility with other device that only support contiguous planes. driver can add support for contiguous planes in the same time. Then the mem_planes can be different from the comp_planes. driver need to handle buffer according mem_planes and comp_planes.
So driver can support NV12 and NV12M.
Signed-off-by Ming Qian <[email protected]>
---
 drivers/media/platform/amphion/vdec.c        | 187 ++++++++++---------
 drivers/media/platform/amphion/venc.c        |  33 ++--
 drivers/media/platform/amphion/vpu.h         |   4 +-
 drivers/media/platform/amphion/vpu_dbg.c     |   8 +-
 drivers/media/platform/amphion/vpu_helpers.c |  48 ++++-
 drivers/media/platform/amphion/vpu_helpers.h |   2 +
 drivers/media/platform/amphion/vpu_malone.c  |   3 +-
 drivers/media/platform/amphion/vpu_v4l2.c    | 168 ++++++++++++-----
 drivers/media/platform/amphion/vpu_v4l2.h    |   3 +-
 drivers/media/platform/amphion/vpu_windsor.c |   8 +-
 10 files changed, 299 insertions(+), 165 deletions(-)
[snip]
Quoted 23 lines
Quoted 9 lines
drivers/media/platform/amphion/vpu_helpers.c +9 -0
+const struct vpu_format *vpu_helper_find_sibling(struct vpu_inst
+*inst, u32 type, u32 pixelfmt) {
+     const struct vpu_format *fmt;
+     const struct vpu_format *sibling;
+
+     fmt = vpu_helper_find_format(inst, type, pixelfmt);
+     if (!fmt)
+             return NULL;
+     if (!fmt->sibling)
drivers/media/platform/amphion/vpu_helpers.c const struct vpu_format *vpu_helper_find_format(struct vpu_inst *inst, u32 type, · line 80
Quoted 10 lines
drivers/media/platform/amphion/vpu_helpers.c +10 -0
+             return NULL;
+     sibling = vpu_helper_find_format(inst, type, fmt->sibling);
+     if (!sibling)
+             return NULL;
+     if (sibling->sibling != fmt->pixfmt)
+             return NULL;
+     if (sibling->comp_planes != fmt->comp_planes)
+             return NULL;
+     return sibling;
+}
I think we can limit the use of "if" statement here. What about this?
const struct vpu_format *vpu_helper_find_sibling(struct vpu_inst *inst, u32
Quoted 16 lines
type, u32 pixelfmt) {
       const struct vpu_format *fmt;
       const struct vpu_format *sibling;

       fmt = vpu_helper_find_format(inst, type, pixelfmt);
       if (!fmt || !fmt->sibling)
               return NULL;

       sibling = vpu_helper_find_format(inst, type, fmt->sibling);
       if (!sibling || (sibling->sibling != fmt->pixfmt) ||
               (sibling->comp_planes != fmt->comp_planes))
               return NULL;

       return sibling;
}
OK, I'll apply this change in v5
Quoted 14 lines
Quoted 14 lines
drivers/media/platform/amphion/vpu_helpers.c +14 -0
+
+bool vpu_helper_match_format(struct vpu_inst *inst, u32 type, u32
+fmta, u32 fmtb) {
+     const struct vpu_format *sibling;
+
+     if (fmta == fmtb)
+             return true;
+
+     sibling = vpu_helper_find_sibling(inst, type, fmta);
+     if (sibling && sibling->pixfmt == fmtb)
+             return true;
+     return false;
+}
+
[snip]
Quoted 7 lines
drivers/media/platform/amphion/vpu_malone.c bool vpu_malone_check_fmt(enum vpu_core_type
Quoted 3 lines
drivers/media/platform/amphion/vpu_malone.c +0 -0
@@ -583,7 +583,8 @@ bool vpu_malone_check_fmt(enum vpu_core_type
type, u32 pixelfmt)
Quoted 3 lines
drivers/media/platform/amphion/vpu_malone.c +0 -0
      if (!vpu_imx8q_check_fmt(type, pixelfmt))
              return false;
 
Quoted 9 lines
drivers/media/platform/amphion/vpu_malone.c bool vpu_malone_check_fmt(enum vpu_core_type · line 586
Quoted 1 line
drivers/media/platform/amphion/vpu_malone.c +0 -1
-     if (pixelfmt == V4L2_PIX_FMT_NV12M_8L128 || pixelfmt ==
V4L2_PIX_FMT_NV12M_10BE_8L128)
drivers/media/platform/amphion/vpu_malone.c bool vpu_malone_check_fmt(enum vpu_core_type · line 586
Quoted 1 line
drivers/media/platform/amphion/vpu_malone.c +1 -0
+     if (pixelfmt == V4L2_PIX_FMT_NV12_8L128 || pixelfmt ==
V4L2_PIX_FMT_NV12_10BE_8L128 ||
drivers/media/platform/amphion/vpu_malone.c bool vpu_malone_check_fmt(enum vpu_core_type · line 588
Quoted 2 lines
drivers/media/platform/amphion/vpu_malone.c +2 -0
+         pixelfmt == V4L2_PIX_FMT_NV12M_8L128 || pixelfmt ==
+ V4L2_PIX_FMT_NV12M_10BE_8L128)
^Here are we using spaces instead of tab or I'm wrong?
It's following the rule of checkpatch.pl
Quoted 3 lines
Quoted 3 lines
drivers/media/platform/amphion/vpu_malone.c +0 -0
              return true;
      if (vpu_malone_format_remap(pixelfmt) == MALONE_FMT_NULL)
              return false;
[snip]
Quoted 9 lines
Quoted 9 lines
drivers/media/platform/amphion/vpu_malone.c +2 -4
+static int vpu_calc_fmt_sizeimage(struct vpu_inst *inst, struct
+vpu_format *fmt) {
      u32 stride = 1;
-     u32 bytesperline;
-     u32 sizeimage;
-     const struct vpu_format *fmt;
-     const struct vpu_core_resources *res;
      int i;
Quoted 15 lines
drivers/media/platform/amphion/vpu_malone.c
Quoted 12 lines
drivers/media/platform/amphion/vpu_malone.c +6 -6
-     fmt = vpu_helper_find_format(inst, type, pixmp->pixelformat);
-     if (!fmt) {
-             fmt = vpu_helper_enum_format(inst, type, 0);
-             if (!fmt)
-                     return NULL;
-             pixmp->pixelformat = fmt->pixfmt;
+     if (!(fmt->flags & V4L2_FMT_FLAG_COMPRESSED)) {
+             const struct vpu_core_resources *res =
+ vpu_get_resource(inst);
+
+             if (res)
+                     stride = res->stride;
If res=NULL stride=1 it is ok? Or we need to return some error?
If res is NULL, it means there is no additional alignment constraints So it's ok to set stride to 1 in this case.
Quoted 39 lines
drivers/media/platform/amphion/vpu_malone.c
Quoted 7 lines
      }
drivers/media/platform/amphion/vpu_malone.c +0 -5
-     res = vpu_get_resource(inst);
-     if (res)
-             stride = res->stride;
-     if (pixmp->width)
-             pixmp->width = vpu_helper_valid_frame_width(inst,
pixmp->width);
drivers/media/platform/amphion/vpu_malone.c
Quoted 2 lines
drivers/media/platform/amphion/vpu_malone.c +0 -2
-     if (pixmp->height)
-             pixmp->height = vpu_helper_valid_frame_height(inst,
pixmp->height);
drivers/media/platform/amphion/vpu_malone.c
Quoted 3 lines
drivers/media/platform/amphion/vpu_malone.c +3 -0
+     for (i = 0; i < fmt->comp_planes; i++) {
+             fmt->sizeimage[i] = vpu_helper_get_plane_size(fmt->pixfmt,
+
fmt->width,
Quoted 1 line
+
fmt->height,
drivers/media/platform/amphion/vpu_malone.c
Quoted 3 lines
drivers/media/platform/amphion/vpu_malone.c +3 -0
+                                                           i,
+                                                           stride,
+
fmt->field != V4L2_FIELD_NONE ? 1 : 0,
Quoted 1 line
+
&fmt->bytesperline[i]);
drivers/media/platform/amphion/vpu_malone.c
Quoted 3 lines
drivers/media/platform/amphion/vpu_malone.c +3 -0
+             fmt->sizeimage[i] = max_t(u32, fmt->sizeimage[i], PAGE_SIZE);
+             if (fmt->flags & V4L2_FMT_FLAG_COMPRESSED) {
+                     fmt->sizeimage[i] = clamp_val(fmt->sizeimage[i],
SZ_128K, SZ_8M);
Quoted 12 lines
drivers/media/platform/amphion/vpu_malone.c +12 -0
+                     fmt->bytesperline[i] = 0;
+             }
+     }
+
+     return 0;
+}
+
+u32 vpu_get_fmt_plane_size(struct vpu_format *fmt, u32 plane_no) {
+     u32 size;
+     int i;
+
+     if (plane_no >= fmt->mem_planes)
Quoted 9 lines
drivers/media/platform/amphion/vpu_malone.c
Quoted 6 lines
drivers/media/platform/amphion/vpu_malone.c +6 -0
+             return 0;
+
+     if (fmt->comp_planes == fmt->mem_planes)
+             return fmt->sizeimage[plane_no];
+     if (plane_no < fmt->mem_planes - 1)
+             return fmt->sizeimage[plane_no];
I like a space here but is my personal opinion :)

Quoted 3 lines

OK, I'll add a space line here in v5
[snip]

Thanks for clarifications.

Regards,
Tommaso

Signature · 10 lines

Tommaso Merciai
Embedded Linux Engineer
[email protected]
__________________________________

Amarula Solutions SRL
Via Le Canevare 30, 31100 Treviso, Veneto, IT
T. +39 042 243 5310
[email protected]
www.amarulasolutions.com

Ming Qian

author maintainer

2022-08-09 06:51 UTC #16 lore raw

PATCH v4 3/4 media: amphion: decoder add support for contiguous planes

In reply to Ming Qian

decoder add support for contiguous formats V4L2_PIX_FMT_NV12_8L128 and V4L2_PIX_FMT_NV12_10BE_8L128

Signed-off-by Ming Qian <[email protected]>

---
 drivers/media/platform/amphion/vdec.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

drivers/media/platform/amphion/vdec.c +16 -0

index 48ab664fa7ef..9c3324717cbc 100644
@@ -72,12 +72,28 @@ static const struct vpu_format vdec_formats[] = {
 		.mem_planes = 2,
 		.comp_planes = 2,
 		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.sibling = V4L2_PIX_FMT_NV12_8L128,
+	},
+	{
+		.pixfmt = V4L2_PIX_FMT_NV12_8L128,
+		.mem_planes = 1,
+		.comp_planes = 2,
+		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.sibling = V4L2_PIX_FMT_NV12M_8L128,
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_NV12M_10BE_8L128,
 		.mem_planes = 2,
 		.comp_planes = 2,
 		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.sibling = V4L2_PIX_FMT_NV12_10BE_8L128,
+	},
+	{
+		.pixfmt = V4L2_PIX_FMT_NV12_10BE_8L128,
+		.mem_planes = 1,
+		.comp_planes = 2,
+		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.sibling = V4L2_PIX_FMT_NV12M_10BE_8L128
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_H264,

Signature · 1 line

2.37.1

Tommaso Merciai

Reviewed-by on v4 3/4

2022-08-22 11:30 UTC #17 lore raw

Re: [PATCH v4 3/4] media: amphion: decoder add support for contiguous planes

Hi Ming,

Quoted 44 lines · Ming Qian

decoder add support for contiguous formats V4L2_PIX_FMT_NV12_8L128 and V4L2_PIX_FMT_NV12_10BE_8L128

Signed-off-by Ming Qian <[email protected]>

---
 drivers/media/platform/amphion/vdec.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

drivers/media/platform/amphion/vdec.c +16 -0

index 48ab664fa7ef..9c3324717cbc 100644
@@ -72,12 +72,28 @@ static const struct vpu_format vdec_formats[] = {
 		.mem_planes = 2,
 		.comp_planes = 2,
 		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.sibling = V4L2_PIX_FMT_NV12_8L128,
+	},
+	{
+		.pixfmt = V4L2_PIX_FMT_NV12_8L128,
+		.mem_planes = 1,
+		.comp_planes = 2,
+		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.sibling = V4L2_PIX_FMT_NV12M_8L128,
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_NV12M_10BE_8L128,
 		.mem_planes = 2,
 		.comp_planes = 2,
 		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.sibling = V4L2_PIX_FMT_NV12_10BE_8L128,
+	},
+	{
+		.pixfmt = V4L2_PIX_FMT_NV12_10BE_8L128,
+		.mem_planes = 1,
+		.comp_planes = 2,
+		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.sibling = V4L2_PIX_FMT_NV12M_10BE_8L128
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_H264,

Signature · 1 line

2.37.1

Looks good to me.

Unfortunately I don't have hw for testing more. I tried to build and reviewed the code seems ok for me.

Reviewed-by Tommaso Merciai <[email protected]>

Tommaso

Signature · 10 lines

Tommaso Merciai
Embedded Linux Engineer
[email protected]
__________________________________

Amarula Solutions SRL
Via Le Canevare 30, 31100 Treviso, Veneto, IT
T. +39 042 243 5310
[email protected]
www.amarulasolutions.com

Ming Qian

author maintainer

2022-08-09 06:51 UTC #18 lore raw

PATCH v4 4/4 media: amphion: encoder add support for contiguous planes

In reply to Ming Qian

encoder add support for contiguous formats NV12

Signed-off-by Ming Qian <[email protected]>

---
 drivers/media/platform/amphion/venc.c | 8 ++++++++
 1 file changed, 8 insertions(+)

drivers/media/platform/amphion/venc.c +8 -0

index 060a1ee78b17..3cbe8ce637e5 100644
@@ -72,6 +72,14 @@ static const struct vpu_format venc_formats[] = {
 		.mem_planes = 2,
 		.comp_planes = 2,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+		.sibling = V4L2_PIX_FMT_NV12,
+	},
+	{
+		.pixfmt = V4L2_PIX_FMT_NV12,
+		.mem_planes = 1,
+		.comp_planes = 2,
+		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+		.sibling = V4L2_PIX_FMT_NV12M,
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_H264,

Signature · 1 line

2.37.1

Tommaso Merciai

Reviewed-by on v4 4/4

2022-08-22 11:32 UTC #19 lore raw

Re: [PATCH v4 4/4] media: amphion: encoder add support for contiguous planes

Hi Ming,

Quoted 29 lines · Ming Qian

encoder add support for contiguous formats NV12

Signed-off-by Ming Qian <[email protected]>

---
 drivers/media/platform/amphion/venc.c | 8 ++++++++
 1 file changed, 8 insertions(+)

drivers/media/platform/amphion/venc.c +8 -0

index 060a1ee78b17..3cbe8ce637e5 100644
@@ -72,6 +72,14 @@ static const struct vpu_format venc_formats[] = {
 		.mem_planes = 2,
 		.comp_planes = 2,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+		.sibling = V4L2_PIX_FMT_NV12,
+	},
+	{
+		.pixfmt = V4L2_PIX_FMT_NV12,
+		.mem_planes = 1,
+		.comp_planes = 2,
+		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+		.sibling = V4L2_PIX_FMT_NV12M,
 	},
 	{
 		.pixfmt = V4L2_PIX_FMT_H264,

Signature · 1 line

2.37.1

Looks good.

Reviewed-by Tommaso Merciai <[email protected]>

Signature · 10 lines

Tommaso Merciai
Embedded Linux Engineer
[email protected]
__________________________________

Amarula Solutions SRL
Via Le Canevare 30, 31100 Treviso, Veneto, IT
T. +39 042 243 5310
[email protected]
www.amarulasolutions.com