Hantro JPEG Encoding Padding Bug
Hello,
yesterday I enabled the Hantro JPEG encoder on my RK3566[1], and noticed a peculiar thing: when encoding a 1920x1080 video[2] with gstreamer through the hardware JPEG encoder using v4l2jpegenc, it'd result in a 1920x1088 output with a green bar at the bottom[3].
I asked on the #linux-media IRC channel on OFTC about this, and was told by Nicolas Dufresne (hello, fellow Nicolas!) that I should post to this list to make sure this doesn't fall through the cracks.
The kernel used was based on 5.18-rc4, and the GStreamer version is 1.20.1.
The GStreamer command used was:
gst-launch-1.0 filesrc location=panduroll.mp4 ! \
qtdemux name=demux demux.video_0 ! decodebin ! \
videoconvert ! v4l2jpegenc ! matroskamux ! \
filesink location=pandu_but_mjpeg.mkvRegards,
Nicolas Frattaroli
[1]: https://patchwork.kernel.org/project/linux-rockchip/list/?series=636371
[2]: https://overviewer.org/~pillow/up/cd92d13cc0/panduroll.mp4
[3]: https://overviewer.org/~pillow/up/f46371b207/pandu_but_mjpeg.mkv
Le jeudi 28 avril 2022 à 14:04 +0200, Nicolas Frattaroli a écrit :
Quoted 14 lines
Hello,
yesterday I enabled the Hantro JPEG encoder on my RK3566[1], and noticed a peculiar thing: when encoding a 1920x1080 video[2] with gstreamer through the hardware JPEG encoder using v4l2jpegenc, it'd result in a 1920x1088 output with a green bar at the bottom[3].
I asked on the #linux-media IRC channel on OFTC about this, and was told by Nicolas Dufresne (hello, fellow Nicolas!) that I should post to this list to make sure this doesn't fall through the cracks.
The kernel used was based on 5.18-rc4, and the GStreamer version is 1.20.1.
Quoted 6 lines
The GStreamer command used was:
gst-launch-1.0 filesrc location=panduroll.mp4 ! \ qtdemux name=demux demux.video_0 ! decodebin ! \ videoconvert ! v4l2jpegenc ! matroskamux ! \ filesink location=pandu_but_mjpeg.mkv
I've reproduced the issue, it looks like GStreamer videoencoder does not call S_SELECTION. Older driver were simply using the CAPTURE width/height and would set the crop automatically, hence why it worked in other cases. Here's a patch to fix GStreamer, looking for feedback.
(quoted diff) +0 -0
b/subprojects/gst-plugins-good/sys/v4l2/gstv4l2object.c index 8609d823ec..de791aa936 100644
subprojects/gst-plugins-good/sys/v4l2/gstv4l2object.c +0 -1
@@ -3190,7 +3190,6 @@ gst_v4l2_object_reset_compose_region (GstV4l2Object * obj)struct v4l2_selection sel = {0 };GST_V4L2_CHECK_OPEN (obj); -GST_V4L2_CHECK_NOT_ACTIVE (obj); sel.type = obj->type; sel.target =V4L2_SEL_TGT_COMPOSE_DEFAULT ;
@@ -4353,7 +4352,6 @@ gst_v4l2_object_set_crop (GstV4l2Object * obj, struct
v4l2_rect * crop_rect)
struct v4l2_crop crop = { 0 };
GST_V4L2_CHECK_OPEN (obj);
- GST_V4L2_CHECK_NOT_ACTIVE (obj);
sel.type = obj->type;
sel.target = V4L2_SEL_TGT_CROP;
(quoted diff) +0 -0
b/subprojects/gst-plugins-good/sys/v4l2/gstv4l2videoenc.c index 19496b73f0..04fab87a48 100644
subprojects/gst-plugins-good/sys/v4l2/gstv4l2videoenc.c +6 -0
@@ -342,6 +342,9 @@ gst_v4l2_video_enc_set_format (GstVideoEncoder * encoder,return FALSE; } +/* best effort */ +gst_v4l2_object_setup_padding (self->v4l2output); + self->input_state =gst_video_codec_state_ref (state);GST_DEBUG_OBJECT (self,"output caps: %" GST_PTR_FORMAT , state->caps);
@@ -876,6 +879,9 @@ gst_v4l2_video_enc_decide_allocation (GstVideoEncoder * }gst_caps_unref (caps); +/* best effort */ +gst_v4l2_object_setup_padding (self->v4l2capture); +if (gst_v4l2_object_decide_allocation (self->v4l2capture, query)) {GstVideoEncoderClass *enc_class =GST_VIDEO_ENCODER_CLASS (parent_class); ret = enc_class->decide_allocation (encoder, query);
---
Quoted 9 lines
Regards,
Nicolas Frattaroli[1]: https://patchwork.kernel.org/project/linux-rockchip/list/?series=636371
[2]: https://overviewer.org/~pillow/up/cd92d13cc0/panduroll.mp4
[3]: https://overviewer.org/~pillow/up/f46371b207/pandu_but_mjpeg.mkv