Skip to contentSkip to conversation

Hantro JPEG Encoding Padding Bug

DISCUSSION Nicolas Frattaroli 2 participants 2 messages Apr 28, 2022 Media (V4L/DVB)

lore
Share & tools
lore
#0 lore raw

Hello,

yesterday I enabled the Hantro JPEG encoder on my RK3566[1], and noticed a peculiar thing: when encoding a 1920x1080 video[2] with gstreamer through the hardware JPEG encoder using v4l2jpegenc, it'd result in a 1920x1088 output with a green bar at the bottom[3].

I asked on the #linux-media IRC channel on OFTC about this, and was told by Nicolas Dufresne (hello, fellow Nicolas!) that I should post to this list to make sure this doesn't fall through the cracks.

The kernel used was based on 5.18-rc4, and the GStreamer version is 1.20.1.

The GStreamer command used was:

gst-launch-1.0 filesrc location=panduroll.mp4 ! \
               qtdemux name=demux demux.video_0 ! decodebin ! \
               videoconvert ! v4l2jpegenc ! matroskamux ! \
               filesink location=pandu_but_mjpeg.mkv

Regards,
Nicolas Frattaroli

[1]: https://patchwork.kernel.org/project/linux-rockchip/list/?series=636371
[2]: https://overviewer.org/~pillow/up/cd92d13cc0/panduroll.mp4
[3]: https://overviewer.org/~pillow/up/f46371b207/pandu_but_mjpeg.mkv

#1 lore raw

Le jeudi 28 avril 2022 à 14:04 +0200, Nicolas Frattaroli a écrit :

Quoted 14 lines

Hello,

yesterday I enabled the Hantro JPEG encoder on my RK3566[1], and noticed a peculiar thing: when encoding a 1920x1080 video[2] with gstreamer through the hardware JPEG encoder using v4l2jpegenc, it'd result in a 1920x1088 output with a green bar at the bottom[3].

I asked on the #linux-media IRC channel on OFTC about this, and was told by Nicolas Dufresne (hello, fellow Nicolas!) that I should post to this list to make sure this doesn't fall through the cracks.

The kernel used was based on 5.18-rc4, and the GStreamer version is 1.20.1.

Quoted 6 lines

The GStreamer command used was:

gst-launch-1.0 filesrc location=panduroll.mp4 ! \
               qtdemux name=demux demux.video_0 ! decodebin ! \
               videoconvert ! v4l2jpegenc ! matroskamux ! \
               filesink location=pandu_but_mjpeg.mkv

I've reproduced the issue, it looks like GStreamer videoencoder does not call S_SELECTION. Older driver were simply using the CAPTURE width/height and would set the crop automatically, hence why it worked in other cases. Here's a patch to fix GStreamer, looking for feedback.

(quoted diff) +0 -0

b/subprojects/gst-plugins-good/sys/v4l2/gstv4l2object.c index 8609d823ec..de791aa936 100644

subprojects/gst-plugins-good/sys/v4l2/gstv4l2object.c +0 -1
@@ -3190,7 +3190,6 @@ gst_v4l2_object_reset_compose_region (GstV4l2Object * obj)
   struct v4l2_selection sel = { 0 };
 
   GST_V4L2_CHECK_OPEN (obj);
-  GST_V4L2_CHECK_NOT_ACTIVE (obj);
 
   sel.type = obj->type;
   sel.target = V4L2_SEL_TGT_COMPOSE_DEFAULT;
@@ -4353,7 +4352,6 @@ gst_v4l2_object_set_crop (GstV4l2Object * obj, struct

v4l2_rect * crop_rect)
struct v4l2_crop crop = { 0 };

GST_V4L2_CHECK_OPEN (obj);
- GST_V4L2_CHECK_NOT_ACTIVE (obj);

sel.type = obj->type;
sel.target = V4L2_SEL_TGT_CROP;

(quoted diff) +0 -0

b/subprojects/gst-plugins-good/sys/v4l2/gstv4l2videoenc.c index 19496b73f0..04fab87a48 100644

subprojects/gst-plugins-good/sys/v4l2/gstv4l2videoenc.c +6 -0
@@ -342,6 +342,9 @@ gst_v4l2_video_enc_set_format (GstVideoEncoder * encoder,
     return FALSE;
   }
 
+  /* best effort */
+  gst_v4l2_object_setup_padding (self->v4l2output);
+
   self->input_state = gst_video_codec_state_ref (state);
 
   GST_DEBUG_OBJECT (self, "output caps: %" GST_PTR_FORMAT, state->caps);
@@ -876,6 +879,9 @@ gst_v4l2_video_enc_decide_allocation (GstVideoEncoder *
   }
   gst_caps_unref (caps);
 
+  /* best effort */
+  gst_v4l2_object_setup_padding (self->v4l2capture);
+
   if (gst_v4l2_object_decide_allocation (self->v4l2capture, query)) {
     GstVideoEncoderClass *enc_class = GST_VIDEO_ENCODER_CLASS (parent_class);
     ret = enc_class->decide_allocation (encoder, query);

---

Quoted 9 lines

Regards,
Nicolas Frattaroli

[1]: https://patchwork.kernel.org/project/linux-rockchip/list/?series=636371
[2]: https://overviewer.org/~pillow/up/cd92d13cc0/panduroll.mp4
[3]: https://overviewer.org/~pillow/up/f46371b207/pandu_but_mjpeg.mkv