I’ve built a ‘seemless’ segment seeking non-linear h264 player using a copy of gstreamer that is a part of the imx8QM multimedia reference image.
The pipeline contains the only h264 decoder available within this reference image (v4l2h264dec).
It appears that If I attempt to change the rate of playback during non flushing segment seeking playback using a seek with GST_SEEK_FLAG_INSTANT_RATE_CHANGE. The rate will change, v4l2h264dec receives the change (the rate im changing is from 1.0 → 0.5:
0:00:08.290096482 2013 0xaaaaee7b08d0 DEBUG videodecoder gstvideodecoder.c:1696:gst_video_decoder_sink_event:<v4l2h264dec> received event 46090, instant-rate-change
0:00:08.290171103 2013 0xaaaaee7b08d0 DEBUG videodecoder gstvideodecoder.c:1116:gst_video_decoder_push_event:<v4l2h264dec> pushing event instant-rate-change
0:00:08.290524336 2013 0xaaaaee7b08d0 DEBUG videodecoder gstvideodecoder.c:1887:gst_video_decoder_src_event:<v4l2h264dec> received event 66817, instant-rate-sync-time
0:00:08.290589458 2013 0xaaaaee7b08d0 DEBUG videodecoder gstvideodecoder.c:1787:gst_video_decoder_src_event_default:<v4l2h264dec> received event 66817, instant-rate-sync-time
0:00:08.295414717 2013 0xaaaaee8034c0 LOG videodecoder gstvideodecoder.c:2808:gst_video_decoder_chain:<v4l2h264dec> chain PTS 0:00:07.850000000, DTS 0:00:07.850000000 duration 0:00:00.016666666 size 124425 flags 4200
0:00:08.295494088 2013 0xaaaaee8034c0 LOG videodecoder gstvideodecoder.c:2970:gst_video_decoder_new_frame:<v4l2h264dec> Created new frame 0xffff74055bc0 (sfn:490)
0:00:08.295537336 2013 0xaaaaee8034c0 DEBUG videodecoder gstvideodecoder.c:2485:gst_video_decoder_chain_forward:<v4l2h264dec> Marking current_frame as sync point
0:00:08.295583584 2013 0xaaaaee8034c0 DEBUG videodecoder gstvideodecoder.c:3958:gst_video_decoder_decode_frame:<v4l2h264dec> PTS delta 0 ms
0:00:08.295648456 2013 0xaaaaee8034c0 LOG videodecoder gstvideodecoder.c:3995:gst_video_decoder_decode_frame:<v4l2h264dec> frame 0xffff74055bc0 PTS 0:00:07.850000000, DTS 0:00:07.850000000, dist 0
0:00:08.295705953 2013 0xaaaaee8034c0 DEBUG videodecoder gstvideodecoder.c:4008:gst_video_decoder_decode_frame:<v4l2h264dec> decoder frame list getting long: 24 frames,possible internal leaking?
but then when the player reaches the end of the segment, which queues up another segment, the decoder doesn’t keep the rate of playback and causes a timestamping issue:
0:00:09.177487945 2013 0xaaaaee8034c0 DEBUG videodecoder gstvideodecoder.c:1537:gst_video_decoder_sink_event_default:<v4l2h264dec> received TIME SEGMENT time segment start=0:00:07.900000000, offset=0:00:00.000000000, stop=0:00:09.849999606, rate=1.000000, applied_rate=1.000000, flags=0x308, time=0:00:07.900000000, base=0:00:07.833333333, position 0:00:07.900000000, duration 99:99:99.999999999
PB_PerpetualBloom| EarlyExitsLeft: 14 PerpetualBloom_E
start:474end:591
Seeking! StartFrame:474 EndFrame :5911
s
S
0:00:09.702977222 2081 0xffff7000e9e0 WARN videodecoder gstvideodecoder.c:3680:gst_video_decoder_clip_and_push_buf:<v4l2h264dec> Dropping frame due to QoS. start:0:00:07.983333333 deadline:0:00:07.916666666 earliest_time:0:00:08.537991763
0:00:09.703211343 2081 0xffff7000e9e0 WARN videodecoder gstvideodecoder.c:3680:gst_video_decoder_clip_and_push_buf:<v4l2h264dec> Dropping frame due to QoS. start:0:00:08.000000000 deadline:0:00:07.933333333 earliest_time:0:00:08.537991763
0:00:09.703379965 2081 0xffff7000e9e0 WARN videodecoder gstvideodecoder.c:3680:gst_video_decoder_clip_and_push_buf:<v4l2h264dec> Dropping frame due to QoS. start:0:00:08.016666666 deadline:0:00:07.949999999 earliest_time:0:00:08.537991763
0:00:09.704061704 2081 0xffff7000e9e0 WARN videodecoder gstvideodecoder.c:3680:gst_video_decoder_clip_and_push_buf:<v4l2h264dec> Dropping frame due to QoS. start:0:00:08.033333333 deadline:0:00:07.966666666 earliest_time:0:00:08.537991763
0:00:09.715495520 2081 0xffff7000e9e0 WARN videodecoder gstvideodecoder.c:3680:gst_video_decoder_clip_and_push_buf:<v4l2h264dec> Dropping frame due to QoS. start:0:00:08.050000000 deadline:0:00:07.983333333 earliest_time:0:00:08.537991763
0:00:09.726342219 2081 0xffff7000e9e0 WARN videodecoder gstvideodecoder.c:3680:gst_video_decoder_clip_and_push_buf:<v4l2h264dec> Dropping frame due to QoS. start:0:00:08.066666666 deadline:0:00:07.999999999 earliest_time:0:00:08.537991763
0:00:09.737347791 2081 0xffff7000e9e0 WARN videodecoder gstvideodecoder.c:3680:gst_video_decoder_clip_and_push_buf:<v4l2h264dec> Dropping frame due to QoS. start:0:00:08.083333333 deadline:0:00:08.016666666 earliest_time:0:00:08.537991763
Is there an issue with this decoder? Or are there any tweaks I can make to the pipe or this plug-in to rectify this issue?
The pipe i’m using is as follows:
filesrc ! qtdemux ! queue ! h264parse ! v4l2h264dec ! imxvideoconvert_g2d ! queue ! glimagesink
Thank you,
William
Linux apalis-imx8-14903975 5.15.129-6.6.0-devel+git.c6f626a5642c #1 SMP PREEMPT Sun Dec 24 15:44:39 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux
