I am asking the same question that is in this thread
Is there a VPU Video Encoder currently available for the iMX8QM V1.0B module?
—If there is a better way to “refresh” an old question please let me know and feel free to delete this thread.—
Yes, it is. However, the regular BSP 3.0b2 console image does not include any Gstreamer stuff. So either you build your own custom image or try a Qt 5 image from our CI feed (using the Toradex Easy Installer). However, for the later, be warned that the last few nightly builds may not quite work due to us being in the middle of development.
Sounds good, I do have a custom build of the BSP 3.0b2 which I have incorporate several Gstreamer components. This has allowed me to stream the OV5640 camera module with the following pipeline:
gst-launch-1.0 -v v4l2src device=‘/dev/video4’ ! x264enc ! rtph264pay ! udpsink host=192.168.0.23 port=5200
I would like to replace the x264enc with the appropriate vpuenc_h264. I have tried to add the “imx-vpu” package to my build but if fails with a message that the IMX8 is not a supported platform.
Any guidance as to how to incorporate the appropriate VPU package into the build would be appreciated.
Thanks!
Could you share your local.conf changes and the error log, please? Thanks.
I have set up a custom build for the Apalis module based on LinuxImage3.0
TAG _Console-Image_3.0b2.65-20190830
I have added the following lines in the local.conf
IMAGE_INSTALL_append = "packagegroup-fsl-gstreamer1.0 packagegroup-fsl-gstreamer1.0-full v4l-utils rsync python3 "
IMAGE_INSTALL_append = "imx-gst1.0-plugin imx-codec imx-parser imx-alsa-plugins gstreamer1.0-plugins-ugly x264"
PACKAGECONFIG_append_pn-gstreamer1.0-plugins-ugly = "x264"
CORE_IMAGE_EXTRA_INSTALL += "x264-bin x264-dev imx-gst1.0-plugin"
LICENSE_FLAGS_WHITELIST_append = " commercial"
I built the image for the IMX6 and looked for the encoders using
root@apalis-imx6:~# uname -a
Linux apalis-imx6 4.14.117-3.0.2+ge43e3a2 #1 SMP Mon Sep 30 15:02:10 UTC 2019 armv7l GNU/Linux
root@apalis-imx6:~# gst-inspect-1.0 | grep 264
rtp: rtph264pay: RTP H264 payloader
rtp: rtph264depay: RTP H264 depayloader
vpu: vpuenc_h264: IMX VPU-based AVC/H264 video encoder
imxvpu: imxvpuenc_h264: Freescale VPU h.264 video encoder
libav: avmux_ipod: libav iPod H.264 MP4 (MPEG-4 Part 14) muxer
libav: avdec_h264: libav H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 decoder
x264: x264enc: x264enc
uvch264: uvch264src: UVC H264 Source
uvch264: uvch264mjpgdemux: UVC H264 MJPG Demuxer
typefindfunctions: video/x-h264: h264, x264, 264
videoparsersbad: h264parse: H.264 parser
root@apalis-imx6:~# gst-inspect-1.0 | grep imx
imxaudio: imxuniaudiodec: Freescale i.MX uniaudio decoder
imxaudio: imxmp3audioenc: Freescale i.MX MP3 encoder
imxcompositor: imxcompositor_ipu: IMX ipu Video Compositor
imxcompositor: imxcompositor_g2d: IMX g2d Video Compositor
imxv4l2: imxv4l2sink: IMX Video (video4linux2) Sink
imxv4l2: imxv4l2src: IMX Video (video4linux2) Source
imxeglvivsink: imxeglvivsink: Freescale EGL video sink
imxv4l2video: imxv4l2videosrc: V4L2 CSI Video Source
imxv4l2video: imxv4l2videosink: V4L2 CSI Video Sink
imxipu: imxipuvideotransform: Freescale IPU video transform
imxipu: imxipuvideosink: Freescale IPU video sink
imxipu: imxipucompositor: Freescale IPU video compositor
imxmp3enc: imxmp3enc: imx mp3 audio encoder
imxvideoconvert: imxvideoconvert_ipu: IMX ipu Video Converter
imxvideoconvert: imxvideoconvert_g2d: IMX g2d Video Converter
imxg2d: imxg2dvideosink: Freescale G2D video sink
imxg2d: imxg2dvideotransform: Freescale G2D video transform
imxg2d: imxg2dcompositor: Freescale G2D video compositor
imxg2d: imxg2dtextoverlay: Text overlay
imxg2d: imxg2dtimeoverlay: Time overlay
imxg2d: imxg2dclockoverlay: Clock overlay
imxg2d: imxg2dtextrender: Text renderer
imxvpu: imxvpudec: Freescale VPU video decoder
imxvpu: imxvpuenc_h263: Freescale VPU h.263 video encoder
imxvpu: imxvpuenc_h264: Freescale VPU h.264 video encoder
imxvpu: imxvpuenc_mpeg4: Freescale VPU MPEG-4 video encoder
imxvpu: imxvpuenc_mjpeg: Freescale VPU motion JPEG video encoder
I then went back to the local.conf and change the build to
MACHINE ?= “apalis-imx8”
I checked for the encoders and found did not find them
root@apalis-imx8:~# uname -a
Linux apalis-imx8 4.14.117-3.0.2+ge43e3a2 #1 SMP PREEMPT Mon Sep 30 15:58:42 UTC 2019 aarch64 GNU/Linux
root@apalis-imx8:~# gst-inspect-1.0 |grep 264
x264: x264enc: x264enc
videoparsersbad: h264parse: H.264 parser
typefindfunctions: video/x-h264: h264, x264, 264
uvch264: uvch264mjpgdemux: UVC H264 MJPG Demuxer
uvch264: uvch264src: UVC H264 Source
rtp: rtph264depay: RTP H264 depayloader
rtp: rtph264pay: RTP H264 payloader
libav: avdec_h264: libav H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 decoder
libav: avmux_ipod: libav iPod H.264 MP4 (MPEG-4 Part 14) muxer
video4linux2: v4l2h264dec: V4L2 H264 Decoder
root@apalis-imx8:~# gst-inspect-1.0 |grep imx
imxvideoconvert: imxvideoconvert_g2d: IMX g2d Video Converter
imxcompositor: imxcompositor_g2d: IMX g2d Video Compositor
root@apalis-imx8:~#
I went back the local.conf and added imx-vpu and received an error message
IMAGE_INSTALL_append = "imx-gst1.0-plugin imx-codec imx-parser imx-alsa-plugins gstreamer1.0-plugins-ugly x264 imx-vpu"
ERROR: Nothing RPROVIDES 'imx-vpu' (but /home/sfc/SFC_PH3/oe-core/build/../layers/meta-toradex-demos/recipes-images/images/console-tdx-image.bb RDEPENDS on or otherwise requires it)
imx-vpu was skipped: incompatible with machine apalis-imx8 (not in COMPATIBLE_MACHINE)
Hi
In the i.MX8 and i.MX8X family userspace access to the encoder and decoder is provided through V4L2 devices.
(In my setup its /dev/video12 and /dev/video13)
The gstreamer1.0-plugins-good-video4linux2 package provides the plugins v4l2h264dec and v4l2h264enc. They can be used to use the VPU in a gstreamer pipeline.
For me the following pipelines worked to encode a video from the MIPI CSI-2 camera and then play it:
gst-launch-1.0 videotestsrc num-buffers=300 ! v4l2h264enc ! h264parse ! matroskamux ! filesink location=test.mkv
gst-launch-1.0 filesrc location=test.mkv ! video/x-matroska ! aiurdemux ! h264parse ! v4l2h264dec ! imxvideoconvert_g2d ! waylandsink
This worked to stream to my PC, albeit a lot of frames got dropped somewere between camera and PC screen:
PC:
VCAPS='application/x-rtp, media=video, clock-rate=90000, encoding-name=H264, packetization-mode=1, sprop-parameter-sets="Z0IAKKpgPAETYEIAAAMAAgAAAwB5G0NGmUA\=\,aM48gA\=\=", payload=96'
gst-launch-1.0 udpsrc port=5000 ! $VCAPS ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink -v
Apalis iMX8QM:
gst-launch-1.0 v4l2src device='/dev/video4' ! video/x-raw, width=1280, height=720 ! v4l2h264enc ! rtph264pay ! udpsink host=192.168.10.1 port=5000 -v
I had issues that the plugin v4l2h264enc did not show up in gst-inspect-1.0 or not be useable from the pipeline in some of our builds. That wasn’t the case with the latest nightly build. I don’t know which of our changes to the kernel actually did enable it.
So if this is also an issue in your setup, you could try to install with the Toradex Easy Installer the following nightly build (Press Feeds and then tick the first feed additionally to download the list of nightly builds to choose from):
‘Toradex Embedded Linux Qt5 Demo’, ‘3.0b3.20-nightly’
Max
Thank you for the response and information. I tried the ‘Toradex Embedded Linux Qt5 Demo’, ‘3.0b3.20-nightly and Toradex Embedded Linux Qt5 Demo’, '3.0b3.21-nightly but neither of them worked for me.
After booting I checked if my camera was found and it was
root@apalis-imx8:~# dmesg | grep ov56
[ 3.792953] ov5640_mipi_v3 5-003c: 5-003c supply DOVDD not found, using dummy regulator
[ 3.802924] ov5640_mipi_v3 5-003c: 5-003c supply DVDD not found, using dummy regulator
[ 3.812960] ov5640_mipi_v3 5-003c: 5-003c supply AVDD not found, using dummy regulator
[ 4.829115] ov5640_probe camera mipi ov5640, is found
[ 4.909634] mx8-img-md: Registered sensor subdevice: ov5640_mipi_v3 5-003c (1)
[ 5.014881] mx8-img-md: created link [ov5640_mipi_v3 5-003c] => [mxc-mipi-csi2.1]
I then tried to run the pipelines that you provided above. Unfortunately the v4l2h264enc encoder was not found for me.
root@apalis-imx8:~# gst-launch-1.0 v4l2src device='/dev/video4' ! video/x-raw, width=1280, height=720 ! v4l2h264enc ! rtph264pay ! udpsink host=192.168.0.37 port=5000 -v
[ 260.150823] (null): mxc_isi_capture_open, No remote pad found!
[ 260.156924] (null): mxc_isi_capture_open, No remote pad found!
[ 260.163347] (null): mxc_isi_capture_open, No remote pad found!
[ 260.169423] (null): mxc_isi_capture_open, No remote pad found!
WARNING: erroneous pipeline: no element "v4l2h264enc"
root@apalis-imx8:~#
I have a custom build and will keep working with it to see if we an figure something out.
Any guidance or suggestions is appreciated!
Thank you for the feedback and the help
EDIT: I am running on an Ixora dev board. I do not believe that this would make a difference. I can move back the the evaluation board if you think it might make a difference.
-Todd
I am still having issues but have not put the time in to trying to resolve them yet. My last post from 5 OCT is still my current state.
Any insights are appreciated!
Thanks
Any chance you were able to get back to this?
I tried the latest QT5 nightly build and still had no luck.
Hi @mccandlt
Which version exactly did you use?
Which errors have you seen?
We tested on our side and video encoding is working on iMX8.
Best regards,
Jaski
I have tried multiple versions and nightly build images. I will give it a shot today and document versions.
Which image should I try?
Are the pipelines that you provided above what you used to verify?
There error that I am getting is that the v4l2h264enc is not found.
Thanks!
The regular console image does not contain any multimedia stuff. Either build a custom image or try one of our latest nightly Qt images from the CI feed (selectable from the Toradex Easy Installer).
I tried the LXQt and the Qt5 nightly build with the same result. I have attached the bootlog, dmesg output, and the pipeline execution error for each build as a text file.
The module that I am using is:
Toradex Apalis iMX8 QuadMax 4GB Wi-Fi / BT IT V1.0B, Serial# 06445396
I am running it on an Ixora development board.
bootlog_Apalis-iMX8_LXQt-Image 3.0b3.55-nightly_20191108.txt
dmesg_Apalis-iMX8_LXQt-Image 3.0b3.55-nightly_20191108.txt
Pipeline_error_Apalis-iMX8_LXQt-Image 3.0b3.55-nightly_20191108.txt
bootlog_Apalis-iMX8_Qt5-X11-Image 3.0b3.55-nightly_20191108.txt
dmesg_Apalis-iMX8_Qt5-X11-Image 3.0b3.55-nightly_20191108.txt
Pipeline_error_Apalis-iMX8_Qt5-X11-Image 3.0b3.55-nightly_20191108.txt
Make sure to also clear your U-Boot environment (e.g. env default -a; saveenv; reset) after installation or better yet use the Toradex Easy Installer Clear Flash button before installing any new images.
THANK YOU!!!
I cleared the memory and tried the QT5 nightly build. This image would not boot on my module. Looked like the kernel was crashing on a PCIE issue. Do you have any insights into why that is the case? I have posted another question similar to the PCIE issue regarding the iMX8QP 2G module that I have.
BUT THEN…
I loaded my custom build console image and it worked!
Thanks again for the persistence and helping me through this.
You are very welcome. And yes, we are currently fighting a PCIe regression. The U-Boot environment issue should also get a proper fix soon.