Gstreamer performance on Apalis iMX6 and Apalis TK1

A customer wants to use a 3 camera setup for a medical application. CSI and parallel cameras will be used. They’re using gstreamer to compose a single video with the 3 camera feeds with audio. This output must be 1920x1080, at least 25 fps and will be live-streamed to a display and encoded to a file on a SSD at the same time. The streams must all be in sync because of the audio. (we haven’t tested the audio yet).

Using an Apalis iMX6Q, we ran some tests with the CSI camera module and a parallel camera using the Analog Camera Adapter and the performance is fair when just streaming the video. Encoding it, though, decreases performance drastically, even when using the gstreamer-imx plugins. This is the iMX6Q pipeline:

gst-launch-1.0 -v \
 imxg2dcompositor name=c background-color=0x223344 \
   sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1280 sink_0::height=720 sink_0::fill_color=0x00000000 \
   sink_1::xpos=1280 sink_1::ypos=0 sink_1::width=640 sink_1::height=480 sink_1::fill_color=0x11111111 \
   sink_2::xpos=1280 sink_2::ypos=240 sink_2::width=640 sink_2::height=480 sink_2::fill_color=0x22222222 ! \
   capsfilter caps="video/x-raw, width=1920, height=1080, framerate=30/1" ! tee name=t ! queue2 ! imxeglvivsink \
 imxv4l2videosrc device=/dev/video1 ! queue2 ! c.sink_0 \
 imxv4l2videosrc device=/dev/video0 ! queue2 ! c.sink_1 \
 t. ! queue2 ! imxipuvideotransform ! vpuenc_h264 ! matroskamux ! filesink location="teste.mkv"

NXP spec says it should be possible to decode 2x 1080p H264 streams at 30fps and encode 1x 1080p 30fps, but it seems this performance can’t be achieved if decoding and encoding are done concurrently.

We also tried it on a TK1 (using two USB cameras) and got good performance even when encoding the output to a file. It seems it’s not using GPU acceleration, though. This is the TK1 pipeline:

gst-launch-1.0 -e \
 v4l2src device="/dev/video2" ! 'image/jpeg, width=640, height=480, framerate=30/1' ! jpegparse ! nvjpegdec ! m.sink_0 \
 v4l2src device="/dev/video1" ! 'image/jpeg, width=1280, height=960, framerate=30/1' ! jpegparse ! nvjpegdec ! m.sink_1 \
 videomixer name=m sink_1::xpos=640 ! 'video/x-raw,width=1920,height=1080' ! tee name=t ! queue ! xvimagesink sync=false \
 t. ! queue ! omxh264enc ! matroskamux ! filesink location=file.mkv

So the questions are:

  1. Is this really too much for the Apalis iMX6Q? We’re not gstreamer experts so maybe we haven’t used the best pipeline possible.

  2. Would the TK1 be able to handle this? Is there some way we can use the NVIDIA gstreamer plugins? All tests we ran with their plugins failed.

hi @gustavo.tx

We don’t have any experience regarding performance of the Apalis iMX6 when encoding/decoding videos. We never tried to have more than one video input at iMX6. For Apalis TK1, there is one example, where we used two CSI Cameras.

A customer wants to use a 3 camera setup for a medical application. CSI and parallel cameras will be used. They’re using gstreamer to compose a single video with the 3 camera feeds with audio. This output must be 1920x1080, at least 25 fps and will be live-streamed to a display and encoded to a file on a SSD at the same time. The streams must all be in sync because of the audio. (we haven’t tested the audio yet).

Are these hard requirements or nice to have? I have found following thread regarding the video streaming on iMX6. Regarding the limits of encoding of the videos, I found the following.

Which Software are you using on the modules?

Hey @jaski.tx!

These are hard requirements.

I’ll take a look into these threads you found, thanks!

From what I’ve read, this should really be a bit too much for the iMX6 considering all the simultaneous work.

However, we still haven’t managed to use the NVIDIA accelerated gstreamer plugins on the TK1. Their documentation is not very thorough and some of their stuff just contemplates newer BSP versions (not available for the TK1).

Regarding the software, we’re using our latest 2.8b5 BSP on the Apalis iMX6 and our latest L4T release on the TK1.

You may find NVIDIA’s Linux for Tegra incl. full documentation here:

https://developer.nvidia.com/linux-tegra-r217