A customer wants to use a 3 camera setup for a medical application. CSI and parallel cameras will be used. They’re using gstreamer to compose a single video with the 3 camera feeds with audio. This output must be 1920x1080, at least 25 fps and will be live-streamed to a display and encoded to a file on a SSD at the same time. The streams must all be in sync because of the audio. (we haven’t tested the audio yet).
Using an Apalis iMX6Q, we ran some tests with the CSI camera module and a parallel camera using the Analog Camera Adapter and the performance is fair when just streaming the video. Encoding it, though, decreases performance drastically, even when using the gstreamer-imx plugins. This is the iMX6Q pipeline:
gst-launch-1.0 -v \
imxg2dcompositor name=c background-color=0x223344 \
sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1280 sink_0::height=720 sink_0::fill_color=0x00000000 \
sink_1::xpos=1280 sink_1::ypos=0 sink_1::width=640 sink_1::height=480 sink_1::fill_color=0x11111111 \
sink_2::xpos=1280 sink_2::ypos=240 sink_2::width=640 sink_2::height=480 sink_2::fill_color=0x22222222 ! \
capsfilter caps="video/x-raw, width=1920, height=1080, framerate=30/1" ! tee name=t ! queue2 ! imxeglvivsink \
imxv4l2videosrc device=/dev/video1 ! queue2 ! c.sink_0 \
imxv4l2videosrc device=/dev/video0 ! queue2 ! c.sink_1 \
t. ! queue2 ! imxipuvideotransform ! vpuenc_h264 ! matroskamux ! filesink location="teste.mkv"
NXP spec says it should be possible to decode 2x 1080p H264 streams at 30fps and encode 1x 1080p 30fps, but it seems this performance can’t be achieved if decoding and encoding are done concurrently.
We also tried it on a TK1 (using two USB cameras) and got good performance even when encoding the output to a file. It seems it’s not using GPU acceleration, though. This is the TK1 pipeline:
gst-launch-1.0 -e \
v4l2src device="/dev/video2" ! 'image/jpeg, width=640, height=480, framerate=30/1' ! jpegparse ! nvjpegdec ! m.sink_0 \
v4l2src device="/dev/video1" ! 'image/jpeg, width=1280, height=960, framerate=30/1' ! jpegparse ! nvjpegdec ! m.sink_1 \
videomixer name=m sink_1::xpos=640 ! 'video/x-raw,width=1920,height=1080' ! tee name=t ! queue ! xvimagesink sync=false \
t. ! queue ! omxh264enc ! matroskamux ! filesink location=file.mkv
So the questions are:
-
Is this really too much for the Apalis iMX6Q? We’re not gstreamer experts so maybe we haven’t used the best pipeline possible.
-
Would the TK1 be able to handle this? Is there some way we can use the NVIDIA gstreamer plugins? All tests we ran with their plugins failed.