Hello,
I need to capture rtsp streams from 4 cameras on iMX8 for HW use, I was recommended to use GStreamer. However, I have a problem because the available OpenCV build does not support GStreamer:
print(cv2.getBuildInformation()) return:
Video I/O:
** …**
** GStreamer: NO**
I’m using VS Code with the Torizon extension for Python development with OpenCV. I have further set:
- extrapackages: python3-opencv gstreamer1.0-qt5 libgstreamer1.0-0 gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav gstreamer1.0-doc gstreamer1.0-tools gstreamer1.0-x gstreamer1.0-alsa gstreamer1.0-gl gstreamer1.0-gtk3 gstreamer1.0-pulseaudio
- requirements.txt:
opencv-python
Is it possible to set in VS Code with Torizon so that the OpenCV build supports GStreamer? Or should I use another library? I think it is necessary to use HW acceleration because only CPU is not enough.
Module: Ixora V1.2A with MIMX8QM6AVUFFAB (SBCK2137)
VS Code with Torizon Extension
Greetings @Richard,
I checked with our team on this. For context the opencv package that is available in the containers is built by us based on NXP sources for the relevant hardware support and such. It turns out the version of this package in our Bullseye Debian containers did not have everything configured correctly for builtin Gstreamer support, as you noticed.
However, this will be resolved in the version of opencv that will come with our Bookworm Debian containers. These are not available quite yet but we plan to have it available soon.
That said, I have a question for you. Are you currently developing on TorizonCore 5 or 6? The reason I ask is because the upcoming Bookworm Debian containers are intended for TorizonCore 6 and may not work correctly on TorizonCore 5.
Best Regards,
Jeremias
Actually, wait disregard everything I said in my initial response it is incorrect.
I double-checked your case here and the issue is actually something else entirely. The issue is that you have both python3-opencv in “extrapackages” and opencv-python in requirments.txt. What this does is install the first package with apt and the second one with pip. This appears to cause a conflict and seems like the pip version of the package is used by Python, which is the version of the package that does not have support.
As a test I attempted to reproduce your issue. My first attempt was doing what you did with installing python3-opencv with apt and opencv-python with pip. I got a similar result to you:
Video I/O:
DC1394: NO
FFMPEG: YES
avcodec: YES (59.37.100)
avformat: YES (59.27.100)
avutil: YES (57.28.100)
swscale: YES (6.7.100)
avresample: NO
GStreamer: NO
v4l/v4l2: YES (linux/videodev2.h)
Next, I only installed the apt verison of the package and got this:
Video I/O:
DC1394: YES (2.2.6)
FFMPEG: NO
avcodec: YES (58.91.100)
avformat: YES (58.45.100)
avutil: YES (56.51.100)
swscale: YES (5.7.100)
avresample: YES (4.0.0)
GStreamer: YES (1.16.2)
PvAPI: NO
v4l/v4l2: YES (linux/videodev2.h)
gPhoto2: YES
As you can see there is now proper Gstreamer support. In summary only install the opencv python module with apt (i.e. extrapackages).
Best Regards,
Jeremias
Dear Jeremias,
thank you very much for the analysis. I added that flag to the requirement.txt, because in addition to GStreamer, I need to save the resulting videos in MP4 format, for which I think I need FFMPEG. However, it does not support the version from pip, as can be seen in the overviews above.
If it is not possible to use both, I will have to find another suitable format to save the videos.
Best regards
Richard
for which I think I need FFMPEG
I checked with our team on this. Apparently in the NXP fork of opencv FFMPEG support is currently broken. Which is why you see this result. We don’t know when NXP plans to resolve this but if possible please pursue a different video format as you have said.
Best Regards,
Jeremias