We are working with imx8qm with a FHD display. We created a cranksoftware application with a simple video player which should be able to run 1920x1080 videos, but using standard cranksoftware controls we are experimentig some lag during video reproduction.
We had a similar problem on imx6dl playing 1280x800 videos and we solved using different framebuffers for gstreamer and cranksoftware application. For example we run cranksoftware on /dev/fb0 and gstreamer on /dev/fb1. In this way, the application with the video player is “above” the video, and leaving a “hole” in the center of the application allow the user to see the video correctly without any lags. (we used fbalpha to set framebuffers in correct way).
Now, since in imx8 it isn’t possible to control framebuffer directly, is there a way to “tell” wayland to reproduce gstreamer below cranksoftware application?
Moreover, if we want to reproduce smaller videos, is it possible to set (x,y) position where the gstreamer plays the video?
Here is our current pipeline : filesrc location=CHELT_01.mp4 ! video/quicktime ! aiurdemux ! h264parse ! v4l2h264dec ! imxvideoconvert_g2d ! video/x-raw,format=RGB16 ! waylandsink window-width=1920 window-height=1080. As you can see we only reproduce mp4 videos with gstreamer 1.0
Here is the software version of the module.
Videos are located in MMC, in a subdirectory of the crank application (/home/root/mainapp/movies).
Here is the video with lag and here is video without lag. As you can see, the first video lags after about 3/4 seconds.
Moreover, in the “video without lag” you can’t see the progress bar in the bottom of the screen because it’s part o the cranksoftware application and it’s below the gstreamer.
The 2 videos are running with a similar pipeline, with the only difference that in the “lag video” we used “appsink” as last element to tell cranksoftware to render the video inside it’s own control, and in the “video without lag” we used “waylandsink”.
Now, is it possible to use waylandsink and tell wayland to reproduce gstreamer below cranksoftware application?
First of all appsink should not be used, since this will render raw data without any Hardware acceleration.
Now, is it possible to use waylandsink and tell wayland to reproduce gstreamer below cranksoftware application?
There should be no issues running two windows overlaid via the Wayland interface. You will require some logic to control the window overlaying though. If you run the video without Storyboard running that should show you how the performance is outside of Crank. It could be that the waylandsink isn’t configured properly.
Also the order in which you run things is important so you will need to run gstreamer-backend from Cranksoftware first and then Storyboard. Wayland will stack the Storyboard app on top.
Also the order in which you run things is important so you will need to run gstreamer-backend from Cranksoftware first and then Storyboard. Wayland will stack the Storyboard app on top.
We already do it. Our script to launch storyboard is the following:
So are you able to use xwaylandsink on the Crank Software Image or not?
Do you mean waylandsink?
Maybe it wasn’t clear in my first message, but i was able to use waylandsink without having any lag.
To summarize:
Using appsink i can see the video correctly below user interface (because of the use of the “external” storyboard render extension), but i’m having lag.
Using waylandsink i don’t have any lag but the video is reproduced above the user interface.
Now, i would like to use waylandsink to avoid the lag, but i would also like to manage the “z order” to have storyboard app above the gstreamer video.
The waylandsink creates its own window so the z-order should be managed implicitly by Wayland. It could be that there is a timing issue, maybe the Storyboard application is coming up faster than the gstreamer-backend. You could try adding a sleep after the gstreamer-backend.
Could you provide a sample application instructions to reproduce the issue?
After some discussion with people from Crank Software, I think it would be the best if you address your question to support@cranksoftware.com directly.
I contacted Crank Software support and one of their suggestion was to implement Wayland IVI extension to manage windows.
Is it possible to add this extension to our image? Do you have any suggestion/example on how to do this?
