We are currently trying to run our Angular app via the browser/kiosk image (and weston-vivante).
As the developer docs say, the overall performance is significantly better with Cog than Chromium.
We do have the requirement of also having to run video playback (for video call streaming), but the gstreamer packages should hopefully help there (have yet to test it).
Our main problem at the moment is the scrolling animation performance, as using a touchscreen to perform vertical scrolling through any app menu lists results in really poorly performing experience.
Questions:
- Is there anything we can do parameters wise to improve the behaviour?
- Any chance Toradex has also encountered similar problems?
- Is the graphics stack + browser running on the host (not containerized) as well as other browser choices that come with it worth considering to get additional performance improvements?
- Any chance that Chromium supports acceleration in the future for this gpu/driver? Chromium is the first pick namely for the practicality of wide array of well known flags/arguments to tweak the viewing experience.
System:
Verdin iMX8M Mini
Dahlia (and custom) carrier board
Torizon Core - Yocto Dunfell
Greetings @ebrodlic,
Could you elaborate on how the scrolling performance is poor? I just did a quick test with scrolling in Cog and it didn’t seem abnormally poor. It seemed at least on par with scrolling performance in Chromium. Perhaps you could provide a video showing this issue.
Is there anything we can do parameters wise to improve the behaviour?
Cog does have some configuration options, but I suppose it depends on what the exact performance issue is here. Otherwise I can’t know for sure if any parameter would help.
Any chance Toradex has also encountered similar problems?
As I said at the start of my message when browsing normal web pages I don’t notice any abnormal performance issues with scrolling. However I didn’t try with any web applications like Angular.
Is the graphics stack + browser running on the host (not containerized) as well as other browser choices that come with it worth considering to get additional performance improvements?
The only noticeable difference between running something natively and running something in a container is that there is a slight overhead in terms of flash storage and memory needed. As long as this overhead doesn’t exceed the flash/memory of the hardware there should be no performance differences.
Any chance that Chromium supports acceleration in the future for this gpu/driver?
Unfortunately this isn’t something Toradex has much control/influence over. Chromium would need to integrate support on their side for the NXP GPUs that are used.
Best Regards,
Jeremias
Hello @jeremias.tx and thanks for replying!
Could you elaborate on how the scrolling performance is poor?
The preliminary test that we did was simply a custom Torizon Core build with the bsp for the custom board, followed by running a few containers (node, apache2, weston-vivante, kiosk-browser).
Only configuration change we did was setting the screen rotation.
The scrolling was simply using the device touchscreen to scroll down a vertical list and the performance was really slow.
Unfortunately, I don’t yet have the device at my disposal, but I am working on performing some comparison tests with various web apps at the manufacturers site so I will get back with more details and a video recording, and hopefully a narrowed down idea of the bottleneck.
For reference, the same application stack was much more performant on a Raspberry Pi 4, but let me first get back with more details.
Is the graphics stack + browser running on the host (not containerized) as well as other browser choices that come with it worth considering to get additional performance improvements?
Thanks for clarifying this, it helps a lot to know the full host solution is not worth exploring and does not offer benefits in terms of graphics performance.
Unfortunately this isn’t something Toradex has much control/influence over. Chromium would need to integrate support on their side for the NXP GPUs that are used.
We don’t really have a preference in browser from a simple app perspective, but to me the largest benefit of Chromium is the well known flags (arguments) list that accomplish tweaking the kiosk behaviour/experience.
I fully understand Toradex does not have control over this, but it would be great to know if this is at least something you can influence as a major hardware vendor indirectly?
Thank you for the prompt reply and regards,
Edin
With regards to the scrolling issue, here’s what I suggest. With your preliminary test you have quite a number of factors (custom TorizonCore, various non-torizion containers, etc). If you can provide me a set of instructions that can reproduce this issue on vanilla Toradex hardware and software, this would be a huge help. In this way I’d be able to reproduce the issue with what I have available. Also by reproducing this on vanilla software and hardware we’d eliminate a lot of possible variables.
I fully understand Toradex does not have control over this, but it would be great to know if this is at least something you can influence as a major hardware vendor indirectly?
From the Toradex side we’ve given what influence we can. But in the end it seems that both the the NXP side and the chromium side are not interested in prioritizing this kind of integration. As you can see on this recent thread NXP is still not interested in chromium support: https://community.nxp.com/t5/i-MX-Processors/Chromium-on-iMX8-mini/m-p/1265802
This is why we actually offer Cog as an alternative. The company behind Cog was interested in integrating hardware support resulting in the support we have now. Let me provide your feedback and see what we can do for you on the Cog side. Since getting support on this is far more likely than getting hardware acceleration in chromium at the moment.
With regards to this feedback could you elaborate what kind of arguments/flags does Chromium offer that Cog is missing?
Best Regards,
Jeremias
For current testing, we are looking at two things:
1 - Generic Angular based web apps (Forbes, Xbox) - 10 Best Websites Built With Angular To Keep In Mind For 2021 (monocubed.com)
2 - Our own UI web app (served locally) - I will send you the source link privately. We have also modified a branch for benchmarking purposes, allowing us to easily test scrolling within the app that holds dummy data.
So far the device we used showed:
1 - Forbes site generally scrolls fine, but the big performance hit comes later on as more content (especially image heavy) starts to load.
2 - Our UI app is very light on content, but we noticed a big performance hit simply scrolling the dummy data
Would be great to compare both of these with Toradex, especially due to potential image differences.
Meanwhile we will be doing more testing with our partners, and come back with results here asap.
(Your colleague also had a good suggestion of keeping track of htop stats to monitor for cpu or memory related spikes).
Would be great if we could at least isolate if the problem with scrolling on our UI happens due to:
- Low GPU performance - hard to remedy
- High CPU and Memory usage - could be due to high Javascript runtime usage as well as Angular workings (observables and similar)
- Image issues - perhaps sub-optimal versions/configs/flags in the image we used (and built)
When it comes to Chromium, we would be perfectly fine with running Cog if it solves our performance issues. The Chromium benefits mostly involve things of operative nature.
A gigantic reference list can be found at: List of Chromium Command Line Switches « Peter Beverloo
Of course most of these rarely if ever get used, but some that come to mind are: disable pinch, incognito (for cache/lstorage behaviour), enable ui devtools (remote debugging), and overall strong devtools package.
Thank you very much for the test application it was much appreciated. After some testing and experimenting I believe there’s multiple issues/things at play here.
First of all I don’t think there’s necessarily a performance issue with scrolling on Cog. I think Cog is scaling the scrolling strangely. Let me elaborate what I mean by this. First of all, all scrolling I did was with a mouse that also had a scroll wheel. I did the following tests:
- Use Cog to browse to www.toradex.com. Use my mouse’s scroll wheel to scroll down the page. The result was that scrolling was “slow”. However if I use my mouse cursor to scroll via the scroll bar on the webpage rather than my scroll wheel then I can scroll rather quickly.
- Next I did the same with Chromium. I used the same website (www.toradex.com) and the same mouse. Using the scroll wheel I was able to scroll a lot quicker compared to Cog. When using the scroll bar I had similar scroll speed as in Cog.
- Next I used Cog on your test web application you sent me. Since there was no scrollbar I was forced to scroll via my mouse’s scroll wheel. As with my test on www.toradex.com, scrolling was slow this way.
What this tells me is that I believe there’s no performance issue. Rather the issue is that Cog for some reason scales scrolling input differently than Chromium. Meaning you need to scroll more in Cog to move the webpage an equivalent distance compared to Chromium. Though this is just my theory for now, I’ll probably need to run this by our partners behind Cog to see if this is correct or not.
I then did another test. I tested scrolling on an angular based content heavy site, www.forbes.com.
- First I used Cog. Scrolling with scroll wheel was slow as with previous tests. However scrolling via the scroll bar was also slow and caused the page to stutter/freeze a little.
- Next I used Chromium. Scrolling with the scroll wheel was quick as with previous tests. Scrolling with the scroll bar however was much quicker and smother compared to Cog, with no noticeable stutters or freezes.
This tells me there may be another issue. Either Cog has issues with angular based web apps or perhaps it just has issues with fairly content heavy sites, since forbes is a rather busy website with lots of pictures. I didn’t see any stutters or freezes with your angular based web app you sent, so perhaps it’s just how heavy the content was on the forbes site.
With all that said I think we have a enough information to escalate this further up. Let me check with our partner behind Cog and show them my findings. I’ll let you know if we uncover anything.
Best Regards,
Jeremias
Hello @jeremias.tx,
Thanks for the extensive test.
I also need to inform you that we have just discovered some issues when it comes to the touch input driver that additionally causes odd behaviour, and the manufacturing partners have detected problems in that regard.
My gut tells me its a mix of both the input (touch) to achieve the scrolling, as well as what you described in actual scroll handling in the browser.
That would explain some of the differences we had compared to your case, at least.
Any way, I will get back to you asap with more info.
There was an issue with the touch driver that has been resolved, which has influenced our initial scrolling test.
I believe now we are seeing results closer to what you described.
We would need to know what Verdin module you used for the testing though, as i have just learned that they are available in 2/4 core variants, as well as 1/2 GB memory.
For reference in my above tests I used a “Verdin i.MX8MM Q 2GB WB IT V1.1A”.
What’s the variant of the module you have?
We seem to have multiple variants, but our preliminary tests were done on the DL 1GB module, which is dual core and obviously 1GB ram.
When we repeated the test on the module version you used, we saw a significant increase in browser usability.
Would it be possible that you also cross check on the weaker module, so we have more reference info?
That’s interesting findings. While it’s expected that the higher spec variant would perform better, I’m a bit surprised that the increase is so significantly noticeable. Especially since Cog is more tuned for lower-end hardware. Unfortunately I don’t currently have a DL 1GB variant to experiment with, but this issue is now being investigated internally so I’ll pass this information along.
By the way, when you say the browser usability/performance increased significantly between the Quad 2GB and Dual Lite 1GB variants. Is this statement referring to both the Chromium and Cog browsers?
Also since you have both variants, may I ask what variant you intend to use for your end product? I ask because this possible performance issue related to hardware specs may be an entirely different issue than the scrolling issue. Therefore knowing this would help prioritize the issues on our side.
Best Regards,
Jeremias
So to recap a bit: our preliminary results were heavily impacted by a problem on the actual touch driver itself.
After that has been remedied, overall the performance experience was not great on the 1GB dual-core module. As soon as we tried the 2GB 4-core one, both Chromium and Cog performed much better!
That means that our first report (when I opened the ticket) was infact due to: touch mechanism + overall performance on dual-core module.
There is some touch-related difference when it comes to scrolling now, that feels more to sensitivity that you described rather than performance-related.
But I will soon be doing more testing so we have the final numbers on the matter.
As for the end product, the initial goal was the dual-core module.
But for that we will have to see:
- Can dual-core carry the browser+services workload
- Solve some stability related issues we noticed on the dual-core machine (not yet managed to isolate issue)
- Will the memory footprint with all intended future services (in Docker) force us to go up to 2GB
On the other hand, so far the 2GB module seems to function very well.
Regards,
Edin
Thank you for the further information and testing.
We’re currently in the process of escalating the scrolling sensitivity issue to our partners behind Cog. We’ll see if we can also bring up the performance difference between the 1GB and 2GB variants. Though depending on where the bottleneck occurs there may be little we can do. But we’ll discuss first and see.
Best Regards,
Jeremias
