Chromium Kiosk - ffmpeg support

Currently when running Chromium through the Kiosk image, it is not possible to play .aac audio due to codec limitations explained here: Audio/Video (Chrome only)
Simple way to test it (on any machine) is to just install Chromium and run an .aac sample from: https://docs.espressif.com/projects/esp-adf/en/latest/design-guide/audio-samples.html
Instead of using the built-in player, the browser simply downloads the file.

It seems this is an intentional behaviour of typical Chromium build, as it won’t use the codecs available on the system - for example installing ffmpeg and the related gstreamer packages allow you to play the .aac through ffplay, but it has no effect on Chromium

This is generally solved either by installing a distro-provided prebuilt “extras” package (chromium-ffmpeg-extra, chromium-ffmpeg-codecs) that has the built .so libraries needed, or doing your own Chromium build with extra flags to include proprietary codec support.

To make the first approach more complicated, these packages have been moving towards snaps lately.
So the typical Ubuntu package now involves simply docs about the transition: chromium-codecs-ffmpeg_80.0.3987.163-0ubuntu1_amd64.deb 20.04 LTS Download (pkgs.org)

Additionally, simply moving the .so lib to the install directory is tricky because of the versioning as well as the fact that the Chrome package itself seems to differ (slightly) based on distro.
Debian – File list of package chromium/bullseye/arm64
Ubuntu – File list of package chromium-browser/bionic/arm64

I noticed that the Chromium install in the image came from the Toradex feed:
APT-Sources: Index of /debian/snapshots/20210610T044134Z testing/main arm64 Packages
Indicating an internal build. I was wondering if it would be possible for Toradex to perhaps build and distribute an alternate version with extra codecs?

Regards,
Edin

Greetings @ebrodlic,

I’ll need to take your request internally and see what we can do here. You are correct that we do have an internal build of Chromium. However, this is only to add ozone-wayland support to Chromium, so that it can run on our primarily weston/wayland containers. We do not actually do any active development on Chromium beyond this.

While I bring up your request internally could you clarify a few questions I have:

  • In your own tests were you able to get .aac audio playing with Chromium (on any machine)?
  • I know from our other thread there are still some ongoing issues with the Cog browser. But, out of curiosity does .aac audio work with that browser by default?

Best Regards,
Jeremias

Hello @jeremias.tx

Just to clarify regarding Chromium, the change I proposed is simply related to using alternate GN Flags, not touching anything source-related.
There is a proprietary_codecs flag which should take care of everything, details at Audio/Video - The Chromium Projects

In your own tests were you able to get .aac audio playing with Chromium (on any machine)?

So by default, a typical Chromium installation will not do so out of the box on any machine.
As I mentioned, even on a typical Windows machine, a fresh default install of Chromium will not play .aac.

On Windows, you would normally get an alternate build with all codecs from a specific company or vendor, example: Download latest stable Chromium binaries (64-bit and 32-bit) (woolyss.com)

Or you would get the necessary ffmpeg.so lib for that specific version of Chromium, from an alternate Chromium-based project like NWJS.

On Linux, some distros will also provide the `codecs-extra’ as a separate package that you can install. Prior to that, the proprietary codecs will ofcourse not work.

Lastly, you would simply build with alternate flags yourself.
I am also not sure if there are licencing matters at play here that dictate the options, but it would be great if we knew if this was an option that could be handled by Toradex or not.

Details:

I know from our other thread there are still some ongoing issues with the Cog browser. But, out of curiosity does .aac audio work with that browser by default?

We have not yet managed to run the audio streams in general on Cog.
Granted, we were in a hurry at the time so we focused on Chromium. I can get back on this after I do some more testing.

Regards,
Edin

Since it seems to be a simple GN build flag we can consider this.

Let me discuss this internally and see how best we can handle this and assist you. However keep in mind we’ll need to do some initial investigation, especially any legal/licensing matters that may come up.

This is the first time we’ve had a feature/support request like this, so I can’t give any rough estimates. Do you have a strict timeline for this? Just to help with planning on our side.

Best Regards,
Jeremias

Thanks Jeremias.

Yeah that is why I brought it up, as Im not sure of repercussions and practices out there seem to vary.
We do have a timeline, and I am expecting a showcase roughly in 2,3 months.

Just thought it might be a useful thing (if possible) to provide, as it seems like a type of feature that might benefit anyone using the kiosk image.
Let me know what you folks think about it.

Regards,
Edin

The request is with the R&D team now internally. Keep in mind that I can’t guarantee anything regarding this request at the moment. But I’ll let you know about any potential updates/actions.

Best Regards,
Jeremias

Alright, thanks Jeremias!

Regards,
Edin

Hi @ebrodlic,

Just wanted to get back to you regarding this. On initial investigation from the team it looks like due to legal reasons we will not be able to just distribute chromium with the proprietary codecs and such easily. We’re double checking the legality but this seems to be the case.

Therefore our plan from Toradex would be the following. We would open source the Chromium build that goes into our Debian containers. Then we would provide instructions/documentation on how to rebuild and package this for use in a container, including the proprietary codecs. I believe this is the best we could offer given the situation.

We’re still working on the testing and documentation for this process at this time. But I wanted to check if this sounds agreeable from your side.

Best Regards,
Jeremias

Hello @jeremias.tx

It would be great if you could share the documentation!
We’ve had to take steps to ensure the media resources we use for now are open source codecs, but we can’t know for sure if any new ones would be proprietary, so being able to do the build ourselves would help a lot.

Thanks and hope to hear from you,
Edin

Good to hear this approach will work for you. I will make sure to share the documentation when it’s ready and available.

I just wanted to give you an update and check in ahead of time if this plan is okay for you. I did not want to risk going with a solution that ultimately did not work for you and wasting everyone’s time.

With that said hopefully next time I update you, it will be with the documentation.

Best Regards,
Jeremias