NPU acceleration on IMX8mp with Torizon distro

I have successfully followed the instructions here for building and testing a reference image with NPU acceleration based off of the tdx-wayland distro.

However, I am now trying to port this functionality into my company’s existing BSP, which is based off the torizon distro.

I am running into errors during the bitbake claiming that

imx-gpu-viv was skipped: missing required distro feature 'wayland' (not in DISTRO_FEATURES)

Which I am assuming comes from imx-gpu-viv being a graphics package, and therefore has dependencies from Wayland/X11.

Switching the distro in use to tdx-wayland returns this error:

ERROR: Nothing RPROVIDES 'ostree-devicetree-overlays' (but /home/beb/npu-test/bsp-synchron/build/conf/../../layers/meta-toradex-torizon/recipes-images/images/torizon-core-docker.bb RDEPENDS on or otherwise requires it)

so both distros have something I need.

How would I go about adding the required distro features to my bitbake process? Is that something I can do within bitbake (adding recipes, patches, etc.) or do I have to modify the torizon distro itself to include wayland features?

Note that this is for a totally headless application for processing time series data, so I don’t need any graphics or image processing capability. I just want to be able to access the NPU.

Additionally, what is the actual difference between the tdx-wayland and torizon distros?

Greetings @beb,

Additionally, what is the actual difference between the tdx-wayland and torizon distros?

First of all let’s tackle this questions since it’s key to understanding what’s going on here.

Here at Toradex we provide 2 different Linux based distributions. We have our Yocto Reference BSP and TorizonCore.

In short, we intend customers using the Yocto Reference BSP to modify and customize the base image using Yocto. Customers using TorzionCore are urged to do changes and application development using containers running on top of the base TorizonCore OS. While it is possible to modify TorizonCore using Yocto, as you are, it’s not exactly recommended.

With that out of the way, now to your issue. So as you’ve seen TorizonCore is missing “wayland”. By default we don’t build wayland into TorizonCore because we instead provide it as a container option. We have in our plan to provide a container that provides NPU functionality as well, but it’s not ready yet.

With all that said, you can try to modify the Torizon distro, though this may lead to strange issues since we didn’t intend for customers to work with TorizonCore like this. If you want to add “wayland” to the torizon distro I suggest looking at the line here: meta-toradex-torizon/torizon.inc at dunfell-5.x.y · toradex/meta-toradex-torizon · GitHub

Finally, just to understand, your company has an existing BSP based on “torizon” but you do modifications to it with Yocto? Are containers used at all or are all customization being done in Yocto? I’m just trying to understand what features of our “torizon” distro you’re trying to leverage in your own BSP.

Best Regards,
Jeremias

Hi Jeremias,

Thanks for your overview of the distribution structure.

We have an existing BSP based on Torizon and plan to use the containers to run communication servers and other more straightforward embedded Linux type programs.
The modifications to Yocto are really only happening at this stage because we want to use the NPU, and currently it would seem that support for that only exists on a Yocto level.

What is the timeline for offering NPU support via a container? This would definitely make everything much easier on our end. Is there any documentation or code you are capable of sharing that would let us piece together some brand of container based NPU functionality?

Florence L.

I know there’s not anything substantial to show yet for customers here. Also I don’t see any rough timelines on this yet, though allow me to check on this internally.

Best Regards,
Jeremias

Thanks for checking.

On a related note, are there any existing protocols Toradex might recommend for shuttling data between a container and processes running within the Yocto layer that’s faster than shared file access?

On a related note, are there any existing protocols Toradex might recommend for shuttling data between a container and processes running within the Yocto layer that’s faster than shared file access?

Well our standard recommendation is to use volumes or bind mounts to share data. Though there are certainly other ways. For example, off the top of my head, sockets, dbus, or some kind of network communication. There are more options than this as well, it depends on use-case of course so I can’t really recommend one specifically.

Best Regards,
Jeremias

Are there any updates on the Torizon support for NPU usage from a container?

We do now have an example container that demonstrates how to use the NPU acceleration with Tensorflow Lite: torizon-samples/tflite/tflite-rtsp at bullseye · toradex/torizon-samples · GitHub

This sample builds in the necessary libraries into the container manually. We still don’t have these libraries and packages as Debian packages which is why they’re still being built and installed manually as can be seen in the Dockerfile.

We do plan to do this eventually though other tasks have taken priority at the moment.

Best Regards,
Jeremias

1 Like