Unable to Generate libvx_delegate.so on BSP 7.2 (NPU via TFLite Delegate)

Technical Support
1

Hello,

I am trying to replicate the tutorial below:

https://developer.toradex.com/torizon/application-development/use-cases/machine-learning/torizon-sample-real-time-tensorflow-lite

However, I realized that my board is running a significantly newer BSP than the one used in the tutorial, and I am encountering issues building the VX delegate required to use the NPU.

Device Information (tdx_info):

Software Summary:

Bootloader: U-Boot
Kernel version: 6.6.84-7.2.0-g87bfb710b6f1 #1-Torizon SMP PREEMPT Mon Mar 31 08:47:48 UTC 2025
Kernel command line: root=LABEL=otaroot rootfstype=ext4 quiet logo.nologo vt.global_cursor_default=0 plymouth.ignore-serial-consoles splash fbcon=map:3 ostree=/ostree/boot.1/torizon/157a467dd3b97eaebfbf6e17d0c8859fdd0abc1d397c637c6ecd808f79dc0260/0
Distro name: NAME=“Torizon OS”
Distro version: VERSION_ID=7.2.0-build.13
Distro variant: VARIANT=“Docker”
Hostname: verdin-imx8mp

Hardware info:

HW model: Toradex Verdin iMX8M Plus WB on Verdin Development Board
Toradex version: 0070 V1.1A
Processor arch: aarch64

The tutorial uses BSP 5.15 and corresponding lf-5.15.71_2.2.0 branches.

Since I am on BSP 7.2 (kernel 6.6), I updated the repositories to my branch version:

  • nn-imx

  • tim-vx-imx

  • tensorflow-imx

  • tflite-vx-delegate-imx

The build completes, but the required libraries for NPU inference are not generated:

  • libtim-vx.so

  • libvx_delegate.so

Because of that, I cannot load the delegate:

tf.load_delegate('/usr/lib/libvx_delegate.so')

and inference falls back to CPU.

Is there an updated branch combination or build procedure for Torizon OS 7.2 / kernel 6.6?

Are additional patches or build flag changes required compared to the 5.15-based tutorial?

Is the delegate workflow still the recommended way to access the NPU on BSP 7.2 ? Has anyone successfully built the VX delegate?

Thank you.

2

Hi, I’m the Toradex AI Assistant.

I might make mistakes or occasionally get you wrong. Our support team will step in if needed.

I understand your issue with building the VX delegate for NPU on Torizon OS 7.2. Good news - there’s an updated approach for your Torizon OS 7.2 system that doesn’t require manual building of the VX delegate libraries.

Recommended Approach for Torizon OS 7.2

For Torizon OS 7.2 with kernel 6.6, you should use the pre-built packages from Toradex/NXP feeds rather than building from source(1). The documentation shows a Dockerfile approach that installs the necessary runtime packages:

FROM --platform=$IMAGE_ARCH torizon/debian-imx8:4

RUN apt-get update && \
    apt-get install -y --no-install-recommends \
    python3 python3-venv python3-pip \
    python3-numpy python3-pil \
    libtim-vx \
    libtensorflow-lite2.16.2 \
    tflite-vx-delegate-imx \
    imx-gpu-viv-wayland \
    imx-gpu-viv-wayland-dev

(1)

Quick Test with Pre-built Container

You can immediately test NPU functionality using the pre-built example:

docker run --rm \
--name "tflite-example" \
-v /dev:/dev \
-v /tmp:/tmp \
--device-cgroup-rule "c 199:0 rmw" \
torizon/tensorflow-lite-imx8:4

(1)

This should show NPU performance (around 0.030s per inference) versus CPU fallback (around 0.366s per inference)(1).

Custom Implementation

For your custom implementation, use the delegate finding function from the documentation:

def find_vx_delegate():
    for p in (
        os.getenv("VX_DELEGATE", "/usr/lib/libvx_delegate.so"),
        "/usr/lib/libvx_delegate.so.2",
        "/usr/lib/libvx_delegate.so.1",
    ):
        if os.path.exists(p):
            return p
    return None

delegates = []
vx = find_vx_delegate()
if vx:
    try:
        delegates.append(tfl.load_delegate(vx))
    except Exception:
        delegates = []

(1)

Key Points

  • Don’t build from source: Use the pre-built packages for Torizon OS 7.2(1)
  • Container environment: Set VX_DELEGATE=/usr/lib/libvx_delegate.so environment variable(1)
  • Device permissions: Include --device-cgroup-rule "c 199:0 rmw" for NPU access(1)

The VX delegate workflow is still the recommended approach for NPU access on Torizon OS 7.2. The key difference is using the packaged libraries instead of building from source.

Best regards, Toradex AI Assistant.

Did you like my answer? :+1: :-1:

If I answered your question, please tick the “Solution” checkbox.

For follow-up questions to me, please use @ToradexAI in your post.

1 Like
3

Hi @Felipecosmes,

Fortunately this is now much easier.

To add the libraries needed for the NPU on the Verdin iMX8MP, you can simply install the following packages to the container:

FROM --platform=linux/arm64 torizon/debian-imx8:4

ENV DEBIAN_FRONTEND=noninteractive \
    PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1

RUN apt-get update && \
    apt-get install -y --no-install-recommends \
      python3 python3-venv python3-pip \
      python3-numpy python3-pil \
      libtim-vx \
      libtensorflow-lite2.16.2 \
      tflite-vx-delegate-imx \
      imx-gpu-viv-wayland \
      imx-gpu-viv-wayland-dev

The documentation on the developer website will be updated soon to reflect these changes.

Best Regards,
Bruno

1 Like