Hello Toradex team,
We are currently evaluating Neural Network acceleration on Verdin i.MX95 using TensorFlow Lite + Neutron delegate, and we are facing a blocking issue:
Even official TensorFlow Lite examples provided by Toradex fail with Neutron error 442.
Context
Hardware / Software:
- Module: Toradex Verdin i.MX95
- Tutorial followed:
Building Machine Learning Software with Reference Images for Yocto Project
Building Machine Learning Software with Reference Images for Yocto Project | Toradex Developer Center
Test performed (official Toradex example)
Command used:
python3 label_image.py \
-i grace_hopper.bmp \
-m mobilenet_v1_1.0_224_quant.tflite \
-l labels.txt \
-e /usr/lib/libneutron_delegate.so
Result on Verdin i.MX95:
root@verdin-imx95-12594079:/usr/bin/tensorflow-lite-2.16.2/examples# python3 label_image.py -i grace_hopper.bmp -m mobilenet_v1_1.0_224_quant.tflite -i grace_hopper.bmp -l labels.txt -e /usr/lib/libneutron_delegate.so Loading external delegate from /usr/lib/libneutron_delegate.so with args: {} INFO: NeutronDelegate delegate: 29 nodes delegated out of 31 nodes with 1 partitions. INFO: Created TensorFlow Lite XNNPACK delegate for CPU. fail to create neutron inference job Error: component=‘Neutron Driver’, category=‘internal fault’, code=442 Traceback (most recent call last): File “/usr/bin/tensorflow-lite-2.16.2/examples/label_image.py”, line 120, in interpreter.invoke() File “/usr/lib/python3.12/site-packages/tflite_runtime/interpreter.py”, line 941, in invoke self._interpreter.Invoke() RuntimeError: /usr/src/debug/tensorflow-lite-neutron-delegate/2.16.2/neutron_delegate.cc:261 neutronRC != ENONE (113203 != 0)Node number 31 (NeutronDelegate) failed to invoke.
Same test on Verdin i.MX8M Plus (comparison)
With the same model and same command structure:
python3 label_image.py \
-i grace_hopper.bmp \
-m mobilenet_v1_1.0_224_quant.tflite \
-l labels.txt \
-e /usr/lib/libvx_delegate.so
Result on i.MX8M Plus:
root@verdin-imx8mp-15676722:/usr/bin/tensorflow-lite-2.16.2/examples# python3 label_image.py -i grace_hopper.bmp -m mobilenet_v1_1.0_224_quant.tflite -l labels.txt -e /usr/lib/libvx_delegate.so
Loading external delegate from /usr/lib/libvx_delegate.so with args: {}
INFO: Vx delegate: allowed_cache_mode set to 0.
INFO: Vx delegate: device num set to 0.
INFO: Vx delegate: allowed_builtin_code set to 0.
INFO: Vx delegate: error_during_init set to 0.
INFO: Vx delegate: error_during_prepare set to 0.
INFO: Vx delegate: error_during_invoke set to 0.
W [HandleLayoutInfer:332]Op 162: default layout inference pass.
Warm-up time: 3368.6 ms
Inference time: 3.1 ms
0.870588: military uniform
0.031373: Windsor tie
0.011765: mortarboard
0.007843: bow tie
0.007843: bulletproof vest
NPU acceleration is fully functional on i.MX8M Plus.
Conclusion
This demonstrates that:
- The model is valid.
- The application works correctly.
- The same example and model run successfully on i.MX8M Plus.
- The failure happens only on i.MX95 when using Neutron delegate.
Questions
Could you please confirm:
- Is Neutron officially supported today on Verdin i.MX95?
- Is error 442 a known issue?
- Is a fix or BSP update planned?
- Is Neutron expected to work similarly to i.MX8M Plus (VX delegate)?
- Have you been able to reproduce this internally?
