Mediapipe with NPU support

Technical Support
, ,
1

Good evening,
I’m trying to run a python program with mediapipe in order to use the NPU acceleration of the board. I have read in the documentation how to use an example with tensorflow, but I would like to use it with mediapipe. Does anyone know if it’s possible to do it?

Thanks in advance!

Verdin iMX8M Plus

Verdin Development Board with HDMI Adapter

2

Hi @alfredoIBV !

I have no experience with MediaPipe. Could you please tell us a little bit about why would you like to use it instead of TensorFlow Lite?

By searching in MediaPipe’s repository, I found some entries related to NPU. Does it help you?

Best regards,

3

Hi!
I want to use mediapipe because in one hand we have already our project done with it, and on the other hand, because it make easier the work with machine learning models.

I have already read the entries but they don’t tell anything interesting about how to use it with the NPU.

It’s possible to compile mediapipe in order to use it with the GPU, but I don’t know if it’s possible to do it with the NPU. Also, I have read that mediapipe uses TensorFlow lite in lower layers.

Has anyone get mediapipe works with the NPU?
Thanks!!

5

Hi @alfredoIBV!

After asking internally about MediaPipe, I got more information.

According to NXP, seems like NPU support is not possible: https://www.nxp.com/docs/en/user-guide/IMX-MACHINE-LEARNING-UG.pdf

Acceleration could work:

  • by using OpenCL or OpenGL ES (I’ve been told that this is slow on Verdin iMX8MP), or
  • if MediaPipe is able to use TensorFlow’s external delegates.

It is not possible to run operations on NPU by using OpenCL. It is only possible with OpenVX.

From MediaPipe’s issues section in its Github repository, How to leverage NPU on edge device? · Issue #1425 · google/mediapipe · GitHub recommends nnapi.
Unfortunately, NXP is discontinuing support for it.

But, I you really want to try it, you could try it yourself by following the Building Machine Learning Software with Reference Images for Yocto Project | Toradex Developer Center article where you would need to use version 5.4.70_2.3.0 from NXP instead of 5.15.32_2.0.0 (this newer version is deprecating nnapi already).

You would need to also add the MediaPipe layer.

Please be aware that the process described here was not tested and we do not know if this would work.

Best regards,

6

Is anywhere a step by step tutorial to make a project like this one?

I’m trying to make some demos with TensorFlow but when I use a different image than “grace_hopper.bmp” the results aren’t good. I have tried to change del prediction model without good results.
Thanks!

7

Hello @alfredoIBV ,

Is anywhere a step by step tutorial to make a project like this one?

You can have a look at our Tensorflow Lite sample:

And our knowledge base about machine learning:
https://developer.toradex.com/torizon/application-development/machine-learning/

Best regards,
Josep