I want to build an application which performs an tflite inference on the live camera image, like the demos from the Au-Zone application Zoo. Currently i got the demo from the toradex/torizon-samples repo (branch: bookworm-new) running, but on startup i get the error:
/.venv/lib/python3.11/site-packages/tflite_runtime/interptreter.py, line 109, in del
if self._library is not None:
AttributeError: ‘Delegate’ object has no attribute ‘_library’
But the application itself continues running then and does the inferences.
Another thing i want to know is the “best practice” approcach to get the camera from /dev/video2 running with openCV to have a live-stream where i can perform my inference on.
When i want to connect my Visual Stuido Torizon Extension to the IMX8MP i get the following output in the Development Tools of VS-Code
[Extension Host] [03-26 14:56:09.966] Trying to connect to 172.22.178.10
console.ts:137 [Extension Host] first login success, checking if we should change the default password…
console.ts:137 [Extension Host] Last login: Tue Mar 26 13:52:35 2024 from 172.22.178.150
console.ts:137 [Extension Host] torizon@verdin-imx8mp-07129912:~$
console.ts:137 [Extension Host] loged in!
console.ts:137 [Extension Host] exit
console.ts:137 [Extension Host] logout
console.ts:137 [Extension Host] ret from mkdir docker.service.d …
console.ts:137 [Extension Host] ret from daemon.json…
console.ts:137 [Extension Host] ret from override.conf…
console.ts:137 [Extension Host] ret from daemon-reload…
console.ts:137 [Extension Host] ret from docker restart…
console.ts:137 [Extension Host] ret from docker info…
And the Extension is stuck in the
[03-26 14:56:09.966] Trying to connect to 172.22.178.10
My IMX8MP has the IP-Address 172.22.178.10/24 and my USB-Ethernet-Adapter has the IP-Address 172.22.178.150/24.
This is a build issue with the sample. It currently fails silently without building the required TensorFlow Lite delegate.
We are aware of the issue and I have submitted a Pull Request to solve it. Please note that this PR is still early and we need to do some more tests to be sure that everything is working as expected.
The application is likely running on the CPU as a fallback, not on the NPU.
There are many approaches to this, but it is hard to say that any of them is a “best practice”.
Here are a few options outlined:
Using OpenCV (python examples):
cv.VideoCapture(2): Will open /dev/video2 and automatically resolve how the capture will be turned into BGR OpenCV frames. In general this is a good starting point but does not provide a lot of control.
cv.VideoCapture(‘<gstreamer pipeline>’, cv2.CAP_GSTREAMER): Will initialize and run the given GStreamer pipeline. This approach gives a lot of control over how the data is handled. This is how I tend to use video captures if I am not using GStreamer Directly.
Using GStreamer directly: It is also possible to use the GStreamer library directly. This approach takes more time to get familiar with the GStreamer library, but can give even more control than using OpenCV.
Using Video4Linux: This approach is also possible, but takes much more development effort. It is not recommended unless absolutely necessary.
Your output seems to be correct.
Do you have problems deploying simple command-line applications to the device?
Thank you for the information about the delegate, I noticed that its running on the CPU, but I think that I will wait with the delegate until the fix is tested enough.
Thank you for that, I will try them out and check if I get the camera running.
Yeah, the IDE does not list my Device in the Network Devices list and therefore i cannot connect to it. Could that be because the Maivin from Au-Zone is not running a stock version of Torizon OS?
The inability of the IDE to list the devices may be caused by a network configuration issue, but usually trying to connect directly to the device bypasses the problem.
While it is possible that Au-Zone’s Torizon customizations somehow make the image incompatible with the Torizon IDE extension, I think that is unlikely.
As long as the device is running Torizon OS 6 and has docker available, the extension is expected to work.
These log lines specifically make me believe that the connection with the IDE extension was established successfully at least once.
Can you try to run a simple console application with the Torizon IDE extension and send the error logs here so we can see what may be the problem?
The Maivin is running on Torizon OS 6.5.0.2 so i thinkt that should not be the problem.
I connected the Maivin directly to my pc with a USB-Ethernet adatpter without a fixed IP-Address and not i get this when i try to connect to verdin-imx8mp-xxxxxxxx.
[04-02 15:04:28.287] Trying to connect to verdin-imx8mp-07129912
[04-02 15:04:33.489] ERROR :: Error trying to connect to : connect EINVAL fe80::b932:a5f6:d543:ff3f:22 - Local (:::0)
If I configure a static IP for the Maivin´s network0 and try to connect to that, I get the issue from my previous post.
I cannot run a console application on the device because i cannot start debugging on it. I can´t connect the a network device and therefore i cannot start the remote debugging, because as soon as i want to debug on the device i get this error:
Executing task: pwsh -nop .conf/validateDepsRunning.ps1 VALIDATING ENVIRONMENT Missing settings.json properties, aborting Did you forget to set default device?
If you are facing issues even after setting default device, please remove the registered device and connect it again.
The terminal process “pwsh ‘-nop’, ‘.conf/validateDepsRunning.ps1’” terminated with exit code: 69.
Terminal will be reused by tasks, press any key to close it.
Therefore i can only debug my console application locally.
I would recommend that you connect your Maivin device to a network with internet access alongside your development machine. A direct connection to the Maivin is likely to run into issues with the IDE extension and may not work at all.
my default way to use the Maivin is with a connection to a network with internet access, but there i can not get it to connect to my VS-Code, it will not show up in the Devices list like i mentioned in this post
I tried to reproduce the issue you are facing with the Torizon IDE extension here and using a Maivin with Torizon for Maivin 6.5.0.2, I can also see the problem where the device never connects to the host.
This is unexpected as the initial connection does little more than an ssh connection to the device, but this seems to not be working with Torizon for Maivin when paired with the Torizon IDE extension.
We will investigate this issue and try to find a workaround.
In the meantime, it is possible to manually build and run containers with docker.
We have a lot of documentation about this topic on the developer website:
There is actually a difference in the Torizon for Maivin image which makes the extension try to connect indefinitely.
Ideally this would cause an error instead. I submitted a ticket with the IDE extension team to get this resolved.
That being said, the extension should work regardless with the Torizon for Maivin image.
And to workaround the problem, you can run the following command on your Maivin device:
sudo sed -i 's/PRETTY_NAME="Torizon for Maivin/PRETTY_NAME="TorizonCore for Maivin/' /etc/os-release
With this change you should be able to connect to the device without issues.
VALIDATING ENVIRONMENT
Missing settings.json properties, aborting