Hi, It seems new docker images are not pulled by the device once triggered
docker-compose.yml (169 Bytes)
Please find the attached yml file
Device is accessible
Hi, I’m the Toradex AI Assistant. I’m in early beta, and I would appreciate your feedback.
The Toradex Support team will follow up with you shortly!
Did you like my answer?
Hello,
When facing issues with Docker images not being pulled by the device after an update trigger, it’s important to check a few key areas to diagnose and resolve the problem:
-
Docker Daemon and Network Configuration: Ensure the Docker daemon is running correctly on your device. You can check this by running
docker infoordocker psto see if the daemon responds and if there are any running containers. Also, verify the network configuration to ensure the device has internet access and can reach the Docker registry from which it’s trying to pull images. -
Authentication Issues: If your images are hosted in a private registry, make sure that the device is authenticated correctly. You can use
docker login [registry]to authenticate. Ensure the credentials are stored correctly on the device and that the authentication token has not expired. -
Docker Compose Version: Check the version of Docker Compose you are using. Some features related to image pulling and service updates might behave differently across versions. Ensure you are using a version compatible with your Docker engine and the features you are trying to use.
-
Compose File Check: Review your Docker Compose file to ensure the image tags are specified correctly and that there are no syntax errors. If you are using environment variables in your image tags, ensure they are being resolved correctly.
-
Force Image Pull: You can force Docker Compose to pull new images by using the
docker-compose pullcommand before runningdocker-compose up. This ensures that the latest images are pulled from the registry. -
Logs and Debugging: Check the Docker and Docker Compose logs for any error messages or warnings that could indicate why the images are not being pulled. You can increase the verbosity of the logs with the
-vor--verboseflag in Docker Compose commands to get more detailed output. -
Storage Space: Ensure there is enough storage space on the device to pull new images. Lack of space could prevent new images from being downloaded.
-
Docker Hub Rate Limiting: If you are pulling images from Docker Hub, be aware that Docker Hub imposes rate limits on image pulls. If you exceed these limits, image pulls will fail until the limit is reset.
If after checking these areas the issue persists, it might be helpful to manually pull the image using docker pull [image] to see if the issue is specific to Docker Compose or a broader Docker issue.
Best regards,
Toradex AI Assistant
Please, also check if these links can help you solve your problem:
Build Linux Kernel from Source Code | Toradex Developer Center.
Greetings @khaled.hassan,
The picture you shared, it’s showing the update is still in flight and hasn’t been received by the device yet. By default the device polls the server for updates in 5min intervals. Meaning if you initiate an update on the Web UI, it doesn’t go into effect on the device immediately.
Did you try waiting for some time and did the device perform the update then?
Best Regards,
Jeremias
Hi Jeremias,
I waited for 1 hour. Seems device is not fetching anything. There is some issues.
Is this device you’re trying to update showing up as “online” in the Web UI?
Does the Web UI still show this update as “Pending” like in your screenshot?
Could you get the logs from the update Client on this device. (journalctl -f -n 100 -u aktualizr*)
Best Regards,
Jeremias
Hi Jeremias,
Thanks for your reply. What is the process of uploading modification from torizon cloud to specific device. Registry and device is in same network but torizon cloud cant access registry is it a problem?
Best Regards
Md Khaled Ibn Hassan
What is the process of uploading modification from torizon cloud to specific device.
Generally speaking you just need to upload your update package via TorizonCore Builder usually. Then from the Web UI you can deploy that update package to a device that you have provisioned. A rough outline is documented here: First Steps with Torizon Remote Updates | Toradex Developer Center
Registry and device is in same network but torizon cloud cant access registry is it a problem?
I think you have a misunderstanding. Our Torizon Cloud platform does not need to access your container registry. During the update process what is sent to your device is the docker-compose.yml file. This is all our platform has access to. On the device, once it receives the docker-compose.yml it will use this to perform essentially a docker pull on all the container images defined in this file. This means your devices in the field will need access to your container registry in order to actually pull the container images.
Best Regards,
Jeremias
Hi Jeremias,
Thanks for the clarification. You can see from below image the divce is online
But problem happens when I try to update image. Getting below error when pipeline finished. How to debug this as don’t have sufficient log to know what is going in device side.
I can manually run same program from VS Code in Torizon docker container.
If I change the visibility of the repository to public then it works.
After login I can even pull docker images from device.
Seems it can’t pull because may be it needs to login first to docker registry before pull.
From where I can do it as it is inside device?
Please find the error log below
Logs using " journalctl -f -n 100 -u aktualizr*" in device side
Aug 12 08:24:24 verdin-imx8mp-15207672 aktualizr-torizon[1039]: Current versions in storage and reported by OSTree do not match
Aug 12 08:24:25 verdin-imx8mp-15207672 aktualizr-torizon[1039]: Event: UpdateCheckComplete, Result - No updates available
Aug 12 08:29:24 verdin-imx8mp-15207672 aktualizr-torizon[1039]: Current versions in storage and reported by OSTree do not match
Aug 12 08:29:25 verdin-imx8mp-15207672 aktualizr-torizon[1039]: Event: UpdateCheckComplete, Result - No updates available
Aug 12 08:34:24 verdin-imx8mp-15207672 aktualizr-torizon[1039]: Current versions in storage and reported by OSTree do not match
Aug 12 08:34:25 verdin-imx8mp-15207672 aktualizr-torizon[1039]: Event: UpdateCheckComplete, Result - No updates available
Aug 12 08:39:24 verdin-imx8mp-15207672 aktualizr-torizon[1039]: Current versions in storage and reported by OSTree do not match
Aug 12 08:39:25 verdin-imx8mp-15207672 aktualizr-torizon[1039]: Current version for ECU ID: 7e4f5768e65286376f27ad63f8706a6855e424192c94f57f4f622317415219e6 is unknown
Aug 12 08:39:25 verdin-imx8mp-15207672 aktualizr-torizon[1039]: New updates found in Director metadata. Checking Image repo metadata…
Aug 12 08:39:25 verdin-imx8mp-15207672 aktualizr-torizon[1039]: 1 new update found in both Director and Image repo metadata.
Aug 12 08:39:25 verdin-imx8mp-15207672 aktualizr-torizon[1039]: Event: UpdateCheckComplete, Result - Updates available
Aug 12 08:39:25 verdin-imx8mp-15207672 aktualizr-torizon[1039]: Current version for ECU ID: 7e4f5768e65286376f27ad63f8706a6855e424192c94f57f4f622317415219e6 is unknown
Aug 12 08:39:26 verdin-imx8mp-15207672 aktualizr-torizon[1039]: Event: DownloadProgressReport, Progress at 100%
Aug 12 08:39:26 verdin-imx8mp-15207672 aktualizr-torizon[1039]: Event: DownloadTargetComplete, Result - Success
Aug 12 08:39:26 verdin-imx8mp-15207672 aktualizr-torizon[1039]: Event: AllDownloadsComplete, Result - Success
Aug 12 08:39:26 verdin-imx8mp-15207672 aktualizr-torizon[1039]: Current version for ECU ID: 7e4f5768e65286376f27ad63f8706a6855e424192c94f57f4f622317415219e6 is unknown
Aug 12 08:39:26 verdin-imx8mp-15207672 aktualizr-torizon[1039]: Waiting for Secondaries to connect to start installation…
Aug 12 08:39:27 verdin-imx8mp-15207672 aktualizr-torizon[1039]: Event: InstallStarted
Aug 12 08:39:27 verdin-imx8mp-15207672 aktualizr-torizon[1039]: Running docker-compose pull
Aug 12 08:39:27 verdin-imx8mp-15207672 aktualizr-torizon[1039]: Running command: /usr/bin/docker-compose --file /var/sota/storage/docker-compose/docker-compose.yml.tmp pull --no-parallel
Aug 12 08:39:27 verdin-imx8mp-15207672 aktualizr-torizon[2977]: torizon Pulling
Aug 12 08:39:27 verdin-imx8mp-15207672 aktualizr-torizon[2977]: torizon Error
Aug 12 08:39:27 verdin-imx8mp-15207672 aktualizr-torizon[2977]: Error response from daemon: Head “https://registry.lab.petalite.co.uk/v2/khaled.hassan/torizon/manifests/2.0.1”: denied: access forbidden
Aug 12 08:39:27 verdin-imx8mp-15207672 aktualizr-torizon[1039]: Error running docker-compose pull
Aug 12 08:39:27 verdin-imx8mp-15207672 aktualizr-torizon[1039]: Event: AllInstallsComplete, Result - DOWNLOAD_FAILED
Aug 12 08:39:27 verdin-imx8mp-15207672 aktualizr-torizon[1039]: Current versions in storage and reported by OSTree do not match
Aug 12 08:39:28 verdin-imx8mp-15207672 aktualizr-torizon[1039]: Event: PutManifestComplete, Result - Success
Yes it would appear the device does not have access to your private registry that contains your Docker images. I would recommend you refer to the following article on this topic: Using Private Registries With the Torizon Cloud | Toradex Developer Center
Keep in mind, other than the login/authentication information for this registry, the device will also need network access to the registry too. For example if your registry is only available in some internal company network, then the device will need to be on the same network. This is not very ideal if your device will be deployed somewhere that won’t have direct access to your company’s network. In which case I would suggest using a private registry that is publicly accessible. Like a Dockerhub private registry or something similar. That way your container is still in a private/protected registry but network wise the registry can be accessed on normal public networks.
Best Regards,
Jeremias