Torizoncore-builder: Caching a auto-downloaded base image

I’m working with torizoncore-builder to create a customized image for my project. This needs a base image, as specified in the input section of the tcbuild.yaml file.

This base image can neatly be specified using a full URL, or using a toradex-feed section to autogenerate the download url. Using either of these is useful, since it ensures that when sharing the tcbuild.yaml file between people (we’re using git to track it), people can just run the build and be sure that they use the same base image, without any manual downloads, and updates are also easy.

However, this approach does seem to download the base image again every time you call torizoncore-builder build, making builds slow and use a lot of bandwidth.

Since the upstream images (I assume) never change (new versions have new filenames), it would only be needed to download the image once, keep it locally and just use that version afterwards. Of course, I can switch to use local in my input section, which does that, but then I have to manually download the image beforehand (resorting to storing the download URL in documentation or so) losing the advantages described above.

Is there any way to get the auto-download behavior, but with caching to only download an image once? If not, maybe this could be considered as a new feature?

Hi @matthijs ,

Is there any way to get the auto-download behavior, but with caching to only download an image once?
If not, maybe this could be considered as a new feature?

Currently on TorizonCore Builder it isn’t possible to cache the auto-downloaded image from our feeds, as you described.

I’ve passed your request and your use case to the team here and it will be discussed internally as a possible feature for future releases of TorizonCore Builder. Thank you for your feedback!

Best regards,
Lucas Akira

Ok, then we’ll handle this with some duplicate documentation for now. Thanks for passing this on!

Any update on this? We’re currently working to automate our image generation for production purposes using github actions, where it would be useful to use a URL to automatically download the base image. However, for local development, we do not want to redownload every time, so now we either need to keep local changes in the tcbuild.yaml, or have a separate file for development, neither of which is ideal.

Having this caching feature would fix this perfectly :‐)

There’s no major updates to share regarding this. I can see the request was never prioritized/scheduled due to a lack of customers asking for this.

I can inquire our team again. Though if I were to be honest I still can’t guarantee if, or when this will be looked at by our team given the current priorities and scheduling.

How much of an issue is this for you and your team? It does sound annoying/tedious given your requirements, but it doesn’t sound like a complete blocking issue, correct?

Best Regards,
Jeremias

Correct, we can just use local modifications in our git working copy, which will work, but is a bit annoying (and might lead to accidental changes being committed at some point).

I do wonder how others solve this - wanting to automate image generation with auto-downloading a base image, but also doing local development without redownloading on every build seems like a common thing to desire?

Thanks for following up, we’ll see how annoying things will get in practice (we’re still setting up the autobuilding). If it is annoying enough, maybe I’ll prepare a PR to fix it at some point :wink:

I do wonder how others solve this - wanting to automate image generation with auto-downloading a base image, but also doing local development without redownloading on every build seems like a common thing to desire?

Most of the customers I work with either just download the input image every time. Or, they just have a pre-downloaded local copy of the input image. Again, I’m not saying your request does not make sense. But, we really haven’t heard from other customers that this redownloading of the input image is an annoyance. Or at the least it’s not enough of an annoyance for them to say anything to us.

If it is annoying enough, maybe I’ll prepare a PR to fix it at some point :wink:

This would be appreciated. Otherwise I really can’t guarantee to you when we’d be able to get to this ourselves.

Best Regards,
Jeremias