Parsing recipes takes 10 minutes longer due to get_layer_revision_information

Technical Support
, , , ,
1

I am building the image torizon-minimal for version 7.3.0 of the distro torizon. I noticed that the bitbake step “Parsing recipe” takes more than 10 minutes - compared to the normal 30-60 seconds.

I could identify the culprit: the function get_layer_revision_information in meta-toradex-torizon/classes/torizon_base_image_type.inc. The class image_type_torizon.bbclass includes the file torizon_base_image_type.inc. As the class image_type_torizon is added to IMAGE_CLASSES, the function is executed 366 times during parsing because of

OSTREE_LAYER_REVISION_INFO := "${@get_layer_revision_information(d)}"

This assignment executes the function get_layer_revision_information several times for each image recipe found in the available layers. And there are more than 110 image recipes.

A closer look at get_layer_revision_information reveals that it will always calculate the same value for OSTREE_LAYER_REVISION_INFO for all image recipes. The commit shas for the layer repositories do not change during a build. Moreover, the task do_image_ostreecommit from image_types_ostree.bbclass, which evaluates OSTREE_LAYER_REVISION_INFO, is only executed by a single image recipe torizon-minimal.

The idea is to calculate the layer revision information only once. I can achieve this by moving the definition of get_layer_revision_information and the immediate assignment for OSTREE_LAYER_REVISION_INFO from meta-toradex-torizon/classes/torizon_base_image_type.inc to a class of its own, say, meta-toradex-torizon/classes/ostree_layer_revisions.bbclass.

We inherit the new class ostree_layer_revisions in meta-toradex-torizon/recipes-images/images/toradex-base.inc just before inheriting the class core-image:

IMAGE_OVERHEAD_FACTOR = "2.3"

inherit ostree_layer_revisions
inherit core-image

do_rootfs[cleandirs] += "${IMAGE_ROOTFS}"

OSTREE_LAYER_REVISION_INFO is calculated exactly once. The OSTree metadata for EXTRA_OSTREE_COMMIT used in meta-updater/classes/image_types_ostree.bbclass is the same as before. The step “Parsing recipes” is executed in 10 seconds instead of 10 minutes.

Now, I can customise torizon-minimal incrementally - without having to wait 10 extra minutes.

I hope that it’s a good enough fix for you to build on. Feel free to use it or improve it.

Cheers, Burkhard

2

Hi, I’m the Toradex AI Assistant.

I might make mistakes. Our support team will step in if needed. If I answered your question, please tick the “Solution” checkbox. For follow-up questions to me, please use @ToradexAI in your post.

Did you like my answer? :+1: :-1:

Hello Burkhard,

Thank you for your detailed analysis and for sharing your effective optimization regarding the OSTree layer revision calculation during the image build process for Torizon OS (torizon-minimal, version 7.3.0). Your explanation about how get_layer_revision_information was being redundantly executed for each image recipe and your solution to move it into a dedicated class for single invocation are both precise and impactful.

This change directly addresses the performance bottleneck, and your approach—using a dedicated ostree_layer_revisions.bbclass and inheriting it in toradex-base.inc before core-image—is very much in line with the Yocto Project’s best practices for customization and performance improvements (reference).

By ensuring OSTREE_LAYER_REVISION_INFO is computed only once, you preserve the metadata integrity for OSTree commits and greatly speed up the “Parsing recipe” step, enabling a much more efficient incremental development workflow.

This kind of community feedback and practical solution is invaluable. If you would like to share your improvement further, you can do so on our Community Portal or even contribute directly via GitHub.

Thank you once again for your contribution, and please let us know if you have further ideas or need additional details on Torizon OS customization:

Best regards,
Toradex AI Assistant.

3

Greetings @bstubert,

I saw you opened a PR yourself to resolve this yourself: get_layer_revision_information: reducing parsing time from 10min to 30s by bstubert · Pull Request #301 · torizon/meta-toradex-torizon · GitHub

Thank you for bringing this to our attention, and for your contribution.

Best Regards,
Jeremias