TorizonCore OS with Custom Layers

Technical Support
, , , , ,
1

I was able to build TorizonCore image from source following instructions on your website. I loaded them to the board and everything looked good.
Then I added 3 custom layers in /layers directory, included them in bblayers.conf in the build folder and run the build again. The new TorizonCore image was successfully built with 3 extra layers. However, after loading the image to the board (colibri-imx8x), it seems the new layers were NOT included in the TorizonCore OS. I checked the build file in /etc folder, but custom layers showed up in ā€œLayer Revisionsā€.
Based on my understanding, in order to load the TorizonCore image with extra layers, I just need to untar the ā€œtorizon-core-docker-colibri-imx8x-Tezi_5.7.1-devel-20221115114629+build.0.tarā€ file and load them to the board via EasyInstaller. Did I miss anything?

2

Greetings @zhu007,

Just to make sure I understand so you added these 3 custom layers to your build. Did you then add recipes from these layers to the TorizonCore image recipe?

If all you did was add layers to the build, then this doesnā€™t necessarily add anything to the built TorizonCore image.

That said I donā€™t know anything about these custom layers of yours and Iā€™m not sure what you mean by ā€œthe new layers were NOT included in the TorizonCore OSā€. What from these layers are you trying to add to the final OS image?

Best Regards,
Jeremias

3

Hi Jeremias,

Thanks for the quick response. I apologize if I had confused you. When I added those new Yocto layers, I directly pulled them from git repo which has recipes included. After that, I manually added the new layers as extra layers in the bblayers.conf file in the build folder. Then I ran the bitbake and built the torizon-core-docker image.

Since I built the the TorizonCore from source with additional yocto layer added, I would assume the new built TorizonCore will have the binary files included in the added yocto layer. Hopefully this clarifies my question!

4

I understand that you added layers to the build. But then did you add recipes from these layers to the TorizonCore recipe?

New packages/recipes in Yocto donā€™t get automatically added just because the layer is present. For example this is the TorizonCore recipe in Yocto: meta-toradex-torizon/torizon-core-docker.bb at kirkstone-6.x.y Ā· toradex/meta-toradex-torizon Ā· GitHub

You can see we have to explicitly add the recipes for Docker related packages to the image. Itā€™s not enough to just add the Docker layers to the build.

Best Regards,
Jeremias

5

Youā€™re rightā€¦ I didnā€™t include the package in the TorizonCore recipe, which is torizon-core-docker.bb file. Iā€™ll add the package/recipes to the TorizonCore recipe and try another build!!! Thanks for helping me identify the issue!

6

Glad I was able to help.

7

Hi Jeremias,

I added the missing packages/recipes in the local configuration file before running another build. However, the build failed and complained about missing dependencies. I double-checked all the required layers and they looked OK. Here are the error messages I got:

ERROR: Nothing RPROVIDES ā€˜aziot-edgedā€™ (but /home/yocto/Projects/torizon-os/build/conf/ā€¦/ā€¦/layers/meta-toradex-torizon/recipes-images/images/torizon-core-dcoker.bb RDEPENDS on or otherwise requires it)
aziot-edged was skipped: aziot-edged - aziot-edged: system username lotedge does not have a static ID defined. Add iotedge to one of these files: /home/yocto/Projects/torizon-os/build/conf/ā€¦/ā€¦/layers/meta-toradex-torizon/files/torizon-static-passwd
NOTE: Runtime target ā€˜aziot-edgedā€™ is unbuildable, removingā€¦
Missing or unbuildable dependency chain was: [ā€˜aziot-edgedā€™]
ERROR: Required build target ā€˜torizon-core-dockerā€™ has no buildable providers.
Missing or unbuildable dependency chain was : [ā€˜torzion-core-dockerā€™, ā€˜aziot-edgedā€™]

Actually I was trying to add a few custom layers and build the Azure iotedge runtime into TorizonCore from source. Iā€™m not entirely sure if itā€™s related to the source file for torzionCore image. But I donā€™t know why it complained that system user name does not have a static ID definedā€¦ Is there anything else I need to include or modify in TorizonCore layer after adding the custom recipes or images in the TorizonCore image recipe?

8

It appears that some azure iot edge recipe adds the user ā€œIotedgeā€ to the OS. In TorizonCore we use static IDs for our users and groups. Since youā€™re adding this new user you either need to define a static ID for this user or explicitly ignore this user for the purposes of static ID.

See our commit here for more info on this: torizon: enable useradd-staticids Ā· toradex/meta-toradex-torizon@f83a6db Ā· GitHub

Also see the official Yocto documentation for more info on how static IDs work: Yocto Project Reference Manual

Best Regards,
Jeremias

9

Hi Jeremias,

Thanks again for helping me identify the issue! I believe this undefined static IDs led to the missing dependencies in my build. After adding user ā€œiotedgeā€, it looks there were more users needed to be defined/added in the required files. The build finally started after I included the static IDs for all required users.

10

Glad I was able to help clarify things!

11

Hello Jeremias,

Hope you are doing well! Now I have a few questions for you again. After assigning an static userid for the new added users, I was able to successfully build an custom TorizonCore OS image with Microsoft Azure runtime from source. However, the Azure runtime didnā€™t really work on the built TorizonCore OS due to a missing directory containing important files. In the iotedge recipe, it tried to create a home directory in /var/lib/ folder, but this folder wasnā€™t able to be created for some reasonā€¦ Microsoft engineers checked their recipe but didnā€™t find any issues on their end. They also confirmed the Yocto build worked on other platformsā€¦ I really wound like to pick up your brain and see if you can think of anything that might go wrong in TorizonCoreā€¦

12

it tried to create a home directory in /var/lib/ folder, but this folder wasnā€™t able to be created for some reason

Well this is actually the issue right here. TorizonCore uses an OSTree based filesystem to allow for easy updates. Now in a OSTree filesystem certain directories are treated specially. The /var/lib directory is one of these. In short during the build when the initial filesystem is transformed into an OSTree filesystem the /var/lib directory actually gets cleared out. I imagine whatā€™s happening is that the iotedge recipe does properly install these files in /var/lib, but then the OSTree recipe clears this directory.

That said, what are your alternatives? Just to name a few:

Iā€™m sure thereā€™s other methods but these are the ones I happen to know about. I mean in summary you need a way to indirectly write the files to /var/lib as doing so directly during the build will result in this issue you observed.

Best Regards,
Jeremias

13

Hi Jeremias,

Thanks for your reply. I believe that is the root cause why we couldnā€™t run Azure iotedge runtime properly after building it into TorizonCore OS. In iotedge recipe, it creates the home directory (/var/lib/aziot/) in /var/lib/ directory which is used to start iotedge daemon and its related services.

install -d ${D}${localstatedir}/lib
install -d ${D}${localstatedir}/log
install -d ${D}${localstatedir}/lib/aziot
install -d ${D}${localstatedir}/log/aziot
install -d -m 755 -o iotedge -g iotedge ${D}${localstatedir}/lib/aziot/edged
install -d -m 755 -o iotedge -g iotedge ${D}${localstatedir}/log/aziot/edged

However, I donā€™t think we could easily modify the current iotedge recipe without making additional changes since this recipe is currently compatible with the build environment on most of platforms. Do you think if thereā€™s any other possible workaround in the TorizonCore to prevent the OSTree from cleaning up the files in /var/lib/ folder in the build process?

14

However, I donā€™t think we could easily modify the current iotedge recipe without making additional changes since this recipe is currently compatible with the build environment on most of platforms.

In my previous reply not all of my suggestions involved changing the iotedge recipe. Do none of those options work for you?

If those directories are all that are needed I imagine using the systemd-tmpfiles method I mentioned could work. Or even just a simple script that runs on startup and creates those directories before the iotedge daemon even starts.

Do you think if thereā€™s any other possible workaround in the TorizonCore to prevent the OSTree from cleaning up the files in /var/lib/ folder in the build process?

The OSTree recipe/logic is not maintained by Toradex in our meta-layer. We inherit this from the meta-updater layer. You can try and modify that to change this logic but I imagine the maintainers there had good reason to do this. I canā€™t say what effects might happen if you try to change this default behavior.

Best Regards,
Jeremias

15

Jeremias,

Thanks for the rapid response. Iā€™m not 100% sure if /var/lib/aziot/ is the only directory required to start and run the iotedge daemon/service. But Iā€™ll try the systemd-tmpfiles method to add the missing home directory and see if I could start the edge daemon/service and make the edge runtime to work.
And I agree with you. Modifying the OSTree recipe/logic may not be a good idea and could potentially create unforeseen issues.

16

Let me know how things work out for you.

1 Like
17

Hi Jeremias,

I manually created the missing home directory and its subfolders in /var/lib/ for iotedge daemon and then assigned the owner to for user and group. The issue I had seems to be resolved though there are still a few minor issues. Next step is to try systemd-tmpfiles method and have the OS automatically create the directory in /var/lib/ and assign the correct owner to it.