Is there an easy way to create a dual rootfs tezi image?

I am building a custom yocto image for a Verdin iMX8MP SoM.
My image is based on the tdx-reference-minimal-image.
As I cannot use Torizon OS at present I have been looking at alternative update implementations.

Ideally I am looking to implement one that uses an A/B Update process, but I am unsure if the current recipes have a way for me to “duplicate” the deployment of the rootfs to 2 partitions on the eMMC.

I will also be using the tdx-data-partition and tdx-signed classes from the meta-toradex-security layer in my image.
What I need initially is to have the second copy of the rootfs as a partition on the eMMC along with the existing partitions that are created.
I am also interested in how to add an overlayfs for the /etc directory as when I switch to the tdxref-signed class the rootfs will be read-only.

Is there anything already to make this possible with minimal changes in my custom layer? And are there any caveats/gotchas I should know about before I embark on a possibly impossible mission?

Regards,

1 Like

Hi, I’m the Toradex AI Assistant. The Toradex Support team will follow up with you shortly!

Did you like my answer?

:+1: :-1:


Hello,

To set up a dual root filesystem (rootfs) for A/B updates on your Verdin iMX8MP SoM using Yocto, you can follow these steps:

  1. Partitioning the eMMC: Modify your Yocto build configuration to define two rootfs partitions. This involves editing the wks file used by the wic tool to create the image, ensuring both partitions are included.

  2. Duplicating the Rootfs: Create a custom Yocto image recipe that deploys the rootfs to both partitions. You might need a post-processing script to copy the rootfs to the second partition after the initial deployment.

  3. Bootloader Configuration: Configure your bootloader (e.g., U-Boot) to support booting from either rootfs partition. This typically involves setting environment variables or scripts to switch between partitions based on update status or failure detection.

For implementing an overlay filesystem (overlayfs) for the /etc directory:

  1. Overlayfs Setup: Configure overlayfs to mount a writable layer over the read-only /etc directory. This can be done by creating a writable partition or using a tmpfs for the overlay’s upper layer.

  2. Integration with Yocto: Modify your image recipe to include the necessary overlayfs setup scripts and ensure they execute at boot time to mount the overlay.

  3. Considerations: Be mindful that overlayfs can add complexity in managing persistent changes across reboots and updates. Ensure your update mechanism accounts for changes in the overlay.

Regarding the tdx-data-partition and tdx-signed classes, ensure your partitioning and signing processes are compatible with these classes. You may need to adjust your build configuration to accommodate the security features they provide.

While setting up a dual rootfs and overlayfs can be complex, it is achievable with careful planning and configuration in your Yocto build environment. Thorough testing will help ensure reliability and robustness in your update process.

Best regards,
Toradex AI Assistant

Please, also check if these links can help you solve your problem:

Automount overlay for /etc.

I have had a quick delve into the image_type_tezi.bbclass in meta-toradex-bsp-common, and it would seem this is not currently possible.

As this is something I will need I have created a patch for the kirkstone branch that would appear to add the ability I require.
Is there a way I can submit this as a PR/MR for consideration into the BSP?

0001-Adding-dual-rootfs-support.patch (1.8 KB)

With this patch and adding TEZI_DUAL_ROOT_ENABLED = "1" to local.conf, this appears to create the required manifest.json to create a second rootfs partition with the same content as the existing rootfs partition.
NOTE: this partition in my case will be added to “/dev/mmcblk2p3” and will make my data tezi partition “/dev/mmcblk2p4” - Noted here as this will mean a change to the configuration of the tdx-encrypted variables to point to the correct partition if/when dual boot is configured.

Updated patch as I believe there were a few typos in my original one…
0001-Adding-dual-rootfs-support.patch (1.8 KB)

Latest updated patches from testing on my local builds.

0001-Adding-dual-rootfs-support.patch (1.8 KB)

I have also got a patch that makes the new data partition easier to extract and deploy.
This should extract the specified data mount point from the rootfs and create a data.tar.xz image file that can then be referenced in the image manifest.
Any recipes putting data into the data folder using the normal FILES approach will automatically get pulled out into the new data image archive.
Extra possible benefit that this can also be used in OTA updates (if not using the TorizonOS implementations).

0002-Adding-data-partition-image-creation.patch (4.9 KB)

Another question that could benefit from these patches can be found here:
tdx-encrypted-with-partitions

Hi @izzy_building!

Sorry for the delayed response.

Were you able to solve your issue?

Unfortunately, we cannot merge your patch since Kirkstone is in maintenance mode, which means we only apply fixes and not features. These patches should be for Scarthgap in order to be merged.

About the update, what do you want to achieve with the A/B Update process?

If you need to do OS updates and Subsystem updates, you could use our Torizon Cloud. Besides that, using Torizon, you can do OTA updates and secure offline updates too.

Best regards.
Lucas Azeituno

Hi @lucas_az.tx ,

I have resolved my issues for this.
My solution is to apply these patches manually on the relevant repos.

I will soon be in the process of applying the full set of changes to scarthgap and will re-submit these once completed and tested (there were some small changes to simplify the implementation done here).

Regarding the Torizon OS suggestion, our use-case cannot use the cloud at all. Also, it was a very early decision made on the project specifically not to use Torizon OS (Something I hope to look into and change later).

Regards,

2 Likes

Hi @lucas_az.tx,

I have indeed.
For now I am applying my own patch over the kirkstone branch.

Unfortunately we cannot use torizon for our builds at all as it was vetoed early in the project before I started.
It will likely be a topic that comes up once our full delivery has been completed and it may be that our version 2 devices will have several updates and will use torizon OS instead.

Once we have this all finalised and delivered I will be working on the migration to Scarthgap, at which point I will re-submit the relevant patches in the hope you can merge them in and provide the extra capabilities.

Current additions include:

  • Dual BootFS
  • Dual RootFS
  • GPT Partitioning
  • Tighter integration of all these with the meta-toradex-security layer (but that will be in the GitHub repo and not here)

Regards,
Izzy

Hi @izzy_building

Thanks for providing the patches that you are using. I’m going to raise this point to the P&D team.

But, to be frank, it’s very unlikely that custom patches specific to your use case to be merged into Torizon.


Just asking, but have you considered using the Yocto Torizon Minimal with offline updates?

.

I would also be interested in having this feature added - my case is very similar to that of @izzy_building and I’m currently developing my own A/B boot, that would be also compatible with meta-toradex-security. I am a little bit afraid of any breaking changes introduced there in the future, so having those changes upstreamed would make the project maintenance less stressful :slightly_smiling_face:

Kind Regards,

Bartek

1 Like

Same here :waving_hand:

Would also be interested in such a feature.

Kind Regards,

Uwe

Hi @bosinski @buw3.

Thanks for the feedback.

I have forwarded this feature request to the P&D team.
We are going to keep you updated on this. :partying_face:

Just to get some context, what’s the reason for you not to use our OTA solution with Torizon OS?

In my case there are multiple reasons:

  • we already have all of our high-level embedded software written and ready, including integration with swupdate - all that is left is to add A/B partition scheme and to make sure that Secure Boot and hardening features work as intended, and integrate everything

  • because of that, OTA would be the only Torizon’s feature we could maybe benefit from, as we already have our custom OTA. We don’t need other features as they are already covered by our solutions, adapted to our own on-premise infrastructure.

  • dm-verity is not going work with ostree (source: How to secure the entire rootfs with dm-verity - #3 by jeremias.tx) - it’s a simple, yet very powerful integrity checking mechanism we care for and it’s yet another reason to stick with A/B for us

  • we’ve already talked with a salesperson about potential pricing in our use-case, and unfortunately the higher-ups’ decision was not to go with Torizon

Kind Regards,
Bartek

In out case it’s more or less the same. Management decision was against Torizon - for now. We are in the final state of development in out current project, for which we useign an A/B update machanism.

Kind Regards,

Uwe

Hi @allan.tx ,

In my case our project is nearing the end of initial delivery phase and will be rolling into the support phase.

Due to the initial project requirements very early on, and due to the lack of knowledge within the team it was decided that Torizon was not to be used and we went down an alternate path.

Initially for our Version 1 Product, this will not change as we have not the resources to go “back to square one” with the OS development.
However, after the initial roll-out has completed and we start work on a Version 2 Product, we will indeed bring this discussion back onto the table for discussion and review.
But it will never be used for the original units and as such we already have the various updates process and plans in place for that.

Also, most of our products require some very specific security related measures and accounting, which is provided by our implementation and would be an unknown if switching to Torizon at this time.
We would need a long discussion with yourselves and our customers before we could even consider that change.

Regards,
Izzy

Hi @izzy_building, @buw3 and @bosinski!

We’ve talked with the R&D team, and unfortunately, they won’t merge it because integrating an A/B partition model would conflict with the OSTree-based design and the tools that make Torizon’s update process robust and maintainable.

That said, if you ever consider switching to Torizon OS, we can support you in adapting your system.

Best regards.

1 Like

Hi @lucas_az.tx,

I am not sure I understand that decision as these changes just provide extra recipe variables that a user can override for the required use-case. By default it would remain exactly as-is. This should mean that it doesn’t affect anything (unless the Torizon OS changes make other significant modifications over this BSP repo?).

If this is still refused, would Toradex mind if a fork of the repo existed with these changes applied (likely on GitHub)? As I would not mind creating and maintaining that if it would be preferred (as an unofficial repo of course).

Regards,
Izzy

Hi @izzy_building!

I talked to the R&D team again, and they said that for an eventual PR, it would depend on a few things. For example, it shouldn’t affect our OSTree implementation and must be technically well implemented. I can’t give you certainty that it would be merged, but the maintainers would evaluate it.

Additionally, they don’t mind if you fork the repo and maintain it. It’s up to you :slight_smile:.

Best regards.