The device we are building involves sensitive plastic parts and delicate senors, making it not very convenient to open up when we would need to re-flash the device.
Typical use-case being periodically applying a new build of the image on internal QA-related devices.
Being able to just plug in a USB stick with Easy Installer seems like the best solution for us, but after having read Load Easy Installer From External Media (SD Card/USB Stick) (toradex.com) some things are unclear:
- From docs:
**alternative method, not supported by Toradex officially** , as we technically cannot guarantee that this works all the time and also across different versions. What exactly are the limitations here? Is this more related to the possibility of the OS that is currently on the flash device being in a corrupt state, or is this a software version/compatibility issue?
- Also from docs:
following order on our BSP: 1. SD Card, 2. internal eMMC, 3. USB Stick as the default boot order, is there a reason not to push the USB stick as primary as a situational fallback?
Seems like a generally convenient thing that when you need extra-ordinary operations (like re-flashing), you just plug in the recovery stick, otherwise defaulting onto existing file system.
System:
Verdin iMX8M Mini
Dahlia (and custom) carrier board
Torizon Core - Yocto Dunfell
Greetings @ebrodlic,
Allow me to clarify some of the points here.
What exactly are the limitations here?
So first of all the reason we state this is not a “supported” method is simply because we can not guarantee this function working always. Basically this feature works right now, but it may not work in the future. There are factors outside of Toradex’s control that may break how the functionality here. Furthermore, in case of future breakage there’s no guarantee that we’ll fix/can fix functionality depending on how it broke.
Another limitation is that this method depends on the device at least being able to boot into U-Boot. If for some reason the device becomes bricked and not even U-Boot is operational than this method won’t work.
For these reasons while the method does work at the time of writing, it’s not something to be relied on long term.
is there a reason not to push the USB stick as primary as a situational fallback?
Well very generally speaking there could be minor security concerns with having USB be your primary boot media. Though ultimate boot order depends on use case. Otherwise the default boot order we chose is just what we feel is a good default, since most of our customers want to boot off the internal flash. Basically you’re free to change it from the default if you feel it better suits your use case. Though keep in mind what I said above, we don’t guarantee boot methods apart from the default one to always work in the future. Meaning testing and maintenance of this would fall on the customer to ensure future functionality.
Best Regards,
Jeremias
Hello @jeremias.tx,
Thanks for sharing insight.
Basically this feature works right now, but it may not work in the future.
When you say the future, do you mean potential new Torizon images built on top of a newer (release) branch of open-embedded, say after Dunfell? Or is this dependant on something internal at Toradex?
Roughly what I’m trying to achieve:
- Developer or QA engineer receives a brand new device (Easy Installer pre-installed by default)
- Stable image version (lets say - 1.0) of Torizon Core gets installed and is working correctly
- As the device is used for development purposes, we modify the boot order to scan the usb device first (for future need)
- Developer makes a new a new image version (1.1) with serious (potentially problematic) OS changes
- Version 1.1 passes automated test (qemu) in terms of initial boot and startup
- We insert the Easy Installer stick, reboot, EI starts from RAM, insert stick with 1.1 and install the new image
If we can accomplish that without the need to open the device, and simply via USB port and stick/keyboard/SSH/VNC that would help our development and operations a lot.
Obviously for actual future production updates, a regular update mechanism such as ostree is the choice.
But for early development prototyping, as well as periodically resetting the QA and staging machines, being able to easily go back to last “clean” image build that we have would be a great thing.
Also, for this discussion lets assume that we don’t have u-boot related bricks or can at least detect them via qemu so that they are not an issue.
Does that mean that if we simply:
a) are trying to reinstall a previously working OS version (same release branch)
b) use a previously working Easy Installer on stick version
we would have reliable way to re-install OS from external device?
And that we would only have to worry about potential new major releases from Toradex?
Regards,
Edin
When you say the future…
Basically when I say future I mean any version of the software that isn’t the current release. This feature works right now for this exact version of software. If this feature happens to work in the future it’s a lucky coincidence, since this isn’t something we currently test for.
There’s multiple software change that could affect this feature. Changes in U-Boot, changes from NXP, changes in Easy Installer. This is why we don’t “officially” support this method.
Does that mean that if we simply:
a) are trying to reinstall a previously working OS version (same release branch)
b) use a previously working Easy Installer on stick version
we would have reliable way to re-install OS from external device?
And that we would only have to worry about potential new major releases from Toradex?
I would say yes since if you’re using the same versions of things where this feature is known to work it should continue to work.
As for your development flow/CI is there any particular reason OSTree/OTA couldn’t be used in this situation? With OSTree/OTA you’d be able to rollback to previously known “good” versions.
Alternatively on your testing/development boards you could implement a recovery button as we have with our Verdin development board. Then you could put the module into recovery with a push of a button then load Easy Installer over USB OTG.
Best Regards,
Jeremias
As for your development flow/CI is there any particular reason OSTree/OTA couldn’t be used in this situation? With OSTree/OTA you’d be able to rollback to previously known “good” versions.
Well this would be a nice way to distribute new releases of a relatively production-ready image.
For early development prototyping (for example trying out widely different options, rather than incremental upgrades), it is nice to have a simple way to just try out an image on a device separately.
For development purposes even, I may be content with working with the board directly but the scenario I am mostly interested about is the QA/staging device that has been tampered with, not the one that receives a faulty OS upgrade (thus being able to rollback).
For example, if a machine is used for verification for a prolongued periods time, eventually someone will either manually alter configuration or move content/data around, download/replace files “temporarily”, alter user entries or the device will contain changes (app cache) that may impact new desired “fresh” testing.
For these reasons, it is often better to reinstall a “fresh” clean, rather than navigating through changes.
I am guessing OSTree might be used to revert back to “day 0” and the last successful image build state, but I am not sure about the process yet.
Or is this the case you had in mind?
Alternatively on your testing/development boards you could implement a recovery button as we have with our Verdin development board. Then you could put the module into recovery with a push of a button then load Easy Installer over USB OTG.
This does sound very useful, but I would have to check with the manufacturer.
With regards to the applicability of OTA/OSTree in this use case let me clarify a couple of points.
If your main requirement to use Easy Installer during QA is to flash a fresh/clean image then yes I agree Easy Installer is better than OTA/OSTree for this purpose. This is because Easy Installer will re-flash the entire eMMC leaving no trace of the previous image.
However OSTree could do this to a degree as well. With OSTree almost the entire root filesystem is replaced with the new update package. I say almost since a couple of locations on the filesystem are persistent through updates (/etc, /var). Also OSTree does not affect the bootloader/U-Boot. This is to say that OTA/OSTree won’t wipe absolutely everything to a “clean” state. However if these limitations are kept in mind it could possibly still be used for your QA use-case.
Of course you’re still free to try the method of loading Easy Installer from SD/USB. Just keep in mind my previous comments about this, and be prepared that the method may just suddenly “not work” one day.
Ultimately I think the best course of action here for your use-case would be to implement a recovery button on your QA hardware. This way Easy Installer can be easily loaded through supported means without having to open up your test fixture. However i understand this requires physical hardware changes which aren’t always easy/possible to do.
Best Regards,
Jeremias
Thanks for the tips.
We will try to focus on having accessable recovery button and USB OTG on the testing modules used for verification, alternatively having a USB OTG capable board nearby for flashing when needed.
Sometimes I just dont want to risk not knowing about potential config changes at /etc
Regards,
Edin
Glad I could help out with this. In general a recovery button is usually a good idea, especially if your device can’t be easily opened up to recover using the hardware shorting method.
Best Regards,
Jeremias
