RootFS get's stuck when booting from USB

Technical Support
, , ,
1

I’ve got our IMX8MP booting from SD card and have spent a long time working u-boot out.

I’m now trying usb0.

I can get the device tree and overlays loaded, then the Kernel loads (from USB 0:1) but rootFS get’s stuck.

I’ve tried various bootargs
setenv bootargs "console=ttymxc2,115200 root=/dev/sda2 ro rootwait rootdelay=10 "
setenv bootargs “console=ttymxc2,115200 root=PARTUUID=076c4a2a-01 ro rootwait rootdelay=10”

but with no success.

I’ve been looking around the notion of the SCSI USB drivers not being ready, but I’m confused given the kernel loads.

I’m loading the image on to a USB stick on windows in the same manner as my SD card.

I’m about to patch my u-boot config:

/*ORIG*/
#define BOOT_TARGET_DEVICES(func) \
    func(MMC, mmc, 1) \
    func(MMC, mmc, 2) \
    func(DHCP, dhcp, na)

/*PATCH*/
#define BOOT_TARGET_DEVICES(func) \
    func(MMC, mmc, 1) \
    func(MMC, mmc, 2) \
    func(DHCP, dhcp, na) \
    func(USB, usb, 0)

to see if this helps.

these Kernel settings are all “y” NOT “m”

CONFIG_USB_XHCI_HCD=y
CONFIG_USB_STORAGE=y
CONFIG_SCSI=y
CONFIG_BLK_DEV_SD=y (an "m" by default)

has anyone got any clues?

thanks as usual in advance.

2

Hi, I’m the Toradex AI Assistant.

I might make mistakes. Our support team will step in if needed. If I answered your question, please tick the “Solution” checkbox. For follow-up questions to me, please use @ToradexAI in your post.

Did you like my answer? :+1: :-1:

Hello! Thanks for sharing the details of your setup and what you’ve already tried.

You are on the right track: if the kernel loads from USB, your U-Boot changes and kernel driver settings are largely in order for the early boot stages. The problem likely relates to how the root filesystem on the USB is referenced or how the USB device is being handed over between U-Boot and the kernel.

Here are some things to check:

1. U-Boot Environment Variables

Make sure you set all the necessary variables for USB boot so both the kernel and root filesystem are loaded from USB. Verify the following in U-Boot:

setenv boot_targets 'usb0 mmc1 mmc2 dhcp'
setenv boot_devtype usb
setenv boot_devnum 0
setenv boot_part 1        # First partition (with kernel/device-tree)
setenv root_devtype usb
setenv root_devnum 0
setenv root_part 2        # Second partition (with rootfs)

You can check variables with:

printenv

More details:
U-Boot Boot Variables

2. USB Root Filesystem Discovery

Your kernel bootargs:

  • Try root=/dev/sda2 for the second USB partition.
  • Double-check with ls /dev on the rootfs image (if you can boot it from SD card)—make sure the device name matches.

It’s also good to have both rootwait and rootdelay=10 in place, as you do.

3. Partition Layout & Filesystem

The USB stick should have:

  • Partition 1: FAT, contains kernel + device trees
  • Partition 2: ext4, contains the rootfs

Citation & Example

4. Kernel Message Debugging

If kernel hangs after “Waiting for root device /dev/sda2”, you may want to increase verbosity:

Add debug to your bootargs for verbose logs. Watch for lines like:

scsi host0: usb-storage ...
sd 0:0:0:0: [sda] Attached SCSI removable disk

If you don’t see those, USB/SCSI/Storage driver loading may still be the issue.

5. U-Boot Patch

Your patch to BOOT_TARGET_DEVICES is correct and required for U-Boot to actively scan USB.

Summary Checklist

  1. U-Boot variables: Set both boot and root device/partition variables specifically for USB.
  2. Partition layout: FAT (kernel/DT) then ext4 (rootfs) on stick.
  3. Kernel drivers: All storage drivers as built-in (y).
  4. Bootargs: Use /dev/sda2 initially, add rootwait rootdelay=10 debug.
  5. USB stick preparation: Format under Linux if possible, as Windows can sometimes introduce quirks.
  6. Watch boot logs: Look for USB/SCSI device attach messages.

Let me know what logs you see around SCSI/USB detection and if you spot any messages like “Waiting for root device /dev/sda2”. If the kernel never sees a SCSI device, focus on kernel driver/bootargs; if it sees them but won’t mount, double-check partition format and contents.

You can find more about the procedure here:
How to boot from USB stick
U-Boot variable documentation

Let me know how it goes or if you need example environment settings!

Best regards,
Toradex AI Assistant.

3

Hi @geeday!

The answer from ToradexAI seems reasonable.

Please let us know if it helps you.

Otherwise, I will test it on my side.

Best regards,

4

Hi, thanks for the reply. I’ve been looking over the last few more days.
I see this bit about usb in my logs before the hang, but there are NO other reference to the USB drivers.

 0.076494] SCSI subsystem initialized
[    0.076643] usbcore: registered new interface driver usbfs
[    0.076686] usbcore: registered new interface driver hub
[    0.076730] usbcore: registered new device driver usb

this image is a comparison of the boot logs between LSH: boot from USB and RHS: booting from internal mmc.

image
image1715×769 118 KB

the image I put on the emmc and USB are the same wic file, so I am wondering if the partition-uuids are the same, so the system is getting confused as the same part-uuid ends up in bootargs.

I’ve just spent time trying to build a seperate image for the USB which includes a different part-uuid, I think we can see this from:

Waiting for root device PARTUUID=8787b0e1-95fd-4c57-93a5-49de1b34ef8a.

I’ve read that adding:
init=/bin/sh
to bootargs might help with diagnosis.
then there is the use of initramfs to make sure USB drivers are loaded early enough, but this is getting a little confusing to me now.

5

For reference, I took one of these nightly builds to try as my image:
https://artifacts.toradex.com/artifactory/tdxref-oe-prerelease-frankfurt/scarthgap-7.x.y/monthly/7/verdin-imx8mp/tdx-xwayland/tdx-reference-minimal-image/oedeploy/

the “wic file” section at the end of this link suggest it should be quite simple to do:

6

Hi @geeday !

During my tests, I managed to reproduce the behavior you are getting as well as advance a bit more.

Please allow me to make some more tests tomorrow. I am hopeful that I will be able to share with you a solution.

Best regards,

7

Thanks very much for your time.

8

Hi @geeday !

I advanced a bit, but didn’t manage to sort this out yet.

My setup:

  • Verdin iMX8MP Q 4GB WB IT V1.1A
  • Dahlia V1.1C / Verdin Development Board V1.1A
  • Reference Minimal Image from BSP 7.2.0 (quarterly)

While investigating, I enabled as builtin (=y) the following kernel configs. Please note that I am not sure that all of them are needed (nor if there is need for more configs to be enabled)

  • CONFIG_BLK_DEV_SD
  • CONFIG_USB_UAS
  • CONFIG_USB_DWC3
  • CONFIG_USB_DWC3_IMX8MP
  • CONFIG_PHY_FSL_IMX8MQ_USB

And to boot, I run the command from U-Boot (running from eMMC): run usb_boot

With this, it seems like the kernel tries to load the rootfs from the USB, but, in my case, the USB stick (or the USB HUB) is disconnected and reconnected during boot, which seems to cause issues.

So more time would need to be invested in it to understand what is going on, and more time to try to make it work.

With it, I would like to understand your use case: why you need this?
Is this related to a development workflow?
Or it is related to a requirement of your project?

Best regards,

9

Thanks again; I’ll add those kernel updates today or tomorrow.

We’ve started to design our own “board” that will accommodate the Verdin IMX8MP in our product; I’ve been in contact with one of your business people about the project.

We’re in the process of understanding the board (and O/S); how to update both locally and OTA and how to recover bricks that might end up in the field. I’ve been learning about u-boot and the various boot targets. I expect booting from USB to help with debug and development. It’s quite a big learning curve all round and certain aspects might not be so helpful, but the learing around this will be invaluable

10

Hi @geeday !

Thanks for the details :slight_smile:

Very nice. I hope you are using Torizon for this :wink:

Do you absolutely need to use USB stick? You could use e.g. TFTP/NFS/DHCP to boot your module from the network. IMHO, this method makes things even easier than using USB stick. Not long ago, I was providing help about this in this other thread: NFS -: server not responding, still trying -upstream-Tezi_7.x - iMx6 - #6 by henrique.tx.
What I did there was on a Reference Image of Toradex BSP. But it might help you as well, since I suppose you are on Torizon OS (which gives you OTA capabilities out-of-the-box).

Totally agree with you. :smiley:

Best regards,

11

Hi @geeday !

Good news!

After getting some more help, we managed to make it work.

These are the kernel configs needed as builtin:

CONFIG_BLK_DEV_SD=y
CONFIG_USB_DWC3=y
CONFIG_USB_DWC3_IMX8MP=y
CONFIG_USB_ONBOARD_HUB=y
CONFIG_USB_ROLE_SWITCH=y
CONFIG_PHY_FSL_IMX8MQ_USB=y

Please note that these are specific for Verdin iMX8MP and these changes were made on top of BSP 7.2.0 (quarterly release).

I hope this solves your needs. Let us know how it goes on your side.

FYI , here is the boot log (I edited the original /etc/issue on the rootfs so the final boot message makes it clear that it comes from the USB stick):

boot-usb-verdin-imx8mp-7.2.0.log (47.8 KB)

Best regards,

12

I tried TFTP a while ago, but my images are currently quite large (we have 2 boot partitions and 2 rootfs); I’m adding many more tools than are required for debug purposes. Given this initial large size, TFTP was way too slow and I’ve not really investigated further,

At somepoint we’ll have a shared data partition, which could be GB’s of space (the Verdin has 32GB of space), I need to understand how to create that at the most appropriate time in order to keep my wic file to a sensible size.

Regarding Torizon, it is an option for us, but for now, we’re going through the pain of learbing Yocto :slight_smile: we have access to a resource that is way more experienced than we are, which we have utilised a few times, but they do not know Toradex hardware as much the support you are offering.

13

Awesome, thanks; I’ll try very soon.
I’d like to think the journey we’ve been on is helpful for many more people than me and potentially the wider Toradex development team.

14

Hi @geeday !

I see. AFAIK, TFTP should not be much impacted by image size. TFTP would load kernel and device tree [overlays] (so usually a few MB) and the rootfs would be loaded on demand (and not all of it at once). But I understand that having it working according to your requirements was not something worth investigating at the time :slight_smile:

We look forward to supporting you on Torizon. I do agree that learning Yocto is good, since it is a tool widely used (and we use it to distribute Toradex BSP and Torizon OS as OpenEmbedded/Yocto layers), but Torizon will provide you even more “out-of-the-box value” with all its development and maintenance ecosystem and ready-to-use over-the-air features. One thing is investing time in building the OS, another thing is investing time in creating your application where your project will bring actual value to your end customer :smiley:

Great! Looking forward to knowing how it goes on your side :slight_smile:

Best regards,