Colibri VF61 U-Boot Update Stuck at "ubi0: scanning is finished"

Technical Support
, ,
1

Dear Support Team,

we’re experiencing a problem with image updates on our Colibri VF61 modules using U-Boot.

Our process is: prepare SD card, reboot, then run setupdate and run update. This usually works, but on some devices, the update hangs after “ubi0: scanning is finished” (see screenshot).

The hang duration is inconsistent – sometimes it resolves after 30-90 minutes, other times we have to re-attempt the update multiple times.

  • Is this a known issue?

  • What could cause this long delay after “ubi0: scanning is finished”?

  • Could flash memory defects on the VF61 be a cause?

  • Could the SD card itself be problematic?

Thanks for your help,

Screenshot 2025-09-25 120854
Screenshot 2025-09-25 120854551×608 184 KB

2

Hi, I’m the Toradex AI Assistant.

I might make mistakes or occasionally get you wrong. Our support team will step in if needed.

I understand you’re experiencing hanging issues during image updates on your Colibri VF61 modules. Let me address your concerns based on the available documentation.

Understanding the Issue

The hang after “ubi0: scanning is finished” is related to the UBI (Unsorted Block Images) filesystem initialization process. This is part of the raw NAND flash management system used by the Colibri VF61 (1).

Potential Causes and Solutions

Flash Memory Issues

Yes, flash memory defects on the VF61 can indeed cause these delays. The UBI scanning process examines the entire NAND flash for bad blocks and filesystem integrity (1). When there are marginal or failing blocks, this scanning can take significantly longer.

SD Card Problems

The SD card can also be problematic. Some SD cards may not work properly with the recovery process (2). The documentation specifically mentions:

Some SD card may not work because of the way they are structured (2)

If you suspect SD card issues, try using the recovery image provided by Toradex and replace the files with your latest package (2).

Recovery Mode Alternative

If the standard update process continues to fail, you can use recovery mode as described in the VFxx Recovery Mode documentation. This involves:

  1. Following the recovery mode procedure to run U-Boot from recovery mode (2)
  2. Using the flashloader command to write the bootloader (2)
  3. Using flashimage to write the OS image (2)

Recommended Troubleshooting Steps

  1. Try a different SD card - Use a high-quality SD card formatted with FAT32
  2. Check for bad blocks - The extended scanning time often indicates NAND flash issues
  3. Use recovery mode - If standard updates continue to fail, recovery mode provides an alternative path (2)
  4. Monitor power supply - Ensure stable power during the update process

Flash Layout Considerations

The Colibri VF61 uses raw NAND with UBI/UBIFS filesystem management (1). You can use MTD utilities to examine the flash health once the system is running normally.

The inconsistent timing (30-90 minutes) suggests the UBI scanning is encountering problematic areas in the NAND flash that require additional time to process or work around.

Best regards, Toradex AI Assistant.

Did you like my answer? :+1: :-1:

If I answered your question, please tick the “Solution” checkbox.

For follow-up questions to me, please use @ToradexAI in your post.

3

@ToradexAI ,

I am seeking guidance on how to evaluate the health of the internal Flash memory on a Toradex VF61 module using the mtd-utils tool.

My understanding is that mtd-utils can be used for this purpose, but I’m looking for specific instructions. Could you please provide:

  1. The specific mtd-utils command(s) recommended for evaluating Flash health on the VF61.

  2. An explanation of the output from these commands that would indicate the health status (e.g., error counts, bad blocks, wear leveling information).

  3. Any additional considerations or best practices for Flash health monitoring on this specific hardware.

Furthermore, I’d like to know if mtd-utils is the only way to check the Flash health on the VF61, or if there are other recommended tools or methods available (e.g., kernel interfaces, dedicated diagnostic tools, or specific drivers that expose health metrics)?

Best regards.

4

Flash Health Monitoring on Colibri VF61

The Colibri VF61 uses 512MB NAND (8 Bit) raw NAND flash memory (1), which requires specific approaches for health monitoring compared to eMMC-based modules.

Flash Memory Type on VF61

The Colibri VF61 features Raw NAND (SLC) flash storage (2). Unlike eMMC devices that have built-in controllers providing standardized health metrics, raw NAND requires different monitoring approaches (2).

MTD-Utils for Raw NAND Health Monitoring

For raw NAND devices like the VF61, mtd-utils is indeed the primary tool for flash health assessment. Here are the key commands:

1. Bad Block Information

mtd_debug info /dev/mtdX

(2)

This command provides basic information about the MTD device, including bad block counts.

2. Bad Block Scanning

nandtest /dev/mtdX

(2)

This performs a comprehensive test of the NAND flash, identifying bad blocks and potential issues.

Understanding the Output

When using these tools, look for:

  • Bad block count: Higher numbers indicate flash wear (2)
  • ECC errors: Error correction events that suggest degrading flash health (2)
  • Write/erase cycle estimates: Though not directly provided, can be inferred from bad block progression (2)

Flash Health Considerations for VF61

Endurance Characteristics

The VF61 uses SLC NAND, which provides (3):

  • Highest endurance compared to MLC/TLC
  • Highest lifetime
  • Highest performance
  • High cost per capacity

SLC NAND typically supports around 100K erase cycles (2), significantly higher than MLC flash.

Write Amplification Impact

Be aware that flash cannot be rewritten without being erased first, and this must be done at the erase block level (up to 512 KB) (2). This write amplification effect means writing small amounts of data can trigger larger erase operations (2).

Alternative Monitoring Methods

Besides mtd-utils, you can monitor flash health through:

1. UBI Statistics

Since the VF61 uses UBI/UBIFS filesystem (2), you can check:

cat /sys/class/ubi/ubi0/*/bad_peb_count

2. Kernel Log Monitoring

Monitor dmesg output for:

  • UBI error messages
  • Bad block notifications
  • ECC correction events

3. Filesystem-Level Monitoring

Monitor write patterns and frequency to estimate wear based on your application’s usage patterns (2).

Best Practices for Flash Health

  1. Minimize write operations to extend flash life (2)
  2. Monitor write behavior of your application (2)
  3. Run stress tests to validate flash endurance for your use case (2)
  4. Avoid using full capacity to improve wear leveling efficiency (2)

Important Note

Unlike eMMC devices that provide standardized health metrics (as shown in TorizonCore with mmc-utils (4)), raw NAND devices like the VF61 require more manual monitoring approaches using MTD utilities and system-level observation.

The VF61’s SLC NAND provides excellent endurance characteristics, but proper monitoring using mtd-utils and system observation remains essential for long-term reliability assessment.

Best regards, Toradex AI Assistant.

Did you like my answer? :+1: :-1:

If I answered your question, please tick the “Solution” checkbox.

For follow-up questions to me, please use @ToradexAI in your post.

5

UBI driver reports erase counters when kernel boots. So if you didn’t flasherase whole MTD device, try

dmesg | grep mean to see average and maximum of erase counters.

It is useful to read whole MTD device and look at ECC corrections numbers like this

nanddump /dev/mtd3 > /dev/null

Unfortunately on VF61, 60 byte ECC mode “corrects” properly erased pages producing fake bitflip of bit5 at offset 0x5FD. So you get useless enormously high ECC bitflip numbers. Very old Toradex BSP, 2.6 or older used lower ECC mode (32? bytes) and nanddump should operate properly.

If you wish fixing 60 byte ECC, try patching vf610_nfc.c driver like this

image
image709×561 58 KB

6

Problem Description:

I am encountering an UBI init error 22 on my Colibri VF61 module, preventing proper boot and image updates. This issue arose after a sequence of diagnostic and update attempts.

Sequence of Events:

  1. Initial Symptom: I observed unexplained behavior/crashes, leading me to suspect NAND integrity.

  2. nandtest Execution: I ran nandtest /dev/mtd3. The test completed, reporting some ECC corrections. (Please note: I understand nandtest is destructive).

  3. Post-nandtest Issue: Following nandtest, I attempted to update the module’s image using U-Boot’s run update command. This process failed.

  4. Current State: Now, on every boot, the module displays UBI init error 22 during the UBI attachment phase, and U-Boot reports a “bad CRC” with default environment loading. (See attached screenshot for exact output).

Specific Error Message:

UBI init error 22

Context & Questions:

  • Impact: The module is currently unbootable into the OS and cannot be updated via run update.

  • Root Cause Suspect: Given the nandtest output showing ECC corrections and the subsequent UBI errors, I suspect the underlying NAND flash may be damaged or severely degraded.

  • Action Requested:

    1. Can this issue be resolved by re-flashing the entire module using the Toradex Easy Installer?

    2. Is there a high probability that the Colibri VF61 module’s NAND flash is permanently damaged, requiring module replacement?

    3. Are there any further diagnostics I can run from U-Boot to confirm the hardware integrity?

      Screenshot 2025-11-11 120837
      Screenshot 2025-11-11 120837495×565 10.2 KB

7

@ToradexAI

In a other device a get with the commando “cat /sys/class/ubi/ubi0/bad_peb_count” 8 PEBs is that a normal value?

image

8

Understanding Your VF61 NAND Issues

UBI Error 22 Analysis

The UBI init error 22 you’re experiencing indicates serious NAND flash corruption, likely caused by the destructive nandtest operation. This error typically means the UBI subsystem cannot properly initialize due to corrupted metadata or excessive bad blocks.

Bad PEB Count Assessment

Regarding your question about 8 bad PEBs (Physical Erase Blocks): This is within acceptable limits for NAND flash devices. The Colibri VF61 uses raw NAND flash, and some factory-marked bad blocks are normal (1). However, the combination of UBI errors and bad blocks suggests progressive flash degradation.

Recovery Options

1. Flash Recovery via Recovery Mode

Since your module cannot boot normally, you’ll need to use recovery mode to restore functionality (2). The process involves:

  1. Enter Recovery Mode following the VFxx Recovery Mode procedure
  2. Flash the bootloader using the recovery process (3)
  3. Restore the complete image including filesystem areas

2. Complete NAND Restoration

For severe corruption like yours, you may need to perform a complete flash restoration (3):

>eraseflash filesystem
Are you sure you want to erase flash(0x4020000-0x1FFFFFFF)?y
Erasing 3583 flash blocks from 513.

(3)

This command erases the filesystem area completely, which may help recover from UBI corruption.

Flash Layout Context

The Colibri VF61 flash layout shows that your filesystem area starts at block 512 and extends to the end of the flash (1). The UBI error suggests corruption in this region, which nandtest likely damaged by writing test patterns.

Diagnostic Commands from U-Boot

From U-Boot, you can run these diagnostics:

  1. Check NAND status:

    nand info

  2. Examine bad blocks:

    nand bad

  3. Test basic NAND operations on a small area to verify hardware functionality

Recommendations

Immediate Actions:

  1. Use recovery mode to attempt complete reflashing (2)
  2. Erase the entire filesystem area before attempting to restore (3)
  3. Monitor for additional bad blocks during the recovery process

Hardware Assessment:

  • 8 bad PEBs is acceptable for a NAND device, but combined with UBI errors suggests degradation
  • The module may still be recoverable if the bootloader area remains intact
  • Complete hardware failure is possible if recovery mode attempts fail consistently

Recovery Probability:

  • High chance of recovery if you can successfully enter recovery mode and flash new images
  • Hardware replacement needed if recovery mode fails or if bad block count increases significantly during recovery attempts

The key is attempting recovery mode first - if the bootloader area is intact and you can flash successfully, the module should be recoverable despite the current UBI corruption.

Best regards, Toradex AI Assistant.

Did you like my answer? :+1: :-1:

If I answered your question, please tick the “Solution” checkbox.

For follow-up questions to me, please use @ToradexAI in your post.

9

Above doesn’t test anything, it just reports ECC corrections registered by kernel since last boot. To better understand ECC condition of whole MTD device, it is better to scan read whole MTD with nandtest /dev/mtd3 > /dev/null. Very big numbers mean either natural flash cells discharge over time, perhaps nothing was written for very long and those numbers grow. Or flash wear level is quite high and cells discharge faster due to worn flash.

Did you check UBI erase counters? Mean and max? ubi part ubi command in U-Boot as well is reporting such information. You should see there tens of thousands to start concerning.

Since UBI is dead, you need to recover. The best is to use ubiformat command from Linux, which would preserve precious erase counters. Of course you need to boot from other media like SD or USB with run mmc_boot or run usb_boot. I see your U-Boot is compiled last year, so you should be able to let bitbake create something like wic image and then deploy it to SD/USB. It is possible as well to partition your SD/USB on linux, one root partition, one boot partition, make boot partition bootable, untar to bootpart kernel and dtb, untar rootfs archive to rootfs. It should boot.

If you don’t mind current erase counters, you may erase ubi partition, then your update script should reformat ubi partition automatically with 0 erase counters when update script executes ubi part ubi command. Recreation of ubi volumes is part of your update process; run update should succed.

I’m not sure at which point you get your UBI error. Perhaps it is just broken rootfs volume. Perhaps try first removing rootfs:

ubi part ubi

ubi remove rootfs

end then try again run setupdate; run update

If still error, then of course nand erase.part ubi and you lose your erase counters inadvertently. Repeat setupdate-update.

10

Thanks for your responses.