EasyInstaller installation from USB stick fails

Hardware: Colibri iMX6DL 512MB IT V1.1Y, we use our own carrier board

From time to time we fail to install our WindowsCE and application via Toradex EasyInstaller script from USB stick. The colibri module resulting state is the EasyInstaller does no more start up so we cannot re-try to install from USB stick. In bootloader output we see several BOOTP broadcast tokens.

How could one force the Colibri into this state? Our script works 99% of the time, could this be a timing problem somewhere?
We can recover the Colibris by loading EasyInstaller again = factory default.

Bootloader output (this time from Toradex Colibri Evaluation Board V3.2B):

Industrial temperature grade DDR3 timings, 64bit bus width.
Trying to boot from MMC1


U-Boot 2020.07-5.3.0+git.68f97c8d17f0 (Jul 06 2021 - 15:50:15 +0000)

CPU:   Freescale i.MX6DL rev1.4 at 792MHz
CPU:   Industrial temperature grade (-40C to 105C) at 45C
Reset cause: POR
DRAM:  512 MiB
PMIC:  device id: 0x10, revision id: 0x21, programmed
MMC:   FSL_SDHC: 1, FSL_SDHC: 0
Loading Environment from MMC... OK
In:    serial
Out:   serial
Err:   serial
Model: Toradex Colibri iMX6 DualLite 512MB IT V1.1Y, Serial# 11066458
Net:   eth0: ethernet@2188000
Hit any key to stop autoboot:  0 
MMC: no card present
switch to partitions #0, OK
mmc0(part 0) is current device
** Unrecognized filesystem type **
starting USB...
Bus usb@2184000: USB EHCI 1.00
Bus usb@2184200: USB EHCI 1.00
scanning bus usb@1 USB Device(s) found
scanning bus usb@2184200 for devices... 4 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found

Device 0: unknown device
ethernet@2188000 Waiting for PHY auto negotiation to complete......... TIMEOUT !
Could not initialize PHY ethernet@2188000
BOOTP broadcast 1
BOOTP broadcast 2
BOOTP broadcast 3
BOOTP broadcast 4
BOOTP broadcast 5
BOOTP broadcast 6
BOOTP broadcast 7
BOOTP broadcast 8
BOOTP broadcast 9
BOOTP broadcast 10
BOOTP broadcast 11
BOOTP broadcast 12
BOOTP broadcast 13
BOOTP broadcast 14
BOOTP broadcast 15
BOOTP broadcast 16
BOOTP broadcast 17

Retry time exceeded; starting again
ethernet@2188000 Waiting for PHY auto negotiation to complete......... TIMEOUT !
Could not initialize PHY ethernet@2188000
BOOTP broadcast 1
BOOTP broadcast 2
BOOTP broadcast 3
BOOTP broadcast 4
BOOTP broadcast 5
BOOTP broadcast 6
BOOTP broadcast 7
BOOTP broadcast 8
BOOTP broadcast 9
BOOTP broadcast 10
BOOTP broadcast 11
BOOTP broadcast 12
BOOTP broadcast 13
BOOTP broadcast 14
BOOTP broadcast 15
BOOTP broadcast 16
BOOTP broadcast 17

Retry time exceeded; starting again
## Error inserting "stdout" variable, errno=22
## Error inserting "stdin" variable, errno=22
Colibri iMX6 TEZI #

Bootloader environment:

Colibri iMX6 TEZI # printenv
arch=arm
baudrate=115200
board=colibri_imx6
board_name=colibri_imx6
board_rev=0112
boot_a_script=load ${devtype} ${devnum}:${distro_bootpart} ${scriptaddr} ${prefix}${script}; source ${scriptaddr}
boot_efi_binary=if fdt addr ${fdt_addr_r}; then bootefi bootmgr ${fdt_addr_r};else bootefi bootmgr ${fdtcontroladdr};fi;load ${devtype} ${devnum}:${distro_bootpart} ${kernel_addr_r} efi/boot/bootarm.efi; if fdt addr ${fdt_addr_r}; then bootefi ${kernel_addr_r} ${fdt_addr_r};else bootefi ${kernel_addr_r} ${fdtcontroladdr};fi
boot_extlinux=sysboot ${devtype} ${devnum}:${distro_bootpart} any ${scriptaddr} ${prefix}${boot_syslinux_conf}
boot_file=zImage
boot_net_usb_start=usb start
boot_prefixes=/ /boot/
boot_script_dhcp=boot-tezi.scr
boot_scripts=boot-tezi.scr
boot_syslinux_conf=extlinux/extlinux.conf
boot_targets=mmc1 mmc0 usb0 dhcp 
bootcmd=run distro_bootcmd; usb start ; setenv stdout serial,vidconsole; setenv stdin serial,usbkbd
bootcmd_dhcp=setenv devtype dhcp; if dhcp ${scriptaddr} ${boot_script_dhcp}; then source ${scriptaddr}; fi;setenv efi_fdtfile ${fdtfile}; if test -z "${fdtfile}" -a -n "${soc}"; then setenv efi_fdtfile ${soc}-${board}${boardver}.dtb; fi; setenv efi_old_vci ${bootp_vci};setenv efi_old_arch ${bootp_arch};setenv bootp_vci PXEClient:Arch:00010:UNDI:003000;setenv bootp_arch 0xa;if dhcp ${kernel_addr_r}; then tftpboot ${fdt_addr_r} dtb/${efi_fdtfile};if fdt addr ${fdt_addr_r}; then bootefi ${kernel_addr_r} ${fdt_addr_r}; else bootefi ${kernel_addr_r} ${fdtcontroladdr};fi;fi;setenv bootp_vci ${efi_old_vci};setenv bootp_arch ${efi_old_arch};setenv efi_fdtfile;setenv efi_old_arch;setenv efi_old_vci;
bootcmd_mmc0=devnum=0; run mmc_boot
bootcmd_mmc1=devnum=1; run mmc_boot
bootcmd_usb0=devnum=0; run usb_boot
bootcount=1
bootdelay=1
bootm_size=0x10000000
console=ttymxc0
cpu=armv7
defargs=enable_wait_mode=off galcore.contiguousSize=50331648
devtype=dhcp
distro_bootcmd=for target in ${boot_targets}; do run bootcmd_${target}; done
efi_dtb_prefixes=/ /dtb/ /dtb/current/
ethact=ethernet@2188000
ethaddr=00:14:2d:a8:dc:5a
fdt_addr_r=0x12100000
fdt_file=imx6dl-colibri-eval-v3.dtb
fdt_fixup=;
fdtcontroladdr=2df69610
fdtfile=imx6dl-colibri-eval-v3.dtb
ipaddr=192.168.10.2
kernel_addr_r=0x11000000
load_efi_dtb=load ${devtype} ${devnum}:${distro_bootpart} ${fdt_addr_r} ${prefix}${efi_fdtfile}
loadaddr=0x12000000
mmc_boot=if mmc dev ${devnum}; then devtype=mmc; run scan_dev_for_boot_part; fi
netmask=255.255.255.0
pxefile_addr_r=0x17100000
ramdisk_addr_r=0x12200000
rootpath=/srv/nfs
scan_dev_for_boot=echo Scanning ${devtype} ${devnum}:${distro_bootpart}...; for prefix in ${boot_prefixes}; do run scan_dev_for_extlinux; run scan_dev_for_scripts; done;run scan_dev_for_efi;
scan_dev_for_boot_part=part list ${devtype} ${devnum} -bootable devplist; env exists devplist || setenv devplist 1; for distro_bootpart in ${devplist}; do if fstype ${devtype} ${devnum}:${distro_bootpart} bootfstype; then run scan_dev_for_boot; fi; done; setenv devplist
scan_dev_for_efi=setenv efi_fdtfile ${fdtfile}; if test -z "${fdtfile}" -a -n "${soc}"; then setenv efi_fdtfile ${soc}-${board}${boardver}.dtb; fi; for prefix in ${efi_dtb_prefixes}; do if test -e ${devtype} ${devnum}:${distro_bootpart} ${prefix}${efi_fdtfile}; then run load_efi_dtb; fi;done;if test -e ${devtype} ${devnum}:${distro_bootpart} efi/boot/bootarm.efi; then echo Found EFI removable media binary efi/boot/bootarm.efi; run boot_efi_binary; echo EFI LOAD FAILED: continuing...; fi; setenv efi_fdtfile
scan_dev_for_extlinux=if test -e ${devtype} ${devnum}:${distro_bootpart} ${prefix}${boot_syslinux_conf}; then echo Found ${prefix}${boot_syslinux_conf}; run boot_extlinux; echo SCRIPT FAILED: continuing...; fi
scan_dev_for_scripts=for script in ${boot_scripts}; do if test -e ${devtype} ${devnum}:${distro_bootpart} ${prefix}${script}; then echo Found U-Boot script ${prefix}${script}; run boot_a_script; echo SCRIPT FAILED: continuing...; fi; done
scriptaddr=0x17000000
serial#=11066458
serverip=192.168.10.1
set_blkcnt=setexpr blkcnt ${filesize} + 0x1ff && setexpr blkcnt ${blkcnt} / 0x200
setethupdate=if env exists ethaddr; then; else setenv ethaddr 00:14:2d:00:00:00; fi; tftpboot ${loadaddr} flash_eth.img && source ${loadaddr}
setsdupdate=setenv interface mmc; setenv drive 1; mmc rescan; load ${interface} ${drive}:1 ${loadaddr} flash_blk.img && source ${loadaddr}
setup=setenv setupargs fec_mac=${ethaddr} consoleblank=0 no_console_suspend=1 console=tty1 console=${console},${baudrate}n8
setupdate=run setsdupdate || run setusbupdate || run setethupdate
setusbupdate=usb start && setenv interface usb; setenv drive 0; load ${interface} ${drive}:1 ${loadaddr} flash_blk.img && source ${loadaddr}
soc=mx6
splashimage=0x12000000
splashpos=m,m
uboot_blk=8a
uboot_hwpart=1
uboot_spl_blk=2
update_spl=run set_blkcnt && mmc dev 0 ${uboot_hwpart} && mmc write ${loadaddr} ${uboot_spl_blk} ${blkcnt}
update_uboot=run set_blkcnt && mmc dev 0 ${uboot_hwpart} && mmc write ${loadaddr} ${uboot_blk} ${blkcnt}
usb_boot=usb start; if usb dev ${devnum}; then devtype=usb; run scan_dev_for_boot_part; fi
vendor=toradex
ver=U-Boot 2020.07-5.3.0+git.68f97c8d17f0 (Jul 06 2021 - 15:50:15 +0000)
vidargs=fbmem=8M

Environment size: 5195/8188 bytes
Colibri iMX6 TEZI #

Our EasyInstaller script image.json:

{
  "config_format": 1,
  "autoinstall": true,
  "name": "bootloader and image",
  "description": "No file encryption",
  "icon": "SH_ICON.png",
  "version": "2.1.21097.0",
  "release_date": "2021-04-07",
  "wrapup_script": "wrapup.sh",
  "supported_product_ids": [
    "0015",
    "0017"
  ],
  "blockdevs": [
    {
      "erase": true,
      "name": "mmcblk0",
      "content": {
        "filesystem_type": "raw",
        "rawfiles": [
          {
            "filename": "Backup.i6r",
            "dd_options": "seek=2",
            "size": 8
          },
          {
            "filename": "cleanreg.bin",
            "dd_options": "seek=16386"
          },
          {
            "filename": "EasyInstallerSplash.tss",
            "dd_options": "seek=32770",
            "size": 1
          },
          {
            "filename": "nk.nbx",
            "dd_options": "seek=40962",
            "size": 28
          }
        ]
      },
      "partitions": [
        {
          "partition_size_nominal": 256,
          "want_maximised": true,
          "partition_type": "07",
          "offset_in_sectors": 196608
        }
      ]
    },
    {
      "erase": true,
      "name": "mmcblk0boot0",
      "content": {
        "filesystem_type": "raw",
        "rawfiles": [
          {
            "filename": "SPL-colibri",
            "dd_options": "seek=2"
          },
          {
            "filename": "eboot.spl",
            "dd_options": "seek=138"
          }
        ]
      }
    }
  ]
}

wrapup.sh:

#!/bin/sh
#
# (c) Toradex AG 2016
#
# Apalis/Colibri iMX6 in-field hardware update script
#
# One-time configurations (non-reversible!):
# - Fuse SoC to use eMMC Fast Boot mode
# - Enable eMMC H/W reset capabilities
# Required configurations
# - Configure Boot Bus mode (due to eMMC Fast Boot mode above)
#
# Other configurations
# - Boot from eMMC boot partition (must run as wrapup script)
#

PRODUCT_ID=$1
BOARD_REV=$2
SERIAL=$3
IMAGE_FOLDER=$4

error_exit () {
	echo "$1" 1>&2
	exit 1
}

# Do a basic validation that we do this on one of our modules
case $PRODUCT_ID in
0027|0028|0029|0035) ;;
0014|0015|0016|0017) ;;
*) error_exit "This script is meant to be run on a Apalis/Colibri iMX6. Aborting...";
esac

# Fuse SoC's BOOT_CFG to enable eMMC Fast Boot mode, if necsary
# WARNING: Fusing is a one-time operation, do not change values
# here unless you are absolutely sure what your are doing.
BOOT_CFG=0x5072
if [ ! -f /sys/fsl_otp/HW_OCOTP_CFG4 ]; then
	echo "Fusing not supported."
elif grep -q ${BOOT_CFG} /sys/fsl_otp/HW_OCOTP_CFG4; then
	echo "No new value for BOOT_CFG required."
else
	echo ${BOOT_CFG} > /sys/fsl_otp/HW_OCOTP_CFG4
	if [ "$?" != "0" ]; then
		error_exit "Writing fuse BOOT_CFG failed! Aborting..."
	fi
	echo "Fuse BOOT_CFG updated to ${BOOT_CFG}."
fi

# eMMC configurations
MMCDEV=/dev/mmcblk0

# Enable eMMC H/W Reset feature. This need to be executed before the other
# eMMC settings, it seems that this command resets all settings.
# Since this is a one-time operation, it will fail the second time. Ignore
# errors and redirect stderr to stdout.
mmc hwreset enable ${MMCDEV} 2>&1
if [ "$?" == "0" ]; then
	echo "H/W Reset permanently enabled on ${MMCDEV}"
fi

# Set boot bus mode
if ! mmc bootbus set single_hs x1 x8 ${MMCDEV}; then
	error_exit "Setting boot bus mode failed"
fi

# Enable eMMC boot partition 1 (mmcblkXboot0) and boot ack
# Make sure everything hit the eMMC when execute this command. Otherwise
# the eMMC will reset the configuration.
sync
if ! mmc bootpart enable 1 1 ${MMCDEV}; then
	error_exit "Setting bootpart failed"
fi

mmc extcsd read ${MMCDEV} | grep -e BOOT_BUS_CONDITIONS -e PARTITION_CONFIG -e RST_N_FUNCTION

echo "Apalis/Colibri iMX6 in-field hardware update script ended successfully."

#CTC special: write config block and reboot to install container from autorun
IMAGE_FOLDER=$4

OFFSET=$(($(($(cat /sys/block/mmcblk0boot0/size)-127))*32))
echo "Writing config block to boot partition" 
dd if=${IMAGE_FOLDER}/Backup.cfg of=/dev/mmcblk0boot0 bs=16 skip=1 count=4032 seek=$OFFSET

sleep 10
reboot -f

exit 0

Could you please provide more details about your case

  • Does it happened to “fresh” modules (with factory flashed Toradex Easy Installer) or modules programmed with something else?
    -what script you are talking about? The wrapup.sh got invoked only when OS flashing is done so it can not affect flashing itself. The image.json is not a script but rather a configuration file.

Could you also provide a detailed repro steps so we can try to investigate this issue locally?

I’m afraid you cannot reproduce this locally because

  • we are using our own custom board
  • 99% of the setups are successfull

It happens to fresh modules with factory settings (EasyInstaller). I did misuse the term ‘script’, you are correct image.json is a config file. We do not use any scripts except wrapup.sh.

Can you please give some hints what is needed for the Colibri to go into this state where BOOTP broadcast is displayed?

Does that 1% of “failed” modules behave the same way after reboot? If so - are they behave the same way if inserted in a Colibri Evaluation board?

Yes, in fact we do not see any output of EasyInstaller while installing on our custom board/custom terminal. After a failed installation, when our application does no start up, I move the Colibri to Colibri Evaluation Board to have any bootloader output to find out what happened.

Hi @zorro ,

The modules tries multiple ways to boot. If an eMMC boot is not successful

it will try USB next:

If this failed as well then it will try to boot over the network (TFTP). If there’s nothing to boot from, the module will stay for a while in this loop to try to get a connection going.

One thing I noticed when looking at your log. Is that the USB phase discovered 4 devices? Are you possible using a USB hub between the USB stick and the module? If so, could you try without the hub?

Best Regards
Kevin

The 4 USB devices are present because of catching the bootloader output I have to put the Colibri into the Toradex Evaluation Board. In our custom board there is no other USB device than the installation stick.
Maybe the installation stick is of bad quality (timing).

Hi @zorro ,

I see. Is the behaviour the same if you use a different USB stick?

Would it maybe be possible that you share a snipped of the schematic, showing how you integrated the USB port?

I can make the topic private if you prefer to not make this public.

Best Regards
Kevin

Hello @zorro ,

Have you made any progress on this topic? Do you need additional help on our side?

Best regards,
Josep