Not able to use Easy Installer in custom carrier board

Hi Toradex-Team,

We’ve developed a custom carrier word for the Verdin AM62. Now I’m evaluating if the production process will work. We’ve reused a lot of the Verdin Development Board schematic on our custom development board. For example the reset and recovery buttons and the USB port to flash the Easy Installer, as well as the ethernet port.

The flashing of the easy installer (Verdin-AM62_ToradexEasyInstaller_6.6.0+build.6) in the custom carrier board works. But I can’t get a remote session with the TightVNC Viewer to work. If I used the ip address of the ethernet port I only get a gray screen but to connection seems to be established. If I use the the USB NCM at IP 192.168.11.1 the connection is refused. But I can Ping both addresses.

If I plug in an USB drive with a autoinstall Torizon-Image and a manipulated wrapup.sh (like described here) nothing happens.

But if I use the Verdin Development Board everything described above works as expected.

Here is the hardware information:

Hardware info
------------------------------------------------------------
HW model:                 Toradex Verdin AM62 on Verdin Development Board
Toradex version:          0075 V1.1A
Serial number:            15207085
Processor arch:           aarch64
------------------------------------------------------------
Here is the console output from the Easy Installer flashing on the custom carrier board
U-Boot SPL 2023.04-ti-gca551722e0b5 (Mar 22 2024 - 08:10:43 +0000)
SYSFW ABI: 3.1 (firmware rev 0x0009 '9.2.7--v09.02.07 (Kool Koala)')
SPL initial stack usage: 13408 bytes
Trying to boot from DFU
###################################################DOWNLOAD ... OK
Ctrl+C to exit ...
Authentication passed
Authentication passed
Authentication passed
Loading Environment from nowhere... OK
init_env from device 10 not supported!
Authentication passed
Authentication passed
Starting ATF on ARM64 core...

NOTICE:  BL31: v2.10.0(release):v2.10.0-367-g00f1ec6b87-dirty
NOTICE:  BL31: Built : 16:09:05, Feb  9 2024

U-Boot SPL 2023.04-ti-gca551722e0b5 (Mar 22 2024 - 08:10:43 +0000)
SYSFW ABI: 3.1 (firmware rev 0x0009 '9.2.7--v09.02.07 (Kool Koala)')
SPL initial stack usage: 1856 bytes
Trying to boot from DFU
#################################################DOWNLOAD ... OK
Ctrl+C to exit ...
Authentication passed
Authentication passed


U-Boot 2023.04-ti-gca551722e0b5 (Mar 22 2024 - 08:10:43 +0000)

SoC:   AM62X SR1.0 HS-FS
DRAM:  1 GiB
Core:  143 devices, 31 uclasses, devicetree: separate
MMC:   mmc@fa10000: 0, mmc@fa00000: 1
Loading Environment from nowhere... OK
In:    serial@2800000
Out:   serial@2800000
Err:   serial@2800000
Model: Toradex 0075 Verdin AM62 Dual 1GB WB IT V1.1A
Serial#: 15207085
i2c_write: error waiting for data ACK (status=0x116)
get_tdx_eeprom: cannot find EEPROM by node
MISSING TORADEX CARRIER CONFIG BLOCKS
i2c_write: error waiting for data ACK (status=0x116)
get_tdx_eeprom: cannot find EEPROM by node
am65_cpsw_nuss ethernet@8000000: K3 CPSW: nuss_ver: 0x6BA01103 cpsw_ver: 0x6BA81103 ale_ver: 0x00290105 Ports:2 mdio_freq:1000000
Setting variant to wifi
Net:
Warning: ethernet@8000000port@1 MAC addresses don't match:
Address in ROM is               28:b5:e8:cb:56:e2
Address in environment is       00:14:2d:e8:0a:ad
eth0: ethernet@8000000port@1 [PRIME]Could not get PHY for ethernet@8000000port@1: addr 7
am65_cpsw_nuss_port ethernet@8000000port@2: phy_connect() failed

Hit any key to stop autoboot:  0
generic_phy_get_bulk : no phys property
Starting download of 50334046 bytes
dwc3-generic-peripheral usb@31000000: request 0000000099f1f300 was not queued to ep1in-bulk
..........................................................................
..........................................................................
..........................................................................
..........................................................................
..........................................................................
..............
downloading of 50334046 bytes finished
dwc3-generic-peripheral usb@31000000: request 0000000099f1f300 was not queued to ep1in-bulk
dwc3-generic-peripheral usb@31000000: request 0000000099f1f300 was not queued to ep1in-bulk
Starting download of 82 bytes
dwc3-generic-peripheral usb@31000000: request 0000000099f1f300 was not queued to ep1in-bulk

downloading of 82 bytes finished
dwc3-generic-peripheral usb@31000000: request 0000000099f1f300 was not queued to ep1in-bulk
dwc3-generic-peripheral usb@31000000: request 0000000099f1f300 was not queued to ep1in-bulk
Starting download of 1675 bytes
dwc3-generic-peripheral usb@31000000: request 0000000099f1f300 was not queued to ep1in-bulk

downloading of 1675 bytes finished
dwc3-generic-peripheral usb@31000000: request 0000000099f1f300 was not queued to ep1in-bulk
dwc3-generic-peripheral usb@31000000: request 0000000099f1f300 was not queued to ep1in-bulk
dwc3-generic-peripheral usb@31000000: request 0000000099f1f300 was not queued to ep1in-bulk
## Executing script at 90280000
Bootargs: quiet video=HDMI-A-1:640x480-16@60D rootfstype=squashfs root=/dev/ram autoinstall
## Loading kernel from FIT Image at 90300000 ...
   Using 'conf-ti_k3-am625-verdin-wifi-dev.dtb' configuration
   Verifying Hash Integrity ... OK
   Trying 'kernel-1' kernel subimage
     Description:  Linux kernel
     Type:         Kernel Image
     Compression:  gzip compressed
     Data Start:   0x90300104
     Data Size:    7680490 Bytes = 7.3 MiB
     Architecture: AArch64
     OS:           Linux
     Load Address: 0x80200000
     Entry Point:  0x80200000
     Hash algo:    sha512
     Hash value:   117b55aee6f045bb92089e5365514606d731813497fc7b9091be95cd7908a17c7638f995d5afae91588460427b0ec62f4882103112ff72006791cbb1930d965e
   Verifying Hash Integrity ... sha512+ OK
## Loading ramdisk from FIT Image at 90300000 ...
   Using 'conf-ti_k3-am625-verdin-wifi-dev.dtb' configuration
   Verifying Hash Integrity ... OK
   Trying 'ramdisk-1' ramdisk subimage
     Description:  tezi-initramfs
     Type:         RAMDisk Image
     Compression:  uncompressed
     Data Start:   0x90ae24d8
     Data Size:    42045440 Bytes = 40.1 MiB
     Architecture: AArch64
     OS:           Linux
     Load Address: 0x84000000
     Entry Point:  0x84000000
     Hash algo:    sha512
     Hash value:   da8a5a75cbeb7b43a167acfcf57a84c7287459f484041574e9e0757a314697c8d86eb917a15ff38244b90d09ce360f17ed5d1e012472cfd99756f48e40ad63e2
   Verifying Hash Integrity ... sha512+ OK
   Loading ramdisk from 0x90ae24d8 to 0x84000000
## Loading fdt from FIT Image at 90300000 ...
   Using 'conf-ti_k3-am625-verdin-wifi-dev.dtb' configuration
   Verifying Hash Integrity ... OK
   Trying 'fdt-ti_k3-am625-verdin-wifi-dev.dtb' fdt subimage
     Description:  Flattened Device Tree blob
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x90aa829c
     Data Size:    70049 Bytes = 68.4 KiB
     Architecture: AArch64
     Load Address: 0x83000000
     Hash algo:    sha512
     Hash value:   5aa70705daac5fe5e4e9b32db9d87fcf8107cce1306694eed0b3bb214e97cb25a4966f1ba927104315dcc26494ad58298f28fbe6432cc385e9030d7158f03666
   Verifying Hash Integrity ... sha512+ OK
   Loading fdt from 0x90aa829c to 0x83000000
## Loading fdt from FIT Image at 90300000 ...
   Using 'conf-verdin-am62_dsi-to-hdmi_overlay.dtbo' configuration
   Verifying Hash Integrity ... OK
   Trying 'fdt-verdin-am62_dsi-to-hdmi_overlay.dtbo' fdt subimage
     Description:  Flattened Device Tree blob
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x90adb41c
     Data Size:    3050 Bytes = 3 KiB
     Architecture: AArch64
     Load Address: 0x83080000
     Hash algo:    sha512
     Hash value:   5ced6b5ab11cabe4aeae808365b7164ecc5a43b15e8a8fda545db693e133ca2bfd036bce2c1d6b45456e88eadf9c6ad6521eebf9c8815f72d756c3ece4a1676f
   Verifying Hash Integrity ... sha512+ OK
## Loading fdt from FIT Image at 90300000 ...
   Using 'conf-verdin-am62_spidev_overlay.dtbo' configuration
   Verifying Hash Integrity ... OK
   Trying 'fdt-verdin-am62_spidev_overlay.dtbo' fdt subimage
     Description:  Flattened Device Tree blob
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x90ae21b0
     Data Size:    560 Bytes = 560 Bytes
     Architecture: AArch64
     Load Address: 0x83080000
     Hash algo:    sha512
     Hash value:   4bd8efa1ddf06d7e8fa7d3aea24a556c7745e945a554d32215f71140f637f01ed033f5bcc06b23f5ec23e046b2fe6f822927fc121ed58f76e8625d8d011a5a58
   Verifying Hash Integrity ... sha512+ OK
   Booting using the fdt blob at 0x83000000
Working FDT set to 83000000
   Uncompressing Kernel Image
   Loading Ramdisk to 966ca000, end 98ee3000 ... OK
ERROR: reserving fdt memory region failed (addr=9e780000 size=80000 flags=4)
ERROR: reserving fdt memory region failed (addr=9e800000 size=1800000 flags=4)
ERROR: reserving fdt memory region failed (addr=9db00000 size=c00000 flags=4)
   Loading Device Tree to 00000000966b5000, end 00000000966c95db ... OK
Working FDT set to 966b5000

Starting kernel ...

[    0.335618] ti-sci-clk 44043000.system-controller:clock-controller: recalc-rate failed for dev=81, clk=20, ret=-19
[    0.720282] mdio_bus 8000f00.mdio: MDIO device at address 7 is missing.
[    0.732827] debugfs: Directory 'pd:182' with parent 'pm_genpd' already present!
[    0.748785] ti-sci-clk 44043000.system-controller:clock-controller: is_prepared failed for dev=81, clk=20, ret=-19
Running /etc/rc.local...
Toradex Easy Installer 6.6.0+build.6 ()
Starting udev
[    2.619622] ina2xx 1-0040: error configuring the device: -121
[    2.747633] k3-m4-rproc 5000000.m4fss: device does not reserved memory regions, ret = -22
[    2.756933] k3-m4-rproc 5000000.m4fss: reserved memory init failed, ret = -22
[    2.871787] platform 78000000.r5f: device does not have reserved memory regions, ret = -22
[    2.900160] k3_r5_rproc bus@f0000:bus@b00000:r5fss@78000000: reserved memory init failed, ret = -22
[    2.917741] k3_r5_rproc bus@f0000:bus@b00000:r5fss@78000000: k3_r5_cluster_rproc_init failed, ret = -22
[    2.971826] omap_i2c 20020000.i2c: Arbitration lost
System time was Fri Mar  9 12:44:01 UTC 2018.
Setting the System Clock using the Hardware Clock as reference...
System Clock set. System local time is now Fri Mar  9 12:44:01 UTC 2018.
[    3.998431] omap_i2c 20020000.i2c: controller timed out
[    4.027785] omap_i2c 20020000.i2c: Arbitration lost
[    5.054554] omap_i2c 20020000.i2c: controller timed out
[    5.083900] omap_i2c 20020000.i2c: Arbitration lost
[    6.110409] omap_i2c 20020000.i2c: controller timed out
[    6.139785] omap_i2c 20020000.i2c: Arbitration lost
[    7.166428] omap_i2c 20020000.i2c: controller timed out
[    7.195877] omap_i2c 20020000.i2c: Arbitration lost
[    8.222505] omap_i2c 20020000.i2c: controller timed out
[    8.331809] omap_i2c 20020000.i2c: Arbitration lost

Welcome to the Toradex Easy Installer

This is a Linux based installer for Toradex modules. Currently, the installer
does not have a serial console interface. You can use the Toradex Easy Installer
via any of the available display interfaces using USB mouse/keyboard or via a
network connection using VNC. Use:
  # ip addr show
to display the Ethernet IP address(es) or use USB NCM at IP 192.168.11.1.

Check our documentation at:
  https://developer.toradex.com/software/toradex-easy-installer
/ # [    9.342483] omap_i2c 20020000.i2c: controller timed out
[    9.371890] omap_i2c 20020000.i2c: Arbitration lost
[   10.398655] omap_i2c 20020000.i2c: controller timed out
[   10.427825] omap_i2c 20020000.i2c: Arbitration lost
Here is the console output from the Easy Installer flashing on the Verdin Development Board
U-Boot SPL 2023.04-ti-gca551722e0b5 (Mar 22 2024 - 08:10:43 +0000)
SYSFW ABI: 3.1 (firmware rev 0x0009 '9.2.7--v09.02.07 (Kool Koala)')
SPL initial stack usage: 13408 bytes
Trying to boot from DFU
###################################################DOWNLOAD ... OK
Ctrl+C to exit ...
Authentication passed
Authentication passed
Authentication passed
Loading Environment from nowhere... OK
init_env from device 10 not supported!
Authentication passed
Authentication passed
Starting ATF on ARM64 core...

NOTICE:  BL31: v2.10.0(release):v2.10.0-367-g00f1ec6b87-dirty
NOTICE:  BL31: Built : 16:09:05, Feb  9 2024

U-Boot SPL 2023.04-ti-gca551722e0b5 (Mar 22 2024 - 08:10:43 +0000)
SYSFW ABI: 3.1 (firmware rev 0x0009 '9.2.7--v09.02.07 (Kool Koala)')
SPL initial stack usage: 1856 bytes
Trying to boot from DFU
#################################################DOWNLOAD ... OK
Ctrl+C to exit ...
Authentication passed
Authentication passed


U-Boot 2023.04-ti-gca551722e0b5 (Mar 22 2024 - 08:10:43 +0000)

SoC:   AM62X SR1.0 HS-FS
DRAM:  1 GiB
Core:  143 devices, 31 uclasses, devicetree: separate
MMC:   mmc@fa10000: 0, mmc@fa00000: 1
Loading Environment from nowhere... OK
In:    serial@2800000
Out:   serial@2800000
Err:   serial@2800000
Model: Toradex 0075 Verdin AM62 Dual 1GB WB IT V1.1A
Serial#: 15207085
Carrier: Toradex Verdin Development Board V1.1F, Serial# 11305462
am65_cpsw_nuss ethernet@8000000: K3 CPSW: nuss_ver: 0x6BA01103 cpsw_ver: 0x6BA81103 ale_ver: 0x00290105 Ports:2 mdio_freq:1000000
Setting variant to wifi
Net:
Warning: ethernet@8000000port@1 MAC addresses don't match:
Address in ROM is               28:b5:e8:cb:56:e2
Address in environment is       00:14:2d:e8:0a:ad
eth0: ethernet@8000000port@1 [PRIME], eth1: ethernet@8000000port@2
Hit any key to stop autoboot:  0
generic_phy_get_bulk : no phys property
Starting download of 50334046 bytes
dwc3-generic-peripheral usb@31000000: request 0000000099f1f440 was not queued to ep1in-bulk
..........................................................................
..........................................................................
..........................................................................
..........................................................................
..........................................................................
..............
downloading of 50334046 bytes finished
dwc3-generic-peripheral usb@31000000: request 0000000099f1f440 was not queued to ep1in-bulk
dwc3-generic-peripheral usb@31000000: request 0000000099f1f440 was not queued to ep1in-bulk
Starting download of 82 bytes
dwc3-generic-peripheral usb@31000000: request 0000000099f1f440 was not queued to ep1in-bulk

downloading of 82 bytes finished
dwc3-generic-peripheral usb@31000000: request 0000000099f1f440 was not queued to ep1in-bulk
dwc3-generic-peripheral usb@31000000: request 0000000099f1f440 was not queued to ep1in-bulk
Starting download of 1675 bytes
dwc3-generic-peripheral usb@31000000: request 0000000099f1f440 was not queued to ep1in-bulk

downloading of 1675 bytes finished
dwc3-generic-peripheral usb@31000000: request 0000000099f1f440 was not queued to ep1in-bulk
dwc3-generic-peripheral usb@31000000: request 0000000099f1f440 was not queued to ep1in-bulk
dwc3-generic-peripheral usb@31000000: request 0000000099f1f440 was not queued to ep1in-bulk
## Executing script at 90280000
Bootargs: quiet video=HDMI-A-1:640x480-16@60D rootfstype=squashfs root=/dev/ram autoinstall
## Loading kernel from FIT Image at 90300000 ...
   Using 'conf-ti_k3-am625-verdin-wifi-dev.dtb' configuration
   Verifying Hash Integrity ... OK
   Trying 'kernel-1' kernel subimage
     Description:  Linux kernel
     Type:         Kernel Image
     Compression:  gzip compressed
     Data Start:   0x90300104
     Data Size:    7680490 Bytes = 7.3 MiB
     Architecture: AArch64
     OS:           Linux
     Load Address: 0x80200000
     Entry Point:  0x80200000
     Hash algo:    sha512
     Hash value:   117b55aee6f045bb92089e5365514606d731813497fc7b9091be95cd7908a17c7638f995d5afae91588460427b0ec62f4882103112ff72006791cbb1930d965e
   Verifying Hash Integrity ... sha512+ OK
## Loading ramdisk from FIT Image at 90300000 ...
   Using 'conf-ti_k3-am625-verdin-wifi-dev.dtb' configuration
   Verifying Hash Integrity ... OK
   Trying 'ramdisk-1' ramdisk subimage
     Description:  tezi-initramfs
     Type:         RAMDisk Image
     Compression:  uncompressed
     Data Start:   0x90ae24d8
     Data Size:    42045440 Bytes = 40.1 MiB
     Architecture: AArch64
     OS:           Linux
     Load Address: 0x84000000
     Entry Point:  0x84000000
     Hash algo:    sha512
     Hash value:   da8a5a75cbeb7b43a167acfcf57a84c7287459f484041574e9e0757a314697c8d86eb917a15ff38244b90d09ce360f17ed5d1e012472cfd99756f48e40ad63e2
   Verifying Hash Integrity ... sha512+ OK
   Loading ramdisk from 0x90ae24d8 to 0x84000000
## Loading fdt from FIT Image at 90300000 ...
   Using 'conf-ti_k3-am625-verdin-wifi-dev.dtb' configuration
   Verifying Hash Integrity ... OK
   Trying 'fdt-ti_k3-am625-verdin-wifi-dev.dtb' fdt subimage
     Description:  Flattened Device Tree blob
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x90aa829c
     Data Size:    70049 Bytes = 68.4 KiB
     Architecture: AArch64
     Load Address: 0x83000000
     Hash algo:    sha512
     Hash value:   5aa70705daac5fe5e4e9b32db9d87fcf8107cce1306694eed0b3bb214e97cb25a4966f1ba927104315dcc26494ad58298f28fbe6432cc385e9030d7158f03666
   Verifying Hash Integrity ... sha512+ OK
   Loading fdt from 0x90aa829c to 0x83000000
## Loading fdt from FIT Image at 90300000 ...
   Using 'conf-verdin-am62_dsi-to-hdmi_overlay.dtbo' configuration
   Verifying Hash Integrity ... OK
   Trying 'fdt-verdin-am62_dsi-to-hdmi_overlay.dtbo' fdt subimage
     Description:  Flattened Device Tree blob
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x90adb41c
     Data Size:    3050 Bytes = 3 KiB
     Architecture: AArch64
     Load Address: 0x83080000
     Hash algo:    sha512
     Hash value:   5ced6b5ab11cabe4aeae808365b7164ecc5a43b15e8a8fda545db693e133ca2bfd036bce2c1d6b45456e88eadf9c6ad6521eebf9c8815f72d756c3ece4a1676f
   Verifying Hash Integrity ... sha512+ OK
## Loading fdt from FIT Image at 90300000 ...
   Using 'conf-verdin-am62_spidev_overlay.dtbo' configuration
   Verifying Hash Integrity ... OK
   Trying 'fdt-verdin-am62_spidev_overlay.dtbo' fdt subimage
     Description:  Flattened Device Tree blob
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x90ae21b0
     Data Size:    560 Bytes = 560 Bytes
     Architecture: AArch64
     Load Address: 0x83080000
     Hash algo:    sha512
     Hash value:   4bd8efa1ddf06d7e8fa7d3aea24a556c7745e945a554d32215f71140f637f01ed033f5bcc06b23f5ec23e046b2fe6f822927fc121ed58f76e8625d8d011a5a58
   Verifying Hash Integrity ... sha512+ OK
   Booting using the fdt blob at 0x83000000
Working FDT set to 83000000
   Uncompressing Kernel Image
   Loading Ramdisk to 966ca000, end 98ee3000 ... OK
ERROR: reserving fdt memory region failed (addr=9e780000 size=80000 flags=4)
ERROR: reserving fdt memory region failed (addr=9e800000 size=1800000 flags=4)
ERROR: reserving fdt memory region failed (addr=9db00000 size=c00000 flags=4)
   Loading Device Tree to 00000000966b5000, end 00000000966c95db ... OK
Working FDT set to 966b5000

Starting kernel ...

[    0.330424] ti-sci-clk 44043000.system-controller:clock-controller: recalc-rate failed for dev=81, clk=20, ret=-19
[    0.396300] rtc-ds1307 0-0032: hctosys: unable to read the hardware clock
[    0.702572] debugfs: Directory 'pd:182' with parent 'pm_genpd' already present!
[    0.719057] ti-sci-clk 44043000.system-controller:clock-controller: is_prepared failed for dev=81, clk=20, ret=-19
Running /etc/rc.local...
Toradex Easy Installer 6.6.0+build.6 ()
Starting udev
[    2.739770] k3-m4-rproc 5000000.m4fss: device does not reserved memory regions, ret = -22
[    2.764747] k3-m4-rproc 5000000.m4fss: reserved memory init failed, ret = -22
[    2.832172] platform 78000000.r5f: device does not have reserved memory regions, ret = -22
[    2.846827] k3_r5_rproc bus@f0000:bus@b00000:r5fss@78000000: reserved memory init failed, ret = -22
[    2.861355] k3_r5_rproc bus@f0000:bus@b00000:r5fss@78000000: k3_r5_cluster_rproc_init failed, ret = -22
System time was Thu Jan  1 00:00:03 UTC 1970.
Setting the System Clock using the Hardware Clock as reference...
hwclock: RTC_RD_TIME: Invalid argument
System Clock set. System local time is now Thu Jan  1 00:00:03 UTC 1970.
Fri Mar  9 12:34:56 UTC 2018
Saving the System Clock time to the Hardware Clock...
Hardware Clock updated to Fri Mar  9 12:34:56 UTC 2018.

Welcome to the Toradex Easy Installer

This is a Linux based installer for Toradex modules. Currently, the installer
does not have a serial console interface. You can use the Toradex Easy Installer
via any of the available display interfaces using USB mouse/keyboard or via a
network connection using VNC. Use:
  # ip addr show
to display the Ethernet IP address(es) or use USB NCM at IP 192.168.11.1.

Check our documentation at:
  https://developer.toradex.com/software/toradex-easy-installer

There are some differences/errors on the custom carrier board log, but i don’t know if the are related to my problem.

Like:

i2c_write: error waiting for data ACK (status=0x116)
get_tdx_eeprom: cannot find EEPROM by node
MISSING TORADEX CARRIER CONFIG BLOCKS
i2c_write: error waiting for data ACK (status=0x116)
get_tdx_eeprom: cannot find EEPROM by node

or

[    2.971826] omap_i2c 20020000.i2c: Arbitration lost
System time was Fri Mar  9 12:44:01 UTC 2018.
Setting the System Clock using the Hardware Clock as reference...
System Clock set. System local time is now Fri Mar  9 12:44:01 UTC 2018.
[    3.998431] omap_i2c 20020000.i2c: controller timed out
[    4.027785] omap_i2c 20020000.i2c: Arbitration lost
[    5.054554] omap_i2c 20020000.i2c: controller timed out
[    5.083900] omap_i2c 20020000.i2c: Arbitration lost
[    6.110409] omap_i2c 20020000.i2c: controller timed out
[    6.139785] omap_i2c 20020000.i2c: Arbitration lost
[    7.166428] omap_i2c 20020000.i2c: controller timed out
[    7.195877] omap_i2c 20020000.i2c: Arbitration lost
[    8.222505] omap_i2c 20020000.i2c: controller timed out
[    8.331809] omap_i2c 20020000.i2c: Arbitration lost

Do you have any idea what is the problem and how to fix it? Do I have to configure the Easy Installer somehow?

Kind Regards
Uwe

Greetings @buw3,

I’m checking internally to see if anyone else has seen this behavior before.

Just one question. On your custom carrier board is there a difference in terms of i2c? Much of the differences in the logs messages seem to be complaining about the i2c subsystem. I don’t think this would have an effect on Easy Installer. But it would be good to know if there is a difference in i2c on your custom carrier board versus our carrier board.

Also can you get the logs from the following:

$ journalctl -b -u weston

$ cat /var/volatile/tezi.log

Best Regards,
Jeremias

Hallo @jeremias.tx,

I checked back with the hardware developer. Our carrier board does not contain the following i2c stuff:

  • i2c stuff for power suppy (IMHO no related to the problem)
  • i2c audio chip (IMHO no related to the problem)
  • MIPI DSI (IMHO no related to the problem - because if the Verdin DSI to HDMI Adapter is not connected to the Verdin Eval Board everything sill works)
  • i2c stuff for PCIE (IMHO no related to the problem)
  • EEPROM IC29 (addr 0x57) - maybe this is somehow the problem?

As the logs say - the software is trying to connect to it:

get_tdx_eeprom: cannot find EEPROM by node
MISSING TORADEX CARRIER CONFIG BLOCKS
i2c_write: error waiting for data ACK (status=0x116)
get_tdx_eeprom: cannot find EEPROM by node

What does this EEPROM contains?

I also removed the jumper A1 and A2 on X3 to cut off the ic2 on the Verdin Eval Board. This cause the same error messages in the logs but the Easy Installer remote connection and GUI still works as expected.


Ethernet - maybe another straw
We are also just have one Ethernet-Jack (ETH_1)! The ETH_2 is not present and the logs say:

eth0: ethernet@8000000port@1 [PRIME]Could not get PHY for ethernet@8000000port@1: addr 7
am65_cpsw_nuss_port ethernet@8000000port@2: phy_connect() failed

Does this maybe cause the problem?

But we also disabled the ETH_2 on the Verdin Eval Board by pulling the RESET_N on IC22 to GND and still everything worked as expected.


Weston - an additional Information
After flashing the Easy Installer the ps command only shows this entry regarding to weston on our custom carrier board:

 229 root     32888 S    /usr/bin/weston --backend=vnc-backend.so --modules=s

in comparison to the Verdin Eval board:

 232 root     26004 S    /usr/bin/weston --tty=2 --current-mode
 235 root     31540 S    /usr/bin/weston --backend=vnc-backend.so --shell=ful

So the GUI application is not working and thats why there is just a gray screen?


TEZI log

The command you mentioned

returned the following on both carrier boards:

# cat /var/volatile/tezi.log
qt.qpa.wayland: Failed to load client buffer integration: "wayland-egl"
qt.qpa.wayland: Available client buffer integrations: ()
USB Gadget: NCM enabled
Trying to read configblock from "mmcblk0boot0" at -512
Running Command:  "/usr/bin/avahi-set-host-name" ("verdin-am62-15207085")
Finished with exit code: 0
avahi-set-host-name:
""
USB Gadget: NCM enabled
"AVAHI_CLIENT_S_REGISTERING"
Scanning media initially...
Network up in 0.096 seconds
NCM up in 0.166 seconds
Tezi Server lookup successfully
Downloading image list from  "http://tezi.toradex.com/image_list.json?PID8=00751100"
Downloading image list from  "http://tezi.toradex.com/image_list_3rdparty.json?PID8=00751100"
"AVAHI_CLIENT_S_RUNNING"
"AVAHI_BROWSER_CACHE_EXHAUSTED"
"AVAHI_BROWSER_ALL_FOR_NOW"

Thanks for your help.
Kind Regard
Uwe

What does this EEPROM contains?

On our modules that feature an EEPROM, it contains the toradex config block that contains some information about the hardware. Though this shouldn’t be needed for Easy Installer to at least run. As proven by the test you ran on our eval board.

Does this maybe cause the problem?

This shouldn’t have any effect either. Easy Installer can run without any network connections.

So the GUI application is not working and thats why there is just a gray screen?

This is my impression yes. The gray screen indicates that Weston is running, but since you don’t see anything else that means the Easy Installer GUI app is not running.

returned the following on both carrier boards:

You’re saying the log file here is identical on both carrier boards? That’s odd, I was hoping to see some kind of error or crash since it appears that Easy Installer isn’t running on your carrier board. Maybe then it is actually running? In the output for ps do you see any application running for Easy Installer? The application itself should be called “tezi” or something similar.

By the way you didn’t return the output of journalctl -b -u weston. Could you share this please.

Best Regards,
Jeremias

Hi @jeremias.tx,

thanks for the answers :slight_smile:

let me get that straight… I could swear the output on both carrier boards contained the same prints, just in a different order. So I figured they are “equal”. But I’ve redone the commands and here are both prints - they are NOT equal:

Custom carrier board tezi.log
/ # cat /var/volatile/tezi.log
Failed to create wl_display (No such file or directory)
qt.qpa.plugin: Could not load the Qt platform plugin "wayland" in "" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: minimal, offscreen, vnc, wayland.
Verdin eval board tezi.log
/ # cat /var/volatile/tezi.log
qt.qpa.wayland: Failed to load client buffer integration: "wayland-egl"
qt.qpa.wayland: Available client buffer integrations: ()
USB Gadget: NCM enabled
Trying to read configblock from "mmcblk0boot0" at -512
Running Command:  "/usr/bin/avahi-set-host-name" ("verdin-am62-15207085")
Finished with exit code: 0
avahi-set-host-name:
""
USB Gadget: NCM enabled
"AVAHI_CLIENT_S_REGISTERING"
Scanning media initially...
NCM up in 0.315 seconds
Network up in 1.219 seconds
Tezi Server lookup successfully
Downloading image list from  "http://tezi.toradex.com/image_list.json?PID8=00751100"
Downloading image list from  "http://tezi.toradex.com/image_list_3rdparty.json?PID8=00751100"
"AVAHI_CLIENT_S_RUNNING"
"AVAHI_BROWSER_CACHE_EXHAUSTED"
"AVAHI_BROWSER_ALL_FOR_NOW"

I didn’t know that the GUI application was called "tezi", but this was a good hint. Only on the Verdin eval board the process was running, but not on our custom carrier board.

/ # ps | grep tezi
  248 root      412m S    /usr/bin/tezi -autoinstall -platform wayland
  305 root      3412 R    grep tezi

So i tried to run it manually on your board… and what a surprise the Easy Installer GUI worked - I got an image on the remote desktop and was able to flash the SOM!

/ # /usr/bin/tezi -autoinstall -platform wayland
qt.qpa.wayland: Failed to load client buffer integration: "wayland-egl"
qt.qpa.wayland: Available client buffer integrations: ()
USB Gadget: NCM enabled
Trying to read configblock from "mmcblk0boot0" at -512
Running Command:  "/usr/bin/avahi-set-host-name" ("verdin-am62-15207085")
Finished with exit code: 0
avahi-set-host-name:
""
USB Gadget: NCM enabled
"AVAHI_CLIENT_S_REGISTERING"
Scanning media initially...
Network up in 0.096 seconds
NCM up in 0.164 seconds
Tezi Server lookup successfully
Downloading image list from  "http://tezi.toradex.com/image_list.json?PID8=00751100"
Downloading image list from  "http://tezi.toradex.com/image_list_3rdparty.json?PID8=00751100"
"AVAHI_CLIENT_S_RUNNING"
"AVAHI_BROWSER_CACHE_EXHAUSTED"
"AVAHI_BROWSER_ALL_FOR_NOW"

So why isn’t the tezi process able the start automatically in the first place on your hardware?

Yeah, I didn’t share that on purpose, because there is no output. After flashing the Easy Installer with the recovery script and running the command it returns the following on both carrier boards:

/ # journalctl -b -u weston
sh: journalctl: not found

And after starting into the Torizon linux the command returns:

torizon@verdin-am62-15207085:~$ journalctl -b -u weston
-- No entries --

Kind Regards
Uwe

Thank you for double-checking the logs here. With the latest information we can confirm that Easy Installer is not running on your custom carrier board for whatever reason. Though it’s interesting that starting the tezi application manually does work.

This suggests there’s some kind of "bad’ initial state when the system boots up where tezi can’t execute properly. Now how and why your custom carrier board causes this, is still unclear. But, at least now we have the logs showing Easy Installer is indeed not starting on boot.

Let me take these logs to our team and see what can be determined. At least for the time-being you have an short-term workaround for using Easy Installer.

Best Regards,
Jeremias

1 Like

Hi @jeremias.tx
I’m just curious. Are there any news on this topic? Did the Torizon-Team get an idea through the logs of what causes this problem?

Best Regards,
Uwe

Our team has a few ideas. So it’s obvious that Weston is starting in your system automatically as expected, but Easy Installer is not. Furthermore as per your tests, manually starting Easy Installer afterwards does work.

In Easy Installer the startup is automated by /etc/rc.local. The contents of this is something like:

nohup /usr/bin/weston --tty=2 --current-mode >> ${RW_MEMORY_DIR}/${WESTON_LOGFIL
sleep 0.5                                                                       
nohup /usr/bin/tezi $DEFAULT_KBD $DEFAULT_LANG $AUTOINSTALL $FULLSCREEN \       
        -platform wayland < /dev/null >> ${RW_MEMORY_DIR}/${TEZI_LOGFILE} &

Basically we start Weston then after a 0.5 second delay we start the Easy Installer application. Looking at the logs for Easy Installer you provided, it looks like the application is complaining that there is no Wayland/Weston client available.

Our theory is that perhaps on your custom carrier board the timings of things are slightly different. Maybe it’s taking longer for certain software stacks in the system to startup which causes Weston to take longer to start. If the timing is affected in this way, then maybe the 0.5 second delay isn’t enough of a delay. This would cause the Easy Installer application to be started before Weston is ready.

This would explain why you can start the Easy Installer application afterwards manually. Since by then Weston has had enough time to start and stabilize. It’s difficult to say for sure though what on your custom carrier board could cause such a change in timing.

Assuming this theory is correct you may be able to try some things. Perhaps you can modify the Easy Installer image to increase this delay in /etc/rc.local. Even though it’s a bit unorthodox, we do have a rough guide on modifying Easy Installer: How to Modify a Toradex Easy Installer Image | Toradex Developer Center

Best Regards,
Jeremias