Same SOM MCU only problem

For a Project we bought SOMs of the verdin-am62
We wanted to implement mcu only sleepmode and encountered problems with the MCU in the processor saying it received 149 over IPC while it should say 1.

Initially we thought it was a software version issue and a problem where the version of the TI MCU+ SDK differs from the TI Processor SDK, but this is not the problem as our versions match, and the MCU software works on 1 SOM.

After a lot of trial and error we figured we’d switch the SOM with another one and on that one the MCU only mode sleep worked.

On The nonworking SOM the MCU receives 149 over IPC and the processor eventually resets, likely due to the watchdog triggering.
On a working SOM the MCU receives 1 over IPC and the processor eventually resumes from suspend due to MCU wakeup trigger.

The MCU firmware is setup such that it triggers a wakeup after waiting on a timer for a certain time.

I used the same SD-card and mallow board for testing the SOMs, the mallow-board is V1.1C

We don’t understand why this is happening, but both SOMs are exactly the same, the Same Version (V1.2A), same processor markings (didn’t look any further then that).

Is this a hardware issue?
Is there some kind of voltage or oscillator that is unstable or an race/edge condition?

We Are hoping you could help us further with this, as we depend on this MCU only sleep mode to work.

The two SOMs in question are marked as follows:
Verdin AM62 S 512MB WB IT
V1.2A

The serial numbers are:
15599560 (Not Working)
15599343 (Working)

Build System

Everything is Build with Yocto and the following versions were used:

export YOCTO_RELEASE="scarthgap"
export YOCTO_VERSION=refs/tags/7.2.0
export MACHINE="verdin-am62"
export DISTRO="tdx-xwayland"
export IMAGE="tdx-reference-minimal-image"

Working SOM info (15599343)

tdx-info

Software summary
------------------------------------------------------------
Bootloader:               U-Boot
Kernel version:           6.6.129-7.2.0-devel #1 SMP PREEMPT Wed Apr  1 11:47:27 UTC 2026
Kernel command line:      root=PARTUUID=076c4a2a-01 ro rootwait console=tty1 console=ttyS2,115200 no_console_suspend rauc.slot=A
Distro name:              NAME="TDX Wayland with XWayland"
Distro version:           VERSION_ID=7.2.0-devel-20251204100250-build.0
Distro variant:           -
Hostname:                 verdin-am62-15599343
------------------------------------------------------------

Hardware info
------------------------------------------------------------
HW model:                 Toradex Verdin AM62 WB on Verdin Development Board
Toradex version:          0072 V1.2A
Serial number:            15599343
Processor arch:           aarch64
------------------------------------------------------------

suspend commands

echo 100 > /sys/devices/system/cpu/cpu0/power/pm_qos_resume_latency_us
echo mem > /sys/power/state

processor console output

root@verdin-am62-15599343:~# echo 100 > /sys/devices/system/cpu/cpu0/power/pm_qos_resume_latency_us
root@verdin-am62-15599343:~# echo mem > /sys/power/state
[  408.417982] PM: suspend entry (deep)
[  408.502152] Filesystems sync: 0.073 seconds
[  408.533584] Freezing user space processes
[  408.547271] Freezing user space processes completed (elapsed 0.004 seconds)
[  408.561325] OOM killer disabled.
[  408.564602] Freezing remaining freezable tasks
[  408.578771] Freezing remaining freezable tasks completed (elapsed 0.001 seconds)
[  408.623971] am65-cpsw-nuss 8000000.ethernet eth0: Link is Down
[  408.654732] ti-sci 44043000.system-controller: ti_sci_cmd_set_device_constraint: device: 117: state: 1: ret 0
[  408.673938] spark-led spi0.0: Attempted to sync while suspend
[  408.673962] ti-sci 44043000.system-controller: ti_sci_cmd_set_device_constraint: device: 178: state: 1: ret 0
[  408.709320] ti-sci 44043000.system-controller: ti_sci_cmd_set_device_constraint: device: 179: state: 1: ret 0
[  408.729614] ti-sci 44043000.system-controller: ti_sci_cmd_set_latency_constraint: latency: 100: state: 1: ret 0
[[  411.644470] am65-cpsw-nuss 8000000.ethernet: set new flow-id-base 19
[  411.676145] am65-cpsw-nuss 8000000.ethernet eth0: PHY [8000f00.mdio:00] driver [TI DP83867] (irq=360)
[  411.697891] am65-cpsw-nuss 8000000.ethernet eth0: configuring for phy/rgmii-rxid link mode
[  411.722978] usb-conn-gpio connector: repeated role: device
[  411.829024] OOM killer enabled.
[  411.837280] Restarting tasks ... done.
[  411.861909] random: crng reseeded on system resumption
[  411.879780] k3-m4-rproc 5000000.m4fss: Core is on in resume
[  411.901165] k3-m4-rproc 5000000.m4fss: received echo reply from 5000000.m4fss
root@verdin-am62-15599343:~# [  411.913685] PM: suspend exit
[  414.650577] am65-cpsw-nuss 8000000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx

root@verdin-am62-15599343:~#

mcu serial output

[LPM DMTIMER WAKEUP] Remote Core waiting for messages at end point 13 ... !!!
[LPM DMTIMER WAKEUP] Remote Core waiting for messages at end point 14 ... !!!
[LPM DMTIMER WAKEUP] Next MCU mode is 1
[LPM DMTIMER WAKEUP] Suspend request to MCU-only mode received
[LPM DMTIMER WAKEUP] Starting the timer.
[LPM DMTIMER WAKEUP] Once it expires, will resume the kernel from MCU only mode
[LPM DMTIMER WAKEUP] In while 0!
[LPM DMTIMER WAKEUP] Main domain resumed due to MCU DMTIMER 10
[LPM DMTIMER WAKEUP] Ticks: 10003, increment: 10


Non Working SOM info (15599560)

tdx-info

Software summary
------------------------------------------------------------
Bootloader:               U-Boot
Kernel version:           6.6.129-7.2.0-devel #1 SMP PREEMPT Wed Apr  1 11:47:27 UTC 2026
Kernel command line:      root=PARTUUID=076c4a2a-01 ro rootwait console=tty1 console=ttyS2,115200 no_console_suspend rauc.slot=A
Distro name:              NAME="TDX Wayland with XWayland"
Distro version:           VERSION_ID=7.2.0-devel-20251204100250-build.0
Distro variant:           -
Hostname:                 verdin-am62-15599343
------------------------------------------------------------

Hardware info
------------------------------------------------------------
HW model:                 Toradex Verdin AM62 WB on Verdin Development Board
Toradex version:          0072 V1.2A
Serial number:            15599560
Processor arch:           aarch64
------------------------------------------------------------

suspend commands

echo 100 > /sys/devices/system/cpu/cpu0/power/pm_qos_resume_latency_us
echo mem > /sys/power/state

processor console output

root@verdin-am62-15599343:~# echo 100 > /sys/devices/system/cpu/cpu0/power/pm_qos_resume_latency_us
root@verdin-am62-15599343:~# echo mem > /sys/power/state
[  295.169009] PM: suspend entry (deep)
[  295.189457] Filesystems sync: 0.009 seconds
[  295.215453] Freezing user space processes
[  295.226955] Freezing user space processes completed (elapsed 0.011 seconds)
[  295.226977] OOM killer disabled.
[  295.226981] Freezing remaining freezable tasks
[  295.239774] Freezing remaining freezable tasks completed (elapsed 0.012 seconds)
[  295.306039] am65-cpsw-nuss 8000000.ethernet eth0: Link is Down
[  295.351301] spark-led spi0.0: Attempted to sync while suspend

System hangs at this point

mcu serial output

[LPM DMTIMER WAKEUP] Remote Core waiting for messages at end point 13 ... !!!
[LPM DMTIMER WAKEUP] Remote Core waiting for messages at end point 14 ... !!!
[LPM DMTIMER WAKEUP] Next MCU mode is 149

System hangs at this point


both reports above show the same hostname with the same serial number because the same sd-card was used, tdx-info Serial does show a difference.

Also Noticed that the working one shows line such as:

[  408.654732] ti-sci 44043000.system-controller: ti_sci_cmd_set_device_constraint: device: 117: state: 1: ret 0

The non working doesn’t and this might be a divergence ?

Additionally i thought it had to do with the r5f core too, which showed it didn’t have mboxes, so i added this to my device tree overlays:

/ {
        compatible = "toradex,verdin-am62";
};

&mailbox0_cluster0 {
        status = "okay";

        mbox_r5_0: mbox-r5-0 {
                ti,mbox-rx = <2 0 0>;
                ti,mbox-tx = <3 0 0>;
        };
};

&wkup_r5fss0_core0 {
        mboxes = <&mailbox0_cluster0 &mbox_r5_0>;
        memory-region = <&wkup_r5fss0_core0_dma_memory_region>,
                        <&wkup_r5fss0_core0_memory_region>;
};

I will attach the binary for the MCU firmware and the source.

Another issue we ran into was TI documentation mentioning setting pm_qos_resume_latency_us to 100000 in newer docs while we had to set it to 100 which was mentioned in older docs.

Older Docs

Newer Docs

Hi, I’m the Toradex AI Assistant.

I might make mistakes or occasionally get you wrong. Our support team will step in if needed.


This appears to be a hardware-specific issue affecting one of your Verdin AM62 modules. The fact that the MCU receives value “149” instead of “1” over IPC suggests there may be a hardware-level communication problem between the main processor and the MCU core on the non-working module.

Analysis of the Issue

The key difference between your modules is:

  • Working SOM (15599343): MCU receives “1” over IPC and properly enters MCU-only sleep mode
  • Non-working SOM (15599560): MCU receives “149” over IPC and the system hangs

The missing ti-sci device constraint messages in the non-working module’s output suggests the system isn’t progressing through the normal suspend sequence (1).

Troubleshooting Steps

1. Verify Device Tree Configuration

Your device tree overlay for enabling the R5F core looks correct. The heterogeneous multicore processing setup requires proper mailbox configuration (1):

&mailbox0_cluster0 {
    status = "okay";
    
    mbox_r5_0: mbox-r5-0 {
        ti,mbox-rx = <2 0 0>;
        ti,mbox-tx = <3 0 0>;
    };
};

&wkup_r5fss0_core0 {
    mboxes = <&mailbox0_cluster0 &mbox_r5_0>;
    memory-region = <&wkup_r5fss0_core0_dma_memory_region>,
                    <&wkup_r5fss0_core0_memory_region>;
};

2. Check for Hardware Variations

Even though both modules are marked V1.2A, there could be subtle hardware differences:

  • Verify the processor stepping/revision numbers
  • Check if there are any visible differences in component markings
  • Test with a different carrier board to rule out board-specific issues

3. Firmware Validation

Ensure your MCU firmware is properly handling the IPC communication and that the value “149” isn’t being interpreted as a valid command by your firmware.

Potential Hardware Issue

Given that:

  • Both modules have identical markings (V1.2A)
  • Same software works on one but not the other
  • The IPC value corruption (149 vs 1) suggests communication issues

This strongly indicates a hardware defect in the non-working module (15599560). The IPC communication pathway between the main processor and MCU may have a hardware fault.

Recommendation

Since you’ve confirmed the software works correctly on one module but consistently fails on the other with identical software and hardware setup, I recommend contacting Toradex support directly with the serial numbers (15599560 - non-working, 15599343 - working) to investigate a potential hardware defect and possible replacement.

The pm_qos_resume_latency_us value of 100 you’re using is correct for this application based on the working module’s behavior.

Best regards,
Toradex AI Assistant.


Did you like my answer? :+1: :-1:

If I answered your question, please tick the “Solution” checkbox.

For follow-up questions to me, please use @ToradexAI in your post.

Here are my device tree setup for m4f and mcu binary + source

/dts-v1/;
/plugin/;

/ {
	compatible = "toradex,verdin-am62";
};

&mcu_m4fss {
	compatible = "ti,am64-m4fss";
	mboxes = <&mailbox0_cluster0 &mbox_m4_0>;
	mbox-names = "tx", "rx";
	memory-region = <&mcu_m4fss_dma_memory_region>,
			<&mcu_m4fss_memory_region>;
	firmware-name = "am62-mcu-m4f0_0-fw";
	wakeup-source;
	status = "okay";
};
/dts-v1/;
/plugin/;

/ {
	compatible = "toradex,verdin-am62";
};

// more /proc/device-tree/bus\@f0000/bus\@4000000/serial\@4a00000/status
&mcu_uart0 {
	status = "disabled";
};

main.c (3.8 KB)
lpm_mcu_dmtimer_wakeup.c (17.0 KB)
linker.cmd (2.6 KB)

lpm_mcu_dmtimer_wakeup_am62x-sk_m4fss0-0_freertos_ti-arm-clang.out (455.3 KB)

Your device tree overlays look correct for enabling the M4 core and disabling the MCU UART. However, there are some important considerations regarding the remoteproc setup for the M4 core that might be relevant to your IPC communication issue.

Device Tree Analysis

Your M4 device tree overlay follows the correct structure (1). The configuration you have:

&mcu_m4fss {
	compatible = "ti,am64-m4fss";
	mboxes = <&mailbox0_cluster0 &mbox_m4_0>;
	mbox-names = "tx", "rx";
	memory-region = <&mcu_m4fss_dma_memory_region>,
			<&mcu_m4fss_memory_region>;
	firmware-name = "am62-mcu-m4f0_0-fw";
	wakeup-source;
	status = "okay";
};

This matches the recommended approach for enabling the M4 core (2).

Potential Missing Configuration

Based on community discussions about remoteproc issues with the M4 core, you may need additional memory region definitions in your device tree (3). A working configuration includes:

&{/} {
    reserved-memory {
        #address-cells = <2>;
        #size-cells = <2>;
        ranges;

        m4_rts_ddr: ipc-memories@9c800000 {
                compatible = "shared-dma-pool";
                reg = <0x00 0x9c800000 0x00 0x00300000>;        /* 3 megabyte */
                no-map;
        };

        mcu_m4fss_dma_memory_region: m4f-dma-memory@9cb00000 {
            compatible = "shared-dma-pool";
            reg = <0x00 0x9cb00000 0x00 0x100000>;
            no-map;
        };

        mcu_m4fss_memory_region: m4f-memory@9cc00000 {
            compatible = "shared-dma-pool";
            reg = <0x00 0x9cc00000 0x00 0xe00000>;
            no-map;
        };
    };

    remoteproc@5000000 {
        compatible = "ti,am64-m4fss";
        reg = <0x00 0x5000000 0x00 0x30000>,
              <0x00 0x5040000 0x00 0x10000>;
        reg-names = "iram", "dram";
        resets = <&k3_reset 9 1>;
        firmware-name = "";
        mboxes = <&mailbox0_cluster0>, <&mbox_m4_0>;
        memory-region = <&mcu_m4fss_dma_memory_region>,
                        <&mcu_m4fss_memory_region>,
                        <&m4_rts_ddr>;
        ti,sci = <&dmsc>;
        ti,sci-dev-id = <9>;
        ti,sci-proc-ids = <0x18 0xff>;
     };
};

Firmware Loading Process

To load and run your firmware, follow these steps (2):

  1. Copy the firmware binary to the module:
$ scp ${SDK_INSTALL_PATH}/examples/hello_world/am62x-sk/m4fss0-0_freertos/ti-arm-clang/hello_world.release.out root@<hostname>:/home/torizon/am62-mcu-m4f0_0-fw
  1. Copy the firmware to /lib/firmware:
# sudo mount -o remount,rw /usr
# sudo mv am62-mcu-m4f0_0-fw /lib/firmware
  1. Start the firmware:
# sudo reboot

Hardware Issue Assessment

Given that your device tree configuration appears correct and the same firmware works on one module but not the other, this strongly suggests a hardware-level issue with the non-working module (15599560). The IPC value corruption (149 instead of 1) indicates a communication problem between the main processor and M4 core that is likely hardware-related.

The fact that the working module shows proper ti-sci device constraint messages while the non-working one doesn’t further supports this being a hardware defect rather than a software configuration issue.

Best regards,
Toradex AI Assistant.


Did you like my answer? :+1: :-1:

If I answered your question, please tick the “Solution” checkbox.

For follow-up questions to me, please use @ToradexAI in your post.

I tried with a different mallow with version V1.1B the non working SOM still did not work.
I tried a different Mallow with version V1.1C the non working SOM still did not work.

I tried a different SOM type and it didn’t work either:
Verdin am62 D 1GB IT
V1.2A
PN 00741200
SN 08676560

Though the output is different, and complains about mailboxes, but this might be a miss configuration.

tdx-info

Software summary
------------------------------------------------------------
Bootloader:               U-Boot
Kernel version:           6.6.129-7.2.0-devel #1 SMP PREEMPT Wed Apr  1 11:47:27 UTC 2026
Kernel command line:      root=PARTUUID=076c4a2a-01 ro rootwait console=tty1 console=ttyS2,115200 no_console_suspend rauc.slot=A
Distro name:              NAME="TDX Wayland with XWayland"
Distro version:           VERSION_ID=7.2.0-devel-20251204100250-build.0
Distro variant:           -
Hostname:                 verdin-am62-15599343
------------------------------------------------------------

Hardware info
------------------------------------------------------------
HW model:                 Toradex Verdin AM62 on Verdin Development Board
Toradex version:          0074 V1.2A
Serial number:            08676560
Processor arch:           aarch64
------------------------------------------------------------

suspend commands

echo 100 > /sys/devices/system/cpu/cpu0/power/pm_qos_resume_latency_us
echo mem > /sys/power/state

processor console output

root@verdin-am62-15599343:~# echo mem > /sys/power/state
[  282.374426] PM: suspend entry (deep)
[  282.446304] Filesystems sync: 0.064 seconds
[  282.468260] Freezing user space processes
[  282.489869] Freezing user space processes completed (elapsed 0.008 seconds)
[  282.496993] OOM killer disabled.
[  282.500268] Freezing remaining freezable tasks
[  282.505886] Freezing remaining freezable tasks completed (elapsed 0.001 seconds)
[  283.517748] ti-sci 44043000.system-controller: Mbox timedout in resp(caller: sci_clk_determine_rate+0x68/0xec)
[  283.527891] ti-sci 44043000.system-controller: Mbox send fail -110
[  283.534135] ti-sci-clk 44043000.system-controller:clock-controller: determine-rate failed for dev=135, clk=0, ret=-110
[  284.573747] ti-sci 44043000.system-controller: Mbox timedout in resp(caller: sci_clk_determine_rate+0x68/0xec)
[  284.583881] ti-sci 44043000.system-controller: Mbox send fail -110
[  284.590127] ti-sci-clk 44043000.system-controller:clock-controller: determine-rate failed for dev=135, clk=0, ret=-110
[  285.629739] ti-sci 44043000.system-controller: Mbox timedout in resp(caller: sci_clk_determine_rate+0x68/0xec)
[  285.639846] ti-sci 44043000.system-controller: Mbox send fail -110
[  285.646086] ti-sci-clk 44043000.system-controller:clock-controller: determine-rate failed for dev=135, clk=0, ret=-110
[  285.656882] cpu cpu0: _opp_config_clk_single: failed to set clock rate: -22
[  285.663912] cpufreq: __target_index: Failed to change cpu frequency: -22
[  285.670684] cpufreq: cpufreq_generic_suspend: unable to set suspend-freq: 1250000. err: -22
[  285.680575] cpufreq: cpufreq_suspend: Failed to suspend driver: cpufreq-dt
[  286.717754] ti-sci 44043000.system-controller: Mbox timedout in resp(caller: ti_sci_cmd_put_clock+0x18/0x24)
[  286.730881] ti-sci 44043000.system-controller: Mbox send fail -110
[  286.738788] ti-sci-clk 44043000.system-controller:clock-controller: unprepare failed for dev=58, clk=6, ret=-110
[  287.773754] ti-sci 44043000.system-controller: Mbox timedout in resp(caller: ti_sci_cmd_put_clock+0x18/0x24)
[  287.786862] ti-sci 44043000.system-controller: Mbox send fail -110
[  287.794746] ti-sci-clk 44043000.system-controller:clock-controller: unprepare failed for dev=58, clk=8, ret=-110
[  288.829760] ti-sci 44043000.system-controller: Mbox timedout in resp(caller: ti_sci_cmd_put_clock+0x18/0x24)
[  288.842916] ti-sci 44043000.system-controller: Mbox send fail -110
[  288.850924] ti-sci-clk 44043000.system-controller:clock-controller: unprepare failed for dev=57, clk=6, ret=-110
[  289.885758] ti-sci 44043000.system-controller: Mbox timedout in resp(caller: ti_sci_cmd_put_clock+0x18/0x24)
[  289.898819] ti-sci 44043000.system-controller: Mbox send fail -110
[  289.906697] ti-sci-clk 44043000.system-controller:clock-controller: unprepare failed for dev=57, clk=8, ret=-110
[  290.941754] ti-sci 44043000.system-controller: Mbox timedout in resp(caller: ti_sci_cmd_free_event_map+0x74/0xe4)
[  290.955304] ti-sci 44043000.system-controller: Mbox send fail -110
[  291.965747] ti-sci 44043000.system-controller: Mbox timedout in resp(caller: k3_ringacc_ring_free+0x164/0x238)
[  291.978963] ti-sci 44043000.system-controller: RM_RA:Mbox config send fail -110
[  291.989375] ti-udma 485c0000.dma-controller: TISCI ring free fail (-110) ring_idx 118
[  293.021746] ti-sci 44043000.system-controller: Mbox timedout in resp(caller: ti_sci_cmd_free_event_map+0x74/0xe4)
[  293.035314] ti-sci 44043000.system-controller: Mbox send fail -110
[  294.045749] ti-sci 44043000.system-controller: Mbox timedout in resp(caller: k3_ringacc_ring_free+0x164/0x238)
[  294.058984] ti-sci 44043000.system-controller: RM_RA:Mbox config send fail -110
[  294.069435] ti-udma 485c0000.dma-controller: TISCI ring free fail (-110) ring_idx 19
[  295.101751] ti-sci 44043000.system-controller: Mbox timedout in resp(caller: ti_sci_cmd_free_event_map+0x74/0xe4)
[  295.115331] ti-sci 44043000.system-controller: Mbox send fail -110
^C[  296.125750] ti-sci 44043000.system-controller: Mbox timedout in resp(caller: k3_ringacc_ring_free+0x164/0x238)
[  296.138999] ti-sci 44043000.system-controller: RM_RA:Mbox config send fail -110
[  296.149452] ti-udma 485c0000.dma-controller: TISCI ring free fail (-110) ring_idx 27
[  297.181751] ti-sci 44043000.system-controller: Mbox timedout in resp(caller: ti_sci_cmd_free_event_map+0x74/0xe4)
[  297.195390] ti-sci 44043000.system-controller: Mbox send fail -110
[  298.205746] ti-sci 44043000.system-controller: Mbox timedout in resp(caller: k3_ringacc_ring_free+0x164/0x238)
[  298.219006] ti-sci 44043000.system-controller: RM_RA:Mbox config send fail -110
[  298.229465] ti-udma 485c0000.dma-controller: TISCI ring free fail (-110) ring_idx 35
[  299.261747] ti-sci 44043000.system-controller: Mbox timedout in resp(caller: ti_sci_cmd_free_event_map+0x74/0xe4)
[  299.275382] ti-sci 44043000.system-controller: Mbox send fail -110
[  300.285748] ti-sci 44043000.system-controller: Mbox timedout in resp(caller: k3_ringacc_ring_free+0x164/0x238)
[  300.299039] ti-sci 44043000.system-controller: RM_RA:Mbox config send fail -110
[  300.309512] ti-udma 485c0000.dma-controller: TISCI ring free fail (-110) ring_idx 43
[  301.341750] ti-sci 44043000.system-controller: Mbox timedout in resp(caller: ti_sci_cmd_free_event_map+0x74/0xe4)
[  301.355387] ti-sci 44043000.system-controller: Mbox send fail -110

System keeps printing errors at this point.

mcu serial output

No further output

Here is a discussion we had on the TI forum:
https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1637404/am625-am62x-mcu-only-mode-wrong-host-state/

Further more the problem i mentioned earlier was a device-tree issue.

I have been doing another test with a new SOM new out of the package:
Verdin AM62 D 1GB IT
V1.2A
PN 00741200
SN 08832885

mallow:
V1.1C
PN 01611102
SN 12867573

tdx-info

Software summary
------------------------------------------------------------
Bootloader:               U-Boot
Kernel version:           6.6.138-7.2.0-devel #1 SMP PREEMPT Fri May  8 07:27:41 UTC 2026
Kernel command line:      root=PARTUUID=076c4a2a-01 ro rootwait console=tty1 console=ttyS2,115200 no_console_suspend rauc.slot=A
Distro name:              NAME="TDX Wayland with XWayland"
Distro version:           VERSION_ID=7.2.0-devel-20251204100250-build.0
Distro variant:           -
Hostname:                 verdin-am62-08832885
------------------------------------------------------------

Hardware info
------------------------------------------------------------
HW model:                 Toradex Verdin AM62 on Verdin Development Board
Toradex version:          0074 V1.2A
Serial number:            08832885
Processor arch:           aarch64
------------------------------------------------------------

suspend commands

echo 100 > /sys/devices/system/cpu/cpu0/power/pm_qos_resume_latency_us
echo mem > /sys/power/state

processor console output

root@verdin-am62-08832885:/usr/lib/firmware# echo 100 > /sys/devices/system/cpu/cpu0/power/pm_qos_resume_latency_us
root@verdin-am62-08832885:/usr/lib/firmware# echo mem > /sys/power/state
[  634.551094] PM: suspend entry (deep)
[  634.597532] Filesystems sync: 0.039 seconds
[  634.618269] Freezing user space processes
[  634.628197] Freezing user space processes completed (elapsed 0.001 seconds)
[  634.635323] OOM killer disabled.
[  634.638590] Freezing remaining freezable tasks
[  634.644488] Freezing remaining freezable tasks completed (elapsed 0.001 seconds)
[  634.682397] am65-cpsw-nuss 8000000.ethernet eth0: Link is Down
[  634.724116] spark-led spi0.0: Attempted to sync while suspend

after a while (a good 3-5 minutes):

[ERROR:   Unexpected affinity info state.
PANIC in EL3.
x30            = 0x000000009e78619c
x0             = 0x0000000002800000
x1             = 0x0000000000000060
x2             = 0x00000[  966.368279] am65-cpsw-nuss 8000000.ethernet: set new flow-id-base 19
00000000060
x3             = 0x000000009e7931c0
x4             = 0x0000000030cd183f
x5             = 0x0000000030cd183b
x6             = 0x75f55f4a0654f9d4
x7             = 0x0000000000000000
x8             = 0x000000009e7806e4
x9             = 0x000000009e780174
x10            = 0x000000009e78105c
x11            = 0x0000000000000001
x12            = 0x000000009e78a1f0
x13            = 0x000000009e78a1f0
x14            = 0x000000000[  966.414329] am65-cpsw-nuss 8000000.ethernet eth0: PHY [8000f00.mdio:00] driver [TI DP83867] (irq=360)
0000004
x15            = 0x000000009e780130
x16            = 0xcb6a3242c82c3cc4
x17            = 0x2dc661b69a1f3e09
x18            = 0x7bf5cd55c1133e7f
x19            = 0x000000009e780130
x20            = 0x0000000000000001
x21            = 0x1fc6cf71c67a7097
x22            = 0xc193db1cfeb39e59
x23      [  966.451394] am65-cpsw-nuss 8000000.ethernet eth0: configuring for phy/rgmii-rxid link mode
      = 0x931d396ad3ec7151
x24            = 0xe9ffbd0796dfc7f6
x25            = 0x4f0929bc63d736bd
x26            = 0xa452c0fa452942d3
x27            = 0x498963ce40848045
x28            = 0x5f4d6960eba50849
x29            = 0x000000009e78dec0
scr_el3        = 0x0000000000000238
sctlr_el3      = 0x0000000030cd183f
cptr_el3       = 0x0000000080000400
tcr_el3        = 0x0000000080803520
daif           = 0x00000000000002c0
mair_el[  966.498849] usb-conn-gpio connector: repeated role: device
3       = 0x00000000004404ff
spsr_el3       = 0x00000000000001cd
elr_el3        = 0x735b075c4c71aac0
ttbr0_el3      = 0x000000009e7931c0
esr_el3        = 0x000000009e1a642c
far_el3        = 0xf57f7ffe07860280
spsr_el1       = 0x0000000040121104
elr_el1        = 0xbe7c66c229dea0c1
spsr_abt       = 0x000000000e1b6724
spsr_und       = 0x0000000082111408
spsr_irq       = 0x0000000086123f01
spsr_fiq       = 0x0000000094155322
sctlr_el1      = 0x0000000000c50838
actlr_el1      = 0x0000000000000000
cpacr_el1      = 0x0000000000000000
csselr_el1     = 0x0000000000000000
sp_el1         = 0xa270cf79acddab74
esr_el1        = 0x0000000081d408f2
ttbr0_el1      = 0x025c914a86882a02
ttbr1_el1      = 0x0402e0a185008b4a
mair_el1       = 0x44e048e000098aa4
amair_el1      = 0x0000000000000000
tcr_el1        = 0x0000000000000000
tpidr_el1      = 0x2108000010300090
tpidr_el0      = 0x840c0480001ee180
tpidrro_el0    = 0x00821833160a0320
par_el1        = 0xb400000202048108
mpidr_el1      = 0x0000000080000001
afsr0_el1      = 0x0000000000000000
afsr1_el1      = 0x0000000000000000
contextidr_el1 = 0x0000000000000000
vbar_el1       [  966.606750] OOM killer enabled.
= 0x0000000000000000
cntp_ctl_el0   = 0x0000000000000000
cntp_cval_el0  = 0x0000002c57cc4777
cntv_ctl_el0   = 0x0000000000000002
cntv_cval_el0  = 0x03b8174ef65e6ea2
cntkctl_el1    = 0x0000000000[  966.618666] Restarting tasks ... done.
000000
sp_el0         = 0x000000009e78dec0
isr_el1        = 0x0000000000000000
dacr32_el2     = 0x0000000044996095
ifsr32_el2     = 0x0000000000000610
cpuectlr_el1   = 0x0000000000000040
cpumerrsr_el1  = 0x0000000001080594
l2merrsr_el1   = 0x0000000011185eb0
cpuactlr_el1   = 0x00001000090ca000
[  966.670009] random: crng reseeded on system resumption
[  966.714804] k3-m4-rproc 5000000.m4fss: Core is on in resume
[  966.771440] PM: suspend exit
root@verdin-am62-08832885:/usr/lib/firmware# [  966.836250] systemd[1]: systemd-udevd.service: Watchdog timeout (limit 3min)!
[  966.861225] systemd[1]: systemd-udevd.service: Killing process 165 (systemd-udevd) with signal SIGABRT.
[  966.984165] systemd[1]: systemd-journald.service: Main process exited, code=dumped, status=6/ABRT
[  967.023795] systemd[1]: systemd-journald.service: Failed with result 'watchdog'.
[  967.058299] systemd[1]: systemd-journald.service: Consumed 2.008s CPU time.
[  967.101665] systemd[1]: systemd-udevd.service: Main process exited, code=dumped, status=6/ABRT
[  967.157109] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.
[  967.212656] systemd[1]: Journal Audit Socket was skipped because of an unmet condition check (ConditionSecurity=audit).
[  967.308901] systemd[1]: Starting Journal Service...
[  967.347716] systemd[1]: systemd-udevd.service: Failed with result 'watchdog'.
[  967.380749] systemd[1]: systemd-udevd.service: Consumed 10.925s CPU time.
[  967.444324] systemd[1]: systemd-udevd.service: Scheduled restart job, restart counter is at 1.
[  967.602241] systemd[1]: Starting Rule-based Manager for Device Events and Files...
[  967.752961] systemd[1]: game-engine.service: Main process exited, code=exited, status=203/EXEC
[  967.793919] systemd[1]: game-engine.service: Failed with result 'exit-code'.
[  968.497431] systemd[1]: systemd-tmpfiles-clean.service: Deactivated successfully.
[  968.535194] systemd[1]: Finished Cleanup of Temporary Directories.
[  968.777988] systemd-journald[1368]: Collecting audit messages is disabled.
[  968.857892] systemd-journald[1368]: File /var/log/journal/e76a5b762961402188db567b51dc37ca/system.journal corrupted or uncleanly shut down, renaming and replacing.
[  969.145173] systemd[1]: Started Rule-based Manager for Device Events and Files.
[  969.177501] systemd[1]: Started Journal Service.
[  969.367910] am65-cpsw-nuss 8000000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx

mcu serial output

[LPM DMTIMER WAKEUP] Remote Core waiting for messages at end point 13 ... !!!
[LPM DMTIMER WAKEUP] Remote Core waiting for messages at end point 14 ... !!!
[LPM DMTIMER WAKEUP] Next MCU mode is 149

no further messages

u-boot

u-boot says it has loaded the correct device tree:

U-Boot SPL 2023.04-ti-gc997b1b09fba (Jun 19 2024 - 16:10:02 +0000)
SYSFW ABI: 3.1 (firmware rev 0x0009 '9.2.8--v09.02.08 (Kool Koala)')
Changed A53 CPU frequency to 1250000000Hz (T grade) in DT
SPL initial stack usage: 13440 bytes
Trying to boot from MMC1
Authentication passed
Authentication passed
Authentication passed
Loading Environment from nowhere... OK
init_env from device 9 not supported!
Authentication passed
Authentication passed
Starting ATF on ARM64 core...

NOTICE:  BL31: v2.10.0(release):v2.10.0-367-g00f1ec6b87-dirty
NOTICE:  BL31: Built : 16:09:05, Feb  9 2024

U-Boot SPL 2023.04-ti-gc997b1b09fba (Jun 19 2024 - 16:10:02 +0000)
SYSFW ABI: 3.1 (firmware rev 0x0009 '9.2.8--v09.02.08 (Kool Koala)')
SPL initial stack usage: 1920 bytes
Trying to boot from MMC1
Authentication passed
Authentication passed


U-Boot 2023.04-ti-gc997b1b09fba (Jun 19 2024 - 16:10:02 +0000)

SoC:   AM62X SR1.0 HS-FS
DRAM:  1 GiB
Core:  143 devices, 31 uclasses, devicetree: separate
MMC:   mmc@fa10000: 0, mmc@fa00000: 1
Loading Environment from MMC... OK
In:    serial@2800000
Out:   serial@2800000
Err:   serial@2800000
Model: Toradex 0074 Verdin AM62 Dual 1GB IT V1.2A
Serial#: 08832885
Carrier: Toradex UNKNOWN CARRIER BOARD V1.1C, Serial# 12867573
am65_cpsw_nuss ethernet@8000000: K3 CPSW: nuss_ver: 0x6BA01103 cpsw_ver: 0x6BA81103 ale_ver: 0x00290105 Ports:2 mdio_freq:1000000
Net:
Warning: ethernet@8000000port@1 MAC addresses don't match:
Address in ROM is               c0:d6:0a:e2:5f:65
Address in environment is       00:14:2d:86:c7:75
eth0: ethernet@8000000port@1 [PRIME]Could not get PHY for ethernet@8000000port@1: addr 7
am65_cpsw_nuss_port ethernet@8000000port@2: phy_connect() failed

Hit any key to stop autoboot:  0
switch to partitions #0, OK
mmc1 is current device
Scanning mmc 1:1...
Found U-Boot script /boot/boot.scr
6007 bytes read in 10 ms (585.9 KiB/s)
## Executing script at 90280000
Saving Environment to MMC... Writing to MMC(0)... OK
RAUC: Booting slot A (partition 1)
ttyS2: No match
Loading DeviceTree: /boot/k3-am625-verdin-nonwifi-dev.dtb
73111 bytes read in 11 ms (6.3 MiB/s)
168 bytes read in 11 ms (14.6 KiB/s)
Working FDT set to 90200000
Applying Overlay: ledring_overlay.dtbo
1619 bytes read in 12 ms (130.9 KiB/s)
Applying Overlay: I2C_overlay.dtbo
1819 bytes read in 12 ms (147.5 KiB/s)
Applying Overlay: cc1310_overlay.dtbo
1357 bytes read in 12 ms (110.4 KiB/s)
Applying Overlay: gpio_overlay.dtbo
1835 bytes read in 12 ms (148.4 KiB/s)
Applying Overlay: m4f_overlay.dtbo
871 bytes read in 12 ms (70.3 KiB/s)
Applying Overlay: am62x_mcu.dtbo
403 bytes read in 11 ms (35.2 KiB/s)
Applying Overlay: tas5808_overlay.dtbo
3908 bytes read in 12 ms (317.4 KiB/s)
Applying Overlay: wifi_disable_overlay.dtbo
527 bytes read in 12 ms (42 KiB/s)
8085395 bytes read in 94 ms (82 MiB/s)
Bootargs: root=PARTUUID=076c4a2a-01 ro rootwait console=tty1 console=ttyS2,115200 no_console_suspend rauc.slot=A
   Uncompressing Kernel Image
## Flattened Device Tree blob at 90200000
   Booting using the fdt blob at 0x90200000
Working FDT set to 90200000
   Loading Device Tree to 0000000098ea4000, end 0000000098ed8fff ... OK
Working FDT set to 98ea4000

Starting kernel ...

We resolved the issue,

Turns out the bootloader was to blame, we had an old u-boot version which was loading an old DM WKUP R5F firmware version, and due to this we had the communication issues, even when the Processor SDK version and MCU+ SDK version matched.