Verdin IMX8M Mini, CAN controller returns IRQ handler mcp251xfd_handle_tefif() returned -22

We have been running into CAN bus issues lately where the mcp2518FD CAN controller dies and doesn’t recover until you run ip link set down/up commands. We don’t have heavy traffic on the CAN controller and there isn’t any special configurations/modifications besides patches.

The board we are using with the Verdin is the mallow carrier board. We are running kernel version 6.6.54 from the 7.1.0 Toradex BSP release, and we applied patches up to 7.6.2 of the Toradex BSP release. With these patches, we have seen the driver go from a full day to a few days without it crashing, compared to without the patches, we were able to get it to crash around every 5–10 minutes. We are also using the default device tree that is provided for the mallow and the bitrate we have configured for the CAN controller is 1Mbps.

Is this a known issue and are there any fixes available? Is it possible we have something configured incorrectly?

image

Thanks!

Jason

Hi, I’m the Toradex AI Assistant.

I might make mistakes or occasionally get you wrong. Our support team will step in if needed.


Before I provide specific guidance, could you please confirm the following details about your setup:

  1. Hardware: You mentioned Verdin iMX8M Mini with Mallow Carrier Board - which specific Verdin iMX8M Mini variant are you using (e.g., 0055, 0057, 0059, 0068)?
  2. OS Version: You mentioned BSP 7.1.0 with patches up to 7.6.2 - are you running a Yocto-based image or Torizon OS?
  3. CAN Configuration: Are you using the on-module MCP2518FD controller, or an external controller connected via SPI?

You can gather this information by running the tdx-info command on your device. For more details, see tdx-info documentation.

Initial Observations

The error message mcp251xfd_handle_tefif() returned -22 indicates an -EINVAL error in the Transmit Event FIFO interrupt handler of the MCP2518FD driver. This suggests the driver encountered an unexpected state during CAN frame transmission handling.

Based on the community discussion about CAN clock source changes(1), there was a known issue with Verdin iMX8M Mini where the CAN clock source changed from 20MHz to 40MHz in certain hardware revisions (starting with BSP 5.7.0). This clock frequency mismatch can cause communication issues between CAN controllers.

Potential Causes

  1. Clock Configuration Mismatch: If your hardware revision uses a different clock frequency than what’s configured in the device tree
  2. SPI Communication Issues: Problems with the SPI interface between the SoC and the MCP2518FD controller
  3. Timing/Bitrate Configuration: Incorrect bit-timing parameters for your 1Mbps configuration

Once you provide the specific hardware and OS version details, I can give you more targeted troubleshooting steps and potential fixes.

Best regards,
Toradex AI Assistant.


Did you like my answer? :+1: :-1:

If I answered your question, please tick the “Solution” checkbox.

For follow-up questions to me, please use @ToradexAI in your post.

Thank you for providing those details. Based on your configuration with the Verdin iMX8M Mini 0059 V1.1D, this is indeed a known issue related to the CAN clock source change.

The Root Cause

The Verdin iMX8M Mini 0059 variant experienced a product change where the CAN clock source changed from 20MHz to 40MHz(1). Since you’re running kernel 6.6.54 from BSP 7.1.0 (even with patches up to 7.6.2), there may be a clock frequency mismatch causing the MCP2518FD controller instability you’re experiencing.

Solution: Apply Clock Source Fix

You need to modify the CAN clock frequency in the device tree. Since you’re building Torizon OS with Yocto and adding your own layers, you can apply this fix through your custom meta-layer(1).

Step 1: Create a Kernel Patch

Create a patch file to change the clock frequency from 40MHz back to 20MHz:

imx8mm-verdin-clock-fix.patch

diff --git a/arch/arm64/boot/dts/freescale/imx8mm-verdin.dtsi b/arch/arm64/boot/dts/freescale/imx8mm-verdin.dtsi
index d03e33a..a5ac27c 100755
--- a/arch/arm64/boot/dts/freescale/imx8mm-verdin.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mm-verdin.dtsi
@@ -35,7 +35,7 @@
  clk40m: oscillator {
  compatible = "fixed-clock";
  #clock-cells = <0>;
- clock-frequency = <40000000>;
+ clock-frequency = <20000000>;
  };
  
  gpio-keys {

(1)

Step 2: Add Patch to Your Yocto Build

In your custom meta-layer, create a .bbappend file for the kernel recipe(2):

  1. Create the directory structure:
$ mkdir -p recipes-kernel/linux/linux-toradex-ti/linux-toradex-ti
  1. Create the .bbappend file:

linux-toradex-ti_%.bbappend

FILESEXTRAPATHS:prepend := "${THISDIR}/${PN}:"

SRC_URI += " \
    file://imx8mm-verdin-clock-fix.patch \
    "

(2)

  1. Place your imx8mm-verdin-clock-fix.patch file in the recipes-kernel/linux/linux-toradex-ti/linux-toradex-ti directory.

Step 3: Rebuild Your Image

Rebuild your Torizon OS image with the patch applied:

$ bitbake <your-image-name>

(2)

Additional Considerations

Since you’re running with PREEMPT_RT, ensure that your real-time configuration isn’t introducing additional latency that could affect SPI communication with the MCP2518FD controller.

After applying this fix and deploying the updated image, monitor your CAN bus stability. The clock frequency mismatch is the most likely cause of the -EINVAL errors you’re seeing in the TEF (Transmit Event FIFO) interrupt handler.

If issues persist after this fix, please let me know and we can investigate other potential causes such as SPI timing parameters or bitrate configuration.

Best regards,
Toradex AI Assistant.


Did you like my answer? :+1: :-1:

If I answered your question, please tick the “Solution” checkbox.

For follow-up questions to me, please use @ToradexAI in your post.

Could you go a little deeper into the clock configuration and why this might be the cause since the V1.1D has the 40 MHz clock @ToradexAI

You’re absolutely right to question this - let me clarify the clock configuration issue more precisely.

The Clock Configuration Change

According to the Product Change Notification, the V1.1D hardware revision introduced a change where “the frequency of the CAN clock source has been changed from 20MHz to 40MHz”(1). This means your V1.1D hardware physically has a 40MHz clock source.

Why the Clock Mismatch Causes Issues

The problem occurs when there’s a mismatch between the hardware clock frequency and what the device tree tells the driver. Here’s what happens:

  1. Hardware Reality: Your V1.1D board has a 40MHz oscillator feeding the MCP2518FD CAN controller
  2. Driver Configuration: The device tree must correctly specify this clock frequency so the MCP2518FD driver can calculate proper bit timing
  3. Bit Timing Calculations: The driver uses the clock frequency to derive the Time Quantum (TQ) and configure the CAN bit rate

The Actual Issue

Looking at the CAN documentation more carefully, the workaround patch I initially suggested (changing 40MHz to 20MHz) was actually meant for older hardware revisions that were incorrectly configured(2).

Since you have V1.1D with the 40MHz clock, your device tree should already be correct at 40MHz. The -EINVAL error you’re seeing is likely not caused by clock frequency mismatch, but rather by:

  1. SPI Communication Issues: The MCP2518FD is connected via SPI, and timing issues on the SPI bus can cause the driver to read invalid data from the controller’s registers
  2. Interrupt Handling Problems: The PREEMPT_RT kernel you’re running may be introducing timing issues in interrupt handling
  3. FIFO Overflow/Underflow: The Transmit Event FIFO (TEF) interrupt handler is failing, suggesting the controller’s internal FIFOs may be getting into an unexpected state

What You Should Check

Instead of modifying the clock frequency, verify:

  1. Device Tree Clock Setting: Confirm your device tree has:
clk40m: oscillator {
    compatible = "fixed-clock";
    #clock-cells = <0>;
    clock-frequency = <40000000>;
};

(2)

  1. SPI Configuration: Check your SPI bus frequency and timing parameters for the MCP2518FD connection

  2. Kernel Patches: You mentioned applying patches up to 7.6.2 - verify these include all MCP2518FD driver fixes

I apologize for the initial confusion about reverting to 20MHz. Since you have V1.1D, you should keep the 40MHz configuration. The issue is likely elsewhere in the driver or SPI communication layer.

Could you share:

  • Your current device tree configuration for the CAN controller?
  • The exact BSP patches you’ve applied?
  • Any custom SPI or CAN configuration you’ve made?

This will help identify the actual root cause of the TEF interrupt handler failures.

Best regards,
Toradex AI Assistant.


Did you like my answer? :+1: :-1:

If I answered your question, please tick the “Solution” checkbox.

For follow-up questions to me, please use @ToradexAI in your post.

@ToradexAI we are using the default device tree that is provided for this version so clk40m is 40MHz.

The patches applied were

From 3c1c18551e6ac1b988d0a05c5650e3f6c95a1b8a Mon Sep 17 00:00:00 2001
From: Marc Kleine-Budde <mkl@pengutronix.de>
Date: Tue, 1 Oct 2024 16:56:22 +0200
Subject: can: mcp251xfd: mcp251xfd_get_tef_len(): fix length calculation

Commit b8e0ddd36ce9 ("can: mcp251xfd: tef: prepare to workaround
broken TEF FIFO tail index erratum") introduced
mcp251xfd_get_tef_len() to get the number of unhandled transmit events
from the Transmit Event FIFO (TEF).

As the TEF has no head pointer, the driver uses the TX FIFO's tail
pointer instead, assuming that send frames are completed. However the
check for the TEF being full was not correct. This leads to the driver
stop working if the TEF is full.

Fix the TEF full check by assuming that if, from the driver's point of
view, there are no free TX buffers in the chip and the TX FIFO is
empty, all messages must have been sent and the TEF must therefore be
full.

Reported-by: Sven Schuchmann <schuchmann@schleissheimer.de>
Closes: https://patch.msgid.link/FR3P281MB155216711EFF900AD9791B7ED9692@FR3P281MB1552.DEUP281.PROD.OUTLOOK.COM
Fixes: b8e0ddd36ce9 ("can: mcp251xfd: tef: prepare to workaround broken TEF FIFO tail index erratum")
Tested-by: Sven Schuchmann <schuchmann@schleissheimer.de>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20241104-mcp251xfd-fix-length-calculation-v3-1-608b6e7e2197@pengutronix.de
Signed-off-by: Marc Kleine-Budde <mkl@pengutronix.de>
---
 drivers/net/can/spi/mcp251xfd/mcp251xfd-tef.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

(limited to 'drivers/net/can/spi/mcp251xfd')

diff --git a/drivers/net/can/spi/mcp251xfd/mcp251xfd-tef.c b/drivers/net/can/spi/mcp251xfd/mcp251xfd-tef.c
index f732556d233a..d3ac865933fd 100644
--- a/drivers/net/can/spi/mcp251xfd/mcp251xfd-tef.c
+++ b/drivers/net/can/spi/mcp251xfd/mcp251xfd-tef.c
@@ -16,9 +16,9 @@
 
 #include "mcp251xfd.h"
 
-static inline bool mcp251xfd_tx_fifo_sta_full(u32 fifo_sta)
+static inline bool mcp251xfd_tx_fifo_sta_empty(u32 fifo_sta)
 {
-	return !(fifo_sta & MCP251XFD_REG_FIFOSTA_TFNRFNIF);
+	return fifo_sta & MCP251XFD_REG_FIFOSTA_TFERFFIF;
 }
 
 static inline int
@@ -122,7 +122,11 @@ mcp251xfd_get_tef_len(struct mcp251xfd_priv *priv, u8 *len_p)
 	if (err)
 		return err;
 
-	if (mcp251xfd_tx_fifo_sta_full(fifo_sta)) {
+	/* If the chip says the TX-FIFO is empty, but there are no TX
+	 * buffers free in the ring, we assume all have been sent.
+	 */
+	if (mcp251xfd_tx_fifo_sta_empty(fifo_sta) &&
+	    mcp251xfd_get_tx_free(tx_ring) == 0) {
 		*len_p = tx_ring->obj_num;
 		return 0;
 	}
-- 
cgit v1.2.3


From eb9a839b3d8a989be5970035a5cf29bcd6ffd24d Mon Sep 17 00:00:00 2001
From: Marc Kleine-Budde <mkl@pengutronix.de>
Date: Fri, 25 Oct 2024 14:34:40 +0200
Subject: can: mcp251xfd: mcp251xfd_ring_alloc(): fix coalescing configuration
 when switching CAN modes

Since commit 50ea5449c563 ("can: mcp251xfd: fix ring configuration
when switching from CAN-CC to CAN-FD mode"), the current ring and
coalescing configuration is passed to can_ram_get_layout(). That fixed
the issue when switching between CAN-CC and CAN-FD mode with
configured ring (rx, tx) and/or coalescing parameters (rx-frames-irq,
tx-frames-irq).

However 50ea5449c563 ("can: mcp251xfd: fix ring configuration when
switching from CAN-CC to CAN-FD mode"), introduced a regression when
switching CAN modes with disabled coalescing configuration: Even if
the previous CAN mode has no coalescing configured, the new mode is
configured with active coalescing. This leads to delayed receiving of
CAN-FD frames.

This comes from the fact, that ethtool uses usecs = 0 and max_frames =
1 to disable coalescing, however the driver uses internally
priv->{rx,tx}_obj_num_coalesce_irq = 0 to indicate disabled
coalescing.

Fix the regression by assigning struct ethtool_coalesce
ec->{rx,tx}_max_coalesced_frames_irq = 1 if coalescing is disabled in
the driver as can_ram_get_layout() expects this.

Reported-by: https://github.com/vdh-robothania
Closes: https://github.com/raspberrypi/linux/issues/6407
Fixes: 50ea5449c563 ("can: mcp251xfd: fix ring configuration when switching from CAN-CC to CAN-FD mode")
Cc: stable@vger.kernel.org
Reviewed-by: Simon Horman <horms@kernel.org>
Link: https://patch.msgid.link/20241025-mcp251xfd-fix-coalesing-v1-1-9d11416de1df@pengutronix.de
Signed-off-by: Marc Kleine-Budde <mkl@pengutronix.de>
---
 drivers/net/can/spi/mcp251xfd/mcp251xfd-ring.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

(limited to 'drivers/net/can/spi/mcp251xfd')

diff --git a/drivers/net/can/spi/mcp251xfd/mcp251xfd-ring.c b/drivers/net/can/spi/mcp251xfd/mcp251xfd-ring.c
index e684991fa391..7209a831f0f2 100644
--- a/drivers/net/can/spi/mcp251xfd/mcp251xfd-ring.c
+++ b/drivers/net/can/spi/mcp251xfd/mcp251xfd-ring.c
@@ -2,7 +2,7 @@
 //
 // mcp251xfd - Microchip MCP251xFD Family CAN controller driver
 //
-// Copyright (c) 2019, 2020, 2021 Pengutronix,
+// Copyright (c) 2019, 2020, 2021, 2024 Pengutronix,
 //               Marc Kleine-Budde <kernel@pengutronix.de>
 //
 // Based on:
@@ -483,9 +483,11 @@ int mcp251xfd_ring_alloc(struct mcp251xfd_priv *priv)
 		};
 		const struct ethtool_coalesce ec = {
 			.rx_coalesce_usecs_irq = priv->rx_coalesce_usecs_irq,
-			.rx_max_coalesced_frames_irq = priv->rx_obj_num_coalesce_irq,
+			.rx_max_coalesced_frames_irq = priv->rx_obj_num_coalesce_irq == 0 ?
+				1 : priv->rx_obj_num_coalesce_irq,
 			.tx_coalesce_usecs_irq = priv->tx_coalesce_usecs_irq,
-			.tx_max_coalesced_frames_irq = priv->tx_obj_num_coalesce_irq,
+			.tx_max_coalesced_frames_irq = priv->tx_obj_num_coalesce_irq == 0 ?
+				1 : priv->tx_obj_num_coalesce_irq,
 		};
 		struct can_ram_layout layout;
 
-- 
cgit v1.2.3
From 30447a1bc0e066e492552b3e5ffeb63c1605dfe2 Mon Sep 17 00:00:00 2001
From: Marc Kleine-Budde <mkl@pengutronix.de>
Date: Sun, 24 Nov 2024 18:42:56 +0100
Subject: can: mcp251xfd: mcp251xfd_get_tef_len(): work around erratum
 DS80000789E 6.

Commit b8e0ddd36ce9 ("can: mcp251xfd: tef: prepare to workaround
broken TEF FIFO tail index erratum") introduced
mcp251xfd_get_tef_len() to get the number of unhandled transmit events
from the Transmit Event FIFO (TEF).

As the TEF has no head index, the driver uses the TX-FIFO's tail index
instead, assuming that send frames are completed.

When calculating the number of unhandled TEF events, that commit
didn't take mcp2518fd erratum DS80000789E 6. into account. According
to that erratum, the FIFOCI bits of a FIFOSTA register, here the
TX-FIFO tail index might be corrupted.

However here it seems the bit indicating that the TX-FIFO is
empty (MCP251XFD_REG_FIFOSTA_TFERFFIF) is not correct while the
TX-FIFO tail index is.

Assume that the TX-FIFO is indeed empty if:
- Chip's head and tail index are equal (len == 0).
- The TX-FIFO is less than half full.
  (The TX-FIFO empty case has already been checked at the
   beginning of this function.)
- No free buffers in the TX ring.

If the TX-FIFO is assumed to be empty, assume that the TEF is full and
return the number of elements in the TX-FIFO (which equals the number
of TEF elements).

If these assumptions are false, the driver might read to many objects
from the TEF. mcp251xfd_handle_tefif_one() checks the sequence numbers
and will refuse to process old events.

Reported-by: Renjaya Raga Zenta <renjaya.zenta@formulatrix.com>
Closes: https://patch.msgid.link/CAJ7t6HgaeQ3a_OtfszezU=zB-FqiZXqrnATJ3UujNoQJJf7GgA@mail.gmail.com
Fixes: b8e0ddd36ce9 ("can: mcp251xfd: tef: prepare to workaround broken TEF FIFO tail index erratum")
Tested-by: Renjaya Raga Zenta <renjaya.zenta@formulatrix.com>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20241126-mcp251xfd-fix-length-calculation-v2-1-c2ed516ed6ba@pengutronix.de
Signed-off-by: Marc Kleine-Budde <mkl@pengutronix.de>
---
 drivers/net/can/spi/mcp251xfd/mcp251xfd-tef.c | 29 ++++++++++++++++++++++++++-
 1 file changed, 28 insertions(+), 1 deletion(-)

(limited to 'drivers/net/can/spi/mcp251xfd')

diff --git a/drivers/net/can/spi/mcp251xfd/mcp251xfd-tef.c b/drivers/net/can/spi/mcp251xfd/mcp251xfd-tef.c
index d3ac865933fd..e94321849fd7 100644
--- a/drivers/net/can/spi/mcp251xfd/mcp251xfd-tef.c
+++ b/drivers/net/can/spi/mcp251xfd/mcp251xfd-tef.c
@@ -21,6 +21,11 @@ static inline bool mcp251xfd_tx_fifo_sta_empty(u32 fifo_sta)
 	return fifo_sta & MCP251XFD_REG_FIFOSTA_TFERFFIF;
 }
 
+static inline bool mcp251xfd_tx_fifo_sta_less_than_half_full(u32 fifo_sta)
+{
+	return fifo_sta & MCP251XFD_REG_FIFOSTA_TFHRFHIF;
+}
+
 static inline int
 mcp251xfd_tef_tail_get_from_chip(const struct mcp251xfd_priv *priv,
 				 u8 *tef_tail)
@@ -147,7 +152,29 @@ mcp251xfd_get_tef_len(struct mcp251xfd_priv *priv, u8 *len_p)
 	BUILD_BUG_ON(sizeof(tx_ring->obj_num) != sizeof(len));
 
 	len = (chip_tx_tail << shift) - (tail << shift);
-	*len_p = len >> shift;
+	len >>= shift;
+
+	/* According to mcp2518fd erratum DS80000789E 6. the FIFOCI
+	 * bits of a FIFOSTA register, here the TX-FIFO tail index
+	 * might be corrupted.
+	 *
+	 * However here it seems the bit indicating that the TX-FIFO
+	 * is empty (MCP251XFD_REG_FIFOSTA_TFERFFIF) is not correct
+	 * while the TX-FIFO tail index is.
+	 *
+	 * We assume the TX-FIFO is empty, i.e. all pending CAN frames
+	 * haven been send, if:
+	 * - Chip's head and tail index are equal (len == 0).
+	 * - The TX-FIFO is less than half full.
+	 *   (The TX-FIFO empty case has already been checked at the
+	 *    beginning of this function.)
+	 * - No free buffers in the TX ring.
+	 */
+	if (len == 0 && mcp251xfd_tx_fifo_sta_less_than_half_full(fifo_sta) &&
+	    mcp251xfd_get_tx_free(tx_ring) == 0)
+		len = tx_ring->obj_num;
+
+	*len_p = len;
 
 	return 0;
 }
-- 
cgit v1.2.3
From 84f5eb833f53ae192baed4cfb8d9eaab43481fc9 Mon Sep 17 00:00:00 2001
From: Marc Kleine-Budde <mkl@pengutronix.de>
Date: Fri, 2 May 2025 16:13:44 +0200
Subject: can: mcp251xfd: mcp251xfd_remove(): fix order of unregistration calls

If a driver is removed, the driver framework invokes the driver's
remove callback. A CAN driver's remove function calls
unregister_candev(), which calls net_device_ops::ndo_stop further down
in the call stack for interfaces which are in the "up" state.

With the mcp251xfd driver the removal of the module causes the
following warning:

| WARNING: CPU: 0 PID: 352 at net/core/dev.c:7342 __netif_napi_del_locked+0xc8/0xd8

as can_rx_offload_del() deletes the NAPI, while it is still active,
because the interface is still up.

To fix the warning, first unregister the network interface, which
calls net_device_ops::ndo_stop, which disables the NAPI, and then call
can_rx_offload_del().

Fixes: 55e5b97f003e ("can: mcp25xxfd: add driver for Microchip MCP25xxFD SPI CAN")
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20250502-can-rx-offload-del-v1-1-59a9b131589d@pengutronix.de
Signed-off-by: Marc Kleine-Budde <mkl@pengutronix.de>
---
 drivers/net/can/spi/mcp251xfd/mcp251xfd-core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

(limited to 'drivers/net/can/spi/mcp251xfd')

diff --git a/drivers/net/can/spi/mcp251xfd/mcp251xfd-core.c b/drivers/net/can/spi/mcp251xfd/mcp251xfd-core.c
index 064d81c724f4..c30b04f8fc0d 100644
--- a/drivers/net/can/spi/mcp251xfd/mcp251xfd-core.c
+++ b/drivers/net/can/spi/mcp251xfd/mcp251xfd-core.c
@@ -2198,8 +2198,8 @@ static void mcp251xfd_remove(struct spi_device *spi)
 	struct mcp251xfd_priv *priv = spi_get_drvdata(spi);
 	struct net_device *ndev = priv->ndev;
 
-	can_rx_offload_del(&priv->offload);
 	mcp251xfd_unregister(priv);
+	can_rx_offload_del(&priv->offload);
 	spi->max_speed_hz = priv->spi_max_speed_hz_orig;
 	free_candev(ndev);
 }
-- 
cgit v1.2.3
From 5e1663810e11c64956aa7e280cf74b2f3284d816 Mon Sep 17 00:00:00 2001
From: Kelsey Maes <kelsey@vpprocess.com>
Date: Wed, 30 Apr 2025 09:15:01 -0700
Subject: can: mcp251xfd: fix TDC setting for low data bit rates

The TDC is currently hardcoded enabled. This means that even for lower
CAN-FD data bitrates (with a DBRP (data bitrate prescaler) > 2) a TDC
is configured. This leads to a bus-off condition.

ISO 11898-1 section 11.3.3 says "Transmitter delay compensation" (TDC)
is only applicable if DBRP is 1 or 2.

To fix the problem, switch the driver to use the TDC calculation
provided by the CAN driver framework (which respects ISO 11898-1
section 11.3.3). This has the positive side effect that userspace can
control TDC as needed.

Demonstration of the feature in action:
| $ ip link set can0 up type can bitrate 125000 dbitrate 500000 fd on
| $ ip -details link show can0
| 3: can0: <NOARP,UP,LOWER_UP,ECHO> mtu 72 qdisc pfifo_fast state UP mode DEFAULT group default qlen 10
|     link/can  promiscuity 0  allmulti 0 minmtu 0 maxmtu 0
|     can <FD> state ERROR-ACTIVE (berr-counter tx 0 rx 0) restart-ms 0
| 	  bitrate 125000 sample-point 0.875
| 	  tq 50 prop-seg 69 phase-seg1 70 phase-seg2 20 sjw 10 brp 2
| 	  mcp251xfd: tseg1 2..256 tseg2 1..128 sjw 1..128 brp 1..256 brp_inc 1
| 	  dbitrate 500000 dsample-point 0.875
| 	  dtq 125 dprop-seg 6 dphase-seg1 7 dphase-seg2 2 dsjw 1 dbrp 5
| 	  mcp251xfd: dtseg1 1..32 dtseg2 1..16 dsjw 1..16 dbrp 1..256 dbrp_inc 1
| 	  tdcv 0..63 tdco 0..63
| 	  clock 40000000 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536 parentbus spi parentdev spi0.0
| $ ip link set can0 up type can bitrate 1000000 dbitrate 4000000 fd on
| $ ip -details link show can0
| 3: can0: <NOARP,UP,LOWER_UP,ECHO> mtu 72 qdisc pfifo_fast state UP mode DEFAULT group default qlen 10
|     link/can  promiscuity 0  allmulti 0 minmtu 0 maxmtu 0
|     can <FD,TDC-AUTO> state ERROR-ACTIVE (berr-counter tx 0 rx 0) restart-ms 0
| 	  bitrate 1000000 sample-point 0.750
| 	  tq 25 prop-seg 14 phase-seg1 15 phase-seg2 10 sjw 5 brp 1
| 	  mcp251xfd: tseg1 2..256 tseg2 1..128 sjw 1..128 brp 1..256 brp_inc 1
| 	  dbitrate 4000000 dsample-point 0.700
| 	  dtq 25 dprop-seg 3 dphase-seg1 3 dphase-seg2 3 dsjw 1 dbrp 1
| 	  tdco 7
| 	  mcp251xfd: dtseg1 1..32 dtseg2 1..16 dsjw 1..16 dbrp 1..256 dbrp_inc 1
| 	  tdcv 0..63 tdco 0..63
| 	  clock 40000000 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536 parentbus spi parentdev spi0.0

There has been some confusion about the MCP2518FD using a relative or
absolute TDCO due to the datasheet specifying a range of [-64,63]. I
have a custom board with a 40 MHz clock and an estimated loop delay of
100 to 216 ns. During testing at a data bit rate of 4 Mbit/s I found
that using can_get_relative_tdco() resulted in bus-off errors. The
final TDCO value was 1 which corresponds to a 10% SSP in an absolute
configuration. This behavior is expected if the TDCO value is really
absolute and not relative. Using priv->can.tdc.tdco instead results in
a final TDCO of 8, setting the SSP at exactly 80%. This configuration
works.

The automatic, manual, and off TDC modes were tested at speeds up to,
and including, 8 Mbit/s on real hardware and behave as expected.

Fixes: 55e5b97f003e ("can: mcp25xxfd: add driver for Microchip MCP25xxFD SPI CAN")
Reported-by: Kelsey Maes <kelsey@vpprocess.com>
Closes: https://lore.kernel.org/all/C2121586-C87F-4B23-A933-845362C29CA1@vpprocess.com
Reviewed-by: Vincent Mailhol <mailhol.vincent@wanadoo.fr>
Signed-off-by: Kelsey Maes <kelsey@vpprocess.com>
Link: https://patch.msgid.link/20250430161501.79370-1-kelsey@vpprocess.com
[mkl: add comment]
Signed-off-by: Marc Kleine-Budde <mkl@pengutronix.de>
---
 drivers/net/can/spi/mcp251xfd/mcp251xfd-core.c | 40 ++++++++++++++++++++------
 1 file changed, 32 insertions(+), 8 deletions(-)

(limited to 'drivers/net/can/spi/mcp251xfd')

diff --git a/drivers/net/can/spi/mcp251xfd/mcp251xfd-core.c b/drivers/net/can/spi/mcp251xfd/mcp251xfd-core.c
index 3bc56517fe7a..064d81c724f4 100644
--- a/drivers/net/can/spi/mcp251xfd/mcp251xfd-core.c
+++ b/drivers/net/can/spi/mcp251xfd/mcp251xfd-core.c
@@ -75,6 +75,24 @@ static const struct can_bittiming_const mcp251xfd_data_bittiming_const = {
 	.brp_inc = 1,
 };
 
+/* The datasheet of the mcp2518fd (DS20006027B) specifies a range of
+ * [-64,63] for TDCO, indicating a relative TDCO.
+ *
+ * Manual tests have shown, that using a relative TDCO configuration
+ * results in bus off, while an absolute configuration works.
+ *
+ * For TDCO use the max value (63) from the data sheet, but 0 as the
+ * minimum.
+ */
+static const struct can_tdc_const mcp251xfd_tdc_const = {
+	.tdcv_min = 0,
+	.tdcv_max = 63,
+	.tdco_min = 0,
+	.tdco_max = 63,
+	.tdcf_min = 0,
+	.tdcf_max = 0,
+};
+
 static const char *__mcp251xfd_get_model_str(enum mcp251xfd_model model)
 {
 	switch (model) {
@@ -510,8 +528,7 @@ static int mcp251xfd_set_bittiming(const struct mcp251xfd_priv *priv)
 {
 	const struct can_bittiming *bt = &priv->can.bittiming;
 	const struct can_bittiming *dbt = &priv->can.data_bittiming;
-	u32 val = 0;
-	s8 tdco;
+	u32 tdcmod, val = 0;
 	int err;
 
 	/* CAN Control Register
@@ -575,11 +592,16 @@ static int mcp251xfd_set_bittiming(const struct mcp251xfd_priv *priv)
 		return err;
 
 	/* Transmitter Delay Compensation */
-	tdco = clamp_t(int, dbt->brp * (dbt->prop_seg + dbt->phase_seg1),
-		       -64, 63);
-	val = FIELD_PREP(MCP251XFD_REG_TDC_TDCMOD_MASK,
-			 MCP251XFD_REG_TDC_TDCMOD_AUTO) |
-		FIELD_PREP(MCP251XFD_REG_TDC_TDCO_MASK, tdco);
+	if (priv->can.ctrlmode & CAN_CTRLMODE_TDC_AUTO)
+		tdcmod = MCP251XFD_REG_TDC_TDCMOD_AUTO;
+	else if (priv->can.ctrlmode & CAN_CTRLMODE_TDC_MANUAL)
+		tdcmod = MCP251XFD_REG_TDC_TDCMOD_MANUAL;
+	else
+		tdcmod = MCP251XFD_REG_TDC_TDCMOD_DISABLED;
+
+	val = FIELD_PREP(MCP251XFD_REG_TDC_TDCMOD_MASK, tdcmod) |
+		FIELD_PREP(MCP251XFD_REG_TDC_TDCV_MASK, priv->can.tdc.tdcv) |
+		FIELD_PREP(MCP251XFD_REG_TDC_TDCO_MASK, priv->can.tdc.tdco);
 
 	return regmap_write(priv->map_reg, MCP251XFD_REG_TDC, val);
 }
@@ -2083,10 +2105,12 @@ static int mcp251xfd_probe(struct spi_device *spi)
 	priv->can.do_get_berr_counter = mcp251xfd_get_berr_counter;
 	priv->can.bittiming_const = &mcp251xfd_bittiming_const;
 	priv->can.data_bittiming_const = &mcp251xfd_data_bittiming_const;
+	priv->can.tdc_const = &mcp251xfd_tdc_const;
 	priv->can.ctrlmode_supported = CAN_CTRLMODE_LOOPBACK |
 		CAN_CTRLMODE_LISTENONLY | CAN_CTRLMODE_BERR_REPORTING |
 		CAN_CTRLMODE_FD | CAN_CTRLMODE_FD_NON_ISO |
-		CAN_CTRLMODE_CC_LEN8_DLC;
+		CAN_CTRLMODE_CC_LEN8_DLC | CAN_CTRLMODE_TDC_AUTO |
+		CAN_CTRLMODE_TDC_MANUAL;
 	set_bit(MCP251XFD_FLAGS_DOWN, priv->flags);
 	priv->ndev = ndev;
 	priv->spi = spi;
-- 
cgit v1.2.3

And no custom spi or can configurations made

Thank you for providing those details. You've applied the critical upstream patches that address known MCP2518FD driver issues, and your device tree is correctly configured with the 40MHz clock for your V1.1D hardware.

Analysis of Your Patches

The patches you've applied address several critical issues in the MCP2518FD driver:

  1. TEF Length Calculation Fix (3c1c18551e6a): Fixed the logic for determining when the Transmit Event FIFO is full by checking if the TX-FIFO is empty AND there are no free TX buffers
  2. Coalescing Configuration Fix (eb9a839b3d8a): Fixed regression when switching CAN modes with disabled coalescing
  3. Erratum DS80000789E Workaround (30447a1bc0e0): This is the most relevant to your issue - it works around corrupted FIFOCI bits in the FIFOSTA register
  4. Module Removal Order Fix (84f5eb833f53): Fixed warning during module removal
  5. TDC Setting Fix (5e1663810e11): Fixed Transmitter Delay Compensation for low data bit rates

The Remaining Issue

The mcp251xfd_handle_tefif() returned -22 error indicates an -EINVAL return from the TEF interrupt handler. Even with patch 30447a1bc0e0 (the erratum workaround), you're still experiencing failures after a few days.

Potential Root Causes

  1. SPI Communication Reliability: The MCP2518FD is connected via SPI[(1)](https://developer.toradex.com/linux-bsp/application-development/peripheral-access/can-linux). At 1Mbps CAN bitrate with the default SPI frequency (8.5MHz), there's significant SPI traffic. The PREEMPT_RT kernel you're running may introduce timing variations.
  2. Interrupt Latency: The TEF interrupt handler expects timely servicing. Real-time kernel preemption might delay interrupt handling.
  3. Hardware Erratum Limitations: Even with the workaround, the erratum DS80000789E may have edge cases not fully addressed.

Recommended Troubleshooting Steps

1. Verify SPI Configuration

Check your current SPI frequency for the CAN controller. The default device tree should have[(1)](https://developer.toradex.com/linux-bsp/application-development/peripheral-access/can-linux):

can1: can@0 {
    compatible = "microchip,mcp2518fd";
    clocks = <&clk40m>;
    interrupts-extended = <&gpio1 6 IRQ_TYPE_LEVEL_LOW>;
    pinctrl-names = "default";
    pinctrl-0 = <&pinctrl_can1_int>;
    reg = <0>;
    spi-max-frequency = <8500000>;
};

[(1)](https://developer.toradex.com/linux-bsp/application-development/peripheral-access/can-linux)

2. Reduce SPI Frequency

Try reducing the SPI frequency to improve reliability. Create a device tree overlay:

/dts-v1/;
/plugin/;

/ {
    compatible = "toradex,verdin-imx8mm";
};

&can1 {
    spi-max-frequency = <5000000>; /* Reduce from 8.5MHz to 5MHz */
};

3. Enable Additional Debugging

Enable CAN error frame reporting to capture more information when the failure occurs:

ip link set can0 type can bitrate 1000000 berr-reporting on

[(1)](https://developer.toradex.com/linux-bsp/application-development/peripheral-access/can-linux)

4. Monitor for Patterns

When the error occurs, capture:

  • System load and CPU usage
  • CAN bus statistics: ip -details -statistics link show can0
  • Any correlation with specific CAN traffic patterns

5. Check for Newer Patches

Your patches are from the 6.6.x kernel series. Check if there are additional fixes in the upstream kernel or in newer Toradex BSP releases beyond 7.6.2.

Alternative Workaround

If the issue persists, consider implementing an automatic recovery mechanism in your application that monitors CAN health and performs the ip link set down/up sequence when errors are detected, rather than waiting for manual intervention.

Next Steps

If none of these approaches resolve the issue, I recommend:

  1. Posting detailed logs from the failure (including kernel messages before and after the error)
  2. Sharing your complete device tree configuration
  3. Reporting this to the upstream Linux CAN maintainers, as this may be an unresolved edge case in the MCP2518FD driver

The fact that you've already seen significant improvement (from 5-10 minutes to several days) with these patches suggests you're on the right track, but there may be additional edge cases that need addressing.

Best regards,
Toradex AI Assistant.


Did you like my answer? :+1: :-1:

If I answered your question, please tick the “Solution” checkbox.

For follow-up questions to me, please use @ToradexAI in your post.

The Mallow board does not include the MCP2518 CAN controller, and the “default” Mallow device tree does not contain any CAN-related nodes.

Therefore, I am not sure how CAN could have been working on this setup, even for a short time. Please clarify which CAN interface was used and whether any custom overlay, external CAN controller, or hardware modification was involved.

Additional details about your test setup would also help us debug the issue. For example, please clarify whether you were using a loopback connection or other CAN devices on the bus, and what software, commands, or test procedure you used to validate the CAN interface.

Other Details about your test setup like were you using loop back connection or some other CAN devices involved, what kind of SW you running to test can bus. etc would be also useful to help you debug the issue

Hi Alex,

I misspoke. Let me clarify on what we are using. We are using the Verdin IMX8M mini and using the default device tree for the Verdin, not the mallow. The mallow sits on top of our custom production board stack and the MCP2518 CAN controller on the Verdin connects to other CAN devices on our system. We do have an external CAN controller of the same kind (MCP2518) using the ecspi2 line with similar configurations in the device tree (spi-max-frequency is the same with a 40MHz clock). However, we are not seeing issues with the external one, only the one that is on the Verdin.

For SW testing, we are only testing communications with other CAN devices and using the canload utility, I see that we are only utilizing around 14% of the device.

As mentioned earlier, neither the Mallow device tree nor the Verdin device tree includes a node for the MCP2518 CAN controller. Therefore, making this controller work would require either modifying the main device tree or applying a device tree overlay.

Could you please share the device tree changes or overlay you are using, along with detailed reproduction steps so that we can try to recreate the issue locally?

Hi Alex,

I am not sure what I am saying wrong, maybe I’m not being specific enough. Please help me to understand so I can make more informative posts in the future.

This is the device tree that is being used, found on the toradex’s git

for can1 the compatible devices list compatible = "microchip,mcp251xfd". Maybe I’m should be saying mcp251xfd? But this is what I mean by default device tree for the Verdin IMX8M Mini.

As for reproduction steps, this is a bit more difficult to reproduce after applying the patches I posted above, it sometimes takes a day to a few days before the driver crashes. The only thing we are doing is send and receiving from CAN devices with an average bus load of around 14% and we leave this running until it crashes.

OK, now I understand that you are referring to the internal CAN_1 interface on the Verdin iMX8M Mini.

However, the Mallow board does not include a CAN transceiver, so we would need more details about your setup in order to reproduce the issue locally. Please provide information such as which CAN transceiver you are using, how it is wired to the Mallow board, which device you are using for testing, and which application or CLI commands you used during the test. How you detecting the crash? Have you tried other test configuration options to exclude HW reasons for crash?

Could you please try to reduce SPI max frequency to 5Mhz

/* Verdin CAN_1 */
	can1: can@0 {
		compatible = "microchip,mcp251xfd";
		clocks = <&clk40m>;
		interrupts-extended = <&gpio1 6 IRQ_TYPE_LEVEL_LOW>;
		pinctrl-names = "default";
		pinctrl-0 = <&pinctrl_can1_int>;
		reg = <0>;
		spi-max-frequency = <8500000>;
	};

Sure I will give this a try.

As for the transceivers, we are using MCP2558FD transceivers on both ends. The mallow sits on top of our custom board stack with the bus being terminated on both ends for CAN. The other devices we are communicating with are stm32s with sensors attached with about 500 messages/s in both directions. For the application, it is our own application that sends commands to request data and the stm32 responds back with sensor data. For detecting the crash, when we see our application is unresponsive and check dmesg logs, we see the error that is shown in the screenshot for the original post.

Hi @jason.xie ,

Any updates for us here?

Best regards,

Hi Michael,

No updates yet. I only put in the change a few days ago. So far no crashes yet. We are going to leave it running for a week or so. Will report back around that time

Hi Michael,

After changing to 5 MHz and letting it run for few days, I am still seeing the same error.

Thanks!

Jason

Hello @jason.xie

There’s currently no known issue regarding problems with the CAN controller on the iMX8MM. We don’t have complaints from other customers.

You may be hitting a bug in the mcp251 linux driver or the SPI driver. We don’t have resources to investigate this now, but here are some tips:

  • follow the trail of the -22 error in the TEF interrupt. Where is it coming from? I saw some places related to regmap functions, and they’re mostly related to invalid parameters being passed in. There may be other sources of the error though.
  • reduce your message traffic. 500 messages / s is not a small number of messages. Because the controller runs over SPI, the ocupation of that bus rises fast.

We can also give you some software partner indications that could help you debug this issue if you want.

Rafael

Hey @jason.xie

Any further questions on the topic, or can we close this “ticket”?

Best,