DeviceTree eeprom entry not binding driver

Me again :slight_smile: Thanks for all the support so far, I’ve encountered another issue where I can’t seem to get things to behave as expected.

Preface: I have wired an M24256 (ST 24c256) to a suitable bus (i2c3 on my Verdin development board - other i2c busses already have EEPROMs on them, this bus has no devices as indicated by i2c-detect).

I am able to get the eeprom to work and appear properly in SysFS manually, e.g. echo 24c256 0x50 > /sys/bus/i2c/devices/i2c-2/new_device will create a 2-0050 device entry containing an eeprom file which can be read and written to access the EEPROM.

However, I am trying to get this to work automatically via devicetree configuration. I know it should be possible because there are several 24c02 EEPROM nodes already present in the Verdin tree.

No matter what configuration values I try, the system is not correctly binding the AT24 driver to this device. Here is my DTS file being applied by torizoncore-builder:

// M24256 EEPROM (Motherboard) DTS. 

/dts-v1/;
/plugin/;

// Note this will need to be adjusted according to 
// the target I2C bus. 
// I2C4 is I2C_1 (pins 12/14) on the Verdin dev board.
// I2C3 is I2C_2 (pins 95/93) on the Verdin dev board.
// Chip can appear at 0x50-57 depending on pin strapping, 
// and 0x58 is the identification page.
&i2c3 {
	m24256: at24@50 {
		compatible = "st,24c256", "atmel,24c256";
		pagesize = <64>;
		size = <32768>;
		reg = <0x50>;
		status = "okay";
	};
};

On reboot, there is now a node at /sys/bus/i2c/devices/2-0050/ (this is missing without the DTS being included) but it is not bound to any driver and no eeprom file is present. dmesg | grep at24 only reveals entries for the 24c02 eeproms have been registered, not my 24c256. i2c-detect 2 confirms it’s not bound either - shows 50 instead of UU

The only way I can get it to bind and start working as expected is echo 2-0050 > /sys/bus/i2c/drivers/at24/bind to manually force the driver to bind this address.

Note I have tried a number of variations of the compatible line, none of which seem to have had any success.

Feels like I’m missing something obvious but I’m at a loss and the search terms I’ve tried using to find answers are too generic to net anything suitable to my issue.

Is there some magic that happens behind the scenes to make the various Toradex 24c02s “just work”? (Carrier board, SoM, DSI adapter all automatically bind the correct driver and have the SysFS eeprom file)

As an interesting (but aside) data point we have also connected a 1-wire EEPROM to the development board via a bridge chip and it worked without any fuss as soon as I added the DTS entries.

Hi @bw908,

Searching for drivers and docs, your node appears to be correct.

I found this eeprom.rst under Documentation/misc-devices/eeprom.rst:

Lacking functionality
---------------------

* Full support for larger devices (24C04, 24C08, 24C16). These are not
  typically found on a PC. These devices will appear as separate devices at
  multiple addresses.

* Support for really large devices (24C32, 24C64, 24C128, 24C256, 24C512).
  These devices require two-byte address fields and are not supported.

* Enable Writing. Again, no technical reason why not, but making it easy
  to change the contents of the EEPROMs (on DIMMs anyway) also makes it easy
  to disable the DIMMs (potentially preventing the computer from booting)
  until the values are restored somehow.

And it says the driver doesn’t support the 24C256. However, as you said it works with the sysfs (have you checked if you can write or read something?), I believe this is only a not updated documentation. Also, I found the at24.txt which mentions your device:

Required properties:

  - compatible: Must be a "<manufacturer>,<model>" pair. The following <model>
                values are supported (assuming "atmel" as manufacturer):

                "atmel,24c00",
                "atmel,24c01",
                "atmel,24cs01",
                "atmel,24c02",
                "atmel,24cs02",
                "atmel,24mac402",
                "atmel,24mac602",
                "atmel,spd",
                "atmel,24c04",
                "atmel,24cs04",
                "atmel,24c08",
                "atmel,24cs08",
                "atmel,24c16",
                "atmel,24cs16",
                "atmel,24c32",
                "atmel,24cs32",
                "atmel,24c64",
                "atmel,24cs64",
                "atmel,24c128",
                "atmel,24c256",
                "atmel,24c512",
                "atmel,24c1024",
                "atmel,24c2048",

This driver is already enabled in our kernel and it should work “out of the box”.

How are you compiling the overlay? Have you checked if it’s being applied? You can check in the boot process or the overlays.txt file.

Best Regards,
Hiago.

Yes, I can read/write the eeprom file with data, and when I do either the bind or manual setup through new_device, dmesg will show a message from the at24 driver showing that the device is bound correctly.

I also think the documentation seems to be out of date because searching for devicetree 24c256 yeilds results from other folks that had it “just work” for them :thinking: (and there is support for writing, which is also old information)

I am adding the DTS in the same way I am applying my other customizations, namely:

      add:
        - [other entries]
        - dts/m24256.dts

in tcbuild.yaml.

The output of torizoncore-builder indicates it is successfully applied.

Hi @bw908,

Have you checked if there is some kind of conflict between the addresses of the EEPROMs (0x50)?

The eeprom in the imx8mm-verdin.dtsi defines:

	eeprom@50 {
		compatible = "st,24c02";
		pagesize = <16>;
		reg = <0x50>;
	};

And checking with my verdin plus with a fresh installation of TorizonCore, I can see the 0x50 is reserved:

root@73a01951c9eb:~# i2cdetect -y 3
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:                         -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- UU -- -- -- -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- 2c -- -- --
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
40: UU -- -- -- -- -- -- -- -- -- -- -- -- -- -- UU
50: UU -- -- -- -- -- -- UU -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
70: -- -- -- -- -- -- -- --

I checked i2c-3 because I saw you are trying to use i2c3 as well.

Best Regards,
Hiago.

I am using i2c-2 (from the OS perspective) where this address is free. (this corresponds to i2c3 in the device tree)

That’s actually a minor beef of mine in that the naming convention was very poorly chosen for these interfaces, with different numbering depending whether you are looking at the evaluation board, SoM, in the OS, or the devicetree entries… clearly I’m not the only one that gets tripped up by this :rofl:

Hi @bw908,

Just to be sure, you are trying to use i2c3 SODIMM 95 and 93, correct?

	pinctrl_i2c3: i2c3grp {
		fsl,pins = <
			MX8MM_IOMUXC_I2C3_SCL_I2C3_SCL			0x40000146	/* SODIMM 95 */
			MX8MM_IOMUXC_I2C3_SDA_I2C3_SDA			0x40000146	/* SODIMM 93 */
		>;
	};

Which is i2c2 from the software perspective, correct?

Sorry about the error. You are correct, the numbers can get confusing sometimes.

Best Regards,
Hiago.

Yes, that’s correct.

Hi @bw908,

Thanks for confirming.

I made a test here applying only this overlay:

/dts-v1/;
/plugin/;

/ {
	compatible = "toradex,verdin-imx8mp";
};

&i2c3 {
    eeprom_256: eeprom@50 {
		compatible = "st,24c256", "atmel,24c256";
		pagesize = <64>;
		reg = <0x50>;
        size = <32768>;
		status = "okay";
    };
};

You can download it here if needed: Download - Toradex File Sharing Platform

Next, I added this overlay and removed the default ones:

torizon@verdin-imx8mp-07174398:~$ cat /boot/ostree/torizon-f7b681e3828a3014c7fa4c1e407cc9729f43b39f687e69ca6efa6db8796d9ac1/dtb/overlays.txt
fdt_overlays=verdin-imx8mp_24c256.dtbo

I don’t have the EEPROM here to test, so I can’t see it on the dmesg:

torizon@verdin-imx8mp-07174398:~$ dmesg | grep -i at24
[    1.402152] at24 0-0050: 256 byte 24c02 EEPROM, writable, 16 bytes/write
[    1.432673] at24 3-0050: 256 byte 24c02 EEPROM, writable, 16 bytes/write
[    1.433805] at24 3-0057: 256 byte 24c02 EEPROM, writable, 16 bytes/write

However, I can see that it’s assigned on the kernel bus:

torizon@verdin-imx8mp-07174398:~$ ls -lh /sys/bus/i2c/devices/
total 0
lrwxrwxrwx 1 root root 0 Nov 23 20:19 0-0025 -> ../../../devices/platform/soc@0/30800000.bus/30a20000.i2c/i2c-0/0-0025
lrwxrwxrwx 1 root root 0 Nov 23 20:19 0-0032 -> ../../../devices/platform/soc@0/30800000.bus/30a20000.i2c/i2c-0/0-0032
lrwxrwxrwx 1 root root 0 Nov 23 20:19 0-0048 -> ../../../devices/platform/soc@0/30800000.bus/30a20000.i2c/i2c-0/0-0048
lrwxrwxrwx 1 root root 0 Nov 23 20:19 0-0049 -> ../../../devices/platform/soc@0/30800000.bus/30a20000.i2c/i2c-0/0-0049
lrwxrwxrwx 1 root root 0 Nov 23 20:19 0-0050 -> ../../../devices/platform/soc@0/30800000.bus/30a20000.i2c/i2c-0/0-0050
lrwxrwxrwx 1 root root 0 Nov 23 20:19 2-0050 -> ../../../devices/platform/soc@0/30800000.bus/30a40000.i2c/i2c-2/2-0050
lrwxrwxrwx 1 root root 0 Nov 23 20:19 3-001a -> ../../../devices/platform/soc@0/30800000.bus/30a50000.i2c/i2c-3/3-001a
lrwxrwxrwx 1 root root 0 Nov 23 20:19 3-0040 -> ../../../devices/platform/soc@0/30800000.bus/30a50000.i2c/i2c-3/3-0040
lrwxrwxrwx 1 root root 0 Nov 23 20:19 3-004f -> ../../../devices/platform/soc@0/30800000.bus/30a50000.i2c/i2c-3/3-004f
lrwxrwxrwx 1 root root 0 Nov 23 20:19 3-0050 -> ../../../devices/platform/soc@0/30800000.bus/30a50000.i2c/i2c-3/3-0050
lrwxrwxrwx 1 root root 0 Nov 23 20:19 3-0057 -> ../../../devices/platform/soc@0/30800000.bus/30a50000.i2c/i2c-3/3-0057
lrwxrwxrwx 1 root root 0 Nov 23 20:19 i2c-0 -> ../../../devices/platform/soc@0/30800000.bus/30a20000.i2c/i2c-0
lrwxrwxrwx 1 root root 0 Nov 23 20:19 i2c-1 -> ../../../devices/platform/soc@0/30800000.bus/30a30000.i2c/i2c-1
lrwxrwxrwx 1 root root 0 Nov 23 20:19 i2c-2 -> ../../../devices/platform/soc@0/30800000.bus/30a40000.i2c/i2c-2
lrwxrwxrwx 1 root root 0 Nov 23 20:19 i2c-3 -> ../../../devices/platform/soc@0/30800000.bus/30a50000.i2c/i2c-3

With this line:

lrwxrwxrwx 1 root root 0 Nov 23 20:19 2-0050 -> ../../../devices/platform/soc@0/30800000.bus/30a40000.i2c/i2c-2/2-0050

So I believe it was added somehow. Can you please test this overlay? Remember to activate only this overlay, so we can remove other conflicts that might be happening. You can check the overlays.txt file.

Best Regards,
Hiago.

That’s the thing - with the overlay, I can see the device is being bound on the kernel bus (2-0050 exists immediately on boot; it does not without the overlay applied). Effectively, it’s in the same state as you are seeing without the “real” hardware attached.

On my system the at24 driver is not being bound to it (udevadm test /sys/bus/i2c/devices/2-0050) shows that it has no driver and is just a generic “SPI device” with no eeprom specialization. (whereas udevadm test on e.g. 3-0050 shows it has been assigned the at24 driver)

Unfortunately I’m not sure this is something that can be debugged/reproduced without a physical eeprom attached to the development board. The closest you might be able to get to seeing this situation is e.g. editing the "compatible = “Atmel,24c02” value for one of the other eeproms to something that isn’t recognized as a valid EEPROM name. (though as you’ve also verified, atmel, 24c256 should be recognized)

Hi @bw908,

Which kernel version are you using? Could you test the same thing on 5.15 BSP 6?

Best Regards,
Hiago.

We are currently working with ...verdin-imx8mp-Tezi_5.6.0+build.13.tar - don’t have hardware at the moment to confirm the exact kernel version in use.

Hi @bw908,

This image uses kernel 5.4 downstream.

I would like to recommend you enable the FTRACE support to debug the i2c bus inside your image. Here is how to do it:

  1. Activate the FTRACE and DYNAMIC_FTRACE.

Here is the config diff from the default configuration:

--- .config	2022-11-28 17:14:21.517680962 -0300
+++ /home/hiago/defconfig	2022-11-28 17:10:21.400355628 -0300
@@ -231,6 +231,7 @@
 CONFIG_SLUB_CPU_PARTIAL=y
 CONFIG_SYSTEM_DATA_VERIFICATION=y
 CONFIG_PROFILING=y
+CONFIG_TRACEPOINTS=y
 # end of General setup
 
 CONFIG_ARM64=y
@@ -660,6 +661,7 @@
 # CONFIG_KPROBES is not set
 CONFIG_JUMP_LABEL=y
 # CONFIG_STATIC_KEYS_SELFTEST is not set
+CONFIG_UPROBES=y
 CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS=y
 CONFIG_HAVE_KPROBES=y
 CONFIG_HAVE_KRETPROBES=y
@@ -1234,6 +1236,7 @@
 # Network testing
 #
 CONFIG_NET_PKTGEN=y
+# CONFIG_NET_DROP_MONITOR is not set
 # end of Network testing
 # end of Networking options
 
@@ -6717,6 +6720,7 @@
 CONFIG_PSTORE_COMPRESS_DEFAULT="deflate"
 # CONFIG_PSTORE_CONSOLE is not set
 # CONFIG_PSTORE_PMSG is not set
+# CONFIG_PSTORE_FTRACE is not set
 # CONFIG_PSTORE_RAM is not set
 # CONFIG_SYSV_FS is not set
 # CONFIG_UFS_FS is not set
@@ -7088,6 +7092,8 @@
 # CONFIG_SYSTEM_BLACKLIST_KEYRING is not set
 # end of Certificates for signature checking
 
+CONFIG_BINARY_PRINTF=y
+
 #
 # Library routines
 #
@@ -7262,6 +7268,7 @@
 # CONFIG_DEBUG_PAGEALLOC is not set
 # CONFIG_PAGE_OWNER is not set
 # CONFIG_PAGE_POISONING is not set
+# CONFIG_DEBUG_PAGE_REF is not set
 # CONFIG_DEBUG_RODATA_TEST is not set
 # CONFIG_DEBUG_OBJECTS is not set
 # CONFIG_SLUB_DEBUG_ON is not set
@@ -7322,7 +7329,7 @@
 # CONFIG_WW_MUTEX_SELFTEST is not set
 # end of Lock Debugging (spinlocks, mutexes, etc...)
 
-# CONFIG_STACKTRACE is not set
+CONFIG_STACKTRACE=y
 # CONFIG_WARN_ALL_UNSEEDED_RANDOM is not set
 # CONFIG_DEBUG_KOBJECT is not set
 CONFIG_HAVE_DEBUG_BUGVERBOSE=y
@@ -7349,14 +7356,49 @@
 # CONFIG_NOTIFIER_ERROR_INJECTION is not set
 # CONFIG_FAULT_INJECTION is not set
 # CONFIG_LATENCYTOP is not set
+CONFIG_NOP_TRACER=y
 CONFIG_HAVE_FUNCTION_TRACER=y
 CONFIG_HAVE_FUNCTION_GRAPH_TRACER=y
 CONFIG_HAVE_DYNAMIC_FTRACE=y
 CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
 CONFIG_HAVE_SYSCALL_TRACEPOINTS=y
 CONFIG_HAVE_C_RECORDMCOUNT=y
+CONFIG_TRACE_CLOCK=y
+CONFIG_RING_BUFFER=y
+CONFIG_EVENT_TRACING=y
+CONFIG_CONTEXT_SWITCH_TRACER=y
+CONFIG_TRACING=y
+CONFIG_GENERIC_TRACER=y
 CONFIG_TRACING_SUPPORT=y
-# CONFIG_FTRACE is not set
+CONFIG_FTRACE=y
+CONFIG_FUNCTION_TRACER=y
+CONFIG_FUNCTION_GRAPH_TRACER=y
+# CONFIG_PREEMPTIRQ_EVENTS is not set
+# CONFIG_IRQSOFF_TRACER is not set
+# CONFIG_PREEMPT_TRACER is not set
+# CONFIG_SCHED_TRACER is not set
+# CONFIG_HWLAT_TRACER is not set
+# CONFIG_FTRACE_SYSCALLS is not set
+# CONFIG_TRACER_SNAPSHOT is not set
+CONFIG_BRANCH_PROFILE_NONE=y
+# CONFIG_PROFILE_ANNOTATED_BRANCHES is not set
+# CONFIG_PROFILE_ALL_BRANCHES is not set
+# CONFIG_STACK_TRACER is not set
+# CONFIG_BLK_DEV_IO_TRACE is not set
+CONFIG_UPROBE_EVENTS=y
+CONFIG_BPF_EVENTS=y
+CONFIG_DYNAMIC_EVENTS=y
+CONFIG_PROBE_EVENTS=y
+CONFIG_DYNAMIC_FTRACE=y
+# CONFIG_FUNCTION_PROFILER is not set
+CONFIG_FTRACE_MCOUNT_RECORD=y
+# CONFIG_FTRACE_STARTUP_TEST is not set
+# CONFIG_HIST_TRIGGERS is not set
+# CONFIG_TRACEPOINT_BENCHMARK is not set
+# CONFIG_RING_BUFFER_BENCHMARK is not set
+# CONFIG_RING_BUFFER_STARTUP_TEST is not set
+# CONFIG_PREEMPTIRQ_DELAY_TEST is not set
+# CONFIG_TRACE_EVAL_MAP_FILE is not set
 CONFIG_RUNTIME_TESTING_MENU=y
 # CONFIG_LKDTM is not set
 # CONFIG_TEST_LIST_SORT is not set
  1. Copy the new kernel image `linux-toradex/arch/arm64/boot/Image.gz´ to your image and boot the module with the new kernel.

This can be done with Yocto as well. Here are some instructions on how to do it:

  1. Enable the I2C debug inside your module (there is no need to enable debugfs here since it’s already enabled):
# echo nop > /sys/kernel/debug/tracing/current_tracer
# echo 1 > /sys/kernel/debug/tracing/events/i2c/enable
# echo 1 > /sys/kernel/debug/tracing/tracing_on
  1. Finally you can try to connect to your eeprom and, after that, check the trace file:
root@verdin-imx8mp-07174398:~# cat /sys/kernel/debug/tracing/trace
# tracer: nop
#
# entries-in-buffer/entries-written: 24/24   #P:4
#
#                                _-----=> irqs-off
#                               / _----=> need-resched
#                              | / _---=> hardirq/softirq
#                              || / _--=> preempt-depth
#                              ||| /     delay
#           TASK-PID     CPU#  ||||   TIMESTAMP  FUNCTION
#              | |         |   ||||      |         |
    kworker/2:1H-235     [002] ....   133.933778: i2c_write: i2c-0 #0 a=025 f=0000 l=1 [1c]
    kworker/2:1H-235     [002] ....   133.933784: i2c_read: i2c-0 #1 a=025 f=0001 l=1
    kworker/2:1H-235     [002] ....   133.935303: i2c_reply: i2c-0 #1 a=025 f=0001 l=1 [30]
    kworker/2:1H-235     [002] ....   133.935306: i2c_result: i2c-0 n=2 ret=2
    kworker/2:1H-235     [002] ....   142.893769: i2c_write: i2c-0 #0 a=025 f=0000 l=1 [1c]
    kworker/2:1H-235     [002] ....   142.893775: i2c_read: i2c-0 #1 a=025 f=0001 l=1
    kworker/2:1H-235     [002] ....   142.895297: i2c_reply: i2c-0 #1 a=025 f=0001 l=1 [30]
    kworker/2:1H-235     [002] ....   142.895300: i2c_result: i2c-0 n=2 ret=2
    kworker/2:1H-235     [002] ....   149.038357: i2c_write: i2c-0 #0 a=025 f=0000 l=1 [1c]
    kworker/2:1H-235     [002] ....   149.038363: i2c_read: i2c-0 #1 a=025 f=0001 l=1
    kworker/2:1H-235     [002] ....   149.041108: i2c_reply: i2c-0 #1 a=025 f=0001 l=1 [30]
    kworker/2:1H-235     [002] ....   149.041110: i2c_result: i2c-0 n=2 ret=2
    kworker/0:3H-515     [000] ....   158.766032: i2c_write: i2c-0 #0 a=025 f=0000 l=1 [1c]
    kworker/0:3H-515     [000] ....   158.766038: i2c_read: i2c-0 #1 a=025 f=0001 l=1
    kworker/0:3H-515     [000] ....   158.767255: i2c_reply: i2c-0 #1 a=025 f=0001 l=1 [30]
    kworker/0:3H-515     [000] ....   158.767257: i2c_result: i2c-0 n=2 ret=2
    kworker/2:1H-235     [002] ....   162.094046: i2c_write: i2c-0 #0 a=025 f=0000 l=1 [1c]
    kworker/2:1H-235     [002] ....   162.094051: i2c_read: i2c-0 #1 a=025 f=0001 l=1
    kworker/2:1H-235     [002] ....   162.095572: i2c_reply: i2c-0 #1 a=025 f=0001 l=1 [30]
    kworker/2:1H-235     [002] ....   162.095574: i2c_result: i2c-0 n=2 ret=2
    kworker/2:1H-235     [002] ....   172.078067: i2c_write: i2c-0 #0 a=025 f=0000 l=1 [1c]
    kworker/2:1H-235     [002] ....   172.078073: i2c_read: i2c-0 #1 a=025 f=0001 l=1
    kworker/2:1H-235     [002] ....   172.079590: i2c_reply: i2c-0 #1 a=025 f=0001 l=1 [30]
    kworker/2:1H-235     [002] ....   172.079593: i2c_result: i2c-0 n=2 ret=2

Please, check if this will help you find the driver issue.

Best Regards,
Hiago.

Thanks. I will try both the newer kernel/BSP release and if that doesn’t help I will try out the kernel debugging you suggest.

I think I’ve found the issue - I had several items wired to the I2C bus for development purposes using a breadboard. the EEPROM works just fine if it’s the only device on the bus, but has problems if I add an additional one. This leads me to believe it’s just related to this prototype wiring scheme which causes the eeprom to not be detected correctly (hence the driver does not bind it).

1 Like

Hi @bw908,

good to know you find what might be causing this issue!

I would recommend you use the I2C kernel debug features and, if possible, check the connections with an oscilloscope. This might show you more information about what is happening.

Let me know if you need extra help with the driver itself or kernel related info.

Best Regards,
Hiago.