Dear All,
I sometimes get the error: 'Fatal: [Errno 2] No such file or directory: '/dev/verdin-adc1'
If I reboot the device, the device works fine. I have mounted the /dev folder and am reading out the ADCs correctly, since the error is not present each time I reboot the device.
I believe that sometimes the board starts up, but does not map the ADC register to the corresponding files. Does anyone know how to overcome this problem?
Kind regards,
Anouk
Greetings @xAnouk,
First of all what version of TorizonCore are you running here? Also what module is this, Verdin i.MX8M Mini or Plus?
I sometimes get the error: 'Fatal:
[Errno 2] No such file or directory: '/dev/verdin-adc1'
Regarding this error is it actually the case that there is no /dev/verdin-adc1 or is this error message misleading and this does exist? Also how often would you say this happens?
With regards to your application/container are you manually launching it after the device has booted? Or is it automatically launching on boot?
Best Regards,
Jeremias
Further to our thread here: ADC read errors on Verdin 1.1b - Technical Support - Toradex Community
Which @RoccoBr mentioned.
I recently showed on a device in the field that it is the driver which has stopped. You can fix the problem by reloading it:
I removed the adc driver with
sudo rmmod ti_ads1015
Then reloaded it with
sudo insmod /lib/modules/5.4.193-5.7.0+git.f78299297185/kernel/drivers/iio/adc/ti-ads1015.ko
And it then works again.
@RoccoBr is working on this with @drew.tx. My view is that the driver version is ancient with numerous fixes now in place. They are working on updating the driver and so far that testing looks good.
Dear Jeremias,
We are using the Verdin IMX8M Mini.
VERSION="6.2.0+build.2 (kirkstone)"
VERSION_ID=6.2.0-build.2
PRETTY_NAME="TorizonCore 6.2.0+build.2 (kirkstone)"
DISTRO_CODENAME="kirkstone"
BUILD_ID="2"
ANSI_COLOR="1;34"
VARIANT="Docker"
I am asking this for a customer of ours, so I will ask whether the /dev/verdin-adc1 file does not exist or that it is a misleading error and I will verify the TorizonCore version.
The container is started automatically when the device is booted. We are using the application in a docker container.
Kind regards,
Anouk
Hi @xAnouk in addition to the info @jeremias.tx asked for, can you see if the steps mentioned by @edwaugh get you past the issue? We definitely want to know if they are the same issue.
Unfortunately, we have been unable to reproduce this on our carrier boards so any ideas you have regarding reproduction of this issue would be most helpful.
Drew
The container is started automatically when the device is booted. We are using the application in a docker container.
This gives me a bit of idea though it’s a bit of a guess. So the file descriptor /dev/verdin-adc1 is actually a symlink for the true file descriptor for adc as seen here:
root@verdin-imx8mm-06827778:~# ls -l /dev/verdin-adc1
lrwxrwxrwx 1 root root 94 Apr 28 2022 /dev/verdin-adc1 -> /sys/devices/platform/soc@0/30800000.bus/30a20000.i2c/i2c-0/0-0049/iio:device0/in_voltage3_raw
These symlinks get setup by udev via some udev rules and scripts: toradex-adc.sh « verdin-imx8mm « files « udev « recipes-core - meta-toradex-bsp-common.git - Toradex BSP layer, recipes common to all modules
Perhaps the issue is the symlink? Maybe the symlink isn’t getting created in some cases, or maybe your application is trying to access the symlink before it’s created since your container runs on boot you said. Again these are more just theories if anything.
That said, maybe another suggestion is try accessing the true file descriptor rather than the symlink and perhaps that will end up being more stable.
Best Regards,
Jeremias
Dear Drew and Jeremias,
I has been a while, but the problem occured again.
The ADC devices are not visible under the /dev directory, but the true descriptor is also not available.
See below:
I also have the dmesg and journalctl log attached if it helps.
journalctl.log (259.5 KB)
dmesg.log (37.1 KB)
Kind regards,
Anouk
The ADC devices are not visible under the /dev directory, but the true descriptor is also not available.
I see, so the original error message is indeed accurate then. Just to check you didn’t make any device tree changes or customizations that might affect the ADC?
On another point, the ADC here is connected via i2c. Could you check if the ADC device is even being detected on i2c when this issue occurs. For example you should see something like this:
torizon@verdin-imx8mm-06827778:~$ i2cdetect -y -r 0
0 1 2 3 4 5 6 7 8 9 a b c d e f
00: -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- UU -- -- -- -- -- -- -- -- -- --
30: -- -- UU -- -- -- -- -- -- -- -- -- -- -- -- --
40: -- -- -- -- -- -- -- -- -- UU -- -- -- -- -- --
50: UU -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
70: -- -- -- -- -- -- -- --
So the ADC device here has a i2c address of 49. We can see via i2cdetect that there is a device detected at address 49. The UU indicates that this device is being used by some kernel driver, which is expected here.
I’m curious if the issue is that i2c fails to detect the device or if the device is detected by i2c, but then the ADC driver just fails to initialize it.
Best Regards,
Jeremias
Dear Jeremias,
I did make an overlay for the device. However, I think I did not make customizations which should affect the ADC, but maybe you can check it. Could I send the device tree overlay in a private message to you?
I will ask the customer to run the command to check the i2c if the error occurs and report back on it.
Kind regards,
Anouk
Could I send the device tree overlay in a private message to you?
Regarding your device tree customization you sent me, on initial inspection I don’t see anything that would affect ADC or the related I2C interface. At least for now then I don’t think this is the cause of the issue.
I will ask the customer to run the command to check the i2c if the error occurs and report back on it.
Please let us know the results of this. It will help us narrow down whether the issue is more on the I2C or ADC side.
Best Regards,
Jeremias
Hi,
I am “the customer” 
Thank you all for your help.
The issue has been reproduced again. I have now SSH’d into the machine complaining about the missing ADC device and just run the i2cdetect -y -r 0 command, as suggested by @jeremias.tx:
torizon@verdin-imx8mm-07222831:~$ i2cdetect -y -r 0
0 1 2 3 4 5 6 7 8 9 a b c d e f
00: -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- UU -- -- -- -- -- -- -- -- -- --
30: -- -- UU -- -- -- -- -- -- -- -- -- -- -- -- --
40: -- -- -- -- -- -- -- -- -- 49 -- -- -- -- -- --
50: UU -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
70: -- -- -- -- -- -- -- --
I can see other peripherals:
torizon@verdin-imx8mm-07222831:~$ ll /dev/verdin-*
lrwxrwxrwx 1 root root 5 Dec 5 08:03 /dev/verdin-i2c-on-module -> i2c-0
lrwxrwxrwx 1 root root 5 Dec 5 08:03 /dev/verdin-i2c1 -> i2c-3
lrwxrwxrwx 1 root root 5 Dec 5 08:03 /dev/verdin-i2c2 -> i2c-1
lrwxrwxrwx 1 root root 5 Dec 5 08:03 /dev/verdin-i2c4 -> i2c-2
lrwxrwxrwx 1 root root 7 Dec 5 08:03 /dev/verdin-uart2 -> ttymxc2
lrwxrwxrwx 1 root root 7 Dec 5 08:03 /dev/verdin-uart3 -> ttymxc0
lrwxrwxrwx 1 root root 8 Dec 5 08:03 /dev/verdin-watchdog -> watchdog
lrwxrwxrwx 1 root root 9 Dec 5 08:03 /dev/verdin-watchdog-soc -> watchdog0
I can leave the system in this state for a while, in case you come up with more diagnostic tests. Thanks!
Thank you for returning back with more information!
This is interesting, looking at your i2cdetect -y -r 0 output. The i2c interface can see the ADC chip at address 49. But, unlike in my setup it’s not UU. UU indicates that this device is currently in-use, usually by a kernel driver. In your case it seems like the driver for this ADC chip never took control of this device.
This would explain why there is no /dev/verdin-adc1 in you system since the driver never seemed to initialize. Now the question is why? Unfortunately this driver has little to no useful debugging output.
How often would you say this issue occurs on your devices? Are there any conditions or patterns with your system you can notice that may be related to this issue? Anything that may help reproduce or characterize the issue would help us.
Best Regards,
Jeremias
@jeremias.tx - Is there a way to determine if the driver is even running whatsoever? dmesg output? Does the driver get loaded via insmod or is it statically built in and enabled on startup?
@jeremias.tx Thank you for your feedback. I haven’t seen any pattern, sorry.
@DaveM The dmesg output I collected was already uploaded by xAnouk, in case you want to have a look at it.
Is there a way to determine if the driver is even running whatsoever? dmesg output?
Well that’s the tricky thing. Even on my device where the driver behaves well there’s no dmesg output coming from this driver. It just seems this driver has very little to no log output in success or failure. So it’s really hard to tell what’s even going on.
Does the driver get loaded via insmod or is it statically built in and enabled on startup?
I can see the driver via lsmod on my system. It shows up as ti_ads1015.
Thank you for your feedback. I haven’t seen any pattern, sorry.
I was afraid of this. This may be difficult for us to properly investigate, but I’ll discuss this internally and see what can be done.
Best Regards,
Jeremias
Is it that the driver never actually was able to run from startup and create the symlinks, or that the driver did run, the symlinks did get created, but then later the driver crashed and the symlinks were removed?
Of course, you could detect this condition and reload the driver and see if this corrects the problem? I think that’s what the other user did. Figuring out why is a good thing, but you probably also need the board to continue running in the field?
That driver sure could use a bit more log output for sure.
@antonio.astorino I never actually asked you to confirm this. But on the problematic unit could you confirm whether the ti_ads1015 driver is actually loaded or not via lsmod?
Given what has been shown of the issue so far I would assume not. But let’s actually confirm this by checking.
Best Regards,
Jeremias
Hi @jeremias.tx. Here is what I see:
torizon@verdin-imx8mm-14684091:~$ lsmod | grep ti_ads1015
ti_ads1015 28672 0
industrialio_triggered_buffer 16384 1 ti_ads1015
industrialio 90112 3 industrialio_triggered_buffer,kfifo_buf,ti_ads1015
By the way, now that we have 9 systems being turned on and off every day, the problem occurs at least once a day.
Okay so it appears the driver is loaded, but it just fails to do what it’s suppose to do for whatever reason.
By the way, now that we have 9 systems being turned on and off every day, the problem occurs at least once a day.
Could you elaborate on this? Do you mean you do a single power-cycle on each of these 9 systems per day? So 9 on-off cycles a day total? With this relatively low amount of power-cycles you’re able to see the issue on at least one system each day?
I just did a test where I took my own Verdin-iMX8MM (using the same 6.2.0 version as your system) and tried to power-cycle it several dozens of times in a row. It worked every single time for me, /dev/verdin-adc1 was always present on my system despite the many power-cycles I did.
This leads me to think possibly there’s something unique about your environment. So I have some questions:
Best Regards,
Jeremias