Bluetooth Hosting Connection Issue on Colibri iMX8 running Yocto Dunfell 5.7.x

[JormenJanssen]

Dear Technical Support Team,

I am writing to request your assistance in resolving a Bluetooth Low Energy (BLE) connection issue on my Colibri iMX8 device running the latest Yocto Dunfell 5.7.x software.

I have noticed that the BLE hosting feature on my device is not functioning properly. The connection between my device and other BLE devices is intermittent, and sometimes it does not work at all. When the connection fails, the phone displays an “interrogation error” message, and I have also observed the following error message in the Linux kernel logs: “Bluetooth: hci0: ACL packet for unknown connection handle 128”.

We have tested the BLE hosting feature on the Colibri iMX8 device with both the original Bluez version 5.55 and the customized version 5.64, but the issue persists in both cases. We suspect that the issue might be related to the firmware on the device, possibly in combination with the interface diversity. However, we have successfully tested the same test scenario on other devices, such as the Raspberry PI 4B and the Toradex Colibri iMX6ULL, and did not encounter any issues with the BLE hosting feature.

I have created a reproducible test scenario that can be found on the following Github repository: GitHub - riwo/ble-example. There are pre-built static binaries available on the release(s) page which can be used to test without compiling the code your self (Release 1.0.2 · riwo/ble-example · GitHub).
The test scenario includes code that allow a BLE peripheral to establish a connection to the Colibri IMX8X (central device). I have tested this scenario multiple times and have encountered the same issue with the BLE hosting feature.

Furthermore, I would like to mention that we are running a customized version of Bluez 5.64 in Yocto. We have tested the BLE hosting feature using both the in-tree Linux drivers and the proprietary drivers provided by NXP through Toradex.

In our efforts to troubleshoot the BLE hosting issue, we have disabled the battery service in the Bluetooth service by changing the “ExecStart” line in the “/usr/lib/systemd/system/bluetooth.service” file to “ExecStart=/usr/libexec/bluetooth/bluetoothd -P battery”. The reason for disabling the battery service is that it forces pairing on the iPhone, which is not necessary for our testing scenario. We believe that this is not the cause of the issue, as we have also encountered the same problem when the battery service is enabled.

I would be grateful if you could provide me with a solution to this issue as soon as possible. I rely heavily on this feature to establish a stable connection between my device and other BLE devices.

Please let me know if you require any additional information from my end to diagnose and resolve this issue. I appreciate your prompt attention to this matter and thank you in advance for your help.

Sincerely,
Jormen Janssen

[rafael.tx]

Hello Jormen,
thank you for reporting this and also for taking the time to create an example code to reproduce it. I will try it and see if I can reproduce it here with your example, and see if we can find out what is happening in this case.

Best regards,
Rafael

[JormenJanssen]

Hi Rafael,

Any updates on this topic?

[rafael.tx]

Hello Jormen, unfortunately, no updates on this yet. I’m planning to at least have a preliminary look this week or beginning of next week.

[rafael.tx]

Hello Jormen,
I managed to reproduce the issue, but I think my results are a little different than yours.

First I tried to run the static binaries from the github repository, but running ble-example just hangs without any messages.
I compiled the source and added some debug information, and found out that it was hanging on this call:

btmgmt.SetLinkLevelSecurity(false)

After commenting this line, the program goes further:

root@colibri-imx8x-06858909:~# ./main
[103256.723975] debugfs: File 'le_min_key_size' in directory 'hci0' already present!
[103256.731534] debugfs: File 'le_max_key_size' in directory 'hci0' already present!
WARN[0001] map[/hci0/apps/0/service67891000/char0:0x400005b3e0 /hci0/apps/0/service67891000/char1:0x40001a6900 /hci0/apps/0/service67891000/char2:0x40001c5380]
DEBU[0001] Setup adapter 

I then used LightBlue to try a connection to the colibri, but in my case it never succeeds. I always get a connection failure with the following message:

And on the colibri I get this:

[103929.390048] Bluetooth: hci0: ACL packet for unknown connection handle 128

You mention that this is intermittent in your case, how many attempts do you need to get a successful connection? I tried three times here with the same result. Could it be because of the commented security stuff I mentioned before?
I’m not familiar with how BLE works so I don’t really know if I’m doing the test correctly or not.

[JormenJanssen]

Hi Rafael,

After receiving your feedback, I conducted some testing on a freshly installed Toradex BSP 5.7.X Multimedia image from Tezi and obtained mixed results. The testing process proved to be challenging, as achieving a successful attempt was difficult. In order to increase the chances of success, I implemented a workaround (and fixed the btmgt code) in the example code. This workaround involved executing the bluetoothctl binary after exposing the advertisement on the dbus. You can find the code with the workaround at this link: GitHub code.

To disable this hack, you can set the DISABLE_BLUETOOTHCTL_HACK=1 environment variable when running the example. For example: DISABLE_BLUETOOTHCTL_HACK=1 ./ble-example. Please keep in mind that even after closing the test, the hack may persist. To properly retest the functionality, it is recommended to power cycle the board.

To install the (new version 1.0.4) example, you can use the following command:

curl -L https://github.com/riwo/ble-example/releases/download/1.0.4/ble-example_1.0.4_linux_arm64.tar.gz | tar -xz && chmod +x ./ble-example

During testing, I used the command
rfkill unblock bluetooth && ./ble-example
to initiate the test. Please note that if you encounter any issues on your phone, I recommend killing the LightBlue app and retrying the test procedure.

Lastly, regarding security, the configuration was set up specifically to avoid any pairing and rely solely on plain Bluetooth Low Energy bonding, which shouldn’t encrypt traffic.

Based on the kernel message “[ 1733.884559] Bluetooth: hci0: ACL packet for unknown connection handle 128,” it appears to be the real problem. This message suggests that there is an issue with the ACL (Asynchronous Connectionless) packet for an unknown connection handle, specifically handle 128.

In this case, i suspect that the wireless firmware is responsible for this problem. The firmware plays a crucial role in managing the Bluetooth functionality.

You can find a demo video below

[rafael.tx]

OK, with this updated version the connection succeeds. How does the problem manifest then? Should I wait for a while? Or should I go back to the app from time to time to check if there’s still a connection?

[rafael.tx]

I have been reconnecting to the colibri imx8x on the working setup; in this case, I couldn’t reproduce the issue.
Do you reproduce it at all with the “hack”? I ask because from the message that the kernel outputs it seems that there’s some previous setup missing (“unknown connection handle”) and I’m wondering if the extra call you’re making is setting up everything that’s needed for it to properly work.

[rudhi.tx]

Dear @JormenJanssen,

Do you have any updates on this topic?

[JormenJanssen]

Thank you for providing the additional information. Regarding the “hack” to reconnect to the Colibri iMX8x on the working setup, I have not been able to reproduce the issue with this approach. However, it is worth noting that on the other devices we have tested, such as the Colibri iMX6ULL, Raspberry PI 4B, and some Linux devices, we have not observed the requirement for an extra call to set up everything properly. The Colibri iMX8 is the only device exhibiting this behavior, which is a rare occurrence compared to many other devices.

We expect that the behavior of requiring an extra call to set up the necessary components may also be seen on the Apalis iMX8 due to its similar interface diversity. However, we do not currently have access to this device for testing. On the other hand, we do not anticipate encountering the same issue on the Verdin iMX8 devices. Additionally, we have conducted some tests using BSP 6.X and have obtained the same results on the Colibri iMX8.

Considering these findings, we believe that the issue lies within the firmware or software configuration specific to the Colibri iMX8 device, potentially related to the interface diversity. We would greatly appreciate your expertise in diagnosing and resolving this issue. Please let us know if there is any further information or specific testing you would like us to perform.

Additionally, I would like to inform you that we have conducted tests using proprietary firmware provided by NXP through Toradex support. After signing the necessary non-disclosure agreement (NDA), we obtained firmware packages PCIE-WLAN-USB-BT-8997-U16-X86-W16.88.10.p173-16.26.10.p173-C4X16698_V4-MGPL.zip and PCIE-WLAN-USB-BT-8997-U16-X86-W16.88.10.p70-16.26.10.p70-C4X16672_V4-MGPL.zip. Unfortunately, these firmware versions also did not resolve the issue.

We kindly request that Toradex support verify the latest available version for this specific interface diversity on the proprietary NXP portal. It would be helpful to know if there are any firmware updates that address this particular issue.

Due to the project deadline and the urgency of having a functional Bluetooth connection, we have decided to temporarily include the “hack” that allows us to establish a working connection. Although this workaround is not an ideal long-term solution, it will suffice for the time being to meet our project requirements.