Reducing Boot time for Apalis board

Technical Support
, ,
1

Hello,

I have Apalis iMX8QM 8GB board, with ixora carrier, with os: TorizonCore 6.2.0+build.2 (kirkstone).

I have Qt/QML application for user interface which needs: Video acceleration, Bluetooth 2 modes (Classic + BLE), 2 Can buses, Network / Internet access, 3 usb port, Ext4 / NTFS, and some more features.

The problem is booting time takes about 30 seconds and more. These are the analyze results:

systemd-analyze time
Startup finished in 4.236s (kernel) + 25.854s (userspace) = 30.091s
multi-user.target reached after 25.782s in userspace

systemd-analyze blame | head
21.148s plymouth-quit-wait.service
19.099s docker.service
2.738s NetworkManager-wait-online.service
2.098s dev-disk-by\x2dlabel-otaroot.device
1.428s systemd-udev-trigger.service
712ms udisks2.service
607ms containerd.service
552ms user:1000.service
494ms systemd-networkd.service
330ms systemd-random-seed.service

systemd-analyze critical-chain

multi-user.target: 25.782s
greenboot-status.service: 25.698s +81ms
greenboot-task-runner.service: 25.473s +214ms
boot-complete.target: 25.440s
docker.service: 6.337s +19.099s
network-online.target: 6.327s
NetworkManager-wait-online.service: 3.586s +2.738s
NetworkManager.service: 3.501s +79ms
network-pre.target: 3.495s
iptables.service: 3.330s +148ms
basic.target: 3.304s
sockets.target: 3.304s
sshd.socket: 3.289s +13ms
sysinit.target: 3.272s
systemd-timesyncd.service: 3.105s +165ms
systemd-tmpfiles-setup.service: 2.949s +136ms
systemd-journal-flush.service: 2.301s +219ms

systemd-analyze critical-chain greenboot-status.service

greenboot-status.service +81ms
greenboot-task-runner.service: 25.473s +214ms
boot-complete.target: 25.440s
docker.service: 6.337s +19.099s
network-online.target: 6.327s
NetworkManager-wait-online.service: 3.586s +2.738s
NetworkManager.service: 3.501s +79ms
network-pre.target: 3.495s
iptables.service: 3.330s +148ms
basic.target: 3.304s
sockets.target: 3.304s
sshd.socket: 3.289s +13ms
sysinit.target: 3.272s
systemd-timesyncd.service: 3.105s +165ms
systemd-tmpfiles-setup.service: 2.949s +136ms
systemd-journal-flush.service: 2.301s +219ms
ostree-remount.service: 2.017s +174ms
var.mount: 1.906s +14ms
dev-disk-by\x2dlabel-otaroot.device: 901ms +2.098s

How can I reduce the booting time to about 10 seconds or less please?

There are my Dockerfile and docker-compose.yml:

Dockerfile (19.2 KB)

docker-compose.yml (4.1 KB)

Best regards,

Tarek

2

Hi, I’m the Toradex AI Assistant.

I might make mistakes or occasionally get you wrong. Our support team will step in if needed.

Hello Tarek,

Based on your systemd-analyze output, I can see the main bottlenecks in your boot time. Let me provide specific optimization strategies for your Torizon OS setup.

Primary Boot Time Issues

Your analysis shows these major time consumers(1):

  • plymouth-quit-wait.service: 21.148s
  • docker.service: 19.099s
  • NetworkManager-wait-online.service: 2.738s

Optimization Strategies

1. Optimize Docker Service Dependencies

The docker.service is taking 19 seconds and blocking the critical path(1). You can modify its dependencies to reduce boot time by removing unnecessary services:

# systemctl cat docker.service | grep "^After\|^Wants\|^Requires"
After=network-online.target docker.socket firewalld.service usermount.service
Wants=network-online.target
Requires=docker.socket usermount.service

(1)

If your Qt application doesn’t require network, firewall, and usermount services immediately at boot, you can modify the dependencies(1):

# systemctl cat docker.service | grep "^After\|^Wants\|^Requires"
After=docker.socket
Wants=network-online.target
Requires=docker.socket

(1)

After making changes, reload systemd(1):

# sudo systemctl daemon-reload

(1)

2. Address Plymouth Boot Screen

The plymouth-quit-wait.service consuming 21 seconds suggests display initialization issues. For embedded applications, you might consider disabling Plymouth if visual boot feedback isn’t critical.

3. Network Optimization

NetworkManager-wait-online.service is adding 2.7 seconds(1). If your Qt application can handle delayed network connectivity, you can modify this dependency.

4. WiFi Driver Optimization

Torizon OS includes optimizations for WiFi drivers that can affect boot time(1). The system blacklists WiFi drivers by default and loads them later via systemd services to ensure consistent boot performance(1).

Measuring Improvements

After implementing changes, use these commands to measure improvements(1):

# systemd-analyze time

(1)

# systemd-analyze blame | head

(1)

# systemd-analyze critical-chain

(1)

Qt Application Considerations

Since you’re running a Qt/QML application, ensure you’re using the optimized Qt containers for Torizon OS(2). These provide hardware acceleration and are optimized for boot performance(2).

Persistent Configuration

To make systemd service modifications persistent, place override files in /etc/systemd/system(1). You can then capture these changes with TorizonCore Builder for deployment(1).

Expected Results

Based on the documentation example, similar optimizations reduced boot time from 14.5s to 12.6s in userspace(1). With your current 25.8s userspace time, you should see significant improvements by addressing the docker service dependencies and Plymouth service.

Alternative Approach

If these optimizations don’t achieve your target of 10 seconds, you may need to consider creating a custom distribution using Toradex BSP Layers and Reference Images for Yocto Project Software(1).

Best regards,
Toradex AI Assistant.

Did you like my answer? :+1: :-1:

If I answered your question, please tick the “Solution” checkbox.

For follow-up questions to me, please use @ToradexAI in your post.

3

1-

systemctl cat docker.service | grep “^After|^Wants|^Requires”
After=network-online.target docker.socket firewalld.service containerd.service fio-docker-fsck.service
Wants=network-online.target containerd.service
Requires=docker.socket

2- Visual boot feedback isn’t critical, how to disable: Plymouth Boot Screen?

3- What is the meaning of: Delayed network connectivity? When my Qt application starts, it sends Post/Get requests, how to do that?

4- I don’t need WiFi, I need Bluetooth, can I disable WiFi and leave Bluetooth working?

For the Alternative Approach, I don’t prefer to create a custom distribution.

Best regards,

Tarek

4

Hello @tarekmhnds !

1- In regards to the docker.service, if your Qt app can handle network (which, by item 3, I’m assuming it does), we can remove firewalld, and other services, which delays startup.

To do so, you can create an overwrite file:
sudo systemctl edit docker.service
Then, modify said file to:

After=docker.socket
Wants=network-online.target
Requires=docker.socket

After doing so, reload systemd and reboot it for good measure:
sudo systemctl daemon-reload
sudo reboot

2- In regards to the boot screen, since it is not needed and is a major bottleneck as of now, disabling it is an option.
To do so, you first have to mask the plymouth services to prevent them starting:
sudo systemctl mask plymouth-quit-wait.service plymouth-start.service plymouth-read-write.service plymouth-quit.service
Then, connect to the serial console and check the current bootargs with printenv
If you see splash or quiet in the bootargs, remove them:

setenv defargs ${defargs} nosplash
saveenv

And reboot: reset

3- The “Delayed Network Connectivity” message means that your Qt application starts before the network is fully connected and ready for communication. In your application, you can either implement some for of retry logic to wait until the network is ready or use the QTcpSocket class to attempt establishing the connection and rely on its signals to automatically detect when the network becomes available and the connection can finally succeed. I’d recommend the latter.

4- You can do so, but honestly disabling wifi is not something that would affect boot times in any form.

Might be worth also taking a look if there is any other systemd service not in use that could be removed.

Let us know if those changes helped with boot time!

5

Hello @joaom.tx ,

Thanks a lot for your reply.

I did the steps 1 and 2, now

systemctl cat docker.service | grep “^After|^Wants|^Requires” gives:
After=network-online.target docker.socket firewalld.service containerd.service fio-docker-fsck.service
Wants=network-online.target containerd.service
Requires=docker.socket
After=docker.socket
Wants=network-online.target
Requires=docker.socket

Also I masked plymouth services to prevent them starting, and there is no splash or quiet in the bootargs.

But the analyze was like this:

systemd-analyze time
Startup finished in 4.297s (kernel) + 25.601s (userspace) = 29.899s
multi-user.target reached after 25.525s in userspace

systemd-analyze blame | head
19.116s docker.service
2.401s NetworkManager-wait-online.service
2.383s dev-disk-by\x2dlabel-otaroot.device
1.167s usermount.service
1.166s systemd-udev-trigger.service
907ms containerd.service
761ms ModemManager.service
759ms udisks2.service
563ms user@1000.service

I rebooted the board many times and made these analysis and I got nearly the same results about 27 sec ~ 29 sec. But in reality, when I measured the boot time with my Mobily watch, from the moment the power was connected to the board until the user interface (QT/QML) appeared, it took about 10 seconds more: 37 seconds, up to 39 seconds.

How can this problem be solved, please? Do you have other ideas?

Best regards,

Tarek

6

The additional 8-10s you are seeing until the Qt/QML UI appears comes from the container startup time, Qt/QML initialization, and GPU bring-up, which systemd does not account for.

What we also can try is:

  1. Mask NetworkManager-wait-online
    sudo systemctl mask NetworkManager-wait-online.service

  2. Start your Qt container earlier
    Lets try changing that initial overlay so that network-online.target isnt a requirement:

[Unit]
After=docker.service
Requires=docker.service
Wants=network.target
  1. Verify you are using the Toradex Qt Wayland accelerated containers

Also, we recommend that if you need to reduce the boot time substantially beyond what is achievable with Torizon + containers, you should consider building a custom minimal image using Yocto that includes only the specific features required for your project. By removing unused services, libraries, and system components at the OS level, you can achieve a much smaller footprint and significantly faster boot times than what is possible with a general-purpose Torizon image.

7

Hi @joaom.tx ,

I tried the first 2 steps, now I got a reduction of approximately 3 seconds in boot time:

systemd-analyze time

Startup finished in 3.814s (kernel) + 22.818s (userspace) = 26.633s
multi-user.target reached after 22.730s in userspace

systemd-analyze blame | head
17.493s docker.service
2.241s dev-disk-by\x2dlabel-otaroot.device
1.467s usermount.service
1.177s udisks2.service
1.136s systemd-udev-trigger.service
821ms containerd.service
798ms ModemManager.service
674ms systemd-logind.service
570ms sshdgenkeys.service
557ms user@1000.service

For the third suggestion, how can I be sure I’m using Toradex Qt Wayland accelerated container? And how to use it if I’m not using it please? In docker-compose.yml I have: image: torizon/weston-vivante: 3, is this image accelerated?

Best regards,

Tarek

8

Hi @joaom.tx ,

After checking docker journal:

journalctl -u docker.service -b --no-pager
I found these 2 lines:

Binding to IP address without --tlsverify is deprecated.
Startup is intentionally being slowed down to show this message.

Which adds about 15 seconds delay to the start up time.

So, after research I found changing this line at docker.service.d:

ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:2375

To this line:

ExecStart=/usr/bin/dockerd -H fd:// -H tcp://127.0.0.1:2375
solved the issue, and it shortened the boot time by 15 seconds.

Now the analyze results are like this:

systemd-analyze time
Startup finished in 3.857s (kernel) + 9.093s (userspace) = 12.950s
multi-user.target reached after 9.038s in userspace

and:

systemd-analyze blame | head
4.017s docker.service
2.266s dev-disk-by\x2dlabel-otaroot.device
1.450s usermount.service
1.254s systemd-udev-trigger.service
717ms ModemManager.service
640ms containerd.service
572ms udisks2.service
560ms user@1000.service
542ms systemd-logind.service
423ms dbus.service

Now the results are more satisfactory.The board and the containers booted up faster.

But, I found that this change causes problems in linking the Torizon extension for VS Code and the board in the TASK RUNNER section and in the Docker/Container tab, also I don’t know if there are other side problems.
How do I solve this, please, and get rid of the 15-second delay at the same time?

Best regards,

Tarek

Colibri-imx8x Boot Time
9

The change you made does fix the delay, but it isn’t the ideal solution. Docker intentionally slows down startup when the API is exposed without TLS, because this is considered unsafe. That’s why the correct approach is to add the --tlsverify flag to the docker.service configuration.

By switching to:
ExecStart=/usr/bin/dockerd -H fd:// -H tcp://127.0.0.1:2375
the Docker API becomes accessible only on localhost. This removes the warning, but it also prevents external tools from reaching the daemon, which is why you’re seeing those issues.

The proper way forward is to keep Docker accessible remotely, but secure it with TLS. This removes the delay, keeps the setup safe, and restores the extension’s functionality.

  • Revert to the original line:
    ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:2375
  • Add the TLS flag in the override file:
    --tlsverify

This will eliminate the warning, secure the Docker endpoint, and allow VS Code to connect normally again.

10

Hi @joaom.tx ,

I applied TLS, now this is the content of docker.service override.conf:

[Unit]
After=docker.service
Requires=docker.service
Wants=network.target

[Service]
ExecStart=
ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:2375 \
–tlsverify \
–tlscacert=/etc/docker/cert/ca.pem \
–tlscert=/etc/docker/cert/server-cert.pem \
–tlskey=/etc/docker/cert/server-key.pem

The docker journal now has no delay for: tlsverify, but the Containers extension for vs code and Task runner also still can’t reach my Apalis board. I think it need to activate TLS from vs code extension side, or there is other problem. Do you know how to solve this please?

Best regards,

Tarek

11

Hi @tarekmhnds

This issue came up in a discussion I was having with another Torizon user. I’m surprised to see you using the Docker API over TCP. That’s not enabled by default so the lag that @joaom.tx mentioned without “–tls-verify” would not be present in the standard config. Can you share details of why you need this feature and how you are using it? If we are missing a use-case we should be supporting, I would like to bring it to the attention of our product team.

Regards,
Drew

12

Hi @drew.tx ,

I have standard config, I have no special use case, and I did not intend to use Docker API over TCP. I have a QT/QML system for operating in-car entertainment, and this requires a fast boot of the operating system and my own applications.

The whole point is that I want to reduce the boot time of my Apalis board, which is currently around 50 seconds. That’s too long, and I want a boot time of only about 10 seconds.

After checking Docker journal, I found this message:

Binding to IP address without --tlsverify is deprecated.
Startup is intentionally being slowed down to show this message.

This add a delay approximately 15 seconds to the boot time.

But after researching, I found that changing this line at docker.service.d:

ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:2375

To this line:

ExecStart=/usr/bin/dockerd -H fd:// -H tcp://127.0.0.1:2375

solved the issue, and it shortened the boot time by 15 seconds, as I mentioned in previous messages. But, this setting prevented the connection between the board and Torizon VS Code Extension.

Therefore, I am asking for a solution that will eliminate this boot delay and restore the connection between the board and the Torizon extension.

My development PC has Windows11 and WSL2, and my Apalis board has this Torizon system:

ID=torizon
NAME=“TorizonCore”
VERSION=“6.2.0+build.2 (kirkstone)”
VERSION_ID=6.2.0-build.2
PRETTY_NAME=“TorizonCore 6.2.0+build.2 (kirkstone)”
DISTRO_CODENAME=“kirkstone”
BUILD_ID=“2”
ANSI_COLOR=“1;34”
VARIANT=“Docker”

If any further information is needed, please let me know.

Best regards,

Tarek

13

That’s interesting. What does sudo grep ExecStart /lib/systemd/system/docker.service show when run from a shell on your device? I just checked on the 6.8.4-build.40 release which is the LTS version of the Torizon 6 major version and it does not enable the TCP API. My guess is you don’t need the override.conf at all since the TCP API should not be enabled.

I recommend you upgrade the 6.8.4 release and test again. It won’t necessarily get you a 10 second boot time but you will be on the latest v6 release. And not having the TCP API exposed is a good thing.

Drew

14

Hi @drew.tx ,

I ran the grep command and it gave this result:

ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

I will try to upgrade to the 6.8.4 release and test again.

Best regards,

Tarek

15

OK. That is good as the TCP API is not available by default. You added it in your override. Definitely try without that and see if the 15s delay you mentioned is gone.

Drew

16

Hi @tarekmhnds

I just wanted to check in here and see if you had made any progress.

Drew