Portainer cannot start in TC with evaluation container - Nightly

vix · June 22, 2023, 8:42am

I downloaded TorizonCore with evaluation containers 6.3.0-devel-20230621+build.307.container for Verdin iMX8M-Plus mounted on my Verdin Development Board.
I see that the image cannot start as expected, becuase Portainer container cannot run with the following errors:

*  Executing task: docker logs --tail 1000 -f 9742ddb5a108c72e9a92caaac3c9081b11fc270cbebad88afe1754db8808bc95 

2023/05/20 06:40PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:534 > encryption key file not present | filename=portainer
2023/05/20 06:40PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:553 > proceeding without encryption key |
2023/05/20 06:40PM INF github.com/portainer/portainer/api/database/boltdb/db.go:124 > loading PortainerDB | filename=portainer.db
2023/05/20 06:40PM FTL github.com/portainer/portainer/api/cmd/portainer/main.go:600 > error="tls: failed to find any PEM data in certificate input"
2023/05/20 06:40PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:534 > encryption key file not present | filename=portainer
2023/05/20 06:40PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:553 > proceeding without encryption key |
2023/05/20 06:40PM INF github.com/portainer/portainer/api/database/boltdb/db.go:124 > loading PortainerDB | filename=portainer.db
2023/05/20 06:40PM FTL github.com/portainer/portainer/api/cmd/portainer/main.go:600 > error="tls: failed to find any PEM data in certificate input"
2023/05/20 06:40PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:534 > encryption key file not present | filename=portainer
2023/05/20 06:40PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:553 > proceeding without encryption key |
2023/05/20 06:40PM INF github.com/portainer/portainer/api/database/boltdb/db.go:124 > loading PortainerDB | filename=portainer.db
2023/05/20 06:40PM FTL github.com/portainer/portainer/api/cmd/portainer/main.go:600 > error="tls: failed to find any PEM data in certificate input"
2023/05/20 06:40PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:534 > encryption key file not present | filename=portainer
2023/05/20 06:40PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:553 > proceeding without encryption key |
2023/05/20 06:40PM INF github.com/portainer/portainer/api/database/boltdb/db.go:124 > loading PortainerDB | filename=portainer.db
2023/05/20 06:40PM FTL github.com/portainer/portainer/api/cmd/portainer/main.go:600 > error="tls: failed to find any PEM data in certificate input"
2023/05/20 06:40PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:534 > encryption key file not present | filename=portainer
2023/05/20 06:40PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:553 > proceeding without encryption key |
2023/05/20 06:40PM INF github.com/portainer/portainer/api/database/boltdb/db.go:124 > loading PortainerDB | filename=portainer.db
2023/05/20 06:40PM FTL github.com/portainer/portainer/api/cmd/portainer/main.go:600 > error="tls: failed to find any PEM data in certificate input"
2023/05/20 06:40PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:534 > encryption key file not present | filename=portainer
2023/05/20 06:40PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:553 > proceeding without encryption key |
2023/05/20 06:40PM INF github.com/portainer/portainer/api/database/boltdb/db.go:124 > loading PortainerDB | filename=portainer.db
2023/05/20 06:40PM FTL github.com/portainer/portainer/api/cmd/portainer/main.go:600 > error="tls: failed to find any PEM data in certificate input"
2023/05/20 06:40PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:534 > encryption key file not present | filename=portainer
2023/05/20 06:40PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:553 > proceeding without encryption key |
2023/05/20 06:40PM INF github.com/portainer/portainer/api/database/boltdb/db.go:124 > loading PortainerDB | filename=portainer.db
2023/05/20 06:40PM FTL github.com/portainer/portainer/api/cmd/portainer/main.go:600 > error="tls: failed to find any PEM data in certificate input"
2023/05/20 06:41PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:534 > encryption key file not present | filename=portainer
2023/05/20 06:41PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:553 > proceeding without encryption key |
2023/05/20 06:41PM INF github.com/portainer/portainer/api/database/boltdb/db.go:124 > loading PortainerDB | filename=portainer.db
2023/05/20 06:41PM FTL github.com/portainer/portainer/api/cmd/portainer/main.go:600 > error="tls: failed to find any PEM data in certificate input"
2023/05/20 06:41PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:534 > encryption key file not present | filename=portainer
2023/05/20 06:41PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:553 > proceeding without encryption key |
2023/05/20 06:41PM INF github.com/portainer/portainer/api/database/boltdb/db.go:124 > loading PortainerDB | filename=portainer.db
2023/05/20 06:41PM FTL github.com/portainer/portainer/api/cmd/portainer/main.go:600 > error="tls: failed to find any PEM data in certificate input"
2023/05/20 06:41PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:534 > encryption key file not present | filename=portainer
2023/05/20 06:41PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:553 > proceeding without encryption key |
2023/05/20 06:41PM INF github.com/portainer/portainer/api/database/boltdb/db.go:124 > loading PortainerDB | filename=portainer.db
2023/05/20 06:41PM FTL github.com/portainer/portainer/api/cmd/portainer/main.go:600 > error="tls: failed to find any PEM data in certificate input"
2023/05/20 06:42PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:534 > encryption key file not present | filename=portainer
2023/05/20 06:42PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:553 > proceeding without encryption key |
2023/05/20 06:42PM INF github.com/portainer/portainer/api/database/boltdb/db.go:124 > loading PortainerDB | filename=portainer.db
2023/05/20 06:42PM FTL github.com/portainer/portainer/api/cmd/portainer/main.go:600 > error="tls: failed to find any PEM data in certificate input"

immagine
I attach the inspect.json (13.2 KB) obtained from ApolloX with Containers >> Inspect

The latest monthly TorizonCore with evaluation containers
6.3.0-devel-202306+build.8.container runs witoput problems, but Chromium container is the “old” :2 tag
This is the log for portainer from the Monthly build

Executing task: docker logs --tail 1000 -f 4df4e8af5d32bb11a6b7afce601e3da5c952423d66ae3c52576bc0c3b7b4af83 

2023/05/20 07:46PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:534 > encryption key file not present | filename=portainer
2023/05/20 07:46PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:553 > proceeding without encryption key |
2023/05/20 07:46PM INF github.com/portainer/portainer/api/database/boltdb/db.go:124 > loading PortainerDB | filename=portainer.db
2023/05/20 07:46PM INF github.com/portainer/portainer/api/datastore/backup.go:109 > creating DB backup |
2023/05/20 07:46PM INF github.com/portainer/portainer/api/datastore/backup.go:45 > copying DB file | from=/data/portainer.db to=/data/backups/common/portainer.db.2.17.1.20230520194646
2023/05/20 07:46PM INF github.com/portainer/portainer/api/database/boltdb/db.go:124 > loading PortainerDB | filename=portainer.db
2023/05/20 07:46PM INF github.com/portainer/portainer/api/datastore/migrate_data.go:108 > migrating database from version 2.17.1 to 2.17.1 |
2023/05/20 07:46PM INF github.com/portainer/portainer/api/datastore/migrator/migrate_dbversion81.go:16 > updating existing user theme settings |
2023/05/20 07:46PM INF github.com/portainer/portainer/api/datastore/migrator/migrate_ce.go:79 > db migrated to 2.17.1 |
2023/05/20 07:46PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:519 > instance already has defined environments, skipping the environment defined via CLI |
2023/05/20 19:46:46 server: Reverse tunnelling enabled
2023/05/20 19:46:46 server: Fingerprint 02:8f:75:4c:d0:fb:90:92:0f:d9:06:c0:a7:f0:a0:ec
2023/05/20 19:46:46 server: Listening on 0.0.0.0:8000...
2023/05/20 07:46PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:810 > starting Portainer | build_number=27976 go_version=1.19.4 image_tag=linux-arm64-2.17.1 nodejs_version=18.14.2 version=2.17.1 webpack_version=5.68.0 yarn_version=1.22.19
2023/05/20 07:46PM INF github.com/portainer/portainer/api/http/server.go:342 > starting HTTPS server | bind_address=:9443
2023/05/20 07:46PM INF github.com/portainer/portainer/api/http/server.go:327 > starting HTTP server | bind_address=:9000

lucas_a.tx · June 22, 2023, 12:08pm

Hi @vix ,

I installed the TorizonCore version you referred to on a Verdin iMX8M Plus, and Portainer works as expected on my side:

torizon@verdin-imx8mp-14777535:~$ cat /etc/os-release
ID=torizon
NAME="TorizonCore"
VERSION="6.3.0-devel-20230621+build.307 (kirkstone)"
VERSION_ID=6.3.0-devel-20230621-build.307
PRETTY_NAME="TorizonCore 6.3.0-devel-20230621+build.307 (kirkstone)"
DISTRO_CODENAME="kirkstone"
BUILD_ID="307"
ANSI_COLOR="1;34"
VARIANT="Docker"
torizon@verdin-imx8mp-14777535:~$ docker ps -a
CONTAINER ID   IMAGE                    COMMAND                  CREATED         STATUS         PORTS                                                           NAMES
12b76f1c63bf   torizon/chromium         "/usr/bin/start-brow…"   2 minutes ago   Up 2 minutes                                                                   torizon-kiosk-1
5cd7168239c7   torizon/weston-vivante   "/usr/bin/entry.sh"      2 minutes ago   Up 2 minutes                                                                   torizon-weston-1
8e1e97b9ebad   portainer/portainer-ce   "/portainer --templa…"   2 minutes ago   Up 2 minutes   8000/tcp, 9443/tcp, 0.0.0.0:8840->9000/tcp, :::8840->9000/tcp   torizon-portainer-1
torizon@verdin-imx8mp-14777535:~$ docker logs --tail 1000 -f 8e
2022/04/28 05:42PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:534 > encryption key file not present | filename=portainer
2022/04/28 05:42PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:553 > proceeding without encryption key |
2022/04/28 05:42PM INF github.com/portainer/portainer/api/database/boltdb/db.go:124 > loading PortainerDB | filename=portainer.db
2022/04/28 05:42PM INF github.com/portainer/portainer/api/internal/ssl/ssl.go:80 > no cert files found, generating self signed SSL certificates |
2022/04/28 17:42:41 server: Reverse tunnelling enabled
2022/04/28 17:42:41 server: Fingerprint 78:b9:ea:14:ab:07:3f:e6:25:d9:32:c3:ac:60:e0:69
2022/04/28 17:42:41 server: Listening on 0.0.0.0:8000...
2022/04/28 05:42PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:810 > starting Portainer | build_number=27976 go_version=1.19.4 image_tag=linux-arm64-2.17.1 nodejs_version=18.14.2 version=2.17.1 webpack_version=5.68.0 yarn_version=1.22.19
2022/04/28 05:42PM INF github.com/portainer/portainer/api/http/server.go:327 > starting HTTP server | bind_address=:9000
2022/04/28 05:42PM INF github.com/portainer/portainer/api/http/server.go:342 > starting HTTPS server | bind_address=:9443

We appreciate your concern and feedback with our releases. However, keep in mind that our nightly releases are preview only, as stated in our Support Strategy page. In fact, we don’t even guarantee that those images will work or boot at all, given that we don’t do any tests on them.

Best regards,
Lucas Akira

vix · June 22, 2023, 12:27pm

I know.
When I need some features not included yet in the most recent Monthly, the only chanche is giving a try to the Nightly…
And if I see something that doesn’t work, I think it’s a good idea sharing it.

I appreciate Toradex effort in assuring the quality of the builds, and for sure it’s not possible doing this on the Nightlies.

Do you need some more details from my side?

leon.tx · June 22, 2023, 12:45pm

Hi again, @vix!

And if I see something that doesn’t work, I think it’s a good idea sharing it.

Absolutely, thanks for testing our nightlies!

Do you need some more details from my side?

We don’t set up encryption for Portainer as far as I know, so maybe the portainer_data volume has some stray certificate from previous runs.

Could you run the following commands to remove the volume and then restart the board? Make sure you’re running the latest docker-compose.yml that comes with build 307.

torizon@verdin-imx8mp-06817296:~$ docker ps -a
CONTAINER ID   IMAGE                           COMMAND                  CREATED         STATUS                            PORTS     NAMES
06418498f426   torizon/chromium:3              "/usr/bin/start-brow…"   3 minutes ago   Exited (0) About a minute ago               torizon-kiosk-1
ebd2f6dacafa   torizon/weston-vivante:3        "/usr/bin/entry.sh"      3 minutes ago   Exited (143) About a minute ago             torizon-weston-1
251297897b8b   portainer/portainer-ce:2.17.1   "/portainer --templa…"   3 minutes ago   Exited (2) About a minute ago               torizon-portainer-1
torizon@verdin-imx8mp-06817296:~$ docker rm -f 251297897b8b
251297897b8b
torizon@verdin-imx8mp-06817296:~$ docker volume rm -f torizon_portainer_data
torizon_portainer_data

Thanks!

vix · June 22, 2023, 2:15pm

Hello @leon.tx

unfortunately I has already reprogrammed the SoM with the Monthly 6.3.0-devel-202306+build.8.container and then with the Nightly 6.3.0-devel-20230621+build.307.container and it works now.

leon.tx · June 22, 2023, 3:11pm

Hi,

No problem. As the out-of-the-box experience is working, I’ll take this as an outlier and mark Lucas’ reply as the answer.

As a reference, if someone else has the same problem in the future, feel free to try the fix I proposed and re-open this thread if needed.

Cheers!