In the end I got GPT1 to work and finally found out why GPT0 was always enabled. Per default it seems like GPT1 is off, either by having its clock or power gated (or both), which must be enabled in either the SCFW or in the ATF. These files are included in a boot container that is compiled and added to a custom image following the instructions the the toradex documentation.
GPT0 is enabled in the ATF and used during boot for some stuff. This can be seen in the files
imx-atf/plat/imx/imx8qm/imx8qm_bl31_setup.c
imx-atf/plat/imx/imx8qm/imx8qm_psci.c
One of the challenges I had was that even though I changed the imx-boot file in the .tar file that I used for the easy-installer of torizoncore-builder, the new boot container didnt seem to change. The first time I got it to work was when I installed it from a USB-stick instead of using torizoncore-builder deploy. There are some commit hashes for the various firmwares in the boot-container that are printed to the serial console during the boot procedure of the imx8. These can be used to see if the boot-container has changed.
I spent a lot of time trying to understand the compatibility parameter of the device-tree, but in the end, it doesn’t seem like this one matter. I can access the GPT peripherals from linux suing both /dev/mem and a custom built linux-device-driver, even though I havent added or changed any compatibility parameters in the drivers.
TL;DR:
If you get a bus-error when accessing memory it is likely that the peripheral/hardware you are trying to access is disabled. The SCU (system control unit) is used to control the peripherals, and so the SCFW (SCU firmware) must be modified in order to enable the peripherals.