Some time has passed but at this point I want to confirm this answer because I have verified that it works and it might help somebody else.
As suggested by @gustavo.tx the line:
RDC_PDAPn(RDC_PDAP_UART4, D1R | D1W),
should be replaced by:
RDC_PDAPn(RDC_PDAP_UART4, D0R | D0W),
Just create a boot container as is explained in the documentation mentioned by @gustavo.tx as well. If you use the default device tree provided by Toradex you can just put the status of the newly freed serialport to “okay”. Which then looks something like this:
serial@30a60000 {
compatible = "fsl,imx8mm-uart\0fsl,imx6q-uart";
reg = <0x30a60000 0x10000>;
interrupts = <0x00 0x1d 0x04>;
clocks = <0x02 0xbf 0x02 0xbf>;
clock-names = "ipg\0per";
dmas = <0x2a 0x1c 0x04 0x00 0x2a 0x1d 0x04 0x00>;
dma-names = "rx\0tx";
status = "okay";
pinctrl-names = "default";
pinctrl-0 = <0x3e>;
};