I am not an expert but I think you can´t use the bootm command with the boot.scr, it has to be a kernel Image file.
I also had lots of compatibility issues when trying to boot various linux versions from various u-boot versions, perhaps you are experiencing the same. For example on some u-boot versions I could´t get the bootm command to work at all, alternatives are bootz and booti (with different kernel image files respectively)
For one specific uboot and Linux version (i dont´t remember which one) I had to execute the following commands in u-boot:
setenv board colibri-imx8qxp
run bootcmd_mmc1 # this fails, just executed to load some parameters
load mmc 1:1 $kernel_addr_r Image
load mmc 1:1 $fdt_addr_r imx8qxp-colibri-eval-v3.dtb
setenv bootargs "video=DPI-1:640x480-16@60D initcall_blacklist=vpu_driver_init root=/dev/mmcblk1p2 autoinstall clk_ignore_unused pci=nomsi"
booti $kernel_addr_r - $fdt_addr_r
Note that the “Image”-file has to exist on the sd card, if its a zImage file you would have to use the “bootz” command for example. These kernel formats are convertible into one another up to some degree. Probably you can´t just use the same commands but perhaps it gives you a hint to the solution.
Good luck