Dear Bram,
Since you are using a Colibri iMX6ULL 1GB it has an eMMC and can thusly make use of overlays. At this link you will find a community thread about somebody using a sgtl5000 and SAI for audio as an example. You might get some inspiration from it. 
Best regards
Collin