Adding PCIe NIC to iommu group

Technical Support
, , ,
1

Hi community,
I’m trying to get VFIO running on Apalis iMX8QM in conjunction with Apalis Evaluation Board v1.1. However, I noticed that the dts node for pcie_a isn’t linked to SMMU (unlike e.g. sata node in imx8qm-ss-hsio.dtsi). Can you advice on required changes to the device tree? For an image I use a custom build downstream kernel as per documentation

Thanks a lot in advance,
Alll

2

Hi @Alll,

Thanks for reaching out to the Toradex Community!

May I know which version of Linux kernel are you building?
Also, could you please elaborate a bit more on your use case here?
Were you successful in flashing the custom-built image on the module or did it throw some errors? In that case could you please also share the error logs with us?

3

Hi @rudhi.tx thanks for your reply. For the kernel I use head of toradex_5.15-2.1.x-imx branch on toradex-linux repo.

My use case is to passthrough a PCIe NIC to a VM, so basically I need VFIO-PCIe working.
I’m absolutely successful booting the board. The main problem is that pcie devices are not added to any iommu group and that’s not surprising since pcie_a node in the device tree is missing connection to smmu. So my question is which changes in device tree are required to get it working.

Does it make sense?

4

Hi @Alll ,

Given that running apps on a VM usually results in a significant performance overhead, may I ask why you’re planning to use a virtual machine instead of using something like containers?

For instance, Torizon OS is an operating system we maintain for our SoMs that runs applications inside containers. Depending on your use-case it could be an alternative.

About your device tree question, the kernel documentation on device tree bindings is probably the best option to get more information on something specific like SMMU/IOMMU bindings. These pages might be useful:

Best regards,
Lucas Akira