Dear all,
I’m using a Toradex Apalis IMX6 and I am trying to make PCIe working.
On the evaluation board, everything is working however on our board, we have some errors at the initialization.
One precision, we are using a Pericom switch which is not the same as the one on the eval board.
During the init, I have messages saying that memory can’t be assigned :
pci 0000:00:00.0: BAR 8: assigned [mem 0x01000000-0x01dfffff]
pci 0000:00:00.0: BAR 0: assigned [mem 0x01e00000-0x01efffff]
pci 0000:00:00.0: BAR 9: can't assign mem pref (size 0x100000)
pci 0000:00:00.0: BAR 6: can't assign mem pref (size 0x10000)
pci 0000:00:00.0: BAR 7: assigned [io 0x1000-0x2fff]
pci 0000:01:00.0: BAR 8: assigned [mem 0x01000000-0x01dfffff]
pci 0000:01:00.0: BAR 9: can't assign mem pref (size 0x100000)
pci 0000:01:00.0: BAR 7: assigned [io 0x1000-0x2fff]
pci 0000:02:03.0: BAR 8: assigned [mem 0x01000000-0x01bfffff]
pci 0000:02:01.0: BAR 8: assigned [mem 0x01c00000-0x01cfffff]
pci 0000:02:01.0: BAR 9: assigned [mem 0x01d00000-0x01dfffff pref]
pci 0000:02:01.0: BAR 7: assigned [io 0x1000-0x1fff]
pci 0000:02:03.0: BAR 7: assigned [io 0x2000-0x2fff]
pci 0000:03:00.0: BAR 6: assigned [mem 0x01d00000-0x01d0ffff pref]
pci 0000:03:00.0: BAR 5: assigned [mem 0x01c00000-0x01c007ff]
pci 0000:03:00.0: BAR 4: assigned [io 0x1000-0x101f]
pci 0000:03:00.0: BAR 0: assigned [io 0x1020-0x1027]
pci 0000:03:00.0: BAR 2: assigned [io 0x1028-0x102f]
pci 0000:03:00.0: BAR 1: assigned [io 0x1030-0x1033]
pci 0000:03:00.0: BAR 3: assigned [io 0x1034-0x1037]
Then when I do a lspci, I have :
00:00.0 PCI bridge: Synopsys, Inc. Device abcd (rev 01)
01:00.0 PCI bridge: Pericom Semiconductor Device 2404 (rev 05)
02:01.0 PCI bridge: Pericom Semiconductor Device 2404 (rev 05)
02:02.0 PCI bridge: Pericom Semiconductor Device 2404 (rev 05)
02:03.0 PCI bridge: Pericom Semiconductor Device 2404 (rev 05)
03:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9220 PCIe 2.0 x2 2-port SATA 6 Gb/s RAID Controller (rev 11)
05:00.0 Ethernet controller: Intel Corporation Device 1531 (rev 03)
Connected to the switch I have a SATA controller, an Ethernet controller and a mini-PCIe connector.
The devices are discovered, but it is impossible to access them in a High level way (mount the drive, the Ethernet interface is not listed).
But when using the Eval board, there is no error, and the peripherals are all working.
My guess would be that the Device Tree that I use (the one from the Eval Board) is not compatible with our board (due the a different PCIe switch) and that I have to modify the Device Tree to adapt it to our PCIe switch.
Am I right ?
Thanks in advance
Pablo
There shouldn’t be anything PCIe switch specific in the device tree apart from maybe its reset workaround. Have you tested the exact same switch on one of our carrier boards as well? Did you check its design and errata documents? How exactly does your schematic look like?
I think that we shall have some configuration for PCIe in the device tree.
In the linux image supplied by Toradex we have these in the DTS file
pcie@0x01000000 {
compatible = "fsl,imx6q-pcie", "snps,dw-pcie";
reg = <0x1ffc000 0x4000 0x1f00000 0x80000>;
reg-names = "dbi", "config";
#address-cells = <0x3>;
#size-cells = <0x2>;
device_type = "pci";
ranges = <0x81000000 0x0 0x0 0x1f80000 0x0 0x10000
0x82000000 0x0 0x1000000 0x1000000 0x0 0xf00000>;
num-lanes = <0x1>;
interrupts = <0x0 0x78 0x4>;
interrupt-names = "msi";
#interrupt-cells = <0x1>;
interrupt-map-mask = <0x0 0x0 0x0 0x7>;
interrupt-map = <0x0 0x0 0x0 0x1 0x1 0x0 0x7b 0x4 0x0 0x0 0x0 0x2 0x1 0x0 0x7a 0x4 0x0 0x0 0x0 0x3 0x1 0x0 0x79 0x4 0x0 0x0 0x0 0x4 0x1 0x0 0x78 0x4>;
clocks = <0x2 0xbd 0x2 0xbb 0x2 0xce 0x2 0x90>;
clock-names = "pcie_phy", "ref_100m", "pcie_bus", "pcie";
status = "okay";
reset-gpio = <0x4 0x2 0x0>;
reset-ep-gpio = <0x4 0x1c 0x0>;
};
And
pcie-switch@58 {
compatible = "plx,pex8605";
reg = <0x58>;
};
I guess I need to adapt at least the last code sample above in order to make the iMX6 to work with a PI7C9X2 Pericom switch. But I have no idea on how to do it, and on the values to set in the “compatible” field.
As mentioned before a PCIe switch should really not require any additional configuration. The pcie-switch node in the I2C section really only describes our Apalis Evaluation board’s PCIe switch also featuring an additional out-of-band I2C management interface which as far as I know nobody is currently even using (e.g. there isn’t even a driver for any such as of yet). The only thing which may require adjustment is the resetting of your switch (e.g. reset-gpio node) and/or your PCIe devices (e.g. reset-ep-gpio node). It also looks like you are not looking at the proper device tree sources as e.g. those GPIO nodes are already expanded rather than pointing to particular GPIOs e.g. as here.
Thanks for the precisions, I will check the reset pins with the HW team.
For the device tree file, I think we are using an old version (taken from a previous project)
I think you are right
I checked with the HW team, the switch reset pin is connected to the reset_moci signal of the Toradex, the PCI devices’ reset are handled by the switch itself. So I think the switch is not correctly reset.
I tried several configurations for the reset-gpio and reset-ep-gpio, but it seems that they are not correct, can you help me on this ?
Thanks !
By default reset-gpio is actually the Apalis GPIO7 connected to the PCIe switch reset on the Apalis Evaluation board and reset-ep-gpio the RESET_MOCI# signal connected to the mini-PCIe and regular PCIe slots. But even if you are just using the RESET_MOCI# signal connected to your PCIe switch it should still work just fine. Unless of course your switch requires a different polarity on its reset signal or has some special bring-up/timing requirements.
That said you may actually have an entirely different issue involving GEN2 compatibility. For the Apalis Evaluation board we e.g. had to limit the PCIe speed to GEN1 in order to get it working in a reliable way. Maybe your design exhibits the same limitation and you would need to do something like this but changing the PCI ID 0x8605 to the one from your switch e.g. 0x2404 as far as I can see.
Okay we checked the reset, and they look OK. HW team thinks there is no GEN2 incompatibility.
We had a reply from the manufacturer, they tell us that in the switch registers, I/O limit address is lower than the I/O base address so, no I/O packets can be forwarded from root complex to the end point devices. Same for the Memory addresses. And this is for upstream and downstream ports.
Theses registers are at offset 0x1C and 0x20.
They advise us to check the U-Boot code and the check why we would have a different behavior with a different switch.
Do you have an idea on what could happen ?
