Hello,
I’m writing regarding an issue I’m experiencing on the apalis iMX6 modules regarding the initialization of the PCIe devices during boot up. I’m currently building using BSP v5.7.0
Here is filtered version of the bootup log focusing on the keyword pci:
[ 0.254436] imx6q-pcie 1ffc000.pcie: 1ffc000.pcie supply epdev_on not found, using dummy regulator
[ 0.636590] imx6q-pcie 1ffc000.pcie: host bridge /soc/pcie@1ffc000 ranges:
[ 0.636604] imx6q-pcie 1ffc000.pcie: Parsing ranges property…
[ 0.636644] imx6q-pcie 1ffc000.pcie: IO 0x01f80000…0x01f8ffff → 0x00000000
[ 0.636667] imx6q-pcie 1ffc000.pcie: MEM 0x01000000…0x01efffff → 0x01000000
[ 0.739668] imx6q-pcie 1ffc000.pcie: Link up
[ 0.739676] imx6q-pcie 1ffc000.pcie: Link: Gen2 disabled
[ 0.739684] imx6q-pcie 1ffc000.pcie: Link up, Gen1
[ 0.739857] imx6q-pcie 1ffc000.pcie: PCI host bridge to bus 0000:00
[ 0.739873] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 0.739882] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]
[ 0.739890] pci_bus 0000:00: root bus resource [mem 0x01000000-0x01efffff]
[ 0.739897] pci_bus 0000:00: scanning bus
[ 0.739930] pci 0000:00:00.0: [16c3:abcd] type 01 class 0x060400
[ 0.739959] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff]
[ 0.739976] pci 0000:00:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]
[ 0.740009] pci 0000:00:00.0: Limiting cfg_size to 512
[ 0.740062] pci 0000:00:00.0: supports D1
[ 0.740069] pci 0000:00:00.0: PME# supported from D0 D1 D3hot D3cold
[ 0.740080] pci 0000:00:00.0: PME# disabled
[ 0.746274] pci_bus 0000:00: fixups for bus
[ 0.770329] pci 0000:00:00.0: scanning [bus 01-ff] behind bridge, pass 0
[ 0.779078] pci_bus 0000:01: scanning bus
[ 0.788815] pci 0000:01:00.0: [10b5:8603] type 01 class 0x060400
[ 0.795330] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x00003fff]
[ 0.806652] pci 0000:01:00.0: supports D1 D2
[ 1.344515] pci 0000:01:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 1.351846] pci 0000:01:00.0: PME# disabled
[ 1.353327] pci 0000:01:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s x1 link at 0000:00:00.0 (capable of 4.000 Gb/s with 5 GT/s x1 link)
[ 1.373838] pci_bus 0000:01: fixups for bus
[ 1.381877] pci 0000:01:00.0: scanning [bus 00-00] behind bridge, pass 0
[ 1.381887] pci 0000:01:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 1.389990] pci 0000:01:00.0: scanning [bus 00-00] behind bridge, pass 1
[ 1.390305] pci_bus 0000:02: scanning bus
[ 1.390710] pci 0000:02:01.0: [10b5:8603] type 01 class 0x060400
[ 1.397432] pci 0000:02:01.0: supports D1 D2
[ 1.401745] pci 0000:02:01.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 1.408433] pci 0000:02:01.0: PME# disabled
[ 1.409205] pci 0000:02:02.0: [10b5:8603] type 01 class 0x060400
[ 1.415924] pci 0000:02:02.0: supports D1 D2
[ 1.420232] pci 0000:02:02.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 1.426921] pci 0000:02:02.0: PME# disabled
[ 1.433217] pci_bus 0000:02: fixups for bus
[ 1.438936] pci 0000:02:01.0: scanning [bus 00-00] behind bridge, pass 0
[ 1.446847] pci 0000:02:01.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 1.454938] pci 0000:02:02.0: scanning [bus 00-00] behind bridge, pass 0
[ 1.471123] pci 0000:02:02.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 1.486894] pci 0000:02:01.0: scanning [bus 00-00] behind bridge, pass 1
[ 1.495602] pci_bus 0000:03: scanning bus
[ 1.503929] pci 0000:03:00.0: [8086:1531] type 00 class 0x020000
[ 1.510113] pci 0000:03:00.0: reg 0x10: [mem 0x00000000-0x007fffff]
[ 1.525011] pci 0000:03:00.0: reg 0x18: [io 0x0000-0x001f]
[ 1.530647] pci 0000:03:00.0: reg 0x1c: [mem 0x00000000-0x00003fff]
[ 1.544493] pci 0000:03:00.0: PME# supported from D0 D3hot D3cold
[ 1.558848] pci 0000:03:00.0: PME# disabled
[ 1.574467] pci_bus 0000:03: fixups for bus
[ 1.586663] pci_bus 0000:03: bus scan returning with max=03
[ 1.590769] pci_bus 0000:03: busn_res: [bus 03-ff] end is updated to 03
[ 1.597454] pci 0000:02:02.0: scanning [bus 00-00] behind bridge, pass 1
[ 1.609049] pci_bus 0000:04: scanning bus
[ 1.627553] pci 0000:04:00.0: [1ee7:babe] type 00 class 0x070001
[ 1.640701] pci 0000:04:00.0: reg 0x10: [mem 0x00000000-0x000fffff]
[ 1.652814] pci 0000:04:00.0: reg 0x14: [mem 0x00000000-0x0000000f]
[ 1.662962] pci 0000:04:00.0: reg 0x18: [mem 0x00000000-0x0000000f]
[ 1.669307] pci 0000:04:00.0: reg 0x1c: [mem 0x00000000-0x0000000f]
[ 1.675651] pci 0000:04:00.0: reg 0x20: [mem 0x00000000-0x0000000f]
[ 1.685991] pci 0000:04:00.0: reg 0x24: [mem 0x00000000-0x0000000f]
[ 1.698654] pci 0000:04:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s x1 link at 0000:00:00.0 (capable of 4.000 Gb/s with 5 GT/s x1 link)
[ 1.723941] pci_bus 0000:04: fixups for bus
[ 1.729576] pci_bus 0000:04: bus scan returning with max=04
[ 1.735112] pci_bus 0000:04: busn_res: [bus 04-ff] end is updated to 04
[ 1.741764] pci_bus 0000:02: bus scan returning with max=04
[ 1.747439] pci_bus 0000:02: busn_res: [bus 02-ff] end is updated to 04
[ 1.754106] pci_bus 0000:01: bus scan returning with max=04
[ 1.759506] pci 0000:00:00.0: scanning [bus 01-ff] behind bridge, pass 1
[ 1.759517] pci_bus 0000:00: bus scan returning with max=ff
[ 1.759559] pci 0000:00:00.0: BAR 8: no space for [mem size 0x01200000]
[ 1.772297] pci 0000:00:00.0: BAR 8: failed to assign [mem size 0x01200000]
[ 1.785144] pci 0000:00:00.0: BAR 0: assigned [mem 0x01000000-0x010fffff]
[ 1.785160] pci 0000:00:00.0: BAR 9: assigned [mem 0x01100000-0x012fffff pref]
[ 1.797734] pci 0000:00:00.0: BAR 6: assigned [mem 0x01300000-0x0130ffff pref]
[ 1.810656] pci 0000:00:00.0: BAR 7: assigned [io 0x1000-0x1fff]
[ 1.823859] pci 0000:01:00.0: BAR 8: no space for [mem size 0x01000000]
[ 1.835557] pci 0000:01:00.0: BAR 8: failed to assign [mem size 0x01000000]
[ 1.849611] pci 0000:01:00.0: BAR 9: assigned [mem 0x01100000-0x012fffff 64bit pref]
[ 1.862263] pci 0000:01:00.0: BAR 0: no space for [mem size 0x00004000]
[ 1.870024] ehci-pci: EHCI PCI platform driver
[ 1.876580] pci 0000:01:00.0: BAR 0: failed to assign [mem size 0x00004000]
[ 1.876593] pci 0000:01:00.0: BAR 7: assigned [io 0x1000-0x1fff]
[ 1.887691] pci 0000:02:01.0: BAR 8: no space for [mem size 0x00c00000]
[ 1.906099] pci 0000:02:01.0: BAR 8: failed to assign [mem size 0x00c00000]
[ 1.906113] pci 0000:02:01.0: BAR 9: assigned [mem 0x01100000-0x012fffff 64bit pref]
[ 1.918774] pci 0000:02:02.0: BAR 8: no space for [mem size 0x00200000]
[ 1.931851] pci 0000:02:02.0: BAR 8: failed to assign [mem size 0x00200000]
[ 1.977072] pci 0000:02:01.0: BAR 7: assigned [io 0x1000-0x1fff]
[ 1.989465] pci 0000:03:00.0: BAR 0: no space for [mem size 0x00800000]
[ 2.003888] pci 0000:03:00.0: BAR 0: failed to assign [mem size 0x00800000]
[ 2.017755] pci 0000:03:00.0: BAR 3: no space for [mem size 0x00004000]
[ 2.029624] pci 0000:03:00.0: BAR 3: failed to assign [mem size 0x00004000]
[ 2.044299] pci 0000:03:00.0: BAR 2: assigned [io 0x1000-0x101f]
[ 2.055682] pci 0000:02:01.0: PCI bridge to [bus 03]
[ 2.069480] pci 0000:02:01.0: bridge window [io 0x1000-0x1fff]
[ 2.080616] pci 0000:02:01.0: bridge window [mem 0x01100000-0x012fffff 64bit pref]
[ 2.098000] pci 0000:04:00.0: BAR 0: no space for [mem size 0x00100000]
[ 2.110462] pci 0000:04:00.0: BAR 0: failed to assign [mem size 0x00100000]
[ 2.121906] pci 0000:04:00.0: BAR 1: no space for [mem size 0x00000010]
[ 2.133701] pci 0000:04:00.0: BAR 1: failed to assign [mem size 0x00000010]
[ 2.146004] pci 0000:04:00.0: BAR 2: no space for [mem size 0x00000010]
[ 2.157794] pci 0000:04:00.0: BAR 2: failed to assign [mem size 0x00000010]
[ 2.171056] pci 0000:04:00.0: BAR 3: no space for [mem size 0x00000010]
[ 2.183276] pci 0000:04:00.0: BAR 3: failed to assign [mem size 0x00000010]
[ 2.195774] pci 0000:04:00.0: BAR 4: no space for [mem size 0x00000010]
[ 2.195783] pci 0000:04:00.0: BAR 4: failed to assign [mem size 0x00000010]
[ 2.208630] pci 0000:04:00.0: BAR 5: no space for [mem size 0x00000010]
[ 2.219979] pci 0000:04:00.0: BAR 5: failed to assign [mem size 0x00000010]
[ 2.232289] pci 0000:02:02.0: PCI bridge to [bus 04]
[ 2.249472] pci 0000:01:00.0: PCI bridge to [bus 02-04]
[ 2.254746] pci 0000:01:00.0: bridge window [io 0x1000-0x1fff]
[ 2.260887] pci 0000:01:00.0: bridge window [mem 0x01100000-0x012fffff 64bit pref]
[ 2.268695] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[ 2.273955] pci 0000:00:00.0: bridge window [io 0x1000-0x1fff]
[ 2.273967] pci 0000:00:00.0: bridge window [mem 0x01100000-0x012fffff pref]
For some reason, the bridge at 0000:00:00.0 is trying to allocate a BAR size of 18M. From my reading and understanding, the iMX6 has a limited size of 8M (and up to 16M) for BAR.
However, I found that if I force a PCI rescan, all the PCI devices seem to be able to allocate the correct BAR size:
[ 28.044120] pci_bus 0000:00: scanning bus
[ 28.056691] pci 0000:00:00.0: scanning [bus 01-ff] behind bridge, pass 0
[ 28.056722] pci_bus 0000:01: scanning bus
[ 28.068486] pci 0000:01:00.0: scanning [bus 02-04] behind bridge, pass 0
[ 28.068513] pci_bus 0000:02: scanning bus
[ 28.074891] pci 0000:02:01.0: scanning [bus 03-03] behind bridge, pass 0
[ 28.074915] pci_bus 0000:03: scanning bus
[ 28.080924] pci_bus 0000:03: bus scan returning with max=03
[ 28.080953] pci 0000:02:02.0: scanning [bus 04-04] behind bridge, pass 0
[ 28.080974] pci_bus 0000:04: scanning bus
[ 28.081071] pci 0000:04:00.0: [1ee7:babe] type 00 class 0x070001
[ 28.081218] pci 0000:04:00.0: reg 0x10: [mem 0x00000000-0x000fffff]
[ 28.081259] pci 0000:04:00.0: reg 0x14: [mem 0x00000000-0x0000000f]
[ 28.081297] pci 0000:04:00.0: reg 0x18: [mem 0x00000000-0x0000000f]
[ 28.081337] pci 0000:04:00.0: reg 0x1c: [mem 0x00000000-0x0000000f]
[ 28.081375] pci 0000:04:00.0: reg 0x20: [mem 0x00000000-0x0000000f]
[ 28.081413] pci 0000:04:00.0: reg 0x24: [mem 0x00000000-0x0000000f]
[ 28.081894] pci 0000:04:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s x1 link at 0000:00:00.0 (capable of 4.000 Gb/s with 5 GT/s x1 link)
[ 28.090898] pci_bus 0000:04: bus scan returning with max=04
[ 28.090939] pci 0000:02:01.0: scanning [bus 03-03] behind bridge, pass 1
[ 28.090974] pci 0000:02:02.0: scanning [bus 04-04] behind bridge, pass 1
[ 28.090998] pci_bus 0000:02: bus scan returning with max=04
[ 28.091022] pci 0000:01:00.0: scanning [bus 02-04] behind bridge, pass 1
[ 28.091048] pci_bus 0000:01: bus scan returning with max=04
[ 28.091060] pci 0000:00:00.0: scanning [bus 01-ff] behind bridge, pass 1
[ 28.091072] pci_bus 0000:00: bus scan returning with max=ff
[ 28.091099] pci 0000:02:01.0: bridge window [mem 0x00100000-0x000fffff] to [bus 03] add_size 200000 add_align 100000
[ 28.091119] pci 0000:01:00.0: bridge window [mem 0x00100000-0x003fffff] to [bus 02-04] add_size 200000 add_align 100000
[ 28.091132] pci 0000:00:00.0: bridge window [mem 0x00100000-0x003fffff] to [bus 01-ff] add_size 200000 add_align 100000
[ 28.091154] pci 0000:00:00.0: BAR 8: assigned [mem 0x01400000-0x018fffff]
[ 28.091171] pci 0000:01:00.0: BAR 8: assigned [mem 0x01400000-0x018fffff]
[ 28.091185] pci 0000:02:01.0: BAR 8: assigned [mem 0x01400000-0x015fffff]
[ 28.091194] pci 0000:02:02.0: BAR 8: assigned [mem 0x01600000-0x017fffff]
[ 28.091205] pci 0000:02:01.0: PCI bridge to [bus 03]
[ 28.091220] pci 0000:02:01.0: bridge window [io 0x1000-0x1fff]
[ 28.091244] pci 0000:02:01.0: bridge window [mem 0x01400000-0x015fffff]
[ 28.091264] pci 0000:02:01.0: bridge window [mem 0x01100000-0x012fffff 64bit pref]
[ 28.091302] pci 0000:04:00.0: BAR 0: assigned [mem 0x01600000-0x016fffff]
[ 28.091328] pci 0000:04:00.0: BAR 1: assigned [mem 0x01700000-0x0170000f]
[ 28.091352] pci 0000:04:00.0: BAR 2: assigned [mem 0x01700010-0x0170001f]
[ 28.091374] pci 0000:04:00.0: BAR 3: assigned [mem 0x01700020-0x0170002f]
[ 28.091396] pci 0000:04:00.0: BAR 4: assigned [mem 0x01700030-0x0170003f]
[ 28.091420] pci 0000:04:00.0: BAR 5: assigned [mem 0x01700040-0x0170004f]
[ 28.091443] pci 0000:02:02.0: PCI bridge to [bus 04]
[ 28.091469] pci 0000:02:02.0: bridge window [mem 0x01600000-0x017fffff]
[ 28.091508] pci 0000:01:00.0: PCI bridge to [bus 02-04]
[ 28.091520] pci 0000:01:00.0: bridge window [io 0x1000-0x1fff]
[ 28.091544] pci 0000:01:00.0: bridge window [mem 0x01400000-0x018fffff]
[ 28.091563] pci 0000:01:00.0: bridge window [mem 0x01100000-0x012fffff 64bit pref]
[ 28.091593] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[ 28.091601] pci 0000:00:00.0: bridge window [io 0x1000-0x1fff]
[ 28.091609] pci 0000:00:00.0: bridge window [mem 0x01400000-0x018fffff]
[ 28.091617] pci 0000:00:00.0: bridge window [mem 0x01100000-0x012fffff pref]
The list of all the devices connected to the module are:
00:00.0 PCI bridge: Synopsys, Inc. DWC_usb3 / PCIe bridge (rev 01)
01:00.0 PCI bridge: PLX Technology, Inc. PEX 8603 3-lane, 3-Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ab)
02:01.0 PCI bridge: PLX Technology, Inc. PEX 8603 3-lane, 3-Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ab)
02:02.0 PCI bridge: PLX Technology, Inc. PEX 8603 3-lane, 3-Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ab)
03:00.0 Ethernet controller: Intel Corporation I210 Gigabit Unprogrammed (rev 03)
04:00.0 Serial controller: Device 1ee7:babe (rev 01)
The configuration we have at the moment is that there’s a PCIe switch PEX8603 connected to the apalis imx6 module. Through the switch, we have an ethernet controller and a FPGA PCIe connected.
Does anyone has any idea why it would fail on the first scan, with a bogus BAR size request, but then suddenly works upon a rescan?
Thanks in advance.