PCIe Hot-Plugging Support with Apalis TK1

Technical Support
, ,
1

Hi,

I am trying to use the PCIe hot-plugging functionality with a broadcom wireless card.
I uses the GPIO pin gpio-250 (pcie_hotplug_detect ) for the hotplug detection. I have changed the dts file accordingly and /sys/kernel/debug/gpio shows that gpio-250 is available for the hotplug detection. My use case is as follows.

Boot TK1 with the Broadcom PCie card connected, the card got probed and the lspci output as below.

root@apalis-tk1:~# lspci
00:00.0 PCI bridge: NVIDIA Corporation TegraK1 PCIe x4 Bridge (rev a1)
01:00.0 PCI bridge: PLX Technology, Inc. PEX 8605 PCI Express 4-port Gen2 Switch (rev ab)
02:01.0 PCI bridge: PLX Technology, Inc. PEX 8605 PCI Express 4-port Gen2 Switch (rev ab)
02:02.0 PCI bridge: PLX Technology, Inc. PEX 8605 PCI Express 4-port Gen2 Switch (rev ab)
02:03.0 PCI bridge: PLX Technology, Inc. PEX 8605 PCI Express 4-port Gen2 Switch (rev ab)
04:00.0 Network controller: Broadcom Corporation Device 4355 (rev 05)
06:00.0 PCI bridge: NVIDIA Corporation TegraK1 PCIe x1 Bridge (rev a1)
07:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03)

I have loaded the broadcom driver, and unloaded it . Now I want to do a card disconnect. I switched the GPIO pin from low to high and the Pcie bus driver detects the interrupt and calls the hotplug handler and does pcie detach(drivers/pcie/host/pci-tegra.c). Now I removed the Broadcom pcie wireless card. Does lspci and it is working fine

[  178.407857] work_hotplug_handler
[  178.423972] PCIE Hotplug:DisConnected
[  178.428458] PCIE Hotplug: DisConnected
root@apalis-tk1:~# lspci
06:00.0 PCI bridge: NVIDIA Corporation TegraK1 PCIe x1 Bridge (rev a1)
07:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03)

gpio status is

 gpio-250 (pcie_hotplug_detect ) in  hi

Now I connect the card back to the pcie slot. set the GPIO to low, the bus driver detects the connection , does pci attach.

gpio status is

gpio-250 (pcie_hotplug_detect ) in  lo

I am able to load the driver for the broadcom wireless card.lspci shows all the devices properly

root@apalis-tk1:~# lspci
00:00.0 PCI bridge: NVIDIA Corporation TegraK1 PCIe x4 Bridge (rev a1)
01:00.0 PCI bridge: PLX Technology, Inc. PEX 8605 PCI Express 4-port Gen2 Switch (rev ab)
02:01.0 PCI bridge: PLX Technology, Inc. PEX 8605 PCI Express 4-port Gen2 Switch (rev ab)
02:02.0 PCI bridge: PLX Technology, Inc. PEX 8605 PCI Express 4-port Gen2 Switch (rev ab)
02:03.0 PCI bridge: PLX Technology, Inc. PEX 8605 PCI Express 4-port Gen2 Switch (rev ab)
04:00.0 Network controller: Broadcom Corporation Device 4355 (rev 05)
06:00.0 PCI bridge: NVIDIA Corporation TegraK1 PCIe x1 Bridge (rev a1)
07:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03)

So it is clear that PCie-hotplugging worked for the first round. So I repeated all the above steps. but this time I could see that on disconnection only the Intel Ethernet Gigabit Network controller got disconnected. Broadcom and the other PCI bridges still shows as connected.

[  541.195440] work_hotplug_handler
[  541.212234] PCIE Hotplug:DisConnected
[  541.216597] PCIE Hotplug: DisConnected
[  541.225337] igb 0000:07:00.0: removed PHC on enp7s0

root@apalis-tk1:~# lspci
00:00.0 PCI bridge: NVIDIA Corporation TegraK1 PCIe x4 Bridge (rev a1)
01:00.0 PCI bridge: PLX Technology, Inc. PEX 8605 PCI Express 4-port Gen2 Switch (rev ab)
02:01.0 PCI bridge: PLX Technology, Inc. PEX 8605 PCI Express 4-port Gen2 Switch (rev ab)
02:02.0 PCI bridge: PLX Technology, Inc. PEX 8605 PCI Express 4-port Gen2 Switch (rev ab)
02:03.0 PCI bridge: PLX Technology, Inc. PEX 8605 PCI Express 4-port Gen2 Switch (rev ab)
04:00.0 Network controller: Broadcom Corporation Device 4355 (rev 05)
06:00.0 PCI bridge: NVIDIA Corporation TegraK1 PCIe x1 Bridge (rev a1)
root@apalis-tk1:~# cat /sys/kernel/debug/gpio
GPIOs 0-255, platform/6000d000.gpio, tegra-gpio:
 gpio-63  (+V1.05_AVDD_HDMI_PLL) out lo
 gpio-70  (temp_alert          ) in  hi
 gpio-108 (VCC_USBO1           ) out lo
 gpio-109 (VCC_USBH(2A|2C|2D|3|) out hi
 gpio-111 (hdmi_hpd            ) in  hi
 gpio-164 (RESET_MOCI_N        ) out hi
 gpio-171 (sdhci_cd            ) in  lo
 gpio-221 (BKL1_ON             ) out hi
 gpio-233 (PEX_PERST_N         ) out hi
 gpio-235 (WAKE1_MICO          ) in  hi
 gpio-250 (pcie_hotplug_detect ) in  hi

GPIOs 1016-1023, platform/as3722-pinctrl, as3722-gpio, can sleep:
 gpio-1018 (+V3.3               ) out hi

What could be the issue? Any advice would be greatly appreciated.

Best Regards,

Pradeepkumar Soman

2

Just give an overview about the issue. I observe the following during my test.

  1. hot-plug disconnect: all the PCie devices except the Intel Gigabit Ethernet controller getting disconnected.
  2. hot-plug connect: All the devices which got disconnected connects back again.
  3. hot-plug disconnect: Only Intel Gigabit Ethernet controller getting disconnected
  4. hot-plug connect: Intel Gigabit Ethernet controller getting connected back
  5. hot-plug disconnect: all the PCie devices except the Intel Gigabit Ethernet controller getting disconnected.
  6. hot-plug connect: All the devices which got disconnected connects back again.
3

Could anyone help me, I would appreciate.
Thanks for the support
Regards,
Pradeepkumar Soman