2nd ethernet port eth1 on Apalis i.mx8

Android 10
Apalis IMX8QM V1.1B

We have a custom board using an Apalis i.mx8 Quad Max running official Android 10.

We are trying to use both eth0 and eth1 ports. We have a 2nd KSZ9131 phy on our custom board. We can bring up each phy individually OK (ping etc) but cannot bring up and use both phy’s.

We have the 2nd phy RGMII pins set to 2.5 volts and the LDO on Apalis board set to 2.5 volts. This all works OK.

(1)
service dhcpcd_eth0 /system/bin/dhcpcd -ABKL eth0
class main
disabled
oneshot

service iprenew_eth0 /system/bin/dhcpcd -n eth0
class main
disabled
oneshot

Works OK.

(2)
service dhcpcd_eth0 /system/bin/dhcpcd -ABKL eth1
class main
disabled
oneshot

service iprenew_eth0 /system/bin/dhcpcd -n eth1
class main
disabled
oneshot

Works OK.

(3)
service dhcpcd_eth0 /system/bin/dhcpcd -ABKL eth0 eth1
class main
disabled
oneshot

service iprenew_eth0 /system/bin/dhcpcd -n eth0 eth1
class main
disabled
oneshot

Does NOT work OK.

The issues appears to be -
(1) Bringing up second ethernet port automatically
(2) The ipv4 address of the 2nd port

apalis_8qm:/ # ifconfig eth1 up
[ 281.500581] Microchip KSZ9131 Gigabit PHY 5b050000.ethernet-2:07: attached PHY driver [Microchip KSZ9131 Gigabit PHY] (mii_bus:phy_addr=5b050000.ethernet-2:07, irq=POLL)
apalis_8qm:/ # [ 284.592335] fec 5b050000.ethernet eth1: Link is Up - 100Mbps/Full - flow control rx/tx
[ 284.600353] IPv6: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready

apalis_8qm:/ $ ifconfig eth0
eth0 Link encap:Ethernet HWaddr 00:14:2d:67:d5:03 Driver fec
inet addr:192.168.0.92 Bcast:192.168.0.255 Mask:255.255.255.0
inet6 addr: fe80::c7fe:c946:c301:ef9f/64 Scope: Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:912 errors:0 dropped:0 overruns:0 frame:0
TX packets:52 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:143444 TX bytes:5459

apalis_8qm:/ # ifconfig eth1
eth1 Link encap:Ethernet HWaddr 00:14:2d:77:d5:03 Driver fec
inet6 addr: fe80::214:2dff:fe77:d503/64 Scope: Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:107 errors:0 dropped:0 overruns:0 frame:0
TX packets:9 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:16389 TX bytes:746

apalis_8qm:/ # ping -I eth0 8.8.8.8
PING 8.8.8.8 (8.8.8.8) from 192.168.0.92 eth0: 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=60 time=49.1 ms

apalis_8qm:/ # ping -I eth1 8.8.8.8
ping: Warning: source address might be selected on device other than eth1.
PING 8.8.8.8 (8.8.8.8) from 0.0.0.0 eth1: 56(84) bytes of data.
From 127.0.0.1: icmp_seq=1 Destination Host Unreachable

&fec1 {
pinctrl-names = “default”, “sleep”;
pinctrl-0 = <&pinctrl_fec1>;
pinctrl-1 = <&pinctrl_fec1_sleep>;
fsl,magic-packet;
fsl,mii-exclusive;
fsl,rgmii_txc_dly;
phy-handle = <&ethphy0>;
phy-mode = “rgmii-rxid”;

mdio {
	#address-cells = <1>;
	#size-cells = <0>;

	ethphy0: ethernet-phy@7 {
		compatible = "ethernet-phy-ieee802.3-c22";
		interrupt-parent = <&lsio_gpio1>;
		interrupts = <29 IRQ_TYPE_LEVEL_LOW>;
		micrel,led-mode = <0>;
		power-domains = <&pd IMX_SC_R_BOARD_R0>;
		reg = <7>;
	};
};

};

&fec2 {
pinctrl-names = “default”;
pinctrl-0 = <&pinctrl_fec2>;
fsl,magic-packet;
nvmem-cells = <&fec_mac1>;
nvmem-cell-names = “mac-address”;
fsl,mii-exclusive;
fsl,rgmii_txc_dly;
phy-handle = <&ethphy1>;
phy-mode = “rgmii-id”;

phy-reset-duration = <100>;
phy-reset-gpios = <&pca9535_1 10 GPIO_ACTIVE_LOW>;  // PCA9535 gpio pin 10
phy-reset-post-delay = <100>;

mdio {
	#address-cells = <1>;
	#size-cells = <0>;

	ethphy1: ethernet-phy@7 {
		compatible = "ethernet-phy-ieee802.3-c22";

		micrel,led-mode = <0>;
		power-domains = <&pd IMX_SC_R_BOARD_R0>;

		reg = <7>;
	};
};

};

Hi @dhack ,

Thanks for the detailed question!

The issue itself is really strange since both ETH are working separated. Is it possible to do a test following the example presented on the imx8qm-mek.dts device tree? There NXP used the same MDIO to configure both fecs.

I’m not an expert on Android, but it can also be a problem on the software side, if you create separated services for both eth0 and eth1, do you face the same issue? Also, what happens if you manually give IP for them, do you see the same issue?

Please also share the entire dmesg log too.

Best regards,
Daniel Morais

Hi Daniel,

Thank you for the quick reply.

We are using separate mdio busses for the two phy’s. The fec0 mdio bus is on the Apalis module along with the phy. The fec1 mdio bus is on our custom board along with the phy.

I don’t think the mdio bus is the issue since -
(1) both busses are probed OK

[ 3.981747] libphy: fec_enet_mii_bus: probed
[ 3.987639] fec 5b040000.ethernet eth0: registered PHC device 0
[ 4.218216] libphy: fec_enet_mii_bus: probed
[ 4.223965] fec 5b050000.ethernet eth1: registered PHC device 1

(2) if I bring up a service for only fec0 OR fec1 everything works OK. It is just bringing up both devices that there is an issue.

I created separate services for fec0 and fec1 and the problem persists when both services are created together in init.rc.

One service, either fec0 OR fec1 in init.rc is OK.

When I set manually the ip address I have the same issue when I try to ping.

I think the clue is the fact when I have two services in init.rc I always have to manually bring up eth1. It is like something in dhcpcd doesn’t allow more than 1 ethernet port. I cannot find the source code for dhcpcd, I suspect it is a library built in.

apalis_8qm:/ # ifconfig eth1 up
[ 90.046573] Microchip KSZ9131 Gigabit PHY 5b050000.ethernet-2:07: attached PHY driver [Microchip KSZ9131 Gigabit PHY] (mii_bus:phy_addr=5b050000.ethernet-2:07, irq=POLL)
apalis_8qm:/ # [ 93.145188] fec 5b050000.ethernet eth1: Link is Up - 100Mbps/Full - flow control rx/tx
[ 93.153206] IPv6: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready

apalis_8qm:/ # ifconfig eth1 192.168.0.91

apalis_8qm:/ # ifconfig eth1 broadcast 192.168.0.255

apalis_8qm:/ # ifconfig eth1
eth1 Link encap:Ethernet HWaddr 00:14:2d:77:d5:03 Driver fec
inet addr:192.168.0.91 Bcast:192.168.0.255 Mask:255.255.255.0
inet6 addr: fe80::214:2dff:fe77:d503/64 Scope: Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2099 errors:0 dropped:0 overruns:0 frame:0
TX packets:26 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:322563 TX bytes:1744

apalis_8qm:/ # ping -I eth1 8.8.8.8
PING 8.8.8.8 (8.8.8.8) from 192.168.0.91 eth1: 56(84) bytes of data.
From 192.168.0.91: icmp_seq=1 Destination Host Unreachable
From 192.168.0.91: icmp_seq=2 Destination Host Unreachable

You can see there is TX/RX packets so something is happening…

Here is the eth0 ifconfig which comes up automatically -

apalis_8qm:/ $ ifconfig eth0
eth0 Link encap:Ethernet HWaddr 00:14:2d:67:d5:03 Driver fec
inet addr:192.168.0.92 Bcast:192.168.0.255 Mask:255.255.255.0
inet6 addr: fe80::35ed:4608:82f0:6d20/64 Scope: Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:87 errors:0 dropped:0 overruns:0 frame:0
TX packets:38 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:17216 TX bytes:4387

Here is the full dmesg when I set two individual services in init.rc
dmesg individual service.txt (57.1 KB)

Something I have changed has stopped me ever getting eth1 to work, this was working!!

I am working back through my changes, it may give me some clues.

Hi @dhack ,

Thanks for the information!

Were you able to make eth1 work back?

On your device-tree it is being used the parameter fsl,rgmii_txc_dly is, can you please try using fsl,rgmii_rxc_dly as the example presented by NXP? Also, please make a test not using any rgmii_dly parameter please.

What is strange is that the second PHY isn’t coming up by default, you should not need to execute the ifconfig eth1 up to start communication between the PHY and the MDIO. Can you please check if the Android is someway blocking it on the system boot? This question on the NXP community may help you with this.

Best regards,
Daniel Morais

Hi Daniel,

I got the 2nd port working by disabling fec1 in the device tree -

//&fec1 {
// status = “okay”;
//};

&fec2 {
status = “okay”;
};

fec2 (0x5b050000) then comes up as eth0 -

[ 31.235878] Microchip KSZ9131 Gigabit PHY 5b050000.ethernet-1:07: attached PHY driver [Microchip KSZ9131 Gigabit PHY] (mii_bus:phy_addr=5b050000.ethernet-1:07, irq=POLL)
[ 34.319774] fec 5b050000.ethernet eth0: Link is Up - 100Mbps/Full - flow control rx/tx
[ 34.327800] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

On closer inspection dhcpcd makes no difference. It looks like the issue is in Linux/Device tree. So still the issue is getting fec1 & fec2 (eth0 & eth1) both coming up and working together

OK, I have worked out how to get both interfaces up at boot. You need to change to this -

<!--For Android we support eth0 eth1 now -->
<string translatable="false" name="config_ethernet_iface_regex">eth\\d</string>

in file /device/toradex/imx8q/apalis_8qm/overlay/frameworks/base/core/res/res/values/config.xml

But now I don’t get ipv4 address for eth0 or eth1!!! Still some more work to do.

I think you can set parameters in this config.xml

Hi @dhack ,

Sorry for the delay in answering!

I’m glad to hear that you were able to enable both ethernets.

Were you able to advance on the issue with the eth’s not getting the ipv4 address?

Best regards,
Daniel Morais

I have had a bit of help from NXP. I started looking at this again today.

Changing and adding these lines -

<!--For Android we support eth0 & eth1 now -->
<string translatable="false" name="config_ethernet_iface_regex">eth\\d</string>

<string-array translatable="false" name="config_ethernet_interfaces">
    <item>eth0;12,13,14,15,16,18,19</item>
    <item>eth1;12,13,14,15,16,18,19</item>
</string-array>

<!-- List of regexpressions describing the interface (if any) that represent tetherable

to -
/device/toradex/imx8q/apalis_8qm/overlay/frameworks/base/core/res/res/values/config.xml

brings both ports up OK at boot and most of the time brings up ipv4 address for eth1

I’m looking at why sometimes when I detach then attach ethernet cable to eth1 the port does not get an ipv4 address

Hi @dhack ,

I’m glad to hear that you were able to advance.

Please let me know if you need any further assistance.

Best regards,
Daniel Morais