iMX7D second ethernet issue with systemd-networkd

Hello everyone,

I’m experimenting some troubles in using the second ethernet on my custom board based on the Colibri iMX7.
My need is to boot up in DHCP mode and then, runtime, modify the ethernet configurations with static IPs.

I followed this article and I choose to use systemd-networkd service

Here is what I’m experiencing:

  • I can correctly see both ETH0 and ETH1 from userspace but only ETH0 can obtain DHCP IP
  • If I write config files (eth0.network and eth1.network) with static IPs, restarting the systemd-networkd service with both ethernet cable plugged in, both ETH0 and ETH1 stop sending pings. Removing ETH1 cable cause the ETH0 to work again.
  • Rebooting the device after the config file definition makes both ethernet to work great with fixed IPs
  • If I restart the systemd-networkd service with the only ETH0 ethernet cable linked in, the service restarts with no problems.

Here is the configuration for both ethernet devices in my custom device tree:

As a second PHY I am using a Micrel KSZ8041, already used for the FEC1.

// Ethernet 1
&fec1 {
	pinctrl-names = "default";
	pinctrl-0 = <&pinctrl_fec1_1588>;
	status = "okay";
	phy-handle = <&ethphy0>;
	mdio {
        #address-cells = <1>;
        #size-cells = <0>;
        ethphy0: ethernet-phy@0 {
             reg = <0>;
             micrel,led-mode = <1>;
        };
    };
};

// Ethernet 2
&fec2 {
	pinctrl-names = "default";
	pinctrl-0 = <&pinctrl_fec2_1588 &pinctrl_fec2>;
	clocks = <&clks IMX7D_ENET2_IPG_ROOT_CLK>,
              <&clks IMX7D_ENET_AXI_ROOT_CLK>,
              <&clks IMX7D_ENET2_TIME_ROOT_CLK>,
              <&clks IMX7D_PLL_ENET_MAIN_50M_CLK>;
    clock-names = "ipg", "ahb", "ptp", "enet_clk_ref";
	assigned-clocks =     <&clks IMX7D_ENET2_TIME_ROOT_SRC>,
	                        <&clks IMX7D_ENET2_TIME_ROOT_CLK>,
	                        <&clks IMX7D_ENET_PHY_REF_ROOT_DIV>;
    assigned-clock-parents = <&clks IMX7D_PLL_ENET_MAIN_100M_CLK>;
    assigned-clock-rates = <0>, <100000000>, <50000000>;
	phy-mode = "rmii";
	fsl,mii-exclusive;
	status = "okay";
	phy-handle = <&ethphy1>;
	mdio {
        #address-cells = <1>;
        #size-cells = <0>;
        ethphy1: ethernet-phy@1 {
             reg = <1>;
             micrel,led-mode = <1>;
        };
    };
};

In U-Boot i can see both PHY:
[upload|jGEND3Uw/Zdh2M/GSDm918SgQ5o=]

And here is the dmesg output:

[upload|x91yE3eKLu1rifO9O9KzGMBTLS0=]

I tried also to disable (through Yocto build) all connman services, without success.

I suppose that i need to better define (in u-boot or kernel) the presence of the second ethernet.

Any suggestion?

Thank you in advance for your help,

Luca

HI @lupo

Thanks for writing to the Toradex Community!

I followed this article and I choose to use systemd-networkd service

Is the second Ethernet Interface working well if you use regular Bsp with devicetree changes and without removing connman?

Rebooting the device after the config file definition makes both ethernet to work great with fixed IPs

Which IPs did you set?

If I restart the systemd-networkd service with the only ETH0 ethernet cable linked in, the service restarts with no problems.

What happen if you start the service with only ETH1?

Best regards,
Jaski

Hi @jaski.tx,

Is the second Ethernet Interface
working well if you use regular Bsp
with devicetree changes and without
removing connman?

Using Connman only eth0 can obtain a DHCP IP. ETH1 got an IP not in range of our subnet (I suppose is a fallback IP).
[upload|Rbp2t7SEZKycGnRZz2j9877e4J0=]

I tried also to setup a fixed IP through connman for the ETH0, but ETH1 still continue to not obtain a valid IP from DHCP.

Last test was to connect only ETH1 but still not working in DHCP…

Which IPs did you set?

When I tried static configuration, i used:

eth0 = 192.168.54.1/22

eth1 = 192.168.2.198/22

With wireshark I can see that only ETH0 accept an IP from our DHCP server. When the ETH1 receive an IP, a new DHCP Discover is sent.

[upload|vf1t8PFm9lXFLAuVVfFBVyrCTBM=]

HI @lupo

Using Connman only eth0 can obtain a DHCP IP. ETH1 got an IP not in range of our subnet (I suppose is a fallback IP).

This means, the eth1 interface does not work properly. You need to check your hardware schmematic and device-tree changes. Could you share these in a private message?

Thanks and best regards,
Jaski

Hi @jaski.tx ,

I’m sorry for the late reply but, due to lockdown measures here in Italy, I’m taking the issue up now.
I tried to connect my device in another network (at home) and both the ethernets got an address from DHCP.
Thus, I suppose the issue is at DHCP server side.
I will investigate further more.

Best regards,

Luca

Please note that, in general, connman’s goal is simply to have a internet connection. So it won’t activate a second interface should it already have a working one. You would need to disable connman or at least instruct it to ignore (a) certain interface(s).

Hi @jaski.tx, @marcel.tx ,

I’m back to you with new important updates regarding to this issue.
Investigating more deeply, I noticed a problem tied to the speed negotiated beetween the board and the network switch.
By default, the negotiated speed is 100Mbps and only the eth0 is able to obtain a dynamic IP.
Another strange thing is that I’m not always able to ping eth0 while the eth1 is also connected… Seems there is some kind of conflict beetween them.

Through the ethtool command, I tried this config:

ethtool -s eth0 speed 10 duplex full;
ethtool -s eth1 speed 10 duplex full;

and all works correctly!

So, I have 2 ethernet ports that can works correctly at a limited speed of 10Mbps and by raising up the speed to 100Mbps can bring to a connection instability.

Do you have any suggestions?

Thanks

Luca

Hello Lupo,

can you share you schematic. I like to take a look at it.

Best Regards,

Matthias Gohlke

Hi @matthias.tx ,

in our carrier board we have added a second PHY like this schematic, with an external 50MHz clock connected to the pin 106 MX7D_PAD_EPDC_BDR0__CCM_ENET_REF_CLK2.
The device tree is the one I wrote in my first question.
I think the problem is related to either the external clock definition or mdio definition.

I followed also this post but with no success.

Now I am in a situation with a stable ping on both interfaces until I unplug one ethernet. When I disconnect one cable I can see lost pings also on the other interface… after some lost ping, it resumes to reply.

Which is the correct way to define a second ethernet in a colibri imx7d?

Thanks,

Luca

Hi Lupo, can you send us your custom board so that we can test it in the lab ?

Best Regards,

Matthias