Toradex Verdin IMX8MP Second Ethernet with AR8033 PHY on custom board

Hi,

For the 2nd ethernet interface on our custom board, we use AR8033 as pin to pin instead of KSZ phy on the verdin development board. Our device tree is as follows.

imx8mp-verdin.dtsi:

/* Verdin ETH_1 (On-module PHY) */
&eqos {
	phy-handle = <&ethphy0>;
	phy-mode = "rgmii-id";
	phy-supply = <&reg_module_eth1phy>;
	pinctrl-names = "default";
	pinctrl-0 = <&pinctrl_eqos>;
	snps,force_thresh_dma_mode;
	snps,mtl-rx-config = <&mtl_rx_setup>;
	snps,mtl-tx-config = <&mtl_tx_setup>;

	mdio {
		compatible = "snps,dwmac-mdio";
		#address-cells = <1>;
		#size-cells = <0>;

		ethphy0: ethernet-phy@7 {
			compatible = "ethernet-phy-ieee802.3-c22";
			eee-broken-100tx;
			eee-broken-1000t;
			interrupt-parent = <&gpio1>;
			interrupts = <10 IRQ_TYPE_LEVEL_LOW>;
			micrel,led-mode = <0>;
			reg = <7>;
		};
	};

	mtl_rx_setup: rx-queues-config {
		snps,rx-queues-to-use = <5>;
		snps,rx-sched-sp;

		queue0 {
			snps,dcb-algorithm;
			snps,priority = <0x1>;
			snps,map-to-dma-channel = <0>;
		};

		queue1 {
			snps,dcb-algorithm;
			snps,priority = <0x2>;
			snps,map-to-dma-channel = <1>;
		};

		queue2 {
			snps,dcb-algorithm;
			snps,priority = <0x4>;
			snps,map-to-dma-channel = <2>;
		};

		queue3 {
			snps,dcb-algorithm;
			snps,priority = <0x8>;
			snps,map-to-dma-channel = <3>;
		};

		queue4 {
			snps,dcb-algorithm;
			snps,priority = <0xf0>;
			snps,map-to-dma-channel = <4>;
		};
	};

	mtl_tx_setup: tx-queues-config {
		snps,tx-queues-to-use = <5>;
		snps,tx-sched-sp;

		queue0 {
			snps,dcb-algorithm;
			snps,priority = <0x1>;
		};

		queue1 {
			snps,dcb-algorithm;
			snps,priority = <0x2>;
		};

		queue2 {
			snps,dcb-algorithm;
			snps,priority = <0x4>;
		};

		queue3 {
			snps,dcb-algorithm;
			snps,priority = <0x8>;
		};

		queue4 {
			snps,dcb-algorithm;
			snps,priority = <0xf0>;
		};
	};
};

/* Verdin ETH_2_RGMII */
&fec {
	fsl,magic-packet;
	phy-handle = <&ethphy1>;
	phy-supply = <&reg_module_eth1phy>;
	phy-mode = "rgmii-id";
	pinctrl-names = "default";
	pinctrl-0 = <&pinctrl_fec>;
	status = "okay";

	fixed-link {
		speed = <1000>;
		full-duplex;
	};
	
	mdio {
		#address-cells = <1>;
		#size-cells = <0>;
		
		ethphy1: ethernet-phy@7 {
			reg = <0x07>;	
			compatible = "ethernet-phy-id004d.d074","ethernet-phy-ieee802.3-c22";
			qca,clk-out-frequency = <125000000>;
			qca,clk-out-strength = <AR803X_STRENGTH_FULL>;
			qca,keep-pll-enabled;
			at803x,eee-disabled;
			at803x,vddio-1p8v;
			at803x,led-act-blind-workaround;

			vddio-supply = <&reg_1p8v>;
		};
	};
};

In addition, at803x.c driver for at8033 is enabled with defconfig.

In response to the “mdio list” command in uboot I get the following result.

Verdin iMX8MP # mdio list
FEC1:
7 - Generic PHY ↔ ethernet@30be0000
ethernet@30bf0000:

After a ping attempt I get the following result.

ping 192.168.10.11
ethernet@30bf0000 Waiting for PHY auto negotiation to complete......... TIMEOUT !
phy_startup() failed: -110FAILED: -110Using ethernet@30be0000 device

Abort
ping failed; host 192.168.10.11 is not alive
Verdin iMX8MP # ping 192.168.10.11
Using ethernet@30be0000 device

Abort
ping failed; host 192.168.10.11 is not alive
Verdin iMX8MP # ping 192.168.10.11
Using ethernet@30be0000 device

Abort
ping failed; host 192.168.10.11 is not alive
Verdin iMX8MP # ping 192.168.10.11
Using ethernet@30be0000 device

Abort
ping failed; host 192.168.10.11 is not alive
Verdin iMX8MP # ping 192.168.10.11
Using ethernet@30be0000 device

Abort
ping failed; host 192.168.10.11 is not alive
**Verdin iMX8MP # mdio list**
**FEC1:**
**7 - Generic PHY <--> ethernet@30be0000**
**ethernet@30bf0000:**
**7 - Micrel ksz9131 <--> ethernet@30bf0000**

When I read the 7th address with mdio read over the ethernet@30bf0000 bus, I can communicate with the on board phy by reading the correct information. But for AR8033, when I read the 7th address over FEC1 bus, I cannot read the correct information (PHY ID). All registers are 0x0000 or 0xFFFF.

When I branch to the kernel, I get the following output and the result I obtained with ifconfig is also shared below. It works when the eth1 cable is plugged in. However, the eth0 I added can’t ping at all, even though it gets ip and is always on.

[ 6.997453] Qualcomm Atheros AR8031/AR8033 30be0000.ethernet-1:00: attached PHY driver (mii_bus:phy_addr=30be0000.ethernet-1:00, irq=POLL)
[ 7.014054] fec 30be0000.ethernet eth0: Link is Up - Unknown/Full - flow control rx/tx
[ 7.018926] imx-dwmac 30bf0000.ethernet eth1: PHY [stmmac-0:07] driver [Microchip KSZ9131 Gigabit PHY] (irq=88)

ifconfig result:
root@verdin-imx8mp-14871170:~# ifconfig
**eth0      Link encap:Ethernet  HWaddr 00:14:2D:F2:EA:82**
**          inet addr:169.254.28.147  Bcast:169.254.255.255  Mask:255.255.0.0**
**          inet6 addr: fe80::214:2dff:fef2:ea82/64 Scope:Link**
**          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1**
**          RX packets:0 errors:0 dropped:0 overruns:0 frame:0**
**          TX packets:44 errors:0 dropped:0 overruns:0 carrier:0**
**          collisions:0 txqueuelen:1000**
**          RX bytes:0 (0.0 B)  TX bytes:8360 (8.1 KiB)**

**eth1      Link encap:Ethernet  HWaddr 00:14:2D:E2:EA:82**
**          UP BROADCAST MULTICAST  MTU:1500  Metric:1**
**          RX packets:0 errors:0 dropped:0 overruns:0 frame:0**
**          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0**
**          collisions:0 txqueuelen:1000**
**          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)**
**          Interrupt:54**

**lo        Link encap:Local Loopback**
**          inet addr:127.0.0.1  Mask:255.0.0.0**
**          inet6 addr: ::1/128 Scope:Host**
**          UP LOOPBACK RUNNING  MTU:65536  Metric:1**
**          RX packets:106 errors:0 dropped:0 overruns:0 frame:0**
**          TX packets:106 errors:0 dropped:0 overruns:0 carrier:0**
**          collisions:0 txqueuelen:1000**
**          RX bytes:8722 (8.5 KiB)  TX bytes:8722 (8.5 KiB)**

**usb0      Link encap:Ethernet  HWaddr D2:B7:82:AC:18:C5**
**          UP BROADCAST MULTICAST  MTU:1500  Metric:1**
**          RX packets:0 errors:0 dropped:0 overruns:0 frame:0**
**          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0**
**          collisions:0 txqueuelen:1000**
**          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)**

Finally, there is this output during boot, but the opposite happens when you branch into the kernel, is it normal?
Net: eth1: ethernet@30be0000, eth0: ethernet@30bf0000 [PRIME]
Normal Boot

We have been trying to solve this problem for a long time. Can you help me?

Hi @eepvs !

From your description, it is hard to pinpoint what could be the problem. It could be hardware (e.g. some routing issues) and software (device tree, driver, etc.).

Since you are trying to solve this for a long time, you are most probably eager to get to a solution as fast as possible. So I really recommend you to go to a Toradex Partner: Embedded Computing | Hardware & Software Service Partners. If you would like some introduction, you can send an email to your Account Manager at Toradex and we can help you with that.

Best regards,

Thanks for your reply Henrique,

We solved the problem. MDIO pin is not connected in PCB design.

But now we have an another problem. PHY only working in 100mbps speed. When we set 1000mbps, eth0 is down. Do you have a suggestion?

Hi @eepvs !

Great to hear that you solved your first issue.

Does dmesg show some clue? Could you please share its output?

Also, can you please share the output of:

  • ip -details -statistics show eth0
  • ethtool eth0

You might want to take a look at the ethtool command to check if its sub-commands might help you to find the root cause.

Best regards,

Thanks Henrique, we solved the problem. It was a line break problem on hardware.

But now we have another problem. We can’t use two ethernet same time, The ethernet which using AR8033 PHY works properly at 1gbps. However, when I plug in the cable of the other ethernet line. The newly connected interface becomes link up and works properly. But the other ethernet constantly goes up/down at first, then crashes completely. We cannot turn the link up again without turning it on and off. Do you have any suggestions?

Hi @eepvs !

Good to know! Great news :smiley:

About your new issue: it is hard to find out what is going on without more information/context…

Just to be sure that I understood what is going on, I will write down what I (think I) understood and also some questions:

  1. Your PHY (AR8033 / Verdin ETH_2_RGMII / fec) is working well at 1Gbps, and not network cable is connected to the other PHY (Verdin ETH_1 / eqos)
    1.1. Is the cable connected to ETH_2_RGMII since before the boot? If yes, does it work if you connect it only after the boot?
  2. Inserting the network cable into Verdin ETH_1 results in the strange behavior you said, then it crashes and if you try to re-connect it, nothing happens.
    2.1. Does ETH_1 work if you have it connected (with ETH_2_RGMII also connected) before turning on the module?
    2.2. Does ETH_1 work if you have it first connected and only after you connect ETH_2_RGMII?
    2.3. Does ETH_1 work if you do connect it without having ETH_2_RGMII connected?

Could you please share some logs? It would be great if you share the logs with an indication of the cable events (e.g. making it clear when the cable was connected/disconnected).

Please, also share the output of tdx-info (Getting Device Information with Tdx-Info | Toradex Developer Center).

Best regards,

Hi Henrique, thank you for your quick reply.

I cannot share the logs because I cannot access the device at the moment. However, I can answer your questions to make it clearer.

1.1. Is the cable connected to ETH_2_RGMII since before the boot? If yes, does it work if you connect it only after the boot?

Yes, the cable is connected before booting because this line is used to communicate with another unit on the card, not outside. So it goes on the PCB, not the cable. But we also tried to link up by connecting this line with a cable after booting and it works.

2.1. Does ETH_1 work if you have it connected (with ETH_2_RGMII also connected) before turning on the module?
2.2. Does ETH_1 work if you have it first connected and only after you connect ETH_2_RGMII?
2.3. Does ETH_1 work if you do connect it without having ETH_2_RGMII connected?

Yes, ETH1 works without any down.

I will add the outputs you want.

Hello again,

I want to share som logs about problem,

First, when the device is booted with eth_2 connected, eth0 is up.

[ 14.465174] fec 30be0000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off
[ 14.472824] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

After that, ı try ping to opposite device, it was succesfully.

root@verdin-imx8mp-14871161:~# ping 192.168.1.50
PING 192.168.1.50 (192.168.1.50): 56 data bytes
64 bytes from 192.168.1.50: seq=0 ttl=64 time=1006.645 ms
64 bytes from 192.168.1.50: seq=1 ttl=64 time=6.230 ms
64 bytes from 192.168.1.50: seq=2 ttl=64 time=1.679 ms
64 bytes from 192.168.1.50: seq=3 ttl=64 time=1.674 ms
64 bytes from 192.168.1.50: seq=4 ttl=64 time=1.393 ms
64 bytes from 192.168.1.50: seq=5 ttl=64 time=1.700 ms

While the ping was in progress, I connected the eth_1 cable and received the following log result.

[ 76.929047] fec 30be0000.ethernet eth0: Link is Down
[ 80.454074] imx-dwmac 30bf0000.ethernet eth1: Link is Up - 100Mbps/Full - flow control rx/tx
[ 80.462586] IPv6: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready

Then I disconnected the eth_1 cable and when I tried to down/up eth0, I got the following result.

root@verdin-imx8mp-14871161:~# ifconfig eth0 down
root@verdin-imx8mp-14871161:~# ifconfig eth0 up
[ 138.117696] Qualcomm Atheros AR8031/AR8033 30be0000.ethernet-1:00: phy_poll_reset failed: -110
[ 138.126438] fec 30be0000.ethernet eth0: Unable to connect to phy
ifconfig: SIOCSIFFLAGS: No such device

and this problem cannot be solved without turning the device off and on. However, every time I plug in eth_1 it works fine.

Also, i want to add an information. in U-boot i can’t see eth_2 but it is working after boot. Eth_2 phy address is 0 on hardware and my dts file, is this a problem?

I am sharing boot log, as you can see in the log, the device that appears as eth0 in uboot is the interface that works as eth1 after the kernel is branched, why is it cross?

U-Boot 2022.04-6.3.0-devel+git.c71ae7141f30 (May 15 2023 - 16:20:01 +0000)

CPU: i.MX8MP[8] rev1.1 1600 MHz (running at 1200 MHz)
CPU: Industrial temperature grade (-40C to 105C) at 28C
Reset cause: POR
DRAM: 4 GiB
Core: 89 devices, 23 uclasses, devicetree: separate
WDT: Started watchdog@30280000 with servicing (60s timeout)
MMC: FSL_SDHC: 1, FSL_SDHC: 2
Loading Environment from MMC… OK
In: serial@30880000
Out: serial@30880000
Err: serial@30880000
Model: Toradex 0063 Verdin iMX8M Plus Quad 4GB IT V1.1A
Serial#: 14871161
get_tdx_eeprom: cannot find EEPROM by node
MISSING TORADEX CARRIER CONFIG BLOCKS
get_tdx_eeprom: cannot find EEPROM by node
SEC0: RNG instantiated

BuildInfo:
** - ATF 3c1583b**

Setting variant to nonwifi
Net: Could not get PHY for FEC1: addr 7
Could not get PHY for FEC1: addr 7
eth0: ethernet@30bf0000 [PRIME]
Normal Boot