Ethernet problems with 4 wire cabling

Hello @rudhi.tx,

i have tried both types of cable.

Dear Andreas @aweger, in case you still read this thread, did you ever resolve this issue? I’m experiencing something similar and would like to know if you found a resolution.

Best regards, Peter

Hello @psv,

Thanks for reaching out!
Could you please create a new thread with more details regarding your hardware and software setup?

1 Like

Dear Peter @psv,

Unfortunately, we have not found a working solution. We have changed our layout and now lead all pairs of lines out. If there are house installations with only four wires, then it is up to IT to configure the switches accordingly.

Best regards, Andreas

1 Like

Thans @aweger.

I debugged a bit on our system, and in the end it seems that it’s an electrical problem, probably noise related. Changing the filtering of the eth lines on the host side of the magnetics improves the performance for us. In general it seems (which also the internet seems to agree on) that 100Mbit on two pairs is less robust, so it requires more of the design and system components to be “correct”.

Hello, @psv and @aweger,

I faced the same issue with Apalis i.mx 6.
Carried board is custom design and it uses 100 Mbit isolation transformer by Pulse.
Other 2 pairs of Ethernet are left floating as described in design guide.
The set-up is more or less similar as your 2-pair cable tests.

Ethernet speed is limited to 100 Mbps in device tree. Respectively the advertised speed is as follows:

Supported ports: [ TP MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
        Supported pause frame use: Symmetric
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
        Advertised pause frame use: Symmetric
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Speed: Unknown!
        Duplex: Unknown! (255)
        Port: MII
        PHYAD: 7
        Transceiver: internal
        Auto-negotiation: on
        Supports Wake-on: d
        Wake-on: d
        Link detected: no

When Apalis is connected to a computer or non-manageable gigabit switch connection is established really quick. Tested it with several Dell, HP and Lenovo PCs/laptops and TP Link, ZyXel and Planet switches.
When manageable switch is used, link is not established. Tested it with Cisco 250 Series and HPE 1820 Series. Once auto-negotiation is turned off and speed is forced to 100 Mbps, full duplex everything works fine.

I will share with you if I found a solution.
Best regards,
Ivan

@Ivan_Gorchev,

Thanks for reporting this. We were able to reproduce the issue with a 4-wire cable and a manageable switch as you pointed out. However, we could also solve it partly by following the instructions from @emanuele.tx:

Try to remove the device tree max-speed = <100>; and run this command before connecting cable:
sudo ethtool -s eth0 advertise 0x0f

After this, the link came up and was autonegotiated to 100mbps. You could already test this.
However, then there is a chance that you might run into the other issue pointed out by @aweger, where the link goes up and down every few seconds. If that happens, should check if the pause is enabled asymmetrically (e.g. rx on/tx off):

root@apalis-imx6-10571902:~# ethtool -a eth0
Pause parameters for eth0:
Autonegotiate:  on
RX:             on
TX:             on
RX negotiated: off
TX negotiated: off

If that’s the case you might be affected by the KSZ9031RNX Silicon Errata linked here. The only solution as far as I know is the workaround proposed in the errata itself:

image

Please let me know if this helps.

Dear @rudhi.tx,

Thank you very much for your reply. The device, used in the module we have is KSZ9031RNXIC. This device is affected by KSZ9031RNX errata.

This is the result after removing max-speed = <100> from device tree and execution of ethtool -s eth0 advertise 0x0f. Pause is disabled.

root@apalis-imx6-10797840:~# ethtool eth0
Settings for eth0:
        Supported ports: [ TP MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Supported pause frame use: Symmetric
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
        Advertised pause frame use: No
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Speed: Unknown!
        Duplex: Unknown! (255)
        Port: MII
        PHYAD: 7
        Transceiver: internal
        Auto-negotiation: on
        Supports Wake-on: d
        Wake-on: d
        Link detected: no

root@apalis-imx6-10797840:~#ethtool -a eth0
Pause parameters for eth0:
Autonegotiate:  on
RX:             off
TX:             off

Unfortunately link was not established. I left the devices connected and link was established after 40 minutes. Meanwhile link goes up and down, as @aweger mentioned:

[ 3454.813197] fec 2188000.ethernet eth0: Link is Up - 100Mbps/Full - flow control off
[ 3454.813256] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[ 3454.820130] fec 2188000.ethernet eth0: Link is Down
[ 3457.434326] fec 2188000.ethernet eth0: Link is Up - 100Mbps/Full - flow control off
[ 3457.434385] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[ 3457.439511] fec 2188000.ethernet eth0: Link is Down

I’ve disconnected the wire, executed again ethtool -s eth0 advertise 0x0f and enabled the pause on both RX and TX:

root@apalis-imx6-10797840:~# ethtool -a eth0
Pause parameters for eth0:
Autonegotiate:  on
RX:             on
TX:             on

Unfortunately link was not established. I left the devices connected and link was established after 5 minutes. Meanwhile link goes up and down, as @aweger mentioned.

root@apalis-imx6-10797840:~# ethtool eth0
Settings for eth0:
        Supported ports: [ TP MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Supported pause frame use: Symmetric
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
        Advertised pause frame use: No
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Link partner advertised link modes:  10baseT/Half 10baseT/Full
                                             100baseT/Half 100baseT/Full
        Link partner advertised pause frame use: No
        Link partner advertised auto-negotiation: Yes
        Link partner advertised FEC modes: Not reported
        Speed: 100Mb/s
        Duplex: Full
        Port: MII
        PHYAD: 7
        Transceiver: internal
        Auto-negotiation: on
        Supports Wake-on: d
        Wake-on: d
        Link detected: yes
root@apalis-imx6-10797840:~# ethtool -a eth0
Pause parameters for eth0:
Autonegotiate:  on
RX:             on
TX:             on
RX negotiated:  off
TX negotiated:  off

Hello @Ivan_Gorchev,

Supported pause frame use: Symmetric

From this, it looks like the Asymmetric Pause capability bit is disabled. Could you please also try this to manually disable the autonegotiation and turn the Rx and Tx off:

ethtool -A eth0 autoneg off rx off tx off

Hello, @rudhi.tx

Here is the result:

root@apalis-imx6-10797840:~# ethtool -a eth0
Pause parameters for eth0:
Autonegotiate:  off
RX:             off
TX:             off

root@apalis-imx6-10797840:~# ethtool eth0
Settings for eth0:
        Supported ports: [ TP MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Supported pause frame use: No
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
        Advertised pause frame use: No
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Speed: Unknown!
        Duplex: Unknown! (255)
        Port: MII
        PHYAD: 7
        Transceiver: internal
        Auto-negotiation: on
        Supports Wake-on: d
        Wake-on: d
        Link detected: no
root@apalis-imx6-10797840:~#

Again link goes up and down:

[ 1217.633602] fec 2188000.ethernet eth0: Link is Up - 100Mbps/Full - flow control off
[ 1217.633661] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[ 1217.640739] fec 2188000.ethernet eth0: Link is Down
[ 1220.179207] fec 2188000.ethernet eth0: Link is Up - 100Mbps/Full - flow control off
[ 1220.179265] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[ 1220.186702] fec 2188000.ethernet eth0: Link is Down

Could you please check the KSZ9031RNX errata:

Module 5: Auto-Negotiation link-up failure / long link-up time due to default FLP interval setting
The device’s Auto-Negotiation FLP (Fast Link Pulse) burst-to-burst timing defaults to 8ms. IEEE Standard specifies this
timing to be 16ms +/-8ms. Some link partners, such as Intel G-PHY controllers, were observed in bench tests to have
tighter timing requirements that need to detect the FLP interval timing centered at 16ms.

Is there a way to check if BSP 5 Ethernet physical driver changes the default configuration?

Best regards,
Ivan Gorchev

Hi @Ivan_Gorchev !

Have you also tried it on a Toradex carrier board? Either Ixora or Apalis Evaluation Board?

If not, please test it there and let us know the outcome.

Best regards,

Hi, @henrique.tx!

We performed the test with Ixora V1.2A and Cisco 250 Series switch. 1Gbps/full duplex link is established within a second.
Unfortunately this is not our use case. Ixora board provides wiring of all 4 pairs. In our design we would like to use the Apalis module in 100Mbps/full duplex mode with 2 pairs wired as described in Apalis Carfrier Board design guide, chapter 2.4.2.3 10/100Mbit Ethernet Schematic Example.

We performed second test with Ixora V1.2A, Cisco 250 Series switch, 4 pair patch cable, Apalis auto negotiation advertised links modes limited to 10/100Mb, half/full duplex via:

ethtool -s eth0 advertise 0x0f

No link established. The result in dmesg is endless link up/down.

Best regards,
Ivan

1 Like

Hi @Ivan_Gorchev

Can you try the following test script together with phytool?

Copy test.sh and phytool to the home directory of your module and execute the script. On my modules it shows the following values, maybe for you they are different:

root@apalis-imx6-10575737:~# ./test.sh
Current register values
0x0006
0x1a80
New register values
0x0006
0x1a80

phytool (694.5 KB)
test.sh (851 Bytes)

Afterwards you can restart autoneg by doing:

ifconfig eth0 down && ifconfig eth0 up

Regards,
Stefan

1 Like

Hi @stefan_e.tx

Thank you very much for the script. The results are the same as yours:

root@apalis-imx6-10797840:/home# ./test.sh
Current register values
0x0006
0x1a80
New register values
0x0006
0x1a80
root@apalis-imx6-10797840:/home#

With kind regards,
Ivan

Hi @Ivan_Gorchev

Unfortunately, I’m running out of ideas. The script makes sure that the proposed workaround is applied. However, it was already applied before. Because we can not reproduce the issue, it is hard to debug it.

What you can try is to use phytool to read out some registers to figure out what is going on:

./phytool read eth0/7/0x1
./phytool read eth0/7/0x4
./phytool read eth0/7/0x5

Is e.g. one of the fault bits set? Maybe you find some other diagnostics that help, here is the link to the datasheet:

Regards,
Stefan

Hi, @stefan_e.tx ,

Thank you very much for the message. Please find the result below:

./phytool read eth0/7/0x1
0x7949
./phytool read eth0/7/0x4
0x01e1
./phytool read eth0/7/0x5
0x0000

It’s really strange register 5 is 0x0000.

After 30-40 minutes link is established. Register 5 changes to:

./phytool read eth0/7/0x5
0xc1e1

Best regards,
Ivan

Hi @Ivan_Gorchev

I have no explanation for this. Can you try to also read some more registers?

./phytool read eth0/7/0x6
./phytool read eth0/7/0x7
./phytool read eth0/7/0x8
./phytool read eth0/7/0x9
./phytool read eth0/7/0xa

Also when the Link is down, what do you see on the other side? Could it be that it is configured for 1GBit/s when the issue happens (through autoneg)? Do you have ethtool available to read out more information from there?

Regards,
Stefan

Hi, @stephan.tx,

Please find the value of registers below:

./phytool read eth0/7/0x6
0x0004
./phytool read eth0/7/0x7
0x2001
./phytool read eth0/7/0x8
0x0000
./phytool read eth0/7/0x9
0x0000
./phytool read eth0/7/0xa
0x0000

I’m not really familiar with Cisco CLI and the provided tools, but will do everything possible to check what happens from it’s side.

With kind regards,
Ivan

Hi @Ivan_Gorchev

This all looks like the link partner is not detected. I’m not sure why this can happen… However, the last thing you could try is to check that bit 6 in register 0 is set to 0 as written in https://ww1.microchip.com/downloads/en/devicedoc/00002117f.pdf section 3.8.

./phytool read eth0/7/0x0
./phytool write eth0/7/0x0 <make sure bit 6 is 0>

Also, you can check if there is traffic on differential pairs A (pins 2, 3) and B (pins 5, 6) with an oscilloscope. Just make sure that you see something happening on the pins.

Regards,
Stefan

Hi, @stefan_e.tx,

thank you for the hints. Here is the result:

./phytool read eth0/7/0x0
0x1140
./phytool write eth0/7/0x0 0x1100
./phytool read eth0/7/0x0
0x1100

Unfortunately link state didn’t change.
I will perform the measurements with oscilloscope later today.

Best regards,
Ivan