Advanced Search

Replaced RPi 5 with HP desktop running LinuxCNC 0.29.1, connected to 7i96s over CAT6 cable.

sudo chrt 99 ping -i .001 -q 10.10.10.10

Quick test shows 0% packet loss for 40,000 packets.

And a longer test:

2312351 sent, 2312350 received  <<<<<<<  Loss of 1 packet

rtt min/avg/max/mdev = 0.177/0.266/0.455/0.019 ms

I think this is particularly concerning. The ethernet connection just seems unstable. As pointed out, there simply should not be packet loss.

I will try replacing the Pi 5 with a PC tomorrow to trying eliminating the 7i96s as part of the problem.

cnc@linuxcnc:~/Downloads/7i96s/configs/hostmot2$ sudo chrt 99 ping -i .001 -q 10.10.10.10
PING 10.10.10.10 (10.10.10.10) 56(84) bytes of data.

--- 10.10.10.10 ping statistics ---
7234 packets transmitted, 7233 received, 0.0138236% packet loss, time 7240ms
rtt min/avg/max/mdev = 0.101/0.106/0.132/0.002 ms

After a bit of poking around, I found a directory

/boot/broadcom

which contains a lot of files including

cmdline.txt

and

config.txt

I have added the igb.EEE=0 to the end of cmdline.txt as this has "isolcpus" in it.

Full file =

console=serial0,115200 console=tty1 root=PARTUUID=486f86bd-02 rootfstype=ext4 fsck.repair=yes loglevel=5 net.ifnames=0 processor.max_cstate=1 isolcpus=2,3 firmware_class.path=/lib/firmware/updates/brcm rootwait cfg80211.ieee80211_regdom=NZ igb.EEE=0

Will now reboot...

Here is part of dmesg.

[    0.000000] Kernel command line: coherent_pool=1M 8250.nr_uarts=1 pci=pcie_bus_safe snd_bcm2835.enable_compat_alsa=0 snd_bcm2835.enable_hdmi=1 bcm2708_fb.fbwidth=1920 bcm2708_fb.fbheight=1080 bcm2708_fb.fbdepth=16 bcm2708_fb.fbswap=1 smsc95xx.macaddr=D8:3A:DD:E8:87:4E vc_mem.mem_base=0x3fc00000 vc_mem.mem_size=0x40000000  console=ttyAMA10,115200 console=tty1 root=PARTUUID=486f86bd-02 rootfstype=ext4 fsck.repair=yes loglevel=5 net.ifnames=0 processor.max_cstate=1 isolcpus=2,3 firmware_class.path=/lib/firmware/updates/brcm rootwait cfg80211.ieee80211_regdom=NZ igb.EEE=0

dmesg | grep EEE shows only the line above (e.g. no errors with EEE in them).

Any packet loss means serious driver/hardware issues.
I would not expect to see any loss even after weeks of uptime.

It would be prudent to disable EEE in the kernel with  the boot parameter I mentioned.
From what I read, some cards it is always on and they do not respect the ethtool settings anyway.
Networking Including the ethercat generic driver is not real time 
Ethercat has a very limited range of network drivers that are real time, but I don't think they are distributed in their debs. YOU need to build fro  source (please check me here as I might be wrong)

I have tried several CAT cables (5, 5e, 6) between Pi 5 and 7i96s and it seems that ethernet comms just stops after some short period of time.

(I am using CAT 6 below.)

Whenever it stops, the green ethernet connector LEDs both remain steady ON (they flash during data transfer).

Once they are steady ON, PINGing the 7i96s results in no reply.

cnc@linuxcnc:/usr/sbin$ sudo chrt 99 ping -i .001 -q 10.10.10.10
PING 10.10.10.10 (10.10.10.10) 56(84) bytes of data.

--- 10.10.10.10 ping statistics ---
94246 packets transmitted, 93613 received, 0.671647% packet loss, time 100046ms
rtt min/avg/max/mdev = 0.101/0.113/0.286/0.007 ms
^Ccnc@linuxcnc:/usr/sbinping-q 10.10.10.10
PING 10.10.10.10 (10.10.10.10) 56(84) bytes of data.

Initially there was only 1 packet loss (approx. 1 in 30,000), then the ethernet comms stopped and the packet loss is obviously 100% after that.

WHILE I AM TYPING THIS the comms surprisingly just restarted, indicated by the ping commands receiving a reply.

And after about 100 pings, comms stopped again (both green LEDs steady ON.

And a few minutes later the pings have started again

Ok, bit of a learning curve here... so EtherCat is a protocol over an ethernet physical layer designed for cycle times; ≤ 100 μs with low communication jitter for precise synchronization purposes; ≤ 1 μs.

en.wikipedia.org/wiki/EtherCAT

I'm presuming that Mesa cards use this?

So you are saying that due to this protocol, it is unlikely that EEE would be negotiated, as that would assume the Mesa card supported EEE. The implication therefore would be that the RPi 5 would NOT be using EEE since it would not be negotiated during initial configuration?

ethtool is not installed on the Pi image (0.29.2 IIRC) but I installed it using sudo apt-get install ethtool. However, it won't run...

First, the OS cannot find it (whereis says /usr/sbin) and second, executing ./ethtool --show-eee eth0 results in "netlink error: Operation not supported".

Someone said that ethtool has been deprecated and to use nmcli however I'm not sure if it is possible to access EEE data using nmcli.

FWIW, here is another page discussing EEE (energy efficient ethernet).

askubuntu.com/questions/1411536/how-do-i...-on-my-ethernet-card

EEE is a problem for low-latency, reliable networking, but it seems odd that it'd *ever* be on for Ethercat hardware. It's supposed to be negotiated between devices as part of the autonegotiation process (along with duplex, flow control, and so forth), and I'd be really surprised if many Ethercat devices used hardware that was able to negotiate for EEE.

What does `ethtool --show-eee <device>` show, where <device> is your Ethercat Ethernet interface?

I don't have a RPi 5 handy, but running against a PC with a Realtec RTL8125 connected to an EK1100, I see:

Interestingly, I see more or less the same thing on the port connected to a Unifi switch.  No EEE there, either.  FWIW, RPi 4s look pretty much identical here.  And the one device that I have handy that is connected to a switch that *can* support EEE apparently has EEE support entirely disabled in the driver.

If you *do* see EEE problems with Ethercat hardware, consider throwing a Beckhoff EK1100 or similar in front of your device, maybe?  EEE is negotiated on a per-link basis, and even if you have a device that (somehow) wants to negotiate EEE, then throwing a non-EEE device in the middle should disable it entirely unless it's *really* broken.

The link was broken, so here it is again.

www.juniper.net/documentation/us/en/soft...s-interface-eee.html

rodw wrote:

It may be worth trying this!
forum.linuxcnc.org/9-installing-linuxcnc...i-5?start=110#293279
 

Thanks rodw, I will test this setting tomorrow.

0x2102 wrote:

Now try to reducing those two settings:

sudo ethtool -C eth0 rx-usecs 0
sudo ethtool -C eth0 tx-usecs 0

If that errors out, try 1 or the lowest number the driver accepts.
 

Thanks for suggestion.
I set bot values to 0.
Do you have any suggestion, how to verify, if its working?
I tried 250 000 ns thread and it did not make any difference (random first drive failing to get to OP)

Hmm, wasn't a bad guess. Doing it at the kernel level as I suggested should be more stable. Please try and report back

Ouch.

forums.raspberrypi.com/viewtopic.php?t=360924

It may be worth trying this!
forum.linuxcnc.org/9-installing-linuxcnc...i-5?start=110#293279