[SOLVED] Mesa 7i76E - Joint Following Error - Error Finishing Read

Hello

I'm trying to get a 7i76e system up and running. I'm converting a shopsabre router from WinCnc to Linux CNC.

The router has Mitsubishi servos driven by Mitsubishi drives in step/direction mode.

After running the Pncconf wizard I can start AXIS but I immediately get an error

hm2/hm2_7i76e.0: error finishing read! iter=6
hm2/hm2_7i76e.0: error finishing read! iter=6

Which I can clear. Then when I attemp to jog an axis I get these errors.

note: MAXV max: 1.000 units/sec 60.000 units/min
note: LJOG max: 1.000 units/sec 60.000 units/min
note: LJOG default: 0.250 units/sec 15.000 units/min
note: jog_order='XYZ'
note: jog_invert=set([])
joint 0 following error
emc/task/taskintf.cc 942: Error on joint 0, command number 95
task: main loop took 0.155724 seconds
joint 1 following error
emc/task/taskintf.cc 942: Error on joint 1, command number 162
joint 2 following error
emc/task/taskintf.cc 942: Error on joint 2, command number 229
command (EMC_JOG_STOP) cannot be executed until the machine is out of E-stop and turned on
task: main loop took 0.119844 seconds
task: main loop took 0.114117 seconds
joint 2 following error
emc/task/taskintf.cc 942: Error on joint 2, command number 296


After exit I'm getting the following error

Shutting down and cleaning up LinuxCNC...
Running HAL shutdown script
task: 75482 cycles, min=0.000026, max=0.155724, avg=0.010071, 3 latency excursions (> 10x expected cycle time of 0.010000s)
ioctl SIOCDARP: Network is unreachable
hm2_eth: HostMot2 ethernet driver unloaded
hm2: unloading
Note: Using POSIX realtime

Not sure why I'm getting such a high latency. This CPU has a 5i25/7i76 installed that works with another router. Not sure where to go from here?



Any help would be appreciated.

Attachments:

hal
ini
ping_test
latency_test

These are serious errors:

hm2/hm2_7i76e.0: error finishing read! iter=6
hm2/hm2_7i76e.0: error finishing read! iter=6

Once these happen, LinuxCNC will not be operable
(its recoverable but should never happen)

These usually mean that the Ethernet read function timed out multiple times
(default timeout is 80% of the servo period)

You can get an idea of how close you are to the limit by printing all the tmax values
with:

halcmd show param *.tmax

Thought you may need to slow the servo thread to avoid the communication loss


Also, what does the command:

ip a

report?

The basic latency is pretty poor, are all power management option disabled in the BIOS?

I just did a reboot to double check my BIOS. The Power Saving option is disabled


ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 94:c6:91:d5:8f:3a brd ff:ff:ff:ff:ff:ff
inet 10.10.10.11/8 brd 10.255.255.255 scope global noprefixroute enp1s0
valid_lft forever preferred_lft forever
3: wlp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether d0:c6:37:73:d6:ec brd ff:ff:ff:ff:ff:ff
inet 192.168.2.45/24 brd 192.168.2.255 scope global dynamic noprefixroute wlp4s0
valid_lft 258655sec preferred_lft 258655sec
inet6 fe80::4e1c:71e1:5320:4841/64 scope link noprefixroute
valid_lft forever preferred_lft forever
4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:6b:bc:0d:10 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever
* tmax

halcmd show param *.tmax
Parameters:
Owner Type Dir Value Name
26 s32 RW 0 hm2_7i76e.0.read-request.tmax
26 s32 RW 3288120 hm2_7i76e.0.read.tmax
26 s32 RW 100308 hm2_7i76e.0.write.tmax
19 s32 RW 23512 motion-command-handler.tmax
19 s32 RW 344594 motion-controller.tmax
29 s32 RW 18290 pid.s.do-pid-calcs.tmax
29 s32 RW 27185 pid.x.do-pid-calcs.tmax
29 s32 RW 20013 pid.y.do-pid-calcs.tmax
29 s32 RW 20663 pid.z.do-pid-calcs.tmax
20 s32 RW 3426059 servo-thread.tmax

26 s32 RW 3288120 hm2_7i76e.0.read.tmax

3288120 clock cycles is about 0.84 ms so likely many packets
are being timed-out and dropped

If the Ethernet interface on your PC is a Intel device you need to
disable IRQ coalescing as shown in the hm2_eth manual page

If not and you have all power saving options disabled (and all C states > C1 disabled)
It may be that you will need to lower the servo thread rate to say 500 Hz (2 ms)
If you do this you will also need to set all the PID P values in the ini file to 500

Below is the output from lspci. Ethernet controller is a Realtek, power saving options are off so I will change the servo thread

Do I change the servro thread rate here in the ini ?

[EMCMOT]
EMCMOT = motmod
COMM_TIMEOUT = 1.0
SERVO_PERIOD = 1000000


00:00.0 Host bridge: Intel Corporation Xeon E3-1200 v6/7th Gen Core Processor Host Bridge/DRAM Registers (rev 06)
00:02.0 VGA compatible controller: Intel Corporation HD Graphics 630 (rev 04)
00:14.0 USB controller: Intel Corporation 200 Series/Z370 Chipset Family USB 3.0 xHCI Controller
00:14.2 Signal processing controller: Intel Corporation 200 Series PCH Thermal Subsystem
00:16.0 Communication controller: Intel Corporation 200 Series PCH CSME HECI
00:17.0 SATA controller: Intel Corporation 200 Series PCH SATA controller [AHCI mode]
00:1c.0 PCI bridge: Intel Corporation 200 Series PCH PCI Express Root Port (rev f0)
00:1c.6 PCI bridge: Intel Corporation 200 Series PCH PCI Express Root Port (rev f0)
00:1d.0 PCI bridge: Intel Corporation 200 Series PCH PCI Express Root Port (rev f0)
00:1f.0 ISA bridge: Intel Corporation 200 Series PCH LPC Controller (B250)
00:1f.2 Memory controller: Intel Corporation 200 Series/Z370 Chipset Family Power Management Controller
00:1f.3 Audio device: Intel Corporation 200 Series PCH HD Audio
00:1f.4 SMBus: Intel Corporation 200 Series/Z370 Chipset Family SMBus Controller
01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)
02:00.0 PCI bridge: Texas Instruments XIO2001 PCI Express-to-PCI Bridge
03:00.0 DPIO module: Device 2718:5125 (rev 01)
04:00.0 Network controller: Intel Corporation Intel Dual Band Wireless-AC 3165 Plus Bluetooth (rev 99)

Here is the output from sudo ethtool -c enp1s0

Should all the values be zero?


Coalesce parameters for enp1s0:
Adaptive RX: off TX: off
stats-block-usecs: 0
sample-interval: 0
pkt-rate-low: 0
pkt-rate-high: 0

rx-usecs: 0
rx-frames: 1
rx-usecs-irq: 0
rx-frames-irq: 0

tx-usecs: 102
tx-frames: 4
tx-usecs-irq: 0
tx-frames-irq: 0

rx-usecs-low: 0
rx-frame-low: 0
tx-usecs-low: 0
tx-frame-low: 0

rx-usecs-high: 0
rx-frame-high: 0
tx-usecs-high: 0
tx-frame-high: 0

Yes. you would change:

SERVO_PERIOD = 1000000

to

SERVO_PERIOD = 2000000

and all joint:

P = 1000.0

statements to:

P = 500.0


The Realtek driver does not have a IRQ coalescing option so needs no tweeks

Oops yes P gains should be 500 for a 2 ms servo thread

I update my INI (attached) with the update P and SERVO_PERIOD values but no luck.

I verified all the cstates are disabled. Should I enable cstate 1?

Here is the line in my grub to shut off all cstates

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_idle.max_cstate=0 processor.max_cstate=0 idle=poll"


Here is the output from cpupower monitor


| Nehalem || Mperf
CPU| C3 | C6 | PC3 | PC6 || C0 | Cx | Freq
0| 0.00| 0.00| 0.00| 0.00|| 99.46| 0.54| 3911
1| 0.00| 0.00| 0.00| 0.00|| 99.46| 0.54| 3910