Advanced Search

If gentelmen have any other tips for reducing latency .. share it please

I did changes for my machine and it runs for few days at 3khz without problem...

Reading the datas take less then 250us with timeout 100000ns and 250us is now limit... + Other stuff in the thread and we are under the 300us

Motion is smoother by the sound and scope profile and compensation of position in the velocity mode is faster==smoother ... It is real difference now

 

That's a great condensed source of network latency reducing tips

Only quibble I would have is that:

setp hm2_7i76e.0.read.timeout 100000  

Is not correct as the hm2 read timeout is scaled in percent of the servo period

EDIT: actually that is correct as values larger than 100 are interpreted as ns

But a 100 usec margin is very small on most systems and will likely result in many
unnecessarily dropped packets .

Default is 80% of servo period and this is fine normally




 

Rod share your tips please or add them to my guide ... I really like to learn something new about this topic!

Thanks

Excellent info. I've started working on a similar procedure.There are a few more optimisations but I'll test before sharing anything.
Now PREEMPT_RT is in the mainline, there are a lot of articles emerging about  RT tuning.

I wonder if it would be worth having a sticky thread that is an index to some of these great posts regarding latency and such.

To reduce network latency:

1. Make sure basic host latency setup is done (disable power management etc)
Actions:

sudo apt update
sudo apt install net-tools

Edit your GRUB config:
sudo nano /etc/default/grub

Find the line starting with:
GRUB_CMDLINE_LINUX_DEFAULT=

Append these parameters:
quiet splash isolcpus=nohz,domain,managed_irq,1 intel_idle.max_cstate=0 processor.max_cstate=1 idle=poll

Adjust isolcpus numbers later once you know your CPU layout.
Then update and reboot:
sudo update-grub
sudo reboot

Also:
Disable CPU frequency scaling:

 sudo apt install cpufrequtils
echo 'GOVERNOR="performance"' | sudo tee /etc/default/cpufrequtils
sudo systemctl enable --now cpufrequtils

Disable power-saving for NIC and USB:

 sudo ethtool -s eth0 wol d
sudo systemctl mask sleep.target suspend.target hibernate.target hybrid-sleep.target

2. If you have an intel MAC on the host PC, disable IRQ coalescing.
IRQ coalescing groups interrupts together to reduce CPU load — bad for real-time use.
Run:
sudo ethtool -C eth0 rx-usecs 0 rx-frames 0 tx-usecs 0 tx-frames 0

Make it persistent by adding it to:
sudo nano /etc/network/interfaces

Example:
auto eth0
iface eth0 inet static
    address 192.168.1.10
    netmask 255.255.255.0
    pre-up /sbin/ethtool -C eth0 rx-usecs 0 rx-frames 0 tx-usecs 0 tx-frames 0

3. Use isolcpus to run LinuxCNC on the last CPU
You “isolate” a CPU core so only LinuxCNC and its threads run there.
Example:
 If you have 4 cores (0–3), you isolate the last one:
isolcpus=3 nohz_full=3 rcu_nocbs=3

That goes into your GRUB line (as above). Then:
sudo update-grub
sudo reboot

To verify:
cat /sys/devices/system/cpu/isolated

4.  (in conjunction with #3) Pin the Ethernet IRQ to the last CPU
(and set irqbalance to one-shot mode so it doesn't mess with the IRQ pinning)
First, find your Ethernet IRQ number:
grep eth0 /proc/interrupts

You’ll see something like:
123:   12345   0   0   0   IR-PCI-MSI  eth0

→ IRQ number is 123
Then pin it:
echo 8 | sudo tee /proc/irq/123/smp_affinity

Here 8 is a bitmask (binary 1000) meaning CPU 3.
 (Use 1, 2, 4, 8, etc., depending on which CPU you isolated.)
irqbalance dynamically moves IRQs — you don’t want that after pinning.
Edit:
sudo nano /etc/default/irqbalance

Find this line:
ENABLED="1"

Change to:
ENABLED="1"
ONESHOT="1"

Then restart:
sudo systemctl restart irqbalance

irqbalance dynamically moves IRQs — you don’t want that after pinning.
Edit:
sudo nano /etc/default/irqbalance

Find this line:
ENABLED="1"

Change to:
ENABLED="1"
ONESHOT="1"

Then restart:
sudo systemctl restart irqbalance

5. Set irqbalance to One-Shot Mode
irqbalance dynamically moves IRQs — you don’t want that after pinning.
Edit:
sudo nano /etc/default/irqbalance

Find this line:
ENABLED="1"

Change to:
ENABLED="1"
ONESHOT="1"

Then restart:
sudo systemctl restart irqbalance


To check maximum network latency:
sudo chrt 99 ping -i .001 -q -c 60000 192.168.1.10
( Assuming 192.168.1.10 is the 7I76E IP address ) 

This will run for 1 minute and print statistics


Command Breakdown
sudo chrt 99 ping -i .001 -q -c 60000 192.168.1.10

Part
Meaning
sudo
Run with root privileges (needed for chrt and for such a short ping interval).
chrt 99
Runs the command with real-time priority 99 (highest possible SCHED_FIFO priority).
ping
Standard ICMP ping utility.
-i .001
Interval of 1 millisecond (1000 Hz) between packets — very fast.
-q
Quiet mode (only summary results).
-c 60000
Send 60,000 pings, which takes about 60 seconds at 1 kHz.
192.168.1.10
Target IP address (replace with your remote machine or device).


What It Tests
Latency/jitter of your Ethernet path between your host and the device at 192.168.1.10

Consistency under real-time scheduling (since chrt gives the ping thread real-time priority).

Impact of CPU isolation and IRQ pinning on network determinism.

then

How to Interpret the Results
When it finishes, you’ll get something like:
--- 192.168.1.10 ping statistics ---
60000 packets transmitted, 60000 received, 0% packet loss, time 60040ms
rtt min/avg/max/mdev = 0.120/0.135/0.210/0.015 ms

Value
Meaning
min
Best-case latency.
avg
Typical latency.
max
Worst-case latency (what we care about most).
mdev
Standard deviation (jitter).

Good results for a well-tuned LinuxCNC host:
max under 0.5 ms (ideally < 0.2 ms).

No packet loss.

mdev small (under 0.05 ms).

Bad results (what to look out for):
max spikes to several milliseconds → indicates power management or IRQ contention.

Packet loss → NIC or cable issues.

mdev large → jittery CPU or network driver.

Tips
Make sure your Ethernet link partner (the device at 192.168.1.10) responds quickly — e.g., another PC on the same subnet, or an EtherCAT master/slave device.

Run it while LinuxCNC is idle and again under load (with the machine moving) — compare the two.

For continuous monitoring, use:

 watch -n 1 "ping -q -c 1000 192.168.1.10"
 to sample shorter bursts every second.

then 

edit servo thread time to for example Ts=333 333 ns in the .ini file

finaly

edit setp hm2_7i76e.0.read.timeout 100000      # 100μs or less depends at measured latency

and check the time of servo thread which has to be less then for example < then Ts
 

hello PCW,

base thread or servo thread latency is way under 6us ... 

here are statistics 



what to do next please? regards and thanks

Hmmm, the plot thickens. If I run the qtdragon interface,as in the config file I have uploaded, I get the following error. But if I switch the UI to axis in the ini file, no other change, it works...any ideas how that can be?

Hello

I have a question. Which *.ngc files and variable entries do I need if I want to use a Rack ATC with LinuxCNC 2.9.7 and a Mesa 7i76e?

Regards,
Hilton

 

PCW you are impressive! I checked the original wiring of the Mach3 board the mill came with, and it used single ended wiring using only the negative wires for STEP and DIR, so I tried to change that.
No change when using the UI, it still shows following error. But then I tried the servo tuning test in pncconf, and the steppers are now moving

Now I just need to figure out the following error, but at least now I know that the hardware is working.

Thank you very much, your help is much appreciated!

If you get a following error, no further movement will be possible so a configuration
issue the causes a following error is s show-stopper.

Other possible reasons for no motion are wrong scaling, too short step lengths
high drive requirement (try single  ended wiring rather than differential)

Single ended wiring:

DRIVE STEP+  --> 7I76EU STEP+
DRIVE STEP-  --> 7I76EU GND

etc
 

I think you are spot on with the "enable" output comment, the motors where not locked. I disconnected "EN+ -> TB2 pin 6 +5VP" and bingo, the motors where locked. So far so good, thank you I have measured Dir+/- and the voltage here change when I jog the axis.

But alas, the motors will still not move. How ever, now I get the "Joint 0 following error" immediately when I try to move the X axis, so I suspect this is why nothing is moving. I have tried to read up on the error, and it seems to point at a misconfiguration in the PID/Acelleration/etc part of the config. But I can't really see it being SO wrong, that the motor won't start at all?

Are the drives locked?

Note that on many step drives. the "enable" input is actually a disable input
and disables the drive when powered.

For checking outputs without a oscilloscope, you can measure the DIR output and
see if it changes when jogging in different directions

A following error may indicate a setup issue and needs to be resolved.

Hi,

I have a DMC2 Mini that I am trying to convert from the Chinese Mach3 board that it came with, to a 7i76eu controlled by linuxcnc.

But I am stuck at getting the stepper motors to move. The DMC2 comes with three Nema23 57-76 motors with built-in controllers, similar to this one:
www.aliexpress.com/item/1005006172891348.html
I have wired the motors like this:
EN-  -> TB2 pin 1 GND
EN+ -> TB2 pin 6 +5VP
DR-  -> TB2 pin 4 DIR1-
DR+ -> TB2 pin 5 DIR1+
PU-  -> TB2 pin 2 STEP0-
PU+ -> TB2 pin 3 STEP0+
But nothing happens when I try to jog the axis. The DRO in the UI change, but the steppers are totally silent. After a while jogging around, I usually get a "Join 0 following error".
I can measure +5v on EN, so I believe that the axis is enabled.

I have just installed LinuxCNC from the livecd, and the configuration is made with pncconf.

Any ideas on what could be wrong, or what I can do to debug it? I don't have an oscilloscope, so I have a hard time actually confirming that the MESA card sends out pulses.

Thank you in advance!

HalloIch habe eine frage ?Welche   *.ngc Dateien und Var Einträge brauche ich wenn ich  ein Rack ATC verwenden möchte.Linuxcnc 2.9.7
Mesa 7i76e


mfg
Hilton