Advanced Search

That's a great condensed source of network latency reducing tips

Only quibble I would have is that:

setp hm2_7i76e.0.read.timeout 100000  

Is not correct as the hm2 read timeout is scaled in percent of the servo period

EDIT: actually that is correct as values larger than 100 are interpreted as ns

But a 100 usec margin is very small on most systems and will likely result in many
unnecessarily dropped packets .

Default is 80% of servo period and this is fine normally




 

Rod share your tips please or add them to my guide ... I really like to learn something new about this topic!

Thanks

Excellent info. I've started working on a similar procedure.There are a few more optimisations but I'll test before sharing anything.
Now PREEMPT_RT is in the mainline, there are a lot of articles emerging about  RT tuning.

I wonder if it would be worth having a sticky thread that is an index to some of these great posts regarding latency and such.

To reduce network latency:

1. Make sure basic host latency setup is done (disable power management etc)
Actions:

sudo apt update
sudo apt install net-tools

Edit your GRUB config:
sudo nano /etc/default/grub

Find the line starting with:
GRUB_CMDLINE_LINUX_DEFAULT=

Append these parameters:
quiet splash isolcpus=nohz,domain,managed_irq,1 intel_idle.max_cstate=0 processor.max_cstate=1 idle=poll

Adjust isolcpus numbers later once you know your CPU layout.
Then update and reboot:
sudo update-grub
sudo reboot

Also:
Disable CPU frequency scaling:

 sudo apt install cpufrequtils
echo 'GOVERNOR="performance"' | sudo tee /etc/default/cpufrequtils
sudo systemctl enable --now cpufrequtils

Disable power-saving for NIC and USB:

 sudo ethtool -s eth0 wol d
sudo systemctl mask sleep.target suspend.target hibernate.target hybrid-sleep.target

2. If you have an intel MAC on the host PC, disable IRQ coalescing.
IRQ coalescing groups interrupts together to reduce CPU load — bad for real-time use.
Run:
sudo ethtool -C eth0 rx-usecs 0 rx-frames 0 tx-usecs 0 tx-frames 0

Make it persistent by adding it to:
sudo nano /etc/network/interfaces

Example:
auto eth0
iface eth0 inet static
    address 192.168.1.10
    netmask 255.255.255.0
    pre-up /sbin/ethtool -C eth0 rx-usecs 0 rx-frames 0 tx-usecs 0 tx-frames 0

3. Use isolcpus to run LinuxCNC on the last CPU
You “isolate” a CPU core so only LinuxCNC and its threads run there.
Example:
 If you have 4 cores (0–3), you isolate the last one:
isolcpus=3 nohz_full=3 rcu_nocbs=3

That goes into your GRUB line (as above). Then:
sudo update-grub
sudo reboot

To verify:
cat /sys/devices/system/cpu/isolated

4.  (in conjunction with #3) Pin the Ethernet IRQ to the last CPU
(and set irqbalance to one-shot mode so it doesn't mess with the IRQ pinning)
First, find your Ethernet IRQ number:
grep eth0 /proc/interrupts

You’ll see something like:
123:   12345   0   0   0   IR-PCI-MSI  eth0

→ IRQ number is 123
Then pin it:
echo 8 | sudo tee /proc/irq/123/smp_affinity

Here 8 is a bitmask (binary 1000) meaning CPU 3.
 (Use 1, 2, 4, 8, etc., depending on which CPU you isolated.)
irqbalance dynamically moves IRQs — you don’t want that after pinning.
Edit:
sudo nano /etc/default/irqbalance

Find this line:
ENABLED="1"

Change to:
ENABLED="1"
ONESHOT="1"

Then restart:
sudo systemctl restart irqbalance

irqbalance dynamically moves IRQs — you don’t want that after pinning.
Edit:
sudo nano /etc/default/irqbalance

Find this line:
ENABLED="1"

Change to:
ENABLED="1"
ONESHOT="1"

Then restart:
sudo systemctl restart irqbalance

5. Set irqbalance to One-Shot Mode
irqbalance dynamically moves IRQs — you don’t want that after pinning.
Edit:
sudo nano /etc/default/irqbalance

Find this line:
ENABLED="1"

Change to:
ENABLED="1"
ONESHOT="1"

Then restart:
sudo systemctl restart irqbalance


To check maximum network latency:
sudo chrt 99 ping -i .001 -q -c 60000 192.168.1.10
( Assuming 192.168.1.10 is the 7I76E IP address ) 

This will run for 1 minute and print statistics


Command Breakdown
sudo chrt 99 ping -i .001 -q -c 60000 192.168.1.10

Part
Meaning
sudo
Run with root privileges (needed for chrt and for such a short ping interval).
chrt 99
Runs the command with real-time priority 99 (highest possible SCHED_FIFO priority).
ping
Standard ICMP ping utility.
-i .001
Interval of 1 millisecond (1000 Hz) between packets — very fast.
-q
Quiet mode (only summary results).
-c 60000
Send 60,000 pings, which takes about 60 seconds at 1 kHz.
192.168.1.10
Target IP address (replace with your remote machine or device).


What It Tests
Latency/jitter of your Ethernet path between your host and the device at 192.168.1.10

Consistency under real-time scheduling (since chrt gives the ping thread real-time priority).

Impact of CPU isolation and IRQ pinning on network determinism.

then

How to Interpret the Results
When it finishes, you’ll get something like:
--- 192.168.1.10 ping statistics ---
60000 packets transmitted, 60000 received, 0% packet loss, time 60040ms
rtt min/avg/max/mdev = 0.120/0.135/0.210/0.015 ms

Value
Meaning
min
Best-case latency.
avg
Typical latency.
max
Worst-case latency (what we care about most).
mdev
Standard deviation (jitter).

Good results for a well-tuned LinuxCNC host:
max under 0.5 ms (ideally < 0.2 ms).

No packet loss.

mdev small (under 0.05 ms).

Bad results (what to look out for):
max spikes to several milliseconds → indicates power management or IRQ contention.

Packet loss → NIC or cable issues.

mdev large → jittery CPU or network driver.

Tips
Make sure your Ethernet link partner (the device at 192.168.1.10) responds quickly — e.g., another PC on the same subnet, or an EtherCAT master/slave device.

Run it while LinuxCNC is idle and again under load (with the machine moving) — compare the two.

For continuous monitoring, use:

 watch -n 1 "ping -q -c 1000 192.168.1.10"
 to sample shorter bursts every second.

then 

edit servo thread time to for example Ts=333 333 ns in the .ini file

finaly

edit setp hm2_7i76e.0.read.timeout 100000      # 100μs or less depends at measured latency

and check the time of servo thread which has to be less then for example < then Ts
 

hello PCW,

base thread or servo thread latency is way under 6us ... 

here are statistics 



what to do next please? regards and thanks

Hmmm, the plot thickens. If I run the qtdragon interface,as in the config file I have uploaded, I get the following error. But if I switch the UI to axis in the ini file, no other change, it works...any ideas how that can be?

Hello

I have a question. Which *.ngc files and variable entries do I need if I want to use a Rack ATC with LinuxCNC 2.9.7 and a Mesa 7i76e?

Regards,
Hilton

 

PCW you are impressive! I checked the original wiring of the Mach3 board the mill came with, and it used single ended wiring using only the negative wires for STEP and DIR, so I tried to change that.
No change when using the UI, it still shows following error. But then I tried the servo tuning test in pncconf, and the steppers are now moving

Now I just need to figure out the following error, but at least now I know that the hardware is working.

Thank you very much, your help is much appreciated!

If you get a following error, no further movement will be possible so a configuration
issue the causes a following error is s show-stopper.

Other possible reasons for no motion are wrong scaling, too short step lengths
high drive requirement (try single  ended wiring rather than differential)

Single ended wiring:

DRIVE STEP+  --> 7I76EU STEP+
DRIVE STEP-  --> 7I76EU GND

etc
 

I think you are spot on with the "enable" output comment, the motors where not locked. I disconnected "EN+ -> TB2 pin 6 +5VP" and bingo, the motors where locked. So far so good, thank you I have measured Dir+/- and the voltage here change when I jog the axis.

But alas, the motors will still not move. How ever, now I get the "Joint 0 following error" immediately when I try to move the X axis, so I suspect this is why nothing is moving. I have tried to read up on the error, and it seems to point at a misconfiguration in the PID/Acelleration/etc part of the config. But I can't really see it being SO wrong, that the motor won't start at all?

Are the drives locked?

Note that on many step drives. the "enable" input is actually a disable input
and disables the drive when powered.

For checking outputs without a oscilloscope, you can measure the DIR output and
see if it changes when jogging in different directions

A following error may indicate a setup issue and needs to be resolved.

Hi,

I have a DMC2 Mini that I am trying to convert from the Chinese Mach3 board that it came with, to a 7i76eu controlled by linuxcnc.

But I am stuck at getting the stepper motors to move. The DMC2 comes with three Nema23 57-76 motors with built-in controllers, similar to this one:
www.aliexpress.com/item/1005006172891348.html
I have wired the motors like this:
EN-  -> TB2 pin 1 GND
EN+ -> TB2 pin 6 +5VP
DR-  -> TB2 pin 4 DIR1-
DR+ -> TB2 pin 5 DIR1+
PU-  -> TB2 pin 2 STEP0-
PU+ -> TB2 pin 3 STEP0+
But nothing happens when I try to jog the axis. The DRO in the UI change, but the steppers are totally silent. After a while jogging around, I usually get a "Join 0 following error".
I can measure +5v on EN, so I believe that the axis is enabled.

I have just installed LinuxCNC from the livecd, and the configuration is made with pncconf.

Any ideas on what could be wrong, or what I can do to debug it? I don't have an oscilloscope, so I have a hard time actually confirming that the MESA card sends out pulses.

Thank you in advance!

Are you dropping packets? that is, what is:

hm2_7i76e.0.packet-error-total

A dropped packet will result in a (bogus) following error spike,
as stale stepgen position (from the previous cycle) will be compared
with the commanded position. At a 2 ms servo thread period and 125 mm/sec
velocity, a .25 mm reported following error value is possible from a dropped
packet. Note the this does not mean that there is an actual path deviation
of this magnitude, this is bounded by the max_error PID pin to ~.0127 mm
per dropped packet.

I still think the physical error is most likely mechanical as it only shows up on Y.

To diagnose stepgen errors, the best thing is to plot the following error
with halscope like this:

 


Note that this is in inches, but the velocity and acceleration are about 4x and 6X your machines values
and the inherent stepgen error is less than  0.25 micron.

Another possibility if the system latency is really bad, is loss of DPLL sync, so you might try increasing
the DPLL time from -250 to -500 usec

 

I wrote this up in chat gpt pretty quickly I think it just about covers all of the bases here of the project.

CNC Plasma Table – Electrical & Wiring Requirements

Project: Remote design of electrical system + wiring documentation for LinuxCNC plasma table
Client: TCB Metal Works

1. Project Overview

You will remotely design the complete electrical and control system for a custom CNC plasma cutting table powered by LinuxCNC.

You will create:

Complete wiring diagram set (schematic + physical routing)

Full Bill of Materials (BOM) with real part numbers

Wire gauge, shielding, grounding, and EMI mitigation plan

Bulkhead connector list + panel layout

Mesa board selection + full LinuxCNC I/O map

Integration notes for all components

Testing & commissioning checklist

This is a 100% remote engineering job — no physical inspection of hardware is required.

2. Available Power Sources

Near the machine:

220 VAC available

460 VAC available

No dedicated 110/120 VAC, which must be generated using a control transformer if needed

Engineer must design the incoming power strategy and transformer selection.

3. Motion Hardware
3.1 X & Y Axis – AC Servos (Provided)

3× A6 Series 1000W RS485 AC Servo Motor Kits

3000 rpm

3.18 Nm

17-bit absolute encoder

IP67 motors

Axis layout:

Y axis: 2 servos (dual-drive gantry)

X axis: 1 servo

Notes:

Motor-to-drive cables are already supplied by the manufacturer.

You do not need to design or evaluate those cables.

Engineer must:

Design AC mains wiring to each servo drive

Design control, enable, alarm, and ESTOP signal wiring between drives and Mesa

Include needed fuses, breakers, filters, and terminals

Provide cabinet layout and routing plan

Include proper grounding and shielding recommendations

3.2 Z Axis – Stepper Motor (Needs Driver + Cable)

PV267-D2.8AA stepper motor — Oriental Motor

Stepper driver not provided

Stepper motor cable not provided

Engineer must:

Select a compatible stepper driver suitable for PV267-D2.8AA + LinuxCNC step/dir

Define driver voltage supply requirements

Design:

Step/Dir wiring from Mesa

Motor wiring from driver to Z motor

Required cable shielding, gauge, and insulation

Connector type and strain relief

4. Control Electronics (Mesa + Power)

Engineer must select Mesa hardware capable of supporting:

3× servo axes

1× stepper axis

THC-AD

Ohmic sensing

All limit/floating switches

Plasma start / Arc OK

E-stop & safety I/O

Deliverables:

Recommended Mesa board combination (e.g., 7i76E, 7i96S, 7i85, THCAD, etc.)

Justification for chosen hardware

Complete LinuxCNC I/O mapping

Required DC power supplies (24V / 5V / others)

Proper AC distribution, fuses, breakers, and protection devices

5. Torch Height & Ohmic Sensing
5.1 THC-AD Integration

Engineer must design:

Raw arc voltage wiring from Hypertherm HT2000 to THC-AD

THC-AD signal wiring to Mesa

Required shielding and filtering

Scaling notes for LinuxCNC

5.2 Ohmic Sensing (Planned)

Engineer must:

Recommend a compatible ohmic board

Provide wiring diagram for torch → ohmic board → Mesa

Specify cable type and shielding

Document signal polarity & behavior

6. Hypertherm HT2000 Integration

Engineer must design wiring for:

Required signals

Torch start / trigger

Arc OK

Raw arc voltage (+/–) into THC-AD

Required protection

Isolation (if needed)

Surge suppression

Filtering

EMI protection

Routing rules

Arc voltage wiring must be kept separate from low-voltage control wiring

7. Limit Switches & Floating Head
X Axis

2 limit switches:

X-min

X-max

Y Axis (Dual Gantry Drive)

2 limit switches:

Y-min

Y-max

Engineer must propose a reliable gantry squaring / homing method for the two Y motors.

Z Axis

2 end-of-travel limit switches:

Z-min

Z-max

1 floating-head switch (touch-off)

Engineer must:

Select appropriate switch type (mechanical or prox)

Provide wiring diagrams to Mesa

Specify wire gauge, shielding, and cable routing

Provide LinuxCNC input mapping

Include any debouncing/filtering requirements

8. Wiring, Shielding, & EMI

Engineer must specify:

Wire gauges for:

AC mains

Servo drive feeds

Stepper driver feeds

Stepper motor wiring

Limit/floating/ohmic wires

Plasma signals

DC supply lines

Shielding requirements for:

Step/dir

Limits/floating/ohmic

Arc voltage

THC-AD lines

EMI considerations

Cable separation

Ferrite locations

Filtering recommendations

9. Grounding & Bonding

The engineer must design the grounding/bonding approach.

Important physical detail:

The water table, gantry, and control cabinet are three separate physical assemblies.

The engineer must account for this when designing:

Grounding/bonding layout

EMC/EMI mitigation

Plasma return path considerations

Ohmic sensing grounding

No grounding method (star, bus, single-point, etc.) is mandated — the engineer chooses the correct approach.

10. Control Cabinet & Bulkheads

Engineer must design:

Cabinet Layout

AC distribution, transformers, PSUs

Mesa boards

Servo drives

Stepper driver

Fuses, breakers, contactors

DIN rail layout

Ventilation/fans

Bulkhead Connectors

Used wherever practical for:

Servo cables (using glands or pass-throughs)

Z stepper cable

Limit/floating switches

Ohmic sensing

Plasma start / Arc OK

E-stop / operator controls

Ethernet access

Engineer must specify:

Connector type

Pinout

Panel cutout style

Labeling

11. Documentation Package

Engineer must deliver:

11.1 Wiring Diagram Set

Full schematic

Cabinet wiring

Field wiring

Power distribution

Grounding/bonding

Bulkhead connector pinouts

11.2 BOM

Complete list of:

Mesa hardware

Stepper driver

Transformers / PSUs

Switches

Connectors

Wire/cable

Protection devices

Cabinet hardware

11.3 I/O Map

All LinuxCNC inputs/outputs

Axis definitions

THC/ohmic mappings

E-stop & safety signals

11.4 Commissioning Checklist

Power-up verification

Servo enable/jog

Stepper test

Homing / gantry squaring

Z floating-head test

Plasma start / Arc OK

THC test

Ohmic test

EMI/noise checks