Ethercat random jitter fix

I was able to run 1.5.2 ribalda built on debian 13 on Papagno's hardware, OP are quick, and no perceptible grinding on a simple bench setup, with 0.9.5 linuxcnc-ehtercat sittner, This is not even the good PC hardware, it had 400ns of jitter, and the motors seemed fine, we will need to test better on the machine.
This was mostly to rule out kernel jitter, biggest culprit for slow OP is definitely ethercat 1.6.9
We (me and YangYang) have a refreshed 1.6.9 in the works, but not sure when that will land, squashing some bugs still, the code bases 1.5.2 riblada and 1.6.9 igh went in two separate directions, and they are hard to merge, will put best efforts in, then we will have a version for ourselves we can maintain ourselves, since igh is kinda of unresponsive and or uninterested, no clue...

Good morning everyone.
I'm continuing the discussion on the Forum, on this PC; I don't have Discord.
I ran the last tests requested by Grandixximo with two versions.
Both versions, with reference clock -1 and 1, don't work well; the axes emit noise.
With the value -1, they are very present when you move browser windows; with 1, they are less noticeable but are still there.
With halscope, I monitored the profile error of joint 0 and joint 1, and when the noise occurs, you see a spike in tracking error on both axes at the same time.
I also tried changing all the EtherCAT cables with no results.
Obviously, for those who have been following the discussion, I had to reinstall the HD with Debian 10, which works fine.

I tried optimizing the GroB command, but it doesn't work.
I tried varying synclock from 0 to 300,000 and 500,000, but it doesn't work.

Now I can't stop the customer who has been very helpful.
We could try again on the next machine, in about 3 months.

The problem with the initial delay of the LCEC pins has been resolved, but the synchronization problem is still present.

I've noticed that sometimes everything seems to be fine. You open maybe four browser windows, frecad, and so on, and everything is fine. Then you reboot, and after opening the first browser window, it starts to vibrate, and it stays that way until you reboot several times.

The problem is there, and it's significant. It means we'll be stuck with Debian 10.
I imagine some drives are more tolerant than others, but we can't use a system that only works well with some drives and not with others.

Probably, if the synchronization problems between the kernel and EtherCAT aren't resolved, in my opinion, the solution would be a hardware EtherCAT master.

I want to thank Grandixximo, RoDW, and others for their efforts.
If you'd like, once I've prepared the next machine, we can meet and work directly on the machine. I'd be willing to do anything to resolve the issue.

I remain available for any clarification.

I’m following this thread closely.But I can’t quite picture how the problem manifests itself.Could someone make a video?

I don't think a hardware master could be used with Linuxcnc in the present implementation of Ethercat. Choose this option carefully as it might require replatforming.

@papagno-source
Would love to get access to the actual machine, there were no jumps on the test rig even moving windows.

@andrax
Videos have been posted somewhere in the thread already I think, not everyone gets the same thing, some motors do a small click, some grind, like vibrate making a grinding noise, this is during motion.

That sounds like the noise produced when synchronization is lost during motion. My solution was to change the parameters to `refClockSyncCycles="-1" sync0Cycle="*2"`. After making this adjustment, I no longer experienced any synchronization loss. I am using an Inovance SV660N drive. You can also add the following line to the XML configuration to monitor the number of synchronization loss events: `<pdoEntry idx="200e" subIdx="19" bitLen="16" halPin="SyncLossCount" halType="s32" />`.


The entry `<pdoEntry idx="200e" subIdx="19" bitLen="16" halPin="SyncLossCount" halType="s32" />` is effective only for Inovance SV series drives; it cannot be used with other drive models.

What does it mean to set sync0 Cycle="*2"?
Should it be set for every drive?

I think that with a value of 2, instead of the default 1, the drive updates the PDOs after two master cycles. If so, I don't know if this is correct for machines that need to interpolate; it would be better to be faster.

It would just be a way to avoid the problem we're trying to solve.

Quite a few people have used 500 Hz servo thread instead of normal 1000 Hz on Raspberry Pi etc so I wonder how much of a difference it really makes? Ethercat would still be doing its thing internally. You'd have to test.

It could work, after lots of testing, I think the noise papagno is experiencing is mostly caused by the servo thread spiking above 1ms, so widening the cycle time could certainly help.
The main reason why Debian 10 does not grind in his case, is mostly because Debian 10 with kernel 4.19 gets lower latency, and this is pretty universal IMO, Debian 13 Kernel 6.12 has overall worse latency, although with system setup tooling that you can get from Rodw, the latency can be improved a lot.
There is only one slight latency regression with the new linuxcnc-ethercat, the modulo operation that I currently must use to do phasing is causing slightly more latency on the servo-thread, and I want to fix that, I am preparing patches to modify Linuxcnc to expose a function to allow proper native phasing, so that I can lighten up linuxcnc-ethercat to match the older performance for both positive and negative refClockSyncCycles modes. Previously I was undecided if going that path, or using Sascha Ittner method/hack of moving ecrt_master_application_time out of lcec_main, but I found that Sascha's hack causes issues with papagno's hardware, I tried this with built binaries from Sittner repo, with refClockSyncCycles -1 nothing would move, therefore I am now convinced the proper path is to have a native linuxcnc phasing function, this will require updating both linuxcnc, and rebuilding linuxcnc-ethercat against the new linuxcnc-dev headers, hopefully the patches will be ready and merged soon, will try to keep you guys updated on the progress here, I know there are people following in the background