- Posts: 31
- Thank you received: 0
Ethercat random jitter fix
7 months 2 weeks ago #344141
by TAKUYA
Replied by TAKUYA on topic Ethercat random jitter fix
Installed the latter. So far so good, didnt do any major tests but multiple power cycles. Also, with the ClockSyncCycles set to 5 and syncToRefClock=true, my servos are now instantly OP. no waiting for them to be ready which caused some amplifier errors when i was too quick with the power button recently.
no idea if thats because of the new linuxcnc-ethercat version or because of those parameters.
one thing to note though, the dc-phased pin never went high for me.
no idea if thats because of the new linuxcnc-ethercat version or because of those parameters.
one thing to note though, the dc-phased pin never went high for me.
Please Log in to join the conversation.
- grandixximo
-
Topic Author
- Away
- Platinum Member
-
- Where are my paaaaants?
Less
More
- Posts: 351
- Thank you received: 2
7 months 2 weeks ago #344163
by grandixximo
---
test signature
Replied by grandixximo on topic Ethercat random jitter fix
They are same
github.com/grandixximo/linuxcnc-ethercat
this just has one small optimization
dc-phased is latency dependent, can you plot in halscope, app-phase, pll-error, and servo thread?
github.com/grandixximo/linuxcnc-ethercat
this just has one small optimization
dc-phased is latency dependent, can you plot in halscope, app-phase, pll-error, and servo thread?
---
test signature
Please Log in to join the conversation.
7 months 2 weeks ago #344171
by TAKUYA
Replied by TAKUYA on topic Ethercat random jitter fix
Ah okay gotcha.
Ill plot the mentioned signals this weekend when im back at the machine.
Latency on my host is ~20k with max jitter when opening mulitple tabs max 45k
would lowering/increasing the SyncCycles value help with this?
Ill plot the mentioned signals this weekend when im back at the machine.
Latency on my host is ~20k with max jitter when opening mulitple tabs max 45k
would lowering/increasing the SyncCycles value help with this?
Please Log in to join the conversation.
7 months 2 weeks ago #344176
by Atsu
Replied by Atsu on topic Ethercat random jitter fix
I have the same issue as TAKUYA, the dc-phased pin never goes high. I managed to fix this using Claude AI and now the dc-phased pin goes high. I have attached a detailed notes of what was changed in the code as well as some screenshots from before and after.
NOTE: this was done entirely with the help of AI and I am not qualified enough to say whether it is correct or not! Grandixximo, could you please review the notes and share your thoughts? Opening the attached HTML file in a browser, you can view the notes which were generated. It is too long to copy-paste here.
EDIT: After testing the code more thoroughly, I'm getting jitter again. It's very likely that the AI's "fix" only addressed the dc-phased pin but broke something else... I've reverted back to the master branch ( github.com/grandixximo/linuxcnc-ethercat ), and everything is working again except for the dc-phased pin.
NOTE: this was done entirely with the help of AI and I am not qualified enough to say whether it is correct or not! Grandixximo, could you please review the notes and share your thoughts? Opening the attached HTML file in a browser, you can view the notes which were generated. It is too long to copy-paste here.
EDIT: After testing the code more thoroughly, I'm getting jitter again. It's very likely that the AI's "fix" only addressed the dc-phased pin but broke something else... I've reverted back to the master branch ( github.com/grandixximo/linuxcnc-ethercat ), and everything is working again except for the dc-phased pin.
Please Log in to join the conversation.
7 months 2 weeks ago #344189
by TAKUYA
Replied by TAKUYA on topic Ethercat random jitter fix
So pll-error remains 0 for me. App phase hovering between 9800 and 10500
What i did notice, i had all 3 axis vibrate like crazy upon first Launch today. Rebooting lcnc fixed that.
Edit: did another power cycle. No Vibration but sudden clunking on all 3 axis again
What i did notice, i had all 3 axis vibrate like crazy upon first Launch today. Rebooting lcnc fixed that.
Edit: did another power cycle. No Vibration but sudden clunking on all 3 axis again
Please Log in to join the conversation.
7 months 2 weeks ago #344209
by Atsu
Replied by Atsu on topic Ethercat random jitter fix
Experienced some jitter as well today while running grandixximo's fork. I doubt there is any synchronization being done while the dc-phased pin is false and all pll-*** pins are 0. Ran sudo dmesg to see what is happening in the background. Maybe someone can notice something clear in the log:
During startup:
[ 801.691210] EtherCAT: Requesting master 0...
[ 801.691215] EtherCAT: Successfully requested master 0.
[ 801.691915] EtherCAT 0: Domain0: Logical address 0x00000000, 74 byte, expected working counter 12.
[ 801.691918] EtherCAT 0: Datagram domain0-0-main: Logical offset 0x00000000, 74 byte, type LRW.
[ 801.691932] EtherCAT 0: Master thread exited.
[ 801.691933] EtherCAT 0: Starting EtherCAT-OP thread.
[ 801.691970] EtherCAT WARNING 0: 1 datagram UNMATCHED!
During operation:
[ 807.424674] EtherCAT WARNING 0-1: Slave did not sync after 5000 ms.
[ 808.944250] EtherCAT 0: Slave states on main device: OP.
During shutdown:
[ 841.183382] EtherCAT 0: Master thread exited.
[ 841.183401] EtherCAT 0: Starting EtherCAT-IDLE thread.
[ 841.183449] EtherCAT ERROR 0-0: Failed to receive AL state datagram: Datagram initialized.
[ 841.183462] EtherCAT 0: Releasing master...
[ 841.183464] EtherCAT 0: Released.
[ 841.188247] EtherCAT 0: Slave states on main device: SAFEOP, OP + ERROR.
[ 841.196539] EtherCAT ERROR 0-0: AL status message 0x001A: "Synchronization error".
[ 841.197317] EtherCAT 0-0: Acknowledged state SAFEOP.
[ 841.208612] EtherCAT ERROR 0-1: AL status message 0x001A: "Synchronization error".
[ 841.209116] EtherCAT 0-1: Acknowledged state SAFEOP.
[ 841.220587] EtherCAT ERROR 0-2: AL status message 0x001A: "Synchronization error".
[ 841.221590] EtherCAT 0-2: Acknowledged state SAFEOP.
[ 841.256274] EtherCAT 0: Slave states on main device: PREOP.
[ 841.828318] EtherCAT WARNING: Datagram 0000000031def7fe (master-fsm) was SKIPPED 1 time.
During startup:
[ 801.691210] EtherCAT: Requesting master 0...
[ 801.691215] EtherCAT: Successfully requested master 0.
[ 801.691915] EtherCAT 0: Domain0: Logical address 0x00000000, 74 byte, expected working counter 12.
[ 801.691918] EtherCAT 0: Datagram domain0-0-main: Logical offset 0x00000000, 74 byte, type LRW.
[ 801.691932] EtherCAT 0: Master thread exited.
[ 801.691933] EtherCAT 0: Starting EtherCAT-OP thread.
[ 801.691970] EtherCAT WARNING 0: 1 datagram UNMATCHED!
During operation:
[ 807.424674] EtherCAT WARNING 0-1: Slave did not sync after 5000 ms.
[ 808.944250] EtherCAT 0: Slave states on main device: OP.
During shutdown:
[ 841.183382] EtherCAT 0: Master thread exited.
[ 841.183401] EtherCAT 0: Starting EtherCAT-IDLE thread.
[ 841.183449] EtherCAT ERROR 0-0: Failed to receive AL state datagram: Datagram initialized.
[ 841.183462] EtherCAT 0: Releasing master...
[ 841.183464] EtherCAT 0: Released.
[ 841.188247] EtherCAT 0: Slave states on main device: SAFEOP, OP + ERROR.
[ 841.196539] EtherCAT ERROR 0-0: AL status message 0x001A: "Synchronization error".
[ 841.197317] EtherCAT 0-0: Acknowledged state SAFEOP.
[ 841.208612] EtherCAT ERROR 0-1: AL status message 0x001A: "Synchronization error".
[ 841.209116] EtherCAT 0-1: Acknowledged state SAFEOP.
[ 841.220587] EtherCAT ERROR 0-2: AL status message 0x001A: "Synchronization error".
[ 841.221590] EtherCAT 0-2: Acknowledged state SAFEOP.
[ 841.256274] EtherCAT 0: Slave states on main device: PREOP.
[ 841.828318] EtherCAT WARNING: Datagram 0000000031def7fe (master-fsm) was SKIPPED 1 time.
Please Log in to join the conversation.
- grandixximo
-
Topic Author
- Away
- Platinum Member
-
- Where are my paaaaants?
Less
More
- Posts: 351
- Thank you received: 2
7 months 2 weeks ago #344237
by grandixximo
which tells me you probably do not have DC, is assignActivate set to 300 on your DC capable devices?
Your first device lacks DC, I would advise to keep the first device on your network to be DC active, or you may need to set which slave does the DC timing reference in the XML, I always keep first device DC, have not tested differently, can't say for sure.
<master idx="0" appTimePeriod="1000000" refClockSyncCycles="-1">
<slave idx="xx" type="devicewith DC" >
<dcConf assignActivate="300" sync0Cycle="*1" sync0Shift="250000"/>
</slave>Or you share your XML here, I will take a look, I think you do not have DC at all at the moment, you never even used any of my code changes at all.
---
test signature
Replied by grandixximo on topic Ethercat random jitter fix
The issues the AI found are not issues,I have the same issue as TAKUYA, the dc-phased pin never goes high. I managed to fix this using Claude AI and now the dc-phased pin goes high. I have attached a detailed notes of what was changed in the code as well as some screenshots from before and after.
NOTE: this was done entirely with the help of AI and I am not qualified enough to say whether it is correct or not! Grandixximo, could you please review the notes and share your thoughts? Opening the attached HTML file in a browser, you can view the notes which were generated. It is too long to copy-paste here.
EDIT: After testing the code more thoroughly, I'm getting jitter again. It's very likely that the AI's "fix" only addressed the dc-phased pin but broke something else... I've reverted back to the master branch ( github.com/grandixximo/linuxcnc-ethercat ), and everything is working again except for the dc-phased pin.
- Same-cycle read — No,
reads from the previous cycle's response, already received inCode:reference_clock_timeCode:lcec_read_master
- Missing
— Not needed; DC ref clock time comes from the normal sync datagramsCode:sync_monitor_queue
-
guard — Intentionally skipped in M2R mode (correct behavior)Code:sync_reference_clock
which tells me you probably do not have DC, is assignActivate set to 300 on your DC capable devices?
Your first device lacks DC, I would advise to keep the first device on your network to be DC active, or you may need to set which slave does the DC timing reference in the XML, I always keep first device DC, have not tested differently, can't say for sure.
- EK1100 — coupler, no DC
- EL1008 — digital input, no DC
- EL2008 — digital output, no DC
- 3× StepperOnline A6 — these should have DC, but only if
is set in the XML configCode:assignActivate
<master idx="0" appTimePeriod="1000000" refClockSyncCycles="-1">
<slave idx="xx" type="devicewith DC" >
<dcConf assignActivate="300" sync0Cycle="*1" sync0Shift="250000"/>
</slave>Or you share your XML here, I will take a look, I think you do not have DC at all at the moment, you never even used any of my code changes at all.
---
test signature
Please Log in to join the conversation.
- grandixximo
-
Topic Author
- Away
- Platinum Member
-
- Where are my paaaaants?
Less
More
- Posts: 351
- Thank you received: 2
7 months 2 weeks ago #344239
by grandixximo
also send this command
ethercat master
and share the output
the output should look something like
Master0
Phase: Operation
Active: yes
Slaves: 12
Ethernet devices:
Main: 88:03:8f:28:ca:33 (attached)
Link: UP
Tx frames: 78788870
Tx bytes: 65756113151
Rx frames: 78788869
Rx bytes: 65756112300
Tx errors: 0
Tx frame rate [1/s]: 1000 1000 1000
Tx rate [KByte/s]: 822.0 822.0 821.9
Rx frame rate [1/s]: 1000 1000 1000
Rx rate [KByte/s]: 822.0 822.0 821.9
Common:
Tx frames: 78788870
Tx bytes: 65756113151
Rx frames: 78788869
Rx bytes: 65756112300
Lost frames: 0
Tx frame rate [1/s]: 1000 1000 1000
Tx rate [KByte/s]: 822.0 822.0 821.9
Rx frame rate [1/s]: 1000 1000 1000
Rx rate [KByte/s]: 822.0 822.0 821.9
Loss rate [1/s]: 0 -0 0
Frame loss [%]: 0.0 -0.0 0.0
Distributed clocks:
Reference clock: Slave 0
DC reference time: 826799805203739108
Application time: 826800839056809695
2026-03-14 10:53:59.056809695
---
test signature
Replied by grandixximo on topic Ethercat random jitter fix
This seems a similar issue, please share your XML fileExperienced some jitter as well today while running grandixximo's fork. I doubt there is any synchronization being done while the dc-phased pin is false and all pll-*** pins are 0. Ran sudo dmesg to see what is happening in the background. Maybe someone can notice something clear in the log:
During startup:
[ 801.691210] EtherCAT: Requesting master 0...
[ 801.691215] EtherCAT: Successfully requested master 0.
[ 801.691915] EtherCAT 0: Domain0: Logical address 0x00000000, 74 byte, expected working counter 12.
[ 801.691918] EtherCAT 0: Datagram domain0-0-main: Logical offset 0x00000000, 74 byte, type LRW.
[ 801.691932] EtherCAT 0: Master thread exited.
[ 801.691933] EtherCAT 0: Starting EtherCAT-OP thread.
[ 801.691970] EtherCAT WARNING 0: 1 datagram UNMATCHED!
During operation:
[ 807.424674] EtherCAT WARNING 0-1: Slave did not sync after 5000 ms.
[ 808.944250] EtherCAT 0: Slave states on main device: OP.
During shutdown:
[ 841.183382] EtherCAT 0: Master thread exited.
[ 841.183401] EtherCAT 0: Starting EtherCAT-IDLE thread.
[ 841.183449] EtherCAT ERROR 0-0: Failed to receive AL state datagram: Datagram initialized.
[ 841.183462] EtherCAT 0: Releasing master...
[ 841.183464] EtherCAT 0: Released.
[ 841.188247] EtherCAT 0: Slave states on main device: SAFEOP, OP + ERROR.
[ 841.196539] EtherCAT ERROR 0-0: AL status message 0x001A: "Synchronization error".
[ 841.197317] EtherCAT 0-0: Acknowledged state SAFEOP.
[ 841.208612] EtherCAT ERROR 0-1: AL status message 0x001A: "Synchronization error".
[ 841.209116] EtherCAT 0-1: Acknowledged state SAFEOP.
[ 841.220587] EtherCAT ERROR 0-2: AL status message 0x001A: "Synchronization error".
[ 841.221590] EtherCAT 0-2: Acknowledged state SAFEOP.
[ 841.256274] EtherCAT 0: Slave states on main device: PREOP.
[ 841.828318] EtherCAT WARNING: Datagram 0000000031def7fe (master-fsm) was SKIPPED 1 time.
also send this command
ethercat master
and share the output
the output should look something like
Master0
Phase: Operation
Active: yes
Slaves: 12
Ethernet devices:
Main: 88:03:8f:28:ca:33 (attached)
Link: UP
Tx frames: 78788870
Tx bytes: 65756113151
Rx frames: 78788869
Rx bytes: 65756112300
Tx errors: 0
Tx frame rate [1/s]: 1000 1000 1000
Tx rate [KByte/s]: 822.0 822.0 821.9
Rx frame rate [1/s]: 1000 1000 1000
Rx rate [KByte/s]: 822.0 822.0 821.9
Common:
Tx frames: 78788870
Tx bytes: 65756113151
Rx frames: 78788869
Rx bytes: 65756112300
Lost frames: 0
Tx frame rate [1/s]: 1000 1000 1000
Tx rate [KByte/s]: 822.0 822.0 821.9
Rx frame rate [1/s]: 1000 1000 1000
Rx rate [KByte/s]: 822.0 822.0 821.9
Loss rate [1/s]: 0 -0 0
Frame loss [%]: 0.0 -0.0 0.0
Distributed clocks:
Reference clock: Slave 0
DC reference time: 826799805203739108
Application time: 826800839056809695
2026-03-14 10:53:59.056809695
---
test signature
Please Log in to join the conversation.
- grandixximo
-
Topic Author
- Away
- Platinum Member
-
- Where are my paaaaants?
Less
More
- Posts: 351
- Thank you received: 2
7 months 2 weeks ago #344240
by grandixximo
---
test signature
Replied by grandixximo on topic Ethercat random jitter fix
@TAKUYA
@Atsu
Please share your full XML
and the output of ethercat master command after you have started linuxcnc
@Atsu
Please share your full XML
and the output of ethercat master command after you have started linuxcnc
---
test signature
Please Log in to join the conversation.
7 months 2 weeks ago #344241
by Atsu
Replied by Atsu on topic Ethercat random jitter fix
Thank you for taking the time to go through the messages from the last few days. I suspected as much, the AI's "fix" wasn't really anything useful.
Regarding the first slave lacking DC, you are correct. I actually changed the order of slaves in the chain yesterday so that the A6 servos are now defined first. They do have assignActivate defined. However, that change didn't make any difference to the dc-phased pin or the pll-*** pins.
I have attached my ethercat-conf.xml along with HAL and INI files if they are of any help.
ethercat master command after LinuxCNC has started:
Master0
Phase: Operation
Active: yes
Slaves: 6
Ethernet devices:
Main: 10:60:4b:5b:58:34 (attached)
Link: UP
Tx frames: 146273
Tx bytes: 9824436
Rx frames: 146272
Rx bytes: 9824318
Tx errors: 0
Tx frame rate [1/s]: 800 709 308
Tx rate [KByte/s]: 93.7 84.1 32.2
Rx frame rate [1/s]: 800 709 308
Rx rate [KByte/s]: 93.7 84.1 32.2
Common:
Tx frames: 146273
Tx bytes: 9824436
Rx frames: 146272
Rx bytes: 9824318
Lost frames: 0
Tx frame rate [1/s]: 800 709 308
Tx rate [KByte/s]: 93.7 84.1 32.2
Rx frame rate [1/s]: 800 709 308
Rx rate [KByte/s]: 93.7 84.1 32.2
Loss rate [1/s]: 0 -0 0
Frame loss [%]: 0.0 -0.0 0.0
Distributed clocks:
Reference clock: Slave 0
DC reference time: 826802114159640000
Application time: 826802133398422723
2026-03-14 11:15:33.398422723
Regarding the first slave lacking DC, you are correct. I actually changed the order of slaves in the chain yesterday so that the A6 servos are now defined first. They do have assignActivate defined. However, that change didn't make any difference to the dc-phased pin or the pll-*** pins.
I have attached my ethercat-conf.xml along with HAL and INI files if they are of any help.
ethercat master command after LinuxCNC has started:
Master0
Phase: Operation
Active: yes
Slaves: 6
Ethernet devices:
Main: 10:60:4b:5b:58:34 (attached)
Link: UP
Tx frames: 146273
Tx bytes: 9824436
Rx frames: 146272
Rx bytes: 9824318
Tx errors: 0
Tx frame rate [1/s]: 800 709 308
Tx rate [KByte/s]: 93.7 84.1 32.2
Rx frame rate [1/s]: 800 709 308
Rx rate [KByte/s]: 93.7 84.1 32.2
Common:
Tx frames: 146273
Tx bytes: 9824436
Rx frames: 146272
Rx bytes: 9824318
Lost frames: 0
Tx frame rate [1/s]: 800 709 308
Tx rate [KByte/s]: 93.7 84.1 32.2
Rx frame rate [1/s]: 800 709 308
Rx rate [KByte/s]: 93.7 84.1 32.2
Loss rate [1/s]: 0 -0 0
Frame loss [%]: 0.0 -0.0 0.0
Distributed clocks:
Reference clock: Slave 0
DC reference time: 826802114159640000
Application time: 826802133398422723
2026-03-14 11:15:33.398422723
Please Log in to join the conversation.
Time to create page: 0.277 seconds
