Basic probe fails on latch return

So, this is going to be a lengthy post. I'm having trouble with qt_dragon_hd and probing via the "Basic Probe" interface. Sometimes, probing fails sporadically for no reason. What happens is that when I probe towards Z- (towards the work piece), when the probe tip hits the target, it stops there and does not retract for a slow probe.

Up until yesterday, I had not been able to reproduce this error consistently. But now I can, and it might be easier to track this issue down.

I'm on 2.9.4, running a MESA card for my router. I have been able to reproduce it and have taken status dumps in between every action I do to reproduce the fault and actually resetting the fault so I can make one probe action again, instead of restarting LinuxCNC.

I have seen other people having similar problems that I think are related to this:
Thread #1: forum.linuxcnc.org/qtvcp/54027-retract-n...-works-after-a-while
Thread #2: forum.linuxcnc.org/49-basic-configuratio...e-with-qtdragon-hang

So, the first procedure for me to reproduce the issue:

• 1: Start LinuxCNC and home all axes.
• 2: Load the 3D probe and issue an "M61 T99" in MDI.
• 3: I can now probe several times without issues.
• STATUS FILE SAVE: A3-4_status_after_homing_and_several_successful_probes.txt
• 4: Issue an "S6000 M3" to start the spindle.
• 5: Issue an "M05" to stop the spindle.
• STATUS FILE SAVE: A5-6_status_after_M05.txt
• 6: Issue an "M02" to end the program (should not matter for testing, just in MDI mode).
• STATUS FILE SAVE: A6-7_status_after_M02.txt
• 7: Try to probe, fails.
• STATUS FILE SAVE: A7-_status_after_failed_probe.txt

To "reset" the issue, this works:

• 1: Issue a probing action that is bound to fail.
• STATUS FILE SAVE: B1-2_status_during_retract_halt_probe_fail.txt
• 2: Wait for the 30 sec delay to get the error message "latch return failed" (row 1508 in probe_routines.py, I think).
• STATUS FILE SAVE: B2-3_status_after_retract_halt_probe_fail.txt
• 3: Issue a "G1 Z1 F200" (the same as the probe routine that failed) in MDI. Z will not move after this command.
• STATUS FILE SAVE: B3-4_status_during_MDI_G1Z1F200_after_probe_fail.txt
• 4: Press the "STOP" button in the UI.
• STATUS FILE SAVE: B4-5_status_after_pressing_stop_button_after_probe_fail.txt
• 5: Reissue the "G1 Z1 F200"; now Z is moved to Z1.
• STATUS FILE SAVE: B5-6_status_after_successful_MDI_G1Z1F200_after_probe_fail.txt
• 6: Do a probe; this succeeds.
• STATUS FILE SAVE: B6-7_status_after_successful_secondary_probing.txt
• 7: Try a second time; this fails.

Here is my config: github.com/hmunktell/printNC/tree/dustshoe

If I run the sim.qtdragon_hd_xyz on the same machine as my CNC, probing is working fine and I have not been able to get the issue. Also, on my desktop, in a virtualbox, running the same sim, everything is fine.

I can not really read out anything else from the status files other than that the probing routine tries to do the latch return, but motion is not up to this task. It almost feels like a race condition somewhere, or a flag that is set after the first probe that is then locked from clearing when starting the spindle. But it works fine in the sim environment; it might be that the internals in LinuxCNC are not handled the same.

Unfortunately, I have no other computer to test this on right now. I'm attaching the status log files. If I can assist with further testing, I'm willing to, as this is rather cumbersome.

Best regards, Henrik

Hello,

Thank you for your efforts, I thought I was the only one.
But I did not pursue this further due to time constraints, especially since no further reactions were made to my post.
The switching states are displayed properly on my mesa card. However, if I quickly press my mesa card in a row, the mesa card reacts correctly, but the LED in QT_DRAGON_HD sometimes does not react as expected.
Possibly a timing problem with signal processing.
I have a flying button that measures both workpiece and tool length. Workpiece height is the built -in routine and tool length and change your own, but that makes no difference.

Many greetings Frank.

Hi,

Thank you for your time and response. I think the UI polls data from the STATUS data. Probably, if a signal is only high fraction of a second, the UI might not refresh quickly to show that. But I think the underlying logic in HAL is catching all states. 

I just run linuxcnc with debug and verbose output from the terminal, but nothing other than the latch return error is printed to the console during the issue  .

Can now confirm that versaprobe has the same issue for me.

This might be related
github.com/LinuxCNC/linuxcnc/issues/3486
I have something with spindle-at-speed in HAL, have not looked into it as I just see this now, and need to sleep.... If possible....

Yes! Got it working now. My spindle.0.at-speed that was taken from hy_vfd.spindle-at-speed was the problem.

hy_vfd.spindle-at-speed is not set to TRUE when MDI M05 and spindle actually comes to a stop. This behavior is not according to the docs.  linuxcnc.org/docs/html/man/man1/hy_vfd.1.html

So, instead, now I'm taking the spindle-speed-fb from hy_vfd and compare that to speed-command in a near component, and that sets spindle.0.at-speed correctly.

Now probing and movement works as expected.

github.com/hmunktell/printNC/commit/46ca...22d4761f4831403e5867

Are you saying that the spindle-at-speed signal interrupted the probe routines?

Sort of. It looks like Spindle-at-speed needs to be true for motion commands to executed. And since there are G1 commands in the probe routine, and the inconsistency in hy_vfd mentioned in my previous post, execution was halted in probe latch return (G1 move) and the timeout error occurs.

To be clear, I do not see any fault or bug in the probing routine. And I think the way linuxcnc is handling spindle-at-speed signal is correct for motion.

The docs of hy_vfd sais:
But the source code for hy_vfd mentions that the spindle also needs to be turned on for this signal to work
github.com/LinuxCNC/linuxcnc/blob/27706f...ng-vfd/hy_vfd.c#L123  

As I hade net hy_vfd.spindle-at-speed to spindle.0.at-speed, after a spindle start and stop, this signal never got TRUE when the spindle stopped, and all subsequent movements where halted. 

Interestingly, G38 moves is not dependent of spindle-at-speed signal, do not know if this is intentional or a bug in linuxcnc code.

h_munktell wrote:

Code:

[code]Interestingly, G38 moves is not dependent of spindle-at-speed signal, do not know if this is intentional or a bug in linuxcnc code.

[/code]

That does seem inconsistent, but at the same time you almost always _don't_ want the spindle to spin with the probe loaded. Especially if it's a wired probe. 
 

For other random probe failures, see also github.com/LinuxCNC/linuxcnc/pull/3537