Frozen if absolute encoder cable detached?

I am running 2.8 on our mill. It has absolute encoders with SSI interface(Mesa 7i74). Today we noticed that if we detach absolute encoder cable while control is powered(it was an accident not intentional unplugging) LinuxCNC got "frozen" Estop did not react, none of buttons on user interface reacted. We got an error stating that configuration of limits is incorrect. Maybe it is a bug?


Tero

You should get a following error and that should at the least at the minimum disable all drives
( not sure what the GUI/HAL file dependencies are but with my sample files and Axis, this
requires restoring "machine power (F2) before continuing )

The configuration limits error may be something in LinuxCNC that does not deal with large position errors properly
( It may not be possible to get this error with quadrature encoders so its not well tested )

This also brings up an issue I've discovered with the absolute encoder module: the module currently
always assumes that absolute encoders can wrap, This is required for single turn absolute encoders to extend the count
to multiple revolutions but can cause problems with multi-turn and linear absolute encoders. The problem is this: if you are using a multi-turn or linear absolute encoder and you get transient bad data (say by unplugging the encoder)
this can cause the wrap detection logic to add/subtract what it thinks is a full turn ( 24 bits for a 24 bit encoder! )
and even when the data is correct, ( encoder re-plugged ) the absolute encoder value is still wrong

This should never happen on a multi-turn or linear absolute encoder so Andy Pugh has modified
the absolute encoder code to include a multi-turn option that disables the wrap logic

In addition, I think because of the possibility of transient errors on non error checked protocols like
SSI, I think there should be some minimal sanity checking in the absolute encoder driver
perhaps an acceleration limit that discards obviously bogus data and sets a bad data flag

I got an error report after forcing quit for LinuxCNC. Machine was at sill when this happened but I noticed Z-axis slowly moving while UI was frozen. So drives were enabled... I will check if I still can find crash log somewhere tomorrow. But it was a bit hasty situation, And I was totally amazed of "What is going on".


Tero

That's rather scary, with crazy SSI data and HAL servo thread running I dont see how you cannot get a
following error and instantly lose drive enables.

(and if the servo thread quit running you should get a watchdog time-out on both the FPGA and the sserial remotes)

I cannot duplicate this with either 2.7.8 or master (The freeze may be a GUI issue)
If I set a crazy feedback value as you would get with a unplugged SSI encoder, ( I used 2e9 ) I get

joint 0 following error
Exceeded positive soft limit on joint 0

on 2.7.8

and

joint 0 following error
emc/task/taskintf.cc 943: Error on joint 0, command number 1776
On Soft Limit
Identity kinematics are MISCONFIGURED
Exceeded POSITIVE soft limit (500.00000) on joint 0

Stop, fix joints and axis LIMITS, then Restart

On master

In ether case the amp enable is set false

in your hal file you have

net x-enable joint.0.amp-enable-out
net x-enable hm2_5i25.0.7i83.0.1.analogena0

This should disable your drive _IF_ your drive is enabled via the 7I83s ENA pins

You can duplicate this kind of fault easily by unlinking the feedback pin and setting it to a crazy number:

halcmd unlinkp joint.0.motor-pos-fb
halcmd setp joint.0.motor-pos-fb 2000000000

I am trying to remember this as accurately as I can:

Problem was with Z-axis(joint 2). It was reading something like -2421.xxx in DRO(G54) It should of been quite near 349.000 and I have -419.000 in G54 Z offset. So absolute position should of been -70.000(There was an error of about 2770 mm and with our resolution it comes to 2770 * 10000). I was trying to change from manual to MDI when I noticed that those buttons did not react(hard switches connected to halui pins). Then I tried from Gmoccapy via touchscreen but it was "frozen". After that I saw that E-stop did not react either(also hard switch). So I force quit LinuxCNC. After that I got LinuxCNC crash report(that usually comes at startup if something is wrong in hal config). But I was too hasty and lost it. After that I restarted LinuxCNC and saw an error about incorrect limits configuration. After start DRO reading was same. So I went to inspect mechanical things(like if axis was on hard limit) and then I saw this axis slowly moving to negative direction. At that point I did shutdown PC completely, switched off mains and waited. And then I checked encoder connectors and saw that Z-encoder connector was detached. Plugged it in restarted and everything was ok. And yes I do have enable pins in use. I cannot be 100% sure this is how it went but this is my best memory image.

Tero