Intermittent Z Axis Spike with Yaskawa Servos, 5i25/6i25 and 7i77

Folks,

I have an issue on a 5 axis, 6 joint machine that I’m finishing up. The machine is an XYYZAB configuration that is taking advantage of the new joints/axes capabilities in master. The X, Y1, Y2, A, and B axes all work just fine and all axes have been tuned and work smoothly. The problem I’m having is that the Z axis occasionally experiences a motion spike, even at standstill. When I say occasionally, it is something that may happen every few minutes or it can go an hour or more between events.

At first, I thought it was an encoder noise issue. A differential capture with a 4 channel scope showed some high frequency noise on the encoder lines, but not enough that I would expect it to be a problem with differential signals. Nonetheless, I’ve gone back over things and taken steps to filter any common mode noise and installed ferrite chokes on all low speed and analog signals. I even went so far as to purchase and install the galvanic isolator for between the host card and the 7i77. I’ve tried 3ft and even 18 inch cables between the host card and the 7i77. This thing really has me stumped.

The real kicker is that halscope shows the f-error of the joint and corresponding PID counter-response in sync, presumably as a result of the 5i25 read/write, PID calcs, and scope all running in the same thread. Anyone have any ideas on how to proceed? I have swapped out the servo drive and motor as well just to make sure it wasn’t In the hardware and I still see the same occasional glitch.

Also pertinent, I have done extensive load / latency testing in the computer and latency never goes above 7000. I have, however seen an occasional serial DOIT error stating that it didn’t complete, but these don’t correspond to the spikes that I’m seeing on the Z axis. I’ll also add that if I use SigmaWin to interface the drive directly, it never exhibits the issues. That leaves me with only the loop facing linuxcnc that is in play.

Any thoughts on where to go from here? The halscope traces I’ve captures so far don’t really tell me whether a spike in f-error or a spike in PID output is happening first.

Here is what I'm seeing in halscope. You can see the sharp front edge on the encoder velocity spike but a much more damped f-error curve, which is why I initially looked at this as the encoder inputs on the 7i77 picking up on some kind of a brief noise burst. I've gone over just about everything I can think of at this point and can't seem to drag this gremlin out into the light. I'm no tuning guru, so by no means am I ruling out the possibility of having made an error in PID tuning somewhere.

Can you re-number your axis in hal so Z is moved to another encoder/ analog out?
That would check if this is specific to a 7I77 channel or external hardware

Encoder noise seems unlikely to cause such a repeatable glitch and should be possible to eliminate
with encoder input filtering

I can move it over in place of the B axis temporarily to see if there is something funky going on with that channel on the 7i77. I'll let you know what shakes out of that test.

OK, so I have bad news and I have bad news... There is definitely something wrong with the 7i77. I moved the Z axis to encoder 5; and still saw the issue. I then moved the analog out over to 5 and the drive didn't enable. A quick check with the meter showed that the enable component (I'm assuming a FET) isn't activating to enable the drive. That's a problem, but its gets even more strange. I then decided to move the drive to analog 3 because my A axis is on it and hooked up, so I know that output works, right? Not so fast. I plugged in, changed the hal, looked like testing could commence. I jogged around a bit in joint mode to make sure things were all good and they appeared to be so until I hit the home axis button and Z started happily wandering in the wrong direction. Yep, you read that correctly. It jogs in the right orientation and with no changes to the ini file, it goes backwards when homing and plugged into encoder 5 and analog 0. I never noticed a problem with the A axis using encoder3 and analog0, but then I don't have homing to a sensor and index pulse set up yet on that axis. Before I go down the path of declaring the 7i77 as dead hardware, I'm wondering if there isn't something squirrely going on with software or firmware here.

I think its time for a whiskey and coke... I'm having a hard time accepting what just turned up whiel testing this.

I then moved the analog out over to 5 and the drive didn't enable. A quick check with the meter showed that the enable component (I'm assuming a FET) isn't activating to enable the drive.

This is expected ( Axis 5 is normally used for spindle control so has a independent enable (axis 0..4 have a common enable ) )

The homing in the wrong direction error Is much more likely a hal or setup error than software of firmware

OK, so hm2_5i25.0.7i77.0.1.spinena. Got it. I moved everything over to 5 and we'll see if I get the spike.

As for homing, I generally would agree with you, but here's the catch. I'm not touching the INI file at all and the only thing I'm changing int the HAL are the encoder and analog numbers, Z jogs and homes fine on encoder4/analog4 as well as encoder5/analog5 now that i tied in spinena. I'm going to set up halscope and let it sit for a while to see if I get the same spiking.

Well, good new and bad news. The good news is that moving the Z axis to encoder5 and analog 5 seems to have resolved the problem. The bad news is that it means the 7i77 looks like it needs to be replaced.

At the risk of being told this is impossible, here is what I decided to try, figuring it couldn't hurt if I was staring at having to replace the 7i77 anyway. I grabbed a copy of the 7i77 firmware and reflashed the card. However impossible this may seem, the reflash corrected the homing reversal i saw before when using analog0. I plugged the Z axis back into encoder 4 and analog 4 and will leave it sit idle overnight with halscope running to check if the axis spikes again. Best case, maybe the sserial firmware on the card was somehow corrupted?

No such luck. As of this morning, I have a triggered halscope with the dreaded spike.