OK I realized I was being dumb and the line inverting the Z-count was right there.
I tried ilowpass and I saw the following things:
1. "Scale" is terrible. It just skips mpg pulses, which is detrimental to the feel and utility of it. e,g. "scale=3.0" on 0.01in/tick means the first tick moves 0.01", the second nothing, the third nothing, then the fourth moves 0.01". So, 1.0 is the only good value for this.
2. If I reduce gain to the point where it seems to be doing real filtering, I can get a "joint 2 following error", which I have not seen for a long time, and I'm perplexed why this would be. With a 1ms servo thread:
setp ilowpass.0.scale 1.0
setp ilowpass.0.gain 0.05
net jog-counts_in xhc-hb04.jog.counts => ilowpass.0.in
net jog-counts_out ilowpass.0.out => axis.0.jog-counts axis.1.jog-counts axis.2.jog-counts
With gain set to 0.1, it doesn't get a following error, but smoothing performance isn't very good either.
axis.2.jog-counts is still subject to accel and velocity limits, right? I have no problem with running fast 3D carving g-code, or keyboard, or mdi, or non-ilowpassed mpg, even when spun wildly.
I suppose when spun wildly, ilowpass would slow the gantry response leading to a larger accumulation of mpg counts. But how could that break the joint following? I've had the router set with slow velocities and high mpg distance-per-tick and it caches a lot of ticks (and distance) and continued to move for like 5 sec after the mpg wheel was stopped, but never did this cause a following error.
EDIT: Really ilowpass DOES NOT improve things in any significant way. What I'm seeing is it's just slowing down an accumulation of pulses, but that's not the problem here. Like the coarse 0.1"/step scale, with a machine with high accel, you turn the knob slowly and the pulses don't get backed up, but then the machine feels EXACTLY the same as it does a high accel for each 0.1" pulse, then stops, then gets a new pulse command. You can add a very low gain but it's not going to help. It does nothing to resolve the shaking.
I need to vary acceleration here, and lowpassing doesn't achieve that.
EDIT2: OK, I see what they're doing, you're supposed to scale up a pulse by like 1000x, lowpass it, and divide by 1000 to achieve the same scale. Thing is, xhc-hb04.jog.counts is already calculated by the xhc-hb04 mux depending on what speed is selected. So I can't hardcode in a constant like the integrator's guide uses. I don't see the exposure of the scale=1,2,3 selection, just the selected scale (1,10,100). Actually I really need that because filtering is different on different scales and potentially axes. But I don't see the exposure.
I could divide by 1000x again at:
net jog-scale xhc-hb04.jog.scale => axis.0.jog-scale axis.1.jog-scale axis.2.jog-scale
But can you do an arithmetic divide on an float inside of a HAL? Google is telling me that's... awkward. Well, there's a mult2 component, but you have to add it to a thread, and I'll need to create another component, invert, to tie it directly to ilowpass.0.scale.
What gets me is that when going into 0.001" stepsize, lowpass is still dividing by 1000x, making a 0.000001" stepsize inside lowpass. I don't see how that breaks anything right now, but it looks pretty crazy. That lowpass divider just isn't correct for the 0.001" stepsize but there's no way to adjust it by stepsize. I actually could make the divider change by axis, by creating 3x lowpass all fed by xhc-hb04.jog.counts and outputting to independent mult2 and then to each axis.x.jog-counts. But there's not a lot of need for that.