Spindle won't hit max RPM

I feel like I'm taking 2 steps forward, 1 step back. Trying to set up my milling machine with 7i76e board. I'll try to keep this short and sweet.
I had the spindle working as expected with t gear ratios. Max 3500rpm and 10000rpm based on 2 different pulleys. Everything was good, but I also wanted to set up rigid tapping and also closed loop spindle feedback. Well, through reading this forum over and over, I figured out the PID feedback and the rigid tapping. But in the process, now the spindle suddenly won't hit full speed in high gear. It used to go to 10000, no problem. No it hits ~7500 . I checked with a cheapo tachometer and it is indeed about 7500. The Halscope also shows 10000 commanded, 7500 feedback and an error of about 2500. So even though it sees the error, it's not correcting. At 7500, increasing the spindle speed to 200% does not do anything. It's like it just won't output any more to the VFD. VFD readout is what you would expect for the RPM.

So I obviously screwed something up in my attempts to fix the encoder issues and PID.
Attached is HAL and INI. Much praise and thanks for any help.

The encoder scale looks very low, correct for a 1 slot encoder
with quadrature = a semi circle disk.

The encoder disc is 4 slot. (low, I know. And the reason my rigid tapping is jittery). I thought it was pulse x4 for quad. But if I set the encoder scale to 16 it's off by a factor of 4. Linux readout = 500 and handheld tach reads 2000. I haven't figured that one out either.

A scale of 16 is correct for a 4 slot disk in quadrature
I would look at your encoder signals in halscope

photos.app.goo.gl/upemsjWDrc6s46PdA

Halscope looks good to me. A and B alternating with an index every 4 pulses.

Thanks for helping, btw.

"setp hm2_7i76e.0.encoder.04.counter-mode 1"

You have the spindle encoder set for up/down counter mode
instead of quadrature

Progress!!! RPM readout is correct. But PID will no longer adjust spindle speed (which is way off).

Checked encoder counts. The velocity was negative when rotating CW. So I reversed the encoder scale to -16.0. Velocity and position are now positive, but hm2_7i76e.0.encoder.04.count is negative. Is this why PID won't work?

I think was where I started this whole mess. PID wasn't working and so I started shifting negatives and positives, 0's and 1's to get that to work, and screwed everything in the process.

Tested rigid tapping, it did attempt it but couldn't keep up with the spindle at 1700rpm and threw a following error. Commanded was 500rpm

Note that encoder counts are not scaled so you should not expect them to change
based on the sign of the encoder scale. The good thing is that it doesn't matter
since you are not using encoder counts but rather encoder position (which is scaled)

I would back up a bit and try open loop control (FF0 = 1.0 all other PID parameters = 0)
just to verify that that works correctly

Brilliant! Starting over with PID fixed the issue. A simple P-2, I=0, D=0 gives decent results.

Thank you so much for the help.

BTW, I tried out my simple threading program. For 13 TPI (2mm pitch) 300rpm was a little jittery. I'm assuming because of the low encoder pulse count.
500rpm seemed usable. I tried 1500 for kicks and it overshot the hole by about an inch. Luckily I was cutting in air.

The reason for the low encoder count is that I mistakenly chose optical sensors with a 1khz freq limit. At 10,000 rpm a disc with 50 pulses was not producing nice square waves and the RPM readout was dancing all over the place. I'll have to look at getting some new sensors with higher pulse rate.