Mesa 7i76E and YL620 VFD: Reference Voltage Not Dropping to 0 and Maxing at 8V

Hi,I have two boxes for my self-built CNC. One contains the Mesa card, stepper drivers, and 24V/36V power supplies, while the other contains the VFD. Both boxes are connected with a ~2m cable. I believe it is connected correctly. Spindle+ to 10V, Spindle Out to V11, and Spindle- to GND.Just by powering everything up, the spindle out reads roughly 1.2V, and if I start the spindle, it reads 8.1V. Also, the VFD shows ~20-40Hz in the off state and ~350-370Hz in the on state. That is obviously not what I want.

I tried setting 03.06 to 150 and 03.07 to 800, but that didn't help
 Did I miss something? Is the cable simply too long?

If you get 1.2V on the spindle output  without LinuxCNC even running,
There either a wiring error or the 7I76E spindle output has been damaged
(most likely the OP-AMP U26)

I tested my setup with cables as short as possible. Everything worked as expected. So the long cables are probably the culprit. Maybe capacity of the cables? or Resistence (0.2ohm)?

I ordered usb to modbus converters, hopefully they are more stable

The resistance of the cables in not important, the issue is most likely
EMI from the VFD. This may be improved with a common mode choke
(large ferrite bead) on the three wires going to the VFD. It also may be
that your voltmeter cannot read DC voltages accurately  in the presence
of large amounts of EMI

A USB/MODBUS adapter will likely also be susceptible to EMI so standard
EMI control measures may be needed in either case. These measures include line
filters on the incoming AC line power, common mode chokes on the interface
wiring and common mode chokes on the UVW motor wiring.