Are you 100% sure all of your motor and drive connections are correct? Nothing is crossed, So that Linuxcnc Y1 command signals and encoder feed back are connected to the drive for Y1 and that is connected to the motor power and encoders for the Y1 motor. And the same for Y2, If any of those things are confused between Y1 and Y2, wildly disastrous things could happen, and tuning would be impossible. (This really does kind of sound to me like one of the command signals or encoder feedback signals might be crossed between Y1 and Y2.)
You should be able to run the Y1 and Y2 open loop, using the PID. You simply don't connect the encoder feedbacks to the PID loop feedbacks. Instead connect the PID feedbacks to the sudo feedbacks from the hardware stepgenerators. (using P=1000 and FF1=1 (all other PID settings set to 0). If you can get that to work. Then you could try reconnecting only one of the encoder feedbacks and tuining only that one side. If you can get that to work acceptibly, then you could try adding the feedback from the 2nd motor.
