Using transformer with 3 28.5V secondary windings to supply 57VAC/80VDC

I think you blow the current sensing resistor. There is no current limiting resistor in today's stepper drivers.
In normal operation, it's impossible, the board will measure the current, and open the transistor when it reach the maximum.
Either the current sensing is dead, either the power stage is damaged, and can't stop the current (a damaged mosfet is always closed). It's more probably the power stage.

Turns out there's a pseudo-short in my flyback diode board between VCC and one of the motor terminals.

Probably the sense resistor.

The fact that I accidentally set the driver to max current probably was not helpful.

I'll replace the sense resistor ASAP.

I'm also promising to never, ever run the driver without a circuit breaker.