Questions about milling machines with linear scales

Glass scales intended for machine tools have a carriage inside the read head that rides directly on the scale with little ball bearings. The read head is isolated from the carriage by a ball joint on the end of a piece of spring wire. This keeps the gap between the scale and reticle constant even when the motion of the slide is not perfectly straight and flat. The presence of the ball joint introduces backlash into the assembly.

I'm using tenths gauges to measure everything. The machine itself is old and worn. The spindle is surprisingly stiff, I can get a couple of tenths if I pull on it hard enough. The X axis is also pretty good, but the Y has a big dip in the center. I'm planning on moving the limit switches in so I can tighten the gibs further without risk of wedging.

Slop from the table rocking on loose ways, will be magnified depending on the position of actual milling vs the pivot point (which would normally be centered on the drive screw nut.) The further away from the pivot point the spindle is, the greater the change seen, and may not even be visible to the linear scale depending on how/where the scale is mounted to the axis and the read heads distance from that pivot point. (Conversely, if the scale is mounted so that it sees more movement from the rocking than the spindle, the control would compensate and over correct from the perspective of the spindle.)

The table (X axis) has dovetail ways and a ball screw down the center. The saddle (Y axis) has box ways and a ball screw on the side. The Y axis scale is mounted on the same side as the ball screw. The spindle is a about 6 inches away from the scale and ball screw. So it seems likely the Y axis is the issue. I'll see how much I can tighten the gib and try again.

Tightening the gibs has reduced (but not eliminated) the error. Pulling on the table to prevent the rocking improved it a little more but resulted in the 45 degrees being different.

Slop and play in a mechanical system can never truly be eliminated. Here is where the law of diminishing returns starts to rear its ugly head. Pick all of the low hanging fruit, and hopefully the results will be good enough. From there it gets harder/more expensive to improve, and you will pay more for smaller and smaller increments of improvements. You may need to resort to scraping ways, improving ball screw mounts (the mounting bearings for screws are notorious sources of slop), higher precision ball screws (less worn).

(At some point you might be better off cutting your losses and getting a better condition machine, or just decide to live with what you've got.)

I had a badly worn (1938) manual Bridgeport that I converted to CNC by adding a ballscrew assembly. Everything had some amount of spring to it, but there was a lot of wear in the ways that made it possible for the table to rock in several different ways. I finally found a much newer R2E3 Bridgeport (BOSS8) machine that was built for CNC from the ground up. Also, the ways are in vastly better condition. Well, there is SURE a difference in accuracy of machining! The old machine was tolerable, the new one is excellent! I also made a MUCH more compact CNC retrofit in the existing servo cabinet on the right side, and completely removed the electrical cabinet on the back of the base.
Jon