Encoders setup

There is a HAL pin that reports quadrature error.

github.com/LinuxCNC/linuxcnc/blob/master...tmot2/encoder.c#L503

You could try enabling that to see if the encoder pulses are consistent.

(I gave a link to the source-code because the pins do not seem to be mentioned in the documentation)

The additional weak (4.7K) pullups on the 7I80 inputs will not hurt anything.
it sounds like the (probably much stiffer) pullups on the C53 board encoder outputs
were not powered. I suspect they need a 5V power source (pin 26?)

The signal issue was indeed the pull-up resistors, and after speaking also with Arturo from CNC4PC he confirmed that these resistors are there for the Smoothstepper, as it is not IEEE1284 compatible. As for the power, it is not an issue because it has a separate power supply path in case pin 26 cannot be used (the case of mesa board).
But although the electrical part is resolved and the signals are passing to 7i80 the problem of missed counts still exists. Yesterday and today I went to whole lot of trouble replacing the encoders with some rotary encoders I have because I was convinced that the DIY encoders are not working properly, only to find out that the problem is not the encoders.
It was a lot of surprise for me to find out that the problem is actually the count frequency. When I reduce the jog speed to 50mm/min the count is as it should be. At 100mm/min counts are starting to be missed.
I am pretty sure it is not the encoder board, as I have the exact same one on my CNC router with a smoothstepper and I can get up to 6000mm/min without loosing a single count from the encoders. Of course someone would ask how many counts per unit for each encoder…
The ones on the CNC router have a 400counts per mm and the ones I tested on the other setup are 512 per unit, so no big difference there.
I can not believe that the setup I have with the 7i80 as a controller can not handle these kind of speeds, even an arduino nano can…
So I am sure I have done something wrong with the configuration of linuxcnc.
Any idea? So that I can close this issue once for all.

The 7I80- can count encoder counts up to about 25 MHz (filter off) so its not likely
to be the issue.

If you removed the breakout board pullups you now only have
the fairly weak 4.7K pullups on the 7I80 so this _will_ limit your
maximum count rate. (for the BOBs stated 10 MHz count rate
you would need close to 300 Ohm pullups)

Count errors may also be because of signal integrity issues so you might try
enabling the encoder filter and/or lowering the encoder sample rate